Culture Secretary Lucy Frazer has blamed heavy Tory losses in the local elections on the pandemic and the Ukraine war.
She also noted the Conservative Party had been in power for “a long time”.
The Cabinet minister conceded that voters were “frustrated and angry”, but claimed Rishi Sunak was starting to regain the public’s trust.
The Prime Minister has been blamed by some Tories after the party shed 960 councillors in Thursday’s poll.
Ms Frazer told Sky News’ Sophy Ridge On Sunday programme: “If we could look at the context for the whole local election, we’ve been in power for a long time.
“We’ve just had a pandemic which has disrupted many people’s lives and has had consequences for the economy, as has the war in Ukraine, which is going on.”
Pressed on whether she understood the scale of her party’s losses, she said: “Totally, it’s really important that we listen to people. I know people are frustrated and angry.”
She said she recognised the results were “not good”, adding: “I totally understand that we need to do better and I think we are going to deliver.”
The party was rebuilding trust as Mr Sunak was “starting to deliver in a quiet way” his five priorities including curbing inflation, Ms Frazer argued.
“I totally recognise we’ve had a really difficult few years.
“I do think that the Prime Minister, who’s now been in office for six months, is getting the country back on track and is delivering. I think we’re starting to gain the trust of the British public.”
Pressed on whether the Tories needed to offer more than Mr Sunak’s promises from January, she told the BBC’s Laura Kuenssberg On Sunday programme: “We absolutely need to reflect. I think Rishi’s only been the Prime Minister for six months.”
Kuenssberg said: “It’s not five minutes, it’s six months.”
“It’s not five minutes but these are huge challenges,” Ms Frazer replied.
Some Tories blamed the electoral hammering on the lack of tax cuts.
Former minister Sir John Redwood tweeted: “Last Thursday many Conservative voters went on strike. They do not want to vote for higher taxes, anti enterprise policies and a failure to take back control of our borders.”
Ms Frazer insisted the Government was doing those things, saying: “The biggest tax cut we can make is a cut in inflation”.
Tory MP Anthony Browne denied that Mr Sunak was “under threat in any way” after the dire results.
“I think a lot of it was a protest vote,” the South Cambridgeshire MP told the BBC’s Westminster Hour, noting the pandemic and “all the shenanigans of last year, which, for people, this was their first chance to actually vote against that”.
“Rishi has calmed things down a lot.”
He also claimed that “Labour has not got the momentum it needs to win a general election”.
Meanwhile, Labour was celebrating “pretty encouraging” local election results, which saw Sir Keir Starmer’s party gain 635 seats and take control of another 22 local authorities.
Shadow health secretary Wes Streeting told Ridge: “I think for the next general election Labour feels confident but not complacent.”
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here