The veteran of government, who is influential in the Tory party, criticised using borrowing to pay for slashing taxes as being “not Conservative”.
It comes as the Conservative party chairman said MPs who vote against the Government’s plan for tax cuts will lose the whip.
Mr Gove welcomed the Prime Minister acknowledging she had made mistakes around the mini-budget but said she displayed an “inadequate realisation” of the scale of the problem.
He told the Sunday With Laura Kuenssberg show he is “profoundly” concerned that Chancellor Kwasi Kwarteng is paying for £45 billion of tax cuts through increased borrowing.
Mr Gove said cutting the 45 per cent income tax rate for the highest earners was a “display of the wrong values”.
He even suggested he could vote against the plans in the House of Commons, as Conservative critics eye a possible rebellion.
“I don’t believe it’s right,” he said of the budget when pressed on the BBC One programme.
He welcomed Ms Truss admitting mistakes over the financial announcement, after she told the same programme they could have been better “laying the ground” for the plans that have sparked a backlash on the financial and mortgage markets.
But he said there remains “an inadequate realisation at the top of Government about the scale of change required”.
Mr Gove said there were “two major things” that were problematic with the plans set out by the Prime Minister and Chancellor.
“The first is the sheer risk of using borrowed money to fund tax cuts.
“That’s not Conservative,” he said.
The second, Mr Gove argued, was the move to cut the top rate of income tax and axe the cap on bankers’ bonuses “at a time when people are suffering”.
Conservative chairman Jake Berry said on Sunday that Conservative MPs who vote against the Government’s plan for tax cuts will lose the whip.
It has been reported that some Tory MPs are preparing to vote with Labour to prevent measures including abolishing the top rate of income tax.
Asked on Sky’s Sophy Ridge on Sunday whether this would result in them losing the party whip, Conservative chairman Jake Berry said: “Yes.”
He also urged Tory MPs to unite behind Liz Truss and her programme, saying she had “a mandate both from colleagues and our membership”.
He said: “I’m sure that if we do that it will lead ultimately to long-term electoral success.”