At Commons Business Questions on Thursday, Mr Rees-Mogg accused the SNP of having a "shameful" record at Holyrood.
SNP Commons spokesman Tommy Sheppard had criticised the Leader of the House of Commons over his comments last week that "constitutional tinkering" under former Labour prime minister Tony Blair should be undone.
Mr Sheppard said on Thursday: "If the UK Government is about to reverse a policy of more than two decades, surely this Parliament ought to debate the matter?
"The current Government has no UK mandate for this change and it represents a total and absolute disrespect for Scottish public opinion.
"It seems that in the coming Scottish general election, two alternative paths will be offered to the electorate - either the Government's assault on devolution ends up with direct rule from Westminster or the Scottish people defend their right to decide matters for themselves by demanding the choice of becoming an independent country."
Mr Sheppard added: "Yesterday, another opinion poll by Ipsos MORI recorded a 12-point lead for independence, the 15th in a row to record a Yes majority.
"The question to this Parliament is whether it will respect the results of the election in Scotland or whether it will ignore them and aim to govern without the consent of the people?
"Does [Mr Rees-Mogg] agree that at some point, we must have this debate?"
Mr Rees-Mogg responded: "The failure of the SNP is not something I would have thought [Mr Sheppard] would wish to boast about.
"The SNP in government has failed on Scottish education, it has failed on Scottish health, it has failed on Scottish law and order.
"It is a shameful record in Scotland of the SNP. They have let the people of Scotland down."
He added: "And [Mr Sheppard] loathes genuine elections because in 2014, he lost, L-O-S-T, lost.
"He had an election and it was going to be for a generation and the generation has hardly lasted six years before he quotes opinion polls.
"Opinion polls aren't proper elections where people go out and put their cross and put it in a ballot box.
"They are a taster of opinion at a passing moment of fancy, whereas a general election is what was had in 2014 and that is the result supported by the people of the United Kingdom."
Mr Rees-Mogg said the coronavirus pandemic had highlighted the strength of the union.
He told Mr Sheppard: "It is worth reminding him of the success of the United Kingdom.
"The UK taxpayer provided £8.2 billion to help Scotland through the pandemic, it is of fundamental importance and shows the strength of this nation together.
"It is the SNP that has been the failure and it is not devolution that has been the failure."
He added: "Devolution could work perfectly well if only the Conservatives were in charge in Scotland, which would make a triumphant success of it.
"And if you don't believe me, you just have to look at how well London was run when it had a Conservative mayor and what a hash is being made of it by a socialist mayor."