The comments come as the Prime Minister faces mounting calls to resign following the publication of senior civil servant Sue Gray's investigation into Covid rule-breaking in No 10 and Whitehall during the lockdowns in England.
Former health minister Steve Brine was discovered to have added his name to the list of Tory MPs to have handed in letters of no confidence in the Prime Minister.
Newton Abbot MP Anne Marie Morris is also among the letter writers, having confirmed to PA news agency that she has had the Tory whip restored after it was removed in January for her decision to support an opposition move to cut VAT on energy bills.
Sir Graham Brady, chairman of the 1922 Committee of backbench Tories, will be obliged to order a confidence vote if he receives 54 letters demanding one.
More than 20 MPs have publicly stated that they no longer support Mr Johnson but more may have penned letters given the process is carried out in secret.
Mr Brine said Ms Gray's report had not altered his view that it was "inevitable the Prime Minister would face a vote of confidence".
"All I can do as a backbencher is seek to trigger that process and (some time ago actually) I have done that," he confirmed in a statement published on his website on Wednesday but only picked up by the media on Saturday.
"I have said throughout this sorry saga I cannot and will not defend the indefensible. Rule-makers cannot be law-breakers."
The band of rebels increased on Friday, with Sir Bob Neill, Conservative chairman of the Justice Select Committee, confirming he had submitted a letter of no confidence in Mr Johnson's leadership.
Alicia Kearns, a member of the 2019 intake, accused the No 10 incumbent of "misleading" Parliament with reassurances that coronavirus laws were upheld.
Cabinet Office official Ms Gray detailed in her report how leaving-dos got out of hand, with staff throwing up after drinking excessively, being rude to security staff and even damaging a child's swing in the Downing Street garden.
She found the Prime Minister - who was slapped with a fine by police for attending his own birthday bash in June 2020 when indoor mixing was forbidden - attended a number of leaving-dos in No 10 during the lockdown months in England.
Ms Gray said he gave speeches and joined in with drinking alcohol at such events despite the public being banned from socialising in a bid to stop the spread of the virus.
The warning by Mr Davis of increasing mutiny comes after polling company YouGov produced new modelling suggesting the Conservatives would lose all but three of 88 "battleground" constituencies if a general election were held on Saturday, putting the Government's Commons majority in jeopardy.
The predicted outcome would see Mr Johnson's own Uxbridge and South Ruislip seat "likely fall" into Labour hands and Red Wall seats such as Blyth Valley and Stoke-on-Trent North also revert back to Sir Keir Starmer's opposition.
Only Ashfield, Bassetlaw, and Dudley North would remain blue, according to YouGov.
James Johnson, a former Downing Street pollster during Theresa May's tenure, said the results could prove to be even worse than the model predicts.
"Remember that MRP does not pick up tactical voting or local seat effects," he said on Twitter.
"Tactical voting helps Labour and the Lib Dems at the moment, so it's highly likely that the current picture for the Conservatives is even worse than this polling indicates."
Mr Davis, a former Brexit secretary, told BBC Radio 4's Today programme that potential Tory losses and the "distraction" of questions about the No 10 parties was focusing minds in terms of thinking about the party's leadership.
He implored Mr Johnson to quit in January, telling him to "in the name of God, go", and said he had not "changed my mind about that".
Asked whether discontent was spreading in the Tory party, Mr Davis said: "There is no doubt about that, for two reasons.
"Number one, frankly they see their own seats disappearing in many cases, they see themselves losing the next election on the back of this.
"Also, it has a bad effect on the country ... it is a distraction on everything you do and it doesn't help the reputation of the country."
Former justice secretary Robert Buckland told Radio 4's The Week In Westminster programme that should the Tories experience "heavy defeats" in the Wakefield and Tiverton and Honiton by-elections next month then "changes will have to be made".
However, Mr Buckland said he hoped the party could avoid a loss on such a scale that it would "lead Conservative members to really fundamentally think again about who is to lead them".
Despite facing criticism over his partygate defence, the Prime Minister chose to announce changes to the ministerial code this week in a move his rivals said watered down punishments for ministers.
An update said ministers will not automatically lose their jobs if they breach the standards code, and can instead apologise or possibly have their salary suspended instead.
Chris Bryant, the chairman of the Commons Standards Committee, told the Today programme the move demonstrated why an independent process was required when it came to judging possible ministerial wrongdoing.
The Labour MP, who has recused himself from chairing the Privileges Committee investigation into whether Mr Johnson misled Parliament with his partygate reassurances, said the current system means "it still all lies in the Prime Minister's hands and we know that the Prime Minister always finds himself innocent in the court of his own opinion".