The Scottish party leader had urged the new prime minister to keep faith in Mr Mundell, who was one of the longest-serving ministers in Whitehall and a veteran of the 2014 Better Together campaign.
But his experience as a high-profile supporter of the Union counted for little on Wednesday evening as the new Tory leader stamped his authority with a sweeping Cabinet reshuffle.
Mr Mundell, 57, had been roundly criticised by opposition parties on Tuesday after he opted not to follow the example of several of his cabinet colleagues by resigning before Mr Johnson entered Downing Street.
But he received firm support from Ms Davidson and other party colleagues in Scotland.
Mr Johnson, who has repeatedly refused to rule out the prospect of a no-deal Brexit, is unlikely to have welcomed Mr Mundell’s previous opposition to such a scenario.
Last year, the Scottish secretary claimed that a no-deal Brexit would be “catastrophic” for Scotland and that such an outcome would be unacceptable for the country.
“I believe it would be a fundamental threat to the continuation of the United Kingdom and therefore in the round you have to weigh up all the issues,” he said in 2018.
“These are difficult and complex judgments but I’m not prepared to countenance a no-deal outcome for Scotland.”
Mr Mundell was however popular among Scottish Conservatives and Ms Davidson in particular.
She paid a fulsome tribute to the former Scottish secretary tonight, describing him as a “trusted advisor and committed Unionist” whose work had helped two prime ministers since his appointment in the wake of the 2015 general election.
She added: “He’s often been a calming influence through the most turbulent times.”
Mr Mundell said he would “of course” support the new Government. He added: “As I said to the prime minister this afternoon I will also hold him to account on his commitments to the Union.”
A source close to Mr Mundell said: “He would have been prepared to make compromises and serve in a Boris cabinet on the basis that he feels Scotland is at a very critical moment, with the First Minister demanding an independence referendum next year, and on the basis that Boris says he wants to get a (Brexit) deal.”
The SNP said Ms Davidson had been “left utterly humiliated” after urging Mr Johnson to keep Mr Mundell in his position.
SNP MP Tommy Sheppard said that Mundell had become a “national embarrassment” over his hypocrisy after he previously criticised Boris Johnson for behaving “in a way that suggests he’s only focused on his own self-interest and not in the interests of our country.”
He added: “Ruth Davidson is a busted flush – Boris Johnson has humiliated her and left her authority non-existent following the sacking of David Mundell as Scottish Secretary.
“That was despite her bid to keep him in post, which was completely ignored by the new PM.”
The shadow Secretary of State for Scotland, Lesley Laird, said: “Instead of departing with dignity David Mundell has been kicked out of the Cabinet by Boris Johnson.
“David Mundell’s time in the Scotland Office will be remembered for his spineless failure to stand up for our country’s interests instead of his party’s.
“The only job he cared about saving was his own, and in the end he couldn’t even manage that.”
Mr Mundell’s departure is a blow for Ms Davidson, who viewed him as a trusted ally in maintaining support for the Union.
In a statement, she said: “David’s record of service at the Scotland Office - first as a minister in the coalition government, then as secretary of state these last four years - has been exemplary.
“He has been a trusted advisor to two prime ministers, held his nerve during the independence referendum and faced down Nicola Sturgeon when she called for a re-run in 2017.
“He is passionate about opening doors for Scottish businesses in Whitehall and using the might of the UK Government to promote them abroad, having led countless overseas trade delegations.
“While David chooses to conduct himself publicly in a typically understated manner, his strategic brain has been at the heart of the rebuilding project of the Scottish Conservatives.
“The party could not have enjoyed so many recent successes without his input, effort and ability to make Scotland’s case.”