Mr Ross insisted he is "in this for the long run" amid reports there have already been informal discussions about ditching him as leader.
He spoke to journalists after the Tories lost more than 60 councillors across Scotland, with Mr Ross blaming partygate.
He said: "Clearly it wasn't a good night and I'm disappointed, and I want to say directly to all the voters who chose to stay at home that we understand why they did.
"They are rightly angry and the challenge for me and the party is to win back their trust and their support, and convince them that we deserve their vote next time."
There have been widespread reports of internal unhappiness following the election results.
One Tory MSP said Mr Ross had "lost all credibility", while a senior UK Conservative said he had "handled the partygate affair in an idiotic manner".
Mr Ross initially led calls for Boris Johnson to quit over the Downing Street parties but later U-turned following the war in Ukraine, insisting stability was needed.
Asked about the speculation over his future, Mr Ross said: "There are always questions about leadership immediately after an election.
"I've been clear that I'm in this for the long run.
"We had our best-ever result in a Scottish Parliament election under my leadership just 12 months ago, and we have had a difficult night on Thursday into the count on Friday.
"So I was here when we had a very good result, and I'm still here when we’ve had a disappointing result."
The Scottish Tory leader said no one in the party had spoken to him directly about his position, adding: "I understand colleagues are as disappointed as I am with the results. We didn't want to lose great colleagues.
"We wanted to see our outstanding candidates elected, and in many cases they weren't."
He said he is not ignoring the challenges, adding: "I hope colleagues, some who say things to the press, also feel free and have done to say things to me when I speak to them on the phone or in person."
Mr Ross argued his partygate U-turn did not change the result.
He said: "I don't think, had I not changed my position in light of the atrocious conflict in Ukraine, it would have changed the situation, because voters I was speaking to were unhappy with the Prime Minister and unhappy with partygate.
"Had I maintained my position despite the war in Europe, those voters would still have been unhappy with partygate, and still unhappy with the Prime Minister because he remains in post."
Asked if there were any circumstances in which he would re-submit his letter of no confidence in the Prime Minister, Mr Ross said: “There are always circumstances.
“It's certainly, and I can assure people who have never done it, it's not an easy thing to put your letter in in the first place and I thought long and hard about that before I put it in, but I also looked at the situation in Ukraine, and couldn't ignore what was going on there.
“But there are always options that any MP has to look at in the current situation."