The Scottish Labour leader was speaking to The Scotsman’s political podcast, The Steamie, prior to the suspension of election campaigning on Friday due to the death of the Duke of Edinburgh.
Mr Sarwar is aiming to be the first Scottish Labour leader to gain seats at a Holyrood election since the very first Scottish Parliamentary election in 1999.
Since then, the party which led to fight for devolution has lost seats in each election, with polling suggesting the same is likely to happen in the election in May should their support remain around the 18 per cent mark on both the constituency and regional list vote.
Asked why Scottish Labour had seen a continued decline, Mr Sarwar joked “how long have you got?”
"I wish I could say there was just one mistake on that journey,” he said.
The son of the first Muslim MP and first Asian MP elected in Scotland, Mohammed Sarwar, who is now the first Asian leader of one of Scotland’s main political parties, said the party has made a “series of errors and mistakes” since 1999.
He said: "To answer directly, one part is that we championed devolution but we didn’t devolve ourselves.
"We also had the final days of a 13-year UK Labour government and at the same time we were having challenges here in Scotland, so the tail end of a Labour government, the loss of popularity over that period I think made life more difficult for us in Scotland as well.
"You could look at us being too worried about going it alone perhaps as a minority government and we perhaps paid the price for some of that along the way.”
Mr Sarwar added that a failure to grasp the issues of identity and the constitutional divide in recent years has also impacted Labour support.
Since the independence referendum in 2014, Labour has had four leaders and no electoral recovery.
Mr Sarwar, who has been in post since late February, said: "We didn’t really get comfortable I think with Scottish identity and pride in Scotland in terms of how we spoke and how we delivered our ideas and projected ourselves as a political party.
"Then we had a period where just we looked like we were talking to ourselves and talking about conflicts within and talking about our past and our history rather than talking about the country and talking about the future.
“The aftermath of the referendum, the fact that the party has never naturally been comfortable talking about that frame of unionism and nationalism.”
"The challenge has been in recent times that we’ve accelerated that decline rather than slowed that decline down.
"The task I have is not just a task over the next four and a half weeks, it is actually a task over the next five years, is to not just stop the decline but go against that wave and rebuild the Labour Party again."
Readers can listen to the full interview with Mr Sarwar on The Steamie.
If you like what you hear, please hit subscribe to never miss an episode – and leave a rating and review. It helps others discover the show and allows us to hear your feedback