The Scottish Labour leader’s party lost East Lothian to the SNP in the first seat to go between the two parties of the election.
It came as the SNP also took the Ayr constituency from the Conservatives, which had been held by John Scott since almost the beginning of the Scottish Parliament.
Speaking after the defeat in East Lothian, Mr Sarwar admitted it was “disappointing”.
He said: “Of course I’m particularly disappointed for our amazing candidate Martin Whitfield.
“But again, look at where we were ten weeks ago – we were at 14 per cent in the opinion polls and coming fourth.
“Even my biggest critics can see that over the course of the last ten weeks we’ve got the Labour Party back on the pitch.
“I think people can see that I’m a different kind of leader, trying to build a different kind of party, and together rebuild a different kind of country.”
East Lothian saw the SNP’s Paul McLennan beat Martin Whitfield with a majority of 1,179, despite Labour having held the constituency since the start of devolution.
The SNP secured 17,968 votes to Labour's 16,789.
Mr Sarwar also admitted his party were still on a journey to build a “credible alternative” to the SNP.
He said: “I’m pleased that we doubled the actual number of votes that we got and we increased the share of our vote by 9 per cent.
“I think you can see the immense progress we have made in the last ten weeks.”
It came as Scottish Labour sources blamed the bad results on Tory campaign tactics, accusing them of spending so long attacking unionist parties they lost sight of the goal, possibly handing the SNP a majority.
Despite this, they insisted: “Compared to our position just ten weeks ago, Labour are a transformed party.
“The real risk at this election was Scottish Labour bleeding out, but we are now a stable, hopeful and re-energised party.”
They also hit out at the Conservative campaign tactics, claiming their “divisive” tactics were seeing Douglas Ross’s party lose seats.
Mr Sarwar had hoped his party would be able to mount a strong challenge to replace the Scottish Conservatives as the opposition, but they may now be heading for third place.
It came as the Tories made a series of losses, but continued to insist the strategy was working.
The party lost Ayr to the SNP by just 170 votes after seeing its majority of 750 overturned.
Siobhan Brown won with 18,881 votes to the Tory's 18,711, with a turnout of 68 per cent.
The party also lost Edinburgh Central to the SNP after record numbers of voters turned out to vote in the Scottish capital.
It means the SNP’s former Westminster leader Angus Robertson has been elected as an MSP after losing his Moray MP seat in 2017.
The Scottish Tories were not confident of holding the seat, which was won in 2016 by former party leader Ruth Davidson.
In a sign the campaign may not have been a success, Ms Davidson appeared to distance herself from the result, saying "I wasn't part of the planners".
As well as losses, the party missed out on target seats including Banffshire and Buchan, and Angus North and Mearns.
In Banffshire and Buchan, Aberdeenshire councillor Karen Adam won the constituency previously held by party veteran Stewart Stevenson after his retirement.
The Tory candidate Mark Findlater enjoyed a 10.8 per cent swing, but missed out on the seat by less than 1,000 votes.
The Tories also failed to see off Scottish minister Richard Lochhead despite a strong challenge from Tim Eagle. Mr Lochead got 19,987 votes, compared to 16,823 for Mr Eagle.
Georgia Strachan, the Labour candidate, won 2,169 votes.
Despite the results, Scottish Tory sources were bullish and insisted the voting was holding up.
A source said: “The SNP are looking strong in constituencies, but our vote has held firm and even increased in neck-and-neck seats.
“The early signs indicate our party list vote strategy could prove to be successful.”
It was also a difficult day for the Liberal Democrats, who missed out on the target seat of Caithness, Sutherland and Ross seat after the SNP’s Maree Todd fought off a close challenge from Molly Nolan.
Willie Rennie was re-elected himself in North-East Fife with a record majority, but his party had made the far north constituency one of their target seats.
A Lib Dem source said: “In the areas where we’re strong, our message clearly got through and our candidates were returned with big majorities.
"However it’s disappointing that in other areas we were unable to move the dial.
"[On Saturday] we will find out what the list vote holds, but for tonight at least we can claim to be Scotland’s biggest opposition party.”
Results from a total of 48 seats were due to be confirmed by the end of Friday, with the final constituency and regional list results on Saturday.
Despite the promising results for the SNP, the First Minister talked down the chances of a majority on Friday.
She said: “A majority has always been a very, very long shot.
"The Holyrood system is a proportional representation system, in 2011 we effectively broke that system.
"So it would be good to do, but I have never taken that for granted.”