The newly elected Scottish Labour leader told BBC Scotland’s Disclosure programme it was not acceptable, but insisted he had only recently take over the party.
His comments come during the documentary which airs tonight at 10:45pm on BBC Scotland.
He said: “I’m acutely conscious of it. It’s not acceptable, I have come into this late, once our selection process is near completed but it’s not good enough.
“It’s not acceptable. And I would also say me being a BAME leader doesn’t make up for it.
“The idea just like having a female leader doesn’t automatically end sexism or having an ethnic minority leader ends inequality and prejudice around race and religion, we’ve still got a huge way to go.”
Mr Sarwar also revealed his own personal experience of being an Asian Muslim during an era when there have only been four people of colour elected to the Scottish Parliament.
He explained: “I’ve been in front line politics for ten years and the idea that I could speak as openly about issues of race or religion or prejudice or hate in the round ten years ago compared to now I just, I wouldn’t have done it.
“And that’s partly because you do worry about being pigeon-holed, you do worry about being framed as being either the Asian candidate or the Muslim candidate rather than the Glasgow candidate or a wider Labour candidate.
“So I think there is that issue around pigeon-holing and also people making claims that you aren’t mainstream, which is part of the challenge.”
There are just 10 women of colour candidates standing in May’s election - six SNP, two Liberal Democrats, one Conservative and one for the Green Party.
Despite the low number, it is still the highest in the history of the Scottish Parliament.
Women make up 51 per cent of Scotland’s population, but only hold 36 per cent of the seats at Holyrood, and there has never been a woman of colour sitting at the parliament.
The SNP, Labour, the Greens and the Liberal Democrats have all pledged that 50 per cent of candidates at this election will be women.
Both Scottish Labour and the Green Party have systems to increase the number of women getting the top places on the additional member voting lists.
Talat Yaqoob, from campaign group Women 50:50 called for a better focus on tackling the issue.
She said: ‘We have never had any women of colour. And that is extraordinary. And I know a lot of people will say women of colour make up such a small proportion of the Scottish population.
“None of this is easy, and largely none of it’s easy because you’ve got a lot of sexist attitudes, and racist attitudes to tackle in your own political party.
“And one of the reasons why political parties, I think are concerned about this, or nervous about taking these measures forward to elect more women, and particularly women of colour, is because they don’t want to be accused of giving anybody an unfair advantage.
“But the reality here is, you have been giving men an unfair advantage for decades, and nobody has batted an eyelid.”