The Scottish Labour leader and chair of the Cross-Party Group on Tackling Islamophobia on Wednesday questioned the findings in the Commission on Race and Ethnic Disparities report, which was ordered by Downing Street in the wake of the Black Lives Matter protests.
A foreword to the report by chairman Tony Sewell, an education consultant and ex-charity boss, said: "We no longer see a Britain where the system is deliberately rigged against ethnic minorities."
It even suggested there was a silver lining to the legacy of slavery, saying: "There is a new story about the Caribbean experience which speaks to the slave period not only being about profit and suffering, but how culturally African people transformed themselves into a re-modelled African/Britain."
The report also recommend ending the use of the term BAME and for organisations to stop funding unconscious bias training.
Success in education and, to a lesser extent, the economy “should be regarded as a model for other white-majority countries”, the report added.
Now Mr Sarwar has demanded parties work together to tackle racism in all its forms.
He said: “The report’s conclusion won’t reflect the lived experiences of many people in the UK.
“No organisation or institution is immune to racism, and the first ever public inquiry into Islamophobia in Scotland found widespread concerns about this.
“We have established beyond doubt that everyday racism exists and the focus must be on overcoming hatred and prejudice. Rather than a debate about whether it is structural or not, we need to focus on action.
“This requires politicians to come together on a cross-party basis, because the fight against hate is a fight for all of us.”
Marsha de Cordova MP, Labour’s shadow women and equalities secretary, said: "This report was an opportunity to seriously engage with the reality of inequality and institutional racism in the UK. Instead we have a divisive polemic which cherry picks statistics.
“To downplay institutional racism in a pandemic where black, Asian and ethnic minority people have died disproportionately and are now twice as likely to be unemployed is an insult.
“The government must urgently explain how they came to publish content which glorifies the slave trade and immediately disassociate themselves from these remarks.”
The SNP also questioned the report, with its candidate for Glasgow Kelvin Kaukab Stewart urging the response to be a “wake-up call”.
He said: "It's evident that the review fails to properly grasp the scale of the issue and concerns raised, and there are questions over the cynical manner in which this review was trailed before its full publication.
"We all need to have an open and honest conversation on race and the systematic and structural issues that perpetuate inequality. Only by acknowledging and understanding institutional racism will we be able to effectively tackle it in all aspects of life.
"It is vital that rather than brushing the issue under the carpet, the Tory Government now heeds the concerns and works properly with communities and organisations."
Prime Minister Boris Johnson said the UK Government would consider the implications of the report's recommendations for future policy and remained "fully committed to building a fairer Britain".