Hate crime incidents linked to the victim's race have fallen by a fifth in the same period.
New figures from the Scottish Government showed that racially aggravated hated crimes fell by 20% over the period 2014-15 to 2019-20, going from 4,967 to 3,969.
At the same time, hate crimes against the transgender community increased from 48 to 96 and the figures showed a 30% rise in crimes where sexual orientation was the aggravating factor, with 1,314 recorded last year, compared to 1,010 in 2014-15.
Race-related incidents accounted for more than three-fifths (62%) of the 6,448 recorded hate crimes in 2019-20.
Meanwhile, 20% of incidents were linked to sexual orientation - with the report stating the "vast majority" of these involved the perpetrator showing "prejudice towards the gay and lesbian community".
A further 8% of hate crimes were linked to religion while 4% were related to disability. Transgender identity was seen as the aggravating factor for 1% of hate crimes and in 5% there were multiple aggravators.
Just over half of hate crimes recorded last year were incidences of "threatening and abusive behaviour", with this making up 51% of such attacks, however 12 were classed under "common assault".
There were also 1,080 recorded hate crimes in 2018-19 where the victim was a police officer.
Assistant Chief Constable Gary Ritchie, of Police Scotland, said: "Hate crime is abhorrent and insidious and will not be tolerated in Scotland.
"Targeting anyone because of a disability, their race, religion, sexuality or transgender identity - or a perception of these characteristics - is despicable."
He added that hate crime was an "under-reported offence", which could see victims "targeted on numerous occasions before they report to our officers".
Justice Secretary Humza Yousaf said the report "provides more detail of how hate crime affects our communities and is a timely reminder that crime motivated by hate is an all too common occurrence, with real-life impacts on victims and those around them".
He added: "Just from those that are reported, we know that an average 18 hate crimes are committed every day in Scotland.
"The fact that visible minority ethnic groups, which represent 4% of the population, experience two-thirds of all race-related hate crimes shows we have much more to do to overcome prejudice, while the findings also highlight the need for more to be done to ensure workplaces are an environment free from hatred and from fear.
"The report also highlights the risk that police face in the line of duty on a daily basis where almost one in five recorded hate crimes involved a victim that was a police officer."
Mr Yousaf , who is taking controversial reforms of hate crime legislation through the Scottish Parliament, said: "The legislation currently before Parliament makes clear that this type of behaviour will not be tolerated - and will ensure Scotland's justice system can bring perpetrators to account and provide protection for individuals and communities harmed by hate crimes."
Communities Secretary Aileen Campbell said: "Hate crime legislation is only part of our wider programme of work to tackle hate crime and build connected communities.
"This report helps us to understand the key characteristics of both victims and perpetrators.”
Reporting by PA