Jim Dempster, a Labour councillor in Dumfries and Galloway, has already said he is “deeply sorry” for remarks made about Scottish Transport Minister Humza Yousaf hiding “under his burka”.
Now further allegations of racial abuse - which Mr Dempster denies - have been made against the local politician.
The claims by a man and a woman reportedly centre on a period in the late 1980s and early 1990s when both were children in the village of Sanquhar.
One of the accusers claim Mr Dempster would pick on him and regularly call him names.
The other accuser also made a number of claims, including one that Mr Dempster “chased me around his shop with an air freshener saying I smelled of curry”.
A Scottish Labour Party spokesman said an investigation already underway into Mr Dempster’s behaviour would “consider any new allegations”.
But Mr Yousaf was clear that he should quit as a councillor.
On Tuesday Mr Dempster said he was “deeply sorry” for his comment about Mr Yousaf - made during a meeting with Transport Scotland officials - adding that he had asked for further diversity training, and had also visisted a local mosque.
This was dismissed as a “tokenistic photo op” by the minister as he repeated calls for the councillor to quit.
Mr Yousaf said he had received “correspondence to suggest that Cllr Dempster’s Islamophobic slur was not a one-off”.
On Thursday he told BBC Radio Scotland that the two alleged victims had contacted him “when they saw Jim Dempster’s face across the news”.
Speaking on the Good Morning Scotland programme, Mr Yousaf said: “One of them contacted me by email on the day that the allegations surfaced of the Islamophobic slur Jim Dempster made against me. Another one contacted me a couple of days ago.
“It’s just really disturbing, these incidents took place allegedly a number of years ago but both victims tell me they really had just suppressed them but when they saw Jim Dempster’s face across the news they just said they had to come forward.
“They wish to remain anonymous but told quite graphic accounts of sustained racial abuse over a decade.”
Mr Yousaf said both he and Labour MSP Anas Sarwar - who recently launched a campaign against “Everyday Islamophobia” - had spoken to the victims.
The Transport Minister said: “One of the victims told me he was haunted by it, he said he had suppressed it, he said the 10 years that he lived in the area in Sanquhar and he was a victim of that abuse were the worst 10 years of his life.
“He said when he saw Jim Dempster’s face again on the news he couldn’t sleep. It really has had an impact on them, I really applaud them on their bravery having spoken to both of them.”
While stressing the new allegations have still to be investigated, he claimed they “point to a disturbing pattern of behaviour”.
Mr Yousaf said: “What Councillor Dempster doesn’t deny is making an Islamophobic slur. We have to ask ourselves, somebody who admits such inexcusable behaviour - do they have a role in our politics and our political life?
“And the fact that he’s been trying to get to mosques, do photo ops, to me shows he is more interested in saving his political career than he is in showing any true remorse, so I repeat my calls on him to resign.”
He added: “The honourable thing for Jim Dempster to do would be to resign instead of trying to scramble to save his political career.”
Councillor Elaine Murray, leader of Dumfries and Galloway Council, said: “The new allegations are extremely serious and if found to have substance I have no doubt that Jim will be expelled from the Labour Party.
“Jim denies the allegations and there is a process of investigation to be undertaken. The Scottish legal tradition is that a person is innocent unless proved guilty.
“I appreciate that the accusers wish to preserve their anonymity but I would urge them to take part in the investigation, I understand that they are both known to Anas Sarwar and I am sure that he would arrange for this to be done sensitively.”
A Scottish Labour spokesman said: “James Dempster has been suspended by the Scottish Labour Party, and an investigation is ongoing.
“The investigation will consider any new allegations. The Labour Party has a zero tolerance policy towards racism and all forms of discrimination.”