The incident is said to have taken place at Stonehaven Harbour by supporters making their way north for Rangers' heated opening game of the season.
David Officer, the father of the three-year-old, alleged that Rangers fans had entered a verbal spat with his in-laws before kicking over the child's sand sculpture.
He said: “One of the Rangers supporter buses stopped in Stonehaven on Sunday and their fans went down to the beach where my eldest son and his grandparents were.
“They shouted at my father-in-law and then kicked my son’s sandcastle over before they left."
The 37-year-old has revealed that his son wasn't affected by the loutish behaviour, but hinted that incidents like this may put off parents from visiting the beach.
“I’m more annoyed for my father-in-law and mother-in-law who had to deal with it," he said. "The beach is a great place for kids to go and spend time. My son wasn’t affected by it, but I just don’t get it. I don’t see the point in all of that. It’s just not something you do."
The disgruntled father then took aim at the supposed perpetrators claiming "all clubs have their fair share of idiots – but it does seem like you tend to hear more of this behaviour from Rangers fans."
“And my father-in-law is a Rangers fan, so I’m not just picking on them."
“I wish something had been done – maybe more policing among themselves – like not letting them back on the bus until they apologised or help rebuild the sandcastle.”
'Unwanted in any of our communities'
The incident has since been reported to the police who have described the incident as "completely unacceptable".
Chief Inspector Martin Mackay said that police made efforts to catch the perpetrators.
“Officers in Stonehaven were made aware of a minor incident yesterday morning involving a handful of men who we believe to be football fans that had stopped off in the town for a short time on their way to the game.
“A unit was despatched and searches made, however they had already left the area."
He added: “Incidents like this may be low-level in nature and very rare, however they are completely unacceptable and unwanted in any of our communities.”
Officer revealed that he has since received a lot of harassment for sharing his story on social media.
He tweeted: "Still getting lots of harassment because I called out Rangers fans on their behaviour.
"Reporting through twitter does nothing. Blocking helps, but I know it's still there. I guess if I go quiet eventually they'll go away, but then they've got what they wanted."
Officer has also praised Rangers fans who have been in touch and apologised on behalf of fellow supporters.
He said: "I will say, thanks to the handful of Rangers fans who have messaged me to apologise. It is genuinely appreciated."