Ian Hamilton, who died last year, broke into the abbey in 1950 and removed the stone, alongside a group of students.
The stone was used to inaugurate Scottish royalty for centuries, before being removed from the country by King Edward I in 1296.
It is currently on display in Edinburgh Castle after being officially returned to Scotland in 1996.
However there are plans for the stone to be moved to Westminster Abbey for the coronation, with a guarantee it will return to Scotland afterwards.
Ian Hamilton’s son Jamie told the BBC: “I think his view would be, it’s ridiculous.”
He added: “If people want to be crowned on it they should come to Scotland and be crowned on it. I think that would be his standpoint.”
Historic Environment Scotland are the custodians of the ancient stone, which is displayed alongside Scotland’s crown jewels.
Head of collections Kathy Richmond told the broadcaster: “We have a royal warrant which says that the stone is to be kept in Scotland and it’s to be transported back to Westminster Abbey for any future coronation.”
Earlier this month former first minister and Alba Party leader Alex Salmond said the stone should not be used in the King’s coronation.
Mr Salmond admitted it was not his “most serious policy” but that the Scottish Government should refuse any request for the stone, due to the refusal to allow a second independence referendum.