The former deputy prime minister, who is a staunch Remainer, likened the arguments for leaving the UK to those for leaving the EU, and called for further decentralisation of power across the country instead.
He said independence would result in Whitehall being “substituted” by Edinburgh, arguing that greater autonomy should be given to cities such as Glasgow and Dundee.
The Tory peer made the comments during an appearance at the Festival of Politics at the Scottish Parliament.
“I am appalled by the prospect of the fracturing of the United Kingdom,” he said, when asked by an audience member for his views.
“It’s the same argument (as Brexit).
“If I look at the record of the British Empire and the British Commonwealth, and the incredible strengths which Scotland has contributed to this 200, 300-year human achievement, the idea of fracturing it, I find unbelievable.
“If I had one question I would ask about independence - is this really substituting Edinburgh for Whitehall, and substituting Edinburgh for Whitehall, with the centralism moved a few hundred miles north, will that really change anything very much in Scotland?”
He continued: “What I have been arguing for for many years now is the recreation of the power centres of the 18th century.
“If I was looking at the Scottish economy in that context, I would be asking myself about Glasgow, Edinburgh, Aberdeen, Dundee and whether they are being devolved power to.
“Whether they are being recreated as the dynamics that they were when those cities were in their heyday.”
Lord Heseltine’s remarks follow the SNP’s autumn conference earlier this week, during which Nicola Sturgeon called for “pragmatism and patience” from independence supporters.
The First Minister has said she is waiting until the shape of any Brexit deal becomes clear before she makes a judgement on a second referendum on leaving the UK.