The Association had publicised the event, due to take place at Dumfries’ Aston Hotel, on its website but all mentions of the brunch have since been deleted from the page.
The planned event has been criticised by both opposition MSPs and the Scottish Conservatives.
SNP MSP for the south of Scotland Joan McAlpine said: “The Tory toffs in the No campaign are obviously taking Scotland’s voters for granted.
“Despite cancelling this event under orders, the No campaign are obviously counting their pheasants and champagne in advance, which is very foolish.”
A spokesman for the Scottish Conservatives said: “We are not taking anything for granted ahead of September 18.
“This event was not appropriate and it has now been cancelled.”