The ornate wall, which runs next to the entrance of the hotel and sits 25 feet above Market Street, next to Waverley Station, was found in pieces on Saturday morning.
Luckily the debris, which included three pillars and a sizeable piece of banister, had fallen inward rather than tumbling on to the street.
Employees at the hotel said they had heard the damage was caused by vandalism early on Saturday morning.
The city council said somebody could have caused "accidental vandalism" by climbing on the wall and causing it to fall over but admitted the damage was a mystery.
Marion Williams, director of the Cockburn Association, said that the incident was "worrying" and could have caused serious injury.
She said: "If it was a safe and sturdy wall, somebody climbing on it would not cause it to be knocked over. If that was the case it must have been in a state of disrepair. It would take a lot to break thick stone.
"Given the wall's prime location it is extra worrying. It doesn't bear thinking about what could have happened if it had fallen on to the street below.
"The council have a duty of care to keep buildings in repair, and this is especially important in major tourist areas. Given all of the movement on and around North Bridge it could be structural damage, so the council need structural engineers to complete a proper assessment of the area."
A council spokesman confirmed that city officials were talking to a scaffolding company, which would erect scaffolding around the wall – thought to date back to the late 19th century – within the next couple of days.
It plans to repair the gap in the coming weeks but a spokesman admitted that it might have to wait until spring, when the council had already planned a restoration of the famous Scotsman steps next to the hotel. The spokesman added that the wall would be assessed when it is repaired to ensure it is not unstable.
Although police attended the scene on Saturday, a spokesman said they would not be investigating the incident further.
A council spokesman said: "We are currently assessing the damage made to the stonework and repairs will be carried out as necessary."