The 4,000 watercolour - called 'I Cannae Hear Ye' - by Borders artist Tom Scott, was stolen from the Signet Library in Parliament Square earlier this month.
It was discovered hanging on a railing outside the library at 8.15am this morning by a member of staff.
At the time of the theft Robert Pirrie, chief executive of the WS Society, which owns the library, said it was a complete mystery how someone had made off with it without being noticed.
The WS Society - Scotland's independent association for lawyers - closed its premises for a festive break from 31 December to 5 January, opening only once on New Year's Day to allow 200 people access for a ticket-only lecture at 3:30pm.
The painting was most probably taken during this time, as there were no signs of forced entry to the building.
Police said they are continuing to investigate the circumstances surrounding the artwork's disappearance and are eager to speak to anyone with information.
A police spokesman said: "Obviously, we are delighted for the library that the painting has been returned.
"Nevertheless, we still must establish whether any criminality was involved when the painting was taken.
"Anyone with information that can assist with our enquiries should contact police immediately on 0131 311 3131, or Crimestoppers in confidence and complete anonymity on 0800 555 111."