Murdo Fraser said the Scottish Government should "follow England's lead" in creating the new elected positions.
In a speech, he told the Institute of Revenues Rating and Valuation that “at the very least” Glasgow, Edinburgh, Aberdeen, Dundee, Perth, Inverness and Stirling should be "given the opportunity" to appoint mayoral representatives.
Elected mayors were introduced in England and Wales in 2000 with the first election of a London mayor. Since then cities and "metro-regions" have elected mayors with a variety of powers.
Mr Fraser said the changes south of the border "have been a success" and that Scotland failing to introduce mayors was a “substantial weakness”.
Local mayors could have powers over economic development, health spending and possibly even welfare, he added.
“We would like to see, at the very least, our seven cities in Scotland having elected mayors, or provosts, with a similar executive power to those south of the border,” he said.
“This would give the opportunity to devolve more power from the centre.
“The SNP is very keen on devolution from Westminster to Edinburgh, but not at all keen on devolution from Edinburgh any further afield. This situation needs to be reversed, with much greater autonomy for local areas.
“It would be a fundamental shift from our current structure of local government, and would allow council finance to be put in its proper context.
“We can see a Scotland made up of a patchwork of vibrant city regions, larger rural areas, all with well-known local leaders – all being a counterweight to an over-centralised Holyrood administration.”
The introduction of elected mayors has been mooted previously, particularly in Edinburgh. Scottish Labour MSP Daniel Johnson has said a directly-elected mayor would "stand up for Edinburgh's needs and give the city fresh direction" while economist and former SNP MSP, Andrew Wilson, has said a mayor would be able to promote Edinburgh to the world.
Think tank Reform Scotland, a non-party policy group, has also argued that a system of elected city leaders could stop party political "buck-passing" in councils, and between Holyrood and local government.
However, Mark Littlewood, director-general of the Institute of Economic Affairs has said the adoption of mayors in England and Wales has failed to etheir "reignite much interest in municipal democracy" or encouraged a drive towards more "radical policy".
There are now 23 directly-elected mayors in England and their powers vary significantly. Two councils - Stoke-on-Trent and Hartlepool - did have elected mayors but then voted to scrap them.