The euphoric crowds, including people of all races, hope the turnout will speed up the official end of Mr Mugabe’s rule, which is widely blamed for the collapse of an economy that was once one of Africa’s wealthiest.
The 93-year-old is virtually powerless and has been deserted by most of his allies after Zimbabwe’s generals placed him under house arrest, allowing him limited movement while talks on his exit from office unfold.
But many Zimbabweans are growing impatient and want him to leave immediately.
As the jubilant crowds gathered on some main streets in central Harare, motorists were honking their horns and people were whistling and cheering.
In a gathering that even days ago would have drawn an immediate police crackdown, Zimbabweans giddy with joy raced through the streets, raising their arms in triumph.
Some had posters with an image of the military commander who swept in earlier this week and put Mr Mugabe under house arrest, with the slogan: “Go, go, our general!!!”
Marchers handed flags to soldiers, who accepted and waved.
“It’s like Christmas,” said one marcher, Fred Mubay, who said Zimbabweans have been suffering for a long time.
Mr Mugabe, the world’s oldest head of state, is said to be asking for more time amid negotiations that seek his exit with a veneer of dignity.
But the crowds in Harare on Saturday were making it clear the country was ready to move on without him.
Concerns remain about who next would be in charge and what freedoms might be available if the military lingered in power - or if Mr Mugabe’s longtime but recently fired deputy led a new government.
However people revelled in the rare chance to speak out, in an event approved by the military.
Veterans of the long liberation war against white minority rule, once close allies of Mr Mugabe, took part, along with opposition activists who have faced police crackdowns by the Mugabe government.
One driver was so jubilant that he got out of his moving car and danced in front of it for a couple of minutes as the empty vehicle coasted slowly down a street lined with cheering crowds.
Some of the demonstrators were removing street signs with the name of President Robert Mugabe and stomping on them.