A mountain of refuse containing everything from fridges, washing machines, baths, toilets and rotting bin bags to hazardous building waste had been piling up on ground beneath the M8 in the Blochairn area of the city.
Organised crime gangs are thought to be behind “systematic” illegal dumping at the site.
City leaders, residents and anti-litter campaigners had condemned the “horrendous” eyesore and highlighted the potential risks to public health.
Responsibility for the site, which was fenced off, but had been easily accessible after the gate was broken off, lies with engineering and infrastructure services company Amey.
The Scottish Environment Protection Agency (Sepa) has warned that criminals will be “found and caught”.
Now with under 50 days to go until world leaders and thousands of visitors are due to arrive in the city to attend the landmark United Nations climate conference, workers have turned up with diggers and rubbish skips to begin clearing away the debris.
Glasgow City Council said it was clear ordinary residents were not behind the tipping, which is being carried out on an industrial scale.
The removal operation has been welcomed.
A spokesman for the local authority said: “This site is not maintained by the council, but we fully welcome the work by the site manager to remove the illegally dumped waste and ensure the site is subsequently secured.
“Those responsible for fly-tipping at this site have shown a complete disregard for the environment and for the safety of any individual living or working in the vicinity of this environmental crime.
“We are working with Sepa and other partners to identify the culprits so that enforcement action can be taken.”
Unauthorised disposal of rubbish is against the law, carrying penalties of imprisonment and fines up to £40,000.
Serious and organised waste crime is estimated to cost at least £600 million a year to the UK economy, with Scots paying £53m annually for removal of illegal rubbish.
A spokesperson for Amey said: “Specialist operatives and equipment have begun the laborious process of identifying, separating and clearing of hazardous materials dumped at the M8 Junction 3 site.
“Once the area has been made safe all the remaining rubbish will be removed and the area cleaned before a new permanent barrier is erected.
“This is a crucial part of the operation, as we need to ensure this site remains clear, especially when we are tackling organised fly-tipping on a large scale.
“We will continue to work with our partners and the communities we serve to reduce the scourge of fly-tipping across our network.”
Jennifer Shearer, head of enforcement at Sepa, said: “This type of crime has a real impact on local communities, legitimate business and the public agencies who are working to tackle it.
“Waste crime can cause serious pollution. It puts communities at risk and places further stress on legitimate operators.”
Big names due to attend COP26, which runs from October 31 to November 12, include the Queen and the Pope, US president Joe Biden, naturalist and television presenter Sir David Attenborough and teenage climate campaigner Greta Thunberg.