Passenger numbers fell by 6 per cent to 438 million in 2009-10, in the biggest annual reduction since 1998.
However, north of the Central Belt, the number of people boarding buses plummeted by one quarter last year, and the only area to show growth was around Edinburgh.
The total number of journeys is at its lowest since at least the 1999 total of 455 million, since when they have climbed to peak at 498 million in 2007.
Comparisons with the totals for previous years are not possible because of changes to the way the figures are compiled.
Experts said the third consecutive annual fall was largely due to increased unemployment and the squeeze on household spending reducing shopping and leisure trips.
However, they predicted the situation could get even worse from feared fare increases and service cuts caused by a reduction in funding from the Scottish Government and councils.
The figures show free bus travel by the over-60s and disabled people also fell for a third year, to 146 million journeys.
Labour infrastructure spokesman Richard Baker said: “This shows a real crisis in Scotland buses, and coupled with changes to the support given to bus companies, there is an impending disaster.
“People are taking fewer journeys and prices are rising.”
Chris Cheek, editor of the Bus Industry Monitor website, said: “The fact there are fewer jobs and fewer trips to retail centres means there will fewer bus passengers.
“That will always trump the number of people who leave their cars at home because of high fuel prices.
“Many drivers will use their cars because they are paying for them anyway, rather than take another form of transport.”
Gavin Booth, senior officer for Scotland for watchdog body Bus Users UK, said northern Scotland had seen the biggest fall in passengers because services were less frequent and more prone to cuts in council funding.
Liberal Democrat transport spokesman Jim Hume said: “Times are tight at the moment for bus companies with the rising cost of fuel and operating costs, and the SNP are doing nothing to help bus companies.
“The SNP have landed a blow on bus companies with a cut to the bus operators’ grant.”
A Transport Scotland spokeswoman said: “A number of factors are likely to have impacted on bus patronage in recent years, not least the two unprecedentedly bad winters, challenging economic conditions, and effective use of smartcard technology to combat fraudulent use of concessionary travel entitlements.
“It is simply untrue to say this government has reduced funding for local bus services over this period.”
She said it had increased by 7 per cent between 2007-8 and 2010-11, with total support for the industry of £250 million a year.