A survey by the British Medical Association in Scotland found almost two-thirds (64.7%) of those questioned - mainly consultants and GPs - had either received a large pension tax bill or were expecting one.
As a result, 40% of consultants and almost the same proportion (39.8%) of GPs are now thinking about cutting back their work for the NHS.
Meanwhile, 22.3% said they could take early retirement as a result of these bills - which have previously been estimated to cost doctors an average of £18,500.
The BMA released the findings of its survey as paediatric consultant Graeme Eunson took over as chairman of the organisation's Scottish consultants committee.
It comes just a week after official figures showed the NHS in Scotland is short of more than 500 consultants - with vacancy rates at their highest since 2007.
Another doctor said the "pensions changes have messed up a lot of lives and are likely to lead to early retirements at a time when recruitment and retention is already a problem".
Dr Eunson said the number of doctors who could quit the NHS because of large pension tax bills was "extremely concerning".
"These bills are forcing many senior doctors to cut the hours they work or decline or reduce waiting list initiative work," he said.
"It's extremely concerning that almost a quarter of those who took part said they were considering early retirement as a result.
"In this current state, when we are desperately calling out for more doctors, the last thing we need is to lose experienced, talented doctors to early retirement - forced out by punitive bills that actually financially penalise them for taking on work.
"This is bad for the NHS, bad for care and bad for patients. If we want to preserve already vulnerable services we need action, and we need it now."
The BMA carried out the survey after warning annual and lifetime allowances for pensions were acting in a "complex and punishing combination" for doctors, leaving some medics worse off if they worked additional hours.
A total of 593 medics, including 251 consultants and 299 GPs, took part in the research.
flexibility for pension schemes.
He said: "Recently, the English Department of Health launched new guidance which includes greater flexibility in the pensions scheme.
"This is a step forward and we desperately need the Scottish Government to act quickly to match or better this offer - which they have indicated they will - and we are pushing them hard to publish the details.
"We are already seeing trusts in England offer this flexibility, and the Scottish health service urgently needs the same."
He insisted ultimately the UK Government needed to act.
"The long-term answer to this lies in pensions tax reform, which is the responsibility of the Treasury," Dr Eunson said.
"As our figures show today, this situation is impacting on doctors and care now.
"We cannot afford to wait any longer for solutions to be put in place or we will go on losing vital capacity from the workforce."