A former private secretary to the Speaker, Angus Sinclair, told BBC Two’s Newsnight that Mr Bercow undermined him by mimicking him, swearing and shouting, and once even smashed a mobile phone by throwing it on to the desk in front of him.
The Speaker’s office issued a firm denial of the allegations, which come amid an independent judge-led inquiry into claims of bullying of staff in the Palace of Westminster.
Tory MP Andrew Bridgen, a long-standing critic of the Speaker, said Mr Bercow should “consider his position”.
He pointed out that Mr Bercow had originally said he would serve no more than nine years in the Speaker’s chair after being elected to the post in 2009.
The North West Leicestershire MP told BBC Radio 4’s Today programme: “I think, at a time when we are looking for culture change in the House of Commons with regard to bullying and harassment, I think that’s very difficult if the titular head of that organisation is mired in these allegations.”
Mr Bridgen noted the Speaker’s denial of bullying, but added: “We don’t know whether he has misled the House unless there is a full investigation, but he holds such power in the House of Commons that it is very difficult to have an independent investigation.”
He said there was an “atmosphere of fear and intimidation of staff at the House of Commons”.
His comments were dismissed by Labour MP Barry Sheerman as “total nonsense”.
Mr Sheerman said Mr Bercow had been the “best reforming Speaker for 100 years” but a “small group” of parliamentary officials “hated” his modernising efforts.
An investigation into the working culture at Westminster is being led by Dame Laura Cox following an earlier Newsnight report alleging that clerks and other officials were bullied by MPs including Mr Bercow.
But critics say it risks giving a “clean slate” to perpetrators, because it will examine the working culture in the Commons but will not investigate individual cases or reopen past complaints.
Mr Sinclair was private secretary to Mr Bercow’s predecessor, Michael Martin, and retained the position when the new Speaker took over in 2009.
He told Newsnight that Mr Bercow undermined him in front of other staff, shouted, swore and attempted to physically intimidate him.
The Speaker was prone to “over-the-top anger”, he claimed, adding: “I’m not sure he was completely in control of it. The arms would wave around.”
And he said: “There was one afternoon I was working at my desk and he came in and was absolutely furious about something...
“There was a lot of bad language and suddenly his mobile phone, which he’d been holding, was flung on the desk in front of me and broke into a lot of bits.”
Following the 2010 general election, he said the Speaker told him he was no longer needed.
He said he was given “compulsory early retirement”, with an £86,250 pay-off dependent on him signing a non-disclosure agreement barring him from making complaints about his treatment in the House.
Newsnight has previously reported that his successor, Kate Emms, was allegedly bullied by the Speaker - a claim Mr Bercow denies.
Mr Sinclair said he felt that if he had not signed the non-disclosure agreement, but had instead put in a complaint, she might not have been put in that position.
“What I’d done was sign a cover-up and, in a cynical way, I’d been paid to do it, and that’s not a good feeling,” he said.
A spokesman for the Speaker’s office said: “Mr Speaker strenuously denies that there is any substance to any of these allegations.
“Mr Speaker has a superb team of dedicated, effective and long-serving staff - five of whom have worked for him very happily for a combined total of over 40 years.”