He also said he was forced to resign as an independent adviser after the government “tried to silence” him.
But his comments were branded “complete nonsense” by Conservative MP Chris Philp.
The Labour peer has stood down as chairman of the National Infrastructure Commission, claiming the Prime Minister had sided with Ukip and the “Tory hard right”.
And he lashed out at the government’s handling of the East Coast rail franchise, claiming a taxpayer-funded “bailout” was evidence that Whitehall was too distracted by Brexit to focus on running the country. Whitehall sources indicated that Lord Adonis, whose frequent criticism of Brexit had caused outrage among Tory Eurosceptics, had “jumped before he was pushed”.
However, the peer insisted it was his decision to quit, accusing Downing Street insiders of “dirty tricks”.
He urged others to follow his example in speaking out.
He said yesterday: “I don’t believe the Brexit policy is sustainable and I believe that people like me who have a leadership position should be out there next year arguing passionately with the British people as to why staying in the EU is the right thing to do, alongside sorting out the deep social crisis which Britain is now in the midst of, which I think is a good part of the reason why Brexit started in the first place.”
Lord Adonis said the 17.4 million voters who backed Brexit in the referendum were “absolutely entitled to their view” but “I’m absolutely entitled, indeed I believe it’s my duty, to say why I think that was wrong”.
“Very few of the people who voted for Brexit voted, I believe, to make themselves poorer,” he said.
A government source said: “He’s been moving closer towards the exit door with each new onslaught he makes against Brexit.
“He’s now walked through the door before he was pushed.”
Former Conservative party leader Iain Duncan Smith said: “Lord Adonis’s departure is long overdue. It’s a bit rich for him to pontificate on what he calls populism, but what most would refer to as democracy, when he himself has never been elected by a public vote. He has instead relied on preferment from others.”