The Labour leader urged Tories to do consider what they were a part of after Boris Johnson wrongly accused him on Monday of failing to prosecute Jimmy Savile.
Speaking at Prime Minister’s Questions, Sir Keir was joined in condemning Mr Johnson’s comments by the Speaker.
Sir Keir said: “I just want to say to the members opposite, theirs is the party of Winston Churchill.
“Our parties stood together as we defeated fascism in Europe.
"Now their leader stands in the House of Commons parroting the conspiracy theories of violent fascists to try and score cheap political points. He knows exactly what he is doing, it is time to restore some dignity.”
Mr Johnson refused to apologise, instead doubling down on the false claim.
He replied: “On his first point, I don’t want to make heavy weather of this but I am told that in 2013, he apologised and took full responsibility for what had happened on his watch. I think that was the right thing to do.”
Sir Keir had apologised at the time for the organisations failings, but had nothing to do with the decision.
Mr Johnson was also rebuked by the Commons Speaker over his discredited claim.
Sir Lindsay began Prime Minister’s Questions by reminding MPs about the rules of parliamentary etiquette when it comes to alleging another member has lied or misled the House.
He said it is not for him to change the current rules unilaterally, adding: “Therefore, I ask members to respect this approach.
“I know feelings run high on important issues we discuss but there are plenty of ways of making strong feelings felt within the rules and without placing the chair in the invidious position of having to order members to withdraw on seeking their suspension.”
Earlier, former minister Tobias Ellwood became the latest senior Conservative figure to criticise the Prime Minister’s comments.
Speaking on Sky News, he said: “Who advised the Prime Minister to say this?
“We’re better than this, we must seek to improve our standards and rise above where we are today.”
Simon Hoare, the Conservative chairman of the Commons Northern Ireland committee tweeted: “The Jimmy Savile false allegation should be withdrawn.”
On Tuesday, Sir Charles Walker, the vice chairman of the Tory backbench 1922 Committee, told Channel 4 News that the comment had been a “mistake” and Mr Johnson should apologise.
“I wish he hadn’t done it. I suspect he wishes he hadn’t done it,” he said.
“I think it might have been at the time in the heat of battle, because it gets very heated on the floor of the House. It’s very febrile. But I think it was a mistake.
“Of course he should withdraw it. It didn’t add to the overall debate. It’s really good though to be able to apologise. I think you never get marked down for an apology. It’s not a sign of weakness.”
However, Communities Secretary Michael Gove insisted the Prime Minister had nothing to apologise for.
He added: “Keir Starmer acknowledged that mistakes had been made by the organisation of which he was head, to his credit. He was very clear about those mistakes.”