The MP for Hitchin and Harpenden told BBC Radio 4's Today programme: "The reason why I voted for the Prime Minister (in the confidence ballot) was that he was clear that he felt he earned the right to rebuild trust in him - rebuild trust in the Government - after the issue of parties at Downing Street.
"I thought that was right, to give him that time.
"But I think that in the last few weeks we've seen that things haven't improved. They've got a lot worse.
"I think the behaviour of Downing Street over the Chris Pincher affair was really appalling. And I, personally, just couldn't think I could defend that sort of behaviour any longer."