The MP for the safe Conservative seat of South Staffordshire since 2010, Mr Williamson served as Chief Whip until his promotion, and is famous for having a pet tarantula in his office.
Sir Michael quit after admitting his behaviour had “fallen below the high standards required” in the role and acknowledging that what might have been acceptable in the past was no longer appropriate.
Mr Williamson, who is a key ally of Theresa May, will replace him in the job, having never held ministerial office.
He did not speak to reporters as he left Number 10.
Foreign Secretary Boris Johnson said Sir Michael had “a record to be extremely proud of” as Defence Secretary.
Mr Johnson told the Press Association: “I’ve known Michael for many years. He has been a great colleague and a great Defence Secretary.
Mr Williamson has been replaced as Chief Whip by his Deputy, Stirling-born Julian Smith, who represents the seat of Skipton and Ripon in Yorkshire.