The philosopher was dismissed as a housing tsar with immediate effect, the Ministry of Housing, Communities and Local Government confirmed.
A spokeswoman said: “Professor Sir Roger Scruton has been dismissed as chairman of the Building Better Building Beautiful Commission with immediate effect, following his unacceptable comments.
“A new chair will be appointed by the Secretary of State, to take this important work forward, in due course.”
Downing Street said Sir Roger’s comments were “deeply offensive and completely unacceptable” and that it was right that he had been dismissed.
“He was appointed because of his expertise in the built environment, but his comments are clearly distracting from the work of the commission and it is no longer right for him to act as a Government adviser,” a No.10 spokeswoman said.
The Government dismissed calls to fire the academic as a housing adviser in November after it emerged Sir Roger had said Islamophobia was a “propaganda word” and described homosexuality as “not normal”.
Sir Roger came under renewed criticism after commenting on the controversy.
The New Statesman America quoted Sir Roger as repeating his claim that Islamophobia was a propaganda word “invented by the Muslim Brotherhood in order to stop discussion of a major issue”.
Sir Roger said it was “nonsense” for critics to accuse Hungarian prime minister Viktor Orban of anti-semitism or Islamophobia.
He said: “The Hungarians were extremely alarmed by the sudden invasion of huge tribes of Muslims from the Middle East.”
Referring to philanthropist George Soros, Sir Roger said: “Anybody who doesn’t think that there’s a Soros empire in Hungary has not observed the facts.”
Labour led calls for him to be sacked, with the shadow women and equalities secretary Dawn Butler accusing him of invoking the “language of white supremacists”.
“His claim that Islamophobia does not exist, a few weeks after the devastating attack in Christchurch, is extremely dangerous, and his defence of the prejudice stoked by Viktor Orban’s government in Hungary is appalling,” she said.
“He should never have been appointed, but the Government should have sacked him when his horrific comments about rape, homosexuality, Islamophobia and anti-semitism came to light.
“Instead, they defended him, with the Secretary of State for Housing, Communities and Local Government praising him as ‘a very brave defender of free speech’.
“Theresa May must sack Roger Scruton immediately.
“If she doesn’t, it will be further evidence that she is turning a blind eye to the deep-rooted prejudices and racist views in the Conservative Party, and will again signal that her Government endorses these disgusting views.”
Remarking on last November’s controversy, Sir Roger said: “It’s upsetting because it’s meant to undermine your authority.
“And authority is the only thing I have, authority that comes from hard work and thinking.
“What surprised me was the kind of people who repeated this. You expect people who spend their lives on Twitter to have this store of malice but when it comes up in Parliament, as it did, I was astonished.”