Lib Dem, Labour, Tory, Green and independent councillors came together to vote unanimously for the motion at York Racecourse with members of the public joining in the chorus of condemnation for the duke.
Many said they wanted Andrew to relinquish his Duke of York title and remove what Labour councillor Aisling Musson called “his stain of an association with this city”.
They said that, if the duke failed to act, the Queen or the Government should step in a remove the title.
Speaking after the meeting, Lib Dem Darryl Smalley, proposer of the motion, said: “The Honorary Freedom of York is the highest honour we, as a city, can bestow on those who represent the very best of York. The honour is held by many notable and accomplished people who carry it with pride and responsibly.
“Having been stripped of his military roles and royal patronages by the Queen, we believe that it is right to remove all links that Prince Andrew still has with our great city.
“I was pleased to see councillors of all parties support this motion and make it clear that it is no longer appropriate for Prince Andrew to represent York and its residents.
“The removal of this honorary title sends the right message that we as a city stand with victims of abuse.
“The next logical step is now for Prince Andrew to do the right thing and relinquish his Duke of York title.
“If he fails to do so, the Government and Buckingham Palace must step in to remove his title to finally end Prince Andrew’s connection to York.”
Honorary freedom recipients include the Duke of Wellington, Sir Winston Churchill, film composer John Barry and actress Dame Judi Dench.
Andrew was given the freedom as a wedding present from York in 1987 and he is the first person ever to have it removed.
First to speak from the public gallery was Gwen Swinburn, who said: “Mr Andrew Windsor is an utter disgrace.
“He brings shame and reputational damage everywhere he goes, including to our city.
“The motion to withdraw his status goes nowhere near far enough. He needs to be be declared persona non grata in York.”
Ms Musson told the meeting: “We owe it to the people of York, particularly those who have been affected by sexual violence, abuse or human trafficking.
“Our first duty is not to our reputation but to their well-being and protection, and to remove this stain of an association with this city which I’m sure has weighed heavily on their minds.”
The motion was brought following the settlement of Andrew’s sexual assault civil case with Virginia Giuffre.
All councillors voted in favour except the Lord Mayor and Lord Mayor elect who said they were abstaining as was the convention in their office.
The duke has faced widespread calls for him to give up his Duke of York title, which is one of his last remaining major titles after the Queen stripped him of his honorary military roles and he gave up his HRH style.
Both Buckingham Palace and a spokesperson for the Duke of York declined to comment.