The three-term governor's decision, which will take effect in two weeks, was announced as momentum built in the legislature to remove him by impeachment.
It came after New York' s attorney general released the results of an investigation that found Mr Cuomo sexually harassed at least 11 women.
Investigators said he subjected women to unwanted kisses, groped their breasts or buttocks or otherwise touched them inappropriately, made insinuating remarks about their looks and their sex lives, and created a work environment "rife with fear and intimidation".
Kathy Hochul, a 62-year-old Democrat and former member of Congress from the Buffalo area, will become the state's 57th governor and the first woman to hold the post.
She said stepping down was "the right thing to do and in the best interest of New Yorkers".
Even as the scandal mushroomed, Mr Cuomo was planning to run for re-election next year.
He still faces the possibility of criminal charges, with a number of prosecutors around the state moving to investigate him.
The string of accusations that spelled his downfall began to unfold in news reports in December and went on for months.
He called some of the allegations fabricated, forcefully denying he touched anyone inappropriately, but he acknowledged making some aides uncomfortable with comments he said he intended as playful, and apologised for some of his behaviour.
He portrayed some of the encounters as misunderstandings attributable to "generational or cultural" differences, a reference in part to his upbringing in an affectionate Italian-American family.
As a defiant Mr Cuomo clung to office, state legislators launched an impeachment investigation, and nearly the entire Democratic establishment in New York deserted him - not only over the accusations, but also because of the discovery that his administration had concealed thousands of Covid-19 deaths among nursing home patients.
The harassment investigation ordered by New York attorney general Letitia James and conducted by two outside lawyers corroborated the women's accounts.