Gebrselassie, who has been a dominant force in distance running for the last two decades, made the decision after he limped out of the ING New York Marathon with a knee injury.
The 37-year-old pulled up after 16 miles of yesterday's race before telling a press conference: "I have never thought about retiring before but now I am. I am not going to run any more."
The Ethiopian broke the marathon world record in Berlin two years ago when he clocked a time of two hours, three minutes and 59 seconds at the age of 35. He almost pulled out of yesterday's race - his first New York Marathon - after struggling with an injury to his right knee all week.
The popular runner had an MRI scan yesterday, which showed he had fluid and tendinitis in his knee. He was clearly in distress as he pulled up, grimacing while crossing the Queensboro Bridge with ten miles of the marathon to go.
That appears to have prompted the veteran's decision to call time on a career which many consider the greatest in distance running history.
"When I announced this, maybe everybody will become a little bit shocked," said Gebrselassie."I don't want to complain anymore after this. It's better to stop here. I have had no discussion with my manager, with anybody, only myself. That's why it's better to stop here.
"I never think about retiring. But for the first time, this is the day."
Gebrselassie battled an impoverished upbringing to grab the attention of the world when he won the 10,000 metres at the World Championships in Stuttgart at the age of 20.
Three more victories at the same event, and Olympic Gold at Sydney and Atlanta, would cement his claim as one of the leading athletes in the world.
After 2004, Gebrselassie chose to move away from the track to win four times in the Dubai Marathon, once in Amsterdam and four times in Berlin, where he broke the world record in 2007 and 2008.
Gebrselassie had hoped to beat the course record of two hours seven minutes and 43 seconds set in 2001 by his countryman Tesfaye Jifar, but in the end Ethiopian Gebre Gebremariam won the race in a time of 2:10.39.