A spokesman for the family of the long-serving disc jockey confirmed he had died “peacefully at home” with his wife, Alicia, by his side.
Sir Jimmy was part of the original line-up of DJs at BBC Radio 1 when the station was launched in 1967, and went on to enjoy a 35 year long career on the airwaves.
He was best known for his regular afternoon programme on BBC Radio 2, which he began in 1973 and hosted until his retirement in 2002.
Sir Jimmy, who had a successful music career before becoming established as a broadcaster, interviewed each prime minister of the day since Harold Macmillan during his time at the station. The late Baroness Thatcher was a guest on his show no fewer than 14 times.
The Queen is believed to have been among fans of the “housewives’ choice” during his time on air.
Sir Jimmy’s former Radio 2 colleague, Ken Bruce, was among those to pay tribute.
He said: “So very sad to hear about the death of my old friend Sir Jimmy Young. One of the most able broadcasters I ever worked with.”
Time Team presenter and Blackadder actor Tony Robinson said: “So sad. Such memories from my childhood and teens. I loved his versions of The Man From Laramie and Unchained Melody.”
Television presenter Piers Morgan said: “Another giant of British broadcasting dies just months after his great friend & colleague Terry Wogan.”
Born Leslie Ronald Young in Cinderford, Gloucestershire, Sir Jimmy was a miner’s son who developed a love for music at a young age.
His mother taught him to play the piano at the age of seven and he became a choirboy at Gloucester Cathedral.
Prior to his broadcasting career, Sir Jimmy was a singer who enjoyed a run of hit singles during the 1950s. He also worked as a clerk in the Ministry of Education and a manager of a hair salon.
In the final year of his career, Sr Jimmy had been off air for several months after surgery but returned for his final fortnight.
He would later return to Radio 2 in 2002 where he joined Desmond Carrington on the weekly show, Icons of the 50s, contributing one of his own records to each programme.
Asked to sum up Sir Jimmy’s appeal, Jeremy Vine, his successor at Radio 2, once said: “Jimmy was just a totally ordinary, honest bloke, the least pretentious person you could ever imagine. And his audience adored him for it.”