Mr Biden said at an event in Washington on Monday evening "it matters" that for the first time a person of colour, who is also the child of Indian immigrants, could rise to the highest political office in the country.
"And whether it's the United Kingdom, where just today, we've got news that Rishi Sunak is now the prime minister. As my brother would say, 'go figure'," Mr Biden said.
"And the Conservative Party, expected to become the Prime Minister, I think, tomorrow when he goes to see the King.
"Pretty astounding. A groundbreaking milestone. And it matters, it matters."
Diwali is a five-day "Festival of Lights" and a celebration by Hindus, Sikhs and Jains of the triumph of good over evil.
Mr Sunak's grandparents were from Punjab state before the Indian subcontinent was divided into two countries, India and Pakistan, in 1947 after British colonial rule ended. His family settled in the UK in the 1960s and he was born in Southampton in 1980.
His rise to power has prompted a sense of pride among Indians, with that country's leader Narendra Modi earlier offering him "special" Diwali wishes as the "living bridge" of UK Indians.
Prime Minister Modi tweeted: "As you become UK PM, I look forward to working closely together on global issues, and implementing Roadmap 2030. Special Diwali wishes to the 'living bridge' of UK Indians, as we transform our historic ties into a modern partnership."
Mr Sunak won the Tory leadership contest without a vote being cast after rivals Penny Mordaunt and Boris Johnson dropped out, and will replace Liz Truss in No 10 on Tuesday.
Indian media were impressed by his victory, with New Delhi Television announcing "Indian son rises over the empire" and India Today news channel taking a jibe at the UK's economic and political turbulence, using the Hindi term for someone of Indian background: "Battered Britain gets 'desi' big boss."
Sunak is set to meet with King Charles III today to become Prime Minister.