Harry and Meghan "need some time away to recharge" and decide their plans for next year, a royal source told The Sun.
Former actress Meghan has celebrated with the royals on the monarch's Norfolk estate for the last two years - including before she married Harry.
The duke and Californian-born duchess are said to be taking a six-week break from royal duties and are expected to be heading to the US for Thanksgiving with Meghan's mother Doria Ragland later this month.
A source said the couple have not yet finalised where to spend Christmas.
Options are thought to include their Windsor home Frogmore Cottage, remaining with Ms Ragland in America, or even a foreign holiday.
Well-wishers gather each year to see the royals take their traditional stroll on December 25 to the morning service at the Church of St Mary Magdalene in Sandringham.
The Windsors enjoy a festive lunch back at Sandringham House and then settle down together to watch the Queen's Christmas Day speech.
Over the past year, Harry and Meghan have faced reports of a rift, first between Meghan and the Duchess of Cambridge and then between Harry and his brother the Duke of Cambridge.
In a television documentary last month, Harry, when asked about the situation, said he loved his brother dearly but they were "on different paths at the moment" and have "good days" and "bad days" in their relationship.
William, the Queen and other senior royals were said to have been left "very worried" about the Sussexes after their emotional appearance in the ITV programme, filmed on their recent tour to Africa.
Harry described his mental health and the way he deals with the pressures of his life as a matter of "constant management".
Meghan admitted feeling vulnerable, and spoke of the difficulty in coping with intense tabloid interest, saying: "It's not enough to just survive something, that's not the point of life. You have got to thrive."
The couple's high-profile visit to Africa, carried out at the request of the Government, was overshadowed on the penultimate day when the duke condemned the British tabloid press, as
Meghan launched a lawsuit against Associated Newspapers.
The duchess is suing over an alleged breach of copyright and privacy after the Mail on Sunday published a private letter between her and her estranged father.
The newspaper said it stands by its story and will be "defending this case vigorously".
Buckingham Palace declined to comment.