The King said he and his mother felt “the greatest admiration for the Scottish people, for their magnificent achievements and their indomitable spirit” and it was a comfort to know the “true affection” in which she was held.
In a speech that also quoted Robert Burns, he pledged to take up his new duties “with resolve to seek always the welfare of our country and its people”.
Earlier, Nicola Sturgeon led heartfelt tributes in Holyrood to “the anchor of our nation”.
The First Minister sparked laughter from the new King when she recalled how her husband, SNP chief executive Peter Murrell, saved one of the Queen’s corgis from being electrocuted at Balmoral.
She told the chamber: “My husband and I were with The Queen before dinner when the drawing room light started to flicker.
"To my great alarm – he was, after all, in the presence of Her Majesty – my husband suddenly leapt up and darted across the floor.
"Peter had spotted the cause of the flickering light. One of the Queen’s young corgis, a beautiful pup called Sandy, was eating through a lamp switch.
"Thankfully, tragedy was averted and Sandy emerged unscathed – though not before a ticking off from his mistress.”
Elsewhere, Ms Sturgeon said she cherished the memory of a train journey from Edinburgh to Tweedbank with the Queen and Prince Philip to mark the opening of the Borders Railway in 2015.
"It was one of the great privileges of my life,” the First Minister said.
She recalled first seeing the Queen when she was nine years old. The monarch had visited Irvine, the First Minister's hometown, to open the Magnum Leisure Centre in 1979.
Moving a Motion of Condolence in Holyrood, Ms Sturgeon offered her “heartfelt condolences” to the King on the death of “Queen Elizabeth, Queen of Scots”.
Clad in a kilt of Prince Charles Edward Stewart tartan, the new King, who was accompanied by the Queen Consort, said: “I know that the Scottish Parliament and the people of Scotland share with me a profound sense of grief at the death of my beloved mother.
"Through all the years of her reign, the Queen, like so many generations of our family before her, found in the hills of this land and in the hearts of its people a haven and a home.
"My mother felt, as I do, the greatest admiration for the Scottish people, for their magnificent achievements and their indomitable spirit.
"And it was the greatest comfort for her to know, in turn, the true affection in which she was held.
"The knowledge of that deep and abiding bond must be to us a solace as we mourn the end of a life of incomparable service.
"If I might paraphrase the worlds of the great Robert Burns, my dear mother was the friend of man, the friend of truth, the friend of age and guide of youth.
“Few hearts like hers, with virtue warmed, few heads with knowledge so informed.”
The King said he was "determined with God’s help, and with yours” to follow the Queen’s example.
He said his Scottish titles, such as Duke of Rothesay, now passed to his eldest son, Prince William, “who I know will be as proud as I have been to bear the symbols of this ancient kingdom”.
He added: “I take up my new duties with thankfulness for all that Scotland has given me, with resolve to seek always the welfare of our country and its people, and with wholehearted trust in your goodwill and good counsel, as we take forward that task together.”
Scottish Conservative leader Douglas Ross said the Queen “seemed as permanent as the stones of Edinburgh Castle".
The length of her reign “meant that she was an anchor to our history”, he added.
Scottish Labour leader Anas Sarwar said the Queen “brought our nation together”, adding: “We know how much she loved Scotland, and we loved her back.”
He said the Queen reminded him of his own grandmother.
Scottish Liberal Democrat leader Alex Cole-Hamilton said: “From the Blitz to the pandemic, through times of tempest and times of calm, sustained as always by the Duke of Edinburgh, her beloved strength and stay, she was the embodiment of constancy and forbearance in the face of remarkable change.”
Patrick Harvie, the co-leader of the Scottish Greens, who does not support the monarchy, hailed the “extraordinary, progressive” social change that took place during the Queen’s reign.
He spoke of his hope that the new King would have the opportunity to witness change “just as transformational, and more”.
Alison Johnstone, Presiding Officer of the Scottish Parliament, started the session by paying tribute to the Queen for her “extraordinary life and work” before leading MSPs, who were dressed in dark colours, in a two-minute silence.
Former first ministers, including Alex Salmond and Jack McConnell, were among the guests watching from the public gallery.
MSP Stuart McMillan played the bagpipes as the Presiding Officer escorted the King and Queen Consort from the chamber, before the monarch met with young people in the main hall of the parliament.
The Queen visited the Scottish Parliament ten times, attending six opening ceremonies, the tenth and twentieth anniversary sessions, the opening of the parliament building in 2004 and a sitting in Aberdeen for the Golden Jubilee in 2002.
"No-one would ever argue that Scottish politics is the business of the meek, the passive or the fainthearted," she said as she addressed Holyrood in 2011.
For this special session, at least, the rivalries that define and divide Scottish politics were put firmly to one side.