Their home is strikingly modern, although it has the feel of a comfortable country house. There is a grand piano set on a marble plinth and a large walk-in drinks cupboard.
The house, overlooking Muirfield golf course in Gullane, was named The Pelicans, after the birds in the family crest, which had been inset into the floor.
And the luxurious villa would end up at the epicentre of Britain's most notorious corruption scandal – the Poulson Affair.
The scandal would eventually implicate ministers in Ted Heath's government and lead to the resignation of Home Secretary Reginald Maudling, as well as the jailing of some of its central players.
It brought to light allegations of Masonic influences and a UK-wide network of corrupt MPs, politicians and council officials who were able to supply Poulson with influence and lucrative contracts in the 1950s and 1960s.
Pottinger was in his mid-50s and enjoying the fruits of a highly successful career at the heart of the Edinburgh establishment when his world came crashing down.
Nicknamed "Gorgeous George", he was on course to become Scotland's top civil servant and craved the knighthood that traditionally went with the job.
A member at Muirfield, he was at the club – dining with two sheriffs and a High Court judge – when he received the fateful call that was to shatter his glittering career.
The Fraud Squad had turned up. His wife phoned the club and told him: "We've got visitors. You have to come home at once."
That was June 22, 1973. Pottinger had already been suspended on full pay from his post in charge of the Department of Agriculture and Fisheries in Scotland for nearly a year, following allegations which surfaced at international architect John Poulson's bankruptcy hearing.
Pottinger walked home, where he was arrested before being held overnight in the city police headquarters in the High Street.
The next day he was driven to Leeds to appear in court alongside Poulson on corruption charges.
The pair had met ten years earlier. Poulson, an architect with the biggest practice in Europe, had been approached about the design of the tourist complex proposed for Aviemore.
Soon afterwards, Pottinger was seconded to help the late Lord Fraser of Allander develop tourism in the Highlands and became involved with the Aviemore plan.
The pair developed a close relationship, regularly dining together, almost certainly at Poulson's expense. But what no-one suspected at the time was how far that "friendship" had extended. The corruption trial jury heard how the architect showered the civil servant with "gifts", including foreign holidays, Savile Row suits and a Rover car. He even designed his home for free and paid 20,000 towards the mortgage.
The court heard the gifts were worth 30,000, the equivalent of more than 500,000 today.
Pottinger later spoke of Poulson's "compulsive generosity" towards The Pelicans.
He recalled: "I wanted to build a house of my own and I had earlier been disappointed in my efforts to secure a site at Carberry. Now I looked round for somewhere else and John, with his undeniable reputation and his friendship for me, seemed the obvious man to draw up the plans.
"The arrangement was that I would obtain the maximum mortgage available and he would cover the balance as a gift. It amounted to more than 20,000.
"I thought he had paid more than he intended, but at the time he was a very wealthy man."
Soon after Pottinger was jailed, the house was sold for 50,000.
Conservative Home Secretary Reginald Maudling was forced to resign as he had been involved with one of Poulson's companies and was later shown to have influenced the government of Malta to win a contract for him.
Two other MPs, Tory John Cordle and Labour's Albert Roberts, were censured for their involvement. T Dan Smith, leader of Newcastle City Council, was jailed.
Pottinger believed he had done nothing wrong – and claimed if the trial was in Scotland, he would have been found not guilty.
He could not be tried in Scotland because, as a member of both Muirfield and Edinburgh's exclusive New Club, he knew nearly all the country's judges.
Former Evening News business editor Ian Burrell, who gave Pottinger lifts to and from the trial, says the civil servant was not in a position to influence work being given to Poulson.
"But the police believed Poulson was waiting until Pottinger became head of the civil service in Scotland," he recalled. "Then he would have pressured him for government contracts. Poulson had carefully filed every single holiday postcard, Christmas card and birthday card sent to him by Pottinger. He had also ordered his secretary to tape all phone calls made by Pottinger to his office."
Pottinger and Poulson were each jailed for five years, although Pottinger's sentence was later reduced by a year on appeal.
He was dismissed from the civil service, forfeited the lump sum of 12,000 payable on his retirement and had his pension cut in half.
After his release he moved to Cambridgeshire where he wrote books until his death, aged 81, in 1998.
Unqualified architect who exploited victims' greed
JOHN POULSON built the biggest architectural practice in Europe by simply bribing MPs, councillors and civil servants.
In the 1960s he was said to be earning 1 million a year in fees.
His efforts allegedly became concentrated on bribing Labour councils in the north of England and Scotland after discovering they were cheaper to corrupt than Conservatives.
An unremarkable – and unqualified – architect, he was incredibly skilled at sizing up the people he met and ruthless at exploiting their greed.
He tended to soften up his prey by taking them out for lavish hotel dinners, before showering them with "gifts".
One of the most prominent politicians to accept his bribes was T Dan Smith, the leader of Newcastle City Council, who was eventually jailed for six years.
The highest-ranking figure to be forced out as a result of the scandal was Conservative Home Secretary Reginald Maudling, who was a business associate of Poulson.
The architect was ultimately brought down as a result of his own greed, over-stretching himself to the point of bankruptcy
Poulson, who died in 1993, was sentenced to seven years in prison in 1974. He later wrote an autobiography, appropriately entitled The Price. However, it never made it into bookshops following a flurry of libel writs from former business associates he named as taking bribes.