The UK foreign secretary was advised to personally ask his Afghan counterpart for urgent support, only to delegate it to a junior minister.
Mr Raab had been on holiday at a five-star resort on the Greek island of Crete on Sunday – the same day Taliban militants entered Afghanistan's capital Kabul.
Now opposition parties have demanded the senior Tory Minister resign or be sacked.
Labour's shadow foreign secretary Lisa Nandy said: "How can Boris Johnson allow the foreign secretary to continue in his role after yet another catastrophic failure of judgement?
"If Dominic Raab doesn't have the decency to resign, the Prime Minister must show a shred of leadership and sack him."
Labour’s shadow Scotland secretary Ian Murray said: “This is a gut-wrenching dereliction of duty.
“Raab deserted the brave interpreters who have supported the British Armed Forces, refusing to help rescue them while he relaxed on a sunbed in Crete.
“His position is untenable and if he doesn’t have the decency to resign, then the Prime Minister must show leadership and sack him without delay.
"This government is just appalling.”
By delegating the call to his Afghan counterpart, Hanif Atmar, there was a delay in the conversation happening.
The SNP Westminster leader Ian Blackford also demanded Mr Raab be sacked.
He said: “Tory ministers cannot wash their hands of responsibility for this foreign policy disaster.
"Dominic Raab has failed to perform his basic duties as foreign secretary and he has put people's lives at risk.
"His position is completely untenable and he must resign, or be sacked.”
Lib Dem Layla Moran, the party’s foreign affairs spokesperson, said: “Dominic Raab must resign today.
"If he does not, the Prime Minister should finally show some leadership, and sack him.”
Liberal Democrat leader MP Jamie Stone labelled the move “shameful”.
The Caithness, Sutherland and Easter Ross MP said: “Dominic Raab must resign. There is absolutely no excuse for Raab’s delay and delegation when it came to making this phone call.
“He has betrayed the brave interpreters and staff who have aided UK Armed Forces personnel for the past 20 years.
“Raab has personally overseen one of the biggest foreign policy disasters of our lifetimes and, as a result, the security of our country is now at risk.
“It is utterly revealing of this Cabinet that, when it came to it, our foreign secretary chose a beach bed in Crete over the people of Afghanistan, fleeing for their lives from a crisis that he himself had a part to play in.”
Thursday morning defence secretary Ben Wallace defended Mr Raab over the accusations.
He said: "What I do know is that at no stage in the last few weeks have I had a problem with the foreign secretary or anyone else in that department in making sure we should process, get people through.
"The facts are changing rapidly on the ground and, as he said yesterday in Parliament, he has managed to do all his calls when he needed to, his Cobra calls etc."
The Foreign Office has appeared to confirm the reports, saying: “The foreign secretary was engaged on a range of other calls and this one was delegated to another minister.”
The UK is meanwhile launching a diplomatic push to encourage allies to join it in offering to take in Afghan refugees fleeing the Taliban regime.
The UK Government has announced Britain will take up to 20,000 people wanting to exit Afghanistan as part of its resettlement scheme, with 5,000 due to be accepted in the next 12 months.
Downing Street said the government would be encouraging international partners to emulate “one of the most generous asylum schemes in British history” – but Labour said the offer was not bold enough.
Mr Wallace has said the UK is using “every space possible” on its evacuation flights leaving Afghanistan.
The defence secretary said other nations had been offered places on UK planes, with Nato interpreters and some Japanese people “in need” among those to receive help.
Reports have suggested evacuation flights to other countries had left Kabul with only a handful of people on board.
But Mr Wallace told Times Radio: “Our people are getting through, we haven’t sent a single empty plane home.
“And I don’t think many other nations have.
“I can’t speak for other nations, obviously, but fundamentally the key here is when we have a plane, if we have a single empty seat, we will offer it to other nations.
“We’ve taken out interpreters who work for Nato, for example, we’ve taken out fellow European or other [people] … we took some Japanese people out recently who were in need, so we will use every space on our planes possible.”
The British ambassador to Afghanistan Sir Laurie Bristow has said Foreign Office personnel hope to get “at least” 1,000 people out of the country every day – but warned there were “days, not weeks” left to complete the mission.
On Wednesday, Prime Minister Boris Johnson said the UK had secured the “safe return” of 306 UK nationals and 2,052 Afghan nationals as part of its resettlement programme, with a further 2,000 Afghan applications completed and more being processed.
Mr Wallace said UK troops would stay in Afghanistan “as long as the United States forces are running the airport”.
US President Joe Biden suggested his troops would stay in Afghanistan until the evacuation of Americans is complete, even if that runs past the August 31 deadline for withdrawal.
Mr Wallace told BBC Breakfast: “They’re doing everything from the control tower to the firefighters, they are running that airport.”