The Deputy First Minister said security of medical supplies had been one of the “critical risks” identified by the Scottish Government after Brexit.
The prospect of a no-deal scenario is growing, with no agreement yet struck between the EU and UK ahead of the Brexit transition period ending on December 31.
On Thursday, UK Health Secretary Matt Hancock said he was "confident" a no-deal Brexit would not delay supplies and stressed that contingency measures were also in place.
The Pfizer vaccine, which the drugs company announced is 90 per cent successful at preventing Covid-19, is being manufactured in Belgium.
Mr Swinney said Scottish Government ministers were concerned about the issue of vaccine supplies being hit.
"We have seen the developments of the arrangements that are in place to deal with the consequences of Brexit, but none of us are certain about what those arrangements will be,” he said.
“There must be anxiety that there is the potential for disruption to medical supplies as a consequence of Brexit.
"That was one of the critical risks that the Scottish Government identified when we looked at the possibility of a no-deal Brexit scenario."
He said work had since been done on how supplies could be maintained, but it was "really damaging" to be leaving the EU during the pandemic.
Mr Swinney said: "There’s deep anxiety within the government about these questions.
"We hope to avoid any of these examples and we will work to make sure that is the case.
"But I certainly can't say to you in all honesty today that I don't think there's a possibility there could be disruption to either medical supplies or vaccine supplies as a consequence of Brexit - but we will do everything in our power to avoid that happening."
On Thursday night, Mr Hancock said the UK could fly in vaccine supplies to avoid any issues around border checks.
The Cabinet minister told BBC Question Time: "We have a plan for the vaccine, which is being manufactured in Belgium, and if necessary we can fly in order to avoid those problems.”