A statement from the core workforce said they believe the firm’s new owners want to drop full-time staff in favour of agency workers.
Scottish Labour demanded swift action from Finance Secretary Derek Mackay to ensure that skills are not lost “for good”.
The Scottish Government said it is confident everything is being done to secure new contracts and restore employment to past levels.
And the firm’s owners insisted the “high quality skills of Scotland’s workforce have a critical part to play in our development”.
The comments come a week after a union claimed that further redundancies at BiFab have led to the effective closure of the yards.
GMB Scotland said from a core workforce of 400 and 1,000 contractors, only seven staff have been offered part-time positions for maintenance and security of the sites at Arnish on Lewis and Burntisland and Methil in Fife.
In addition, 12 senior managers will be retained by new owner Canadian company JV Driver, which acquired the firm through its subsidiary DF Barnes in April in a Scottish Government-backed rescue package.
In a statement, the Bifab core workforce said unions believe “it has been the intention of Barnes / Driver all along to sack the full-time union core workforce and then start back non-union agency personnel”.
It continued: “We believe that this will seriously damage any hope of winning contracts for the yards.”
It added that a lack of full-time employees would mean that “government money has been spent creating temporary working instead of full-time working”.
Labour’s economy spokeswoman Jackie Baillie said: “Laying off the highly-skilled core workforce would devastate families and communities - and lead to a loss of skills and experience that would be impossible to replace.
“The SNP government simply cannot be in a position where it has allowed itself to be a facilitator for making skilled workers redundant and replaced with insecure agency work.
“That is an outrage and SNP Finance Secretary Derek Mackay must immediately address these reports and take swift action to ensure these skills are not lost for good.”
The Scottish Government said it is in regular contact with BiFab, including at a meeting this week.
A spokeswoman said: “It was clear at the point new ownership was secured that conditions would remain challenging for the yards and new contracts would have to be won to secure future work.
“The workforce played a major role in securing new investment for the yards and the prospect of securing future contracts which DF Barnes is working toward.
“We have confidence that everything possible is being done to secure new contracts and restore employment to previous levels.”
DF Barnes said it has made a long term commitment to supporting all three yards.
“Working closely with the Scottish Government, our absolute focus is on securing new contracts that will strengthen communities in Fife and on the Isle of Lewis. We remain confident of success,” a spokesman said.
“We informed the trade unions in Scotland some months ago that, while there was a gap between projects, we would need to reduce the workforce ...
“We believe that the high quality skills of Scotland’s workforce have a critical part to play in our development, but we need to maintain a laser sharp focus on attracting new orders in order to secure everyone’s future.”