The Deputy First Minister defended officials against claims they undermined the inquiry, saying they acted “legitimately and appropriately” within the law.
Helen Holland, a spokeswoman for survivors group In Care Survivors, said she currently has no faith in the inquiry and called for assurances of its “absolute independence”.
The inquiry’s chair Susan O’Brien QC resigned on Monday after formal proceedings were launched to remove her following claims she made comments that were ‘’offensive’’ to survivors.
Panel member Professor Michael Lamb has also stepped down, saying the review is ‘’doomed’’ due to interference by ministers.
In her resignation letter, Ms O’Brien’s lawyer said the inquiry’s work ‘’will have no value’’ if its independence cannot be guaranteed.
Mr Swinney said Scottish Government officials have a duty under the Inquiries Act 2005 in relation to the use of public finances.
He told BBC Radio Scotland’s Good Morning Scotland programme: “That is the area where Scottish Government officials have, in my view, legitimately and appropriately been taking forward their responsibilities.”
Mr Swinney said a “protracted discussion” had taken place earlier in the year about whether it was necessary for members of counsel to take all the statements from survivors at a cost of about £100 an hour.
He said: “There are other ways in which inquiries have taken evidence in a professional fashion that have cost significantly less than that.
“The Government has an obligation under the Inquiries Act to make sure public money is being used effectively and wisely.”
Mr Swinney added: “We have got an obligation to ensure that we manage the cost. There are countless inquiries that have cost enormous amounts of money and have caused public concern.
“I’m satisfied that Scottish Government officials have exercised their responsibilities consistent with the Inquiries Act and I would expect nothing else of them.”
He moved to reassure survivors, insisting the inquiry will be “free to take all of the evidence to look into all of the issues it needs to look into without fear or favour”.
He said: “I can well understand the concerns of survivor groups and I welcome the opportunity I will have on Thursday to meet with survivor groups.
“I go into that discussion with the intention of building confidence around the inquiry and building confidence around the personal relationship that needs to exist between me and the survivors groups, as the minister responsible for this, to make sure that they can have full confidence in the thoroughness of the process.”
Speaking to the same programme, Ms Holland said: “I think it’s been awful, and the people who are most affected by this are the survivors themselves - the very people who should be protected.
“This whole event has been traumatising for survivors. Last week I spent just two nights constantly on the phone to survivors, phoning with concerns that they had about the whole thing.”
On accusations of interference, she added: “We need answers. We need to know exactly what happened, exactly what the interference is and at what level that interference has been there.
“At every step of the way, everybody who has been involved with this since 2007 has said there needs to be absolute independence from Government.
“I would be lying if I sat here and said I had faith in the inquiry. As it stands at the moment, no. But it can be rectified if it’s dealt with properly.”