The SNP leader is due to make a statement to MSPs just before First Ministers Questions at Holyrood today.
But Mr Carlaw told Sky News: "The whole Parliament will be in shock.
"I think we are all bewildered that Derek could have undertaken such a colossal lapse of judgement.
"The First Minister is going to be making a statement before First Minister's Questions this morning.
"I don't think she believes that simply accepting his resignation from Government is an end to the matter.
"I think there will be huge question to be asked, parents who will be concerned, and I think many colleagues in parliament who will wonder if Mr Mackay can continue to enjoy the confidence of his constituents or Parliament.
"It's a huge blow to politics in Scotland."
Mr Carlaw compared the resignation with the case of Mark MacDonald who was forced to quit as children's minister a few years ago after sending inappropriate text messages.
"The First Minister then immediately suspended the individual from the Scottish National Party. He now sits as an independent member.
"I think having set that bar if the First Minister thinks, however illustrious Mr Mackay was to the Scottish Government, that there can be different standards set now, and that she can simply accept his resignation from Government.
"I don't think that that will be regarded by many as appropriate and effective action. Parliament has a responsibility to set the tone and protect the reputation of politics in Scotland."