Fettes College in Edinburgh have taken the action as the Scottish Child Abuse Inquiry looks into allegations against Anthony Chenevix-Trench following allegations he would beat pupils for his own sexual gratification.
The school, whose alumni include former Prime Minister Tony Blair, has also carried out its own internal investigation over the claims.
Chenevix-Trench, was appointed to his post at Fettes in 1971 and remained until his death in 1979, aged 60.
David Blackie, 71, who claims to have endured brutal sadism from Chenevix-Trench, campaigned for the school to remove the portrait.
A statement released by Fettes’ governors said: “We have no information to suggest that any crime was committed by Mr Chenevix-Trench while at Fettes.
“However, as we have reported to the Scottish Child Abuse Inquiry, his specific judgment in handling two specific matters did not measure up to the excellent pastoral care we pride ourselves on at Fettes today.”