Reports said troops had entered one building and were planning to push cautiously through the rest of the huge compound amid fears of explosives.
IS militants are believed to have fled to the north-east of the city.
The government has been trying to retake Ramadi for weeks.
The mainly Sunni Arab city, about 55 miles west of Baghdad, fell to IS in May, and was seen as an embarrassing defeat for the army.
In recent days, troops have been picking their way through booby-trapped streets and buildings as they pushed towards the city centre, seizing several districts on the way.
They were reported to be within a few hundred yards of the former provincial administrative headquarters on Saturday.
When sniper fire from the compound stopped, and aerial surveillance detected no human activity, a group of Iraqi soldiers moved in.
They reportedly entered what used to be the city’s department of health, housing a blood bank.
The Iraqi military believes the militants have headed north-east; fighting meanwhile is reported to be under way to the south-west of the compound.
The operation to recapture Ramadi began in early November, but has made slow progress, mainly because the government chose not to use the powerful Shia-dominated paramilitary force that helped it regain the northern city of Tikrit, to avoid increasing sectarian tensions.
Meanwhile on Saturday, the Army of Islam and allied militant groups in Syria mourned the killing of Zahran Alloush, a top Syrian rebel commander who led one of the most powerful groups battling president Bashar al-Assad’s forces, while government supporters and the Islamic State group cheered his death – a reflection of his role in fighting both sides in the Syrian civil war.
Alloush was killed in airstrikes that targeted the group’s headquarters during a meeting on Friday. He was instantly killed along with a number of senior commanders of his Army of Islam group and those of the ultraconservative Ahrar al-Sham and the Faylaq al-Rahman groups.
The Syrian army claimed responsibility for the airstrike that killed Alloush.