IS claimed responsibility for the attack in a statement posted online on a militant website commonly used by the extremists.
Haider al-Abadi sacked the commander of Baghdad Operations, a statement from his office said, as the embattled prime minister faced growing protests at the site of a large-scale bombing, where at least 186 people were killed earlier this week. The statement said other security and intelligence officials were also sacked, but did not specify who.
The Thursday night attack began with a volley of mortar fire on Sayyid Mohammed shrine and a nearby market in Balad, 50 miles (80km) north of the Iraqi capital.
A suicide bomber first targeted policemen guarding the shrine’s entrance, allowing a second bomber to push into the courtyard with nine gunmen, who targeted security forces and civilians who had gathered inside to celebrate Eid al-Fitr, the holiday marking the end of the holy month of Ramadan.
A third bomber was killed before he detonated his explosives, police officials said. According to hospital officials 37 were killed in the attack and 62 wounded.
In Baghdad, protests were growing at the scene of last weekend’s car bomb attack, which was one of the deadliest since the 2003 US-led invasion of Iraq. Many Iraqis blame their political leadership for security lapses that allow such large-scale bombings to happen in territory far from the front-line fighting against IS.
Small-scale bombings occur on a near-daily basis in Baghdad, and in May a string of larger attacks, many of them claimed by IS, killed more than 200 people in a week.
After the Baghdad attack, Mr Abadi announced new security measures, but it was unclear if any of the measures have yet been implemented.
The minister of interior submitted his resignation on Tuesday, but Mr Abadi has not accepted it.
The Islamic State group was pushed out of Fallujah last month after holding the city just west of Baghdad for more than two years.
Despite a string of territorial defeats, IS still holds pockets of territory in northern and western Iraq, including the country’s second largest city of Mosul.