The former Hoops player and manager was given a warm reception from the home fans before McGregor, left out of Gordon Strachan’s recent Scotland squad to some surprise, drove in the opener in the 15th minute.
Two stunning strikes by McGinn after the break put the Easter Road ahead before McGregor levelled with 10 minutes remaining to extend Celtic’s unbeaten domestic run to 58 games.
The Hoops’ convincing 3-0 Champions League win over Anderlecht on Wednesday night ensured a packed stadium for their return and they were treated to another 90 minutes of excitement.
Celtic boss Brendan Rodgers, without injured captain Scott Brown and fellow midfielder Stuart Armstrong, made five changes.
French attacker Odsonne Edouard started alongside fellow countryman Moussa Dembele with McGregor, James Forrest and Jonny Hayes back in the side.
Anthony Stokes, Efe Ambrose and Dylan McGeouch were ex-Celts in the Hibs side, adding another sub-plot to the encounter
Hoops keeper Craig Gordon tipped a 25-yard free-kick from Stokes over the bar in the third minute as Hibs started confidently.
However, gradually the champions played themselves into the game.
When Dembele played in McGregor he fired low past Hibs keeper Ross Laidlaw from 16 yards for his fifth goal of the season.
That appeared to be Celtic into their stride and moments later Laidlaw pulled out a decent save from midfielder Tom Rogic’s drive.
Hibs, though, were not out of it and in the 34th minute Celtic defender Dedryck Boyata had to speed back to deny Martin Boyle who had raced on to a Lewis Stevenson pass.
At the other end, Laidlaw blocked an Edouard effort after he was set up by Hayes in a swift counter-attack.
Scott Sinclair replaced Edouard for the start of the second half and within three minutes had drawn a good save from Laidlaw, his drive from 12 yards parried and then cleared by Stevenson.
McGinn’s equaliser in the 53rd minute stunned Celtic Park.
As the home side set themselves up to defend their penalty area midfielder Marvin Bartley drifted along the edge of the box and when the ball fell neatly for the former St Mirren player who fired low past Gordon.
The match turned in Hibs favour and after Leigh Griffiths replaced Dembele and only a remarkable Gordon save in the 65th minute kept the home side on level terms.
When a McGinn corner fell to Steven Whittaker a yard out it looked a certain goal but somehow the Scotland keeper beat away his shot for another corner.
McGregor cleared the bar moment later as an exciting finale ensued and when McGinn fired high past Gordon from 22 yards after Mikael Lustig had headed clear an Ambrose there was an air of disbelief around the stadium.
Celtic rallied and after Griffiths’ cross was knocked back into the path of McGregor after several players challenged for it, he steered the ball in from 12 yards.
There was a Celtic penalty claim in the 87th minute when Ambrose appeared to impede Sinclair before the Hoops substitute had his shot parried away by Laidlaw.
Referee Willie Collum ignored the appeals to the anger of the home supporters, who, when the dust settled, will recognise that they had witnessed their side given a rare domestic test.