Oliver bundled the ball home with only 11 minutes left after team-mates Ross Forbes and Jamie McDonagh had hit Hibs cross bar in the space of a few seconds but Cummings, who had replaced defender Liam Fontaine only a few minutes earlier curled home a fantastic free-kick to win his side a hard-earned draw.
Kris Commons was pitched straight into action - his first appearance since the end of April - after signing a 28 day emergency loan for the Easter Road club, the Celtic midfielder handed the No 15 shirt as Jason Cummings dropped to the bench once again.
But it was Morton, unbeaten at home since March 28, who took the game to Hibs, Oliver passing up a glorious chance to put his side ahead as he swept McDoinagh's low cross over the bar when he should at least hit the target.
Neil Lennon's side were struggling to match Morton's energy which gave the Hibs players little time on the ball, the upshot being a lack of composure and inability to retain possession to any great extent.
Hibs goalkeeper Ross Laidlaw did well to get down to a powerfully struck low shot from Mark Tidser before a bit of confusion in the home defence saw goalkeeper Derek Gaston left stranded as the ball fell to Brian Graham. He had the chance to loft it back over Gaston and into the empty net but screwed his effort horribly wide.
Commons had enjoyed a somewhat quiet debut but he did send one glorious pass out left for Lewis Stevenson to take on his chest as he cut inside before, on his right foot, he snet his shot across goal and wide of the far post.
McDonagh almost enjoyed a huge stroke of luck when, having driven at the Hibs defence the ball broke off Liam Fontaine before crashing back off the Morton player before looping over Laidlaw's crossbar.
Hibs looked to get Commons on the ball more at the start of the second half, the former Scotland star spraying a couple of searching passes forward but it was Morton who enjoyed the game's best chance, Oliver nodding Ricki Lamie's cross down for McDonagh who's snapt shot brought out a smart save from Laidlaw as he got down at his left hand post.
Commons was certainly exerting a greater influence and an ambitious chip from the edge of the penalty area just cleared Gaston's crossbar and Hibs went closer when after Andrew Shinnie's initial effort had been blocked skipper David Gray latched onto the loose ball only to see Thomas O'Ware deflect his shot wide.
But at the other end it took a terrific last-ditch challenge from Darren McGregor to halt the run of McDonagh after he'd sped through the centre of Hibs defence. The game was swinging from end to end, though, and O'Ware was again on hand to get his body in the way as Grant Holt got on the end of Martin Boyle's low cross.
Morton. though, took a deserved lead 11 minutes from time in astonishing fashion, Forbes dipping effort came crashing back off the bar for McDonagh who saw his shot do exactly the same before Oliver managed to bundle it over the line.
The home side's lead lasted barely two minutes, Cummings stepping up to curl a fantastic free-kick from 25 yards into the bottom left hand corner of Gaston's net.It was the 21-year-old's ninth goal of the season but his first in exactly three months,
Morton: Gaston, McDonagh, O'Ware, Gunning, Lamie, Forbes, Tidser, Murdoch, Russell, Oyenuga (Kilday 16), Oliver.
Substitutes not used: McNeil, Doyle, Nesbitt, Scullion, Strapp.
Hibs: Laidlaw, McGregor, Hanlon, Fontaine (Cummings 76), Gray, Bartley, Shinnie, Stevenson, Commons, Boyle, Graham (Holt 65).
Substitutes not used: Virtanen, Keatings, Forster, Eardley, Crane.
Referee: Steven McLean.