A drab fourth-round clash burst in to life 16 minutes from time when Louis Moult gave Mark McGhee’s team a surprise lead in Govan.
But with the clock ticking down, veteran Miller - who turned 37 last month - struck twice inside five minutes to book Gers’ place in Sunday’s fifth-round draw.
Rangers boss Mark Warburton gave recent loan signing from Arsenal Jon Toral his first start but left his other new recruit, Bournemouth’s Emerson Hyndman, on the bench.
McGhee, meanwhile, admitted this week he was interested in taking out-of-favour Gers wideman Michael O’Halloran on loan but the Well boss arrived at Ibrox to find the forward had been given a rare start for the hosts.
Rangers pulled and probed at their opponents for much of the first half an hour without causing too much concern.
An early strike from Lee Wallace was deflected wide while Miller saw a header clip the top of the crossbar.
But the Light Blues should have had a penalty seven minutes before the break when O’Halloran’s driving run into the box was halted by Stevie Hammell’s clumsy challenge. Referee Craig Thomson, though, was unconvinced by Gers’ appeals.
The home support were starting to voice their frustration at the lack of cutting edge being shown by their team.
And there were more groans when Miller kicked at fresh air after Barrie McKay sent the ball into the danger area.
Toral looked tidy but never really gave his new side the attacking impetus they required before his debut came to an end just after the hour mark when he was replaced by Josh Windass.
The former Accrington playmaker did his best to provide a spark but went after a McKay cross with his foot when he really should have thrown his head at it. He did better as he charged into the box moments later but could only fire wide.
Hyndman’s introduction after 72 minutes for Jason Holt would prove vital but only after the visitors struck first.
The Rangers defence looked well positioned as Chris Cadden sent a cross into the box but Rob Kiernan’s failure to track Moult left the striker free to climb above Lee Hodson and power home with his head.
It appeared Gers’ last hope of claiming a trophy this campaign were about to be snuffed out but Miller proved himself the hero with six minutes remaining as he launched himself at substitute Martyn Waghorn’s cross, sending his header past the stranded Craig Samson.
Motherwell could have snuck back in front were it not for an impressive double save from Wes Foderingham to deny Carl McHugh and Scott McDonald.
But the drama was not over there as Miller completed a remarkable comeback in the 89th minute when, after Stephen McManus gave the ball away, he raced on to Hyndman’s perfect through pass and slotted in the winner.