Miller scored in each half and passed to Josh Windass for the third goal after former Rangers striker Kyle Lafferty had opened the scoring with a spectacular free-kick.
Caretaker boss Graeme Murty ended Miler’s unexplained exile from the first-team squad, which Caixinha had maintained since reports of a heated team meeting after defeat by Celtic on September 23.
Caixiha’s decision-making often looked questionable during his seven-month spell in charge and Miller even made him look foolish after his dismissal on Thursday with a performance that vindicated Murty’s decision to hand him the captain’s armband.
Miller was in for the suspended Graham Dorrans while Ryan Jack kept his place after the club appealed against his midweek red card in the 1-1 home draw against Kilmarnock which sealed Caixinha’s fate.
Danny Wilson partnered Ross McCrorie in place of the suspended Bruno Alves while Carlos Pena was a notable absentee despite returning to training after missing the Killie game through injury.
Hearts were without four injured midfielders and Connor Randall dropped out as teenagers Lewis Moore and Harry Cochrane came in. And they lost another early in the second half when Jamie Walker went off with a hamstring injury.
Rangers made slightly the better start in an uninspiring opening spell. Declan John got the first shot on target and Miller over-hit a pass which might have played Alfredo Morelos in.
Lafferty lit up the game in the 24th minute after McCrorie was penalised for pulling the shorts of Esmael Goncalves’ more than 25 yards out. The Northern Ireland international curled the ball over the wall and into the top corner before a subdued celebration.
Ross Callachan was clean through moments later but Wes Foderingham got down well to save and soon stopped Rafal Grzelak reaching John Souttar’s free-kick.
Hearts goalkeeper Jack Hamilton handled a through ball a yard outside his box but referee Craig Thomson played on after looking toward his assistant, Douglas Ross, the Tory MP who has quit midweek football duties to focus on his main job.
Rangers levelled in the 42nd minute. Morelos turned Christophe Berra and played a reverse pass through to Miller, whose shot spun off Souttar and over Hamilton.
Foderingham tipped over Moore’s long-range strike moments later but Hearts looked a spent force after Walker pulled up five minutes into the second half.
Miller headed homed James Tavernier’s cross in the 65th minute and then played a brilliant long-range pass seven minutes later to Windass, who turned Grzelak and shot inside Hamilton’s near post.
Hearts had brought on a defender, Krystian Nowak, and striker, Cole Stockton, for their two teenage midfielders and they could not get back into the game.
Hamilton saved from Morelos and Miller was substituted to a rapturous ovation from the 14,000 Rangers fans in the 32,852 crowd before embracing Murty.