Former Liverpool boss Rodgers, who replaced Ronny Deila in May, saw his side fall behind to two goals from Celje captain Danijel Miskic either side of the break before Nadir Ciftci and Tom Rogic hauled Celtic level in the second half.
Celtic lost the first goal in the 28th minute when midfielder Miskic fired home with a first-time shot from inside the box following a low corner.
Leigh Griffiths curled a shot just off target soon after as Celtic looked to get back on level terms and the visitors had a penalty appeal waved away five minutes before the break when winger James Forrest was sent tumbling in the box.
Rodgers made wholesale changes during the break before Celtic fell further behind in the 50th minute, with Miskic curling home a free-kick via a slight deflection.
Celtic pulled one back six minutes later through Ciftci’s header following Stuart Armstrong’s cross and were rewarded for concerted pressure in the 75th minute when Rogic chipped home a fine finish.
It was Ciftci’s first appearance for Celtic since leaving the club in February to go on loan with the Turkish side Eskişehirspor.