Tries from Callum Gibbins, Nikola Matawalu and Kyle Steyn - along with 12 points from the boot of Adam Hastings - was enough to secure victory for Dave Rennie's side.
La Rochelle scored two tries of their own courtesy of Dany Priso and Zeno Kieft with Jules Plisson kicking 14 points.
Lively right-wing Vincent Rattez sparked the hosts into life with a 35-metre break after bouncing off a weak tackle from Fraser Brown.
Rattez approached the 22 before offloading to Priso, who touched down underneath the crossbar with Plisson converting.
La Rochelle began to apply pressure in the Glasgow half and they extended their lead with a further two successful penalties from Plisson.
Hastings clawed one penalty back for Glasgow after the hosts were penalised at the proceeding scrum.
La Rochelle blindside flanker Kieft claimed their second try after Glasgow slapped down a lineout just five metres from their own try line.
The ball appeared to be heading into the hands of DTH Van Der Merwe but it was fortuitously gathered by Kieft for the try.
Glasgow pulled themselves back into the game on the stroke of half-time. With Priso penalised for not rolling away the visitors turned down an almost certain three points to go for the corner.
The driving lineout was well worked allowing Gibbins to power over from short range with Hastings converting meaning Glasgow trailed 18-13 at the interval.
Plisson extended La Rochelle's lead with a straightforward penalty before Matawalu intercepted a pass from Victor Vito to run in unopposed from 70 metres out.
The conversion was missed but La Rochelle proceeded to build some pressure, allowing Plisson to extend their lead to four points.
But the visitors hit straight back and had a try disallowed when second-row Scott Cummings appeared to score. However, upon referral to the television match official, the try was ruled out because there was a hand holding the ball up.
Steyn put Glasgow ahead as he ran in from 45 metres out after gathering a perfectly executed cross-kick from Hastings, who converted to put the visitors 27-24 ahead with 15 minutes remaining.
La Rochelle threw everything at Glasgow in the closing stages of the game and very nearly went back ahead when Rattez burst clear only to be denied a try by a last gasp tackle from Ali Price.
The hosts laid siege to Glasgow's line in the closing stages of the game but the Scottish outfit managed to hold on for a crucial victory which keeps alive their European hopes.