Craig Halkett, Craig Sibbald and Scott Pittman were on the scoresheet after Saints captain Joe Shaughnessy had opened the scoring in the 14th minute.
The victory moved the West Lothian side 19 points clear of the Ladbrokes Premiership relegation zone and to within five of the top six.
Holt was thrilled with his team’s response to going behind.
He said: “It’s a nice present and I asked them for it before the game. I asked them for three points. We pride ourselves on clean sheets and not giving goals away, but I said I’d rather take three points today.
“It’s always more satisfying when you have to come from behind to do it. It shows character, resilience and everything we’re about.”
Asked if his side could consider themselves safe with nine games left, Holt said: “There are still 27 points to play for. We don’t want to focus on being safe already and the season peters out because a malaise sets in.”
The hosts were the first to threaten inside three minutes when Steven Lawson’s deflected drive from 18 yards came back off a post.
Livingston kept up the pressure and stand-in striker Scott Robinson had a shot blocked inside the area by Shaughnessy before referee John Beaton waved away appeals for a penalty after Pittman fell inside the area following a challenge from the Saints captain.
The visitors gradually settled into the match, with Chris Kane getting off a tame shot that was comfortably collected by Liam Kelly.
There was better to come for Saints in the 14th minute as Shaughnessy gave Tommy Wright’s side the lead. The ball dropped to the defender from Sean Goss’ corner and the skipper slammed the ball into net from six yards.
The woodwork then spared Alan Lithgow’s blushes in the 28th minute as the Livingston defender came close to scoring an own goal. Lithgow flicked a header from Goss’ free-kick past his own goalkeeper, but the ball came back off an upright and was eventually cleared.
Lithgow turned provider as the hosts restored parity. The defender swung in a cross from the left and Halkett nodded into the gaping net from close range.
The home side then took the lead just seconds after the restart when Sibbald headed in at the back post from a Robinson cross.
Saints substitute Tony Watt wasted a great opportunity to level in the 89th minute when he fired Richard Foster’s low cross over the bar at the back post with the goal at his mercy.
Saints were made to rue that miss after Pittman drove a third high into the net after connecting with Lawson’s cross at the front post.
Saints manager Wright, whose side have not won in nine matches and are four points adrift of the top six, said: “Tony’s chance is a big one. He’s obviously gutted and he’s apologised.
“You don’t miss them deliberately, but it’s the best chance of the game.”