Scrum-half John Cooney landed a winning penalty in the final act of a dramatic match at Kingspan Stadium as Edinburgh blew a 17-point lead in the second half.
Edinburgh looked to have rescued the win when Simon Hickey edged them back in front 30 seconds from time with his seventh successful kick of the night. But Richard Cockerill’s men failed to win the restart and were pinged as they tried to hold Ulster at bay and scrum-half Cooney stepped forward and held his nerve to kick the winner from 38 metres on the right.
It was tough luck on an Edinburgh side who were anxious to bounce back from their 17-13 loss away to Ospreys on the opening weekend and they were good value for their 16-6 interval lead.
Ulster, who snatched victory at the death against Scarlets on the opening weekend, were on the back foot from the early stages as Edinburgh half-backs Henry Pyrgos and Hickey probed at every opportunity.
Ulster were forced to cough up penalties and Hickey began punishing them, landing the opening score with a fine kick from the ten-metre line after just four minutes.
He doubled it shortly afterwards from 35 metres and with Edinburgh minimising the impact of Ulster centre Stuart McCloskey, their standout player against Scarlets, the visitors continued to dominate.
Viliame Mata and Luke Hamilton were busy throughout in a dominant back row, ensuring Edinburgh got plenty of go-forward ball and they used it well.
A superb looping pass from Pyrgos put Tom Brown in for the opening try in the left corner, with Hickey landing the conversion to make it 13-0 after just 15 minutes.
Edinburgh lost centre Mark Bennett to a hamstring injury at the end of the opening quarter but they maintained their composure and dealt capably with the Ulster challenge.
Ulster went to the corner with a penalty but couldn’t find a way through an excellent defence and in the end Cooney opted for the posts and reduced the margin with a kick after 20 minutes.
But Edinburgh forced a penalty from the restart after Ulster No 8 Nick Timoney was nailed on his own 22 as Edinburgh piled in and Hickey steered the ball between the posts from the right wing to make it 16-3.
Cooney hit back with a penalty in front of the posts 13 minutes from the break as Ulster enjoyed a good spell.
But Edinburgh ended the opening half deep inside the Ulster half seeking to increase their lead.
Edinburgh lost Mata to an arm injury at half-time but Hamilton continued to impress in his second game since joining from Leicester Tigers as Ulster came looking for an early score after the break. But it was Edinburgh who got the score. Pyrgos forced a turnover inside his 22 and on the break Blair Kinghorn made the most ground before feeding James Johnstone who beat Henry Speight to score a converted try and make it 23-6.
Ulster hit back and got their first try of the campaign when full-back Will Addison got over on the right after good pressure for a converted try, but almost immediately Hickey landed a penalty to put Edinburgh ahead by 26-13 after 57 minutes.
Addison turned provider to set Cooney up for a converted try after 61 minutes, cutting the gap to 26-20 and ensuring a nervous finish for the Scottish side. And Ulster made them pay with the try of the night when Speight and Cooney combined from deep inside their own half and Craig Gilroy sprinted down the right and scored, with Cooney landing the conversion to edge them in front for the first time with nine minutes of the match remaining.
That set the scene for the dramatic finish as Cooney broke Edinburgh’s hearts with his last-gasp winner.