Lawrence Shankland put the visitors ahead in Riga from the penalty spot in the first half before Alan Forrest secured the win and three points late on.
Craig Gordon had to play his part with a few excellent stops but the victory leaves Hearts second in the group going into the third match with Fiorentina and Tynecastle Park.
As well as earning Scotland some crucial coefficient points, a win in the Conference League group stage earns the club €500,000.
Next up for Hearts is a meeting with Motherwell at Fir Park in the Premiership. Centre-back Craig Halkett will miss it after picking up another hamstring injury in the first half of the win over RFS having just come back into the team.
He was replaced by Lewis Neilson with the teenager putting in an assured display.
“It’s always a risk but hopefully it is not as bad as the last time,” Robbie Neilson said on Halkett’s injury. “He felt a wee twinge in the same spot and felt it was better to come.
“He won’t make Sunday but then we have a couple of weeks and I am hoping he’ll be back after that.
“It keeps just flagging up a wee bit, which is disappointing. We’ll see.
“Lewis Neilson was outstanding, for a 19-year-old kid to come in and play Conference League like that against a very good team. Physically he did well and on the ball he was outstanding as well. In time he will be a top player.”