It looked like the Gorgie side had done the hard bit in the first leg of their Europa League second qualifying round by leaving Malta last week with a goalless draw.
However, in a scrappy first 45 minutes, attacker Jamie Walker hit the bar before midfielder Prince Buaben had a weak penalty saved by visiting keeper Miroslav Kopric.
In the 55th minute defender Christian Bubalovic stunned the home support when he drove the visitors ahead before Edward Herrera grabbed a second 12 minutes later to send their 56 travelling supporters wild with delight.
Striker Conor Sammon reduced the deficit in the 74th minute but it turned out to be no more than a consolation with some Jambos fans booing their side off at the final whistle.
The warning signs were there early for Hearts. The Maltese side started with some confidence and Gorgie keeper Jack Hamilton had to parry a whipped-in cross from Vito Plut before instantly making a point-blank save from Shaun Bajada.
It became clear that a robust Birkirkara side were not going to be pushovers but in the 27th minute the hosts came close to getting the breakthrough.
Callum Paterson, playing in midfield, got the break of the ball on the right-hand side and his floated cross was met by Walker whose acrobatic shot beat Kopric but crashed off the bar before it was cleared to safety.
Another golden opportunity was soon presented to Hearts. A mazy run into the box by Sam Nicholson in the 34th minute ended when he was tripped by Cain Attard but Buaben’s penalty was too close to the keeper who dived to his left to save.
Gorgie defender Faycal Rherras guided a shot past the far post moments later before a Walker effort from wide on the left, after he had robbed Kopric who had strayed out his area, was cleared away from the line by the head of defender Emerson Marcelina.
Hearts started the second half better but in the 54th minute, after the Gorgie defence had failed to clear a free-kick from Dimitrov Srdan, the ball eventually fell to Bubalovic and he hammered a shot high past Hamilton.
The Tynecastle side shook themselves and rallied. Moments later, Nicholson sent a drive from the edge of the box against the crossbar before Arnaud Djoum blazed a shot high over the bar.
However, in the 67th minute, it was all but over for the Edinburgh side when Herrera raced on to a pass from Plut to steer the ball through the legs of Hamilton and the home supporters vented their frustration.
Sammon gave Hearts a lifeline when he headed in a Djoum cross for his first goal for the club and, although they piled forward in the closing stages, there was no way back.