It is understood the man, who has not yet been named, required urgent medical treatment before kick-off as supporters entered Tynecastle.
Paramedics rushed to the turnstiles in front of the Wheatfield Stand to perform CPR treatment, but the man could not be revived and passed away shortly after.
Fans have since posted heartfelt messages of condolence online describing the deceased as an ‘absolute legend’, a ‘very upstanding man’ and a ‘real family man’.
On the Jambos Kickback twitter page, his family wrote: “The Jambo in question was our father who saw his first Hearts game at the age [of] five in 1939.
“Despite a valiant struggle he lost his fight for life just after 9pm last night.
“My brothers and I would like to pass on our heartfelt thanks to all the paramedics, St Andrew’s ambulance staff, stewards and members of the public who came to his assistance and tried desperately to revive him.
“Many thanks also to the Hearts family for all your messages of condolence and support.”
A Police Scotland spokesperson said: “During the Scottish Cup match between Heart of Midlothian and Partick Thistle on Tuesday, March 12, an 84-year-old man took unwell and was taken to the Royal Infirmary of Edinburgh.
“He sadly passed away a short time later and our deepest sympathies are with his family.
“There are no suspicious circumstances surrounding this death.”
Sign up to our newsletter: enter your email in the box at the top of this article to get daily updates straight to your inbox.