The man has been identified as 44-year-old Malik Faisal Akram, orginally from Blackburn in Lancashire.
He was killed in a "shooting incident" after the FBI entered the building at Congregation Beth Israel in Colleyville, Texas, on Saturday.
All four hostages were unharmed in the incident which President Joe Biden described as "an act of terror".
During the incident - which saw police first called at 11am and ended with the release of the final hostages at around 9pm - the man could be heard ranting on a livestream in what appeared to be a British accent.
Yesterday, the Foreign Office confirmed he was British and the Metropolitan Police said counter-terror officers were in contact with US authorities and colleagues from the FBI.
Mr Biden said, in an update to the press, that he did not have all the details but it was believed Akram had "got the weapons on the street", adding: "He purchased them when he landed."
He said there were "no bombs that we know of", and that Akram is thought to have "spent the first night in a homeless shelter".
Akram's family said they are "absolutely devastated" and "do not condone any of his actions", according to a statement shared on the Blackburn Muslim Community Facebook page.
It stated: "We can't say much now as their is an ongoing FBI investigation. We would like to say that we as a family do not condone any of his actions and would like to sincerely apologize wholeheartedly to all the victims involved in the unfortunate incident."
The statement, attributed to Akram's brother Gulbar, said the hostage-taker "was suffering from mental health issues".
It added: "We would also like to add that any attack on any human being be it a Jew, Christian or Muslim etc is wrong and should always be condemned.
"It is absolutely inexcusable for a Muslim to attack a Jew or for any Jew to attack a Muslim, Christian, Hindu vice versa."
Foreign Secretary Liz Truss condemned the "act of terrorism and anti-semitism", while the British Ambassador to the United States Karen Pierce said British authorities are providing "full support to Texas and US law enforcement agencies".
Akram is said to have demanded the release of a Pakistani neuroscientist who was convicted of trying to kill US army officers in Afghanistan.
Speaking to reporters after the incident, FBI special agent in charge Matt DeSarno said they believed the man was "singularly focused on one issue and it was not specifically related to the Jewish community", and added they will continue to "work to find motive".
Confirming that the hostage-taker had died, he said there would be "an independent investigation of the shooting incident".
He said the FBI had been in contact with their legal attache offices in London and Israel for an investigation with "global reach".