Police discovered 330 kg - around 730lbs - of the drug in a truck in the western city of Essen and arrested two men, public prosecutor Anette Milk said on Thursday.
The haul was more than the entire amount of heroin seized in Germany during 2013, which totalled around 270 kilograms.
Local media speculated that the haul could have provided 1.3 million doses.
Two brothers aged 30 and 35, and a Dutch national living in Belgium were arrested in connection with the discovery.
Broadcaster Deutsche Welle reported the siblings as being members of a Syrian-Iraqi gang that police detectives believe are responsible for the smuggling attempt.
The truck came from Iran, but authorities have not confirmed if the heroin originated from the Middle Eastern country.
Iran has a border with Afghanistan, the world’s largest producer of opium poppies, and has experienced high rates of heroin usage among its people in recent years.