"All foreign militants should leave, otherwise they would be crushed," Pervez Musharraf told a rally of some 80,000 people in Lahore.
His comments came amid tension between Pakistan and Afghanistan over claims the former is not doing enough to stop Taleban and al-Qaeda forces launching cross-border attacks from within Pakistan.
They also came weeks after Pakistani security forces backed by helicopter gunships attacked a suspected al-Qaeda hideout in a remote tribal region of northwestern Pakistan bordering Afghanistan and killed dozens of suspects, most foreigners.
The military said the dead included a Chechen commander linked to al-Qaeda. Since then, authorities have been urging tribal elders to evict foreigners.
Although Mr Musharraf did not elaborate on the militants or their number, Pakistani security officials have said hundreds of foreigners, including Arabs, Uzbeks, Chechen and Afghans are hiding in the north and south Waziristan tribal regions.
Mr Musharraf said foreign militants were also involved in acts of terrorism within Pakistan. "I will not tolerate the presence of these terrorists in Pakistan," he told the rally.
Pakistan, an ally of the United States in its actions to combat terrorism, stopped backing Afghanistan's former Taliban regime after the 11 September, 2001, attacks. Since then, security forces have arrested more than 750 al-Qaeda suspects. Most have been turned over to the US.
The Lahore rally marked the 100th anniversary of the ruling Pakistan Muslim League.