Mr Blackford and fellow SNP MP Stephen Gethins were leaving the Cabinet Office shortly after 4pm following a meeting to discuss Brexit with minister David Lidington.
Media interviews that were scheduled to take place outside the Cabinet Office had to be abandoned. The pair were escorted back to parliament by a police officer.
Mr Gethins said: “Ian took it with characteristic good humour, but no one should have to face that kind of abuse.”
There has been growing concern about the safety of MPs and their staff around Westminster, with parliamentarians this week being advised to take taxis to and from work.