The blaze, involving railway arches - along with six cars and a telephone box - is now under control, according to the London Fire Brigade.
The fire is believed have started at some nearby garages and was first reported at 1.43pm on Monday.
Its cause is not yet known, though Southwark Police said the incident is not thought to be terror-related.
In a video posted on social media, a huge plume of smoke can be seen billowing up from railway arches near a set of train tracks in the densely-populated neighbourhood.
Widespread road closures are currently in place around the station.
Firefighters, whose numbers at the scene have swelled from 70 to more than 100, told nearby residents to “close all doors and windows”.
Police also asked travellers to avoid the area.
It is understood that paramedics attended to six people at the scene – two of whom were treated for smoke inhalation.
One person was taken to hospital, according to the London Ambulance Service.
A restaurant worker close to Elephant and Castle station said she was evacuated after the fire broke out.
Alex Leckie, 30, assistant manager at Tupi, said: “The entire building was evacuated, we began to smell smoke and the store owner went to check, we saw smoke billowing out and what I’m told is the mechanics were on fire, the smoke pillar started enveloping the apartment blocks as well and we kept hearing intermittent bangs.
“The police have indicated it could be some time (until they are allowed back inside). They have repeatedly pushed the crowds back even from the Castle Square Park and cordoned off the roads, there is a very, very heavy presence.
“At this point emergency crews have everyone well clear of the area.”
Martin Bobrowski, a builder who is working opposite the station, said: “I heard the police come and I thought there was an accident nearby, but then they started blocking the road off. I then came outside for a cigarette and saw all the black smoke,” he explained.
“The smell was awful. It smelt like a burning electrical cable smell.”
In a statement at around 4:30pm, the London Fire Brigade said: “The Fire in Elephant and Castle is now under control.
"Firefighters will remain at the scene for the next hours to dampen down the scene.
It added: "Road closure are still in place in the area.”
London Mayor, Sadiq Khan, thanked the emergency services.
In a tweet, he wrote: “My thoughts are with all those affected by the very serious fire in Elephant and Castle, which is now under control.
"I’m in close contact with the Fire Commissioner and want to thank our emergency services who acted quickly to ensure residents & passengers were safely evacuated.”
National Rail told commuters that trains between London Blackfriars and Denmark Hill and Herne Hill may be cancelled or delayed because of the fire.
Initially, the company said that “major disruption” could last until 3:30pm, though that projection has now been extended to the end of the day.
Tube trains were prevented from stopping at the nearby Elephant and Castle underground station, though normal services have now resumed.