The 15-year-old building, home to hundreds of migrant workers and their families, collapsed in the Lalita Park area. Emergency vehicles faced difficulty navigating its narrow alleys to arrive at the scene.
"The scale of the tragedy is unprecedented," New Delhi's top elected official, Sheila Dikshit, said as she visited the site.
Local people said they heard a rumble like thunder as the building collapsed.
"There was nothing left, only people helping each other. There were so many dead bodies, there was no movement at all," said Dil Nawaz Ahmed, a 25-year-old journalist who lives nearby. He dug into the piles of concrete, bricks and mortar with other volunteers and managed to help free five injured, he said. But he mainly pulled out bodies, which he carried to waiting ambulances several blocks away.
The cause of the collapse was not immediately clear. One official said the building may have been weakened by water damage following monsoon rains.
City official Yoginder Chandolia said: "During the recent flood, water reached the foundation and weakened it considerably, resulting in the collapse."
However, residents claimed the landlord was constructing an additional storey on the top floor.
Officials ordered the evacuation of a neighbouring, twin building, fearing it might also collapse, Mrs Dikshit said. Both the collapsed building and the neighbouring block had flooded basements, she said.
Police later said they had filed charges against landlord Amrit Singh but had yet to locate him. Residents said he had fled right after the collapse.
Local volunteers digging for survivors with their hands were later joined by police and firefighters, who used jackhammers to cut through the iron rods jutting from the ruins.
Police brought in sniffer dogs to locate people trapped under the debris. Many rescuers were still working with sledgehammers and their hands last night to remove the rubble, stone by stone and pull out bodies as neighbours watched from nearby rooftops.
Lalita Park is a congested neighbourhood near the banks of the Yamuna river that houses millions of impoverished workers who stream into New Delhi from villages hoping to get jobs in the Indian capital.
"A large number of people come to Delhi in search of jobs.This was cheap accommodation for a lot of them," said Deep Mathur, a city official.
Poor construction material and inadequate foundations are often blamed for building collapses in India. In New Delhi, where land is at a premium, unscrupulous builders often break building laws to add additional floors to existing structures.
Mrs Dikshit said the government would probe whether the landlord had the necessary permits to build an extra storey. The building - like its twin next door - was constructed with crude bricks and splintered wooden window frames.
It housed migrants from the states of Bihar and West Bengal in 80 rooms, with anywhere from three to six residents in each room, residents said. The rooms rented for 2,000 to 3,000 rupees ($44 to $66) a month.
Residents expressed anger that police and firefighters had taken so long to arrive.
"They took more than 45 minutes to reach the site. And then there was confusion about how they were going to bring in the ambulances," resident Mohinder Singh said. But officials said they had enormous problems navigating the muddy alleys. "Our biggest hurdle was to get vehicles through. Even ambulances got stuck," said Mr Mathur.