City officials and Louisiana governor John Bel Edwards confirmed the death.
The building was under construction at the corner of Rampart Street and Canal Street, a broad boulevard just outside the Quarter, lined with restaurants hotels and retailers.
Canal Street, which carries six lanes of traffic, separates the Quarter from the city's main business district.
WWL-TV aired and tweeted a viewer's dramatic video of the collapse, showing upper floors falling on top of each other before one side of the building toppled to the ground.
Another video on social media was taken by someone aboard one of the city's famous streetcars as it approached the site while the building was collapsing.
It showed what looked like a metal structure - part of the building or a piece of construction equipment - tumbling to the ground and people running from the scene as clouds of dust
billowed up and around the streetcar, obscuring the view like a thick fog.
Authorities say 18 people were taken to hospital for treatment. All were considered stable.
Evacuees included guests at a hostel across the street.
"I heard a huge noise and thought it was a plane crashing. Then, the hostel shook," guest Sue Hurley, 68, told The Associated Press.
She said she was reminded of news accounts of the 9/11 terrorist attacks. "The noise was as strong as the 9/11 crash. The dust was as thick."
Another hostel guest, Michael Arbeiter, 30, from Germany, said he was just getting out of the shower when the room shook.
"I'm not sure what happened but they told us to get out of here," he said. "I'm supposed to stay until Monday. Thank God it was not another 9/11."
Mr Edwards urged people to stay away from the area, which was still considered unstable.
An unsupported crane listed away from the building site. As dust settled, twisted metal, concrete pilings and other wreckage covered part of Rampart Street.
"It was a deep rumbling sound," Matt Worges, who saw the collapse from a nearby building, told The Times-Picayune/The New Orleans Advocate.
"Like an airplane maybe. It drew my head immediately."