He was found unconscious and pulled from the water after going overboard as he and others attempted to reach the UK.
French rescue teams sprang into action in the early hours of Friday. The man was taken back to shore, but declared dead.
A manslaughter investigation has been opened, the Boulogne-sur-Mer prosecutor said.
A total of 32 people were rescued off Berck, near Calais, in a state of hypothermia.
In UK waters, multiple Border Force vessels have been active off the Kent coast for much of the day.
Around 35 people were seen being escorted ashore in Dover by immigration staff on Friday afternoon, including a frail man being supported by two officers.
The death comes less than two months after at least 27 people died when their boat sank off the coast of France in November.
Despite the dangers, more than 450 people have made the life-threatening trip across the Channel in small boats so far this year, following a record-breaking year in 2021.
At least 271 people aboard ten small boats reached the UK on Thursday, including a young child wrapped up in an orange lifejacket and wearing a beanie hat.
Mike Adamson, chief executive at British Red Cross, said: “It is devastating to hear that another person has lost their life attempting to cross the Channel. Our thoughts are with them and their loved ones, who may not even know yet what has happened.
“There are no simple answers, but we urge the Government to rethink its plans for making the UK’s asylum system harder to access. This should start with ambitious plans for new safe routes and a commitment to resettle 10,000 people a year.”
Steve Valdez-Symonds, Amnesty International UK’s refugee and migrant rights director, said: “Once again someone has lost their life in tragic and avoidable circumstances, and our thoughts are with the family and friends suffering this devastating loss.
“This tragedy is made all the more unbearable because of the refusal of governments on both sides of the Channel to address the needs and rights of people compelled to attempt these dangerous journeys.”