The poll also found half of Remain supporters back a reduction in the numbers of low-skilled workers arriving from the EU.
Publishing the findings, think tank British Future described Brexit as a “reset moment” for immigration policy.
It proposed a system which would control low-skilled migration through an annual cap while allowing skilled migrants to come to the UK as before.
Sunder Katwala, Director of British Future, said: “There is public support, across political and referendum divides, for an immigration system that combines the UK control demanded in the referendum with openness to the migration that our economy will continue to need.
“A new post-Brexit immigration system that differentiates between skilled and low-skilled EU immigration sounds like common sense to most people.
“They can see that we need doctors, engineers and other professionals but they want more control over low-skilled immigration.
“Even there, the public knows we need people to pick the fruit and veg, build more houses and care for the elderly.”
The report, based on a survey of more than 3,600 people, says more than a third of respondents (38 per cent) would like to see high-skilled migration to the UK from EU countries increase, while 48 per cent support the status quo.
A total of 82 per cent of Brexit supporters would be happy for high-skilled immigration from the EU to remain at current levels (51 per cent) or increase (31 per cent), according to the paper.
Concern with immigration numbers is focused primarily on low-skilled migration, the findings suggest.
Nearly two in three respondents (64 per cent), including 50 per cent of Remain voters, expressed a preference for a reduction in immigration from the EU for low-skilled workers.
The survey also examined attitudes on immigration by profession.
It found most people would prefer migration of overseas doctors and nurses, scientists and researchers, engineers, IT specialists and business and finance professionals to stay the same or increase.
Three-quarters of those polled expressed a preference for the number of care workers coming to Britain to either stay the same or go up.
Majorities were happy for the numbers of migrant construction workers (63 per cent), waiters (52 per cent) and fruit-pickers (63 per cent) to hold at current levels or increase.
The Government’s plans for a post-Brexit immigration system have come under intense scrutiny since the referendum last June.
An analysis published earlier this year showed that EU nationals account for as many as one in 10 employees in some sectors of the UK economy.