More than half of the rubbish collected from Ben Nevis in the Highlands is made up of the fruit peel, according to the John Muir Trust.
The charity says up to 1,000 banana skins can be found on the summit plateau at any time.
In the cold temperatures they can take up to two years to biodegrade.
Hillwalkers are being urged to treat their fruit waste like any other rubbish and take it back down the mountain with them.
Conservation officer Sarah Lewis said: "Banana skins are a particular problem because people think they will quickly disappear.
"Sadly this isn't the case.
"We've often caught walkers in the process of chucking their banana skin on the path.
"When you speak to them about it they say it is not a problem because they will biodegrade.
"Quite simply, if you carry something up, you should carry it back down."
At 4,406ft (1,344m), Ben Nevis is the highest peak in the UK.
Cool conditions at the summit means the breakdown process of natural waste takes significantly longer than it would at lower levels.
Skins can become buried in large areas of packed snow and remain there until after it has melted.