The brain's visual cortex, which processes information, can be three times bigger in some individuals than in others.
According to the new research, this has an effect on how people perceive what they "see".
While different individuals might have their own thoughts and emotions, it is generally assumed that their understanding of visual images is the same.
But a set of experiments involving optical illusions has demonstrated this is not the case.
Scientists at the Wellcome Trust Centre for Neuroimaging at University College London showed a series of optical illusions to 30 volunteers while scanning their brains.
They included the famous Ebbinghaus illusion, in which two circles of the same size are surrounded by large or small "petals". Most people will see the circle surrounded by large petals as smaller.
In another visual puzzle called the Ponzo illusion, volunteers were shown two identically sized circles superimposed on to the image of a tunnel. The circle placed further back in the tunnel tends to appear larger than the one placed near the front.
By adapting the illusions, the researchers showed that different volunteers saw them differently.
For instance, while some people saw a big difference in size between the two circles in the examples, others barely noticed any at all.
The functional magnetic resonance imaging (fMRI) brain scans revealed a strong link between visual cortex size and the extent to which volunteers were "fooled" by what they saw in the tests.
Smaller visual cortex areas were associated with more pronounced illusions.
The findings are published today in the journal Nature Neuroscience.