The Electoral Reform Society (ERS) said the way list ballots were counted favoured larger political parties and many potential voters felt little motivation to make their voice heard at Scottish Parliament elections.
Turnout on May 5 was 55.6 per cent - up five points from the previous Holyrood poll in 2011, but a significant drop from the 85 per cent who participated in the 2014 independence referendum.
The ERS said previous research into non-voters revealed many felt ‘voting had no effect on their lives’.
“We know these people are predominantly young and poor, and often those who need political representation the most,” said Willie Sullivan, ERS director in Scotland.
“Those who shout loudest often get the most resources, so we should all be concerned about the lack of voice for those who chose not to vote.”
The SNP won 63 seats at the election, two short of an overall majority and six less than its 2011 total.
Based on regional list votes, the party secured 48.8 per cent of seats from 41.7 per cent of the vote.
The Conservatives won 22.9 per cent of the vote and 24 per cent of seats.
Smaller parties including UKIP, RISE and others won 9.7 per cent of the vote but returned no MSPs.
Campaigners believe a small change to Additional Member System at Holyrood is required to ensure fairness.
ERS believes list seats should in future be allocated via the Sainte-Laguë method which is less favourable to large parties than the present D’Hondt system.
The society argues a “more proportional” result would encourage more people to vote.
Sullivan called for the SNP and other parties to “reboot” democracy during the new parliament by considering modifications to the voting system as well as considering the introduction of a second chamber, which could be made up of ordinary citizens.
“A big part of the SNP’s success has been their ability to tap into popular disillusionment with what people see as the political establishment, be that Westminster or the Scottish Labour Party,” he said.
“Their success must be seen as a mandate for reform of our political system, and they should use it to make Scotland the most democratic country in the world.”
Patrick Harvie MSP, co-convener of the Scottish Greens, said there was a growing case for reform, including how Holyrood’s voting system works and the way parliament carries out its business.
“While no system will ever be perfect, we should be aiming to achieve the fairest and most proportional result, while minimising the tactical games many people feel forced into playing,” he said. “Sainte-Lague would certainly be a step in the right direction.
“Beyond the voting system, the Scottish Greens believe the public should have more opportunities for involvement in the legislative process, through measures such as citizens’ juries, public scrutiny committees and perhaps Public Questions sessions, in which members of the public can pose questions to Scottish ministers.”
The Sainte-Laguë method is named after the French mathematician André Sainte-Laguë, and is similar to the current system used to allocate seats on Holyrood list vote.
It was proposed by the Conservative-Liberal Democrat coalition government in 2011 as the method for calculating the distribution of seats in future elections to the House of Lords before reform of the upper chamber was dropped.
An SNP spokesman said: “We are proud that the Scottish Parliament already has a proportional electoral system which ensures that parties share of the seats more accurately reflects their support across the country.
“While we have much work left to do to improve voter engagement and turnout, we are pleased that turnout in last week’s election reached its highest level since 1999. We will look to see a similar improvement in turnout in next year’s local elections.
“We believe proportional representation has been shown to work in Scotland at a national and local level – and that the case for reform of the electoral system at Westminster is now unanswerable.”