Liam Kerr, his Scottish Conservatives’ shadow, said reaching the target recommended by official advisers the Climate Change Committee would require 4,000 new charging points a year.
He said despite Mr Matheson planning a “seamless, accessible and available” network, “it’s difficult to see how it’s going to happen”.
Mr Matheson said of the recommendation for 30,000 charging points: “It is a significant challenge because it is significant piece of infrastructure that needs to be put in place.”
Scottish Labour transport spokesperson Colin Smyth said “we need to go further and a lot faster”, pointing to Transport Scotland guidance for one charge point per ten vehicles, which suggested 50,000 to 100,000 points would be required.
The MSPs were responding to the publication of the Scottish Government agency’s “Draft Vision for Scotland’s Public Electric Vehicle Charging Network” document, which includes a £60 million fund for increasing the number of charge points, which half the money is expected to come from the private sector over the next four years.
The strategy also involves improving their design and siting with experts from the V&A Dundee, including at poorly-lit locations to boost safety, and to prevent pedestrian access to pavements being impeded.
Mr Matheson said: “Existing charging infrastructure does not always adequately serve people with mobility needs, and women drivers have also raised concerns about some charge points being in poorly-lit locations and feeling unsafe.
"We can do things better.”
Mr Matheson said Scotland already had the largest number of charge point per head in the UK outside London.
He said electric cars now accounted for more than one in five new car sales.
The minister said there would be an announcement “in the very near future” over requiring charging points to be provided with all new homes.
He also stressed despite a target of cutting vehicle travel by 20 per cent by 2030, “cars and vans will still have a role to play and particularly in rural areas”, but they needed to be electric.
Mr Matheson said: “We have invested over £50m to create a network with over 2,100 public charge points across Scotland.
"With demand for electric vehicles rapidly increasing thanks to government incentives, public and private sector partnerships will now be key in attracting investment and scaling provision at pace.
“The £60m public electric vehicle infrastructure fund will draw in commercial investment so the charging network works for everyone, while at the same time potentially doubling the size of our public network in Scotland.”
Aberdeenshire East SNP Gillian Martin called for more rapid chargers.
She said: “It has become very apparent to me as a lone female driver that reliable rapid charge is absolutely essential so you are not spending hours alone in your car.”
Neil Swanson, director of the Electric Vehicle Association Scotland, whose members want urgent progress, said: “The shift from petrol and diesel to electric vehicles is going to be one of the largest examples of public or private policy implementation of the next decade.”