The line between Edinburgh and Tweedbank also suffers from stops being missed and overcrowding, said rail consultant and author David Spaven.
Timekeeping had improved up to mid-summer, since when it had been “volatile with an underlying downward trend.”
He added: “There has been just one week without train cancellations since May last year. Overall, this is not the performance one would expect from a new railway, long after ‘teething troubles’ can reasonably be blamed.”
Spaven, whose book Waverley Route – The Battle For The Borders Railway has just reached its third edition, said “skip-stopping” had also plagued the line.
He claimed that missing stops to minimise delays to other trains had contributed to poor passenger numbers at Midlothian stations.
Spaven said it was ironic the focus was on plans to extend the line “while fundamental actions to sort out the existing sub-optimal railway remain largely undiscussed”.
Campaign for Borders Rail spokesman Allan McLean said: “More of the existing railway could be double track but I am optimistic about performance improving.”
The route, which opened in September 2015, attracted 1.3m passenger journeys in its first year – only in line with targets despite previous rail re-openings far exceeding them. The ScotRail Alliance refused to provide second- year figures.
A spokesperson said: “The Borders Railway has been huge success, opening up a wealth of tourism and employment opportunities.
“We will continue to work closely with local communities, businesses and stakeholders to support the vision of making the region a great place to visit, work, live and study.”
A spokesperson for the Scottish Government’s Transport Scotland agency, referring to the line’s first year figures, said: “The Borders Railway goes from strength to strength, as can be evidenced not only by impressive passenger numbers but also the continued inward investment in the area. Seasonal fluctuations in performance are to be expected.”