ScotRail is trying to establish why safety systems failed to prevent the train moving off with some of its doors still open.
The alarm was raised by an off-duty ScotRail worker who was aboard when it reached Shawfair and the train was taken out of service.
One source said the train had travelled “a fair distance, perhaps several miles” with the doors open.
At least three passengers were in the middle carriage involved in the incident, which happened at about 10:30am on Sunday.
The UK Department for Transport’s rail accident investigation branch (RAIB) was notified.
The class 170 train was taken to ScotRail’s Haymarket depot in Edinburgh for investigation.
ScotRail said it was unclear how far the train had travelled with the doors open.
They are usually locked and unlocked centrally on arrival and before departure at stations by the train conductor, who can perform the operation from various points within the train.
Safety systems should prevent trains from moving with doors still open, with warning lights being triggered and the brakes applied.
A ScotRail spokesperson said: “Our 0947 Tweedbank to Edinburgh service on Sunday, 27 September had to terminate at Shawfair due to a fault with the train doors.
"Customers on board were all transferred onto a following train to complete their journey.
“Our specialist teams are now looking in to exactly what happened as part of our investigation procedures.”
ScotRail said it would submit its investigation and repair plans to the RAIB, which would then decide whether to launch its own investigation.