Emergency services were called to Eyemouth harbour at 12.20am this morning after two people dialled 999.
The cyclist had managed to clamber on board a small fishing boat by the time they arrived.
Eyemouth lifeboat took him to shore where he was transferred to an ambulance and taken to hospital.
It is thought the man swerved when his dog ran in front of his bike and he tumbled into the water.
The man was not thought to be injured but was mildly hypothermic after up to 17 minutes in the water.