Martin said: "I am really enjoying my football with Peebles - it's a great little club with good team spirit. That's the first time I've ever scored a hat-trick at any level, even when I played as a boy."
The derby sprung to life on the half hour when Streamers defender Gary Hamilton was sent off for tugging back Lewis Montgomery inside the box.
Martin's penalty was saved by Mark Walters but the midfielder blasted the rebound into the net. Just three minutes later the same player charged into the area to steer Alan Chalmers' low cross beyond Walters from close range.
Coldstream's ten men were thrown a lifeline on 39 minutes when a Stevie Doohan corner fell to Darren Lockhart and the former Civil Service player rattled in an unstoppable shot to pull a goal back.
But Martin reinstated Rovers' two-goal lead before the break as he darted clear of the home defence to chip Walters from 25 yards. Coldstream had to wait until the 89th minute before James Guy cut in from the right flank to find the net from a tight angle.
Coldstream: Walters, P Crawford, Hamilton, Brown, Doohan, Lockhart, Hunter, Guy, J Crawford, Christie, Zavaroni. Subs: Irvine, Hebdon, Rae.
Peebles: Callan, Urey, Anderson, Innes, Howe, Martin, Hunter, Morrison, Montgomery, Black, Chalmers. Subs: Collin, Leslie.
Peebles Rovers 3