The Scottish former world number one began the third round clash in fine form, racing away to an early break before Zverev was able to settle into his rhythm and narrowly take the first set.
The second set began in similar fashion, with Murray again going up a break before allowing Zverev back into the match.
Murray made the most of some uncharacteristic mistakes from Zverev to force a tiebreak, but the 2020 US Open finalist was able to win the crucial points down the stretch to make it through to the fourth round.
Zverev paid tribute to Murray's performance in his on-court interview after the match.
"He's the only one of the Big Four that I hadn't beaten yet, so I'm happy that I've done it today," he said.
"I always stayed in the match, even though I was down a break in both sets. I always knew I had a chance. Especially the second set I think was an extremely high level from both of us. It could have gone both ways.
"Obviously it was a fantastic match. I thought Andy played extremely well, maybe as well as he's played since the (hip) surgery. I hope he continues playing the same way, because tennis did miss him for a long time and I think it's good to have him back."