Britain s three-time grand slam winner was looking for only his second win over a top-10 player since returning from the career-saving hip surgery he underwent two years ago.
There were certainly encouraging signs for Murray, who came through the physical test against the big-hitting fourth seed.
But the shot-making on the big points which propelled him to the very top of the sport has yet to fully return as he slipped to a 7-5 6-2 defeat at the AMRO World Tennis Tournament in Rotterdam.
"From the positive side I thought I moved well for the first hour or so," said Murray. "He's one of the bigger hitters on the tour but in the first set I didn't think he was getting too many free points from the back of the court. He was having to work hard to get the ball through me.
"I did some things quite well in the first set but when it mattered I didn't play well. I lost my serve from 30-0 up. To me that's just not good enough.
"When you are playing one of the best players in the world, if they come up with something great to win those points, you come off and it's understandable.
"But I felt at the end of the first set I blew it a bit. Then I had 15-40 to break back and I messed up a little bit there as well.
"Second set he was much better than me, no question, but at the end of the first set I messed up.
"I'm not used to making those errors in those moments. I'm finding that frustrating mentally and find myself looking back on those moments too much."
Murray's first serve was at least carrying significantly more venom than in Monday's three-set win over Robin Haase in the first round.
The Scot was also comfortable engaging Rublev in long, punishing rallies early on, waiting for the 23-year-old to make the first error.
But the familiar chuntering surfaced towards the end of the first set, Murray at one point exclaiming "I'm totally unprepared for this", although two aces on his way to a hold for 5-4 suggested otherwise.
However, a pair of double-faults gave Rublev the break for 6-5 and a frustrated Murray smashed his racket into the ground.
He failed to capitalise on two break points as the first set slipped away in just over an hour.
Rublev, a renowned frontrunner, broke for 3-2 in the second and wrapped up victory with five games in a row.
Murray is still weighing up his schedule as he tries to build up his match sharpness without compromising his fitness.
"I want to play as much as I can but also bearing in mind I have to be smart with the decisions about which tournaments to play," he added.
"I'll have next week off and then hopefully be able to play in Dubai and Miami and then see from there."
There was more bad news for Britain as Cameron Norrie went down 6-2 6-2 to Russian Karen Khachanov.