The world number four finally overcame Baghdatis under the Centre Court roof at 11.02pm in their third-round match, the latest finish in Wimbledon history and two minutes past the curfew.
They face each other at the same stage of the Olympic tournament, although with the draw only half the size, this time it is for a place in the quarter-finals.
Murray said: "I had a tough match with him at Wimbledon a few weeks ago. I certainly won't be taking anything for granted there. It's going to be very, very tough. He's a very, very good grass-court player. I'll need to play very well to win that."
Murray comfortably saw off Jarkko Nieminen 6-2 6-4 yesterday to reach the last 16, Murray will play his singles match third on Centre Court this afternoon before moving to Court 18 where he and Laura Robson will face Czech pair Radek Stepanek and Lucie Hradecka.
The singles remains Britain's best chance of a medal, with Murray determined to make amends after missing out in Beijing four years ago, and he believes the intense spotlight on the hosts' athletes could be the reason for some below-par performances so far.
He said: "It can be tough if you're not used to it. Because of Wimbledon, I've had a decent amount of experience dealing with it, but it's not easy, and for a lot of athletes this is a once-in-a-lifetime opportunity.
"As long as everyone keeps working hard and doing their best, that's all we can do. There's a long way to go still and I'm sure we are going to do well."
The fifth seed finally won his second-round clash with Milos Raonic 6-3 3-6 25-23 after three hours and 57 minutes on Wimbledon's Court One - with the third set alone taking three hours, another Olympic record.
In an era dominated by Roger Federer, Rafael Nadal and Novak Djokovic, Tsonga was pleased there was something left for him.
The Frenchman said: "It's good because this is the only way to write my name in history for the moment. With Rafa, Roger and Novak, even Andy (Murray), it's tough to go through big tournaments. So I'm really happy. I hope I will have some more."