The No 1-ranked Spaniard had never dropped a set to Jurgen Melzer in three previous meetings but went down 6-1, 3-6, 6-3.
Nadal falls to season's first pre-quarter-final defeat
Rafael Nadal suffered a surprise 6-1 3-6 6-3 defeat by Austria's Jurgen Melzer in the third round of the Shanghai Masters, while Novak Djokovic and Andy Murray marched on.
Nadal, who won the Tokyo Open on Sunday and was playing in his third tournament back-to-back, had not lost a set to Melzer in their three previous meetings.
The crowd was left stunned as he lost that record in wretched fashion, dropping serve twice in the first set amid a series of unforced errors on his way to defeat by the 13th seed.
The world No 2 Djokovic produced a scorching display to reach the last eight with a 6-1 6-1 victory over France's Richard Gasquet
"I did really well from the start. I put pressure on him. I was aggressive and I knew I had to be," said the US Open runner-up who secured his 18th career title at the China Open on Monday.
Murray secured his place at the ATP season-ender in London next month with a routine 6-3 6-4 win over another Frenchman, Jeremy Chardy.
"I didn't get broken today. I served a lot into his forehand and varied the serve, which helped," said Murray, who lost in the quarter-finals of the China Open last week,
The fourth seed will next meet France's Jo-Wilfred Tsonga, who celebrated wildly after beating German Florian Mayer 7-5 6-3.
Spaniard Guillermo Garcia-Lopez also advanced after beating seventh seed Czech Tomas Berdych 7-6 6-3.