Guptill had to abandon his innings initially on just three, with runners no longer allowed in international cricket, but he returned at the fall of the seventh wicket to get the hosts over the line in pursuit of 258.
Forty-one runs were still needed when he joined McCullum with four overs left. But the opener was to hit the winning run in the penultimate over to finish unbeaten on 27 as the Kiwis took a 1-0 lead with two more matches to play after this first one-day international at Seddon Park.
Jonathan Trott (68), Ian Bell (64) and Joe Root (56) had helped the tourists to a near par total.
But Kane Williamson (74) and then McCullum (69no) kept New Zealand in it before Guptill's late heroics proved decisive.
England's prospects were almost immediately enhanced in the field when James Anderson marked his return to action with a record-breaking 529th international wicket.
Playing for the first time since the Nagpur Test in December, Anderson bowled BJ Watling through the gate with an inswinger - the last ball of his first over.
He therefore went ahead of Ian Botham as England's all-time leading wicket-taker across all international formats.
New Zealand were soon minus Watling's partner Guptill too after he pulled up and had to retire hurt.
Ross Taylor and Grant Elliott both reached the 20s before falling to Chris Woakes. Taylor lost control with a hook shot and was caught at deep backward square, and Elliott was well held at point by Eoin Morgan.
Williamson coasted past a 70-ball 50 and, after surviving a tough chance to Anderson at midwicket off a fierce pull at Graeme Swann on 73, was still carrying the game when he was run out just a single later. Sent back by his captain, he was just short of his ground when Woakes got back to his stumps and deftly transferred Bell's throw back behind him to break the bails.
It was a smart piece of work by the seamer, to add to his wickets.
Steven Finn soon had James Franklin caught behind attempting to hook in the power play; then just when the McCullum brothers were gathering ominous momentum, Nathan was given out lbw pushing forward to Swann.
After a torturous wait for DRS, third official Rod Tucker upheld Gary Baxter's original decision on the basis ball had not definitely hit bat before pad.
It seemed a hammer blow. But Brendon McCullum - who would hit three sixes and six fours in all - survived a tough chance to Trott in the outfield on 52 off Finn, and Guptill and he took telling advantage.
England's trio of half-centurions had laid the foundations for a near par total in stands of 84 and 89 for the second and third wickets.
But when they were gone, the anticipated big finish did not materialise as Mitchell McClenaghan took four for 56. After being put in, Alastair Cook went in the fourth over to a good ball from McClenaghan - bowled through the gate in defence by one that held its line rather than swinging away.
Trott tucked McClenaghan off his pads for four from the first ball he faced, but needed another 60 deliveries before he registered his second boundary.
Bell mistimed a slog-sweep in Nathan McCullum's first over and was dropped at deep midwicket by Watling on 26, and went on to his 50 from 67 balls with a chip from the crease off Andrew Ellis straight over long-on for his second six.
But soon afterwards, up the wicket to try to manufacture runs into the off side off Franklin, he edged behind.
Joined by Root, Trott gradually increased his strike rate. But the big shots were hard work for both players.
Root was reprieved on 32 when the economical Kyle Mills thought he had him caught behind, as did umpire Baxter, only for DRS to overturn the decision.
Trott went selflessly, bowled making room in the power play; then Morgan followed him for just a single when he speared a low catch to point, very well held by Guptill.
Root posted the quickest of England's 50s, from 58 balls but containing just four fours.
Jos Buttler hit two sixes off just 13 balls. But immediately after the second, crashing McClenaghan over wide long-on off the back foot, he toe-ended a full toss straight to extra-cover.
When Root also went trying to slog, bowled by Franklin, England were in danger of falling disappointingly short.
The tail chipped in to help salvage 68 for seven in the last 10 available overs - a statistic which would pale in comparison to Guptill's 24 from just 10 balls in his triumphant late return.
Follow us @SprtNationalUAE