Pakistan, chasing 261 for victory, were dismissed for 231 with a ball to spare after all of India's five bowlers took two wickets apiece.
Misbah-ul-Haq, who top-scored for Pakistan with 56, helped take them from 208 for nine to a situation where an unlikely 30 was needed off the last over.
More on India v Pakistan
• Dravid: India are favourites against Pakistan in today's World Cup semi
• When India v Pakistan matches reached boiling point
• India captain Dhoni wary of Pakistan counterpart Afridi's spin
• A two-point agenda for everyone at Mohali
• Time for some hyperbole as Afridi blames media
But Zaheer Khan held his nerve, bowling four dot balls before Misbah holed out to Virat Kohli to the delight of a capacity crowd. India's 260 for nine after winning the toss against their arch-rivals was built around man-of-the-match Sachin Tendulkar's 85.
But they might have made considerably less had not the star batsman been dropped four times in an innings that still left him one short of a hundred international hundreds.
Left-arm fast bowler Wahab Riaz led Pakistan's attack with career-best figures of five for 46.
Pakistan saw Kamran Akmal start their pursuit with a first ball four off Zaheer. But the left-arm quick had his revenge with a slower ball the opener sliced to Yuvraj Singh at point to leave Pakistan 44 for one.
Mohammad Hafeez made 43 before he was caught behind after trying to sweep seamer Munaf Patel.
Yuvraj, out for a golden duck, atoned in front of his home crowd with two wickets for no runs in five balls as Pakistan slumped to 106 for four.
The left-arm spinner bowled Asad Shafiq (30) as the batsman made room to cut and then had the experienced Younus Khan (13) caught at cover by Suresh Raina.
Umar Akmal, however, counter-attacked in trademark style, pulling and driving Yuvraj for the first two sixes of the match. But on 29 he was bowled by an arm ball from local hero Harbhajan Singh to leave Pakistan 142 for five.
Pakistan required 84 off the last 10 overs and needed a big innings from big-hitting but out of form skipper Shahid Afridi.
Afridi only managed 19 before he sliced Harbhajan to Virender Sehwag at cover-point and at 184 for seven the game was almost up for Pakistan.
Earlier, the 25-year-old Riaz kept his side in the match. Riaz, on as first change, struck fifth ball when he had dangerman Sehwag, hitting across the line, lbw for 38 including nine boundaries.
Tendulkar, opening with Sehwag, subsequently square-drove Riaz for four to become the first batsman ever to score 400 runs in three separate World Cups.
But off-spinner Saeed Ajmal nearly had him out twice in successive balls for 23, a reviewed lbw and tight stumping call both going against Pakistan.
Then, on 27, Tendulkar pulled leg-spinner Afridi hard to short mid-wicket only for Misbah to drop the catch. Tendulkar was dropped again, on 45, when Younus at mid-off failed to hold on to a mistimed drive off leg-spinner Afridi.
Riaz, however, had Kohli caught by Umar Akmal at backward point and next ball clean bowled Yuvraj with a swinging full toss.
Tendulkar was given a third reprieve by Pakistan on 70 when wicketkeeper Kamran Akmal failed to hold a tough chance off an edged Afridi leg-break.
And he was on 81 when a leaping Umar Akmal at short mid-wicket dropped a checked drive against off-spinner Hafeez.
However, Afridi ended his innings when he took a sharp catch at short extra-cover off Ajmal. Tendulkar faced 115 balls with 11 fours.
Riaz had Dhoni lbw before completing his five-wicket haul when Zaheer was caught behind.
Virender Sehwag lbw b Wahab Riaz 38
Sachin Tendulkar c Shahid Afridi b Saeed Ajmal 85
Gautam Gambhir st Kamran Akmal b Mohammad Hafeez 27
Virat Kohli c Umar Akmal b Wahab Riaz 9
Yuvraj Singh b Wahab Riaz 0
MS Dhoni lbw b Wahab Riaz 25
Suresh Raina not out 36
Harbhajan Singh st Kamran Akmal b Saeed Ajmal 12
Zaheer Khan c Kamran Akmal b Wahab Riaz 9
Ashish Nehra run out 1
Munaf Patel not out 0
Extras: (lb 8 w 8 nb 2) 18
Total: (for nine wickets; 50 overs) 260
Fall of wickets: 1-48, 2-116, 3-141, 4-141, 5-187, 6-205, 7-236, 8-256, 9-258
Bowling: Umar Gul 8-0-69-0 (2nb, 1w), Abdul Razzaq 2-0-14-0, Wahab Riaz 10-0-46-5 (4w), Saeed Ajmal 10-0-44-2 (2w), Shahid Afridi 10-0-45-0, Mohammad Hafeez 10-0-34-1
Kamran Akmal c Yuvraj b Zaheer 19
Mohammad Hafeez c Dhoni b Patel 43
Asad Shafiq b Yuvraj 30
Younus Khan c Raina b Yuvraj 13
Misbah-ul-Haq c Kohli b Zaheer 56
Umar Akmal b Harbhajan 29
Abdul Razzaq b Patel 3
Shahid Afridi c Sehwag b Harbhajan 19
Wahab Riaz c Tendulkar b Nehra 8
Umar Gul lbw b Nehra 2
Saeed Ajmal not out 1
Extras (w8) 8
Total (all out, 49.5 overs) 231
Fall of wickets: 1-44 (K Akmal), 2-70 (Hafeez), 3-103 (Shafiq), 4-106 (Younus), 5-142 (U Akmal), 6-150 (Razzaq), 7-184 (Afridi), 8-199 (Riaz), 9-208 (Gul), 10-231 (Misbah)
Bowling: Zaheer 9.5-0-58-2 (6w); Nehra 10-0-33-2 (1w); Patel 10-1-40-2 (1w); Harbhajan 10-0-43-2; Yuvraj 10-1-57-2.
Result: India won by 29 runs
Man-of-the-match: Sachin Tendulkar