Blaine Gabbert and Cecil Shorts III combined with a 80-yard touchdown pass to win the match,, while the New Orleans Saints are still winless.
NFL round-up: Jacksonville Jaguars steal win from Indianapolis Colts
Blaine Gabbert, Jacksonville's second-year quarterback, combined with Cecil Shorts III for an 80-yard touchdown pass with 45 seconds remaining as the Jaguars took a 22-17 win over the Indianapolis Colts.
New coach Mike Mularkey picked up his first Jaguars win despite his side trailing 17-16 with under a minute left before Gabbert linked up with Shorts.
The New Orleans Saints remain winless since coach Sean Payton was suspended in connection with the NFL's bounty investigation as Ryan Succop kicked six field goals in a 27-24 win for the Kansas City Chiefs, including one to force overtime and the clinching score.
The Dallas Cowboys went 2-1 as Tony Romo finally had some short scoring drives in a 16-10 victory over the Tampa Bay Buccaneers.
The Arizona Cardinals made their best start for 38 years as they went 3-0 with a 27-6 win over the Philadelphia Eagles which saw quarterback Michael Vick sacked five times.
Kevin Kolb starred against his former team, throwing for 222 yards and two touchdowns as James Sanders returned one of Vick's two first-half fumbles for a touchdown.
Matt Ryan threw touchdown passes to three different receivers and safety Thomas DeCoud had two interceptions and a fumble recovery as the Atlanta Falcons won 27-3 at the San Diego Chargers to remain undefeated.
Payton Manning failed for a second straight game to rouse the Denver Broncos in a 31-25 defeat to the Houston Texans who reached 3-0 for the first time in their 11-season history with Matt Schaub throwing for four touchdowns.
Sebastian Janikowski kicked a 43-yard field goal on the last play as the Oakland Raiders scored the final 13 points to beat the Pittsburgh Steelers 34-31.
But there was bad news for the Raiders as wide receiver Darrius Heyward-Bey was taken to hospital with a neck injury early in the fourth quarter after a helmet-to-helmet hit in the end zone with Steelers safety Ryan Munday that was not called a penalty.
Justin Tucker kicked a 27-yard field goal to give the Baltimore Ravens a 31-30 victory over the New England Patriots in a rematch of the AFC championship game.
When the teams met in January, the Patriots squeezed out a three-point win at home to advance to the Super Bowl.
The game-winning kick followed a pass interference call against Devin McCourty on Jacoby Jones.
Torrey Smith, who was playing less than 24 hours after the death of his 19-year-old brother, had six catches for 127 yards and two touchdowns for the Ravens.