Kyle Walker struck from 30-yards out in the 73rd minute to lift Tottenham Hotspur past Arsene Wenger's stuttering Arsenal side.
Tottenham's 'have-a-go' hero Walker sinks Arsenal
Rafael van der Vaart's controversial first-half effort - he appeared to use his arm to bring the ball down - opened the scoring for Spurs before Aaron Ramsey equalised six minutes after the break.
Walker then scored with a superb strike which swerved at the last moment to beat Gunners keeper Wojciech Szczesny.
Walker told Sky Sports 1: "It doesn't get much better than that. I just picked the ball up and hit it.
"I had one of them against Fulham for Aston Villa, so I thought why not take a chance.
"I just thought why not have a go. If you don't buy a lottery ticket, you don't win the lottery.
"Arsenal are a good team and we found it difficult at times, but we dug deep and got the result that we wanted."
Spurs boss Harry Redknapp praised his young defender, who has spent time on loan with QPR and Aston Villa.
"He's a great attacking full-back and he loves to get forward," Redknapp said.
"It was a good strike from Karl. He's done ever so well and he's had a good start to the season."
And Redknapp was pleased his tactics came to fruition as Arsenal's poor start to the season continued.
"They had a lot of possession in the first-half. It was difficult. They played with three in midfield with one up, and so they were always going to hold sway in terms of possession," Redknapp said.
"I played two strikers, so I went for a very attacking team and we got the goal before half-time.
"To be honest, I nearly made a change before half-time to thicken us up in there, but I thought we'll see how we go.
"They got off to a better start in the second-half, but we upped it and got better and stronger and in the second-half we had all the chances.
The win moves Tottenham into sixth place in the Premier League while Arsenal are now in 15th.
Arsene Wenger, who marked 15 years as Arsenal manager on Saturday, has seen his side concede 14 goals in four away games in the club’s worst start to a season in 58 years.
Questioned on Arsenal’s early season underachievement, Wenger said: “It bothers me. We are not where we want to be and it is important we respond quickly.”
Wenger thought both goals could have been avoided.
“I think the first goal was handball and second yellow card,” he said. “The referee, maybe, cannot see it, but how can the linesman not see it? After the second goal, I think we defended badly.”
Wenger’s frustrations were clear in an apparent post-match confrontation with Clive Allen, Tottenham’s assistant manager.
There was further bad news for Arsenal as right-back Bacary Sagna suffered a serious ankle injury in the second half and was carried off.