England had been pegged back to 1-1, missed a host of chances and denied two penalties before Kane snuck in with a late header to send the nation wild.
Kane opened the scoring before a controversial penalty allowed Tunisia back into the match through Ferjani Sassi just before half time.
And it was down to the skipper to rescue all three points late on.
Kyle Walker had been bombarded with messages on Twitter before Kane’s goal after his mistake had led to the equaliser.
England fans were slating the Manchester City defender for costing them two points – and he was clearly relieved as the second Kane goal went in.
And the referee had become such a figure of complaint that his Wikipedia page had been edited to say he was from Tunisia.
Wilmar Roldan decided Walker’s elbow impeded Fakhreddine Ben Youssef, but then riled England fans further by refusing to award England a spot kick on at least two occasions over the course of the game.
But England overcame Tunisia, the referee, VAR and their own poor finishing at the death with captain Kane leading the charge and sending them on towards Panama on Sunday.
‘We kept fighting’
Jordan Henderson admits the late winner was a bonus and thinks it can generate momentum as they go through the gears in Russia.
He said: ”It’s a massive boost, a massive bonus, we kept going. We kept fighting and got the late goal and a big win.
“I felt positive at 1-1, the pace did drop off a bit. We had to keep creating chances, we didn’t play as well second half but they defended deep.
“Winning the first game is always massive.”
‘Teams won’t want to play England’
Rio Ferdinand believes England sent out a warning with their never-say-die attitude.
They face Panama next before rounding off the group against Belgium.
Results in the previous games have opened up potentially easier routes through the knockout stages now.
Germany and Brazil both failing to win their opening games has left those predicted routes off balance.
However none of that would matter should England fail to get through their own group.
But the points tonight puts them on course for that and Ferdinand thinks people will be worried about coming up against Kane and co.
Speaking as a pundit on BBC he said: “We are are where we should be.
“The energy in the first half was impressive and there will be teams out there who won’t want to come up against them.
“The two substitutes were important too.”
‘Southgate has created team spirit’
Kane believes the late winner shows the spirit Gareth Southgate has instilled into the team is working and will stand them in good stead going on.
England kept knocking on the door deep into stoppage time and were rewarded.
“It’s massive,” Kane said. “I’m so proud of the lads. It’s tough. We played so well especially in the first half and we could have scored a few more. We kept going. It’s a World Cup, you go to the last second. I’m absolutely buzzing.
“We’ve done it for a while [had good resilience] since the gaffer has been here – he’s instilled it into us. We’ve got a great bond off the pitch so it’s great to see it on the pitch. We’ll get onto the plane happy tonight.
“We could have had a couple of penalties, especially when you look at theirs. A few corners, they were trying to grab, hold and stop us running. Maybe a bit of justice to score at the back post at the end. That’s football, that’s the ref. It showed good character to get on with it.
“I’m really proud. We did a lot of work on training field and were itching to play. Get the nerves and excitement out of the play.”
‘Result of hard work’
While Southgate beamed with pride at the final whistle and insisted they were worth the victory.
He said: ”I was happy of the way we kept playing even though the clock was running down. We stayed patient, we didn’t just throw the ball in the box. We deserved the win.
“We created so many clear cut chances, especially in the first half, and were in total control in the second half. We were strong on set plays all night. Even if we’d drawn, we‘d have been proud of the performance.
“We’ll do well to make as many chances in a game again in this tournament. The movement, pace, control from the back with the ball was pleasing. We wore them down. Good teams score late goals – if you dominate the ball like that the opposition tire.
“It’s difficult to highlight individuals as it was a collective performance. The squad have been brilliant. We got this result over the last three weeks, not just tonight. That’s from everyone who worked so hard together.”
