The penalty was awarded after Kyle Walker caught Fakhreddine Ben Youssef with his arm.
The Tottenham Hotspur striker bookended a heroic performance and scored either side of a Ferjani Sassi equaliser from the penalty spot, netting a 91st-minute victor as Gareth Southgate's side clinched three points in shaky fashion.
Harry Kane, Dele Alli, Raheem Sterling and Jesse Lingard will be expected to provide the creative spark and goals as England bid to join Belgium at the top of the section following Roberto Martinez's team's 3-0 win over Panama earlier on Monday.
Maguire said: "I had a couple of nervy passes in the first half - it's a big occasion and your first game at this high pressure".
'Brilliant, ' because Kane had to twist his neck and side-spin the ball into a recess of Tunisia's net.
"Credit to the lads, they kept going, kept going to the last second".
Tunisia scarcely managed an attack in the second half but were resolute in defence. "We had so much joy from corners".
England is one of the deepest teams in the tournament and are just a level above Tunisia here.
England were dominating but struggling to create anything of note.
"I definitely didn't expect the manager to drop him against Slovenia and as soon as we all saw that we knew that that gaffer was not somebody to be messed around with", said Rose.
England boss Gareth Southgate says he would have been proud of his players' performance even if they had drawn their World Cup opener against Tunisia.
BBC Sport's Natalie Pirks, on the ground in Volgograd, said: "They are on your face, stick to your lips, get inside your nostrils, your ears and your hair".
However, Harry Maguire flicked a Kieran Trippier delivery into the path of Kane, who made no mistake from point-blank range to break Tunisian hearts.
Tunisia are making quite the surge in the world rankings, however I think this is one of the easier picks in the opening round of matches.