Harry Kane gave England the lead with a penalty kick in the 57th minute. But when Yerry Mina rose above Harry Maguire to head in the tying goal three minutes into injury time at the end of regulation, the heavily pro-Colombia crowd inside Spartak Stadium roared out of despair and right back into the game, and the feeling that England might blow it all - again - hung in the air. England captain Kane, who was chosen as Man of the Match, said that it was unlucky for them to concede a corner and allow the Colombians to level the score in stoppage time.
Pickford made a brilliant one-handed save from Carlos Bacca from Colombia's fifth kick in the shoot-out before Eric Dier converted to ensure England will play Sweden on Saturday.
It was the first time Colombia had been involved in a World Cup shootout and they took first blood when Jordan Henderson missed his spot kick - but for once fortune smiled on England. We had the resilience and the belief to get over the line, whatever it takes. "That shouldn't change now that we're in the knockout phase", he said.
"We were ready for that".
'We knew what we had to do and we stayed calm'.
Colombia's star player and the Golden Boot victor at the 2014 World Cup James Rodriguez, was kept on the bench throughout the match due to an injury.
Later reflecting on Colombia's equaliser on his show, he said: "I cheered Colombia's goal as if I had headed it myself".
Sterling then did well to spin past the imposing Davinson Sanchez although his left-footed strike was charged down, while Kieran Trippier curled wide from a free-kick.
The game itself had been a bad-tempered affair, with eight players booked in total - six Colombians, including Wilmar Barrios, who was lucky to avoid red for a first-half headbutt on Henderson.
And as far as supporters at the city-centre pub were concerned, the victory over Colombia was the sign of further success to come this tournament.
John Stones scored two goals from set-pieces in England's 6-1 win over Panama.
Juan Cuadrado blazed over with 10 minutes to play, and it appeared Colombia's hopes were up when Pickford made a terrific save to turn behind Manuel Uribe's spectacular long-range strike.
Maradona, whose celebration of Argentina's dramatic win over Nigeria that sealed a place in the knockout stages will provide one of the lasting memories of Russian Federation 2018, was particularly scathing of Geiger's decision to award a spot-kick to England when Kane was dragged to ground by Carlos Sanchez.
The Andean nation's capital, Bogota, came to a standstill as people tuned in to watch the "Cafeteros", with electrifying roars and loud gasps surging throughout the city each time either team came close to scoring.