Broom, along with new batsman James Neesham (23), displayed some quality cricket as the duo forged a crucial 27-run partnership before Broom was dismissed by offie Mosaddek Hossain.
But Shakib Al Hasan (114) and Mahmudullah (102 not out) responded to the pressure of a dire position in a must-win game with a brilliant new Bangladesh record partnership of 224 that turned the match upside down and eventually saw New Zealand eliminated.
New Zealand opener Martin Guptill was in excellent touch after the match started an hour late due to rain and drove Bangladesh captain Mashrafe Mortaza for six.
Chasing a total of 266, Bangladesh found themselves 33/4 in less than 12 overs, with three of the top order batsman back in the shed inside the first five overs.
The loser of Friday's Group A match at Sophia Gardens will be out of the tournament.
Williamson has been in fine form with scores of 100 and 87 in the first two group matches, but Bangladesh's Tamim Iqbal made 128 and 95 against England and Australia respectively. In that case, both England and Australia will advance to the last four.
Bangladesh will be the England's Number one fans when they take on Australia at Edgbaston.
Mahmudullah and Mosaddek mopped up the remaining runs to complete an unforgettable win for Bangladesh.
Taylor proceeded to make 63 off 82 deliveries, which included six boundaries, before he was sent packing by Taskin.
They also have a very realistic chance of making it further in the tournament and can definitely churn out a lot of positives from their campaign in this tournament.
New Zealand then lost six wickets for only 64 runs in nearly 12 overs.
Only 62 runs came off the last 10 overs for the loss of four wickets to give Bangladesh the upper hand at the halfway stage.
New Zealand was well set while Williamson (57) and Ross Taylor (63) shared a stand of 83 - their third fifty partnership in as many matches this tournament.
The Kiwis were 201 for 3 in the 39th over and 203 for 4 at the end of the 40th over and they looked set to adding maximum runs in the final 10 overs but that never happened.
Pacer Taskin Ahmed, with 2-43, removed Taylor, but it was offspinner Mosaddek Hossain, who had only one wicket in his last five one-day internationals, who did the most damage. It was Rubel Hossain who removed the risky Martin Guptill for 33 off 35 balls, after trapping him in front with good bowling with one that nipped back off the seam with pace. He scored 33 and was out for a lbw by Rubel Hossain. He fell in a awful mix-up in the 30th over as he was sent back by Taylor after starting for a single. A single in the 45th over made Shakib and Mahmudullah the first Bangladesh pair to have a 200-run partnership in an ODI. Birthday-boy Mushfiqur got hit on the fingers and was also dropped by Ross Taylor at slip on four, before Adam Milne's 146kph thunderbolt blew out his middle stump in the 12th over.