It was the first such vote on Zuma to be held by secret ballot, and dozens of members of his ruling African National Congress party revolted and supported the motion.
Opposition parties had been pushing for a secret ballot so as to encourage ANC MPs to support the motion.
The secret ballot vote was endorsed by National Assembly Speaker Baleka Mbete who qualified the decision as being in the best interests of the country.
A broad coalition of opposition parties and renegade MPs from the ruling African National Congress (ANC) fell short of the simple majority to pass the no-confidence motion against the president.
Mr Zuma denies any wrongdoing.
The decision to hold the vote came against a background of high levels of unemployment, resentment at South Africa's enduring inequality and economic stagnation.
Protesters started to fill the streets of Cape Town, where South Africa's parliament is located, from the morning, with banners urging legislators to "Fire Zuma".
Zuma, 75, has been dogged by allegations of corruption during his eight years in office. He has survived several motions of no confidence before, although the previous ones were by an open vote, and has dealt with accusations of corruption.
Some ANC MPs have indicated that they would vote with their conscience instead of the party line.
Meanwhile EFF leader Julius Malema made an impassioned plea to the ANC's younger MPs and members of the ANC Youth League to think about their future when casting their votes.
South Africa's main opposition party is accusing President Jacob Zuma of "derelict leadership" in its draft resolution in parliament asking for his removal from office.
He told parliament in June this year that the opposition was trying to get a majority they did not have.
Criticism increased following the sacking of the widely-respected finance minister, Pravin Gordhan, in March.
The statement also accused the opposition of attempting "to collapse government, deter service delivery and sow seeds of chaos in society to ultimately grab power".
Sadly the Chief Whip of the party says the MPs who voted against Zuma will face disciplinary measures.