Jones was given the chance to coach the Vols' final two regular season games, but declined.
Brady Hoke is expected to be named as interim coach.
Jones inherited a program that had posted three straight losing seasons under coach Derek Dooley.
The Vols will need an interim head coach and Brady Hoke is the next man up. The Vols haven't won a SEC title since 1998, which was also the year Tennessee won the national championship under coach Philip Fulmer.
Jones had three years remaining on his contract, which owed him $4.1 million annually through the 2020 season.
Hoke was 34-38 at Ball State, 13-12 at San Diego State and 78-70 with the Wolverines, including his best mark of 11-2 in 2011. Hoke, the former head coach at MI, is in his first year at Tennessee. And now, the football program is looking for its fourth head coach in nine years.
As Tennessee's struggles continued, the criticism of Jones mounted. This search will be my sole focus, and I will be in regular contact with Chancellor (Kim) Davenport. By the end of the Missouri game, Tennessee had only five available scholarship offensive linemen, including two true freshmen and two redshirt freshmen. Tennessee's average home attendance climbed from 89,965 the year before Jones' arrival to 100,968 last season.
Tennessee has never had a winless SEC record in 121 years playing football, but must beat LSU and Vanderbilt to prevent that from happening.