Mueller is investigating potential collusion between Trump's campaign and Russian Federation as well as whether the president has unlawfully tried to obstruct the probe.
None of the charges Manafort faces alleges coordination with the Kremlin, and Manafort's attorneys had used that point to argue that Mueller had exceeded his authority by bringing charges against him accusing him of various financial crimes and acting as an unregistered foreign agent on behalf of Ukrainian interests.
Manafort's civil lawsuit relies in part on a law called the Administrative Procedure Act, which dictates how federal agencies write regulations.
Manafort's civil lawsuit against the Justice Department and the special counsel's office, filed in January in the US District Court for the District of Columbia, asked the court to not only declare Mueller's original appointment in May invalid, but also to set "aside all actions taken against Mr. Manafort pursuant to the Appointment Order".
Mueller's team in a filing on Monday rebutted Manafort and asked the court to dismiss his lawsuit, saying it "lacks merit".
Mueller, who was appointed by Trump's Justice Department in May, has brought indictments against 19 people and three Russian companies.
The federal judge overseeing the criminal case against former Trump campaign chairman Paul Manafort expressed exasperation Wednesday about his separate lawsuit challenging the appointment of special counsel Robert Mueller. Mueller's team alleges that Manafort and his business associate, Richard Gates, attempted to hide the money they made from that lobbying work from USA officials.
The memo, attached to Monday night's court argument and not previously disclosed even to Manafort, describes how Rosenstein's public order that appointed Mueller in May left out some details so it didn't confirm "specific investigations involving specific individuals".
The 33-year-old worked closely with Trump's former campaign chairman Paul Manafort and former deputy chairman Rick Gates, both of whom have been indicted with money laundering, conspiracy and tax fraud. Later, Downing clarified that the civil lawsuit was not seeking the dismissal of Manafort's charges but rather "prospective relief", an injunction preventing future action by Mueller's team.
Berman Jackson didn't decide on the lawsuit on Wednesday, and will issue an opinion later.
In addition to authorizing the Russian Federation collusion investigation of Manafort, Rosenstein also specifically authorized Mueller to investigate any crimes related to payments Manafort received from the Ukrainian government during the tenure of former Ukrainian President Viktor Yanukovych.
"Chasing indictment after indictment is not an adequate remedy", Downing said. Downing disputed the extent of Manafort's ties to Russia - Manafort never had an office there, he said - and argued that the Russia section should be interpreted as only applying to activities related to the Trump campaign.
The Justice Department is handling the defense against Manafort's civil suit, but at least a half dozen lawyers from the special counsel's office attended Wednesday's hearing, including Manafort prosecutors Andrew Weissmann, Greg Andres, and Brian Richardson.