"It's amusing, I read in the paper, you all know, you write it, that I have been a failure at controlling the president, or a failure at controlling his tweeting", Kelly said at a press briefing on Thursday.
The chief of staff also hid his face behind a hand as Trump made defiant threats toward North Korea and "rocket man" Kim Jong Un at the United Nations last month.
When asked about President Donald Trump's frustrations, Kelly, speaking to a group of reporters, said, "One of his frustrations is you".
And asked about the president's own frustrations, Kelly replied frankly: "One of his frustrations is you". I would offer, though, it is not the best job I've ever had.
Rest easier, official Washington: President Donald Trump's chief of staff, John Kelly, says he's not going anywhere. Based on Trump's track record of late, these assertions are either lies or grim indictments of Kelly's ability to do his job effectively, and I'm genuinely not sure which explanation is worse for him.
Kelly also dismissed speculation that he might resign soon due to conflicts with Trump, saying that he had just talked to the president and did not think he would be fired.
But on Thursday he was in a humorous mood and said: "You guys with the cameras always catch me when I'm thinking hard, and it looks like I'm frustrated and mad".
A few times he has tweeted his frustration with diplomacy and with Tillerson, though other senior advisers - including Kelly and Defense Secretary James N. Mattis - were also known to favor diplomacy. "We go in now, rather than onesies and twosies, we go in and help him collectively what he needs to understand to make these vital decisions".
"Unless things change, I'm not quitting, I'm not getting fired, and I don't think I'll fire anyone tomorrow", he continued before taking questions. "This is, in my view, the most important job I ever had".
Kelly added that serving as chief of staff is the "hardest job" he's ever done.
"I would believe, I think I speak for the administration, that that state simply can not have the ability to reach the homeland".
How do I know this?
One his frustrations is you.
Kelly was pushed further by the press on the issue of him leaving, to which Kelly turned things back onto the media.