Late show antics from celebrities are nothing new, but people weren't too amused by Penn's cigarette.
At the beginning of the interview, the "Mystic River" and "Milk" actor also said he had taken Ambien, a sleep-inducing drug, and said he isn't acting any more.
The two-time Academy Award victor has written "Bob Honey Who Just Do Stuff: A Novel", which is described as "a scorching, darkly amusing novel about Bob Honey - a modern American man, entrepreneur, and part-time assassin". Probably not. But an amused Colbert, calling back to the Ambien, told Penn, "I'm on board".
"Let's see. I think it's pretty much the same".
The second guest of the night, Roseanne's Sara Gilbert, also reacted to Penn's smoking, telling Colbert, "Wow, I just want to say, Sean Penn makes me look energetic".
He told Colbert, "The greatest thing that an actor can bring to the party is to play well with others-it's the collaboration-and I increasingly don't play well with others", he explained.
Colbert riffed with the star about how he could decipher "Sean Penn on Ambien" from "Sean Penn not on Ambien" and the first giveaway was when Penn took out a cigarette and lit up.
When asked about his motivations for writing the book, Penn said it was a kind of therapy for him living through the Trump era.
But smoking and Ambien aside, Penn got serious and took a moment to praise the Marjory Stoneman Douglas High School student survivors and the organizers of Saturday's March for Our Lives protest, which was sparked by the mass school shooting on February 14.
Penn said: "Within days of that not only are they stating their case with incredible sobriety and articulate words but in such an inclusionary way".