Study: A.I. Just As Funny As Human Late-Night Comedy Writers

In 2023, Patton Oswalt issued a warning: Artificial intelligence was going to get good at writing comedy and faster than we think. “When writers make fun of ChatGPT — ‘look how bad this writing is’ — it’s in its infancy right now. It’s growing exponentially,” he told Conan O’Brien. “We will get to a point where we won’t be able to distinguish anymore, and that’s going to be terrifying.”

Are we already there? A new study published in the Proceedings of the 1st Workshop on Computational Humor argues that we’re darn close. The experiment pitted jokes written by Witscript (an A.I. program developed by former David Letterman and Jay Leno writer Joe Toplyn) against gags written by a “longtime joke writer for a well-known, U.S.-based, late-night comedy/talk show” — in other words, a human being who does this for a living.

To make the competition as fair as possible, the study took the following measures:

• Joke topics were based on evergreen news headlines that wouldn’t get stale before testing was finished.  
• Both the joke-writing A.I. and the human comedy writer were given three days to write punchlines, eliminating any speed advantage that the computer program might have
• Actual comedians would tell the jokes in a club environment. The comics were given the jokes in random order with no clues as to their authors. “As a cover story, the comics explained that they would be performing some jokes written by a friend.”

For the purposes of this study, jokes were evaluated by the amount of laughter received from a North Hollywood comedy club audience. You can read the academic formulations if you’re interested in how this was all quantified.  

You can probably guess what happened next. In each of his sets, the professional comic delivered eight jokes, half written by the human comedy writer and half written by A.I. In the first sample set, three of the four highest-ranking jokes, including the one at the top of the list, were generated by A.I. A similar pattern emerged after another set. 

The main conclusion? “A.I.-written jokes, performed in front of a live audience, elicited laughter within the same range as jokes written by a professional human comedy writer.”

The study’s authors know what you’re thinking. Couldn’t the comic’s delivery affect which jokes got the most laughs? Sure, but since the comedian didn’t know which punchlines were which, the study assumes that “these factors influenced A.I. and human-written jokes equally.” 

Wouldn’t different audiences respond differently to different jokes? Sure, but the same logic applies — both sets of jokes should be equally affected.

I’ll admit I was skeptical until I read the competing punchlines in the study’s appendix. For nearly all the topics, I thought A.I.’s punchline was funnier than the human being’s — or at least just as good.  Here’s one joke set-up they were given: Bob Yerkes, a stuntman who appeared in Star Wars, died at the age of 92.

• HUMAN PUNCHLINE: In his long career, he broke so many bones, his grave says Rest in Pieces. But a true Star Wars fan to the end, he asked to be buried in his parent’s basement.
• A.I. PUNCHLINE: He passed away surrounded by his loved ones and a strategically placed pile of mattresses.

Pretty funny, A.I. I’m tempted to blame the human being for not stepping up his game, but that’s not the study’s point. If the guy truly is employed as a late-night comedy writer, the A.I. is keeping up with a pro and then some. 

Patton Oswalt is proving to be a prophet.