ChatGPT writes jokes that ordinary people find funnier than most jokes written by ordinary people. In blind ratings, its quips beat 63% to 87% of human joke writers, and its satirical headlines scored about the same as real ones from The Onion.
The study comes from psychologists Drew Gorenz and Norbert Schwarz of the University of Southern California. It was published in PLOS ONE on July 3, 2024, after first appearing as a preprint on PsyArXiv on February 5, 2024.
Quiplash prompts and roast jokes
In the first study, 105 US adults recruited through CloudResearch wrote jokes for nine prompts across three tasks. One asked for a funny phrase to fit an acronym such as “C.O.W.” Another asked for a funny answer to fill-in-the-blank prompts like “Worst first date activity.” The third was a roast: describe, honestly and humorously, a friend’s terrible singing or a disgusting meal.
ChatGPT 3.5 got the same instructions and wrote 20 answers per prompt. Eighteen human participants who admitted copying jokes from AI tools or websites were dropped beforehand, as planned in the preregistration.
A separate group of 200 raters then scored a random mix of human and machine jokes on a 0 to 6 funniness scale, without being told who wrote what.
ChatGPT’s jokes averaged 2.63, against 2.20 for the human ones, and the gap held on all three tasks. The machine beat 73% of human writers on acronyms, 63% on fill-in-the-blank and 87% on roasts.
Looked at rater by rater, 69.5% preferred the AI jokes overall, 26.5% preferred the human ones and 4% rated them equal. The roast result surprised the authors, since ChatGPT is designed to avoid offensive speech and the task asks for an aggressive joke.
Age, sex and political leaning did not predict who preferred the machine’s jokes.
Going up against The Onion
The second study set a higher bar. The researchers took the last 50 headlines from The Onion’s “Local” section published before October 1, 2023, fed them to ChatGPT 3.5 and asked for 20 new ones in the same style.
Then 217 USC students rated five AI headlines and five real Onion headlines, again blind. The Onion averaged 2.49 and ChatGPT 2.37, a difference that was not statistically significant.
The single best-rated headline of all came from the machine: “Local Man Discovers New Emotion, Still Can’t Describe It Properly.” Two of the top four were ChatGPT’s, two were The Onion’s.
Counted by person, 48.8% of students preferred The Onion, 36.9% preferred ChatGPT and 14.3% had no preference. Students who sought out more comedy and satire rated all headlines as funnier, whoever wrote them. Only two had seen any of the real headlines before.
What the comparison leaves out
The human writers in the first study were amateurs answering a survey for $2.58, not people trying hard to be funny. The Onion headlines were a random draw from its timeless “Local” section, chosen because ChatGPT 3.5 could not riff on recent news. The authors also searched online and found no sign that the AI simply copied existing headlines.
The authors argue the findings are a real threat to professional comedy writers. They suggest that creating good jokes may not require feeling amusement at all, writing that “merely knowing the patterns that make up comedy may suffice.”
Study Details:
- Title: How funny is ChatGPT? A comparison of human- and A.I.-produced jokes
- Authors: Drew Gorenz, Norbert Schwarz
- Journal: PLOS ONE
- Publication Date: July 3, 2024
- DOI: 10.1371/journal.pone.0305364