Most social media users do not want toxic posts taken down. In experiments with nearly 9,000 Americans, large majorities shown rude or intolerant posts asked platforms to do nothing at all, and only violent threats against LGBTQ people drew a majority in favour of some action.
The study, “Toxic Speech and Limited Demand for Content Moderation on Social Media,” was published online in the American Political Science Review on January 24, 2024. Its authors are Franziska Pradel, Jan Zilinsky and Yannis Theocharis of the Technical University of Munich and Spyros Kosmidis of the University of Oxford.
Three kinds of toxic posts
The researchers separated toxic speech into incivility, intolerance and violent threats. They ran two preregistered sets of experiments with US adults recruited on the platform Prolific in 2022: Study 1 with 5,130 people and Study 2 with 3,734.
Participants saw a social media post attacking a target. In Study 1 the target was an LGBTQ person, a billionaire or a truck driver with Christian bumper stickers. In Study 2 it was a Democrat or a Republican. Control groups saw a post opposing the target without toxic language, or a post with no information about the target. Everyone then chose what Facebook or Twitter should do: leave it, add a warning label, reduce its reach, remove it or suspend the account.
Threats move people, insults barely do
Compared with the control post, uncivil language raised support for any action by 19.3 percentage points and intolerant language by about 19.2 points. A violent threat, a variation of “punch them in the face,” raised it by 41.9 points.
Who was attacked mattered. Averaged across all kinds of attack, 21.4% wanted action when a billionaire was the target, 36% when it was the Christian driver and 58.2% when it was an LGBTQ person. Even a threatening post aimed at the Christian driver led only about 8% to want it removed and about 6% to want the account suspended.
Outside the LGBTQ case, the authors write, large majorities did not support any kind of moderation, not even for threats, and banning an account was the least popular option.
Partisans judge attacks on their own side similarly
In Study 2, uncivil or intolerant posts about a Democrat or Republican raised demand for moderation by 21 to 22 points, and violent threats by 52 points. Democrats did not ask for much more moderation when a fellow Democrat was attacked, and Republicans did not when a Republican was.
Overall, Democrats were around 10% more likely than Republicans to want moderation, and they were also more protective of targets Republicans tend to identify with, such as the religious driver.
The authors note that their sample was younger, better educated and more Democratic than the US population, much like the typical Twitter user. Because Democrats favour moderation more, they suggest a more balanced sample would probably show even less support. Their conclusion is that no critical mass of users is pushing platforms to clamp down.
Study Details:
- Title: Toxic Speech and Limited Demand for Content Moderation on Social Media
- Authors: Franziska Pradel, Jan Zilinsky, Spyros Kosmidis, Yannis Theocharis
- Journal: American Political Science Review
- Publication Date: January 24, 2024
- DOI: 10.1017/S000305542300134X