Consumers who used AI chatbots to write complaints about banks and lenders were more likely to get something back. Using a large language model raised the chance of obtaining relief by an estimated 6.9 percentage points, according to an analysis of more than a million complaints filed with the US Consumer Financial Protection Bureau.
The study is by Minkyu Shin of City University of Hong Kong, Jin Kim of Northeastern University and Jiwoong Shin of the Yale School of Management. It was published online on February 18, 2026, in Nature Human Behaviour.
Spotting AI-written complaints
The team analysed 1,134,512 complaints submitted between 2015 and 2024. Each complaint includes the consumer’s own written account and a record of how the firm responded, including whether it closed the case with monetary or non-monetary relief.
Using a commercial AI detection tool and a strict threshold, the researchers flagged complaints that were likely written with an LLM. Before ChatGPT’s release on November 30, 2022, such complaints were close to zero. Afterwards their share rose sharply, reaching 9.8% by March 2024 in an earlier version of the paper. Adoption was higher in ZIP codes where more residents had limited English.
Separating the tool from the user
People who choose to use chatbots might differ from those who do not, so a simple comparison could mislead. The authors used an instrumental variable approach based on two ZIP-code characteristics measured before ChatGPT existed, internet access and English proficiency, which affect whether people adopt LLMs.
That analysis produced the 6.9 point estimate, with a 95% confidence interval of 4.9 to 8.9 points. It also revealed negative selection: the consumers most likely to adopt LLMs were those who would otherwise tend to get worse outcomes.
A falsification check supported the design. Before ChatGPT, English proficiency did not predict whether complaints won relief.
Same facts, better presentation
To test why, the researchers ran three online controlled experiments with 1,010 US participants. They took real complaints and had ChatGPT edit them for clarity, coherence or professionalism, keeping the facts identical. Participants then rated how likely they would be to offer compensation.
The edited versions were more likely to win compensation than the originals, including among participants with work experience in the finance industry. The improvement came from presentation alone, not from any change in content.
The authors acknowledge that the instruments cannot capture every difference between consumers, which is why they added the experiments. They conclude that LLMs can act as an equalizer and call for policies that widen access to the technology.
The consumers turning to chatbots were disproportionately those who would otherwise fare worse, and adoption ran highest where English was weakest.
Study Details:
- Title: The adoption and efficacy of large language models in US consumer financial complaints
- Authors: Minkyu Shin, Jin Kim, Jiwoong Shin
- Journal: Nature Human Behaviour, April 2026, pages 669-680
- Publication Date: February 18, 2026
- DOI: 10.1038/s41562-026-02409-4