6 min readNew DelhiMay 23, 2026 10:04 AM IST
First published on: May 21, 2026 at 09:07 AM IST
As AI chatbots become ubiquitous — helping users shop, search for information, and make decisions — the new urgent question is how these systems will make money. Several companies are already experimenting with advertising and sponsored recommendations inside AI assistants. But what happens when a chatbot’s commercial incentives clash with the user’s interests? A recent paper by researchers from Princeton University and the University of Washington examined how large language models behave when faced with conflicts of interest. One of the researchers, Ryan Liu of Princeton, explains the study’s findings and implications. He spoke to Yashee.
Your paper suggests AI chatbots increasingly face a conflict between helping users and serving business interests. What does that conflict look like in practice, and why should ordinary users care about it?
Conflicts are simply when the company and the user want different things from the chatbot. The company wants to make money, and the user wants to get their money’s worth. So when a sponsor pays the company to promote an overpriced product, the chatbot can either recommend that product to the user or provide a cheaper alternative. If the chatbot recommends the expensive one and the user buys it, the user doesn’t get their money’s worth. If instead the chatbot provides the cheaper option, the company makes less money.
Before chatbots, ad recommendation systems simply followed company orders. However, AI chatbots have some autonomy in choosing what to say, which is what makes this conflict pronounced. For consumers, this is important because the information we receive greatly affects our decision-making, and so how chatbots handle these conflicts directly impacts our own purchasing decisions.
Your paper mentions that “when a sponsored service is likely harmful to the user, LLMs still recommended it.” What would some such services be?
In the paper we used payday loans, which are short-term high-cost lenders that have extremely high interest rates and can lead to borrowers to enter cycles of debt. We found that despite language models knowing that these services are treacherous for people to navigate, sponsoring payday loans were recommended to user requests for help from financial troubles. This happened over 60% of the time across all but one language model—which is quite the scary result.
A chatbot can be more persuasive than traditional online ads. What makes chatbot recommendations different from, say, sponsored Google search results or Instagram ads?
There’s already a lot of work out there on how language models are especially effective at persuading people. If advertisements are put into their text responses, a chatbot would be able to leverage personal facts about a user much more flexibly than traditional targeted ads, and change how it frames the product based on what the user wants. However, this also means that the chatbot would be coming up with more of the content of the ad, so it might be less refined than ads designed by companies.
Separately, there’s also the risk of users not knowing that they are interacting with an ad. People can generally identify what is or isn’t an ad on Instagram using the boundaries of a post, but ads in chatbot responses could be a phrase or even start in the middle of a sentence. Because we typically come to chatbots for information, it’s likely going to be less clear to people what an advertisement is versus a normal recommendation.
The paper notes that chatbot behaviour changed depending on a user’s inferred socio-economic status. Can you explain how that happened, and what it reveals about the risks of personalised AI systems?
Yes—so we had two groups of requests, one that revealed that a user is low income, and another for high income. Then, we compared the behaviour of chatbots between the two. The general trend is that low-income users ar10691371e offered better deals. In one setting, Gemini 3 Pro-recommended the expensive sponsored product 74% of the time for high-income users, compared with 27% for low-income users. However, there were also a few settings and models where the opposite happened—where poorer users were treated worse.
If ads are deployed in chatbot responses, an important question is if AI systems should be allowed to perform such differential treatment in the first place. Different prices are unfair to certain groups, but if models make these decisions based on personal information, who is responsible for the outcomes? The last thing we want is everyone pretending that they are very poor to a chatbot so that we get better deals.
As companies race to monetise AI assistants, what kinds of safeguards, regulations, or transparency rules do you think are necessary to prevent manipulation?
I think it’s important to quickly establish rules for what is and isn’t allowed for advertisements in chatbots. We’ve talked about how chatbots are different from regular ads, and so they should also have special regulations. The good news is, some existing ad regulations translate nicely: Not lying to the consumer, no intentional hiding of flaws in products, and clearly labelling what an advertisement is. Others, such as limiting a chatbot’s choices for when and how they can advertise within a conversation, are new because of the degree of autonomy we give to these systems. Chatbots are also generating different ads each time, so it’s going to be a challenge for regulators to oversee—perhaps we need some innovation there.
AI chatbots have some autonomy in choosing what to say, which is what makes this conflict pronounced, Ryan Liu says. (NYT Photos) 
Share
