Users’ prompting strategies and ChatGPT’s contextual adaptation shape conversational information-seeking experiences

Why our chats with AI matter

Millions of people now turn to chatbots like ChatGPT to answer questions about health, science, and public policy. This study asks a simple but important question: what actually happens in those conversations? By looking at how ordinary Americans ask questions and how ChatGPT adjusts its replies, the researchers show that subtle differences in wording and topic can shape not only how people feel about the AI, but also what they end up believing about real-world issues.

How the study was set up

The research team recruited a nationally representative group of 937 U.S. adults and asked each person to have a multi-turn information-seeking conversation with ChatGPT. Participants were randomly assigned to one of six topics that covered health (such as COVID-19 vaccines and artificial sweeteners), science (such as climate change and microplastics), and policy (such as immigration and highway infrastructure). Some topics were intentionally controversial, others relatively neutral. Everyone was told to imagine preparing for a neighborhood discussion and to chat with ChatGPT for at least five turns to gather information before reporting their attitudes and impressions.

How people actually talk to ChatGPT

Despite the buzz around “prompt engineering,” the study found that most people do not use fancy tricks when they talk to ChatGPT. Only 19.1% of users employed at least one explicit prompting strategy, such as giving extra background, asking for sources, requesting a certain writing style, or asking for step-by-step reasoning. The vast majority simply typed straightforward questions. Those who did use such strategies were more likely to be college-educated and to lean Democratic in their politics. People who were already very familiar with AI tools were especially likely to fine-tune ChatGPT’s style (for example, asking for shorter or more conversational answers). In contrast, people with more prior knowledge about a topic tended to use fewer strategies, likely because they felt less need for extra guidance.

How ChatGPT changes its answers

The authors then examined the language of both user messages and ChatGPT replies across thousands of turns. They looked not at factual content, but at styles of communication: asking for information, stating facts, sharing personal experience, urging action, and using more cognitively complex language. They also counted how often ChatGPT included web links and how heavily it relied on structured formatting like headings and bullet points. ChatGPT’s replies clearly shifted with the situation. When the topic was controversial, answers tended to be more mentally demanding and action oriented, and they included more external links but were less rigidly formatted. Different prompting tactics also nudged ChatGPT in specific directions: giving more background led to more action-focused advice, style requests pushed the AI toward simpler wording, and content-focused requests encouraged more elaborate, cognitively complex explanations.

How these answers affect people

The most striking finding came from linking ChatGPT’s style back to people’s reactions. Replies that scored higher in cognitive complexity—those that wove together ideas in a more layered, analytic way—had a double-edged effect. On the one hand, users liked these answers less: they rated the responses as lower quality, saw the AI as less likable, and even judged it as less intelligent. On the other hand, these same complex answers were more effective at shifting people’s views on the issues themselves. After just one such conversation, participants showed stronger concern about microplastics and climate change, and greater support for vaccination, immigration, artificial sweeteners, and highway reconstruction, even after accounting for their initial opinions.

What this means for everyday AI use

For everyday users, the study suggests that you do not need expert-level prompts to get useful information from ChatGPT, but the way you ask still matters, and not everyone has equal skill or comfort in shaping those conversations. For designers and policymakers, the work highlights a new kind of digital divide: differences not just in who has access to AI, but in who knows how to talk to it effectively. It also reveals a design tension. Simple, easy-to-read answers make users feel better about the AI, while more mentally demanding answers can quietly push their attitudes in meaningful ways. Building chat systems that are both accessible and transparent about this influence will be essential as people increasingly rely on AI to navigate complex public debates.

Citation: Xue, H., Oh, Y.J., Zhou, X. et al. Users’ prompting strategies and ChatGPT’s contextual adaptation shape conversational information-seeking experiences. Sci Rep 16, 12112 (2026). https://doi.org/10.1038/s41598-026-42465-4

Keywords: conversational AI, digital divide, prompting strategies, ChatGPT, attitude change