Conditional effects of source expertise and pre-existing attitudes on objective knowledge in AI-assisted health information verification

Why this study matters to everyday readers

Many of us now ask tools like ChatGPT for help judging health advice we see online, from diet tips to miracle cures. This study looks at a simple but important question: when people double-check a popular claim about gluten-free diets with the help of AI, what really shapes what they learn—whether the post looks like it comes from an expert, what they already think about gluten, or how sure they are about their own knowledge? The answers reveal when AI can support sound understanding, and when our own beliefs still get in the way.

Health tips, AI helpers, and mixed messages

The researchers focused on a common real-world situation: a Facebook post about gluten-free diets that blends accurate facts with misleading statements. Gluten-free eating is essential for people with celiac disease, but it has also become a lifestyle trend promoted as healthier for everyone, even though evidence is weak. That mix of science, marketing, and rumor makes gluten a useful test case for studying how people sort truth from hype when AI is available as a checking tool. Instead of treating ChatGPT as a persuasive source, the authors saw it as part of a “verification environment” where people share the work of thinking with technology, still needing to judge what to trust.

How the experiment worked

In an online study with 103 participants, everyone saw the same mixed-accuracy Facebook post about gluten-free diets, but the supposed source was changed. For some participants, the post appeared to come from a food-focused media brand that signaled expertise; for others, it was from an ordinary individual user. After reading the post, participants were told to use ChatGPT in any way they wished to check the information. They then completed a detailed quiz that objectively tested what they now knew about gluten, alongside questions about how they felt about gluten-free diets and how much they trusted and valued their ChatGPT search.

When expert labels help—and when they do not

Surprisingly, simply labeling the Facebook post as coming from an expert source did not, on its own, lead to better factual knowledge after the ChatGPT search. On average, people who saw the expert-labeled post did not score higher on the knowledge test than those who saw the non-expert version. But the picture changed when the researchers looked at people’s prior attitudes. Among participants who already felt quite positive about gluten-free diets, the expert label did make a difference: those readers ended up with more correct answers. In contrast, for people with neutral or negative attitudes, the expert label hardly mattered. This suggests that expert signals mainly help those who are already motivated to engage with the topic, rather than automatically boosting learning for everyone.

Uncertain versus confidently wrong

The study also distinguished between two types of knowledge problems: people who openly admitted “I don’t know” and those who held clearly wrong beliefs with confidence. Using patterns of right, wrong, and “don’t know” answers, the authors classified participants as relatively uncertain or relatively misinformed. Their planned test of a three-way pattern—expert label, attitude, and knowledge state together—did not reach the strict threshold for statistical proof, likely because of the modest sample size. Still, exploratory analyses hinted that the expert label and favorable attitudes tended to work together mainly for the uncertain group, who seemed more open to using cues like expertise and AI feedback, while misinformed participants were less responsive.

Designing better AI support for health decisions

To a lay reader, the main takeaway is that AI and expert labels are not magic fixes for online health confusion. In this study, generative AI served as a helpful partner, but whether people actually learned more depended on their motivation and how firmly their beliefs were already set. Expert signals improved factual understanding only for those inclined to engage, and people who were confidently wrong were harder to reach than those who simply did not know. The authors argue that future AI tools and health communication should adapt to these differences—encouraging deeper reflection, making expertise more visible, and tailoring responses to whether a person is uncertain or misinformed—so that AI supports truly informed, not just more influenced, health choices.

Citation: Oh, J., Montag, C., Kohne, J. et al. Conditional effects of source expertise and pre-existing attitudes on objective knowledge in AI-assisted health information verification. Sci Rep 16, 13291 (2026). https://doi.org/10.1038/s41598-026-43698-z

Keywords: generative AI, health misinformation, source credibility, gluten-free diet, online health information