An extended refutation text and funny videos reduce notorious p-value misconceptions

Why this study matters to everyday learners

Many students in the social sciences struggle with statistics, especially when it comes to understanding what a test result actually tells them. This study looks at one small but important piece of that puzzle: the p value, a number that is often treated as a magic stamp of success or failure for scientific findings. The researcher tested a short online lesson, boosted with funny videos, to see whether it can quickly clear up widespread wrong ideas about p values among psychology students.

Common mistakes about what numbers mean

The paper begins by showing that confusion about p values is not a rare problem or limited to beginners. Earlier work has found that almost all students, and many of their teachers, misread p values as the probability that a scientific idea is true or false. In this study, 98 percent of undergraduate psychology students made at least one such mistake before the lesson. Many firmly believed that a low p value tells you the chance that the research hypothesis is correct, which it does not. To get a clearer picture, the researcher did not just check whether answers were right or wrong, but also how sure students felt, which helped separate simple guessing from deeply held false beliefs.

A short online lesson designed to change minds

To tackle these misunderstandings, the author built a brief digital module that students could complete in about ten minutes. The core of the module was an extended refutation text, a carefully written explanation that first states the common wrong ideas, then explains why they are wrong, and finally offers a clearer way to think. The text went beyond a few correcting sentences. It walked students through how null hypothesis testing is usually used, what a p value actually describes, and why it does not give the probability that a hypothesis is true. It also briefly introduced alternative ways to look at data, such as focusing on effect sizes or approaches that directly talk about the probability of hypotheses.

Adding humor to ease fear of statistics

Because statistics is often linked with anxiety and frustration, the study tested whether a bit of humor could make the lesson more effective. Before reading either the refutation text or a neutral placeholder text, students watched one of two short video sequences. One sequence contained lighthearted clips like a bouncing dog and harmless mishaps, chosen to lift mood; the other was a neutral university information video. A pilot test confirmed that the funny clips were in fact seen as much more amusing. The idea was that a quick boost in mood could help students invest more effort and engage more openly with the material that followed.

What the experiment revealed

The study used a two by two design, combining funny versus neutral videos with either the refutation text or the neutral text. Students completed a questionnaire on p value statements before and after the lesson, always reporting both their answer and how certain they felt. From this, the researcher calculated a misconception score and a conceptual change score that captured how much each student’s understanding shifted. The extended refutation text had a large positive effect: students who read it showed much stronger movement away from confident wrong beliefs than those who read the neutral text. On top of that, seeing funny videos before the lesson slightly increased mental effort, gave an extra boost to the reduction of misconceptions, and led students to rate the learning module more favorably.

Takeaway for students and teachers

For lay readers, the central message is that short, targeted explanations can meaningfully improve how students understand a tricky statistical idea, especially when those explanations directly confront common myths and offer clearer alternatives. In this case, a ten minute online refresher helped students move away from treating a p value as a simple truth meter for hypotheses. A brief dose of humor before learning further increased engagement and learning gains. While this study focused on psychology undergraduates and one specific topic, it suggests that well crafted, myth busting texts, paired with mood lifting media, can be a practical tool for clarifying key ideas in statistics and perhaps other fields that many students find intimidating.

Citation: Hefter, M.H. An extended refutation text and funny videos reduce notorious p-value misconceptions. Sci Rep 16, 16017 (2026). https://doi.org/10.1038/s41598-026-53375-w

Keywords: p value misconceptions, statistics education, refutation text, humor and learning, psychology students