On the "significance" of the p-value

Boix Campos, J.

Vivum Editorial Team

Submitted by J. Boix

Abstract

This article explains what p-values measure, why they become unreliable at scale, and how to correct for multiple testing. Interactive visualizations are essential to the scientific argument because they allow readers to directly manipulate the relationship between sample size, effect size, and p-values; revealing how the same threshold can be crossed by trivial effects given enough data. The resample-able multiple testing grid demonstrates the inevitability of false positives under repeated testing, while the dynamic correction method comparison shows how Bonferroni and Benjamini-Hochberg procedures make different trade-offs between sensitivity and specificity. Static figures cannot convey these parameter-dependent relationships or let readers explore the bias-variance tradeoff in real time, which is central to understanding when "statistical significance" diverges from scientific significance.

Original submitted snapshot ↗

Discussion