Effect size is significantly more important than statistical significance

# · 🔥 375 · 💬 159 · 8 days ago · www.argmin.net · stochastician
Though the sample size looked enormous, the effective number of samples was only 600 because the treatment was applied to individual villages. The study reports that 0.76% of the people in the control villages were symptomatic and seropositive whereas 0.69% of the people in the treatment villages were symptomatic and seropositive. Let's say one treatment-control pair consists of two villages with 10,000 people each and pair two consists of villages with 6,000 people. In the larger treatment village, there is an outbreak with 136 cases, but in the smaller treatment village there are no cases. In the control villages, the larger village observes 75 cases and the smaller 46 cases. Based on the captions on the tables, it appears that they modeled the rate of symptomatic seroprevalence in each village as a normal distribution whose mean is a function of the village cluster and some other covariates. I think a milder version of Rutherford's aphorism should guide scientific investigation: If the effect size is so small that we need sophisticated statistics, maybe that means the effect isn't real.
Effect size is significantly more important than statistical significance



Archive | Send Feedback | WebAssembly Version (beta)