In a previous PEDro blog, physiotherapists were encouraged to stop using null hypothesis statistical tests in clinical research. Such tests involve the reporting of p-values and the interpretation of results as statistically significant or non-significant. There are many problems with p-values and claims of statistical significance, as discussed in detail in a Research Note published in Journal of Physiotherapy, which was the topic of the previous PEDro blog. That blog foreshadowed that physiotherapy journal editors would be releasing guidance on this issue. That time has now come.
Twelve leading physiotherapy journals are co-publishing an editorial in early 2022 that outlines the expectations of their editorial board members regarding statistical inference. The take-home message is that editorial board members of those 12 journals will expect manuscripts to use estimation methods instead of null hypothesis statistical tests. Estimation methods do not use p-values and statements about statistical (non-)significance. Instead, estimates are reported with confidence intervals.
Confidence intervals can be reported around numerous types of estimates. For example, a randomised controlled trial might report the mean between-group difference as the trial’s estimate of the effect of the experimental intervention in people with a particular disease. An observational study might report a proportion as an estimate of the prevalence of some characteristic in people with a particular clinical condition. Statistics such as these should be thought of as estimates because they are calculated from the study’s participants, who represent only a fraction of the entire clinical population of interest; therefore, the estimate is unlikely to be exactly equal to the true value in the wider clinical population.
Each time a confidence interval is reported, in one sense it indicates fundamentally the same thing regardless of which statistic is being reported; it indicates the range of values around the main estimate where the true effect probably lies. For example, a randomised controlled trial in people with ataxia might estimate that the experimental intervention improves gait speed by 7 m/min (because the mean between-group difference is 7 m/min). When this estimate is reported with a confidence interval of 5 to 9, it reminds us that there is some uncertainty in the estimate of 7 m/min but it reassures us that the true average effect of the intervention is probably somewhere between 5 and 9 m/min. A person with ataxia and their physiotherapist can jointly decide whether this estimate is precise enough and beneficial enough to make it worth undertaking the intervention. However, the authors reporting the trial should attempt to interpret whether the intervention would generally be considered worthwhile by people with ataxia. Similarly, an observational study reporting that 15% of 100 competitive breaststroke swimmers have patellofemoral arthritis should report a confidence interval around the prevalence estimate; in this case, the confidence interval would be 9 to 23. The authors should again interpret what clinical and research implications arise from this evidence that the true prevalence lies somewhere between 9 and 23% in that sporting population.
The editorial provides a list of resources to help authors in making this transition from p-values and statistical significance to estimation methods. The resources include articles that explain confidence intervals from first principles, articles that explain how to interpret confidence intervals, and advice about how to calculate confidence intervals from your own data and from published summary data.
The editors acknowledge that it will take time to make this transition, so editors will give authors the opportunity to revise manuscripts to incorporate estimation methods if the manuscript seems otherwise potentially viable for publication.
The joint editorial was initiated by the International Society of Physiotherapy Journal Editors but the move to estimation methods is not just happening in the physiotherapy profession. Leading statisticians have called upon editors to abandon the concept of statistical significance and editors are responding in research fields from neuroscience to nursing and from pharmacy to psychology.
The estimation train is leaving the station so make sure you’re on it. Read the editorial, which is freely available in full text via the link below, to ensure you understand what these physiotherapy journal editors expect in the analysis of research data.
