Earlier this year a study assessed the accuracy of abstracts of articles reporting the results of randomised controlled trials evaluating physiotherapy interventions for low back pain. The main findings were that abstracts were often incomplete, contained spin, and were inconsistent with the full-text of the article. The authors concluded that physiotherapists should not rely on abstracts when using trials to guide practice and that journals could improve the accuracy of abstracts by increasing the word count and providing education. In response to these data, the Archives of Physical Medicine and Rehabilitation journal has revised some of its editorial policies. The changes include increasing the word limit of abstracts to 300 words for original research and systematic review articles and enhancing education efforts for authors, reviewers and editors in order to minimise spin and assure that abstracts fully reflect the findings – and limitations – reported in the manuscripts.
A recent study has extended this evaluation of abstracts to articles reporting the results of systematic reviews. The aim of this study was to determine if abstracts of articles reporting the results of systematic reviews evaluating physiotherapy interventions for low back pain contain spin and if abstracts consistently represent the full-text of the article. Spin was defined as an “overstated interpretation of study results.”
Systematic reviews indexed in PEDro in January 2018 were included in the analysis if they: (a) evaluated a physiotherapy intervention for the treatment of low back pain; (b) were published in 2015-2017; and, (c) were written in English, Spanish, or Portuguese. Spin was assessed using a 7-item checklist. Consistency between the abstract and full-text was determined by completing the spin checklist separately for the abstract and the full-text then calculating agreement between the two sections using Kappa coefficients. The methodological quality of the included reviews was measured using the AMSTAR-2 (A MeaSurement Tool to Assess systematic Reviews).
Sixty-six systematic reviews were included in the analyses. The methodological quality of the included reviews ranged from ‘critically low’ to ‘high.’ There was some form of spin in 80% of abstracts. Selective reporting of harm outcomes for the experimental intervention was the most common type of spin (73% of abstracts), and the title claiming a beneficial effect that was not supported by the findings was the least common (0% of abstracts). One item of the 7-item checklist (safety based on non-statistically significant results) had ‘substantial’ agreement between the abstract and full-text (Kappa = 0.62). All other items had ‘fair’ to ‘moderate’ agreement (Kappa = 0.27-0.55).
Interestingly, there were stark differences in quality, spin and agreement between the abstract and full-text when Cochrane reviews were compared to non-Cochrane reviews. The quality of Cochrane reviews ranged from ‘moderate’ to ‘high,’ while the quality of non-Cochrane reviews ranged from ‘critically low’ to ‘low.’ Compared to non-Cochrane reviews, the abstracts of Cochrane reviews contained less spin (6/57 non-Cochrane reviews and 7/9 Cochrane reviews were free from spin) and had higher consistency (Kappa = 0.27-0.60 for non-Cochrane reviews, 0.78-1.00 for Cochrane reviews). An incidental finding was that four of the nine Cochrane reviews had also been republished in other journals. Compared to the original Cochrane versions of the reviews, the republications had lower AMSTAR-2 ratings and contained more spin in the abstract.
These results have implications for journals publishing the results of physiotherapy research, authors seeking to get their research published, and physiotherapists using research articles to guide practice. Editorial processes and peer review could be enhanced to ensure that abstracts do not contain spin. One simple strategy could be for journals mandate the use of reporting checklists like the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for abstracts checklist. Authors should write the abstract only when the manuscript is completed, paying due attention to consistency and the omission of spin. Physiotherapists need to read beyond the abstract when using systematic reviews to guide practice and seek out Cochrane reviews, when available, to answer their clinical questions.
