PEDro is currently working on a new database that will index studies and reviews that evaluate the accuracy of diagnostic tests used by physiotherapists. Called DiTA, this new database project is being led by Mark Kaizik, Rob Herbert and Mark Hancock.
An investigation of the measurement properties of quality assessment tools for diagnostic test accuracy studies was conducted to inform the development of DiTA. The main aims of the investigation were to determine the reliability, measurement error, internal consistency, convergent validity, and floor and ceiling effects of three tools commonly used to evaluate the quality of diagnostic test accuracy studies.
50 diagnostic test accuracy studies in the field of musculoskeletal, orthopaedic or sports physiotherapy that were published in English were randomly selected from DiTA. Three tools were evaluated: Quality Assessment of Diagnostic Accuracy Studies (QUADAS), Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) and Diagnostic Accuracy Quality Scale (DAQS). Two physiotherapists independently rated each study using each tool (the order of both the study and tool were randomised). Summary scores were calculated to facilitate the analyses. 13/14 QUADAS items, 5/7 QUADAS-2 domains, and 14/21 DAQS items had less than moderate inter-rater reliability (Kappa< =0.40). Inter-rater reliability for summary scores ranged from poor (Intraclass Correlation Coefficient 0.27, QUADAS) to moderate (0.45, DAQS). Standard error of measurement was 2.7 points was for the 0-28 point QUADAS tool, 1.8 for the 0-14 point QUADAS-2, and 3.6 for the 0-42 point DAQS. Internal consistency was acceptable (Cronbach’s alpha>0.70) for the QUADAS-2 tool only. Convergent validity was acceptable (Pearson’s correlation>0.70) for half of the analyses: QUADAS vs DAQS (both rater 1 and rater 2), and QUADAS-2 vs DAQS (rater 1). Floor or ceiling effects were not present in any tool. The study concludes that all three tools provide unreliable estimates of quality for studies evaluating the accuracy of diagnostic tests used by physiotherapists.
