Research

Internal consistency reliability is a poor predictor of responsiveness

Milo A Puhan¹ , Dianne Bryant² , Gordon H Guyatt^2,3 , Diane Heels-Ansdell² and Holger J Schünemann²

¹  Horten Centre, University Hospital, Postfach Nord, 8091 Zurich, Switzerland

²  Department of Clinical Epidemiology and Biostatistics, McMaster University, 1200 Main St. W., Health Sciences Centre, Room 2C12, Hamilton, Ontario, L8N 3Z5 Canada

³  Department of Medicine, McMaster University, Health Sciences Centre, Hamilton, Ontario, L8N 3Z5 Canada

author email corresponding author email

Health and Quality of Life Outcomes 2005, 3:33doi:10.1186/1477-7525-3-33

Published:	9 May 2005

Abstract

Background

Whether responsiveness represents a measurement property of health-related quality of life (HRQL) instruments that is distinct from reliability and validity is an issue of debate. We addressed the claims of a recent study, which suggested that investigators could rely on internal consistency to reflect instrument responsiveness.

Methods

516 patients with chronic obstructive pulmonary disease or knee injury participating in four longitudinal studies completed generic and disease-specific HRQL questionnaires before and after an intervention that impacted on HRQL. We used Pearson correlation coefficients and linear regression to assess the relationship between internal consistency reliability (expressed as Cronbach's alpha), instrument type (generic and disease-specific) and responsiveness (expressed as the standardised response mean, SRM).

Results

Mean Cronbach's alpha was 0.83 (SD 0.08) and mean SRM was 0.59 (SD 0.33). The correlation between Cronbach's alpha and SRMs was 0.10 (95% CI -0.12 to 0.32) across all studies. Cronbach's alpha alone did not explain variability in SRMs (p = 0.59, r² = 0.01) whereas the type of instrument was a strong predictor of the SRM (p = 0.012, r² = 0.37). In multivariable models applied to individual studies Cronbach's alpha consistently failed to predict SRMs (regression coefficients between -0.45 and 1.58, p-values between 0.15 and 0.98) whereas the type of instrument did predict SRMs (regression coefficients between -0.25 to -0.59, p-values between <0.01 and 0.05).

Conclusion

Investigators must look to data other than internal consistency reliability to select a responsive instrument for use as an outcome in clinical trials.