You are here

Attributes of a Good Outcome Measure

Choosing the appropriate health status measurement instrument for a specific purpose is a difficult and time-consuming task. The choice depends on a number of factors such as the proposed use of the instrument, the concept to be measured, the readability of the questions, the requirements and costs associated with the use of the instrument, the burden on the participants, and, last but not least, the measurement properties of the instruments (validity and reliability).

Several authors have suggested standards for the development and evaluation of instruments to measure health status (1-4). One of the most elaborate lists was proposed by the Scientific Advisory Committee (SAC) of the Medical Outcomes Trust (1,2). The SAC defined a set of eight key attributes of instruments to measure health status and (health-related) quality of life, which are briefly presented below.

Attributes to be assessed

  • The concept to be measured needs to be defined properly and should match its intended use.
  • Reliability is the degree to which the instrument is free of random error, which means free from errors in measurement caused by chance factors that influence measurement.
  • Validity is the degree to which the instrument measures what it purports to measure.
  • Responsiveness is an instruments ability to detect change over time.
  • Interpretability is the degree to which one can assign easily understood meaning to an instrument's score.
  • Burden refers to the time, effort and other demands placed on those to whom the instrument is administered (respondent burden) or on those who administer the instrument (administrative burden)
  • Alternative means of administration include self report, interviewer-administered, computer assisted, etc. Often it is important to know whether these modes of administration are comparable.
  • Cultural and Language adaptations or translations

For each of these attributes the Scientific Advisory Committee (SAC) of the Medical Outcomes Trust has developed review criteria (or standards) that should be used as much as possible in order to ensure valid and reliable results. We strongly recommend that when selecting and using outcome instruments, practitioners and researchers review the criteria to assess these attributes carefully.

Some authors have included ratings of the various outcome instruments. For example, McDowell (5) has developed a ranking system based on availability of evidence of reliability and validity, where one star indicates no evidence to five stars, which indicate excellent evidence. After ample discussion, we have decided not to provide a ranking of instruments in the Database for several reasons.

  • There is no firm consensus on existing rating systems.
  • Selection of an instrument should to be based on a number of attributes (as discussed above).
  • There are a number of new instruments in the CAM field which may be very good but are still in the process of being evaluated. Therefore a rating is not possible and a poor rating would not reflect the true value of the instrument.

We will however, refer people to appropriate references on basic validity and reliability testing for each instrument. We will also identify if an instrument is fully developed or is still under development.

References

1 . Perrin, E. (1995). SAC Instrument Review Process. Medical Outcomes Trust Bulletin, September, IV. (with link to pdf of article) http://www.outcomes-trust.org/monitor/nov98/8.html

2 . Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002, 11:193–205. doi: 10.1023/A:1015291021312. [PubMed: 12074258]

3. Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chronic Dis. 1985;38:27–36. doi: 10.1016/0021-9681(85)90005-0. [PubMed: 3972947]

4. Lohr KN, Aaronson NK, Alonso J, Burnam MA, Patrick DL, Perrin EB, Roberts JS. Evaluating quality-of-life and health status instruments: development of scientific review criteria. Clinical Therapeutics. 1996;18:979–992. doi: 10.1016/S0149-2918(96)80054-3. [PubMed: 8930436]

5. McDowell, I. Measuring Health – A guide to rating scales and questionnaires. 3rd edition. New York: Oxford University Press, 2006.