1477-7525-5-46 1477-7525 Research <p>Injection Drug User Quality of Life Scale (IDUQOL): Findings from a content validation study</p> Hubley M Anita anita.hubley@ubc.ca Palepu Anita anita@hivnet.ubc.ca

Measurement, Evaluation, and Research Methodology, Department of Educational and Counselling Psychology, and Special Education, University of British Columbia, Vancouver, BC, Canada

Division of Internal Medicine, Department of Medicine, University of British Columbia, Vancouver, BC, Canada

Department of Medicine, St. Paul's Hospital, Vancouver, BC, Canada

 Health and Quality of Life Outcomes 1477-7525 2007 5 1 46 http://www.hqlo.com/content/5/1/46 17663783 10.1186/1477-7525-5-46 13 3 2007 30 7 2007 30 7 2007 2007 Hubley and Palepu; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Quality of life studies among injection drug users have primarily focused on health-related measures. The chaotic life-style of many injection drug users (IDUs), however, extends far beyond their health, and impacts upon social relationships, employment opportunities, housing, and day to day survival. Most current quality of life instruments do not capture the realities of people living with addictions. The Injection Drug Users' Quality of Life Scale (IDUQOL) was developed to reflect the life areas of relevance to IDUs. The present study examined the content validity of the IDUQOL using judgmental methods based on subject matter experts' (SMEs) ratings of various elements of this measure (e.g., appropriateness of life areas or items, names and descriptions of life areas, instructions for administration and scoring).

Methods

Six SMEs were provided with a copy of the IDUQOL and its administration and scoring manual and a detailed content validation questionnaire. Two commonly used judgmental measures of inter-rater agreement, the Content Validity Index (CVI) and the Average Deviation Mean Index (ADM), were used to evaluate SMEs' agreement on ratings of IDUQOL elements.

Results

A total of 75 elements of the IDUQOL were examined. The CVI results showed that all elements were endorsed by the required number of SMEs or more. The ADM results showed that acceptable agreement (i.e., practical significance) was obtained for all elements but statistically significant agreement was missed for nine elements. For these elements, SMEs' feedback was examined for ways to improve the elements. Open-ended feedback also provided suggestions for other revisions to the IDUQOL.

Conclusion

The results of the study provided strong evidence in support of the content validity of the IDUQOL and direction for the revision of some IDUQOL elements.

Background

In the health and medical fields, quality of life (QoL) is widely used to evaluate social and clinical interventions, treatment side effects, and disease impact over time 12. Most of these QoL instruments tend to focus on health-related quality of life (HRQOL) or the functional effects of respondents' perceived mental and physical health 3. Gill and Feinstein 3, however, defined QoL as a reflection of respondents' perceptions and reactions to not only their mental and physical health, but also to non-health related aspects of their lives (e.g., family, friends, work). Thus, measurement of QoL needs to encompass more than just the health-related aspects of respondents' lives.

Nearly all studies of QoL in injection drugs users (IDUs) use measures of HRQOL 45678. Commonly used measures of HRQOL with this population include the Opiate Treatment Index 9, Nottingham Health Profile 1011, the Berlin Quality of Life Profile 12 and the MOS surveys (including the SF-36 and the SF-12) 13141516. Many studies have shown that IDUs experience significantly lower HRQOL relative to the general population 1617181920 but, as noted by Fernández Miranda 21, remarkably little published research has examined QoL as an outcome variable in the treatment of drug addiction 2223.

Although previous research with IDUs and related populations (e.g., illicit drug users, HIV/AIDS) has considered the effects of non-health related aspects of respondents' lives on their HRQOL 47152425262728 or even on the initiation or maintenance of drug use 29303132333435, rarely has published research with IDUs used a broadly-defined QoL measure (i.e., one that captures various social, psychological, physical, geographic, and occupational domains of QoL). Two exceptions would be Wasserman and colleagues 36, who examined the psychometric properties of Lehman's 37 Quality of Life Interview – Brief Version with IDUs, and Dunaj and Kovác 38, who compared convicted drug addicts and controls on broadly-defined QoL using the WHOQOL-BREF 39 and ComQol-A5 40. Dunaj and Kovác reported that addicts scored significantly lower than controls in their subjective ratings of areas such as health, emotional well-being, safety, and social standing.

Although broader measures of QoL are beginning to be used with IDUs 3638 and are certainly an improvement over the use of strictly HRQOL measures, measures developed specifically for the IDU population and using a context sensitive approach that considers the many life areas deemed by IDUs as critical to their QoL, are still needed. QoL, as defined by the World Health Organization Quality of Life (WHO-QOL) group, refers to "an individual's perceptions of their position in life in the context of the culture and value systems in which they live, and in relation to their goals, expectations, standards, and concerns" (pp. 1–2) 41. The item content and methods of administration for most available QoL instruments do not measure the QoL of drug users in a culturally-sensitive fashion 42. IDUs live in a distinct environment characterized by a high prevalence of infectious disease, crime, violence, and lack of stable housing. Many IDUs cannot depend on basic necessities and experience considerable instability in many aspects of their lives.

A recently developed broadly-defined QoL measure, the Injection Drug User Quality of Life (IDUQOL) scale, was designed to capture the health and non-health related aspects of IDUs' lives that would be important components of their quality of life, particularly given their individual circumstances and environment 4344. This measure has also been adapted for use in Spanish with injection and non-injection drug users 45. To use an instrument with confidence, it is important that there be evidence of validity – that is, the meaningfulness, usefulness, and appropriateness of an instrument for a given population in a given context 464748. Previous research has examined the factor structure, internal consistency, and test-retest reliability of scores from the IDUQOL as well as the criterion-related, convergent, and discriminant validity of inferences made from the measure 44. Content validity, a critical step in the test development and validation process 495051, refers to the degree to which elements of an assessment tool are representative of the construct of interest and appropriate for a given population 52. Importantly, the elements of interest in a content validation study are not just the content or items of the measure, but all elements of the instrument including the instructions, response format, and scoring procedures 53.

The purpose of the present study was to examine the content validity of the IDUQOL using judgmental methods based on subject matter experts' (SMEs) ratings of the IDUQOL title, items, instructions, response format, scoring procedures, and record form.

Methods

Participants

The sample consisted of a panel of six subject matter experts (SMEs; 50% male), all of whom were researchers working in the area of drug use in the United States or Canada with an average of 10 years experience in the field. Two SMEs were epidemiologists and four SMEs were physicians who also provided addiction and medical care to drug users in their clinical practice. As noted by others 54, there is no set number of SMEs required for content validation studies. Typically, somewhere between three to ten experts is recommended, although a minimum of five SMEs is recommended to control for chance agreement; furthermore, the larger the number of experts, the greater the confidence in the ratings and the easier it is to detect rater outliers 5355.

Measures

The subject matter experts were provided with a copy of the IDUQOL and its administration and scoring manual and a detailed content validation questionnaire. Ethics approval for this study was obtained from the University of British Columbia and Providence Health Care Research Ethics Boards.

Injection Drug User Quality of Life Scale (IDUQOL)

The original IDUQOL, which includes both health and non-health related aspects of QoL 3 and is based on the WHO-QOL group definition of QoL 41, consisted of 20 life areas. Several of these areas (e.g., Drugs, Drug Treatment, Harm Reduction and Neighbourhood Safety) were included in the measure precisely because of their particular relevance to the social and physical reality of IDUs as confirmed by focus groups during the development phase 43. Each IDUQOL life area is represented on a 4 by 4 inch card, with the name of the area printed on the front along with a simple picture. A description of the life area is presented on the back (see Table 1 for a list of all 20 life areas and descriptions). Graphic representation of the life areas is intended to make the instrument more accessible to individuals who have low literacy skills or do not speak English as a first language. When administering the IDUQOL, the interviewer starts by showing the respondent each of the 20 life area cards and describes the area. The participant selects those areas that he/she deems important to his/her quality of life and any remaining cards are set aside. The cards representing important areas are laid out and the participant is given three poker chips for each card. The total number of chips can, therefore, range from 0 (no life areas are important) to 60 (all 20 life areas are important). The participant then distributes the chips across the cards to indicate the level of importance of each life area, with more chips indicating greater importance. Next, the participant provides a satisfaction rating for each area, using a 6-point Likert-type scale anchored by 1 (very dissatisfied) and 6 (very satisfied) and illustrated with six stylised frowning and smiling faces.

<p>Table 1</p>

Injection Drug User Quality of Life (IDUQOL) life area names and descriptions

Life Area

Description

Being Useful

e.g., volunteering, employment, participating in the community, helping others

Community Resources

e.g., food bank, soup kitchen, shelters, outreach programs, social service agencies

Drugs

drug use – e.g., alcohol, heroin, cocaine, crack

Drug Treatment

e.g., detox, recovery house, residential treatment, methadone, abstinence

Education

e.g., formal schooling, literacy programs

Family

e.g., parents, children, siblings, foster families (not friends)

Feeling Good about Yourself

e.g., self-esteem, self-worth

Friends

anyone you consider a friend (but not family)

Harm Reduction

access to, and experience with: e.g., methadone treatment, needle exchange, safe injection programs, prescription heroin

Health

mental and physical health, including HIV, AIDS, Hepatitis C, disability, schizophrenia

Health Care

access to, and experience with: physicians, nurses, hospitals, clinics, ER

Housing

e.g., owning, renting, house, apartment, hotel room, shelters, homeless

Independence and Free Choice

e.g., making your own decisions, autonomy, being able to do things on your own, having individual rights

Leisure Activities

e.g., music, sports, movies, books, parties

Money

e.g., income, welfare, cash flow, meeting your needs

Neighborhood Safety

e.g., crime, violence, police harassment

Partner(s)

e.g., spouse, common-law partner, same-sex partner, girlfriend or boyfriend (not casual partners)

Sex

e.g., sexual intimacy, quantity or quality of sex, sex in exchange for money or drugs, sexual abuse

Spirituality

e.g., religion, faith, belief in a higher being or spiritual world (or not)

Transportation

e.g., car, taxi, public transportation, getting to places you need to go

When scoring the IDUQOL, the importance rating (number of chips) of each area is divided by the total number of chips used by that participant and then multiplied by the satisfaction rating for that area. This produces an area score. Finally, all area scores are summed to obtain an overall quality of life score ranging from 1 (very dissatisfied) to 6 (very satisfied).

IDUQOL content validation questionnaire

The questionnaire was divided into seven sections covering the clarity of the instrument title, ease of administration procedure instructions, clarity of the names and descriptions of the 20 IDUQOL life areas, whether each of the 20 IDUQOL life areas should be included in the measure (including whether any life areas need to be added, revised, or deleted), ease of the response formats used for each of importance and satisfaction ratings, clarity of scoring procedure instructions, and the ease of use of the record form. Experts were also given the opportunity to provide open-ended commentary in each section. As recommended by Lynn 55, a four-point Likert type scale was used in most cases. For questions involving clarity, the following four response options were used: 0 = not at all clear, 1 = somewhat clear, 2 = mostly clear, 3 = very clear. For questions involving ease, the following four response options were used: 0 = not at all easy to follow/use, 1 = somewhat easy to follow/use, 2 = mostly easy to follow/use, 3 = very easy to follow/use. For questions involving inclusion of items, the following three response options were used: 0 = no, 1 = unsure, 2 = yes. Two questions asked about how helpful the provided examples were in the manual; for these, the following four response options were used: 0 = not at all helpful, 1 = somewhat helpful, 2 = mostly helpful, 3 = very helpful.

Procedures

The six SMEs were identified through the second author's professional contacts with nationally and internationally recognized experts in the area of substance abuse epidemiology and treatment. They were sent a letter of invitation and agreed to take part in the study. None of the SMEs were associated with the development of the IDUQOL. The SMEs were mailed a copy of the IDUQOL (which included the 20 life area cards, poker chips, satisfaction rating card, and record form), the administration and scoring manual, and the IDUQOL content validation questionnaire. As suggested by Grant and Davis 56, SMEs were provided with the conceptual basis for the IDUQOL via the brief introduction in the manual in which the definition of QoL underlying this measure, the target population, and how the measure is intended to be used was provided. The SMEs completed the content validation questionnaire at their leisure and independently of one another. All SMEs returned usable questionnaires.

Two commonly used judgmental measures of inter-rater agreement, the Content Validity Index (CVI) 5557 and the Average Deviation Mean Index (ADM) 585960, were used to evaluate SMEs' agreement on ratings of the various IDUQOL elements. The two measures provide very different types of information, however, and should be viewed as complementary. Generally, the CVI indicates the proportion of SMEs that endorse an element as content valid whereas the ADM indicates the degree of disagreement among SMEs in the response option selected regardless of whether they, as a group, endorsed an element or not. Thus, one should first examine the CVI values to determine whether the SMEs endorsed an item or not and then consider the level of agreement among the SMEs by examining the ADM.

The CVI can be computed at the individual item level (I-CVI) and at the level of the overall scale or subscale (S-CVI). I-CVI is computed as the proportion of SMEs that endorse an item. Following standard procedures for four response options 4955, ratings of 2 or 3 were combined and treated as endorsements by SMEs whereas ratings of 0 or 1 were combined and treated as non-endorsements in the present study. When three response options were used, a rating of 2 was treated as an endorsement by SMEs whereas ratings of 0 or 1 were combined and treated as non-endorsements. A minimum of five out of the six SMEs (I-CVI ≥ .83) had to endorse an item to achieve significant evidence (α = .05) of content validity for any given item on the IDUQOL content validity questionnaire and to provide confidence that agreement was not occurring by chance alone 55. Elements that were not endorsed by a minimum of five SMEs were examined further to determine if appropriate revisions could be made.

The S-CVI may be defined and computed a number of different ways, but Polit and Beck 49 recommend using the average proportion of items endorsed by the SMEs (what they refer to as S-CVI/Ave) and computing this as the average of the I-CVI values. This is the approach that will be used in the present study in conjunction with Lynn's 55 description of S-CVI as "the proportion of total items judged content valid" (p. 384). For S-CVI/Ave, the minimum acceptable value is recommended to be .90 49.

The ADM Index measures dispersion of ratings about the mean rating; thus, it is actually a measure of disagreement so lower values indicate higher levels of agreement among SMEs. An advantage of the ADM Index is that it provides a measure of dispersion that is directly interpretable in terms of the original rating scale units. The general cut-off for determining acceptable ADM values is based on c/6, with c referring to the number of response options 59. Thus, using this guideline for practical significance, acceptable ADM values are .50 or less for ratings with three response options and .69 or less for ratings with four response options. Critical values that can be used to evaluate whether an obtained ADM could have been achieved by chance can also be computed. Critical values for ADM at the 5% level of significance, taking into account the number of SMEs and the number of response options in the present study, would be .28 or less for three response options and .44 or less for four response options 58. ADM values that are equal to or below these critical values are unlikely to have been obtained by chance. Elements for which ADM values were neither practically nor statistically significant or were only practically significant were examined further to determine if appropriate revisions could be made.

Results

Content validity evidence for IDUQOL life areas

Table 2 presents the I-CVI and ADM results for each of the 20 IDUQOL life areas indicating whether (a) each life area was appropriate for a QoL measure for IDUs, (b) the name of the life area was clear, and (c) the description of the life area was clear. I-CVI results showed that all individual life areas, including the name used and the description provided, were endorsed by a minimum of five SMEs.

<p>Table 2</p>

Item level CVI (I-CVI) and ADM index values for each of the 20 IDUQOL life areas

Appropriate?a

Nameb

Descriptionb

Life Area

I-CVI

ADM

I-CVI

ADM

I-CVI

ADM

Being Useful

1.00

.00

1.00

.44

0.83

.67

Community Resources

1.00

.00

1.00

.28

1.00

.28

Drugs

1.00

.00

0.83

.55

0.83

.67

Drug Treatment

0.83

.28

1.00

.00

1.00

.28

Education

0.83

.28

1.00

.00

1.00

.28

Family

1.00

.00

1.00

.00

1.00

.44

Feeling Good about Yourself

0.83

.28

1.00

.28

1.00

.50

Friends

1.00

.00

1.00

.00

1.00

.28

Harm Reduction

1.00

.00

1.00

.00

0.83

.33

Health

1.00

.00

1.00

.00

1.00

.28

Health Care

1.00

.00

1.00

.00

1.00

.28

Housing

1.00

.00

1.00

.00

1.00

.00

Independence and Free Choice

0.83

.28

1.00

.50

1.00

.00

Leisure Activities

1.00

.00

1.00

.00

1.00

.00

Money

1.00

.00

1.00

.00

1.00

.28

Neighborhood Safety

1.00

.00

1.00

.00

1.00

.44

Partner(s)

1.00

.00

1.00

.28

1.00

.00

Sex

1.00

.00

1.00

.00

1.00

.44

Spirituality

1.00

.00

1.00

.00

1.00

.00

Transportation

1.00

.00

1.00

.00

1.00

.00

a Ratings were made on a 3-point scale. b Ratings were made on a 4-point scale.

    Note
. I-CVI: 1.00 = endorsement by all six subject matter experts (SMEs); 0.83 = endorsement by five of six SMEs. ADM Index: with a 3-point scale, acceptable values are .50 or less and statistically significant values are .28 or less; with a 4-point scale, acceptable values are .69 or less and statistically significant values are .44 or less. Values that are acceptable, but not statistically significant, are bolded.

The ADM results provide additional information about the extent to which the SMEs agreed on the exact rating (e.g., not at all clear, somewhat clear, mostly clear, very clear) for each life area. In all cases, acceptable agreement (i.e., practical significance) was obtained. In terms of the appropriateness of the life areas, each life area also showed statistically significant agreement among the SMEs. When considering whether the name of the life area was clear, there were only two cases (i.e., Drugs, Independence & Free Choice) in which agreement was acceptable, but not statistically significant – meaning that agreement could have occurred by chance. In the case of Drugs, five SMEs rated this name as 'very clear' whereas one SME thought it was only 'somewhat clear' because drugs could be confused with medications. Independence & Free Choice was rated as 'mostly clear' by half of the SMEs and 'very clear' by the other half of SMEs with the problem being different interpretations of the word "independence". When considering whether the description of the life area was clear, there were three cases (i.e., Being Useful, Drugs, Feeling Good about Yourself) in which agreement was acceptable, but not statistically significant. The description for Being Useful was rated as 'somewhat clear' by one SME, 'mostly clear' by one SME and 'very clear' by four SMEs. Most SME comments were focused on the visual depiction provided on the card and no suggestions for changes or additions to the description were offered. The description for Drugs (i.e., "drug use – e.g., alcohol, heroin, cocaine, crack") was rated as 'somewhat clear' by one SME, 'mostly clear' by two SMEs and 'very clear' by three SMEs. There were two concerns raised by SMEs. The first and most prominent concern was that we limited this life area to use of drugs; the second concern was that we only listed four drugs. The description for Feeling Good about Yourself (i.e., "e.g., self-esteem, self-worth") was rated as 'mostly clear' by half of the SMEs and 'very clear' by the other half of the SMEs. No suggestions were made for changes to the description.

SMEs were also asked if there were any life areas that they would recommend deleting or adding to the IDUQOL. No life areas were recommended for deletion from the IDUQOL. The following additional life areas were suggested: food, pets, personal safety, sense of future (e.g., hopefulness, aspirations), employment (as its own life area separate from Being Useful), and pain.

Table 3 shows that the S-CVI/Ave for the element groupings of Appropriateness, Name clarity, and Description clarity of the IDUQOL life areas ranged from .97 to .99, which exceeded the minimum value of .90 and is also strong evidence of content validity.

<p>Table 3</p>

Scale level CVI (S-CVI/Ave) for elements of the IDUQOL measure and manual

IDUQOL Element

S-CVI/Ave

Appropriateness of IDUQOL Life Areas (20 items)a

0.97

Clarity of IDUQOL Life Area Names (20 items)b

0.99

Clarity of IDUQOL Life Area Descriptions (20 items)b

0.98

Clarity of Title and Target Population (2 items)b

0.92

Ease of Administration Procedure (4 items)b

1.00

Ease of Response Formats (2 items)b

0.83

Ease of Scoring Procedure (3 items)b

0.94

Helpfulness of Provided Examples (3 items)b

1.00

Ease of the Record Form to Use (1 item)b

1.00

a Ratings were made on a 3-point scale. b Ratings were made on a 4-point scale.

    Note
. Ease of Record Form result is the same as the individual level findings; this element is included here for completeness. For S-CVI/Ave, the minimum acceptable value is .90. A scale level ADM Index could not be applied here as it requires scales comprised of essentially parallel items.

Content validity evidence for other IDUQOL elements

Table 3 also shows that the S-CVI/Ave for the element groupings of Clarity of Title and Target Population, Ease of Administration Procedure, Ease of Scoring Procedure, and Helpfulness of Provided Examples ranged from .92 to 1.00. Only Ease of Response Formats produced a S-CVI/Ave (.83) that was below the minimum recommended value of .90, although both items were endorsed by five out of six SMEs.

I-CVI and ADM results for other individual elements (e.g., title, administration instructions, scoring instructions, record form) of the IDUQOL measure and manual are presented in Table 4. I-CVI results showed that all individual elements were endorsed by a minimum of five SMEs. The ADM results show that acceptable agreement (i.e., practical significance) was obtained in all cases, although there were four cases in which agreement was not statistically significant (i.e., Clarity of Title, Response Format – Chips, Response Format – Smiley Faces, Scoring Procedure – Summed Score).

<p>Table 4</p>

Item level CVI (I-CVI) and ADM index values for IDUQOL measure and manual elements

IDUQOL Content Validity Questionnaire Itema

I-CVI

ADM Index

Clarity of Title

0.83

.67

Clarity of Intended Population

1.00

.00

Administration Procedure – Introduction

1.00

.28

Administration Procedure – Respondent Selects Life Areas

1.00

.44

Administration Procedure – Respondent Rates Importance

1.00

.00

Administration Procedure – Respondent Rates Satisfaction

1.00

.00

Response Format – Easy for Respondent to Use Chips

0.83

.67

Response Format – Easy for Respondent to Use Smiley Faces

0.83

.67

Scoring Procedure – Relative Importance Score

1.00

.44

Scoring Procedure – Importance × Satisfaction Score

1.00

.44

Scoring Procedure – Obtain Summed Score

0.83

.67

Example – Relative Importance Score

1.00

.00

Example – Importance × Satisfaction Score

1.00

.00

Example – Completed Sample Record Form

1.00

.00

Ease of the Record Form to Use

1.00

.28

a Ratings were made on a 4-point scale.

    Note
. I-CVI: 1.00 = endorsement by all six subject matter experts (SMEs); 0.83 = endorsement by five of six SMEs. ADM Index: with a 4-point scale, acceptable values are .69 or less and statistically significant values are .44 or less. Values that are acceptable, but not statistically significant, are bolded.

The name or title of the IDUQOL was rated as 'somewhat clear' by one SME, 'mostly clear' by one SME and 'very clear' by four SMEs, with no suggestions made for how to make the title clearer. The Response Format – Chips was rated as 'somewhat easy to use' by one SME, 'mostly easy to use' by one SME, and 'very easy to use' by four SMEs. Suggestions were made for simplifying the instructions given to IDUs about how to use the poker chips to indicate the importance of the different life areas. In addition, it was suggested that poker chips might act as a trigger for IDUs with gambling issues. The Response Format – Smiley Faces was rated as 'somewhat easy to use' by one SME and 'very easy to use' by five SMEs. It was suggested that we consider using an odd-numbered Likert-type scale (rather than our 6-point Likert-type scale) for the smiley faces that would permit a neutral response. The Scoring Procedure – Summed Score was rated as 'somewhat easy to follow' by one SME, 'mostly easy to follow' by one SME, and 'very easy to follow' by four SMEs. It was suggested that we clarify the headings on the record form so they would better match the terms used in the manual.

Suggested revisions to the IDUQOL based on SME feedback

Open-ended feedback and comments from the SMEs resulted in several other suggestions for changes to the IDUQOL measure, materials, and manual. These may be grouped into three points. First, suggestions were made to expand the descriptions for (a) Education, (b) Family, and (c) Sex. Second, it was pointed out that we needed to show greater diversity in our cards involving people – specifically Family and Friends. Third, we were advised to revise the cards depicting (a) Being Useful, and (b) Independence and Free Choice to make them clearer.

Discussion

Test development and validation are ongoing processes designed to ensure measures and the inferences made from them remain appropriate, relevant, and useful for the target population and context of use 47. The IDUQOL was developed as a measure of broadly defined subjective QoL that incorporates both health and non-health related aspects of IDUs' lives. Administration of the IDUQOL was designed to be sensitive to the diversity of literacy levels, English language skills, attention levels, and cognitive abilities of the target population. An important step in test development and validation is the evaluation of content validity. The purpose of the present study was to examine the content validity of various elements of the IDUQOL measure and manual using SMEs and two commonly used judgmental methods (i.e., CVI and ADM). The CVI indicates the proportion of SMEs that endorse an element as content valid whereas the ADM indicates the degree of disagreement among SMEs in the response option selected. Overall, the results of this study provide strong evidence for the content validity of the elements of the IDUQOL measure and manual. Specifically, the I-CVI results supported the content validity of each of the individual elements. These elements include the appropriateness, name, and description of each of the 20 life areas, clarity of the name of the measure, clarity of the target population, ease of each step of the administration procedure, ease of each response format (i.e., chips and smiley face scale), ease of each step of the scoring procedure, helpfulness of each of the provided example boxes in the manual, and ease of use of the IDUQOL record form.

The S-CVI/Ave results also supported the content validity of all of the grouped elements of the IDUQOL measure and manual (e.g., Appropriateness of Life Areas, Ease of Administration Procedure), with the exception of Ease of Response Formats. Two points are worth noting about the Ease of Response Formats case. First, when examined individually using I-CVI, each of the two items under Ease of Response Formats was endorsed by five of the six SMEs, supporting their content validity. In fact, the one SME who supposedly did not endorse either of these items actually circled both 'somewhat easy to use' and 'mostly easy to use' (responses that fell into the 'not endorsed' and 'endorsed' categories, respectively) in each case and indicated that the ease of each response format for the target population was an empirical question that should be piloted instead. Using a conservative approach, we treated this SME's response as a non-endorsement, although it could be argued to be more ambiguous. Second, it should be noted that when there are only two or three items making up a grouping of elements (as is the case for the two-item Ease of Response Formats grouping), the minimum acceptable level of .90 for S-CVI/Ave cannot be reached unless all but one item in the grouping achieves endorsement by all six of the SMEs. Taking each of these points into account, we would argue that the content validity of the response formats should not be discounted, but that care should be taken to ensure, through further study or pilot testing, that these response formats are appropriate to the group of IDUs with whom a researcher or practitioner wishes to use the IDUQOL. Based on interviews we conducted with the experienced staff who administered the IDUQOL to participants in another study 4461, that sample of IDUs did not have difficulty using either response format, although some respondents expressed the desire for a neutral response option on the Likert-type smiley face satisfaction scale.

In terms of the ADM results, practical significance was obtained for all 75 elements of the IDUQOL and manual, indicating a high level of agreement among the SMEs in their ratings. However, statistical significance was not reached for the following nine elements: (a) names for the Drugs and Independence & Free Choice life areas, (b) descriptions for the Being Useful, Drugs, and Feeling Good about Yourself life areas, and (c) clarity of title, ease of response format – chips, ease of response format – smiley faces, and ease of scoring procedure – summed score. Thus, for these elements, it is possible that SME agreement on the ratings may have occurred by chance alone.

Both the CVI and ADM results and the open-ended feedback from the SMEs were used to make seven main revisions to IDUQOL elements. First, the names for the Drugs and Independence & Free Choice life areas were changed to Drugs & Alcohol and Free Choice, respectively. We also decided to take one SME's suggestion to change the Education life area to Education & Training.

Second, descriptions for the life areas of Drugs & Alcohol, Education, Family, and Sex were expanded. The original and revised descriptions for each of these life areas are provided in Figure 1. The descriptions for some other life areas (e.g., Being Useful, Feeling Good about Yourself) remained unchanged because SMEs had only commented on the visual image used or had not provided any suggestions for changes.

<p>Figure 1</p>

Samples of original and revised IDUQOL card descriptions

Samples of original and revised IDUQOL card descriptions.

Third, we added Sense of Future as a life area to the IDUQOL with the following description: "e.g., hopefulness, aspirations, dreams, goals". The SMEs made several suggestions for life areas that could be added to the IDUQOL (i.e., food, pets, personal safety, sense of future, employment, pain). After considerable thought and given previous IDU focus group discussions about important life areas, we ultimately decided to only add Sense of Future as its own life area to the measure. However, we recognize that other researchers and practitioners may want to consider including these suggestions as additional life areas in their own work. Three points about our decision are worth noting: (a) following discussion, we decided that 'personal safety' was an implicit part of Neighborhood Safety, (b) we decided to incorporate 'pain' under our description of the Health life area, and (c) we resisted adding employment as its own life area because IDUs in a concurrent focus group study strongly opposed viewing employment as separate from, or more important than, other aspects of 'being useful in society'.

Fourth, we made revisions to the visual depictions for Friends, Family, Being Useful, and Free Choice. Based on SMEs' comments, we increased the diversity of people in the cards for Friends and Family (see Figure 2). For the Being Useful and Free Choice life areas, SMEs recommended using different images to make these concepts clearer to respondents. The old and new cards are shown in Figure 3.

<p>Figure 2</p>

Original and revised cards to improve diversity of people

Original and revised cards to improve diversity of people.

 <p>Figure 3</p>

Original and revised cards to improve poor graphics on cards

Original and revised cards to improve poor graphics on cards.

Fifth, we were particularly struck by one SME's suggestion that the poker chips used in the response format for importance ratings might act as a trigger for IDUs with gambling issues and so we changed these to unmarked chips. We also incorporated suggestions made by SMEs for simplifying the instructions given to IDUs in the manual about how to use the chips to indicate the importance of the different life areas.

Sixth, we changed the Likert-type smiley face scale used to rate satisfaction from a 6-point scale to a 7-point scale that would permit a neutral response. This was based not only on SMEs' suggestions but also on our own concurrent experiences in administering the IDUQOL to IDUs 4461. We found that it was particularly appropriate to have a neutral option available when the respondent was rating satisfaction with a life area that had not been rated as particularly important.

Seventh, we revised the headings used in the IDUQOL record form so they would better match the terms used in the manual and would make obtaining the summed score easier (see Figure 4.)

<p>Figure 4</p>

Original and revised IDUQOL record form headings

Original and revised IDUQOL record form headings.

Given the strong support provided by this study for the content validity of the IDUQOL and its manual, the revisions made to improve them further, and previous research supporting the validity of inferences made from the IDUQOL 4461, the IDUQOL is a viable instrument for assessing broad-based QoL in IDUs and potentially in non-injection drug users 45. The majority of published research in which QoL is examined with IDUs focuses on HRQOL. Future research is needed that examines the impact of drug use and various treatment options on QoL using broadly defined subjective QoL measures such as the IDUQOL. Future research on the IDUQOL needs to further examine its appropriateness and usefulness with non-injection drug users, its sensitivity to change, and its relationship with other broad-based QoL measures such as the Quality of Life Interview – Brief Version, which was originally developed for use with the mentally ill, or the Personal Wellbeing Index, which is the successor to the ComQol-A5 and was developed for use with the general population 62.

Conclusion

The results from the present study provided strong support for the content validity of the elements of the IDUQOL measure and manual. Further revisions were made based on the CVI and ADM results as well as the open-ended feedback from the SMEs. These revisions included (a) revised names for three life areas (now Free Choice, Drugs & Alcohol, and Education & Training), (b) an expanded description for the Drugs & Alcohol, Education & Training, Family, and Sex life areas, (c) the addition of Sense of Future as a life area, (d) revisions to the visual depictions for Family, Friends, Being Useful and Free Choice, (e) the use of unmarked chips rather than poker chips in the response format for importance ratings, (f) the inclusion of a neutral point on the Likert-type scale of smiley faces for rating satisfaction, and (g) revisions to the headings used in the IDUQOL record form to make obtaining the summed score clearer. These revisions to the IDUQOL resulted in an instrument that is even easier for researchers, practitioners, and program evaluators to use as a way of assessing and tracking changes in QoL over time or as a result of interventions in IDUs.

List of abbreviations

ADM Average Deviation Mean Index

CVI Content Validity Index

HRQOL health-related quality of life

IDUs injection drug users

IDUQOL injection drug user quality of life scale

QoL quality of life

SMEs subject matter experts

Competing interests

The author(s) declare that they have no competing interests.

Authors' contributions

AH obtained funding, designed the study, directed the statistical analyses, prepared the initial draft of the manuscript and conducted revisions. AP conceived of the study, obtained funding, coordinated data collection, and conducted revisions of the manuscript. Each author read and approved the final manuscript.

Acknowledgements

This research was supported by an operating grant from the Canadian Institutes of Health Research (CIHR) to Dr. Anita Palepu and Dr. Anita Hubley. Additional support was provided through a Canadian Institutes for Health Research New Investigator Award and a Michael Smith Foundation for Health Research Senior Scholar Award to Dr. Anita Palepu. We thank Dr. Bruno D. Zumbo and Lara Russell for their comments on an earlier draft of this manuscript.

 <p>Associations of clinical parameters with health-related quality of life in hospitalized persons with HIV disease</p> Globe DR Hays RD Cunningham WE AIDS Care 1999 17 71 86 10.1080/09540129948216 <p>Life quality vs the 'quality of life': Assumptions and underlying prospective quality of life instruments in health care planning</p> Koch T Social Science and Medicine 2000 51 419 427 10.1016/S0277-9536(99)00474-8 <p>A critical appraisal of the quality of quality-of-life measurements</p> Gill TM Feinstein AR Journal of the American Medical Association 1994 272 619 626 10.1001/jama.272.8.619 7726894 <p> The role of personality, coping style, and social support in health-related quality of life in HIV infection</p> Burgess AP Carretero M Elkington A Pasqual-Marsettin E Lobaccaro C Catalán J Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care & Rehabilitation 2000 9 4 423 437 <p>Differences in the quality of life of two groups of drug users</p> Rooney S Freyne A Kelly G O’Connor J Irish Journal of Psychological Medicine 2002 19 55 59 <p>Impact of psychiatric conditions on health-related quality of life in persons with HIV infection</p> Sherbourne CD Hays RD Fleishman JA Vitiello B Magruder KM Bing EG McCaffrey D Burnam A Longshore D Eggan F Bozzette SA Shapiro MF American Journal of Psychiatry 2000 157 248 254 10.1176/appi.ajp.157.2.248 10671395 <p>Psychological distress and quality of life in drug-using and non-drug-using HIV-infected women</p> te Vaarwerk MJE Gaal EA European Journal of Public Health 2001 11 109 115 10.1093/eurpub/11.1.109 11276564 <p>Adverse impact of cigarette smoking on dimensions of health-related quality of life in persons with HIV infection</p> Turner J Page-Shafer K Chin DP Osmond D Mossar M Markstein L Huitsing J Barnes S Clemente V Chesney M the Pulmonary Complications of HIV Infection Study Group AIDS Patient Care and STDs 2001 15 615 624 10.1089/108729101753354617 <p>Development and validation of a multidimensional instrument for assessing outcome of treatment among opiate users: Opiate Treatment Index</p> Darke S Hall W Wodak A Heather N Ward J British Journal of Addiction 1992 87 733 742 10.1111/j.1360-0443.1992.tb02719.x 1591524 <p>Use of the Nottingham Health Profile for measuring health status of patients in methadone maintenance treatment</p> Torrens M San L Martinez A Castillo C Domingo-Salvany A Alonso J Addiction 1997 92 707 716 10.1111/j.1360-0443.1997.tb02937.x 9246798 <p>Characteristics of heroin addicts entering methadone maintenance treatment: Quality of life and gender</p> Puigdollers E Domingo-Salvany A Brugal MT Torrens M Alvaros J Castillo C Magri N Martin S Vazquez JM Substance Use & Misuse 2004 39 1353 1368 10.1081/JA-120039392 15462234 <p>Gender differences in health-related quality of life on admission to a maintenance treatment program</p> Giacomuzzi SM Riemer Y Ertl M Kemmler G Rossler H Hinterhuber H Kurz M European Addiction Research 2005 11 69 75 10.1159/000083035 15785067 <p>Health status at entry to methadone maintenance treatment using the SF-36 health survey questionnaire</p> Ryan CF White JM Addiction 1996 91 39 45 10.1111/j.1360-0443.1996.tb03160.x 8822013 <p>The functioning and well being of persons who seek treatment for drug and alcohol use</p> Stein MD Mulvey KP Plough A Samet JH Journal of Substance Abuse 1998 10 75 84 10.1016/S0899-3289(99)80142-4 9720008 <p>Health related quality of life among both current and former injection drug users who are HIV-infected</p> Préau MP C. Spire B Sobel A Dellamonica P Moatti JP Carrieri MP Drug and Alcohol Dependence 2007 86 175 182 10.1016/j.drugalcdep.2006.06.012 <p>The impact of illicit drug use and harmful drinking on quality of life among injection drug users at high risk for hepatitis C infection</p> Costenbader EC Zule WA Coomes CM Drug and Alcohol Dependence 2007 Feb 21 [E pub ahead of print] <p>Reliability and validity of an HIV-specific health-related quality-of-life measure for use with injecting drug users</p> Carretero MD Burgess AP Soler P Soler M Catalan J AIDS 1996 10 1699 1705 8970691 <p>Health-related quality of life in active injecting drug users with and without chronic hepatitis C virus infection</p> Dalgard O Egeland A Skaug K Vilimas K Steen T Hepatology 2004 39 74 80 10.1002/hep.20014 14752825 <p>Health-related quality of life of patients with HIV: Impact of sociodemographic, clinical, and psychosocial factors</p> Perez IR Baño JR Lopez Ruz MA Jiminez AA Prados MC Liaño JP Rico RM Lima JT Pardal JLP Gomez ML Muñoz N Morales D Marcos M Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care, & Rehabilitation 2005 14 1301 1310 <p>Health-related quality of life of HIV-infected women: Evidence for the reliability, validity, and responsiveness of the Medical Outcomes Study Short-Form 20</p> Smith MY Feldman J Kelly P DeHovitz JA Chirgwin K Minkoff H Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care, & Rehabilitation 1996 5 1 47 55 <p>La calidad de vida en adicciones: Una medida de la efectividad de los tratamientos [Quality of life in addictions: A measure of treatment effectiveness]</p> Fernández Miranda JJ Anales de Psiquiatría 2003 19 377 384 <p>An overview of systematic reviews of the effectiveness of opiate maintenance therapies: Available evidence to inform clinical practice and research</p> Amato L Davoli M Perucci CA Ferri M Faggiano F Mattick RP Journal of Substance Abuse Treatment 2005 28 321 329 10.1016/j.jsat.2005.02.007 15925266 <p>Life activities and life quality of heroin addicts in and out of methadone treatment. </p> Reno RR Aiken LS International Journal of Addictions 1993 28 3 211 232 <p>Health-related quality of life of patients with HIV disease: Impact of hepatitis C coinfection</p> Fleming CA Christiansen D Nunes D Heeren T Thornton D Horsburgh Jr. CR Koziel MJ Graham C Craven DE Clinical Infectious Diseases 2004 38 572 578 10.1086/381263 14765352 <p>Quality of life among women living with HIV: The importance of violence, social support, and self care behaviors</p> Gielen AC McDonnell KA Wu AW O’Campo P Faden R Social Science & Medicine 2001 52 315 322 10.1016/S0277-9536(00)00135-0 <p>Abuse, HIV status, and health-related quality of life among a sample of HIV positive and HIV negative low income women</p> McDonnell KA Gielen AC O’Campo P Burke JG Quality of Life Research 2005 14 945 957 10.1007/s11136-004-3709-z 16041892 <p>Determinants of health-related quality of life of opiate users at entry to low-threshold methadone programs</p> Millson P Challacombe L Villeneuve PJ Strike CJ Fischer B Myers T Shore R Hopkins S European Addiction Research 2006 12 74 82 10.1159/000090426 16543742 <p>Health-related quality of life in veterans and nonveterans with HIV/AIDS</p> Mrus JM Leonard AC Yi MS Sherman SN Fultz SL Justice AC Tsevat J Journal of General Internal Medicine 2006 21 S39 47 10.1111/j.1525-1497.2006.00644.x 17083499 <p>Frequent childhood geographic relocation: Its impact on drug use initiation and the development of alcohol and other drug-related problems among adolescents and young adults</p> DeWit DJ Addictive Behaviors 1998 23 623 634 10.1016/S0306-4603(98)00023-9 9768299 <p>Effects of race, neighborhood, and social network on age at initiation of injection drug use</p> Fuller CM Borrell LN Latkin CA Galea S Ompad DC Strathdee SA Vlahov D American Journal of Public Health 2005 95 689 695 1449242 15798131 10.2105/AJPH.2003.02178 <p>The trouble with peace: The cease-fires and their impact on drug use among youth in Northern Ireland</p> Higgins K McElrath K Youth & Society 2000 32 29 59 10.1177/0044118X00032001003 <p>Injection and non-injection drug use related to social exclusion indicators in two Andalusian cities</p> March JC Oviedo-Joekes E Romero M Drugs: Education, Prevention & Policy 2005 12 437 447 10.1080/09687630500281998 <p>Variables psicosociales en la adherencia al tratemiento antiretroviral en pacientes adscritos a un programa de mantenimiento con metadona [Psychosocial variables in the antiretroviral treatment adherence in methadone maintenance patients]</p> Martín LL Hernández SO Carrobles JA Psicothema 2005 17 575 581 <p>Transitions to injecting drug use among noninjecting heroin users: Social network influence and individual susceptibility</p> Neaigus A Gyarmathy A Miller M Frajzyngier VM Friedman SR Des Jarlais DC Journal of Acquired Immune Deficiency Syndromes 2006 41 493 503 10.1097/01.qai.0000186391.49205.3b 16652059 <p>The impact of family context, deviant attitudes, and emotional distress on adolescent drug use: Longitudinal latent-variable analyses of mothers and their children</p> Newcomb MD Bentler PM Journal of Research in Personality 1988 22 154 176 10.1016/0092-6566(88)90012-8 <p>Psychometric evaluation of the Quality of Life Interview, Brief Version, in injection drug users</p> Wasserman DA Sorensen JL Delucchi KL Masson CL Hall SM Psychology of Addictive Behaviors 2006 20 316 321 10.1037/0893-164X.20.3.316 16938069 <p>Evaluating quality of life for persons with severe mental illness: Assessment toolkit</p> Lehman AF Cambridge, MA , Evaluation Center at Health Services Research Institute 1995 <p>Quality of life of convicted drug addicts; Preliminary report</p> Dunaj R Kovác D Studia Psychologica 2003 45 357 360 <p>Development of the World Health Organisation WHOQOL-BREF quality of life assessment</p> WHOQOL Group Psychological Medicine 1998 28 551 558 10.1017/S0033291798006667 9626712 <p>Comprehensive Quality of Life Scale-Adult: ComQol-A5</p> Cummins RA Melbourne, Australia , School of Psychology, Deakin University 5th 1997 <p>Validation of the United States’ version of the World Health Organization Quality of Life (WHOQOL) instrument</p> Bonomi AE Patrick DL Bushnell DM Martin M Journal of Clinical Epidemiology 2000 53 1 12 10.1016/S0895-4356(99)00123-7 10693897 <p>Substance Abuse: The Challenge of Assessment</p> Metzger DS O'Brien CP Quality of Life Assessments in Clinical Trials New York , Raven Press, Ltd. Spilker B 1990 237 246 <p>Towards more effective public health programming for injection drug users: Development and evaluation of the Injection Drug User Quality of Life Scale.</p> Brogly S Mercier C Bruneau J Palepu A Franco E Substance Use & Misuse 2003 38 965 992 10.1081/JA-120017619 12801151 <p>Injection Drug Users Quality of Life (IDUQOL) scale: A validation study</p> Hubley AM Russell L Palepu A Health and Quality of Life Outcomes 2005 3 43 http://www.hqlo.com/content/3/1/43 <p>Cross cultural adaptation of the Injection Drug User Quality of Life Scale (IDUQOL) in Spanish drug dependent population, with or without injectable consumption : Drug User Quality of Life Scale - Spanish (DUQOL - Spanish)</p> Morales-Manrique CC Valderrama-Zurian JC Castellano-Gomez M Aleixandre-Benavent R Palepu A Addictive Behaviors 2007 32 9 1913 1921 10.1016/j.addbeh.2006.12.006 17240076 <p>Standards for educational and psychological testing</p> APA AERA NCME Washington, DC , American Psychological Association, American Educational Research Association, National Council on Measurement in Education 1985 <p>A dialectic on validity: Where we have been and where we are going</p> Hubley AM Zumbo BD The Journal of General Psychology 1996 123 207 215 <p>Validity of test interpretation and use. Research Report No. 90-11</p> Messick S Educational Testing Service 1990 2112804 <p>The content validity index: Are you sure you know what’s being reported? Critique and recommendations</p> Polit DF Beck CT Research in Nursing & Health 2006 29 489 497 10.1002/nur.20147 16977646 <p>Objectifying content validity: Conducting a content validity study in social work</p> Rubio DM Berg-Weger M Tebb SS Lee ES Rauch S Social Work Research 2003 27 94 104 <p>The construct of content validity</p> Sireci SG Social Indicators Research 1998 45 83 117 10.1023/A:1006985528729 <p>Ensuring content validity: An illustration of the process</p> Beck CT Gable RK Journal of Nursing Measurement 2001 9 2 201 215 11696942 <p>Content validity in psychological assessment: A functional approach to concepts and methods</p> Haynes SN Richard DR Kubany ES Psychological Assessment 1995 7 238 247 10.1037/1040-3590.7.3.238 <p>Measuring community coalition effectiveness using the ICE© instrument</p> Cramer ME Atwood JR Stoner JA Public Health Nursing 2006 23 74 87 10.1111/j.0737-1209.2006.230111.x 16460424 <p>Determination and quantification of content validity</p> Lynn MR Nursing Research 1986 35 6 382 385 10.1097/00006199-198611000-00017 3640358 <p>Selection and use of content experts for instrument development</p> Grant J Davis L Research in Nursing and Health 1997 20 269 274 10.1002/(SICI)1098-240X(199706)20:3<269::AID-NUR9>3.0.CO;2-G <p>A quantitative approach to content validity</p> Lawshe CH Personnel Psychology 1975 28 563 575 10.1111/j.1744-6570.1975.tb01393.x <p>Accurate tests of statistical significance for r WG and average deviation interrater agreement indexes</p> Dunlap WP Burke MJ Smith-Crowe K Journal of Applied Psychology 2003 88 2 356 362 10.1037/0021-9010.88.2.356 12731720 <p>Estimating interrater agreement with the Average Deviation Index: A user's guide</p> Burke MJ Dunlap WP Organizational Research Methods 2002 5 2 159 172 10.1177/1094428102005002002 <p>On average deviation indices for estimating interrater agreement</p> Burke MJ Finkelstein LM Dusig MS Organizational Research Methods 1999 2 49 68 10.1177/109442819921004 <p>Does weighting capture what's important? Revisiting subjective importance weighting with a quality of life measure</p> Russell LB Hubley AM Palepu A Zumbo BD Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement 2006 75 141 167 <p>Personal Wellbeing Index</p> International Wellbeing Group Melbourne, Australia , Australian Centre on Quality of Life, Deakin University. 2006 http://www.deakin.edu.au/research/acqol/instruments/wellbeing_index.htm