Email updates

Keep up to date with the latest news and content from HQLO and BioMed Central.

Open Access Research

An assessment of the construct validity of the ASCOT measure of social care-related quality of life with older people

Juliette N Malley12, Ann-Marie Towers1*, Ann P Netten1, John E Brazier3, Julien E Forder12 and Terry Flynn4

Author Affiliations

1 Personal Social Services Research Unit and Quality and Outcomes of Person-Centred Care Research Unit, University of Kent, Canterbury, Kent CT2 7NF, UK

2 Personal Social Services Research Unit and Quality and Outcomes of Person-Centred Care Research Unit, London School of Economics and Political Science, Houghton Street, London WC2A 2AE, UK

3 Health Economics and Decision Science, School of Health and Related Research, University of Sheffield, Regent Court, 30 Regent Street, Sheffield S1 4DA, UK

4 Centre for the Study of Choice (CenSoC), University of Technology Sydney, Sydney, P.O. Box 123, Broadway, NSW 2007, Australia

For all author emails, please log on.

Health and Quality of Life Outcomes 2012, 10:21  doi:10.1186/1477-7525-10-21

The electronic version of this article is the complete one and can be found online at: http://www.hqlo.com/content/10/1/21


Received:19 August 2011
Accepted:10 February 2012
Published:10 February 2012

© 2012 Malley et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

The adult social care outcomes toolkit (ASCOT) includes a preference-weighted measure of social care-related quality of life for use in economic evaluations. ASCOT has eight attributes: personal cleanliness and comfort, food and drink, control over daily life, personal safety, accommodation cleanliness and comfort, social participation and involvement, occupation and dignity. This paper aims to demonstrate the construct validity of the ASCOT attributes.

Methods

A survey of older people receiving publicly-funded home care services was conducted by face-to-face interview in several sites across England. Additional data on variables hypothesised to be related and unrelated to each of the attributes were also collected. Relationships between these variables and the attributes were analysed through chi-squared tests and analysis of variance, as appropriate, to test the construct validity of each attribute.

Results

301 people were interviewed and approximately 10% of responses were given by a proxy respondent. Results suggest that each attribute captured the extent to which respondents exercised choice in how their outcomes were met. There was also evidence for the validity of the control over daily life, occupation, personal cleanliness and comfort, personal safety, accommodation cleanliness and comfort, and social participation and involvement attributes. There was less evidence regarding the validity of the food and drink and dignity attributes, but this may be a consequence of problems finding good data against which to validate these attributes, as well as problems with the distribution of the food and drink item.

Conclusions

This study provides some evidence for the construct validity of the ASCOT attributes and therefore support for ASCOT's use in economic evaluation. It also demonstrated the feasibility of its use among older people, although the need for proxy respondents in some situations suggests that developing a version that is suitable for proxies would be a useful future direction for this work. Validation of the instrument on a sample of younger social care users would also be useful.

Keywords:
social care; quality of life; ASCOT; validity; long-term care

Background

The term social care is used frequently in the UK to describe a range of long-term care activities, including providing help with personal hygiene, dressing and feeding, as well as help with shopping, keeping active and socialising. Social care services include personal assistance, nursing and residential care homes, and day centres and are usually provided in response to needs arising from physical or sensory impairments, learning difficulties and mental health problems including dementia [1]. The increased cost of this support, associated with improvements in life expectancy among disabled people and in the general population, continues to be the subject of much analysis and debate [2-5]. If we are to target resources effectively we also need to be able to measure the outcomes and value of such services.

Measuring outcomes is never straightforward but is arguably particularly challenging in social care due to the nature of the support provided. Social care services tend to compensate people for the effect of their impairments on their quality of life (QoL) in accordance with local and national policies. Services also aim to do this in a way that is enabling and allows people to make choices as to how their needs are met. If we want to measure the value of social care services we need a measure that reflects the compensatory activity of social care, is sensitive to choice and captures what we have termed social care-related quality of life (SCRQoL), which reflects those aspects of QoL, or attributes, that are the focus of social care support. Finally to generate a single score for use in cost-effectiveness analyses it should be preference-weighted to reflect the relative importance of the SCRQoL states [6].

The ASCOT instrument was developed as a multi-attribute preference-weighted measure of SCRQoL. ASCOT has eight conceptually distinct attributes: Personal cleanliness and comfort, Food and drink, Control over daily life, Personal safety, Accommodation cleanliness and comfort, Social participation and involvement, Occupation and Dignity. Dignity differs from the other attributes since it reflects the impact of the care process on how people feel about themselves, rather than being an aspect of QoL that applies to all of us, whether or not we have care and support from others. There is one item per attribute and each attribute has four response options, reflecting four different outcome states. The top two states both reflect states where outcomes are fully realised but were designed to differ in the extent to which respondents have choice over how the outcome is realised, so that the best state reflects a person with choice and the second state one without choice [6]. To assess the distribution of responses to the ASCOT attributes, a survey was conducted which also afforded an opportunity to assess the validity of the measure.

Validity is an assessment of the extent to which the instrument measures what it is intended to represent, which for the ASCOT measure is the value of social care. A number of different types of validity have been identified, including content, construct, and criterion-related validity [7]. However, the psychometric approach used to assess validity needs some modification to make it applicable for determining the validity of preference-weighted measures, which capture the value of the outcome state not the state itself [8,9]. Brazier et al [9] identify three aspects of preference-weighted measures that require validation: the descriptive system, the valuations, and the empirical validity of the instrument. The valuations derived from the preference study have been examined elsewhere [6,10]. The focus of this paper is on the validity of the descriptive system, which refers to the choice of attributes and the specification of the items in the instrument. Since there is a lack of well-established measures for many of the ASCOT attributes, our approach to validation follows that taken by Coast et al [11], and focuses on the construct validity of each of the attributes: the extent to which each attribute has the expected relationship with other variables and concepts. We first describe the method before discussing the study results.

Methods

The data were collected to assess the distribution of responses to ASCOT and explore its validity. Ethics approval was obtained from the University of Kent Ethics Committee and research governance approval from each of the Local Authorities (LAs) that agreed to participate in the study. A sampling frame was generated from respondents to a national user experience survey (UES) of older people (aged over 65) using home care services who had indicated that they were happy to be approached to take part in further research [12].

Data were collected face-to-face through computer-aided personal interviews in people's homes during 2009 in ten geographically dispersed locations across England. The interviews gathered socio-demographic information and details about service receipt and informal support. To measure health we used the five-dimension Euroqol (EQ-5D) [13,14]; a range of activities of daily living (ADLs) and instrumental activities of daily living (IADLs) were included to measure disability; and the 12-question General Health Questionnaire (GHQ-12)[15-17] was included to capture psychological well-being. The control and autonomy subscale of the CASP-12 [18,19] was used to measure sense of control. To capture QoL a global self-reported seven-point measure of QoL was included. Four measures of aspects of the nature of the locality and environment were included: a five-point interviewer-rated cleanliness and tidiness of respondent's home, the type of area the person lived in (rural, urban and so on), a four-point self-rated design of home question [20] and a four-point question on the accessibility of local area. Measures of social contact and participation included: the UCLA 3-item loneliness scale [21], and measures on the frequency of meeting up with friends and relatives, the frequency of speaking to neighbours, the frequency of speaking to friends and relatives, involvement in organised formal and informal groups and activities in the last 12 months, and unpaid volunteering in last 12 months. Permission was gained to link the interview data to that from the UES, conducted roughly six months previously, which contained questions on perceptions of service quality.

We used hypothesis testing to test construct validity for each ASCOT attribute. Expected relationships with variables were based on evidence from the literature, where this was available, and the views of team members, who brought their expertise in this area, as well as detailed knowledge of the data from which ASCOT items were developed. Table 1 summarises the hypothesised associations for each attribute. Relationships were analysed between all the ASCOT attributes and each variable for transparency and counterfactual evidence.

Table 1. Variables included and expected associations

To assess relationships, we used chi-squared tests (for unordered or ordered categorical variables) or one-way analysis of variance (for continuous variables) in STATA v11. For chi-squared tests, one-sided probability exact tests were used when computationally feasible; where this was not possible, data were recoded to increase numbers in individual cells. Fisher's exact test produces a p-value indicating the probability that the two variables are independent of one another. Unlike chi-squared, there are no accompanying test statistics. Consequently, the p-values only are summarised in the tables and the nature and direction of the associations are reported in the text. Associations significant at the 1% level were taken to be strongly suggestive of a relationship between the attribute and the variable; and relationships significant at between the 1% and 10% level were taken to be weakly suggestive of a relationship between the attribute and the variable. We also considered the patterns of relationships as well as the p-value to assess the direction of the relationships rather than significance alone.

Results

In total, 566 contacts were attempted from a sample of 778, producing 301 (53%) complete interviews. Non-responders were categorised as refusals (n = 18, 3%), deceased (n = 4, 1%) and not contactable (n = 243, 43%). Table 2 shows the characteristics of the 301 participants. As expected among publicly-funded older home care service users, the majority of the sample was female, over the age of 80, single and lived alone [3,12]. However, compared with the population from which this sample was drawn, the sample has slightly fewer females (68% compared with 72%), more people from a white ethnic background (98% compared with 93%) and a smaller proportion over 80 years old (60% compared with 68%) [12].

Table 2. Socio-demographic and economic characteristics of sample members

The distribution of non-proxy responses for each ASCOT attribute is shown in Table 3 along with the proportion of responses answered by a proxy. Ten per cent or fewer of the responses to ASCOT attributes were given by a proxy. This was very similar to the percentages for the EQ-5D, GHQ-12, ADL and QoL question, with proxy response rates ranging from 8.6% to 10.6%.

Table 3. Responses of older people receiving home care to the ASCOT questionnaire (n = 301)

We hypothesised that a number of measures -- GHQ-12, overall QoL, EQ-5D, and CASP control and autonomy subscale -- would be associated with all of the attributes and that the UCLA loneliness scale would be associated with all the attributes except Dignity. In general, associations significant at the 1% level (p < .01) were found between these measures and all of the attributes (see Table 4), except for the relationships between the CASP subscale and Food and Drink and Dignity, and the loneliness scale and Food and Drink and Accommodation, where associations reached the 5% (p < .05) or 10% level (p < .10) of significance.

Table 4. Mean GHQ-12, CASP subscale, EQ-5D and UCLA loneliness scale scores by ASCOT attribute

The relationship between these measures and the attributes were also in the expected direction - positive for GHQ-12, overall QoL, EQ-5D, and CASP subscale and negative for the loneliness scale (see Table 4). Post-hoc tests showed a significant difference in average CASP subscale scores between the top and second response options for all of the attributes except Dignity. For Dignity a significant difference was found between the responses to the second response option and the options that report feeling undermined.

For both the GHQ-12 and EQ-5D, post-hoc tests showed a significant difference in average scores between the top and second response options for all attributes, except Dignity, Safety and Social participation (for EQ-5D only) where the difference was between the top two and the bottom two options. Average scores were also significantly different for each level of Control. Post-hoc tests for the loneliness scale revealed that people reporting the top level were significantly less lonely compared to those reporting worse outcomes, for all attributes except Safety and Occupation where those reporting the top two levels were significantly less lonely than those choosing the bottom two levels, and Dignity where no significant differences were found.

Results for variables hypothesised to be related to each attribute are summarised below. The significance of the relationships between the measures of socio-demographic characteristics, disability, nature of the locality and environment, social contact and support, participation and service quality and the eight attributes are summarised in Tables 5 and 6 and the nature and direction of the associations are reported in the text below.

Table 5. Significance of relationship between ASCOT attributes and socio-demographic characteristics, general quality of life and disability (p-values)

Table 6. Significance of relationship between ASCOT attributes and nature of locality and environment, social contact and support and participation (p-values)

Control over daily life

Control was significantly associated with the CASP subscale (p < .001) and, in comparison with the other attributes for which a difference in average scores between the top two response options was found, post-hoc tests revealed that the difference between the mean CASP scores of each Control state, except that between the lowest two levels, was significant (see Table 5 for means).

Several measures - ADLs, IADLs, home design and accessibility of local area - captured aspects of one's life that may restrict one's ability to manage independently. We therefore expected them to be associated with Control. A positive relationship was observed with home design and accessibility of local area, although the relationship with home design was weaker (p < .05). All of the ADLs (except ability to manage stairs and feed oneself) and IADLs (except prepare hot meals) were significantly (p < .001) associated with Control in the manner expected, with people reporting that they could manage the activity on their own more likely to report the top level. For the most part, poorer outcomes were related to whether they managed with difficulty, needed help or did not do the activity. An exception was that difficulty performing the mobility ADLs was not associated with poorer outcomes.

Control was unexpectedly associated with living status (p < .001), with those reporting living with others experiencing less control over their daily lives. We also did not anticipate the positive association found with speaking to relatives and speaking to neighbours (p < .001).

Personal cleanliness and comfort

Although the Personal cleanliness item captures the compensatory action of services, we did expect the attribute to be associated with ADLs related to personal care, with those reporting they could manage the personal care ADLs on their own being more likely to choose the top response option. This pattern was observed, and significant associations were found for getting in and out of bed, going to the WC/toilet, washing face and hands, bathing (all p < .001) and getting dressed and undressed (p < .05). Unexpectedly, highly significant associations were also found for getting around indoors and feeding (p < .001), as well as for the IADL managing paperwork and finances (p < .001). A positive association was also observed between home design and Personal cleanliness (p < .01), as we hypothesised.

Personal cleanliness had an unexpected positive association with participation in groups and speaking to neighbours; there was no association with speaking and meeting up with relatives and friends. Another unanticipated association was that people living in London or another large city or town were more likely to report the top level, with people living in suburban areas and small towns (not rural areas) appearing worst off, both being more likely than expected to choose the lower two levels.

Food and drink

Food and drink was significantly associated with the 'feed self' ADL (p < .001), with those reporting that they could manage on their own being more likely to choose the top level, as we expected. Although in the anticipated direction, with people reporting that they can manage the activity on their own more likely to report the top level, the association with the IADL preparing hot meals was only significant at the 5% level (p < .05); and there was no association with the household shopping IADL. Again, reflecting the effect of home design on outcomes, a positive relationship was observed between this variable and Food and drink (p < .05).

Food and drink also had a highly significant relationship (p < .001) with using the WC/toilet ADL, with those reporting that they could manage on their own being more likely to choose the top level.

Accommodation cleanliness and comfort

We hypothesised that Accommodation would have a significant, positive association with the interviewer's assessment of the cleanliness and tidiness of the respondent's home and with home design. Both of these relationships were observed (p < .001). Accommodation had strong relationships (p < .01) with several ADLs (getting into and out of bed, using the WC/toilet, and washing face and hands), with those reporting that they could manage on their own being more likely to choose the top level. It also had an unexpected relationship with the type of area the person lives in (p < .01), with people living in London or another large city or town more likely to report the top level. People living in suburban areas and small towns (not rural areas) appeared worst off, both being more likely than expected to report the lower two levels.

Safety

Safety was, as we anticipated, positively associated with home design (p < .01). It was also associated with the type of area the respondent lives in (p = .011), with people living in London or another large city or town being more likely than expected to report the lower two levels; people living in suburban areas being more likely than expected to report the third level; and people in small towns and rural areas being more likely than expected to report the second level.

As expected, sense of Safety was weakly related to living alone (p < .05) and marital status (p < .10), with those who live with others or are married reporting better outcomes. Both items capturing the extent to which the person speaks to others (to relatives/friends and neighbours) were positively associated with safety, although for speaking to neighbours the association was weak (p < .10). There was no association with meeting up with relatives or friends.

An unexpectedly significant, positive association was also found between Safety and people's perceptions of how easy it is to get to all the places in their local area (p = .001). A highly significant association was found with the household shopping IADL (p < .001) and the feed self ADL (p < .01), where people reporting that they can manage the activity on their own were more likely to report the top outcome state.

Social participation

Social participation was significantly associated with the loneliness scale and, unlike the pattern observed for the other attributes, post-hoc tests revealed a significant difference in the average score between each Social participation level (see Table 4). In addition, Social participation had positive associations, with the social contact and support items (frequency of speaking to relative or friends (p = .011), to neighbours (p = .017), and meeting up with relatives or friends (p = .001)) and having volunteered (p < .05). Social participation was not associated with marital status or living alone.

As hypothesised, there was a positive association with accessibility of the local area (p < .05) and a marginally significant association with the type of area the respondent lived in (p < .10): people living in London or another large city were more likely to report good outcomes and those in suburban areas were more likely to report the lower two levels.

Social participation was also strongly, positively associated with self-reported design of home (p < .01) and the interviewer's view of the cleanliness and tidiness of the respondent's home (p < .01).

Occupation

Key anticipated relationships for Occupation were found with participation in groups (p = .01) and volunteering (p = .001). In addition, Occupation was strongly positively associated with the three social contact and support items (frequency of speaking to relative or friends, to neighbours, and meeting up with relatives or friends (all p < .01)).

As expected, there was an association with the area people lived in (p = .014), with people living in London or another large city or town being more likely to report the top level of Occupation. However, it was people living in suburban areas and small towns (not rural areas) who appeared worst off, both being more likely than expected to report the lower levels [22]. Occupation also had a significant, positive association with people's perceptions of how easy it is to get to around their local area (p < .001), as hypothesised.

Occupation was significantly associated with nearly all the ADL items (see Table 5), with people reporting that they could manage the activity on their own more likely to report the top level. For the most part, poorer outcomes were related to whether the person managed with difficulty, needed help or did not do the activity. An exception to this pattern was that for people reporting difficulty performing the ADLs relating to mobility and using the toilet/WC this was not associated with poorer outcomes. There was also a highly significant association with the managing paperwork and finances IADL (p < .001), with people reporting that they can manage the activity on their own more likely to report the top level.

Dignity

Dignity was significantly associated with the question asking whether care workers do the things the user wants done (p < .001). It was also weakly associated with all of the other user-reported service quality variables, except the question about whether the person sees the same care workers, with p-values ranging from 0.028 to 0.064, such that better quality services were associated with better dignity scores. Unexpectedly, no relationship was found (p = .71) with the question asking users to assess their overall treatment by care workers.

An unanticipated association was found with home design (p < .01), with those reporting that how they are helped makes them think and feel better about themselves being the only group more likely than expected to think the design of their home meets all their needs. A significant and unexplained association was also found with the area people live in (p < .01), with those living in London, another large city or town and suburban areas being less likely than expected to choose the top level and more likely to choose the second response option. By contrast, people living in small towns were more likely to choose the top level and less likely to choose the second level.

Discussion

The analyses presented in this paper provide some evidence to support the construct validity of the ASCOT attributes. The difference between the top two levels in CASP control and autonomy subscale scores for each attribute provides good evidence of the ability of the items to capture choice. Although for Dignity a difference was found between the top two and bottom two levels, when cognitively testing this item we found that those choosing the second response option generally had less need for help and received fewer services, which implies, as we found here, that this group are more independent and less likely to experience loss of control compared to those choosing other response categories.

In terms of the attributes, we have very strong evidence to support the validity of the Control and Occupation items, in particular, since these had strong associations with the expected variables and where there were unexpected relationships these could be explained. For instance, it is possible that the association between higher levels of Control and living with others is related to higher levels of impairment among service users who live with others, as eligibility criteria mean that those living alone are more likely to receive services as they have less access to informal support. In support of this, we found that respondents living with others (U = 4765.50, z = -7.38, p < .001) needed help with significantly more ADLs than people living alone. It is also possible that those with more control (and therefore independence) are able to maintain relationships more easily, hence the positive association between Control and speaking to relatives and neighbours [23]. Equally the unexpected finding that it was those living in suburban areas and small towns whom had the worst Occupation outcomes, may be explained by suburban areas lacking a sense of community that is strong in rural areas [22]. The association between Occupation and the ADLs could be due to the loss of independence associated with difficulty or inability to perform ADLs or IADLs and the restriction this places on the activities a person can enjoy.

There was also very good evidence concerning the Personal cleanliness and Accommodation attributes, since both had strong associations in the expected direction with key variables. We were not able to understand well the reason for the relationship of these items with the type of area the person lived in. It may be that this is due to LA policies or a quirk of the sample, and validation on a different sample would provide contrary evidence. However, other unanticipated relationships such as the one between Personal cleanliness and the non-personal care ADLs and IADLs, is likely to be due to similarities in the capabilities required for the tasks. The positive association of Personal cleanliness with speaking to neighbours and participation in groups could reflect an unwillingness of people who do not feel presentable to socialise with people they do not know well [24].

In general, Social participation had the anticipated relationships with other variables, although sometimes the associations were weak. The only expected findings that were not observed was an association with marital status and living with others, which suggests that the people in this sample, at least, did not consider those they live with as being a source of social contact. The unexpected strong positive associations with design of home and the interviewer's view of the cleanliness and tidiness of the respondent's home, may reflect an unwillingness amongst people with inaccessible, unclean or untidy homes to accept guests [24].

Similarly, Safety seemed to capture factors both inside and outside the home that could make a person feel unsafe, but the lack of association with the frequency with which the person met up with relatives and friends was unexpected, particularly given the associations that were found with speaking to relatives, friends and neighbours. This implies that a sense of safety is determined more by the sense that there is someone to turn to or perhaps a sense of connection to a community, as has been found in research elsewhere [23,25,26] rather than regularly meeting up with friends and relatives. This potentially has implications for the nature of social care support that aims to help people feel safe. The unanticipated relationships with variables capturing the ability of the person to get around outdoors may be explained by fear of falling.

The attributes for which we had the weakest evidence of validity were Food and drink and Dignity. Food and drink is not easy to validate as good, easy-to-administer, self-reported measures of nutritional intake are lacking. The few measures we did use in general had the expected relationship with the item, but often the relationships were weak. The lack of relationship with household shopping is not too concerning since this could include shopping for items other than food. As we discuss below, the poor distribution of Food and drink which resulted in a lack of variation may have affected the analysis undertaken here and it would be of value to repeat the analysis conducted here with the new item wording.

Similarly, we lacked good data against which to validate Dignity since the variables that we hypothesised would be strongly associated with Dignity -- the variables capturing aspects of quality associated with the delivery of home care services by care workers and related staff - were collected roughly six months prior to the survey data. In general Dignity had the expected relationships with other variables, but the associations were mostly weak. The lack of relationship with the question about overall treatment by care workers was unexpected, but given the other significant, albeit weak, relationships, and previous evidence that this question is poorly related to service users' attitudes towards their carers [27], we feel the observed lack of association raises more questions about the overall treatment question rather than the Dignity item. The weak associations between Dignity and most of the service quality questions could reasonably be explained by the gap of six months between the collection of both sets of data. Differences in breadth of the questions could also be a factor, since the questions about service quality focus on care workers, whereas Dignity is broader, asking about help and treatment by any person. It would be of value to repeat this analysis with better validation data.

In addition to the evidence presented here, the methods used to develop the measure ensured content and face validity. Expert review with social care stakeholders was used to identify attributes and ensure ASCOT's sensitivity to outcomes of interest to policymakers and relevance to the evaluation of social care interventions. This approach was complemented by a literature review exploring service users' understanding of social care outcomes. Cognitive interviews were conducted to check social care service users' understanding of terms and clarify the wording of the items [6].

ASCOT was developed to fill a gap, as to our knowledge there are no dedicated social care outcome measures. Past studies in the social care field have tended to use health outcome measures, such as the EQ-5D, to assess cost-effectiveness. Whilst these measures share some of the characteristics of ASCOT, they tend to focus on people's functional abilities (such as mobility) rather than the impact of support on their QoL and are limited in the range of outcome states they capture [28,29]. ACSOT was proposed as a measure to capture the full range of social care outcomes and we would expect it to be a more sensitive measure than the EQ-5D. Early findings suggest that the two measures are strongly correlated (r = 0.4), but that ASCOT is more sensitive than the EQ-5D to the impact of social care interventions [6,29].

This study also provides some evidence for the feasibility of using ASCOT with older people. All 301 participants responded to every item in the instrument, although about ten per cent of the responses were given by proxy. The need for proxies is not unexpected in a sample of older people receiving social care, where prevalence of cognitive decline is likely to be quite high. Although cognitive ability was not something we recorded in this study, the fact that the rate of proxy response was no higher for ASCOT than other QoL, health and disability measures (including the EQ-5D, ADLs and GHQ-12), suggests that proxy responses were given because the respondent lacked the capacity to answer survey questions in general rather than the ASCOT questions specifically. Further work examining the types of people for whom a proxy response is required and whether there are systematic differences between responses given by proxy or by self, as has been found elsewhere [30-33], would be helpful, as would the development of a suitable version for use with proxy respondents.

For the most part, the distribution of the items seemed plausible. Although the distributions were skewed towards good outcomes, if services are doing their job properly this type of distribution is to be expected. However, the distributions for Food and drink and Accommodation were highly skewed. We, therefore, tested revised wording in a parallel piece of work [34,35]. The new wording achieved better distributions in a sample of equipment users (see Appendix A) and has now been incorporated into the revised measure.

There were some limitations associated with this study. Firstly, the sample data only included older people receiving publicly-funded home care services. As a result it is only possible to draw conclusions about the feasibility of using the measure and its validity for this client group in this setting. Secondly, the sample obtained here was not ethnically diverse, so we cannot demonstrate the validity of the measure amongst black and minority ethnic (BME) groups. It would therefore be of value to repeat this analysis with other client groups and, given the potential for some members of BME groups to have very specific preferences related to their cultural heritage, on a more ethnically diverse sample. Future work should also consider the reliability of the items.

Conclusion

The current policy emphasis on outcomes in the field of health and social care is unlikely to reduce as fiscal pressures intensify the need to identify value for money. It is important that any measure provide a valid description of the outcomes states it is intended to reflect. The results for ASCOT are encouraging in this respect, although further validation work with a different sample and the development of a version suitable for proxies would be of value.

Abbreviations

The following abbreviations have been used: ADL: Activity of daily living; ASCOT: Adult Social Care Outcomes Toolkit; BME: Black and minority ethnic; CASP-12: 12 item scale measuring control, autonomy, self-realisation and pleasure; EQ-5D: EuroQol 5-item health questionnaire; GHQ-12: 12-item General Health Questionnaire; IADL: Instrumental activity of daily living; LA: Local authority; NIHR: National Institute for Health Research; SCRQoL: Social Care Related Quality of Life; SEG: Socio-economic group; QoL: Quality of Life; UCLA: University of California, Los Angeles; and UES: user experience survey). In tables 5 and 6 we use Accomm as shorthand for accommodation.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

JM designed this part of the broader study, conducted part of the statistical analysis and drafted the manuscript. A-MT conducted part of the statistical analysis, helped to draft the manuscript and is the corresponding author for submission purposes. AN conceived of the study, and participated in its design and organisation and the drafting of the manuscript. JB, TF and JF contributed to the design of the study and the drafting of the manuscript. All authors have read and approved the final manuscript.

Appendix A

Revised wording of items after further testing:

Food and drink

Thinking about the food and drink you get, which of the following statements best describes your situation?

I get all the food and drink I like when I want; I get adequate food and drink at OK times; I don't always get adequate or timely food and drink; I don't always get adequate or timely food and drink, and I think there is a risk to my health

Accommodation cleanliness and comfort

Which of the following statements best describes how clean and comfortable your home is?

My home is as clean and comfortable as I want; My home is adequately clean and comfortable; My home is not quite clean or comfortable enough; My home is not at all clean or comfortable

Acknowledgements

This project was funded by the UK National Institute for Health Research (NIHR) Health Technology Assessment Programme (project No. 06/96/01). The views and opinions expressed are those of the authors alone. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Department of Health. We are very grateful to all those who participated in the research and to Accent, who undertook the fieldwork. We are also grateful to the anonymous reviewers whose input greatly improved this article.

References

  1. Knapp M: The economics of social care. Basingstoke: Macmillan; 1984. OpenURL

  2. Her Majesty's Government: The Coalition: our programme for government. London: TSO; 2010. OpenURL

  3. Wittenberg R, Comas-Herrara A, King D, Malley J, Pickard L, Darton R: Future Demand for Long-Term Care, 2002 to 2041: Projections of Demand for Long-Term Care for Older People in England. Discussion Paper 2330. Canterbury: Personal Social Services Research Unit; 2006. OpenURL

  4. Wanless D: Securing Good Care for Older People. Taking a Long-Term View. London: King's Fund; 2006. OpenURL

  5. Her Majesty's Government: Building the National Care Service. CM 7854. London: The Stationery Office; 2010. OpenURL

  6. Netten A, Burge P, Malley J, Potoglou D, Towers A, Brazier J, Flynn T, Forder J, Wall B: Outcomes of Social Care for Adults: Developing a Preference-Weighted Measure.

    Health Technology Assessment, in press. OpenURL

  7. De Vellis R: Scale Development: Theory and Applications. 2nd edition. London: Sage; 2003. OpenURL

  8. Lenert L, Kaplan RM: Validity and Interpretation of Preference-Based Measures of Health-Related Quality of Life.

    Medical Care 2000, 38:II138-II150. PubMed Abstract | Publisher Full Text OpenURL

  9. Brazier J, Deverill M, Green C, Harper R, Booth A: A review of the use of health status measures in economic evaluation.

    Health Technology Assessment 1999, 3:1-164. PubMed Abstract | Publisher Full Text OpenURL

  10. Potoglou D, Burge P, Flynn T, Netten A, Malley J, Forder J, Brazier J: Best-Worst Scaling vs. Discrete Choice Experiments: An Empirical Comparison using Social Care Data.

    Social Science & Medicine 2011, 72:1717-1727. PubMed Abstract | Publisher Full Text OpenURL

  11. Coast J, Peters TJ, Natarajan L, Sproston K, Flynn T: An assessment of the construct validity of the descriptive system for the ICECAP capability measure for older people.

    Quality of Life Research 2008, 17:967-976. PubMed Abstract | Publisher Full Text OpenURL

  12. Information Centre for Health and Social Care: Personal Social Services Home Care Users in England aged 65 and over, 2008-09 Survey. Leeds: Information Centre for Health and Social Care; 2009. OpenURL

  13. EuroQol group: EuroQol -- A New facility for the measurement of health-related quality of life.

    Health Policy 1990, 16:199-208. PubMed Abstract OpenURL

  14. Dolan P, Gudex C, Kind P, Williams A: The time trade-off method: Results from a general population study.

    Health Economics 1996, 5:141-154. PubMed Abstract | Publisher Full Text OpenURL

  15. Goldberg D, Hilller V: A scaled version of the General Health Questionnaire.

    Psychological Medicine 1979, 9:139-145. PubMed Abstract | Publisher Full Text OpenURL

  16. Goldberg D: Manual of the general health questionnaire. Windsor: NFER; 1978. OpenURL

  17. Goldberg D: The detection of psychiatric illness by questionnaire. London: Oxford University Press; 1972. OpenURL

  18. Hyde M, Wiggins RD, Higgs P, Blane DB: A measure of quality of life in early old age: The theory, development and properties of a needs satisfaction model (CASP-19).

    Aging & Mental Health 2003, 7:186-194. PubMed Abstract | Publisher Full Text OpenURL

  19. Wiggins R, Netuveli G, Hyde M, Higgs P, Blane D: The Evaluation of a Self-enumerated Scale of Quality of Life (CASP-19) in the Context of Research on Ageing: A Combination of Exploratory and Confirmatory Approaches.

    Social Indicators Research 2008, 89:61-77. Publisher Full Text OpenURL

  20. Malley J, Sandhu S, Netten A: Younger Adults' Understanding of Questions for a Service User Experience Survey: A Report to the Information Centre for Health and Social Care. PSSRU Discussion Paper 2360. Canterbury: Personal Social Services Research Unit; 2006. PubMed Abstract OpenURL

  21. Hughes ME, Waite LJ, Hawkley LC, Cacioppo JT: A Short Scale for Measuring Loneliness in Large Surveys.

    Research on Aging 2004, 26:655-672. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  22. Leyden KM: Social Capital and the Built Environment: The Importance of Walkable Neighbourhoods.

    American Journal of Public Health 2003, 93:1546-1551. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  23. Walker RB, Hiller JE: Places and health: A qualitative study to explore how older women living alone perceive the social and physical dimensions of their neighbourhoods.

    Social Science & Medicine 2007, 65:1154-1165. PubMed Abstract | Publisher Full Text OpenURL

  24. Aronson J: Elderly people's accounts of home care rationing: missing voices in long-term care policy debates.

    Ageing & Society 2002, 22:399-418. PubMed Abstract OpenURL

  25. Young AF, Russell A, Powers JR: The sense of belonging to a neighbourhood: can it be measured and is it related to health and well being in older women?

    Social Science & Medicine 2004, 59:2627-2637. PubMed Abstract | Publisher Full Text OpenURL

  26. Flynn T, Chan P, Coast J, Peters TJ: Assessing quality of life among British older people using the ICECAP-O capability measure.

    Applied Health Economics and Health Policy 2011, 9(5):317-29. PubMed Abstract OpenURL

  27. Jones K, Netten A, Francis J, Bebbington A: Using older home care user experiences in performance monitoring.

    Health & Social Care in the Community 2007, 15:322-332. PubMed Abstract | Publisher Full Text OpenURL

  28. Andresen EM, Meyers AR: Health-related quality of life outcomes measures.

    Archives of Physical Medicine and Rehabilitation 2000, 81:S30-S45. PubMed Abstract | Publisher Full Text OpenURL

  29. Forder JE, Caiels J: Measuring the outcomes of long-term care.

    Social Science & Medicine 2011, 73:1766-1774. PubMed Abstract | Publisher Full Text OpenURL

  30. Ellis B, Bannister W, Cox J, Fowler B, Shannon E, Drachman D, Adams R, Giordano L: Utilization of the propensity score method: an exploratory comparison of proxy-completed to self-completed responses in the Medicare Health Outcomes Survey.

    Health and Quality of Life Outcomes 2003, 1:47. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  31. Epstein AM, Hall JA, Tognetti J, Son LH, Conant L Jr: Using Proxies to Evaluate Quality of Life: Can They Provide Valid Information about Patients' Health Status and Satisfaction with Medical Care?

    Medical Care 1989, 27:S91-S98. PubMed Abstract | Publisher Full Text OpenURL

  32. Pierre U, Wood-Dauphinee S, Korner-Bitensky N, Gayton D, Hanley J: Proxy Use of the Canadian SF-36 in Rating Health Status of the Disabled Elderly.

    Journal of Clinical Epidemiology 1998, 51:983-990. PubMed Abstract | Publisher Full Text OpenURL

  33. Andresen EM, Vahle VJ, Lollar D: Proxy Reliability: Health-Related Quality of Life (HRQoL) Measures for People with Disability.

    Quality of Life Research 2001, 10:609-619. PubMed Abstract | Publisher Full Text OpenURL

  34. Malley J, Caiels J, Fox D, McCarthy M, Smith N, Beadle-Brown J, Netten A, Towers A-M: A Report on the Development Studies for the National Adult Social Care User Experience Survey, PSSRU Discussion Paper 2721. Canterbury: Personal Social Services Research Unit, University of Kent; 2010. OpenURL

  35. Malley J: A Comparison of Options for Performance Indicators from the Adult Social Care Survey (ASCS), PSSRU Discussion Paper 2736. Canterbury: Personal Social Services Research Unit, University of Kent; 2010. OpenURL

  36. Bowling A, Iliffe S: Psychological approach to successful ageing predicts future quality of life in older adults.

    Health and Quality of Life Outcomes 2011, 9:13. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  37. Bilotta C, Bowling A, Case A, Nicolini P, Mauri S, Castelli M, Vergani C: Dimensions and correlates of quality of life according to frailty status: a cross-sectional study on community-dwelling older adults referred to an outpatient geriatric service in Italy.

    Health and Quality of Life Outcomes 2010, 8:56. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  38. Ormel J, Lindenberg S, Steverink N, Vonkorff M: Quality of life and social production functions: A framework for understanding health effects.

    Social Science & Medicine 1997, 45:1051-1063. PubMed Abstract | Publisher Full Text OpenURL

  39. Wilson IB, Cleary PD: Linking clinical variables with health-related quality of life: a conceptual model of patient outcomes.

    JAMA 1995, 273:59-65. PubMed Abstract | Publisher Full Text OpenURL

  40. Tanner B, Tilse C, de Jonge D: Restoring and Sustaining Home: The Impact of Home Modifications on the Meaning of Home for Older People.

    Journal of Housing For the Elderly 2008, 22:195-215. Publisher Full Text OpenURL

  41. Gitlin LN, Corcoran M, Winter L, Boyce A, Hauck WW: A Randomized, Controlled Trial of a Home Environmental Intervention.

    The Gerontologist 2001, 41:4-14. PubMed Abstract | Publisher Full Text OpenURL

  42. Bannister J, Fyfe N: Fear and the City.

    Urban Studies (Routledge) 2001, 38:807-813. Publisher Full Text OpenURL

  43. Box S, Hale C, Andrews G: EXPLAINING FEAR OF CRIME.

    British Journal of Criminology 1988, 28:340-356. OpenURL

  44. Victor CR, Scambler SJ, Bowling A, Bond J: The prevalence of, and risk factors for, loneliness in later life: a survey of older people in Great Britain.

    Ageing & Society 2005, 25:357-375. PubMed Abstract | Publisher Full Text OpenURL

  45. Victor C, Scambler S, Bond J, Bowling A: Being alone in later life: loneliness, social isolation and living alone.

    Reviews in Clinical Gerontology 2000, 10:407-417. Publisher Full Text OpenURL

  46. de Jong Gierveld J: A review of loneliness: concept and definitions, determinants and consequences.

    Reviews in Clinical Gerontology 1998, 8:73-80. Publisher Full Text OpenURL