Overview of studies
All but eight [
15,
18‐
24] of the 25 studies identified as reporting psychometric properties of ICECAP-O (including the paper by Milne et al. [
15]) assessed construct validity. (Specific methodology referred to by authors was convergent, divergent and discriminant validity.) Responsiveness and reliability have received much less attention: five studies assessed responsiveness [
16,
22,
24‐
26], just two assessed reliability [
23,
26]. Content validity (including face, item and sampling validity) was assessed in six studies [
15,
18‐
21,
23]. The majority of studies (21) in this category were quantitative studies. The remaining four used qualitative or semi-qualitative techniques and comprised: two ‘think aloud’ studies [
19,
21], one study based on semi-structured interviews [
20], and one semi-qualitative-quantitative study based on the Nominal Group Technique [
18].
The psychometric properties of the ICECAP-O were assessed across a range of patient and general populations: five focussed on the general population only [
23,
27‐
30]; Couzner et al. included both patients (with post-acute needs) and the general population [
31]. Of those studies which exclusively included patients/service users, five focused on cognitive impairment [
15,
32‐
35], three on frailty/social care needs [
21,
26,
36] and two on those at high risk of falling [
22,
37,
38]. Two studies focussed on patients with post-acute needs [
39‐
41]. One study included patients who had previously had a stroke [
16] and one study included the carers of people with dementia [
20]. Five studies focussed on patients undergoing joint surgery [
17‐
19,
24,
42].
The majority of studies were undertaken in Europe with ten conducted in the UK [
15‐
20,
24,
27,
29,
36], five in the Netherlands [
21,
26,
30,
34,
39], and one each in Germany [
33], Spain [
35] and Sweden [
23]. Of the remaining studies, four were carried out in Australia [
28,
31,
40‐
42] and three in Canada [
22,
32,
37,
38].
Supplementary Table 2 provides a full summary of papers reporting psychometric properties.
Construct Validity
A summary of results relating to the assessment of construct validity is presented in Table
1. Of those studies that examined construct validity, the comparators most commonly used can be broadly categorised as ‘sociodemographic characteristics’, ‘general health’, ‘Activities of Daily Living/physical independence’, ‘mental health (non-dementia specific)’, ‘cognitive-impairment’ ‘well-being’ and ‘environment and care quality’. Each is examined in turn.
Table 1
Results relating to construct validity
Socio-demographic characteristics |
Increased age | ICECAP-O does not discriminate between over and under 65’s [NH] [ 28] ICECAP-O discriminates between the young-old (65–75) and old-old (over 75) [expected] [ 29, 33, 35, 36, 39] |
Living with others | None [positive expected] [ 28] |
Living with marital partner | None [positive expected] [ 28] |
Generic health measures (index score) |
EQ-5D-3L or 5L | Positive strong [expected] [ 36] Positive [expected] [ 16, 38, 44] ICECAP-O discriminates between: above and below average health [NH] [ 36] patients and the general population [expected] [ 45] |
Physical health/independence |
Katz Index of Independence in Activities of Daily Living (ADL) | ICECAP-O discriminates between IADL dependent and non-IADL dependent elderly [expected] [ 39] |
Barthel activities of daily living (ADL) Index | Positive strong [expected] [ 33, 36] Positive strong [expected] [ 35] |
Mental health (non-dementia) |
Geriatric depression scale-15 | Negative strong [NH] [ 36] |
Cognitive impairment |
Alzheimer Disease Related Quality of Life (ADRQL) | Positive [expected] [ 33, 35] |
Well-being |
Cantril’s ladder | Positive strong [NH] [ 36] Positive [expected] [ 34, 39] |
Environment and care quality |
Multiple deprivation scores of electoral ward | None [negative weak expected] [ 29] |
A complete table of summarised results regarding construct validity can be found in Supplementary Table 3.
The most commonly used sociodemographic comparator was age, and findings here were mixed. Of the studies using age as comparator, seven were conducted in Europe [
16,
25,
27,
33,
35,
36,
39] (predominantly the UK) and all found a degree of negative association between increasing age and ICECAP-O score; of the three studies conducted in Australia [
28,
40‐
42] two found no relationship and one [
42] found a negative relationship. Findings were also mixed in relation to ‘living with others’: three European studies found positive relationships [
17,
25,
27] and two studies conducted in Australia found no relationship [
28,
42].
Other socio-demographic comparators used were gender [
16,
27,
42], social class [
27], employment status [
28], income level [
28,
36], receiving benefits, having a faith, being an unpaid carer [
25] and being married [
17]. The direction of the relationship between each of these comparators and ICECAP-O scores was found to be as hypothesised although those in relation to social class [
28], employment status [
28] and being an unpaid carer [
27,
29] were not statistically significant.
The most commonly used comparator for general health was the EQ-5D which was used in ten studies [
16,
27,
31,
33,
34,
36,
38‐
42,
44]. The relationship between EQ-5D and ICECAP-O scores was as expected in all studies, with one exception: Coast et al. [
27] found no relationship between the EQ-5D score and Attachment (whereas the study by Keeley [
16] found a positive relationship). Other generic health comparators used were: EQ-5D extended with a cognitive dimension (EQ-5D-3L + C) [
33,
35], EQ-VAS [
34,
36], SF-20 [
39] and SF-36 [
16]. A positive relationship between ICECAP-O scores and better health was both hypothesised and found within each of these studies.
Measures of physical health/independence that were used as comparators were: the Katz Index of Activities of Daily Living [
39], the Barthel Activities of Daily Living index [
33,
35,
36], Instrumental Activities of Daily Living, Physiological Profile Assessment, Short Physical Performance Battery [
37,
38], the Care Dependency Scale [
34], the Modified Rankin Scale [
16], having a disability, pain or a limiting or long-term illness (a survey question), and doing moderate exercise [
25]. All studies hypothesised and found a positive relationship between greater physical health/independence and ICECAP-O scores.
The following measures of (non-dementia) mental health were used as comparators: the Geriatric Depression Scale-15 [
36,
39], the Hospital Anxiety and Depression Scale (HADS), the Herth Hope Index [
40,
41]. All of the studies that stated a hypothesis expected a positive relationship between better mental health and overall ICECAP-O score. All but one study found a positive relationship, the exception being Makai et al. who found no relationship between the overall ICECAP-O score and the HADS.
Measures of cognitive impairment that were used as comparators were the Alzheimer’s Disease Related Quality of Life (ADRQL), the Mini–Mental State Examination and the Global Deterioration Scale [
35]. All studies hypothesised and found a negative relationship between higher levels of cognitive impairment and ICECAP-O score.
The following measures of well-being were used as comparators: Cantril’s Ladder [
34,
36,
39], Social Production Function Instrument for the Level of Well-being [
39], Satisfaction With Life Scale (SWLS) and the Older People’s Quality of Life Questionnaire (OPQOL), and two survey questions about overall life satisfaction and narrative foreclosure in relation to the past and future [
30]. All studies hypothesised and found a positive relationship between higher well-being and overall ICECAP-O index.
A range of other comparators were used that fall broadly under the heading of ‘environment and quality of care’. All studies hypothesised and found a positive relationship between ICECAP-O index scores and a ‘better’ environment (safer/less deprived/greater level of contact with others/better quality care).
Content validity
A summary of results relating to the assessment of content validity is presented in Table
2. Several studies found that participants questioned the relevance of the ICECAP-O domains. Studies found the domains were considered irrelevant for measuring outcomes in clinical trials of patients with hip fracture [
18] or for carers of those with dementia trialling a Global Positioning Satellite technology [
15]. Horwood et al. [
19] found that in a UK population of surgical joint replacement patients, whilst some participants questioned the relevance of the Attachment and Security domains they were considering the relevance specifically in relation to aspects of their illness rather than in relation to their general quality of life. A ‘narrow’ interpretation of items considered not to be relevant was also observed in a study of older people in the Netherlands [
21], again in relation to the ‘Attachment’ and ‘Security’ domains. However, in qualitative interviews with informal carers of people with dementia [
20] in the UK, four themes were identified (social network and relationships; interactions with agencies; recognition of role; and time for oneself), which the researchers noted overlap with ICEAP-O domains. A Swedish study of over 70s found Attachment to be the most relevant domain and enjoyment the least [
23].
Table 2
Content validity results
Haywood et al. (2014) [ 18] | Relevance: ICECAP domains considered to be not important or relevant | Clinical trial of patients with hip fracture (UK) |
Horwood et al. (2014) [ 19] | Relevance: Some participants questioned why ‘love and friendship’ and ‘thinking about the future without concern’ were relevantHowever, they were focusing on the relevance to their operation | Patients undergoing joint replacement surgery (UK) |
Van Leeuwen et al. (2015a) [ 21] | Relevance: Some participants narrowly interpreted ‘Attachment’ and ‘Security’ items respondents tending to concentrate on one aspect of a domain | Dutch older adults |
Hörder et al. (2016) [ 23] | Relevance: Participants gave their highest relevance rating to Attachment and lowest to Enjoyment | Swedish 70-year-olds |
| Four themes were identified: social network and relationships; interactions with agencies; recognition of role; and time for oneself. All overlap with ICECAP-O domains | Carers of people with dementia (UK) |
| Several caregivers criticised the ICECAP-O for having questions that did not seem relevant | Carers of people with dementia (UK) |