Top

Gepubliceerd in:

03-12-2024 | Review

Cognitive interviewing for assessing the content validity of older-person specific outcome measures for quality assessment and economic evaluation: a scoping review

Auteurs: Kiri Lay, Claire Hutchinson, Jia Song, Rachel Milte, Jyoti Khadka, Julie Ratcliffe

Gepubliceerd in: Quality of Life Research | Uitgave 3/2025

Abstract

Purpose

Older people (aged ≥ 65 years) are significant users of health and social care systems. However, many outcome measures for quality assessment and economic evaluation lack content validation for this population. Cognitive interviews are a key approach for generating content validity evidence. This systematic scoping review aimed to synthesise evidence on cognitive interviewing practices with older adults, including those with cognitive impairment and dementia, to determine the content validity of outcome measures used with these populations in health and social care settings.

Methods

A search was conducted across five databases, and results were uploaded to Covidence review management software. Two researchers performed screening and data extraction using a custom template. The initial search identified 8814 articles. After removing duplicates and conducting title and abstract screening, 105 articles remained. Full-text screening excluded 86 references, resulting in 19 included articles.

Results

Studies utilised various cognitive interviewing approaches, predominantly concurrent think-aloud combined with verbal probing. Inductive analysis based on Tourangeau’s four-stage response model was commonly used for analysing verbal protocol data. Only one in four included studies indicated any allowances to alleviate cognitive burden for older participants. There was limited involvement of older people with cognitive impairment and dementia, but studies including these populations reported several modifications enhancing meaningful inclusion.

vorige artikel Opening the contextual black box: a case for idiographic experience sampling of context for clinical applications

volgende artikel Quality of life among obstructive sleep apnoea patients: a systematic review and meta-analysis of EQ-5D studies

Alleen toegankelijk voor geautoriseerde gebruikers

United Nations. (2022). World population prospects 2022: Summary of results (UN DESA/POP/2022/TR/NO. 3.). https://www.un.org/development/desa/pd/sites/www.un.org.development.desa.pd/files/wpp2022_summary_of_results.pdf

Singh-Manoux, A., Kivimaki, M., Glymour, M. M., Elbaz, A., Berr, C., Ebmeier, K. P., Ferrie, J. E., & Dugravot, A. (2012). Timing of onset of cognitive decline: Results from Whitehall II prospective cohort study. BMJ : British Medical Journal, 344, d7622. https://doi.org/10.1136/bmj.d7622CrossRefPubMedPubMedCentral

World Health Organisation & Alzheimer’s Disease International. (2012). Dementia: A public health priority. World Health Organisation. https://www.who.int/publications/i/item/dementia-a-public-health-priority

Black, N. (2013). Patient reported outcome measures could help transform healthcare. BMJ : British Medical Journal, 346, f167. https://doi.org/10.1136/bmj.f167CrossRefPubMed

Dawson, J., Doll, H., Fitzpatrick, R., Jenkinson, C., & Carr, A. J. (2010). The routine use of patient reported outcome measures in healthcare settings. BMJ : British Medical Journal, 340, c186. https://doi.org/10.1136/bmj.c186CrossRefPubMed

Agarwal, A., Pain, T., Levesque, J. F., Girgis, A., Hoffman, A., Karnon, J., King, M. T., Shah, K. K., Morton, R. L., HSRAANZ PROMs Special Interest Group. (2022). Patient-reported outcome measures (PROMs) to guide clinical care: Recommendations and challenges. Medical Journal of Australia, 216(1), 9–11. https://doi.org/10.5694/mja2.51355CrossRefPubMed

Dean, S., Al Sayah, F., & Johnson, J. A. (2021). Measuring value in healthcare from a patients’ perspective. Journal of Patient-Reported Outcomes, 5(Suppl 2), 88. https://doi.org/10.1186/s41687-021-00364-4CrossRefPubMedPubMedCentral

Churruca, K., Pomare, C., Ellis, L. A., Long, J. C., Henderson, S. B., Murphy, L. E. D., Leahy, C. J., & Braithwaite, J. (2021). Patient-reported outcome measures (PROMs): A review of generic and condition-specific measures and a discussion of trends and issues. Health Expectations, 24(4), 1015–1024. https://doi.org/10.1111/hex.13254CrossRefPubMedPubMedCentral

Bulamu, N. B., Kaambwa, B., & Ratcliffe, J. (2015). A systematic review of instruments for measuring outcomes in economic evaluation within aged care. Health and Quality of Life Outcomes, 13, 179. https://doi.org/10.1186/s12955-015-0372-8CrossRefPubMedPubMedCentral

10.

Ratcliffe, J., Laver, K., Couzner, L., Cameron, I. D., Gray, L., & Crotty, M. (2010). Not just about costs: The role of health economics in facilitating decision making in aged care. Age and Ageing, 39(4), 426–429. https://doi.org/10.1093/ageing/afq041CrossRefPubMed

11.

Royal Commision into Aged Care Quality and Safety. (2021). Final report: Care, dignity and respect-list of recommendations. C. o. Australia. https://agedcare.royalcommission.gov.au/sites/default/files/2021-03/final-report-volume-1.pdf

12.

Quality and Outcomes of Person-centered care policy research unit. (2024). The Adult Social Care Outcomes Toolkit (ASCOT). Retrieved March 26, 2024, from https://www.qoru.ac.uk/about/research/measurement/557-2/

13.

Prince, M., Bryce, R., Albanese, E., Wimo, A., Ribeiro, W., & Ferri, C. P. (2013). The global prevalence of dementia: A systematic review and metaanalysis. Alzheimer’s & Dementia, 9(1), 63–75.e62. https://doi.org/10.1016/j.jalz.2012.11.007CrossRef

14.

Australian Institute of Health and Welfare. (2021). Dementia in Australia. AIHW, Australian Government. Retrieved May 13, from https://www.aihw.gov.au/getmedia/4afad8ba-08b5-4092-ab40-049a3cdb94eb/Dementia-in-Australia.pdf.aspx?inline=true

15.

Gagnier, J. J., Lai, J., Mokkink, L. B., & Terwee, C. B. (2021). COSMIN reporting guideline for studies on measurement properties of patient-reported outcome measures. Quality of Life Research, 30(8), 2197–2218. https://doi.org/10.1007/s11136-021-02822-4CrossRefPubMed

16.

Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. (2010). The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiology, 63(7), 737–745. https://doi.org/10.1016/j.jclinepi.2010.02.006CrossRefPubMed

17.

Ericsson, K. A. (1992). Protocol analysis: Verbal reports as data (Rev. ed.). MIT Press.

18.

Lundgren-Laine, H., & Salantera, S. (2010). Think-aloud technique and protocol analysis in clinical decision-making research. Qualitative Health Research, 20(4), 565–575. https://doi.org/10.1177/1049732309354278CrossRefPubMed

19.

Hutchinson, C., Milte, R., Stanley, M., Duff, G., & Ratcliffe, J. (2022). Using discrete choice experiments to elicit the service preferences of people with mild intellectual disability: An exploratory study. Health and Social Care in the Community, 30(4), e1396–e1405. https://doi.org/10.1111/hsc.13547CrossRefPubMed

20.

Bolle, S., Romijn, G., Smets, E. M. A., Loos, E. F., Kunneman, M., & van Weert, J. C. M. (2016). Older cancer patients’ user experiences with web-based health information tools: A think-aloud study. Journal of Medical Internet Research, 18(7), e208–e349. https://doi.org/10.2196/jmir.5618CrossRefPubMedPubMedCentral

21.

Scott, K., Ummer, O., & LeFevre, A. E. (2021). The devil is in the detail: Reflections on the value and application of cognitive interviewing to strengthen quantitative surveys in global health. Health Policy and Planning, 36(6), 982–995. https://doi.org/10.1093/heapol/czab048CrossRefPubMedPubMedCentral

22.

Wright, J., Moghaddam, N., & Dawson, D. L. (2021). Cognitive interviewing in patient-reported outcome measures: A systematic review of methodological processes. Qualitative Psychology, 8(1), 2–29. https://doi.org/10.1037/qup0000145CrossRef

23.

Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71(2), 287–311. https://doi.org/10.1093/poq/nfm006CrossRef

24.

Boeije, H., & Willis, G. (2013). The cognitive interviewing reporting framework (CIRF). Methodology, 9(3), 87–95. https://doi.org/10.1027/1614-2241/a000075CrossRef

25.

Peterson, C. H., Peterson, N. A., & Powell, K. G. (2017). Cognitive interviewing for item development: Validity evidence based on content and response processes. Measurement and Evaluation in Counseling and Development, 50(4), 217–223. https://doi.org/10.1080/07481756.2017.1339564CrossRef

26.

Balza, J. S., Cusatis, R., McDonnell, S. M., Basir, M. A., & Flynn, K. E. (2022). Effective questionnaire design: How to use cognitive interviews to refine questionnaire items. Journal of Neonatal-Perinatal Medicine, 15(2), 345–349. https://doi.org/10.3233/NPM-210848CrossRefPubMedPubMedCentral

27.

DeMuro, C. J., Lewis, S. A., DiBenedetti, D. B., Price, M. A., & Fehnel, S. E. (2012). Successful implementation of cognitive interviews in special populations. Expert Review of Pharmacoeconomics & Outcomes Research, 12(2), 181–187. https://doi.org/10.1586/erp.11.103CrossRef

28.

Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(1), 143. https://doi.org/10.1186/s12874-018-0611-xCrossRefPubMedPubMedCentral

29.

Hyde, M., Wiggins, R. D., Higgs, P., & Blane, D. B. (2003). A measure of quality of life in early old age: The theory, development and properties of a needs satisfaction model (CASP-19). Aging & Mental Health, 7(3), 186–194. https://doi.org/10.1080/1360786031000101157CrossRef

30.

Bailey, C., Kinghorn, P., Orlando, R., Armour, K., Perry, R., Jones, L., & Coast, J. (2016). ‘The ICECAP-SCM tells you more about what I’m going through’: A think-aloud study measuring quality of life among patients receiving supportive and palliative care. Palliative Medicine, 30(7), 642–652. https://doi.org/10.1177/0269216315624890CrossRefPubMed

31.

Holmberg, C., Karner, J. J., Rappenecker, J., & Witt, C. M. (2014). Clinical trial participants’ experiences of completing questionnaires: A qualitative study. British Medical Journal Open, 4(3), e004363. https://doi.org/10.1136/bmjopen-2013-004363CrossRef

32.

Horwood, J., Sutton, E., & Coast, J. (2013). Evaluating the face validity of the ICECAP-O capabilities measure: A “Think Aloud” study with hip and knee arthroplasty patients. Applied Research in Quality of Life, 9(3), 667–682. https://doi.org/10.1007/s11482-013-9264-4CrossRef

33.

van Leeuwen, K. M., Jansen, A. P., Muntinga, M. E., Bosmans, J. E., Westerman, M. J., van Tulder, M. W., & van der Horst, H. E. (2015). Exploration of the content validity and feasibility of the EQ-5D-3L, ICECAP-O and ASCOT in older adults. BMC Health Services Research, 15, 201. https://doi.org/10.1186/s12913-015-0862-8CrossRefPubMedPubMedCentral

34.

van Leeuwen, K. M., Bosmans, J. E., Jansen, A. P., Rand, S. E., Towers, A. M., Smith, N., Razik, K., Trukeschitz, B., van Tulder, M. W., van der Horst, H. E., & Ostelo, R. W. (2015). Dutch translation and cross-cultural validation of the Adult Social Care Outcomes Toolkit (ASCOT). Health and Quality of Life Outcomes, 13, 56. https://doi.org/10.1186/s12955-015-0249-xCrossRefPubMedPubMedCentral

35.

Hill, N. L., Mogle, J., Whitaker, E. B., Gilmore-Bykovskyi, A., Bhargava, S., Bhang, I. Y., Sweeder, L., Tiwari, P. A., & Van Haitsma, K. (2019). Sources of response bias in cognitive self-report items: “Which Memory Are You Talking About?” The Gerontologist, 59(5), 912–924. https://doi.org/10.1093/geront/gny087CrossRefPubMed

36.

Gustafsson, S., Horder, H., Hammar, I. O., & Skoog, I. (2018). Face and content validity and acceptability of the Swedish ICECAP-O capability measure: Cognitive interviews with 70-year-old persons. Health Psychology Research, 6(1), 6496. https://doi.org/10.4081/hpr.2018.6496CrossRefPubMedPubMedCentral

37.

Meyer, R., Drewniak, D., Hovorka, T., & Schenk, L. (2019). Questioning the questionnaire: Methodological challenges in measuring subjective quality of life in nursing homes using cognitive interviewing techniques. Qualitative Health Research, 29(7), 972–986. https://doi.org/10.1177/1049732318812042CrossRefPubMed

38.

Margot-Cattin, I., Kuhne, N., Kottorp, A., Cutchin, M., Öhman, A., & Nygård, L. (2019). Development of a questionnaire to evaluate out-of-home participation for people with dementia. The American Journal of Occupational Therapy, 73(1), 7301205030p7301205031–7301205030p7301205010. https://doi.org/10.5014/ajot.2019.027144CrossRef

39.

Phillipson, L., Smith, L., Caiels, J., Towers, A.-M., & Jenkins, S. (2019). A cohesive research approach to assess care-related quality of life: Lessons learned from adapting an easy read survey with older service users with cognitive impairment. International Journal of Qualitative Methods. https://doi.org/10.1177/1609406919854961CrossRef

40.

Stephan, A., Marques, M. J., Bieber, A., Machado, A. S., Balsinha, C., Handels, R., Makai, P., Gonçalves-Pereira, M., & Dichter, M. N. (2021). Self- and proxy-rating of the ICECAP-O for people with dementia: A cross-sectional linguistic validation study in Germany and Portugal. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen, 162, 24–31. https://doi.org/10.1016/j.zefq.2021.02.008CrossRefPubMed

41.

Rohr, M., Brandstetter, S., Plomer, A. S., Loss, J., Kretschmer, R., & Apfelbacher, C. (2021). A qualitative study exploring content validity and feasibility of frequently used generic health-related quality of life measures in older people with hip fracture: The patients’ perspective. Injury, 52(2), 134–141. https://doi.org/10.1016/j.injury.2020.09.061CrossRefPubMed

42.

Penton, H., Dayson, C., Hulme, C., & Young, T. (2022). A qualitative investigation of older adults’ conceptualization of quality of life and a think-aloud content validation of the EQ-5D-5L, SF-12v2, Warwick Edinburgh Mental Wellbeing Scale, and Office of National Statistics-4. Value in Health, 25(12), 2017–2027. https://doi.org/10.1016/j.jval.2022.04.1735CrossRefPubMed

43.

Wilson, R., Cuthbertson, L., Kazis, L., & Sawatzky, R. (2022). Measuring health status in long-term residential care: Adapting the Veterans RAND 12 Item Health Survey (VR-12(c)). Clinical Gerontologist, 45(3), 562–574. https://doi.org/10.1080/07317115.2020.1752347CrossRefPubMed

44.

Lay, K., Crocker, M., Engel, L., Ratcliffe, J., Charlton, S., & Hutchinson, C. (2023). How do older adults receiving aged care services understand and respond to the EQ-5D-5L? A think-aloud study in residential care. Quality of Life Research. https://doi.org/10.1007/s11136-023-03466-2CrossRefPubMedPubMedCentral

45.

Ploughman, M., Austin, M., Stefanelli, M., & Godwin, M. (2010). Applying cognitive debriefing to pre-test patient-reported outcomes in older people with multiple sclerosis. Quality of Life Research, 19(4), 483–487. https://doi.org/10.1007/s11136-010-9602-zCrossRefPubMed

46.

Lay, K., Crocker, M., Engel, L., Ratcliffe, J., Milte, R., & Hutchinson, C. (2024). Exploring older people’s understanding of the QOL-ACC, a new preference-based quality-of-life measure, for quality assessment and economic evaluation in aged care: The impact of cognitive impairment and dementia. Health and Quality of Life Outcomes, 22(1), 4. https://doi.org/10.1186/s12955-023-02222-xCrossRefPubMedPubMedCentral

47.

Engel, L., Kosowicz, L., Bogatyreva, E., Batchelor, F., Devlin, N., Dow, B., Gilbert, A. S., Mulhern, B., Peasgood, T., & Viney, R. (2023). Face validity of four preference-weighted quality-of-life measures in residential aged care: A think-aloud study. Patient, 16(6), 655–666. https://doi.org/10.1007/s40271-023-00647-6CrossRefPubMedPubMedCentral

48.

Willis, G. B. (2015). Analysis of the cognitive interview in questionnaire design. Oxford University Press.

49.

Schreier, M. (2012). Qualitative content analysis in practice (1st ed., pp. 1–280). SAGE Publications.CrossRef

50.

Antonovsky, A. (1979). Health, stress, and coping (1st ed.). Jossey-Bass Publishers.

51.

Tourangeau, R. (2000). The psychology of survey response. Cambridge University Press.CrossRef

52.

Willis, G. B. (2005). Cognitive interviewing : A tool for improving questionnaire design. Sage Publications.CrossRef

53.

Hall, S., & Beatty, S. (2014). Assessing spiritual well-being in residents of nursing homes for older people using the FACIT-Sp-12: A cognitive interviewing study. Quality of Life Research, 23(6), 1701–1711. https://doi.org/10.1007/s11136-014-0627-6CrossRefPubMed

54.

O’Shea, E., Hopper, L., Marques, M., Goncalves-Pereira, M., Woods, B., Jelley, H., Verhey, F., Kerpershoek, L., Wolfs, C., de Vugt, M., Stephan, A., Bieber, A., Meyer, G., Wimo, A., Michelet, M., Selbaek, G., Portolani, E., Zanetti, O., Irving, K., & Actifcare, C. (2020). A comparison of self and proxy quality of life ratings for people with dementia and their carers: A European prospective cohort study. Aging & Mental Health, 24(1), 162–170. https://doi.org/10.1080/13607863.2018.1517727CrossRef

55.

Andrieu, S., Coley, N., Rolland, Y., Cantet, C., Arnaud, C., Guyonnet, S., Nourhashemi, F., Grand, A., Vellas, B., PLASA group. (2016). Assessing Alzheimer’s disease patients’ quality of life: Discrepancies between patient and caregiver perspectives. Alzheimer’s & Dementia, 12(4), 427–437. https://doi.org/10.1016/j.jalz.2015.09.003CrossRef

56.

Aujla, N., Vedhara, K., Walker, M., & Sprigg, N. (2020). Evaluating a stroke-specific version of the Illness Perception Questionnaire-Revised, using the Think-Aloud method. Journal of Health Psychology, 25(12), 1989–2005. https://doi.org/10.1177/1359105318781942CrossRefPubMed

57.

Leighton, J. P. (2021). Rethinking think-alouds: The often-problematic collection of response process data. Applied Measurement in Education, 34(1), 61–74. https://doi.org/10.1080/08957347.2020.1835911CrossRef

58.

Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859–883. https://doi.org/10.1037/0033-2909.133.5.859CrossRefPubMed

59.

Muehlhausen, W., Byrom, B., Skerritt, B., McCarthy, M., McDowell, B., & Sohn, J. (2018). Standards for instrument migration when implementing paper patient-reported outcome instruments electronically: Recommendations from a qualitative synthesis of cognitive interview and usability studies. Value in Health, 21(1), 41–48. https://doi.org/10.1016/j.jval.2017.07.002CrossRefPubMed

60.

Ng, S. L., Mylopoulos, M., Kangasjarvi, E., Boyd, V. A., Teles, S., Orsino, A., Lingard, L., & Phelan, S. (2020). Critically reflective practice and its sources: A qualitative exploration. Medical Education, 54(4), 312–319. https://doi.org/10.1111/medu.14032CrossRefPubMed

Titel: Cognitive interviewing for assessing the content validity of older-person specific outcome measures for quality assessment and economic evaluation: a scoping review
Auteurs: Kiri Lay
Claire Hutchinson
Jia Song
Rachel Milte
Jyoti Khadka
Julie Ratcliffe
Publicatiedatum: 03-12-2024
Uitgeverij: Springer New York
Gepubliceerd in: Quality of Life Research / Uitgave 3/2025
Print ISSN: 0962-9343
Elektronisch ISSN: 1573-2649
DOI: https://doi.org/10.1007/s11136-024-03849-z

Andere artikelen Uitgave 3/2025

‘The disease ruined my life’: a qualitative study of the quality of life in women with uterine fibroids

Health-related quality of life among extrapulmonary tuberculosis patients and inequalities by disease manifestations: a longitudinal study analysing the impact of TB treatment

Open Access

Myelofibrosis symptom assessment form total symptom score version 4.0: measurement properties from the MOMENTUM phase 3 study

Open Access

Opening the contextual black box: a case for idiographic experience sampling of context for clinical applications

Commentary

Trajectories of dyadic quality of life in young to middle-aged stroke couples: a longitudinal study

Quality of life among obstructive sleep apnoea patients: a systematic review and meta-analysis of EQ-5D studies

Review

Bohn Stafleu van Loghum

Welkom bij THIM Hogeschool voor Fysiotherapie & Bohn Stafleu van Loghum

Registreer

Login

Cognitive interviewing for assessing the content validity of older-person specific outcome measures for quality assessment and economic evaluation: a scoping review

Abstract

Purpose

Methods

Results

Andere artikelen Uitgave 3/2025

‘The disease ruined my life’: a qualitative study of the quality of life in women with uterine fibroids

Health-related quality of life among extrapulmonary tuberculosis patients and inequalities by disease manifestations: a longitudinal study analysing the impact of TB treatment

Myelofibrosis symptom assessment form total symptom score version 4.0: measurement properties from the MOMENTUM phase 3 study

Opening the contextual black box: a case for idiographic experience sampling of context for clinical applications

Trajectories of dyadic quality of life in young to middle-aged stroke couples: a longitudinal study

Quality of life among obstructive sleep apnoea patients: a systematic review and meta-analysis of EQ-5D studies

Bohn Stafleu van Loghum

Welkom bij THIM Hogeschool voor Fysiotherapie & Bohn Stafleu van Loghum

Registreer

Login

Deel dit onderdeel of sectie (kopieer de link)

Abstract

Purpose

Methods

Results

Log in om toegang te krijgen

‘The disease ruined my life’: a qualitative study of the quality of life in women with uterine fibroids

Health-related quality of life among extrapulmonary tuberculosis patients and inequalities by disease manifestations: a longitudinal study analysing the impact of TB treatment

Myelofibrosis symptom assessment form total symptom score version 4.0: measurement properties from the MOMENTUM phase 3 study

Opening the contextual black box: a case for idiographic experience sampling of context for clinical applications

Trajectories of dyadic quality of life in young to middle-aged stroke couples: a longitudinal study

Quality of life among obstructive sleep apnoea patients: a systematic review and meta-analysis of EQ-5D studies