Recalling what we thought we knew about recall periods: a qualitative descriptive study of how adults diagnosed with cancer use recall periods for patient-reported outcome items about physical function

Physical function (PF) (see Box 1) is a recommended outcome used to evaluate treatment benefit and burden of symptoms and side effects in cancer studies [1].

Box 1 Definition of physical function [2].

While PF can be evaluated using different types of outcome assessments, patient-reported outcome measures (PROMs) allow for measurement based on direct report by the patient. PROMs often use a recall period, or the amount of time the respondent is asked to think back in order to answer the questions (e.g., “last 24 h,” “past 7 days”), while other PROM items have no recall period specified. Depending on the context of use for the PROMs and endpoint specification, the choice of recall period should consider a number of factors, such as the recency of the outcome of interest, rate of change in the phenomenon, complexity of the information being retrieved, and how usual or unusual the phenomenon is (e.g., walking to a bus stop, running a marathon) [3]. Recall periods that do not align with the context of use can introduce measurement error that influences the ability to detect a treatment effect.

Previous research has identified at least 13 PF-specific PROMs that could be used in cancer populations [4]. Three widely used PROMs in cancer clinical trials are the Medical Outcomes Study Short Form 36 (SF-36) [5], European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ-C30) [6] and the Patient-Reported Outcomes Measurement Information System® (PROMIS®) PF items [7, 8]. The SF-36 PF asks patients about current physical health status; for example, how their “health now limits you.” The QLQ-C30 does not specify a recall period, asking patients, “Do you have any trouble…” The PROMIS® PF Item Bank version 2.0 [8] does not include a specific recall period, and ask patients “Are you able …” or “Does your health now limit you …” However, an alternative PROMIS PF short form specifically for adults with cancer was recently developed with a 7-day recall period [9]. Other PF measures, such as the World Health Organization Disability Assessment Schedule 2.0 [10] and the Stanford Health Assessment Questionnaire-Disability Index (HAQ-DI) [11] have 30-day and 7-day recall periods, respectively.

Patients receiving anti-cancer treatment may have rapid declines (e.g., stem cell transplant), slow declines (e.g., sarcopenia, cumulative toxicity), increases (e.g., steroids), or cyclical changes (e.g., chemotherapy) in their PF. Changes in PF status within the recall window may influence patients’ certainty about their responses. Accurately capturing patients’ PF includes ensuring that the recall period is long enough to capture potential shifts but not too long that patients have difficulty remembering PF activities.

Some quantitative studies have explored recall periods for PF PROMs. One study tested PROMIS PF items with no specified recall period, a 24-h recall period, and a 7-day recall period [12]. The authors concluded that compared to no specified recall period, the use of a recall period has little or no effect on PROMIS PF responses or scores. In another study, researchers compared a 7-day recall with no specified recall on the PROMIS PF Short Form 8c (PF8c) [9] and concluded that no specified recall period was preferred because there was not a significant effect on PF8c responses based on the recall presented. Yet, we do not know which recall period patients actually use when describing their PF, irrespective of whether a recall period is specified. If, as the previous studies suggest, there is no interpretational difference between the results of a specific recall (e.g., 7-day) or no specified recall within a PROM, we do not know why patients may have used a different recall period than what was presented in the PROM. Therefore, knowing which recall period patients are confident in reporting their PF is essential information to minimizing measurement error.

As part of the Patient Reports of Physical Functioning Study (PROPS) research program, we conducted qualitative interviews with adults diagnosed with cancer to understand their response processes when answering PROM items about PF. We had four sub-goals: (1) describe the timeframe adults diagnosed with cancer naturally used to report when asked about their current PF; (2) summarize how adults describe changes in PF over time; (3) examine the time window adults consider when answering questions posed with different recall periods and what recall period they actually use; and (4) describe the retrieval strategies adults employ when answering questions posed with different recall periods.

This study explored patients’ response processes when answering PROM items about PF with and without a specified recall period. When talking naturally about their PF status, most participants recalled their functioning over 1 month or longer. Findings were different between natural conversation and cognitive testing; when no recall period was offered for items tested in the cognitive interview, slightly more than half the time participants used a recall of 1 week or less. When asked open-ended questions about their PF over the past week, participants found it easy to describe changes (or lack of) in their PF over the prior week. Some participants indicated that their PF fluctuated due to treatment or pain. When answering PROM questions during the cognitive interview, they used a 1-month recall period less often than when describing PF naturally (during concept elicitation). When participants were not offered a recall period in PROM items, most responses focused on the last time they tried/completed the activity in question. For PROM items with a 7-day recall period, almost half of the retrieval strategies used the recall period offered, but the remainder focused on other issues such as the last time they completed an activity or treatment.

While a previous study showed that participants responded similarly for either no recall or 7-day recall [12], our study illuminates the variability in participant retrieval strategies and actual recall periods used for PROM items including no specified recall or 7-day recall. The primary implication of our findings is that offering a 7-day recall period helps to standardize how far back in time participants retrieve information (i.e., there is more consistency in the recall period used); yet even when prompted with a 7-day recall period, almost a third of responses relied on a longer recall period. One possible reason for using a longer recall period is that participants followed conventions of conversation by trying to guess what the interviewer truly wants to know and then adjusting their response strategies to be as helpful as possible [22]. For example, the participant might know that the prior 7 days were unusual (e.g., took a multi-day road trip that involved mostly sitting) and so used a longer period to give a more accurate picture of their PF under usual circumstances. This explanation is also consistent with our findings concerning differences between inpatient and outpatient participants (i.e., inpatients tended to use shorter recall periods corresponding to their recent hospitalization) and the strategy they used in our initial open-ended queries about their physical PF (i.e., participants tended to report PF since a salient clinical event that might have led to a change in their PF).

Specifying a 7-day recall helps but does not ensure consistency among participants in their recall periods, which—as an anonymous reviewer pointed out—raises the question of how much consistency in the use of a recall period is sufficient for clinical research. At this time, there appears to be no standard or agreed-upon threshold for how consistent patients need to be in the use of recall periods. The issue might be similar to that of missing data, where the impact of missing data depends on the mechanism of missingness. Some deviations from the specified recall period could reflect random measurement error if respondents’ deviations are due to unsystematic missingness, such as random lack of attention or discipline in responding. More of this type of inconsistency may result in attenuated associations with other variables and/or lower statistical power. Other, more systematic, inconsistencies in recall period used might reflect conscious decisions by the respondent to use a recall period that they believe is aligned with what the researcher wants to know; for example, choosing a shorter recall period that reflects their time as an inpatient on the assumption that the researcher wants to interpret the scores to assess how the patient is now. If the respondent’s assumption about the researcher’s intended interpretation of the scores is correct, then the deviation may not bias the researcher’s conclusions from the data. If the respondent’s assumption is wrong, it might contribute bias. More work is needed to understand the implications of both the extent and types of deviations from the specified recall period.

An assumption that investigators make with any recall period, is that study participants are using the recall period presented. Our qualitative study is unique in that we assessed if this assumption was true. Many quantitative studies have assessed the difference between PRO scores when patients are presented with multiple daily assessments versus a single assessment with a 7-day recall, for example [23]. In these cases, the shorter recall is widely considered to be the most accurate. The quantitative studies address the question: Are the responses to multiple assessments with a daily recall and a single assessment with #-day recall similar enough so that we can use the single assessment with longer recall? A threshold could be chosen a priori to determine if there is an “acceptable” agreement between responses when different recall periods are used. However, the quantitative work published thus far does not provide a threshold for how consistent patients need to be in recall period use. This qualitative study, and previous quantitative studies, are asking different questions. Our qualitative study is focused on which recall period people report using, rather than how similar are the responses to the PROM items if we ask them with different recall periods. Our study opens the door for us to be curious about how often patients use the recall periods offered, and what risks to validity may be at play in different contexts of use given the potential uncertainty of recall periods being used. Multi-item PF PROMs ask participants about a range of activities. Many participants linked retrieval of their PF to the last time they tried an activity. Future research may be needed to explore whether patients use different recall periods for different activities, potentially influencing the recall period used by patients. Our study was limited in asking only about an activity that was relevant to the participants. Therefore, our data do not extend beyond items addressing activities they conducted less frequently. For activities that are not being completed often, these activities arguably should not be asked about anyway—qualitative studies assessing the content of the PROM items would help us avoid this issue.

Other limitations of this study include the inability to investigate phenomena that might influence recall, such as present-state bias and, potentially, response shift. The contiguous order in which PROM items were presented (first typical recall during concept elicitation, then no recall and finally a 7-day recall) during the cognitive interviews may have influenced which recall period participants used during the latter section of the interview. Another limitation is that participants were recruited from a single medical center. In addition, results are limited to sample-level conclusions, and individual-level patterns for recall strategy were not assessed. Cognitive interviews focused on one activity that was relevant to each participant. This was a strength in that it ensured relevance, but it was also a limitation because we were unable to examine recall periods for activities that participants did not do. This study also lacked representation of participants who were not English-speaking within and outside the United States.

A strength of this study is that we captured participants’ actual recall period and their strategies for retrieving answers. Such data reveal how PF question format can be improved in the future to ensure more accurate answers. Another strength is that we included a substantial proportion of inpatients, which may be an infrequent but highly important period in patients’ lives, particularly related to physical functioning. Results differed slightly between inpatients and outpatients; we chose to include inpatients because research in this group has been limited.

Our results show that presenting a 7-day recall period leads to more consistent retrieval strategies across adults diagnosed with cancer, in comparison with PROM items that do not use a recall period. Investigators should also consider their target population by setting (inpatient versus outpatient), disease activity, and effects of treatment when selecting the appropriate PROM recall period. Some researchers have tried strategies to improve retrieval, including using video recordings of participants at baseline [24]. Future research could test different strategies for improving the accuracy of recall.