The influence of mode of remote delivery on health-related quality of life outcome measures in British Sign Language: a mixed methods pilot randomised crossover trial

Deaf Sign language users often identify themselves as being culturally Deaf and are part of the Deaf community, seeing their language use as a source of identity not an adaptation to hearing loss. Signed languages are visual languages, grammatically distinct from spoken languages; in the UK, British Sign Language (BSL) [1]. A systematic review on health outcomes of Deaf signing populations worldwide has highlighted inequalities in both mental and physical health outcomes for Deaf people in comparison with general populations [2]. For example, Deaf people experience poorer health status overall [3], higher rates of depression/anxiety [4, 5], higher rates of obesity [6] and, in one study, were diagnosed at a more advanced stage of colorectal and prostate cancer [7] than the general population. In the UK, the context for this study, Deaf people experience inequities in accessing services compared to hearing populations [8] resulting in under- and late- diagnoses, poor management of chronic conditions and reduced healthcare choices. Accessible Deaf-specialist sign language primary mental/physical health services are not universally available in the UK. A key barrier in service delivery is the scarcity of healthcare professionals with the expertise and cultural competency to work with Deaf patients [9]. Consequently, Deaf patients living in areas without such professionals with the required expertise are more likely to experience healthcare inequity. In such circumstances, telemedicine has been proposed as a potential way forward to increase the availability of services in signed languages whether directly or through interpreters.

In the UK, the National Health Service (NHS) defines telemedicine as “the use of telecommunication and information technology for the purpose of providing remote health assessments and therapeutic interventions” [10]. For Deaf people this can include live interactions in a signed language such as remote psychological therapy [11], the provision of static information such as health literacy through videos in signed languages [12], and standardised assessments of health/psychological health in a signed language. These can be pre-recorded asynchronous assessments in signed languages [4, 13‐16] or directly delivered assessments by a professional who signs to a Deaf patient/service user in an online clinical session without an interpreter. While patients are strongly recommended to use online portals to complete assessments before their sessions with a clinician [17], there are no clear guidelines for those patients who are Deaf sign language users. In response to the Covid-19 pandemic, services were rapidly forced to accept this medium to deliver clinical, health and social care services which accelerated their use in mainstream services. Findings from the systematic review carried out by Rogers et al. [18] on the effectiveness of telemedicine intervention with Deaf people suggested that delivery of mental health services using telemedicine is feasible and that satisfaction is high (e.g. [11, 19]) but the impact of different modalities of delivery on outcomes is under researched.’Modality’ refers to both medium of delivery e.g. online, or in real life as well as to linguistic modality (spoken, written, signed). It also encompasses features of response such as asynchronous self-completion or synchronous live interaction.

In general population studies, a meta-analytic review examined the equivalence of scores between paper-and-pencil and computer administration when completing Patient-Reported Outcome Measures (PROMs) concluding they were equivalent [20]. However, no similar studies on effects of modality of administration and response have been carried out with regard to signed languages.

The objective of this pilot study was to explore the equivalence of two modes of remote delivery of health-related quality of life outcome measures (EQ-5D-5L BSL, EQ-VAS BSL, and CORE-10 BSL): in asynchronous pre-recorded assessment in BSL and synchronous, live, delivery of the same assessment in BSL (via videoconferencing). The intention was to use the data to determine the sample size required for a confirmatory study, should one be needed.

This is a randomised crossover experimental study with a subsequent nested qualitative exploratory study to assist with interpretation of results. A randomised crossover design was chosen for this study instead of the more traditional randomised parallel-group design because it was important to evaluate both modalities using the same participant. It required fewer participants than a parallel trial as each participant acted as their own control, and the risk of a carry-over effect was small. Participants were randomly allocated to one of two sequences of assessment. Mode A followed by Mode B (Group one = AB), or Mode B followed by Mode A (Group two = BA). Mode A delivered the assessment online, asynchronously, in BSL using pre-recorded test items, derived from the BSL test versions that had previously been tested for reliability and validity [15, 21]. Mode B also delivered the assessment online, but live. Random allocation was done in blocks (block sizes of 2 and 4) through https://​www.​sealedenvelope.​com/​. This block randomisation was carried out to ensure an equal size for each group. A sub-sample of the assessed group took part in individual semi-structured qualitative interviews to explore responses to the two modalities of assessment delivery and response that had been experienced in order to gather data to assist with the interpretation of the results of the experimental study.

Deaf people who use BSL in the UK general population were recruited in various ways including snowball sampling, mail outs, social media and adverts on the project website. All recruitment materials were in BSL (and written English). The inclusion criteria stated participants needed to be Deaf, BSL users, and aged 18 years-or above. A sample size of 30 was chosen based on the recommended rule of thumb to estimate a parameter in Lancaster et al. [22]. This sample size enables the determination of equivalence where the equivalence margin has a Cohen’s d effect size of 0.83 with 80% power and a 5% significance level. Additional power comes from the crossover study design. The data from this study are intended to be used to determine the sample size required for a confirmatory study, should one be needed. Thirty-three Deaf BSL users initially took part completing the first round of assessments but subsequently two withdrew, leaving a total number of thirty-one participants who completed all aspects of data collection (see Fig. 1).

The study received ethical approval from the University of Manchester Research Ethics Committee (ref. 2022–14080-23435). Following individual consent and randomisation, participants, were informed whether allocated to Group one (AB) or two (BA). The assessor, a Deaf signer (Author 1), was different from the Deaf signer on the asynchronous version. She was also the original designer of the BSL validated assessments used in Mode A. The live-signed test items (Mode B) were identical to those in the asynchronous version (Mode A). In neither Mode was an interpreter used. It was assumed that there would be little or no change in responses to the assessment over the one to two weeks between delivery Modes. Demographic information collected included age, gender, hearing status of parents, as well as a self-report of current difficulties (if any) with their physical health and/or mental health (see Table 1). The rules for completing the BSL assessments for both conditions were the same: question repetition was allowed but giving specific examples of what a question meant was not permitted. In Mode A, repetition meant the participant rewatched the recorded question again. In Mode B, the researcher signed the question again.

A purposively sampled sub-group of participants was invited to take part in a one-to-one semi-structured interviews to discuss their responses to the different modes of assessments in BSL. Diversity in sample selection was sought to cover a range of age, gender and sexuality with equal numbers in groups one and two. The aim of the interview was to explore preferences for mode of delivery having experienced both and why one might be more effective than another in their eyes. These data were designed to assist in the interpretation of the quantitative data, rather than to be a standalone qualitative study. Nine participants were invited, 8 responded and 8 were interviewed. Interviews were carried out online via Zoom in BSL by the researcher who carried out the assessment in Mode B and recorded with consent. Participants were sent in advance the topics that would be covered in the interview to permit them time to think prior to interview. These were presented as open questions covering: overall appraisal; pros of Mode A, pros of Mode B, cons of Mode A, cons of Mode B, additional or unexpected reflections. Interviews lasted an average of 9 min (range 6.5 to 15 min).

The equivalence of three outcome measures across two delivery modes was investigated: (i) EQ-5D-5L (BSL version), (ii) EQ-VAS (BSL version), and (iii) CORE-10 (BSL version). The EQ-5D-5L contains five dimensions of health (mobility, self-care, usual activities, pain/discomfort and anxiety/depression) and includes the standard EQ-VAS which asks the participants to rate their health on that day from 0 (‘the worst health state you can imagine’) to 100 (the best health state you can imagine’). The EQ-5D-5L index score, also known as utility value, ranges from 0 (representing death) to 1 (representing perfect health) and is commonly used in health economics to measure the valuation of health states. The reliability and validity of the EQ-5D-5L BSL are reported in the study by Rogers et al. [15]. The short version of the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-10) has ten items from the full version of CORE-OM that measures psychological distress [23]. The BSL version of the CORE-OM has been checked for its reliability and validity [21]. The CORE-6D is not a standalone measure but is a scoring based on six of the CORE-OM items to create a health utility scaling [24] which, like the EQ-5D ranges from a possible zero to 100. Whereas four of the five domains of the EQ-5D-5L are physical and the one (anxiety/depression) is psychological, only one of the six CORE-6D items, “I have been able to do most things I needed to” is arguably physical as well as psychological. Although the scores of the EQ-5D-5L and the CORE-6D encompass both quality of life/health utility scores, they have different foci. In this study, by permission of the CORE authors, the 14 items common to the CORE-10 and the CORE-6D scoring were presented without the other items of the CORE-OM respecting the greater time needed for BSL presentations of questionnaires. The focus of the qualitative semi-structured interviews are provided in the procedure section.

The mean age of participants was 49 years old (SD = 12.9; range: 31–76 years old). At least half of the participants were female, and majority (87.1%) were White British. Eleven had at least one Deaf parent, of whom ten use BSL. Sixteen participants reported currently experiencing mental health difficulties. All participants used BSL with 83.9% listing it as their ‘preferred’ language, 3 (9.7%) preferred Sign Supported English (SSE), and 2 (6.5%) preferred speech/lip-reading. All were happy to take the assessments in BSL. See Table 1 for the demographic information.

Table 2 shows the mean difference score and equivalence bounds for EQ-5D-5L BSL, EQ-VAS BSL, and CORE-10 BSL. The mean score for EQ-5D-5L BSL live and EQ-5D-5L BSL pre-recorded were similar (mean difference = 0.034 with 90% confidence interval of 0.015–0.051) and the EQ-VAS score 0.697 to 8.083. Both lie within the acceptable equivalence limit of − 0.08 to 0.08 for EQ-5D-5L index scores and equivalence bounds of − 10.00 to 1.00 for the EQ-VAS [28] (see Table 2). This suggests that the two modes of live and pre-recorded EQ-5D-5L BSL and EQ-VAS BSL assessments are equivalent.

The mean difference between the CORE-10 BSL live and CORE-10 BSL pre-recorded is 0.17, and the midpoint across mean live and pre-recorded is 10.145. Therefore, the delta 20% is 2.029, meaning the equivalence bounds are (− 2.029, 2.029). The 90% CIs (− 1.4065, 1.1775) for mean differences lies within this equivalence region (− 2.029, 2.029) which suggest the two modes are equivalent (see Fig. 2 for the equivalence margin and 90% confidence intervals for CORE-10 BSL).

The mean score of utility values for the EQ-5D-5L BSL and CORE-6D BSL for two modes are reported in Table 3. The mean scores are similar for both modes (mean utility values of 0.848 for EQ-5D-5L BSL (SD = 0.149) and 0.843 for CORE-6D BSL (0.116) for pre-recorded versions; and 0.814 for the live EQ-5D-5L BSL (SD = 0.160) and 0.856 for the live CORE-6D BSL (SD = 0.098).

For the pre-recorded mode, the Pearson’s correlation coefficient between EQ-5D-5L BSL utility value and CORE-6D BSL utility value is 0.650 which is significant (r = 0.650, n = 31, p < 0.001). A significant association was also found for the live mode between EQ-5D-5L BSL utility value and CORE-6D BSL utility value (r = 0.655, n = 31, p < 0.001).

Positive Pearson’s correlation was found between EQ-5D-5L BSL live and CORE-10 BSL live (r = − 0.408, n = 31, p < 0.023), and between EQ-5D-5L BSL live and CORE-6D BSL live (r = -0.586, n = 31, p < 0.001). For the pre-recorded mode, positive correlation was also reported for between EQ-5D-5L BSL pre-recorded and CORE-10 BSL pre-recorded and between EQ-5D-5L BSL pre-recorded and CORE-6D BSL pre-recorded (r = − 0.546, n = 31, p < 0.002 and r = − 0.714, n = 31, p < 0.001).

Additional sample size calculations for each of the outcomes for the EQ-5D-5L BSL, EQ-VAS BSL and CORE-10 BSL were calculated with the data provided. For each, this was determined as the sample size required for a paired t-test to detect equivalence within the pre-specified equivalence limits based on the observed standard deviation of the differences in outcomes between the two modes of delivery, with 90% power and a 5% significance level. The sample sizes determined were N = 7, N = 12 and N = 28 for EQ-5D-5L BSL, EQ-VAS BSL and CORE-10 BSL respectively. These are very low as the observed differences between the modes of delivery were small. It was determined that the sample size of 30 was adequate to address the aims of this study.

Participants’ demographic details are provided in Supplementary Table 1. All thought that both modes of assessment in BSL were clear, easy to understand and felt able to answer the questions. Neither mode was seen as disadvantaging them, but different pros and cons for each were identified as discussed below. No one mode emerged as preferable, however when asked to make a clear choice it was how safe the style of assessment made them feel, rather than mode of delivery per se that made a difference. Two participants reported a preference for pre-recorded with the main reasons being that they felt safer and less self-conscious when answering the questions. Four participants preferred the live mode as it felt more natural, allowed them to interact, and they felt safer because if they felt vulnerable, they could seek support. Two participants stated that they did not have a preference and thought both would be fine.