Predicting presenteeism using measures of health status

Rheumatoid arthritis (RA) is a fluctuating chronic inflammatory auto-immune condition that primarily causes stiffness and pain in joints and tendons of the hands and feet. It is the most common inflammatory auto-immune condition in the United Kingdom (UK) and if left untreated can cause permanent damage to joints leaving the individual disabled [19]. Typically, disease onset occurs before the age of 65 years old (the current retirement age in the UK) meaning that individuals are frequently affected during their working lifetime [20]. There is substantial evidence to suggest that RA is significantly associated with increases in presenteeism [21].

The relevant study population for this study was defined as adults who were currently in work and had a self-reported diagnosis of RA. A sample of adults (18 years and over) with RA who were currently working in full-time or part-time paid positions were invited to take part in the study. The study sample was identified and recruited using an internet panel provider (ResearchNow, now called Dynata; https://​www.​dynata.​com/​). A sample size of n = 500 was informed in line with published mapping studies listed on the Health economics research centre (HERC) database of mapping studies (version 7.0) [22].

Data were collected using a bespoke online survey. Ethical approval from the University of Manchester was granted (reference number: 16144). Informed consent was taken at the beginning of the survey before the participant completed the survey. Respondents were informed they could leave the survey at any time without providing a reason; however, it was also explained that once the participant clicked “submit” they would not be able to retrieve and withdraw their responses due to anonymisation. The survey collected data on each individual’s: demographics; job type (sedentary, light, medium and heavy) and employment status (full-time, part-time; employed or self-employed); disease severity, measured by the Routine assessment of patient index data three survey (RAPID3 [23]; medications; health status (EQ5D-5L and SF6D); and presenteeism, measured using the Work productivity activity impairment (WPAI) [24]. The WPAI was selected as the relevant measure of presenteeism for this study because it is recommended for use in patients with RA by the Outcomes measures in rheumatology group (OMERACT) [25], adopts a patient perspective, and is relatively short, thereby reducing participant burden. The WPAI asks: ‘During the past 7 days, how much did your rheumatoid arthritis affect your productivity whilst you were working?’. The WPAI records levels of presenteeism using a zero to ten Likert scale where zero indicates ‘RA had no effect on my work’ and ten indicates ‘RA completely prevented me from working’. The WPAI has been well tested for its validity and reliability both within RA and other chronic conditions [26, 27]. The EQ5D-5L and SF6D were transformed into index values using the relevant published algorithms available and acceptable for use during the analysis period of this study [28, 29].

Data analysis involved three stages in line with published recommendations for producing mapping algorithms [30, 31].

Spearman’s rank (r) correlation was used to measure the strength and direction between the measures of health status (EQ5D-5L/SF6D) and presenteeism (WPAI). The potential strength of the correlation was described by categories defined prior to the start of the study: very weak (r = 0 to 0.19); weak (r = 0.2 to 0.39); moderate (r = 0.40 to 0.59); strong (r = 0.6 to 0.79); and very strong (r = 0.8 to 1) [32]. If a sufficient correlation (defined as moderate or above) was identified between the EQ5D-5L [33] and/or SF6D [34] with WPAI [24] then those measures of health status would be taken forward and developed to form a mapping algorithm for presenteeism. Supplementary Appendix 1 describes the approach to understand the performance of the WPAI in this study sample in terms of reliability (internal consistency) measured using Cronbach’s alpha.

Potentially suitable regression methods for producing a mapping algorithm were defined prior to analysis of the data. The dependent variable was defined as the level of presenteeism (using WPAI) and the independent variables included a measure of health status (EQ5D-5L or SF6D) with covariates for age and gender. This study took a parsimonious approach to the inclusion of additional covariates to allow for wider applicability of a specified mapping algorithm; a method used in published algorithms [11, 35]. Age was collected in pre-defined age bands and gender (male; female) was treated as a dummy variable.

There are many potential regression models that can be used to generate a mapping algorithm. Published guidelines for developing a mapping algorithm state that the selection of model type depends on the characteristics of the dependent variable (categorical, ordinal, etc.) and its distribution [30]. Longworth and Rowen [30] explain the need to take into account the bi-model distribution of the EQ5D for algorithms attempting to predict EQ5D values. However, the focus of this study is to develop an algorithm that predicts levels of presenteeism and not utilities for the EQ5D. Presenteeism, measured using the WPAI, can take values from zero to ten, increasing by increments of one and typically exhibits a negative distribution skewed to the left (many zeros). No formal guidelines exist to inform the model type for predicting presenteeism, therefore five types of regression models were selected as potential candidates to develop the mapping algorithm: (1) Ordinary Least Squares (OLS); (2) Tobit; (3) Censored Least Absolute Deviation (CLAD); (4) Ordinal Logit (Ologit); (5) multi-variable logit (mlogit).

OLS models a linear relationship and assumes equal distance between values of the dependent variable; this is consistent with the interpretation of the levels (zero to ten) included in the WPAI. OLS is an unbounded regression model and may produce inconsistent estimators when dealing with censored (left or right) dependent variables [36]. Tobit models are a potentially useful alternative when data are censored. Tobit models allow the analyst to set upper and lower limits for the dependent variable, for example 0 ≤ y ≤ 10. Tobit models are highly sensitive to heteroscedasticity which can lead to inconsistent estimates and affecting the standard errors [37]. Therefore, the use of a CLAD model was explored because it is less sensitive to skewed data and is robust to heteroscedasticity but is also censored at a lower value of zero [38].

Ordinal logit regression models are used for its ability to predict an ordinal dependent variable, for which presenteeism, as measured by the WPAI, is in this study. Ordinal logit models estimate the cumulative probability of observing an outcome using specified explanatory variables. The multinomial logit model, a similar regression model to ordinal logit where it also uses cumulative probabilities to predict an outcome level, was selected for its ability to generate predictions across multiple outcome levels. The observed outcome of the WPAI may take one of multiple levels ranging from zero to ten.

Six model specifications (see Table 1) were run for each of the five regression models. In total, 60 potential mapping models were specified to test their ability to predict presenteeism. The EQ5D-5L and SF6D were incorporated into separate mapping models as: (1) index scores; and (2) dummy variables for each level of severity associated with each domain.

The Root mean square error (RMSE) was used as the metric from which to judge models relative ability to predict presenteeism; a lower RMSE reflects smaller prediction errors. The RMSE was selected as the measures of prediction accuracy because it is able to penalise to a greater extent those predictions that are further away from the actual observed value [39]. The RMSE is an appropriate measure of error where predicted levels of presenteeism that are further away from the actual are interpreted to be considerably worse compare to those that are closer to the true value. The Mean bias error (MBE) is used to estimate the average bias, under or over-prediction, of the model as defined by the sign (negative or positive) and may be used to inform measures to correct to the bias [40].

To calculate the RMSE and MBE for each model, the K-fold method was used to split the sample. There is no ‘gold standard’ method for selecting the most appropriate number of folds, however ten folds is common practice [41] and therefore K = 10 in this study. The RMSE results are reported using graphical plots and across quartiles of the WPAI’s range.

Spearman’s rank correlation suggested a strong and negative correlation between the WPAI and EQ5D-5L (r = − 0.64) and the WPAI and SF6D (r = − 0.60) providing evidence that, in theory, mapping algorithms could be produced using either of these measures of health status.

Table 3 presents information on the predictive ability (RMSE) of all the models ran to predict presenteeism using EQ5D-5L or SF6D data. The MBE for all models was zero indicating zero bias in the models. Overall, the models that used dummy variables for each of the domains of the EQ5D-5L and SF6D produced more accurate estimates compared with those that used the index score and typically, those models that used covariates (age and gender) also performed better compared with models that did not include covariates.

Table 3 reports the RMSE for each model. The model with the smallest RMSE (1.7858) was for the OLS model with SF6D dummy model with age and gender interacted (model 36). The model with the next smallest RMSE was the OLS model with EQ5D dummy model with age and gender interacted (model 33) which had a RMSE that was fractionally larger than model 36 (RMSE = 1.7859). The full algorithms for models 36 and 33 are presented in the supplementary appendix 4. The observed and predicted values of the two model specifications (33 and 36) are illustrated in Fig. 2. The graphical plots suggest the two mapping algorithms were able to predict presenteeism scores with some degree of accuracy. However, both models tended to over-predict levels of presenteeism at observed levels between zero and four and under-predict levels of presenteeism at observed levels of five and over.

The RMSE of models 33 and 36 were compared across the quartile of the range of the presenteeism scale (Table 4). Model 36 had lower RMSEs in three of the four quartiles suggesting that, overall, model 36 generates more accurate predictions of presenteeism compared with model 33.

To our knowledge, this is the second of two studies that have applied mapping algorithms to quantitatively link health status with a concept beyond health and is the first to apply mapping methods to predict levels of presenteeism. Prediction models using health status data for presenteeism are limited and have focussed their efforts on developing models using EQ5D-3L data [11, 12], whereas the study presented here is the first to develop a prediction model for presenteeism using EQ5D-5L and SF6D data.

There is strength in the results of this study because they are based on data collected from individuals who were still working with RA during the time of this study. The results capture the reality of working with RA including nuances such as the ability to adapt and manage a chronic condition. This is in direct contrast with the study design used by Krol and colleagues [11] where individuals were asked to imagine their levels of presenteeism given a specific health state. A potential reason why Krol and colleagues [11] did not find an strong relationship between health status and presenteeism is that individual who have no experience of working with a chronic condition have little understanding of the actual impact it may have on their ability to work. However, it must be noted that the results from this study are far from conclusive and an external validation study is needed to confirm confidence in the algorithms generated in this study.

The developed mapping algorithm must be understood in light of some limitations. Few observations for presenteeism at levels 9 and 10 (very severe levels of presenteeism) meant that the mapping algorithm struggled to predict these high levels of presenteeism (see Supplementary Appendix 3).

This preliminary study used a complete case analysis of a dataset comprising data from all completed surveys. We did not use multiple imputation methods to generate estimates of ‘missing’ data because the literature is currently unclear regarding how to combine multiple imputation within predictive modelling. Research into multiple imputation methods is currently very active with researchers exploring issues related to; the assumptions made when applying imputation methods [42], how to account for imputation uncertainty and its impact on subsequent statistical testing [43], and how model selection is affected after having applied multiple imputation methods [44]. Using a complete case analysis approach will not affect the observed estimated mapping algorithms but may affect the generalisability of the results.

Developing a prediction model based on few observations is not recommended, therefore, we considered the possibility of collapsing observed presenteeism levels eight, nine and ten to make one group. Ultimately, this approach was decided against where the primary purpose of the mapping algorithm was to enable a prediction of presenteeism at all levels. It may be the case that the lack of observations for very high levels of presenteeism (nine and ten) reflects the current health and work status of the individuals sampled in this study where those individuals who are able to continue working do so because they know they are, broadly, able to keep pace with their work and therefore report low, mild or moderate levels of presenteeism. Individuals who might report severe presenteeism may be struggling to remain productive at work and are potentially less likely to engage with studies such as this potentially making them a difficult subgroup to reach. Further research is required to study the characteristics of individuals who work with severe levels of presenteeism. Furthermore, the evidence presented in this study may potentially help towards an improved understanding of the differences between inter-individual levels of presenteeism; however, further research is needed to quantify absolute productivity losses. Potential new methods, such as the Productivity adjusted life years (PALYs), as discussed by Ademi et al. [45] aim to quantify productivity loss and incorporate productivity explicitly in cost-effectiveness studies. A mapping algorithm linked to PALY utilities may be useful, particularly to populate those datasets where PALY utilities have not been collected.

To promote the use of a mapping algorithm, it must be rigorously tested using an external dataset [30]. Unfortunately, and to our knowledge, there is no dataset that has SF6D, EQ5D-5L and WPAI data that can be used to externally validate the algorithms.

The mapping algorithms were developed using an RA population only. Further research is needed to understand whether the models could be used in: (1) populations with diseases similar to RA, for example ankylosing spondylitis; and (2) populations with any other form of chronic physical condition that makes working difficult, for example chronic pain.