Coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was first diagnosed in Wuhan, China, after a series of unexplained cases of severe pneumonia in December 2019 [1]. Despite important precautions that were taken, the virus spread rapidly worldwide and arrived in Belgium in February 2020. From the little experience and data of other countries, we knew that the presence of the infection was highly heterogeneous [2, 3]. The majority of the patients were asymptomatic or showed mild symptoms, but around 20% of the patients required hospitalization, of whom 5–10% became critically ill, requiring intensive care admission and mechanical ventilation, with still a high mortality rate [4].
Consequently, major concerns arose regarding bed and nurse capacity for all intensive care units (ICU) eligible patients during the first wave of the COVID-19 pandemic in Belgium. Due to the scarcity of guidelines for ICU admission and outcome data, mainly age and comorbidities were used as criteria for admission and decision making of coding. During the second wave of infections, with an even higher critical saturation of ICU beds and mortality rate, specific scoring systems to guide ICU admission or predict mortality for patients with SARS- -CoV-2 infection were still lacking.
There are multiple scoring systems in use on the ICU that can help assess organ dysfunction or failure and are useful to evaluate morbidity in critically ill patients. The Sequential Organ Failure Assessment (SOFA) score was initially designed in 1996 to sequentially assess and quantify the severity of organ dysfunction in patients who were critically ill from sepsis [5]. The SOFA uses six different variables, one for each of the major organ systems: respiratory, cardiovascular, liver, coagulation, kidneys, and central nervous system. The function of each organ system is scored from 0 to 4, with an increasing score reflecting worsening organ dysfunction [5, 6]. Although this scoring system was not developed to predict the outcome, an obvious relationship between organ failure and mortality has been demonstrated in several studies [6–8]. Therefore, it has been validated as a prognostic tool to predict mortality in severely ill patients [9, 10]. Additionally, the SOFA score is an increasingly important tool in evaluating the response to the change of therapy in the context of clinical trials [11].
This study aimed to evaluate the prognostic value of the SOFA score and evolution in SOFA score concerning ICU mortality in patients admitted to the ICU with severe SARS-CoV-2 infection.
METHODS
This mono-centric, investigator-initiated, longi-tudinal, retrospective, observational cohort study was performed at the ICU department of the Jessa Hospital, Hasselt, Belgium. This study was approved by the ethical committee of Jessa Hospital, Hasselt, Belgium on 4th January 2021 and registered at clinicaltrials.gov on September 30th, 2020 (NCT04713852). Written informed consent was waived in light of the urgent need to collect data on the ongoing pandemic. This study is reported according to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement [12].
Study population
All adult patients diagnosed with COVID-19 pneumonia and admitted to the ICU between 13th March 2020 and 17th October 2020 were included in the study. Following the World Health Organization (WHO) protocol [13], laboratory confirmation of COVID-19 infection was defined as a positive result on polymerase chain reaction (PCR) assays of nasopharyngeal swab samples or bronchoalveolar lavage. Only laboratory-confirmed patients were included in the analysis. From March 13th, 2020, until October 17th, 2020, data from 116 consecutive patients admitted to the ICU were prospectively entered into a customized database that included medical history, demographic data, clinical symptoms and signs, laboratory results, and clinical outcomes. Body temperature was measured with an electronic inguinal thermometer. A body temperature of 38°C was determined as the cut-off value for diagnosis of fever.
Multi-organ failure variables
The SOFA score was calculated on admission and every 24 hours until ICU discharge or death for every included patient. The SOFA score includes components reflecting the status of coagulation and the hepatic, respiratory, cardiovascular, central nervous, and renal organ systems. Each component score ranges from 0 (normal organ function) to 4 (worst organ function). The SOFA score is the sum of all organ scores resulting in a range from 0 to 24 [5]. The SOFA score for each patient on a given ICU day was the sum of the worst values for all component organ failures in the 24 hours. In cases where the physiological parameters did not match any row, zero points were given. In cases where the physiological parameters matched more than one row, the row representing the highest score was selected [11]. All parameters included in the SOFA scores such as bilirubin, platelets and serum creatinine were measured daily. However, when a value was exceptionally missing, the value of the previous day was used to calculate the SOFA score (last observation carried forward approach) [14]. All COVID-19 patients admitted to the ICU were included in the analysis and the type of respiratory support, i.e., endotracheal tube, high-flow nasal oxygen (HFNO), noninvasive ventilation (NIV) or nasal cannula, was documented as such. Almost all patients were treated at some stage in their care with a nasal cannula and HFNO, which deliver oxygen at variable inspired oxygen percentage or flow rates. For patients on nasal cannula oxygen, an estimated FiO2 was calculated for patients on a nasal cannula (O2 flow in litres per minute × 0.03 + 0.21) [11]. In sedated patients, we also applied ‘a last obser-vation carried forward approach’ [11]. In the rare cases where the Glasgow Coma Scale (GCS) from before the start of sedation was not available, a value of 15/15 was recorded and carried over throughout the duration of sedation administration [11].
The admission SOFA score is defined as the sum of the most severe value for each sub-score at admission to the ICU. The maximum SOFA score is the highest daily SOFA score observed during ICU stay [10]. The change in SOFA score during the first 24/48 hours is calculated as SOFA at day 2/3 minus SOFA at day 1 (admission). We refer to this change as delta SOFA from day 1 to day 2/3.
Outcome parameters
The primary endpoint is either ICU mortality or discharge from the ICU. Hence, the study population was divided into participants who died during their ICU stay and participants who were discharged from the ICU alive. The data set was closed on November 10th, 2020. All participants reached the primary outcome.
Explanatory variables
For the present study, a few variables were selected to test their predictive value for ICU mortality with multiple testing. First, it is widely recognized that mortality after SARS-CoV-2 infection increases exponentially in patients aged over 50 [15]. Second, in our hospital, also patients with a do-not-intubate (DNI) code were admitted to the ICU for high-flow oxygen therapy. Third, a therapy adjustment was implemented throughout the inclusion period: an intensified thromboprophylaxis protocol implemented on March 31st significantly reduced one-month mortality [16]. Thus, the time of admission (before or after 31.03.2020) was included as an explanatory variable.
Statistical analysis
Continuous data are shown as mean ± standard deviation (SD) and categorical data are represented as frequency (%). At ICU admission, Student’s t-test or the Mann-Whitney U test (in the case of non-normally distributed data) was used to compare ICU survivors and ICU non-survivors for continuous variables, and the χ2 test for categorical variables.
Linear mixed models for repeated measures were employed to compare the evolution of mean SOFA scores over time for ICU survivors and ICU non-survivors. For each patient, the SOFA scores of the first two weeks at the ICU were used. Day since ICU admission, group (ICU survivor or ICU non-survivor) and their interaction term were included as fixed effects. A non-linear evolution was allowed by including the quadratic term of day since ICU admission. The linear mixed model also included a random patient effect to take into account the correlation between SOFA scores of the same patient. Based on this model, changes within a group (survivor or non-survivor) and differences between the groups at specific time points or for differences in change were investigated. No correction of multiple testing was used.
The predictive value of the following SOFA variables for predicting ICU mortality was investigated using logistic regression models: SOFA score on day 1, the maximum SOFA score, combination of SOFA score on day 1, and change in SOFA score between days 1 and 2. The χ2 test was used to evaluate the statistical significance of these variables. These logistic models were extended by incorporating the following variables: age, time of admission (before or after 31.03.2020), and DNI status. For each model, the predictive significance of the variables, the OR (95% CI), and area under the receiver operating chara-cteristic (AUC) curve are given. The ROC curve, the calibration plot, and the Hosmer-Lemeshow test are presented for the multiple logistic models when the SOFA variable is statistically significant. A P-value < 0.05 is considered statistically significant. All ana-lyses were conducted with SAS software, version 9.4 of the SAS System for Windows.
RESULTS
The STROBE flowchart depicting inclusion and exclusion is presented in Figure 1. In total, 116 COVID-19 patients were admitted to the ICU from March 13th until October 17th, 2020. Patients admitted to the COVID ICU for reasons other than COVID-19 pneumonia (i.e., neurological trauma, diabetic ketoacidosis, complication after surgery) were excluded, resulting in 103 patients for the statistical analyses. The patients’ baseline characteristics are presented in Table 1. Mean age was 68.2 ± 11.3 years, ranging from 24 to 86 years for the total group. The majority of the patients were male (61.2%). A total of 30 patients (29%) died during their ICU stay and 73 (71%) patients were discharged alive. Arterial hypertension was the most common comorbidity in both groups (60.2% of all patients). Fever (80.6%), dry cough (79.6%), and dyspnoea (72.8%) were the most common symptoms on admission. Age was significantly associated with mortality (mean age 73.7 years versus 65.9 years in the survivor group; P-value 0.01). Other baseline variables associated with mortality were history of dementia (P = 0.02), fever at admission (P < 0.001), higher systolic (P = 0.02) and diastolic (P < 0.01) blood pressure at admission, lower P/F-ratio at admission (P < 0.01), and higher clinical frailty index (P = 0.01). In the total group, the mean SOFA score was 4.5 (SD 3.0) on admission. In the ICU survival group, the mean admission SOFA score was 4.3 (SD 2.9) versus 5.2 (SD 3.3) in the ICU non-survival group (P = 0.10). The distribution of SOFA scores at admission, stratified for ICU survival, is shown in Figure 2.
TABLE 1
Demographics and clinical characteristics at admission to ICU
[i] Data are presented as mean ± standard deviation or as number (%). Characteristics of ICU survivors and ICU non-survivors were compared with Student’s t-test or Mann-Whitney U test for continuous variables and with the χ2 test for categorical variables. A P-value < 0.05 is considered statistically significant. BMI – body mass index.
The mean maximum SOFA score for the ICU non-survivors was 11.7 (SD 4.7) compared to 7.4 (SD 4.3) for the survivors during their stay in the ICU (P < 0.001). The individual course of the SOFA scores for ICU survivors and ICU non-survivors during ICU stay until day 35 is shown in Supplemental Figure 1.
Figure 3 displays the evolution of SOFA scores in time of consecutive patients. Each line corresponds to one patient’s trajectory during their ICU hospitalization displayed until day 35 for those who survived and those who did not survive on the ICU.
FIGURE 3
Individual SOFA scores displayed in three categories with a colour code (≤ 4: green, 5–9: orange, ≥ 9: red, discharge from ICU: white, deceased: grey, light grey: missing SOFA score). A lower SOFA score indicates less organ dysfunction. The patients are plotted according to their admission day: the first admitted patient corresponds to the lowest line. The blue line represents 30.03.2020, as the day that an intensified thromboprophylaxis protocol was implemented in the hospital
The average evolutions of the daily worst SOFA scores for ICU survivors and ICU non-survivors, estimated by the linear mixed model, are depicted in Figure 4.
FIGURE 4
Average evolution of the daily worst SOFA scores stratified for survival (ICU survivors: blue and ICU non-survivors: red), estimated using a linear mixed model
On admission (day 1), the mean worst SOFA score did not differ between the two groups (difference = 0.60, P = 0.333), whereas from day 2, the difference in mean worst SOFA scores gradually increased between both groups (difference at day 2 = 1.18, P = 0.055). On day 3, the score of the ICU non-survivors is 1.76 points higher compared to the survivors (P < 0.01). This difference further increased to 4.08 on day 7 (P < 0.001). The figure demonstrates a rise in SOFA scores in the first week for the entire population; this rise is however less pronounced for the ICU survivors (an increase of 1.60 for survivors vs 5.08 for non-survivors, P < 0.001). For both groups, the change in SOFA score from day 1 to day 2 (first 24 hours) was statistically significant. For the ICU non-survivors there was an increase in SOFA score of 1.03 points (P < 0.001), for the survivors of 0.45 points (P < 0.001). The same significant rise was observed from day 1 to day 3, with an increase in SOFA score of 1.98 points for the non-survivors (P < 0.001) and 0.82 for the survivors (P < 0.001). Hence, the increase in SOFA score in the non-survivor group was significantly stronger, with an additional increase of 0.58 from day 1 to 2 (P < 0.001), 1.16 from day 1 to 3 (P < 0.001), and 3.48 from day 1 to 7 (P < 0.001).
The results of the logistic regression analyses with the SOFA variables as predicted variables are presented in Table 2. Only the maximum SOFA score is significant (OR = 1.23, 95% CI: 1.11–1.37, P < 0.001).
TABLE 2
Overview of logistic regression analyses with different SOFA variables
| OR (95% Ci) | AUC | P-value | |
|---|---|---|---|
| Baseline SOFA score | 1.11 (0.96–1.27) | 0.60 | 0.15 |
| Maximum SOFA score | 1.23 (1.11–1.37) | 0.76 | < 0.001 |
| Baseline SOFA score Delta SOFA score 24 h | 1.12 (0.97–1.31) 1.03 (0.83–1.28) | 0.64 | 0.13 0.77 |
This association remains significant after inclusion of age, DNI status and time of admission (before or after 31.03.2020) in the multiple logistic regression model (OR = 1.53, 95% CI: 1.23–1.89, P < 0.001) (Table 3).
TABLE 3
a) A multiple logistic regression model with the maximum SOFA as an explanatory variable corrected for age, DNI status, and time of admission. b) Multiple logistic regression model with the maximum SOFA as explanatory variable corrected for age and time of admission. The area under the curve equals 0.91
The ROC curve (AUC = 0.91) and calibration plot of this multiple logistic model are shown in Figure 5. The Hosmer-Lemeshow test indicates good calibration (P = 0.838).
DISCUSSION
In this cohort study including 103 consecutive patients admitted to the ICU with severe SARS- -CoV-2 infection and investigating the predictive value of serial evaluation of the SOFA score for ICU mortality, we first noted an increase in SOFA scores in the first week for the entire population, but less pronounced for the ICU survivors. Secondly, no significant difference in the worst SOFA score on admission day between ICU survivors and non-survivors was observed. Thirdly, a combination of SOFA score on admission day and delta SOFA score in the first 48 hours lost its predictive value for ICU mortality. Finally, the association of the maximum SOFA score with ICU mortality remained present after inclusion of age, DNI status, and change of therapy protocol to the predictive model. Moreover, this multiple logistic regression model can discriminate between patients with and without ICU mortality with an AUC of 0.91. The calibration curve, shown in Figure 5, also indicates good calibration of this model.
To date, several studies have investigated the predictive value of the SOFA scoring system for poor outcomes in a COVID-19 population [4, 13, 15–17]. Of those, five focused on patients admitted to the ICU with mixed results [4, 18–21].
The trend in SOFA score
A significant increase in SOFA scores in the first week was found in both the ICU survival and ICU non-survival groups. This increase was however less pronounced for ICU survivors. Based on an analysis of 18 patients, Martinez et al. [20] concluded that an increase in SOFA score in the first 48 hours after admission is associated with significant ICU mortality. This study was underpowered to draw firm conclusions on the association in the evolution of SOFA scores over time and ICU mortality. Bels et al. [4] concluded in a cohort study including 93 mechanically ventilated participants with SARS-CoV-2 infection that a decrease of SOFA scores over time was associated with ICU survival, even after adjustment for age, sex, and comorbidities. However, it has to be pointed out that in this study, patients were only admitted to the ICU when intubation and mechanical ventilation were essential, resulting in higher SOFA scores on admission. At our institution, patients were already admitted to the ICU when in need of high-flow oxygen therapy. Furthermore, Bels et al. [4] drew their conclusions from the analysis of the evolution of SOFA scores over more than 30 days. In our cohort, a decrease of SOFA scores in the ICU survival group was also noted from day 8.
Worst SOFA score on admission day and delta SOFA score in first 48 hours
The observed absence of a significant difference in worst SOFA score on admission day between ICU survivors and non-survivors in this study is in line with the observations of Bels et al. [4]. However, two studies investigating SOFA scores on admission to hospital both came to an opposite result: in a multicentre cohort study, Zhou et al. [11] detected an increased risk of in-hospital death with higher SOFA score on admission, older age, and D-dimer greater than 1 µg mL–1. Liu et al. [22] also found that a SOFA score on admission of ≤ 3 was highly predictive of survival in COVID-19 patients admitted to the hospital ward. These opposite results might be explained by the fact that SOFA scores on admission are high in both ICU survivors and non-survivors, resulting in an impaired discriminative power for mortality of this SOFA variable in an ICU population. Another multicentre cohort study concluded that higher renal and cardiovascular SOFA score components and lower PaO2/FiO2 ratio at ICU admission are independent predictors of 90-day mortality [21].
Evaluating and comparing the level of mean worst SOFA scores at ICU admission, and accordingly reflecting the severity of organ dysfunction of COVID-19 patients admitted to the ICU, may provide interesting insights. Martinez et al. [20] reported a mean SOFA score of 6 (± 4.8) at ICU admission in the ICU non-survivor group compared to 5.2 (± 3.3) in our study. Interestingly, Bels et al. [4] reported remarkably higher mean baseline SOFA scores for both ICU survivors (7.3 [SD 1.9]) and ICU survivors (8.6 [SD 2.8]) [3]. This difference is probably attributable to the limited availability of ICU beds in the Netherlands, which is only 6.4 compared to 15.9 per 100 000 inhabitants in Belgium [16], resulting in very restrictive admission criteria (only patients who require intubation and mechanical ventilation).
A combination of SOFA score on admission day and delta SOFA score in the first 48 hours was also not predictive for ICU mortality. In a non-COVID ICU population, however, an increase in SOFA score in the first 48 hours after ICU admission was highly effective in predicting ICU mortality [10]. Our results echo those of a large multicentre trial including 675 mechanically ventilated patients with SARS-CoV-2 pneumonia. This study also showed that the worst SOFA score within 48 hours before intubation was poorly predictive for mortality and was significantly inferior to simply using age [18].
Maximum SOFA score during ICU stay
In this study, the maximum SOFA score, or the worst SOFA score during the patient’s ICU stay, was strongly associated with ICU mortality. Its predictive power for ICU mortality remained intact after including age, DNI status, and change of therapy protocol in the predictive model. This model comprising maximum SOFA score, age, DNI status, and change of therapy protocol is highly effective in discriminating between patients with and without ICU mortality and also showed good calibration. In a non-COVID ICU population, the maximum SOFA score was also found to be highly predictive for poor outcomes [10]. Other studies evaluating the predictive value for poor outcome of the SOFA scoring system in a COVID-19 population did not analyse the maximum SOFA score [3, 10, 19, 20, 22, 23]. Data on the maximum SOFA score are only available for post-hoc analysis. Consequently, this SOFA variable cannot be incorporated in a predictive model for poor outcomes of critically ill SARS-CoV-2 patients that can support physicians in decisions concerning ICU admission at times of critical saturation of ICU capacity.
Because of the novelty of the disease and its clinical progression, there were some therapy adjustments throughout the inclusion period. Especially an intensified thromboprophylaxis protocol implemented on March 31st, including augmented low-molecular-weight heparin dosing, individually tailored with anti-Xa measurements and twice-weekly ultrasonography screening for deep venous thrombosis (DVT), significantly reduced one-month mortality [16]. The response to this change of therapy concerning trends in SOFA score is clear in Figure 3: SOFA scores above 10 were less often seen after March 31st. In contrast, recent evidence does not support the use of intermediate-dose venous thromboembolism (VTE) prophylaxis (INSPIRATION trial) [24]. The INSPIRATION trial was conducted in several hospitals in Iran between July and November 2020. In this trial, VTE events were only recorded in 19 (3.4%) patients enrolled and the risk of VTE was not significantly different between the intermediate-dose and standard-dose groups (3.3% vs. 3.5%) [24]. In contrast, several mono- and multicentre European first wave ICU cohorts reported incidences of above 40% of thrombotic complications [25–27]. These conflicting results may be explained by variability in disease severity and/or host immune response between different ethnic groups. The observed difference in COVID-19-associated hypercoagulability between European first wave cohorts and the INSPIRATION trial might also be explained by the genetic variation in the SARS-CoV-2 circulating in Europe in the first wave and Iran between July and November 2020. Finally, selection bias and/or detection bias may also explain the low observed incidence of VTE phenomena in the INSPIRATION trial, since 1200 out of 1800 patients were excluded from enrolment and VTE phenomena were not systematically assessed. This study also has some limitations. First, the interpretation of our findings might be limited by the small sample size with low statistical power and the single-centre approach. Second, interpretation of our findings might be further limited due to the retrospective design of this study. Also, a limited amount of laboratory tests was not available in all patients. Finally, since COVID-19 is a new disease with a lack of effective standard therapy, changes in therapy were made throughout the inclusion period which might have affected the patient outcome.
CONCLUSIONS
Evaluation of the SOFA scores in the first 48 hours after ICU admission is not a good prognostic indicator in COVID-19 patients. Only the maximum SOFA score was highly predictive of ICU mortality. Since these data are only available for posthoc analysis, we can conclude that the SOFA scoring system is not useful for the triage of critically ill COVID-19 patients.
