Adjusting EWS scores for altitude above sea level: is it necessary to predict sepsis mortality in the emergency room?

Abstract

Background

Sepsis mortality can be significantly reduced with early diagnosis and appropriate treatment. It is crucial to have tools that enable the early identification of patients at high risk of sepsis mortality from the triage stage. The National Early Warning Score (NEWS) and recently the International Early Warning Score (IEWS), are widely used for this purpose. However, its oxygenation parameters are primarily validated in populations at sea level. Given that patient oxygenation varies with altitude, there is a need to investigate the applicability of this scores at different altitudes. The purpose of this study is to compare the discriminatory capacity of sepsis mortality in emergency departments between the standard NEWS and IEWS scale and a NEWS and IEWS scale adjusted for barometric pressure in Bogotá, a city located 2600 m above sea level. A prospective recruitment was conducted in the triage area of the emergency department, including all patients with suspected sepsis. The scales under evaluation were calculated. Subsequently, the sensitivity, specificity, predictive values, and areas under the curve (AUC) of each scale were assessed for mortality prediction.

Results

A total of 304 patients were recruited. The overall mortality rate was 19.4% and the septic shock mortality rate was 59.3%. The AUC for the standard NEWS was 0.78 (95% CI: 0.72–0.83), and for the standard IEWS was 0.81(95% CI: 0.75–0.87), altitude-adjusted NEWS, it was 0.79(95% CI: 0.73–0.84), and for the altitude-adjusted IEWS was 0.82(95% CI: 0.76–0.88).

Conclusions

Adjustment of oxygen saturation for altitude above sea level in NEWS (NEWSa) does not improve its predictive capacity for mortality in patients with sepsis in the emergency department, however, this same adjustment in the IEWS value significantly improves the predictive capacity compared to NEWS and NEWSa.

Background

Sepsis accounted for approximately 20% of global deaths in 2017 [1]. Delayed diagnosis of sepsis in the emergency room is associated with delays in administering interventions, such as the initiation of antibiotics, which can contribute to increased mortality [2]. Despite the development of multiple treatment strategies, Since 2001, there have been no care protocols that have significantly impacted the reduction of sepsis mortality [3]. Therefore, there is a need to develop prediction tools and risk identification methods for sepsis-related mortality starting from the emergency room. This would enable earlier diagnosis and, consequently, positively impact the reduction of mortality from this disease.

Currently, there is no single recommended scale for predicting sepsis, septic shock, and sepsis-related mortality in the triage area of the emergency department. Despite this, the National Early Warning Score (NEWS) is one of the most widely used scales [4, 5]. Nevertheless, the National Early Warning Score (NEWS) is one of the most widely used scales, often employed alone or with variations in many regions around the world [6]. The scale was developed in the United Kingdom using retrospective patient data, and its discriminatory capacity has been successfully validated, even in some meta-analyses [7, 8].

The International Early Warning Score (IEWS) was recently validated in a multicenter study for the prediction of in-hospital death from the emergency department [9]. The validation study demonstrated that by including age and sex as predictive variables, the mortality discrimination capacity increases compared to the original NEWS [9]. However, this study was conducted in the Netherlands and Denmark, these countries are also at sea level [9].

NEWS and IEWS incorporate clinical variables such as oxygenation and the need for supplemental oxygen. A subsequent modification adjusted the oxygen saturation value for patients with hypercapnic respiratory failure [10]. The concentration of oxygen in the atmosphere depends on the altitude above sea level and the barometric pressure at the location. This results in a decrease in available atmospheric oxygen as altitude increases [11]. Thus, the oxygenation of sepsis patients behaves differently at high altitudes (above 1500 m) compared to sea level. Consequently, prediction scales should be adjusted according to the altitude at which they are applied, as the values used may overestimate the hypoxemia of the patients assessed. Therefore, this study aimed to evaluate whether adjusting the oxygen saturation variable in the NEWS and IEWS scales for altitude influences the ability of the scales to accurately predict sepsis-related mortality in a population of patients residing at high altitudes, specifically at 2600 m above sea level.

Additionally, given that lactate is a recognized factor associated with sepsis mortality [12], as an additional objective, this study proposes to evaluate the competitive advantages of adding arterial lactate levels to the measured values ​​of the EWS scales.

Methods

Study design

This is a prospective study to validate the ability of the EWS (Early Warning Score) scales to predict in-hospital mortality due to sepsis and compare it with the same scales adjusted to altitude (Early Warning Score adjusted). In addition, the study seeks to evaluate the discriminatory capacity of two new scales developed in this research: NEWSaL and IEWSaL.

In this study, we collected patient data prospectively and recorded them in the sepsis database of the Institute of Urgencies and Trauma (ISMET) of the Santa Fe Foundation of Bogotá, from June 2023 to March 2024.

Eligibility criteria

The study was conducted at a high-complexity university hospital situated at an altitude of approximately 2600 m above sea level. It included adult patients aged 18 and older who were evaluated in the triage area with a suspected diagnosis of sepsis, as defined by the Sepsis-3 consensus, regardless of the origin of the sepsis [13]. Patients referred from other institutions, those who received prior treatment at another facility, and pregnant patients were excluded from the study. Patients who were not confirmed to have an infectious process by cultures and/or diagnostic images were also excluded. All patients meeting the eligibility criteria were included sequentially until the proposed sample size was reached.

Methodology

All patients were assessed by a nurse in the triage area of the emergency department. Each nurse received specific training to identify patients with suspected infections. Once a nurse identified a patient with symptoms of infection (such as fever, cough, dysuria, diarrhea, etc.), they calculated the NEWS score. If the score was ≥ 5 points, the patient was admitted to the emergency room, where the emergency physician evaluated whether the patient exhibited clinical signs of sepsis. If sepsis was suspected, studies for SOFA calculation were requested, and treatment was initiated according to institutional guidelines. We used a threshold of 5 points on the scale based on the associated risk of early cardiac arrest, intensive care admission, and mortality at this score level [14].

All patients enrolled in the study were monitored throughout their hospital stay until sepsis was confirmed. This confirmation was based on culture reports and/or diagnostic imaging, all following the Sepsis-3 guidelines, and their outcomes were documented [13].

Altitude-adjusted or NEWSa and IEWSa

To calculate these variables, oxygen saturation values ​​in the NEWS and IEWS scores were replaced by oxygen saturation values ​​adjusted to the altitude of the city where the study was conducted (2640 m above sea level).

For this purpose, we initially used the Severinghaus equation to calculate the approximate alveolar oxygen pressure (PaO2) from the oxygen saturation (SaO2) [15]. Using this approach, we established our reference points for PaO2 and SaO2 at sea level, which were used in the NEWS and IEWS scores. Subsequently, we calculated the arterial oxygen pressure adjusted for atmospheric pressure. To achieve this, we first determined the alveolar oxygen pressure (PAO2) by adjusting for the barometric pressure specific to the city where the study was conducted (560 mmHg), utilizing the alveolar gas equation [16]. Then, starting from a normal alveolar-arterial gradient (5–10 mmHg), we calculated the arterial oxygen pressure [16]. In this way, we obtained the arterial pressure values adjusted for atmospheric pressure. Finally, using the oxygen saturation values ​​reported in the NEWS and IEWS scores, we calculated the expected PaO2 values ​​at sea level. This allowed us to adjust the PaO2 and subsequently the SaO2 values to the atmospheric pressure of the study location (Table 1).

Table 1 Oxygen saturation values in NEWS and NEWSa

Early warning scores adjusted for height above sea level plus arterial lactate

Because arterial lactate is clearly recognized as an important predictor of sepsis mortality [17, 18]; we wanted to evaluate whether adding arterial lactate to altitude-adjusted prediction scores would improve the predictive ability of these scores for in-hospital sepsis death in the emergency department.

NEWSaL

We utilized the known predictive capacity of lactate levels for sepsis mortality and incorporated it into the National Early Warning Score adjusted for altitude above sea level. Therefore, this value corresponds to the sum of the NEWSa score, and the initial arterial lactate value obtained from the patient with suspected sepsis, taken in the emergency department after triage conducted by the nurse.

IEWSaL

This value corresponds to the sum of the IEWSa score, and the initial arterial lactate value obtained from the patient with suspected sepsis, taken in the emergency department after triage conducted by the nurse.

Sample size

For the sample size calculation, an estimated disease prevalence of 18% was assumed, with a significance level of 95% and a power of 80%. Additionally, a difference between the areas under the curve of 0.1 was determined to be considered relevant. With these parameters, a minimum sample size of 232 patients was estimated.

Data analysis plan

The information recorded in the REDcap data collection tool was reviewed to avoid inconsistencies or duplications, ensuring that the data corresponded to each type of variable.

A descriptive analysis of the study variables was conducted using categorical variables, and frequency distribution. For continuous variables, measures of central tendency and dispersion were calculated according to the type of distribution (mean and standard deviation vs. median and interquartile ranges (IQR) for normal or non-normal distribution, respectively), applying the normality test (Shapiro–Wilk). Sensitivity, specificity, and area under the curve (AUC) were documented using standard formulas for a binomial proportion and the corresponding 95% confidence intervals (CI). For the analysis of positive predictive value (PPV) and negative predictive value (NPV), adjustments were made according to the disease prevalence found in the study. The association between categorical variables and mortality was established by calculating the relative risk and testing for independence using the chi-square test. We use the Delong test from the pROC package in R to establish the differences between the ROC curves. A p-value of less than 0.05 was considered statistically significant. All statistical calculations were performed using R Version 1.4.1106 © 2009–2021 RStudio, PBC. The pROC package was used in the size sample calculation.

The study was conducted at the Hospital Universitario Fundación Santa Fe de Bogotá in Colombia and was approved by the institution's research and ethics committee with approval number CCEI-15338–2023. The study did not receive external funding.

Results

A total of 320 patients were initially selected, however, a total of 16 patients were excluded from the analysis; the reasons for exclusion were that three subjects died before confirmation of sepsis and the remaining 13 patients had negative cultures and imaging tests that did not allow confirmation of the diagnosis of infection before hospital discharge (Fig. 1).

Fig. 1

Flowchart for patient inclusion from June 2023 to March 2024

No data were missing. In Table 2 we report the variables evaluated in the cohort of patients studied and their relationship with mortality. There was no difference in mortality between male and female patients; males constituted 57.6% of the population with a mortality rate of 16.5%. The median age was 72 years (range 23–95), and the most common comorbidity was hypertension (43.4%). A history of chronic kidney disease and COPD were significantly associated with mortality. Overall mortality was observed in 59 patients, accounting for 19.4% (Table 2). Additionally, a significant increase in mortality was determined in patients with septic shock, with an odds ratio of 3.62 (95% CI: 2.01–6.60). The most common site of infection was the lung, and no difference was found between the site of sepsis and mortality (Table 2). A significant difference was found in the values of all evaluated scores (NEWS, NEWSa, NEWSaL, IEWS, IEWSa and IEWSaL) and documented in-hospital mortality (Table 2).

Table 2 Patient characteristics

In terms of the discriminative ability of the evaluated scales and their comparison, it was found that the highest Receiver Operating Characteristic (ROC) curve value (AUC) was for IEWSaL at 0.86 (95% CI: 0.81–0.91), while the lowest was for NEWS at 0.78 (95% CI: 0.72–0.83). However, when comparing NEWS with NEWSa, no significant differences were documented in the AUCs (p = 0.132). Similarly, no significant differences were found between IEWS and IEWSa (p = 0.120). However, we found significant differences in the AUC of NEWS and IEWS (p = 0.009) (Fig. 2). Finally, the addition of the lactate value to the original scores improves the AUC of all scales, NEWSL (0.82, 95% CI:0.76–0.88) and NEWSaL(0.83, 95%CI:0.78–0.89), especially of IEWS (IEWSL 0.84, 95%CI:0.79–0.90) and IEWSa (IEWSaL 0.86, 95%CI:0.81–0.91), We found a significant difference between the NEWSL and NEWSaL scales versus IEWSa and IEWSaL, although no significant differences were documented between these last two scales (p = 0.070).

Fig. 2

Receiver operating characteristic curves comparing prediction tools. NEWS: 0.78(95% CI:0.72–0.83), NEWSa: 0.79(95% CI:0.73–0.84), IEWS: 0.81(95% CI:0.75–0.87), IEWSa: 0.82(95% CI:0.76–0.88), IEWSL: 0.84(95% CI:0.79–0.90), IEWSaL: 0.86(95% CI:0.81–0.91)

We performed calibration curves for all the scores evaluated and found that the behavior of the original scores was adequate; however, the calibration curve when adding lactate to the scores has problems, especially at high values ​​on the scale. Figure 3.

Fig. 3

Calibration curves of prediction scales. NEWS: the National Early Warning Score. NEWSa: the National Early Warning Score adjusted for height above sea level, IEWS: the International Early Warning Score, IEWSa: the International Early Warning Score adjusted for height above sea level IEWSL: the International Early Warning Score plus the value of arterial lactate, IEWSaL: the International Early Warning Score adjusted for height above sea level plus the value of arterial lactate

Ultimately, although all scales had similar values, we found that the IEWSaL had the highest values ​​of sensitivity, specificity, positive and negative predictive value (Table 3).

Table 3 Sensitivity, specificity, and predictive values comparing prediction tools

Discussion

The early identification and prediction of mortality due to sepsis, particularly from the emergency department, is a current priority in clinical research [19]. The NEWS and IEWS scales were developed in populations at sea level [4, 9]. Therefore, it is necessary to determine whether adjusting the oxygen saturation variable in the NEWS and IEWS scales by altitude above sea level would improve their predictive capacity. This adjustment has not been previously reported. Our study showed that adjusting for altitude saturation on a scale derived from sea-level populations does not significantly improve the predictive ability of the test. This finding aligns with reports in the literature that adjust the arterial oxygen pressure and inspired oxygen fraction (PaO2/FiO2) ratio for altitude, including studies in special populations such as COVID-19 patients [20], mechanical ventilation [21], and in acute Respiratory Distress Syndrome [22, 23]. It appears that these adjustments, although physiological, do not significantly affect outcomes. These results could support the safe use of prediction scales in high-altitude populations without major adjustments.

In terms of the predictive capacity of the evaluated scales, the AUC value for NEWS in this study was 0.78(95% CI: 0.72–0.83), which is like the values reported by Ruangsomboon et al. [24], and Wang et al. [7] with AUCs of 0.61 (95% CI: 0.58–0.64) and 0.69 (95% CI: 0.65–0.73), respectively. We developed a NEWS score adjusted for altitude above sea level and found that the discrimination capacity improved with respect to the original scale AUC 0.79(95% CI: 0.77–0.84); however, this improvement was not statistically significant. Nevertheless, the NEWSa score values ​​are comparable with other AUCs reported in other studies such as that of Brink et al. [25], who showed an AUC for mortality of the NEWS score of 0.77(95% CI:0.75, 0.80), which suggests that adjusting the oxygen saturation value could be useful; however, the results of this work do not allow us to recommend such adjustment.

The IEWS scale, which includes, in addition to the original variables of the NEWS score, points for the age and sex of the patients, demonstrated that its predictive capacity was better than the NEWS score in a population of patients from the Netherlands and Denmark [9]. For this reason, we evaluated the discrimination capacity of this test in our population and found a significant improvement in the predictive capacity compared to the NEWS score, AUC 0.81(95% CI:0.75–0.87) vs 0.78(95% CI: 0.72–0.83). Interestingly, although the IEWS score showed an improvement in AUC compared to the NEWSa scale, this was not statistically significant AUC 0.81(95% CI:0.75–0.87) vs 0.79(95% CI: 0.77–0.84) respectively. Based on this result we think that it is better to use the IEWS scale over the height above sea level adjustment of the original NEWS scale.

Based on our findings regarding the predictive capacity of the IEWS scale, we wanted to evaluate its discrimination capacity by adjusting the oxygen saturation value to the height above sea level in a similar way to that performed with the NEWS scale. We found that the IEWSa scale has a higher AUC than the IEWS scale, however, this was not significant. AUC 0.82(95% CI:0.76–0.88) vs 0.81(95% CI:0.75–0.87). Although the predictive capacity of IEWSa was significantly superior when compared to NEWS (p = 0.001) and NEWSa (p = 0.004). The mortality discrimination capacity of the IEWSa scale was comparable to that reported by the Candel et al. work AUC 0.87 (95% CI:0.85–0.88) [9]. This data is not surprising if we consider that oxygen saturation is not only related to altitude above sea level, the age of the patients is also a determining factor in the expected value of arterial oxygen saturation, that is, it is expected that the older the patient, the lower the expected oxygen saturation [26]. Based on these results, we believe that the IEWSa scale may be an interesting alternative for predicting mortality from sepsis in high-altitude sites.

Interestingly, the work of Dadeh AA et al. [27] used a similar modification to the conventional NEWS by adding the lactate value. They demonstrated an AUC of 0.83 for predicting in-hospital sepsis mortality, although the AUC value for NEWS without lactate was 0.82, suggesting that the contribution of lactate was not very significant. The study by Jo S et al. [22], with 4624 adults, also showed a good predictive capacity for mortality with NEWS-L, achieving an AUC of 0.87 (95% CI: 0.85–0.90). We decided to add the lactate value to all the scales used and found that the discrimination capacity improved, especially we found that the IEWSL and the IEWSaL were the ones with the best AUC of all 0.84 (95% CI: 0.79–0.90) vs 0.86 (95% CI: 0.81–0.91).

The sensitivity values ​​found in the scales were around 70%, which is similar to that reported by the meta-analysis of Wang et al. [7], who reported a sensitivity of 73% for NEWS. Similarly, the specificity of the scales was also approximately 60%, a value similar to that reported in Wang et al.'s study (52%) [7]. These values ​​are also comparable to the sensitivity (71%) and specificity (85%) records reported by the meta-analysis of Qiu X et al. [28]. The positive and negative predictive values of all the scales were similar, at 0.90 and 0.40, respectively, results comparable to the work of Brink et al. [25].

In this cohort, the mortality rate due to sepsis was 19.4%, which is comparable to other clinical studies that show overall sepsis mortality rates of 24.9% [29] and 22.4% [30]. The mortality rate due to septic shock documented in this study was 33.3% of the total sample, which is comparable to other studies such as the meta-analysis by Bauer et al. [31], which reported a septic shock mortality rate of 34.7%, and the study by Fleischmann et al., which reported a mortality rate of 26% [32]. Similarly, the mortality rate due to sepsis documented in this study was 12.06%, like the 24.4% reported in the meta-analysis by Bauer et al. [29] and the 17% reported in the study by Fleischmann et al. [30]. However, among the group of patients who died, 59.9% had septic shock. A statistically significant association was documented between the presence of septic shock and mortality, with an odds ratio (OR) of 3.62 (95% CI, 2.01–6.60), underscoring the importance of early recognition and management of this condition. We did not find a difference in sepsis mortality based on sex; however, we documented a significant difference between advanced age and sepsis mortality. This finding is not surprising, as it is widely known and reported in the literature [1]. Even in data from populations like this study, mortality in populations over 70 years old with sepsis is reported to be around 40% [33], highlighting the need for more careful assessment of the elderly due to their fragility.

The primary limitation of this study is that it was conducted in a single hospital center, which means that the results should be interpreted with caution when applying the scales to other populations. Additionally, the cohort of patients analyzed did not include many severely ill cases, which subtly limited the analysis of the scales' performance in this type of population. However, the mortality rates were similar to those reported in the literature, which supports the reliability of the study's predictive capacity. The NEWS score should not be used to establish the diagnosis of sepsis. The scope of this work only allows us to determine that it is likely that adjusting the oxygen saturation variable to the height above sea level does not contribute much to the ability of the score to predict death in a cohort of patients with confirmed sepsis before hospital discharge.

Conclusion

The results of this exploratory study indicate that adjusting the oxygen saturation for altitude above sea level when applied to the NEWS scale (NEWSa), does not enhance the predictive capacity for mortality in sepsis patients in the emergency department, however, this same adjustment in the IEWS value significantly improves the predictive capacity compared to NEWS and NEWSa. In addition, although the inclusion of lactate levels to the score of the EWS scales increases the predictive capacity of mortality and especially that of the IEWSaL scale, more studies are needed before recommending its routine use.