The emergency department (ED) plays a central role in the management of acute heart failure (AHF) since about 90% of patients with this condition attend an ED to alleviate their symptoms.1 In the ED, AHF patients are usually treated with diuretic intensification, oxygen supplementation and, if needed, vasodilators and morphine.2 Once these initial treatments have been administered and their effects evaluated, a decision is needed as to whether the patient should be hospitalized or can be discharged with adequate treatment and subsequent follow-up. Although some risk scores have recently been developed for use in EDs in Canada3,4 and the United States5 with the aim of objectively supporting this decision-making process, their implementation is not widespread in Spanish EDs. Therefore, the decision to discharge patients directly from the EDs without hospitalization, currently made in about a quarter of AHF patients in Spain,2 is still empirically-driven by the subjective assessment of emergency physicians.

To address this clinical challenge, the MEESSI (Multiple Estimation of risk based on the Spanish Emergency department Score In patients with AHF) risk score was recently developed for use in AHF patients in Spanish EDs.6 This risk score demonstrates that the individual 30-day risk of mortality of AHF patients admitted to the ED can be reliably estimated using 13 readily available items. This tool has strong risk discrimination (c-statistic of 0.836), adequate model goodness-of-fit, and external validation and is now available online as a friendly website calculator.7 The MEESSI risk score also stratifies patients into 4 clinical categories, corresponding to low-, intermediate-, high- and very high-risk groups. Routine use of this risk model can potentially help clinicians to adequately manage ED admission, particularly by reliably identifying individuals at lower risk who may not require further hospitalization. However, this risk score needs to be externally validated and its accuracy checked in a new patient cohort, with particular emphasis on comparisons between the performance of MEESSI in different patient subgroups based on hospital and ED characteristics. Therefore, the aim of the present study was to validate the performance of the MEESSI risk score in a new cohort of Spanish patients and in several subgroups of hospitals and EDs, as well as to evaluate its accuracy in stratifying AHF patients by risk.

We studied 4711 new patients (2 were excluded from the original 4713 patients due to lack of follow-up) recruited in 19 university and 11 community hospitals (3811 and 900 cases, respectively) in EDs with high-, medium- and low-activity (2695, 1479, and 537 cases, respectively) and in EDs that did or did not participate in the original derivation of the MEESSI (3892 and 819 cases, respectively). Of note, patients were of advanced age (mean 80.8 years ± 10.2), with multiple comorbidities (the most frequent being hypertension in 83.2%, atrial fibrillation in 50.8%, and diabetes mellitus in 41.8%), more than half had had previous episodes of AHF, and two-thirds had preserved left ventricular ejection fraction (> 49%). The remaining clinical characteristics are shown in Table 2. There was less than 10% of missing values in 50 out of the 56 clinical variables (89.3%), and only echocardiographic data, respiratory rate, Barthel index at arrival, potassium, NT-proBNP and troponin levels showed higher proportions of missingness. The final patient destination after ED care is shown in Table 3, with the main destinations for admitted patients (75.4% of cases) being internal medicine and cardiology wards, although the distribution significantly differed according to the risk category (P < .001).

In this validation cohort, 486 (10.3%) patients died within 30 days of ED admission. The MEESSI risk score predicted 30-day mortality using 13 variables (full model) with excellent discrimination (c-statistic, 0.810; 95% confidence interval, 0.790-0.830; P < .001).The distribution of patients across the clinical risk categories defined by the MEESSI score was as follows: 1673 (35.5%) were classified as low risk, 2023 (42.9%) as intermediate risk, 530 (11.3%) as high risk, and 485 (10.3%) as very high risk. The steep-gradient in 30-day mortality remained consistent across risk categories, with cumulative mortalities at 30 days of 2.0%, 7.8%, 17.9%, and 41.4%, respectively (Figure 1). The goodness-of-fit of the model was excellent (Figure 2). The other 7 models for the MEESSI risk score, which can be applied to predict 30-day mortality in the absence of the Barthel index, NT-proBNP and/or troponin, showed c-statistics between 0.752 and 0.806. Our sensitivity analysis using only patients with complete data (ie, not using imputed values for those variables lacking data) rendered similar values for the 8 models, with c-statistics ranging from 0.731 to 0.785 (Figure 3). In addition, the 30-day revisit to the ED was higher with an increase in the risk category (18.5%, 23.5%, 23.7%, and 27.0%, respectively, P = .01), and the same relationship was found with respect to 30-day rehospitalization (11.0%, 17.5%, 19.7%, and 22.5%; P < .001).

Figure 1.

Kaplan-Meier curves showing the 30-day cumulative mortality for the 4 clinical risk groups defined by the MEESSI risk score. Low-, intermediate-, high- and very high-risk categories correspond to groups defined by the cutoffs found in the original derivation study that included the first and second bottom quintiles, the third and fourth quintiles, the ninth decile, and the top decile of 30-day mortality, respectively.

(0.09MB).

Figure 2.

Assessment of the goodness-of-fit of the MEESSI scale in the new patients included in the present study by comparing observed and predicted 30-day mortality. The Hosmer-Lemeshow test (P = .745) indicates that the 30-day mortality estimated by the MEESSI risk score does not significantly deviate from the observed data.

(0.07MB).

Figure 3.

Description of the area-under-the-curve receiver operating characteristic for the full MEESSI-AHF model and for each reduced model using imputed and real data. Estimates in the group “Using imputed data” were performed with all the 4711 patients in all the models, and included real values and also imputed values for missing data. Estimates in the group “Using observed data” were performed using only patients with complete data for all relevant variables: 602 patients in model 1 (full MEESSI risk score model); 1215 patients in model 2; 953 patients in model 3; 662 patients in model 4; 2216 patients in model 5; 1394 patients in model 6; 1048 patients in model 7, and 2669 patients in model 8. 95%CI, 95% confidence interval; NT-proBNP, N-terminal pro-B-type natriuretic peptide.

(0.21MB).

Figure 4 shows the subgroup analysis of the discriminatory capacity of the MEESSI risk score by type of hospital, ED activity volume, and previous participation in score derivation. Very similar c-statistics were obtained with no significant differences among subgroups. Remarkably, the discriminatory capacity of our model was very high (c-statistic: 0.832; 95% confidence interval, 0.791-0.872) in the sample of patients recruited in EDs which did not participate in the derivation of the MEESSI risk score.

Figure 4.

Receiver operating characteristic curves for the MEESSI risk score applied to the complete cohort (A), and for the subgroups of patients from university and community hospitals (B), high-, medium- and low-activity EDs (C), and from EDs that did and did not participate in the original derivation of the MEESSI risk score (D). Receiver operating characteristic curves were obtained with the full MEESSI model and included all patients (those with complete real data and those with some imputation because of missing data). ED, emergency department.

(0.27MB).

We have externally validated the MEESSI risk score in a new cohort of AHF patients from 30 Spanish EDs. The results demonstrate that the MEESSI score has a high discriminatory capacity, which is consistent across several subgroup analyses, including 10 EDs that were not previously involved in the derivation process. The MEESSI scale classifies patients into 4 clinical risk categories. In particular, 35.7% patients were allocated to the low-risk category (cumulative 30-day mortality of 2.0%), and 10.1% were classified into the very high-risk category (30-day cumulative mortality of 41.4%, more than 20-fold higher than low-risk patients). This clear-cut risk stratification based on the prediction of 30-day mortality may help emergency physicians make clinical decisions with respect to AHF patient disposition based on objective risk estimates.

The MEESSI risk score showed a remarkably high discriminatory capacity across 3 cohorts of Spanish patients (c-statistic of 0.836 and 0.828 in the derivation and validation original cohorts, and 0.810 in the current validation study). To our knowledge, only 3 risk scales derived from ED patients have previously been developed to predict mortality: 1 was from the United States (The STRATIFY [Improving Heart Failure Risk Stratification in the Emergency Department]5) and 2 were from Canada (the Ottawa Heart Failure Risk Scale [OHFRS],3 and the Emergency Heart Failure Mortality Risk Grade, [EHMRG]4). However, their discriminatory capacity was lower than that observed with the MEESSI risk score (c-statistics 0.68 for STRATIFY, 0.77 for OHFRS, and 0.807/0.804 for the EHMRG derivation and validation cohorts, respectively). Moreover, since risk scores work with the populations to which they are applied, differences in health care system characteristics and local organization could have an impact on their performance. This is particularly true for AHF in which transition plans, outpatient clinics, walk-in centers, AHF clinics and other specifically devoted disease pathways play an important role in patient management and may influence outcomes.12–14 On testing the performance of the EHMRG score in 2 different Spanish cohorts of 1553 and 2137 AHF ED patients, the c-statistic decreased from the original values (0.807/0.804) to 0.74115 and 0.750,6 respectively. Taking into account that we included all consecutive AHF patients and only excluded ST-segment elevation myocardial infarction patients with concomitant AHF, the MEESSI risk score can potentially be applied to the vast majority of AHF patients diagnosed in EDs. This also makes our risk score different from the remaining scores available, since some were developed excluding substantial subsets of patients (ie, EHMRG excluded palliative patients and OHFRS enrolled a nonconsecutive sample with multiple exclusion criteria). Therefore, our results suggest that the MEESSI risk score not only performs well in Spanish EDs but also covers a global unmet clinical need.

The marked accuracy of the MEESSI risk score for identifying low-risk patients is potentially clinically relevant. We found that patients with AHF discharged to home from Spanish EDs without hospitalization were exposed to a significantly increased risk of adverse outcomes, which is especially high in the short-term. Indeed, the hazard ratios for ED- compared with hospital-discharged patients, after adjustment for patient and center features, was 2.07 (95% confidence interval, 1.19-3.60) for the 7-day ED revisit, and 3.07 (95% confidence interval, 1.92-4.92) for the 7-day hospitalization risk.16 Similar results have been reported in Canada17,18 and the United States.19 This clearly underlines that there is room for improvement in ED decision-making and that the definition and implementation of effective strategies to improve patient selection for direct ED discharge can improve outcomes. In a recent consensus document, experts proposed that in EDs in which extended observation is possible (usually up to 24hours), such as in many Spanish EDs, the direct ED discharge rate should be higher than 40% while maintaining a 7-day revisit rate of less than 10%, a 30-day mortality of less than 2%, and a combined 30-day ED revisit or rehospitalization rate of less than 20% for this subgroup of patients managed without hospitalization.20 The MEESSI score stratified nearly 40% of all patients in the low-risk group, and the 30-day mortality was approximately 2% in this group. In addition, our data suggest that the MEESSI scale could also stratify the risk of a revisit to ED or rehospitalization due to AHF, although this potential capacity will have to be tested in further studies. These figures suggest that it should be feasible to achieve the goals of the previously mentioned expert recommendations, and the MEESSI score could help to achieve these objectives. However, as a large proportion (62.3%) of low-risk patients were hospitalized, it is not known if the good outcomes achieved in these low-risk patients would have remained the same if these patients had been discharged directly from the ED to home without hospitalization.

The detection of AHF patients at very high risk can also assist emergency physicians to better recognize this subset of patients, who have a strong probability of dying within the days following ED admission. The lack of immediate identification of these patients and the delay in offering them intensive care is, in fact, associated with poorer outcomes.21,22 In this regard, the MEESSI risk score could also help to improve outcomes in this challenging patient subgroup, although other risk scales developed in the ED arena can be applied in this setting, such as the recently published EAHFE-3D,23,24 which could even be more successful in these very sick patients. Nonetheless, the early detection of this situation does not necessarily lead to patient admission in intensive care units, since palliative rather than intensive care may be the most adequate approach in some patients in whom AHF is the final stage of chronic advanced cardiomyopathy.25,26 In fact, our data show that most of the patients allocated to the very high-risk category were not admitted to cardiology and intensive/coronary care units, suggesting that a high proportion of these patients were allocated to palliative care.

The MEESSI risk score is a clinically useful tool for acute risk stratification of AHF patients in all Spanish EDs and eventually worldwide. It has been carefully developed in a large, multicenter cohort, including virtually all kinds of AHF patients. It has been internally and externally validated, showing good performance using only variables readily available in the ED arena. In addition, online access to a user-friendly risk calculator that works even in the absence of some variables6 makes it very practical. We hope that the generalization of accurate risk estimation of patients with AHF during ED care by emergency physicians will help to improve early clinical decision-making and, in turn, positively impact short-term outcomes in a syndrome whose prognosis has remained largely unchanged over several decades.

This study was partially supported by grants from the Instituto de Salud Carlos III supported with funds from the Spanish Ministry of Health and FEDER (PI15/01019 and PI15/00773) and Fundació La Marató de TV3 (2015/2510). The “Emergencies: Processes and Pathologies” research group of the IDIBAPS receives financial support from the Catalonian Government for Consolidated Groups of Investigation (GRC 2009/1385 and 2014/0313). Xavier Rosselló has received support from the SEC-CNIC CARDIOJOVEN fellowship program.

The ICA-SEMES Research Group has received unrestricted support from Orion Pharma and Novartis. The present study was designed, performed, analyzed, and written exclusively by the authors independently of these pharmaceutical companies. H. Bueno reports grants and personal fees from Astrazeneca, Daiichi Sankyo, Eli Lilly, Bayer, Sanofi during the conduct of the study; personal fees from Novartis, BMS-Pfizer, Servier outside the submitted work.