The RESCUE registry is a multicenter registry of all-cause CS. This registry was used for the development of a novel risk prediction system, the RESCUE score. Patients were enrolled between January 2014 and December 2018. The study design and main results, including clinical characteristics and predictors of in-hospital mortality, have been published previously.14 The inclusion criteria were as follows: a) systolic blood pressure below 90mmHg for 30minutes or the need for inotropes or vasopressors to attain a systolic blood pressure of 90mmHg or above, and b) the presence of pulmonary congestion and signs of organ dysfunction due to malperfusion (altered mental status, cold periphery, oliguria <0.5mL/kg per hour for the previous 6hours, or blood lactate levels above 2 mmol/L).

Exclusion criteria consisted of out-of-hospital cardiac arrest and evidence of septic or hypovolemic shock. More detailed information regarding participating centers and prospective and retrospective enrollments in this trial is listed in table 1 of the supplementary data. We also included 750 CS patients who were admitted to the cardiac intensive care unit of Samsung Medical Center (SMC) between January 2011 and December 2020 for external validation. The inclusion and exclusion criteria of the cohort for external validation were the same as those of the RESCUE registry (figure 1).

Figure 1.

Study design flow chart. LASSO, least absolute shrinkage and selection operator analysis; SMC, Samsung Medical Center.

(0.2MB).

Data were collected using a web-based case record form, with additional information retrieved from medical records or through telephone contact when necessary. Clinical characteristics, laboratory findings, vital signs, and in-hospital management were analyzed as potential predictors. The worst values for left ventricular ejection fraction and lactic acid levels at the time of the index CS event were recorded.

The inotropic score and vasoactive inotropic score were calculated based on the maximal use of vasoactive agents during the first 48hours of shock. These included norepinephrine, milrinone, vasopressin, dopamine, dobutamine, and epinephrine, using the formula proposed by Gaies et al.15 Coronary angiography, percutaneous coronary intervention, mechanical circulatory support, continuous renal replacement therapy, and intubation with mechanical ventilation were performed promptly, based on the patient's condition and at the physician's discretion. The primary outcome of the study was in-hospital mortality.

The RESCUE registry was registered with ClinicalTrials.gov (Retrospective and Prospective Observational Study to Investigate Clinical Outcomes and Efficacy of Left Ventricular Assist Device for Korean Patients With CS, the RESCUE registry, NCT02985008). Ethics approval for the study protocol was obtained from the institutional review board of Samsung Medical Center (No. 2016-03-130, April 6, 2016), with additional approval from each participating hospital. The name and number of the institutional review boards of all participating hospitals and the approval dates are listed in table 2 of the supplementary data. The requirement for written informed consent was waived for retrospectively enrolled patients. However, written informed consent was obtained from prospectively enrolled patients or their legal representatives. All procedures adhered to the ethical standards of the responsible institutional review committees and the principles outlined in the Helsinki Declaration of 1975.

Baseline patient characteristics are summarized as numbers and proportions for categorical variables, and as means with standard deviations or medians with interquartile ranges (IQR, 25th–75th percentiles) for continuous variables. Comparisons between survivors and nonsurvivors were conducted using the chi-square or Fisher exact test for categorical variables and the Student t-test for continuous variables.

To develop the RESCUE scoring system, multivariable logistic regression analysis was performed with the variables consistently ranked highly for in-hospital mortality by 4 ML algorithms: a) least absolute shrinkage and selection operator (LASSO) analysis, b) support vector machine, c) random forest classifier, and d) extreme gradient boosting. The RESCUE score was constructed using a regression coefficient-based scoring method that incorporated both linear and polynomial regression models. Since the coefficients represent the log odds of each variable, all variables were standardized to a common scale. Logarithmic transformations were applied to normalize disproportionately distributed variables. After constructing the scoring system, the inverse logit transformation was applied to convert the risk scores from the logistic regression model into predicted probabilities of death.

For internal validation, the study population was randomly divided into 2 groups (training and testing sets) at a ratio of 7:3. Ten-fold cross-validation was performed, while external validation was conducted using an independent CS-dedicated cohort. The discriminative performance of the RESCUE score for predicting in-hospital mortality was evaluated using the area under the curve (AUC) of receiver operating characteristic (ROC) analysis. Calibration plots were used to assess the agreement between predictions of in-hospital mortality and the observed outcome rate for a given predicted risk.

To assess the usefulness of the RESCUE score, conventional univariable and multivariable logistic regression analyses with backward stepwise selection were also performed. The net reclassification index (NRI) was used to compare the AUCs of the RESCUE score and conventional logistic models. Additionally, decision curve analysis was conducted to evaluate the clinical implications of the prediction models.

To handle missing data, the ML models used exclusion, median imputation, κ-nearest neighbor imputation, and bag imputation. In the 4 ML algorithms, there was no difference in the performance of the risk prediction models based on the techniques used for handling missing data, even after repeating the process 10 times. Therefore, we decided to exclude missing data in the development and validation of the scoring system. Data were analyzed using IBM SPSS Statistics for Windows version 20.0 (IBM Corp., United States) and R version 4.2.0 (R Foundation for Statistical Computing, Austria), and P values <.05 were considered statistically significant.

A total of 1247 patients from 12 tertiary centers in Korea were enrolled between January 2014 and December 2018. Of these, 954 patients were retrospectively enrolled, while 293 patients were prospectively enrolled. The baseline clinical and laboratory characteristics of the patients are summarized in table 1. A total of 860 patients (69.0%) were men, the mean age was 66 years, and 419 patients (33.6%) died during the index hospitalization. The primary causes of CS included ischemic heart disease (80.1%), followed by dilated cardiomyopathy (6.1%), myocarditis (3.2%), and arrhythmia (2.5%). Comparison of baseline characteristics stratified by in-hospital mortality revealed that nonsurvivors were older (mean age 69 vs 64 years; P <.001), had lower left ventricular ejection fraction (mean 28% vs 39%; P <.001) and had higher lactic acid levels (mean 7.8 mmol/L vs 4.6 mmol/L; P <.001) and vasoactive inotropic scores (mean 70 vs 16; P <.001). There were no significant differences in the primary causes of CS between survivors and nonsurvivors. Nonsurvivors were more likely to have comorbidities, present with in-hospital cardiac arrest (39.6% vs 12.3%; P <.001), require continuous renal replacement therapy (46.8% vs 10.7%; P <.001), and need mechanical ventilation (86.4% vs 41.9%; P <.001).

All the variables listed in table 1 were considered potential predictors. These variables were ranked in order of relative importance by each risk prediction model of the 4 ML algorithms, and the results are shown in figure 1 of the supplementary data. Model performance among the ML algorithms is summarized in figure 2 of the supplementary data. The risk prediction models developed using the ML algorithms demonstrated good predictive performance for in-hospital mortality: the LASSO analysis had an AUC of 0.84 (95% confidence interval [95%CI], 0.81-0.87], the support vector machine an AUC of 0.90 (95%CI, 0.87-0.92), random forest an AUC of 0.98 (95%CI, 0.97-0.99), and extreme gradient boosting an AUC of 0.87 (95%CI, 0.85-0.90).

Among the possible predictors, variables ranked in the top 7 in importance by at least 3 ML models were selected for the final model. The 7 selected variables associated with in-hospital mortality were age, vasoactive inotropic score, left ventricular ejection fraction, lactic acid level, in-hospital cardiac arrest at presentation, continuous renal replacement therapy, and mechanical ventilation.

The final multivariable logistic regression model, the RESCUE score, was derived using these selected variables. A detailed description of the model, including the calculation formula, is provided in table 2. Table 3 provides examples of the estimated predictive probabilities of in-hospital mortality derived from the RESCUE score, as well as the observed outcomes in the study population. The RESCUE score showed excellent predictive performance with an AUC of 0.86 (95%CI, 0.83-0.88) for in-hospital mortality, and the box plot showed excellent discrimination power (figure 2).

Figure 2.

The predictive power of the RESCUE score in the RESCUE registry and the SMC CICU registry. The area under the receiver operating characteristic curves and the box plots of the RESCUE score in the RESCUE registry (A) and the Samsung Medical Center (SMC) CICU registry (B).

(0.33MB).

After univariable logistic regression was performed to assess crude odds ratios (table 4), multivariable logistic regression models were developed using 2 approaches. First, a conventional multivariable logistic regression model using backward stepwise selection was performed, constructed with 10 variables, which was more complex than the RESCUE score (figure 3 of the supplementary data). Next, another multivariable logistic regression model was created, limiting the number of variables to match those of the RESCUE score (figure 4 of the supplementary data). The RESCUE score outperformed the secondary logistic regression model, with an NRI of 0.13 (95%CI, 0.05-0.21; P=.002) in the training set and 0.18 (95%CI, 0.08-0.29; P <.001) in the external validation set.

To assess whether the risk prediction models were not only sensitive and accurate but also clinically appropriate, decision curve analysis was performed for all models. The RESCUE score yielded comparable net clinical benefits within a broad range of risk thresholds compared with the other derived prediction models (figure 3), including both ML-derived risk models and the conventional multivariable logistic regression model.

Figure 3.

Decision curves comparing the net benefit of risk prediction models. Decision curves of all models derived in this study. The RESCUE score yielded comparable net clinical benefits among a broad range of risk thresholds to the other derived prediction models. LASSO, least absolute shrinkage and selection operator analysis.

(0.19MB).

Among the patients in the RESCUE registry, 762 had complete data and served as the internal validation cohort. In this group, the RESCUE score showed good predictive performance, with a 10-fold cross-validated AUC of 0.86 (95%CI, 0.77-0.95). A total of 650 patients with complete data were included in the external validation cohort. External validation also showed excellent predictive performance, with an AUC of 0.80 (95%CI, 0.76-0.84; figure 2). The final model was well calibrated in both the development and external validation cohorts, as evidenced by the calibration plots shown in figure 4. Additionally, we analyzed the predictive power of the RESCUE score in both the training and external validation sets. Subgroups analyses were conducted based on clinical parameters indicative of CS severity, such as the use of extracorporeal membrane oxygenation and the presence of cardiac arrest, as well as the etiology of CS. The overall results were comparable (figure 5 of the supplementary data).

Figure 4.

Calibration plots of the RESCUE score for predicting in-hospital mortality. Calibration plots showing the agreement between the predicted risk of in-hospital mortality and the observed outcome rate for a given predicted risk in the RESCUE registry (A) and the Samsung Medical Center (SMC) CICU registry (B).

(0.17MB).

Figure 5.

Central illustration. Development and validation of the RESCUE score. The RESCUE score for predicting in-hospital mortality from all-cause cardiogenic shock patients. Variables were extracted by machine learning algorithms and the score demonstrated excellent predictive performance. It was well-validated both internally and externally. SMC, Samsung Medical Center.

(0.42MB).

There are several limitations to our study. First, most patients in the developmental cohort were enrolled retrospectively. Although the large sample size used to derive the RESCUE score and its external validation enhance its robustness, the possibility of unmeasured confounding factors remains. Second, the etiology of most CS patients in the developmental cohort was ischemic heart disease, with patients with nonischemic causes representing a heterogeneous group, potentially limiting the applicability of the score to these subpopulations. Third, both the developmental and validation cohorts consisted exclusively of East Asian patients, which may limit the generalizability of the findings to other ethnic populations. Fourth, Impella (Abiomed Inc, United States), a mechanical circulatory support device, was not included in this study due to its unavailability in South Korea.

Fifth, despite the recent decrease in in-hospital mortality from all-cause CS,1 the mortality rate in the RESCUE registry is relatively low. This may be partly explained by the potential for selection bias due to difficulties in obtaining informed consent during prospective enrollment, which could contribute to the relatively lower observed mortality rate. Sixth, the observed risk in the validation set appeared slightly lower than the predicted risk among high-risk patients. As the validation cohort was derived from a single center, institutional differences and unmeasured confounding factors may have contributed to the discrepancy in observed in-hospital mortality. Lastly, the RESCUE score was developed using variables collected at the time of hospitalization, making it challenging to account for the dynamic nature of CS and the changing risk of mortality over time. The recently updated SCAI SHOCK stage emphasizes the dynamic progress of CS, in which patients progress or recover over time.32 Future studies are needed to evaluate temporal trends in the RESCUE score through a large-scale, prospective, protocol-based CS registry.