Publish in this journal
Journal Information
Vol. 72. Issue 3.
Pages 192-194 (March 2019)
Download PDF
More article options
Vol. 72. Issue 3.
Pages 192-194 (March 2019)
DOI: 10.1016/j.rec.2018.10.002
Full text access
Limitations of Current Risk Scoring in Real World Populations: The Importance of External Validation
Limitación de la aplicación de las puntuaciones de riesgo en la clínica: la importancia de la validación
Sabato Sorrentinoa,b,, Usman Babera,, Roxana Mehrana,
Corresponding author

Corresponding author: Interventional Cardiovascular Research and Clinical Trials, The Zena and Michael A. Wiener Cardiovascular Institute, Mount Sinai School of Medicine, One Gustave L. Levy Place, Box 1030, New York, NY 10029, United States.
a Interventional Cardiovascular Research and Clinical Trials, The Zena and Michael A. Wiener Cardiovascular Institute, Mount Sinai School of Medicine, New York, NY, United States
b Division of Cardiology, Department of Medical and Surgical Sciences, University Magna Graecia, Catanzaro, Italy
Related content
Rev Esp Cardiol. 2019;72:215-2310.1016/j.rec.2018.06.004
Sergio Raposeiras-Roubín, Berenice Caneiro Queija, Fabrizio D’Ascenzo, Tim Kinnaird, Albert Ariza-Solé, Sergio Manzano-Fernández, Christian Templin, Lazar Velicki, Ioanna Xanthopoulou, Enrico Cerrato, Giorgio Quadri, Andrea Rognoni, Giacome Boccuzzi, Andrea Montabone, Salma Taha, Alessandro Durante, Sebastiano Gili, Giulia Magnani, Michele Autelli, Alberto Grosso, Pedro Flores Blanco, Alberto Garay, Ferdinando Varbella, Francesco Tomassini, Rafael Cobas Paz, María Cespón Fernández, Isabel Muñoz Pousa, Diego Gallo, Umberto Morbiducci, Alberto Domínguez-Rodríguez, José Antonio Baz-Alonso, Francisco Calvo-Iglesias, Mariano Valdés, Ángel Cequier, Fiorenzo Gaita, Dimitrios Alexopoulos, Andrés Íñiguez-Romo, Emad Abu-Assi
Article information
Full Text
Download PDF
Tables (1)
Table 1. Validation of PARIS Risk Scores in Patients With Acute Coronary Syndrome
Full Text

Dual antiplatelet therapy (DAPT), combining aspirin and a P2Y12 receptor inhibitor, is the cornerstone of secondary antithrombotic prevention in patients undergoing percutaneous coronary intervention (PCI) and stent implantation.1,2 The introduction of safe stent platforms, coupled with a greater appreciation of bleeding-related risks after PCI, have contributed to a general shift and equipoise toward shorter DAPT durations in the setting of stable ischemic heart disease (SIHD). In the Dual Antiplatelet Therapy (DAPT) trial, 9961 patients without ischemic or bleeding events in the initial 12 months after PCI were randomized to receive additional DAPT with aspirin plus clopidogrel or prasugrel for 18 months or to receive aspirin monotherapy. Extended DAPT resulted in a 2.0% absolute reduction in myocardial infarction (MI), a 0.9% absolute increase in moderate or severe bleeding events (Global Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries [GUSTO] moderate or severe) with a borderline increase in overall mortality.3 Numerous pooled analyses comparing short vs longer DAPT durations have corroborated the key message of this trial, namely that extending DAPT in all patients is unnecessary and may be associated with excess harm.4,5 In part, these results may reflect the durable risk of mortality after a bleeding event that appears comparable to that of MI. Alternatively, thrombotic events are characterized by a temporal attenuation in mortality risk, thereby decreasing the clinical imperative for ongoing platelet inhibition. Hence, current guidelines have incorporated these findings in recommendations that stipulate 6-month DAPT duration and the use of risk scores to inform clinical decisions to prolong DAPT among patients with SIHD.1,2 To date, intense investigation has focused on improving perception of risk related to DAPT, and several risk scores have been developed to support physicians during the decision-making process.4,6–10 Notably, the DAPT, PRECISE-DAPT, and PARIS scores were developed from contemporary cohorts of patients undergoing PCI.4,8–10 Notwithstanding the trend toward shorter DAPT durations among patients with SIHD, guidelines remain consistent in recommending at least 1 year of DAPT in patients with acute coronary syndrome (ACS).1,2 Trials comparing short DAPT durations in the setting of ACS have suggested harm while alternative approaches, such as de-escalation, remain unproven. However, ACS patients are also at risk for bleeding complications, and therefore identifying those patients who might be candidates for a shorter or less intense pharmacologic regimen is a clinically intuitive strategy. To address these gaps, in a recent article published in Revista Española de Cardiología, Raposeiras-Roubín et al.11 explored the performance of the PARIS bleeding and coronary thrombosis risk scores in the RENAMI (REgistry of New Antiplatelet therapy in patients with acute Myocardial Infarction) registry, which enrolled a large cohort of patients with ACS discharged with aspirin and a potent P2Y12 receptor inhibitor (ticagrelor or prasugrel). The authors should be congratulated for undertaking this analysis, which is the very first attempt to assess the performances of the PARIS score in ACS patients treated exclusively with potent P2Y12 receptor inhibitors. To properly contextualize these findings, it is important to note several key differences between the PARIS derivation and external validation cohorts. Most importantly, the RENAMI registry is composed entirely of ACS patients (58.0% with ST-segment elevation MI) whereas most PARIS participants presented with stable syndromes.6,12 Secondly, the PARIS risk scores were derived in a clopidogrel-treated cohort whereas potent P2Y12 inhibitors were used in all RENAMI participants. Third, while PARIS was a prospective study with external adjudication, the present report is a retrospective study. Finally, patients were followed up for 1 year as opposed to 2 years in the original PARIS cohort. The authors report that the PARIS score for coronary thrombotic events performed modestly well to discriminate 1-year rates of MI or stent thrombosis (c-statistic, 0.64). In contrast, discrimination for bleeding was relatively poor with a c-statistic of 0.54. Nevertheless, when the authors considered both scales in concert, patients at variable levels of thrombotic and bleeding risk were identified in whom therapeutic decisions may be individualized. Specifically, among patients deemed at high thrombotic risk, only 11.3% were also at high bleeding risk. Analogously, among those at low ischemic risk, 85.5% were also categorized as low thrombotic risk. The authors conclude that such differentiation may be clinically useful in tailoring therapy. For example, high bleeding/low thrombotic risk patients might benefit from short DAPT durations whereas the opposite may be considered for high thrombotic/low bleeding risk individuals.

As an exercise in external validation, the present report is necessary to document the usefulness, or lack thereof, of empiric tools such as the PARIS risk scores to potentially guide clinical decision-making vis a vis DAPT duration (Table 1). In this context, the findings of Raposerias-Roubín et al.11 suggest that the PARIS thrombotic risk score performs reasonably well in an ACS population treated with ticagrelor or prasugrel. Accordingly, key determinants of thrombosis, such as diabetes mellitus, renal impairment and prior revascularization appear to be largely consistent across registries, clinical presentations, and background therapy. In contrast, discrimination for bleeding was not as useful with a c-statistic of 0.54. Although it is expected that the performance of a predictive scale will degrade somewhat when applied in an external cohort, other factors may also play a role. The relatively low rate of bleeding observed in the RENAMI registry is somewhat unexpected, thereby minimizing the level of potential discrimination. This low rate sheds doubt on the validity of bleeding event ascertainment in a retrospective registry. In a separate report with a higher bleeding rate, the PARIS bleeding score demonstrated a higher c-statistic of 0.73. Moreover, factors that play an important role in bleeding risk, such as frailty or malignancy, were not included in the PARIS cohort and could also lead to the observed level of performance. Finally, bleeding risk is not static but rather dynamic over time, yet predictive constructs, including the PARIS scores, are based on assigning risk at a single point in time. More advanced computational approaches are needed to better align risk assignment to longitudinal exposure and actual events.

Table 1.

Validation of PARIS Risk Scores in Patients With Acute Coronary Syndrome

  PARIS Registry10Independent PARIS validation on ACS patients
  Raposeiras-Roubín et al.11  Abu-Assi et al.15  Song et al.16 
Study description  Population: 4190 patients ACS and non-ACS patients
Study design: single arm registry with a 2-y follow-up
Treatment: clopidogrel or prasugrel Platform: DES 
Population: 4310 ACS patients
Study design: single arm registry with a 17 mo follow-up discharged on prasugrel or ticagrelor;
Platform: DES or BMS 
Population: 1926 ACS patients
Study design: ACS patients; Clopidogrel, ticagrelor, and prasugrel
Platform: DES or BMS 
Population: 6088 ACS patients Study design: ACS patients discharged on clopidogrel;
Platform: DES 
MB  2-y: 3.3%  1-y: 1.6%  1-y: 2.8%  1-y: 0.32%;
2-y: 0.5% 
CTE  2-y: 3.8%  1-y: 1.9%  N/A  1-y: 2.2%;
2-y: 2.62% 
c-statistic  MB:
0.72 (derivation)
0.64 (validation)
0.70 (derivation)
0.65 (validation) 
MB: 0.56
CTE: 0.64 
MB: 0.73
CTE: 0.57 

ACS, acute coronary syndrome; BMS, bare metal stents; CTE, coronary thrombotic events; DES, drug-eluting stents; MB, major bleeding; N/A, not available.

From a clinical perspective, the present findings introduce the concept of potentially tailoring therapy for ACS patients, a concept that remains somewhat discordant with current practice guidelines, which recommend at least 1 year of DAPT in such patients. However, a growing evidence base appears to challenge this convention and may ultimately influence practice. To date, 1 randomized trial has compared a short vs long DAPT duration in an ACS cohort and has demonstrated excess risk for coronary thrombosis with short DAPT durations (SMART-DATE trial).13 Conversely, the GLOBAL-LEADERS trial found that antiplatelet monotherapy with ticagrelor was associated with a similar risk of death or MI compared with a conventional DAPT regimen in ACS patients.14 Other studies suggest that lessening the intensity of platelet inhibition by switching to clopidogrel rather than stopping DAPT, or de-escalation, may also be comparable to conventional DAPT regimens consisting of prasugrel or ticagrelor. Finally, the ongoing SHORT-DAPT trial (NCT03218787), which is enrolling patients at high bleeding risk with or without ACS, will test the safety of 3 months of DAPT followed by aspirin monotherapy.

In aggregate, it appears that the initial trend toward shorter DAPT durations is also being extended to ACS patients, albeit in a more nuanced and incremental fashion. Given the degree of thrombotic risk that persists after an acute MI, it will be essential to properly characterize thrombotic and bleeding risk in a reliable and accurate fashion to inform clinical decisions surrounding the duration or intensity of platelet inhibition in such patients. Validating existing tools within ACS cohorts as performed by Raposeiras-Roubín et al.11 is an important first step in this direction. Ultimately, randomized studies that incorporate such tools to guide decisions, as is being performed in the ongoing MASTER DAPT randomized trial (NCT03023020), will be the best arbiters regarding the usefulness of a tailored DAPT approach in high-risk ACS patients.


R. Mehran reports personal fees from Abbott Laboratories, Abiomed, Boston Scientific, CardioKinetix, Cardiovascular Systems, Inc, Medscape, Siemens Medical Solutions, Spectranetics, The Medicines Company, Roivant Sciences, Inc, Volcano Corporation, AstraZeneca, Bayer, Beth Israel Deaconess, BMS, CSL Behring, Eli Lilly/ DSI, Medtronic, Novartis Pharmaceuticals and OrbusNeich; R. Mehran also reports other relationships with Claret Medical, Elixir Medical, Janssen Pharmaceuticals, Osprey Medical, Bristol-Meyers Squibb and Watermark Research Partners outside the submitted work. U. Baber and S. Sorrentino have no conflicts of interest.


G.N. Levine, E.R. Bates, J.A. Bittl, et al.
2016 ACC/AHA guideline focused update on duration of dual antiplatelet therapy in patients with coronary artery disease: A report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines.
J Thorac Cardiovasc Surg., 152 (2016), pp. 1243-1275
M. Valgimigli, H. Bueno, R.A. Byrne, 2017 ESC focused update on dual antiplatelet therapy in coronary artery disease developed in collaboration with EACTS: The Task Force for dual antiplatelet therapy in coronary artery disease of the European Society of Cardiology (ESC) and of the European Association for Cardio-Thoracic Surgery (EACTS), et al.
Eur Heart J., 39 (2018), pp. 213-260
L. Mauri, D.J. Kereiakes, R.W. Yeh, et al.
Twelve or 30 months of dual antiplatelet therapy after drug-eluting stents.
N Engl J Med., 371 (2014), pp. 2155-2166
F. Costa, D. van Klaveren, S. James, et al.
Derivation and validation of the predicting bleeding complications in patients undergoing stent implantation and subsequent dual antiplatelet therapy (PRECISE-DAPT) score: a pooled analysis of individual-patient datasets from clinical trials.
Lancet., 389 (2017), pp. 1025-1034
T. Palmerini, L. Bacchi Reggiani, D. Della Riva, et al.
Bleeding-Related Deaths in Relation to the Duration of Dual-Antiplatelet Therapy After Coronary Stenting.
J Am Coll Cardiol., 69 (2017), pp. 2011-2022
S. Sorrentino, G. Giustino, U. Baber, et al.
Dual Antiplatelet Therapy Cessation and Adverse Events after Drug-Eluting Stent Implantation in Patients at High Risk for Atherothrombosis. (From the PARIS Registry).
A. Carino, S. De Rosa, S. Sorrentino, et al.
Modulation of Circulating MicroRNAs Levels during the Switch from Clopidogrel to Ticagrelor.
Biomed Res Int., 2016 (2016), pp. 3968206
D. Sibbing, D. Aradi, C. Jacobshagen, et al.
A randomised trial on platelet function-guided de-escalation of antiplatelet treatment in ACS patients undergoing PCI. Rationale and design of the Testing Responsiveness to Platelet Inhibition on Chronic Antiplatelet Treatment for Acute Coronary Syndromes (TROPICAL-ACS) Trial.
Thromb Haemost., 117 (2017), pp. 188-195
R.W. Yeh, E.A. Secemsky, D.J. Kereiakes, et al.
Development and Validation of a Prediction Rule for Benefit and Harm of Dual Antiplatelet Therapy Beyond 1 Year After Percutaneous Coronary Intervention.
JAMA., 315 (2016), pp. 1735-1749
U. Baber, R. Mehran, G. Giustino, et al.
Coronary Thrombosis and Major Bleeding After PCI With Drug-Eluting Stents: Risk Scores From PARIS.
J Am Coll Cardiol., 67 (2016), pp. 2224-2234
S. Raposeiras-Roubín, B. Caneiro Queija, F. D’Ascenzo, et al.
Usefulness of the PARIS Score to Evaluate the Ischemic-hemorrhagic Net Benefit With Ticagrelor and Prasugrel After an Acute Coronary Syndrome.
R. Mehran, U. Baber, P.G. Steg, et al.
Cessation of dual antiplatelet treatment and cardiac events after percutaneous coronary intervention (PARIS): 2 year results from a prospective observational study.
Lancet., 382 (2013), pp. 1714-1722
J.Y. Hahn, Y.B. Song, J.H. Oh, et al.
6-month versus 12-month or longer dual antiplatelet therapy after percutaneous coronary intervention in patients with acute coronary syndrome (SMART-DATE): a randomised, open-label, non-inferiority trial.
Lancet., 391 (2018), pp. 1274-1284
P. Vranckx, M. Valgimigli, P. Juni, et al.
Ticagrelor plus aspirin for 1 month, followed by ticagrelor monotherapy for 23 months vs aspirin plus clopidogrel or ticagrelor for 12 months, followed by aspirin monotherapy for 12 months after implantation of a drug-eluting stent: a multicentre, open-label, randomised superiority trial.
E. Abu-Assi, S. Raposeiras-Roubin, R. Cobas-Paz, et al.
Assessing the performance of the PRECISE-DAPT and PARIS risk scores for predicting one-year out-of-hospital bleeding in acute coronary syndrome patients.
EuroIntervention., 13 (2018), pp. 1914-1922
L. Song, C. Guan, H. Yan, et al.
Validation of contemporary risk scores in predicting coronary thrombotic events and major bleeding in patients with acute coronary syndrome after drug-eluting stent implantations.
Catheter Cardiovasc Interv., 91 (2018), pp. 573-581

The authors contributed equally to this work.

Revista Española de Cardiología (English Edition)

Subscribe to our newsletter

Article options
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?