Data were collected from the participants in the ENRICA study, a prospective cohort designed to study CVD risk factors.20 The study comprises 13 105 individuals representative of the noninstitutionalized Spanish population aged 18 years or older. Details on study design and data collection have been reported elsewhere.20 Briefly, the sample was selected using stratified cluster sampling.20 Baseline data collection was carried out between June 2008 and October 2010 in 3 sequential steps.20 First, trained personnel conducted a telephone interview on sociodemographic factors, CVD risk factors, health behaviors, and morbidity. Second, trained nurses obtained blood and urine samples in a first home visit; and finally, a physical examination and a face-to-face diet history were performed in a second home visit.20

The HHS was developed to predict 20-year risk of CVD separately for women and men.16 This tool was the most parsimonious, rather than comprehensive, model for predicting CVD risk and the 9 components selected were: smoking status, BMI, physical activity, intake of cereal fiber, fruit and vegetables, red and processed meat, nuts, sugar-sweetened beverages, and alcohol consumption (figure 1 of the supplementary data). All data required deriving the original sex-specific beta coefficients in ENRICA, which were measured at baseline (2008-2010). The dietary components of HHS were estimated from food consumption over the preceding year obtained with a validated electronic diet history,21 where participants could report up to 861 foods and recipes commonly consumed in Spain. Portion sizes were estimated with the help of 127 digitized photographs and household measures. Leisure-time physical activity was assessed with the Spanish-validated version of the EPIC-cohort questionnaire,22 which we used to calculate moderate-to-vigorous physical activity (3 to 6 metabolic equivalent tasks). BMI was calculated as weight (in kilograms) divided by height (in meters) squared. Grams of alcohol consumed were estimated from the diet history using standard food composition tables.21 Self-reported information on age, sex, and smoking status was collected during the telephone interview. Finally, a higher overall HHS indicated higher risk for CVD.

The main outcome of this study was CVD death; a secondary outcome was all-cause death. Vital status and date of death were retrieved from the National Death Index of Spain, while cause of death was obtained from death certificates collected by the National Institute of Statistics of Spain23; the causes of death were coded by trained nosologists according to the International Classification of Diseases, 10th revision.24 We considered deaths coded from I00 to I99 as deaths from CVD. Matching to the databases was made by investigators blinded to the predictor data using combinations of first and last names, birthdates, and national identity card numbers.25 Length of follow-up was time between the telephone phone interview and date of death or administrative censoring (December 31st, 2020, for CVD mortality; January 31st, 2022 for all-cause mortality), whichever came first.

We followed a complete-case analysis approach using all available observations with complete data on outcomes and predictors. Thus, of the 13 105 participants, those with missing data on smoking status (n = 43), BMI (n=952), alcohol consumption (n=111) or energy intake (n=58) were sequentially excluded, leaving a first analytical sample (hereinafter overall cohort) of 11 941 participants (figure 2 of the supplementary data). A second analytical sample was obtained by additionally excluding those with chronic conditions at baseline (type 2 diabetes [n=790], CVD [n=192], lung disease [n=642], cancer [n=89]), leaving 10 228 participants (4794 men, 5434 women) (figure 2 of the supplementary). Participant characteristics were summarized using descriptive statistics including the mean± standard deviation (SD) for continuous variables, and absolute and relative frequencies for categorical variables.

The performance of the HHS in ENRICA participants to predict 12-year CVD mortality was estimated using sex-specific beta-coefficients (figure 1 of the supplementary), and assuming half of the baseline hazards, derived within the NHS (women) and HPFS (men), due to the difference in follow-up (12 years vs 20 years). Evaluation of performance was based on the discrimination and calibration of the model. Model discrimination was assessed using the Harrell c-statistic for survival data and its 95% confidence interval, with values between 0 and 1 (values closer to 1 indicated better discrimination). We also calculated the Gonen-Heller c-statistic with its 95% confidence interval to account for possible bias due to the censoring pattern, as has been previously suggested.26 Model calibration was assessed using calibration slopes and visualized through calibration plots representing the average predicted risk against the observed risk in deciles. Since the HHS model was developed for a longer-term risk (20 years), for CVD incidence, and in a population with baseline hazard different to that in Spain, we considered the recalibration based on the baseline hazard risk of our population without modifying the specific beta coefficients (figure 3 of the supplementary). We estimated model calibration and discrimination of the calibrated HHS using the same procedures. Finally, we evaluated the calibration performance of the HHS and HHS-cardiovascular disease model (CVDm) by age groups.

We conducted several sensitivity analyses: we evaluated the calibration performance of the HHS and HHS-CVDm by categories of risk defined by the European Society of Cardiology Guidelines.27 Because the baseline CVD-free survival at 12 years in the original HHS was unknown, we estimated model performance for several scenarios, assuming that a) all events occurred in the first 12-years of follow-up (ie, the same baseline CVD-free survival at 12 years as at 20 years [0.966 for women and 0.964 for men]), and b) all events occurred beyond 12 years of follow-up (ie, baseline CVD-free survival at 12 years was close to 1). Finally, we repeated all described procedures to evaluate the predictive performance of the HHS on all-cause mortality.

Analyses were performed using Stata version 16.0 (Stata-Corp LLC, College Station, United States). All P-values were considered statistically significant at P <.05 (2-sided).

The study was approved by the Clinical Research Ethics Committee of La Paz University Hospital in Madrid (Trial registration: NCT02804672), and study participants gave written informed consent.

Baseline characteristics of the 11 941 overall participants, the 10 228 participants without chronic diseases, and the excluded participants are presented in table 1. The mean age at baseline was 47.1±16.5 years for men and 47.7±16.9 years for women in the overall cohort and was 45.3±15.8 years and 46.1±16.3 years, respectively, in the participants without chronic diseases. In both analytic samples, ∼53% were women. Overall, the participants without chronic diseases were younger and had a slightly lower HHS (better) than the overall cohort. No major significant differences were found between included and excluded participants. Compared with the original cohorts in which the HHS was developed, our participants were younger, with higher BMI, higher consumption of cereal fiber, and lower consumption of fruits and vegetables (table 1 of the supplementary data). In addition, men were more physically active and more likely to be current smokers. The main baseline differences between other cohorts where the HHS has been validated (CARDIA17 and the Jackson Heart Study18) and the ENRICA cohort are presented in table1 of the supplementary data.

After a median follow-up of 11.8 [range 0.9-13.4] years, there were 211 CVD deaths (112 men, 99 women). In those without chronic diseases, 110 CVD deaths (59 men, 51 women) were ascertained. The discrimination of the HHS was similar for women (Harrell c-statistic, 0.91; 95%CI, 0.88-0.94) and men (Harrell c-statistic, 0.89; 95%CI, 0.87-0.91) in the overall cohort; however, discrimination was the same for men and women without chronic diseases (Harrell c-statistic, 0.91; 95%CI, 0.88-0.94 for men, and Harrell c-statistic, 0.91; 95%CI, 0.87-0.95 for women) (table 2). The Gonen-Heller c-statistic followed the same pattern but was consistently lower in all cases (Gonen-Heller c-statistic, 0.85; 95%CI, 0.83-0.88 for men, and Gonen-Heller c-statistic, 0.85; 95%CI, 0.83-0.88) for women (table 2). The original HHS had a poor calibration performance in both the overall cohort (slope, 1.48; 95%CI, 1.25-1.71 for men, and slope, 1.28 95%CI, 1.10-1.46 for women) and in those without prior chronic diseases (1.61; 95%CI, 1.30-1.92 for men, and 1.28 95%CI, 1.04-1.52 for women) (table 2). It consistently overestimated the risk of CVD mortality for men and women, particularly among the highest deciles of predicted risk (figure 1).

Figure 1.

Calibration plots (95%CI) of HHS for the prediction of CVD mortality among ENRICA participants (median follow-up of 12 years) assuming the baseline survival function at 12 years in the original development cohorts is half than that at 20 yearsa.

aParticipants free of type 2 diabetes, cardiovascular disease (myocardial infarction, heart failure, stroke), lung disease (asthma, chronic obstructive pulmonary disease), and cancer at baseline.

bn=211 / n=11 941 (men 112/5649, women 99/6292).

cn=110 / n=10 228 (men 59/4794, women 51/5434). Baseline survival function for men, 0.98184; Baseline survival function for women 0.983.

95%CI, 95% confidence interval; CVD, cardiovascular disease; CVDm, cardiovascular disease model; HHS, Healthy Hearth Score.

(0.2MB).

We recalibrated the HHS using our sex-specific baseline CVD survival function (HHS-CVDm) (figure 3 of the supplementary data). The HHS-CVDm calibration became optimal for all but the highest decile of predicted CVD risk of both men and women, where the calibrated HHS underestimated the risk (figure 2). Additionally, the observed and predicted risks of the HHS-CVDm were similar across all age groups except for those> 70 years where there was an underestimation, especially in women (figure 3).

Figure 2.

Calibration plots (95%CI) of the HHS-CVDm for the prediction of CVD mortality at median follow-up (12 years) among the overall ENRICA cohort and for a subset of participants without chronic diseasesa recalibrating by the baseline survival function of the ENRICA cohort.

aParticipants free of type 2 diabetes, cardiovascular disease (myocardial infarction, heart failure, stroke), lung disease (asthma, chronic obstructive pulmonary disease), and cancer at baseline.

bn=211 / N=11 941 (men 112/5649, women 99/6292).

cn=110 / N=10 228 (men 59/4794, women 51/5434). Baseline survival function for men, 0.9999935; Baseline survival function for women 0.9999937.

95%CI, confidence interval; CVD, cardiovascular disease; CVDm, cardiovascular disease model; HHS, Healthy Hearth Score.

(0.16MB).

Figure 3.

Calibration of the HHS-CVDm across different age groups in men and women. HHS, Healthy Hearth Score; CVDm, cardiovascular disease model.

(0.23MB).

In addition, the mean predicted risks became closer to the observed risks for both risk categories (< 2.5% and ≥ 2.5%) when using the HHS-CVDm (table 3).

In sensitivity analysis, we assessed the performance of the HHS in our population assuming alternative extreme scenarios of survival function of CVD at 12 years (figure 4 and 5 of the supplementary data). Similar results were observed when we used the HHS to predict all-cause mortality over a median 12.9 years of follow-up with 938 all-cause deaths (525 men, 413 women). Although the original HHS showed good discrimination for men and women in both the overall cohort and in those without chronic diseases (table 2 of the supplementary data), its calibration performance was poor across most deciles of predicted risk (figure 6 of the supplementary data) but improved to optimal when offset by the baseline survival hazard (figure 7 of the supplementary data).

One of the strengths of the HHS vs other risk equations is that it only includes modifiable lifestyle factors and there is no need for laboratory measurements to calculate risk. This is a strategic advantage for its usefulness in the clinical setting, particularly in low- and middle-income countries with poor access to laboratory services. In addition, the HHS focuses on the most critical health behaviors that can be individually monitored to help improve CVD health, either in nonclinical settings or within health promotion clinics, including in routine work-related health checkups. In addition, while some of the barriers for implementing tools in clinical practice is “time”, a previous qualitative study to assess the strengths, weaknesses and clinical usefulness of this tool among patients and health providers reported an average of 4minutes to finalize the assessment and the results supported the use of the HHS as a CVD primordial prevention tool.38

There were differences in the follow-up time (20 years in the developmental cohorts vs 12-years in our cohort) and outcome (CVD incidence vs CVD deaths) between de NHS and HPFS cohorts in the United States and the ENRICA cohort in Spain. However, we simulated different scenarios with different baseline hazards and, when we additionally offset the baseline survival function, a good HHS performance in the ENRICA cohort was achieved without requiring further adjustments. Furthermore, although information on more than 90% of the total cohort participants was analyzed, most of the exclusions were due to missing data on BMI. Since this is an important risk factor for CVD incidence and mortality, by excluding these participants, we could have introduced some bias in our estimates; however there were no major differences between included and excluded participants. Finally, the resulting limited number of CVD deaths might reduce the precision of our estimates.