The scientific approach used to predict patients’ futures (prognosis) and health care planning is based on estimating risks and benefits. Cardiovascular risk functions are a paradigmatic example of this effort applied to clinical medicine: well-established facts (eg, readily measured risk factors such as diabetes, lipid panel, smoking) can be used to determine the risk of a specific clinical course (cardiovascular disease in the next 10 years) within a margin of error.1 As knowledge advances, this starting point can be gradually improved by broadening the factors used to predict the functions, such as polygenic risk scores.2 If someone has a risk of 1%, we know that 1 out of every 100 similar patients will have a coronary event in the next 10 years.1 However, the issue is that we do not know who those patients are. We resignedly accept this situation because the effort and expense needed to neutralize this low incidence is unacceptable. In contrast, in patients at high or very high risk (> 10% over 10 years), the resources become cost-effective: the population at this level of risk is much smaller, and the number of preventable events is much higher.

To make decisions that ensure equal opportunities and optimal investment in resources, health care planners use information on the incidence and prevalence of diseases, their possible relationship with the use of health resources (cost-effectiveness studies), the benefits and undesirable effects of interventions, and socioeconomic factors relating the use of resources to inpatient mortality for some cardiovascular diseases, as recently published in Revista Española de Cardiología by de la Torre Hernández et al.3 This study concluded that the use of cardiologic technologies varies considerably between autonomous communities, a variability that cannot be explained by differences in gross domestic product, health expenditure per capita, or the number of hospital visits due to the conditions analyzed. Substantial effort has been made to obtain a large database on cardiologic health care activities in various autonomous communities. The study indicates that financial factors are not the reason for intercommunity variability in the use of percutaneous coronary interventions in general, most especially implantable cardioverter-defibrillator, cardiac resynchronization therapy, and percutaneous aortic valve implantation. This variability could be explained by other factors:

• •

  Incidence of heart conditions benefiting from these techniques, now replaced with RECALCAR hospitalization frequency indices. Reports on the Spanish Society of Cardiology (SEC) website4 acknowledge that the minimum data set (MDS) supplied by the Ministry of Health, Consumer Affairs and Social Welfare has coding and coverage limitations. The RECALCAR survey4 itself is voluntary, although it covers 77% of the 161 public hospitals with cardiology units that it identified. The study excluded 229 private facilities with a cardiology department and another 122 facilities at 283 public hospitals with a cardiology department identified by the Ministry of Health in its 2016 report.5 This is possibly because they do not fit RECALCAR's definition of “cardiology unit,” which excludes hospitals with fewer than 200 beds.4 Private hospitals account for 20% of all operational beds. In 2016, public facilities had 3.49 interventional cardiology units per million inhabitants, and private facilities another 2.15, according to the Ministry of Health report.5 The RECALCAR report also mentions that the cardiologic technology resources available per million inhabitants are above the international standards accepted as sufficient.

• •

  The population mortality due to cardiovascular diseases—or weighted by ischemic heart disease, heart failure, etc.—is a data point available for each autonomous community and could have been used for the adjustment instead of the risk-adjusted mortality ratio (RAMR), which measures risk-adjusted inpatient mortality due to these causes in hospitals with available data. The RECALCAR publications6 refer to the method previously described, and the adjustment method is further explained in a 2013 publication.7 This indicator is provided in all reports without a 95% confidence interval, which is sensitive to sample size and in many cases would likely show that the rates in many hospitals are similar. In addition, the RAMRs of the hospitals in each community were averaged without weighting according to the number of patients provided or considering the confidence intervals of these quality estimators. In contrast, the total or community-specific population mortality is not affected by sample size and would be a population system used to make an additional adjustment for differences between autonomous communities.

• •

  The mean age of the population and the life expectancy of each community may both influence and be influenced by the incidence and prevalence of heart disease. These indicators are available from the National Institute of Statistics and are also robust.

• •

  The indication for the procedures studied is a critical point when evaluating the differences in clinical practice between regions or hospitals. The key is to have the necessary information on the indications and contraindications for each case eligible for a certain technique or device. Health care variability is an important component based on the well-reasoned decision on whether or not to use a specific diagnostic or therapeutic technique. This decision depends mainly on the characteristics of individual patients. If the database does not include these characteristics, allowing the indication to be analyzed, then it becomes difficult to judge how many procedures have been carried out properly, how many could have been undertaken and were not, and how many were not performed due to a properly assessed contraindication.

• •

  Variability in clinical practice. Last, the component of insufficient implementation of good clinical practice established in the guidelines would not be explained by the above points, when correctly analyzed.

Out of all the possible designs in biomedical research, that used by de la Torre Hernández et al.3 would be classified as an ecologic correlation study. In this type of study, the unit of comparison is no longer an individual but a population group, in this case an autonomous community. This type of design was used to create the lipid theory of arteriosclerosis and its possible link to the Mediterranean diet more than 50 years ago.8 Admittedly, the ability of this type of design is very limited because it is subject to many potential uncontrollable biases. Therefore, the inferences that can be made are also limited, and its usefulness is limited to the generation of hypotheses that should be tested using more powerful research designs. If the proportion of fatal motorcycle accidents were analyzed, this proportion would probably be seen to be lower in countries with helmet laws, but other factors may also be in play, such as the minimum age for motorcycle use in each country, speed limits, guardrails on roadway curves, infrastructure quality, and other factors we most certainly cannot imagine.

An approach based on analyzing differences between regions using an ecological design, in which individuals are replaced by groups of individuals, is always risky and conflicts with issues on sample representativeness and information biases, diagnostic reliability, health care resources, and varied clinical practices, most of which are not accurately known.9

The sources used by de la Torre Hernández et al.3 are the National Institute of Statistics for economic data from the autonomous communities, publications on health care activity by the Registry of the Spanish Society of Cardiovascular and Endovascular Surgery, the Spanish Registry of Hemodynamics and Interventional Cardiology of the Interventional Cardiology Association of the Spanish Society of Cardiology, the Spanish Pacemaker Registry of the Heart Rhythm Association of the Spanish Society of Cardiology, and the RECALCAR registry of the Spanish Society of Cardiology.

Most registries are based on anonymous, unaudited voluntary reports on health care activity at several hospitals in the autonomous communities. There are 764 hospitals in Spain, of which 421 are private.5 Assuming that all hospitals with a cardiology unit can perform pacemaker implantation, then 512 (67%) hospitals offer this procedure. The 17th Official Report of the Cardiac Pacing Section of the Spanish Society de Cardiology of 2019 presents results from its registry of 109 hospitals which voluntarily participated, and the activity reported accounted for 39% of the level declared by the pacemaker industry.10 In the case of cardiovascular surgery, 49 to 64 facilities were included, depending on the year, out of the 146 hospitals that had cardiac surgery departments in Spain in 2016.5,11 The Hemodynamics and Interventional Cardiology Registry is working on its latest report on 109 hospitals (83 of them public) with catheterization laboratories11 among the 254 existing in 2016.5

None of these registries (neither in the article by de la Torre Hernández et al.3 nor in the registries mentioned10–12) analyze the sample representativeness of hospitals according to autonomous community except for RECALCAR, which places it at a minimum of 60% of public hospitals with cardiology units and at least 200 beds that were invited to participate. Because the information required and available to assess health care quality is incomplete, it is necessary to ask if combining information from the sources used would make it possible to reach sufficiently reliable conclusions at this scale.

A more efficient alternative to a geography-based analysis would be to create objective, robust, independent, and universal indicators of health care quality that could be applied to all centers or to sentinel centers. These indicators should be adjusted for local characteristics (geography, population, and patients) and reported with a 95% confidence interval.

Some years ago, 2 European projects (EURHOBOP13 and EUROTRACS14) developed a tool to establish the percentile of quality of a hospital treating patients with myocardial infarction in the context of European hospitals and according to country, type of hospital, and characteristics of the patients received. The model was created with about 15 000 patients with acute coronary syndrome from 6 European countries, was validated in another 55 000 patients, and had excellent calibration (the c-statistic had a 95% confidence interval of 74%-82%). In the ATHOS study,15 each participating hospital received an individual and confidential assessment of their objective performance in treating myocardial infarction patients based on the EURHOBOP study. Unlike the RAMR, the EURHOBOP system directly provides the percentile ranking of the hospital assessed within the context of hospitals with same characteristics, according to the type of patients received. An online app is available for completely anonymous assessments.16 These types of tools fit perfectly with the SEC-EXCELENTE program.17 In addition, participation bias may always be involved: it is likely that only hospitals who believe they have good outcomes would participate. However, the confidentiality of the information and the possibility to perform anonymous online self-evaluations most certainly encourage curiosity and a desire to improve. A suboptimal result could lead to a pursuit of consulting, implementation of an improvement program, and a subsequent certification quality process.

The STROBE (Strengthening the Reporting of Observational studies in Epidemiology) statement18 lists 22 essential items that should be mentioned and taken into account in any observational study. We would like to specifically mention the following:

Number 3 requires listing the specific objectives, including any prespecified hypotheses. This last point is key to understanding what is expected to be seen in an ecologic study. Specifying the objectives also makes it possible to determine which ones have the necessary statistical power, assuming an accurate estimate of the sample size, and which ones lead to hypotheses.

Number 6 requires that eligibility criteria, sources, and participant selection methods be provided.

Number 10 requires an explanation of how the sample size was determined, or otherwise which statistical power provides the sample size available to fulfill the main objective.

Point 12(a) requires that all statistical methods be specified, including those used to control confounding factors. When a significant number of hypotheses are compared, it is important to correct the P values by multiple comparisons to reduce the number of spurious relationships.

Point 13(a) involves describing the number of participants in each study phase, eg, figures for potentially eligible participants and facilities, those analyzed for inclusion, those confirmed as eligible, those initially included in the study analysis plan, those who had complete follow-up, and those finally analyzed, as well as the reasons for losses to follow-up in each phase. The information should be presented in a flow chart with these descriptive data.

Point 19 invites an honest discussion on the study limitations, taking into account sources of potential bias or imprecision and a rationale for both the direction and extent of any potential bias.

Last, point 21 requires discussing the generalizability of the results, ie, the external validity of the study.

In conclusion, the team of de la Torre et al. has made a laudable effort to obtain and combine data from such a variety of sources, as well as to analyze them thoroughly and to draw conclusions from the results. Nevertheless, we would like to propose to the editors that, to better understand the internal and external validity of observational, descriptive ecologic studies on health care activity that are published, the STROBE publication guidelines should be followed.18 This would allow the representativeness and validity of the data provided to be assessed objectively. Irrespective of this consideration, we believe that the ideal level for assessing health care quality is the health care center, as groupings of facilities by region or health care level are subject to many influencing factors, many of which cannot be known or, consequently, controlled with sufficient precision.