There is growing interest in the study and understanding of the social impact of scientific research. Recent decades have seen the development of a discipline, or a set of disciplines, with various objectives within that common purpose, known as research impact assessment (RIA). More and more research teams in various countries are dedicated to RIA, generally commissioned by management bodies, but also independently driven by social or scientific interest. One of the most important motivations of RIA is awareness of the ever-increasing volume of all types of research activities in the globalized world, their enormous costs, and the requirement to manage them according to their benefits, which should be well known and evaluated. A good example is provided by cardiovascular research, due to its volume and significance and given its outstanding results in recent decades. Cardiologists and cardiovascular researchers should be aware of the role played in society by their research activity and of the usefulness of its evaluation.

The concept of research impact is heterogeneous and has various definitions. In the present reflection, research impact is considered any effect, change, or benefit produced by research in the economy, society, health care policy or services, health, environment, or quality of life,1 particularly when the nonacademic community perceives this change to be of added value. Various illustrative examples of what can be understood by research impact and, thus, the objective of RIA, are shown in the Table.

Research impact assessment has found a conceptual basis in such notions as assuming that society is entitled to the benefits of research and that knowledge production is in itself valuable or useful. An elaboration of this idea is payback, a concept encompassing the eventual return to society of any material or human resources provided for the research.2 Research impact assessment is largely based on this conceptual model and involves evaluation of this payback to determine its characteristics and determinants, thereby facilitating the allocation and distribution of material and human resources for research. From an economic point of view, RIA allows determination of the real use made by research of the resources received from society in the form of taxes and donations. In addition, society would have the right to know this information to improve the management of these resources according to the demands of different social and scientific groups. According to this approach, RIA would help to streamline resources and plan research according to the obtained and expected impacts. For example, a government agency in the United Kingdom (Higher Education Funding Council of England [HEFCE]) has proposed, as one of the criteria for the funding of university centers, the level of the distinct impacts reached of their previous research activities outside the centers themselves, which has stimulated the detailed research of the distinct impacts achieved.3

Although this economic view has so far dominated RIA, research impact can be assessed from other points of view. From the point of view of human solidarity, if it is considered of value that society benefits from the information produced, one requirement of research should be that it promotes the diffusion and application of any knowledge and innovations generated. This standpoint also considered medicine to be a knowledge-oriented activity conducted for the common good, with research as one of its essential pivots. In this case, RIA can help to determine how research is conducted and what barriers and facilitators it encounters, thereby helping to improve it, that is, to enrich understanding of the so-called “pathways to impact”.4

The consideration of RIA as the study of the bilateral relationship between the researcher and society involves recognition of a tacit social contract between them. This contract represents the commitment of the research community to socially relevant and worthwhile work. Thus, broadly speaking, RIA is an activity aimed at determining and evaluating the characteristics and effects of the interaction between the scientific community and society as a whole.

One characteristic of RIA is the multiplicity of its target audiences. In addition to politicians and managers, other important groups include the researchers themselves and clinicians in general (not to mention patients and the public) who are among the end users of the research. The knowledge generated by RIA can not only help these end users to better understand their work and the world in which it is performed, but can also contribute to more responsible research with greater impact. Thus, RIA should play a more important role beyond its usefulness for management and planning.

Research impact assessment is essentially heterogeneous. No wonder it is called a set of disciplines. Its methods vary according to the objectives, which, as mentioned above, also depend on their beneficiaries or target audiences. This is easily understood by recalling that the notion of research impact refers more to a group of social effects, more or less disparate, than to a “single” homogeneous impact. Each of these impacts might interest diverse target audiences and there will be distinct methodologies for their study, combined in various ways. Thus, bibliometric analysis reflects academic success but also sheds light on collaborations; peer review can provide guidance on the strength and effectiveness of the research; qualitative case studies demonstrate how and why the projects achieved their impacts and the pathways followed; and estimates of the quality-adjusted life year (QALY) gained, surveys, analysis, and economic modeling allow estimation of the effects on health and economy, etc. Very broadly speaking, these methods can be applied from the bottom up, by studying the pathways followed by the research projects until they achieve a specific impact (eg, adoption of an innovation), or from the top down, by investigating what types of research and their pathways can be attributed to a specific type of impact (eg, reduced frequency of disease due to an intervention or economic benefit in society).

Research impact assessment poses many methodological problems: it involves a discipline undergoing continuous development and with a long way to go. In addition to the classical problems such as that of attribution (how to establish to what extent a certain impact can be linked to a project or projects), the debate persists on the relative role of quantitative and qualitative methods. The predominance of the former has been blamed for excessive emphasis on direct and quantifiable impacts at the expense of remote and indirect impacts, although the latter are potentially more significant for society.5 In addition, RIA entails a considerable commitment of time, effort, and money, and there is a need to establish which processes and methods are appropriate and efficient.

The main contributions of RIA to cardiology research can be considered in 3 dimensions:

• •

  In very general terms, RIA has determined that the long-time economic return from cardiovascular disease research has been considerable. Various studies (particularly those from the United States6 and Australia7) have considered the notable increase during the second half of the 20th century in the mean survival of the population affected by coronary heart disease, by far the most prevalent disease, and have attributed one- to two-thirds of this increase to the application of research findings (on therapy or prevention). Notably, this increased survival has not been seen in other diseases (cancer or infections). Using various estimates, the authors calculated that the monetary value of this survival exceeded by a factor of up to 2 or 4 the investment that permitted the research and its findings. These first studies of coronary heart disease research became emblematic or a paradigm of the potential benefits of RIA. With a distinct orientation and range of presumptions, in 2008 British authors estimated a lower but still substantial benefit for the United Kingdom from cardiovascular and mental health research (grossly one-third of the investment).8 This benefit was due to health gains and national income improvements. The authors first calculated the amount of research investment. They then estimated the QALY gained between 1985 and 2005 via medical interventions introduced due to this research in 46 combinations of patient groups and interventions. To estimate the benefit, they used a complex methodology involving huge databases and economic evaluations. Thus, the first conclusion of RIA in cardiology was that, overall, cardiovascular research has been a fruitful investment in economic terms for the population as a whole, a useful message for politicians and high-level managers and inspirational for researchers.

• •

  The same group from the United Kingdom, which in various seminal studies established a complex case study-based methodology for the identification and categorization of research impacts, has addressed the question of how research projects are translated into social impacts beyond bibliometrics and academia. The Retrosight (2011) project9 identified basic and clinical research grants awarded for cardiovascular disease and stroke between 1989 and 1993 in the United Kingdom, Canada, and Australia. Via meticulous case studies of the pathways of these projects over time, the project identified several factors favoring the production of impacts, grouped into academic impacts (bibliometric and training) and impacts with greater social range (organizational, health, and economic). Basic research was associated with greater bibliometric impact, but chief among the factors favoring greater social impact was professional cooperation between basic research and clinical or health care research, as well as the inclusion of transfer and application objectives in basic research projects. Engagement with clinicians and patients was also found to facilitate academic and social impacts. The project included detailed case studies illustrating how these factors (eg, genetic studies of fibrillin deficit in Marfan syndrome) have permitted their complex academic development in the form of spin-off projects, diverse institutional dissemination, and application to clinical practice, which eventually lead to health gains.

• •

  Research impact assessment has also contributed to an essential step in the complex pathway from the research project itself to its final impact on health care due to incorporation of the findings into clinical practice. Numerous studies of the follow-up and degree of adherence to research-derived recommendations—such as clinical practice guidelines—belong to the literature on health care quality but also contribute to RIA and share a common territory with it, especially in what is referred to as results attributable to well-defined projects. Indeed, one aspect of RIA is aimed at analyzing and potentiating the dynamics of the relationship between the distinct actors of the research process and the eventual agents of their implementation. These are the so-called productive interactions.10 Similarly, other studies focus on the time lag between discovery and application.11 In cardiology, numerous studies have examined adherence to research-derived recommendations, both in prevention12 and in treatment.13 These studies have also investigated the association of guideline adherence with better quality of care and health outcomes (as in the case of guidelines on acute coronary syndrome). The field of RIA clearly involves a deepening of the understanding of how these findings are produced or hindered and their relationship with specific projects.

To understand the potential relevance of RIA in cardiovascular research, it may be appropriate to recall some of its relatively distinctive features. First, cardiovascular disease shows the largest worldwide research volume of all biomedical research fields, with huge investment of resources. Second, the involvement of the pharmaceutical and medical device industries in this research is also essential in absolute and comparative terms, whether independently or in the form of public-private cooperation, with considerable economic and financial benefits. Finally, the speed of innovation in cardiology in recent decades has been astounding, with the generation of notable changes in clinical practice and health care results that remain to be fully elucidated. In this period, a huge number of preventive, diagnostic, and therapeutic innovations have been introduced, based on an also dazzlingly large volume of research. Therefore, there is no doubt that the impact of cardiovascular research in recent years has been extraordinarily large, with clear effects (not always positive) on public health.

Each of these general characteristics of cardiovascular research in recent decades could give rise to proposals on the possible applications of RIA in the present and future and on the cultural change in researchers possibly induced by these applications.

The strong pace of innovation and its results in clinical practice would seem to represent a particularly attractive and useful field for RIA. There are good reasons to wonder (or doubt) whether the fruits of the latest and current cardiovascular research will be followed by clinical practice changes and such momentous results as in past decades and what will be the net added value of their findings in prevention, diagnosis, and treatment. The strong involvement of industries and financial interests in cardiovascular research leads us to a general comment on RIA. Although there are studies on the economic dynamics of both public and private research, paradoxically, RIA has largely focused on evaluating projects funded with public money or via philanthropic organizations, whereas many of the studies generating changes in clinical practice or affecting medical behavior patterns (that is, whose impact in this regard has been very notable) have been promoted by private interests such as those of large pharmaceutical companies, although increasingly in close cooperation with academic institutions. Perhaps RIA, and not only in the cardiovascular field, should better address the study of the mechanisms through which privately funded studies are transferred to clinical practice and society in general; these mechanisms do not necessarily overlap with those used by publicly funded research. The field of RIA study could also consider the selection and prioritization of research topics according to the funding source (research into drugs and devices is more of a priority for academics and private industry organizations, whereas end users demand more or more extensive studies of issues such as education, health care, diet, exercise, and psychological and behavioral interventions)14. Studies of the economic performance of research (we have already seen that they were pioneers in the application of RIA to coronary heart disease) should probably be diversified to offer a true picture of the benefits generated and their social distribution.

The first of the notable characteristics of cardiovascular research—namely, its sheer volume and the huge investment it represents—is why we need to critically evaluate its uses and the relevant return on that investment. Previous studies raised the following issue: in recent years, the belief has spread that a not inconsiderable part of medical research, both basic and applied, and of the published work is futile, if not invalid; this futility is due to poor or redundant methodological approaches that lead to findings without merit or without added value that cannot reasonably be expected to have social impact.15 Cardiovascular research has apparently scarcely investigated this problem; however, it may represent a waste of money and professional energy. It has been proposed that RIA could help to reduce the volume of futile work (via improved management and planning and through a better understanding of the types of impacts anticipated for projects). In fact, although for partly different reasons, such as the profitability of research investments, one of the missions most strongly promoted by RIA enthusiasts is to help management agencies to prioritize certain lines of research based on their predicted impact. Another related example, also from the UK, might be the dissemination of the impacts of research funded by the Medical Research Council.16 But this is no easy task, with questionable aspects (too extensive to be discussed here), and largely pending effects. In any case, in the long term, a more accurate view of the extent and determinants of the problem should help to improve the social value of research. The volume of cardiology research, its capacity for innovation, and the involvement of powerful organizations17 increase the relevance of this aspect.

Apart from its innovative current applications, RIA might play a powerful and important role by enabling changes in the research community culture in 3 areas. First, as already noted, RIA can identify how and by what mechanisms research can have an impact. This provides the research community and funders with the tools to perform the necessary analysis to identify “what works” in research funding and, therefore, helps to prioritize future funding allocations.

Secondly, RIA permits a cultural change in how scientific research is rewarded. The more RIAs are performed and the more the results of these evaluations are rewarded, the more the researchers themselves are incentivized to systematically identify ways to improve their own personal prioritization and their research impact. This does not mean that RIA incorporation implies the end of blue skies or nongoal-oriented research, which will always have a place in science. Neither would this approach involve the complete replacement of scientific excellence evaluation (currently mainly involving academic journals and peer review). Simply, it would involve a desirable change in the research community culture, particularly in the medical sciences, which have much to offer society, as evident in the case of cardiology.

Finally, a third, more long-term, cultural change is proposed. This change is related to what can happen when the general population, who often fund the research, become aware of RIA results. For example, the Medical Research Council openly provides the public with information on the results of the research it has funded. The expansion of a RIA culture would increase the awareness of the cardiology research community and that of other fields of their connection with the public, whether consisting or not of the patient population with individual interest in the study. Such a change could significantly improve the commitment of the researcher to society.

The introduction and rise of RIA are directly associated with the need to determine the social and economic performance of research in order to improve its management in light of its huge costs. However, RIA can provide other important information for clinicians and researchers: it can help to determine and improve the pathways through which research becomes clinically applied, improving its efficiency. On a more general level, the development of RIA represents a contribution to the awareness of the interrelationship between researchers and society and the implicit “contract” established between them. Determination of the impact of research, for both researchers and managers, could therefore contribute to greater societal involvement. Both groups, and cardiovascular specialists are no exception, have a formidable challenge ahead if they are to achieve all of these objectives.

None declared.