In the information current era of media information, the increase in the number of biomedical publications has gone from being excellent news to become a terrible nightmare. Each year, the MEDLINE database alone includes 560 000 new articles and the central register of the Cochrane Collaboration adds 20 000 trials annually. According to the estimate of Glasziou and Haynes,1 to remain "updated," we should read 1500 original articles and 55 randomized clinical trials daily.
This is probably one of the many reasons that explain the existence of a gap between research and clinical practice; but, basically, the problem is that to search, evaluate and apply knowledge (evidence) in clinical practice is a task that requires the development of skills, to a certain extent novel, to deal with new problems, and that necessitates adequate tools to facilitate this task. As early as 2001, the Institute of Medicine of the United States National Academy of Sciences2 recommended the establishment and maintenance of a global program with the aim of making scientific evidence more useful and accessible for clinicians and patients.
For this purpose, and from this clinical perspective, in which we implicate ourselves from this moment on, we welcome with interest the proposal of Valderas et al,3 which appears in this issue of Revista Española de Cardiología, concerning the construction and validation of a geographic filter for the search for and selection of studies carried out in Spain in PubMed (MEDLINE). This tool, in the opinion of its authors, "enables the retrieval of those references with the greatest external validity for clinical practice in Spain and, employed in a systematic manner, may be extremely useful in the development of clinical practice guidelines and in the definition of clinical pathways."
The proposal includes innovative elements in that it approaches the problem of filtering and the searches with logic that differs from the classic proposals; likewise, the study tests certain unusual validation processes and, finally, it indicates a possible utility of the tool in clinical practice which warrants reflection. Here, we will address these three aspects successively.
Search and Filter Logic
Classically, evidence-based medicine (EBM) has developed tools and strategies to make the application of knowledge to patient care more feasible. This is an important matter that has invariably been on the EBM agenda, and there are a number of advances that should be pointed out.
With the original studies, the most relevant advance (aside from the decision to allow open and cost-free access to MEDLINE) has been the development of methodological filters for information searches in databases. Filters of this type have been implemented for MEDLINE (PubMed), as well as for other databases such as EMBASE, CINAHL, PsyclINFO, etc.
The implicit reflection implicit of methodological filters is that, once a clinical question is formulated in the correct terms (classically in PICO format: Population, Intervention, Comparison, and Outcomes), there is a correspondence between the clinical questions and the optimal study designs or architectures for responding to that type of question (hence the indifferent use in PubMed of methodological filters or clinical searches, which, in this interface, are referred to as clinical queries). Thus, for example, for questions concerning treatment, we will preferentially look for randomized trials; for questions on prognosis, cohort studies, or studies on clinical prediction rules; for diagnostic questions, studies on diagnostic precision, etc.
To mitigate the excess of original studies that we mentioned above, synthesis research was developed. Characteristically, syntheses involve a number of original articles (systematic reviews), in which case, they offer a perspective view but, above all, they simultaneously improve the validity and power of the studies. Thus, these systematic reviews are carried out for randomized trials, observational (cohort) studies, or studies of diagnostic tests, etc. These reviews are found in specific databases, like those contained in the Cochrane Library, and can also be accessed in the clinical queries section in PubMed.
Nevertheless, in the maelstrom of clinical activity, it is often necessary to have easier and more rapid access to evidence. Actually, many clinicians consider that they can function4 (and, in fact, do function) with summaries written by others. This has led to the development of synopses which, in essence, are documents prepared according to explicit methods to summarize studies or systematic reviews. They provide a brief and efficient source of information for clinical use (examples of these synopses would be the Evidence Based collection, which includes Evidence-Based Cardiovascular Medicine, ACP Journal Club and DARE).
Finally, totally or partially automated evidence-based decision-making support systems are being developed. They provide an explicit definition of evidence evaluation processes and have the capacity (or will have in the future) of integrating diverse types of information concerning specific patients or patient registries. The most recent versions of the electronic book UptoDate (http://www.uptodate.com) or of Clinical Evidence (http://www.evidence.org) are steps in this direction.
This gradual description corresponds to the pyramid proposed by Haynes,5 referred to as the "4S" (Systems, Synopses, Syntheses, Studies) strategy. Haynes also suggests that that our traditional clinical search procedure (which focuses on original articles) be revised, to assume a more efficient one based on the orderly use of the 4S system.
It should be pointed out that, while the objective of clinical practice guidelines is to compile or synthesize information pertinent to quality health care, in the 4S strategy, they are not, as such, considered to be syntheses of scientific knowledge. They borrow their capacity for synthesis from systematic reviews carried out by other authors or, on rare occasions, by the editors of the guidelines themselves. However, they are invaluable compendiums of recommendations for the treatment of certain diseases. Their importance from the point of view of health care quality and control of variability in clinical practice is obvious. Revista Española de Cardiología has also compiled and offers clinical practice guidelines of the Spanish Society of Cardiology and other societies.6
The philosophy implicit in methodological filters involves the selection of evidence on the basis of major criteria such as validity, both internal (or the extent to which the design, conduction, and analysis make it possible to obtain unbiased results) and external (understood as the coherence of the results with those of other studies and other available knowledge). A similar philosophy, based on validity, pervades the entire process of the 4S pyramid. In summary, the maxim of this approach (which we have referred to as classic) would be "select, synthesize and summarize that which is valid" (regardless of where it comes from), and then apply it, adapting it to the greatest possible extent to the individual conditions of your patient or the conditions of your patient population (in the case of groups of patients). It would be an adapted version of "think globally, act locally."
The logic behind a geographic filter is different; in essence, the proposal is to search for the results or the evidence in our population or in a reasonably similar population, thus facilitating its application. That is, we put the accent on the external validity, understood, not as it was defined above, but as the capacity to extrapolate the outcome to our population or to our patient. There is, of course, no formal contradiction in the proposals; ideally, the reflection on external validity does not substitute for the assessment of internal validity; however, two problems arise.
On the one hand, in the case of conflicts in validity, how should the evidence to be applied be selected? For example, if the study is highly valid "externally," but questionable "internally," do we prefer a study that is less valid "externally," but more valid "internally"? In such cases, how will we resolve these dilemmas? And how will we incorporate this new search approach into the 4S structure of clinical systematics?
On the other hand, as a research program, that is, as a systematic proposal for knowledge construction for the clinical setting, to propose to have access to valid research (in terms of proximity or similarity) for every location and condition is simply unviable. In contrast, the viability of EBM as a program for generating and utilizing the evidence in patient care depends critically on the assumption that valid knowledge is or will be globalized, although it may be necessary to adapt it in a sensible and ethical manner.
Validation of the Filter
With respect to the process of assessing the filters, the authors point out the analogy between this procedure and studies to assess the precision of diagnostic tests,7 an affirmation with which we are totally in agreement. Continuing along this line, a certain analogy between these studies is maintained for the critical reading of diagnostic studies and those involved in the evaluation of the filters.8,9 Let us consider some of the key elements of this reading.
Classical studies for the validation of methodological filters involve the use of a block of journals selected from a group of clinical journals over a determined time interval. From this point of view, there is a clear definition of the range of situations in which the filter is to be tested and, thus, of the situations in which it could be applied (it would be equivalent to the range of patients in the studies of diagnostic tests).
In the study we are discussing here, the validation sample was constituted with a search for the term "Myocardial Infarction" (MeSH), with certain restrictions and activation of the "explode" option. It is difficult to know in what circumstances and for what other terms the filter could be applied, and it would be interesting to know whether, when the study is repeated with other MeSH terms, the sensitivity remains the same.
Moreover, what the authors considered as the gold standard in the study they present (that is, a manual search performed by two observers), while perfectly defined, could be improved in terms of calibration.
Undoubtedly, these methodological questions (especially the sample selection), although original, do not translate into an obvious advance with respect to the classical designs for filter evaluation. We consider there to be shadows in the design that should be discussed, although this is, of course, a subject to be explored in future studies.
Concerning the Utility in Clinical Practice
The process of knowledge construction and the process of knowledge application in the clinical setting are essentially different. The former is a scientific procedure the purpose of which is to construct knowledge on the basis of groups of patients by means of a defined epistemology; thus, it can lead to value judgments regarding the correction or validity of this knowledge (this has to do with critical reading skills). The latter is a "prudent" procedure in which it is necessary to decide, on the basis of common sense, whether or not the knowledge generated in groups of individuals can be applied in a specific individual. Moreover, these results must be combined with other pertinent information (in order to adapt the evidence) and with the values of the patient and of society.
It is probably easier to agree on the judgments concerning the validity of a study than on its applicability, which is always a "risky judgment" because the situations are often complex. Thus, the affirmation on the behalf of the authors that the filter "opens the door to its systematic application in clinical practice" may be somewhat optimistic.
To mention a few examples, some of them pointed out by the authors, if we propose to apply the equation for cardiovascular risk, adapted for groups of patients in Gerona, in northeastern Spain, to a patient, let's say, from Huelva, in southweastern Spain, actually, this extrapolation is based on the assumed similarity between the two populations, but based on common sense, not on proof.
Another hypothetical situation: what can we do with a large scale multicenter study in which there is only a very limited participation of the Spanish population or of some Spanish group? Is this study more valid "externally" than if the Spanish groups had not participated? Is it more applicable in Spain? Obviously, the response is "it depends" because we would need to have information on other elements, and then judge prudently.
To continuing the review of situtations reviewing situations, let's imagine a systematic review in which one or two studies have been performed in a Spanish population. This raises certain questions. The first is: will our filter find this review? If it does, how will we utilize the results of the study? Should we analyze a subgroup for local studies? If so, what estimator should we attempt to apply: the global estimator of the systematic review or the partial estimator of the subgroup of Spanish studies? The latter questions constitute a particular case of the known dispute between "lumpers" and "splitters," which, in our opinion, would be resolved by applying the general estimator rather than that of the Spanish population, a decision based on the so-called Stein's paradox.10
Undoubtedly, a number of questions could be added to those mentioned above with regard to the use of the filter in the clinical setting, but we would prefer, after reiterating our welcome of the tool, to conclude: the filter will probably be useful for the estimation of scientific production or other bibliometric purposes, from whence, in a way, it comes and where, as reported in the article, it was tested successfully. In some of the examples indicated by the authors, such as the development of guidelines or the allocation of resources, etc., it is clear that the filter truly promises to be useful and should be explored. With respect to clinical practice, there is no doubt that it indicates an area for reflection, but it is uncertain whether it will offer any practical advantages.
Unquestionably, rather than a replacement for existing technology, the geographic filter is complementary to it and, in any case, its use in the clinical setting appears to require further conceptual and pragmatic development.
Performed with the support of the European Union project "European Union Evidence-Based Medicine Unity" (www.ebm-unity.org), Leonardo da Vinci (Community Vocational Training Action Programme).
Correspondence: Dr. J.B. Cabello López
Joaquín Orozco, 6, 1.o F.
03006 Alicante. España.
E-mail: jbcabello@redcaspe.org