Publish in this journal
Journal Information
Vol. 59. Issue 12.
Pages 1244-1251 (December 2006)
Share
Share
Download PDF
More article options
Vol. 59. Issue 12.
Pages 1244-1251 (December 2006)
DOI: 10.1016/S1885-5857(07)60080-2
Full text access
Development of a Geographic Filter for PubMed to Identify Studies Performed in Spain
Construcción de un filtro geográfico para la identificación en PubMed de estudios realizados en España
Visits
...
José M Valderasa, Joan Mendivilb, Antoni Paradac, Marina Losada-Yáñezd, Jordi Alonsoe
a Unidad de Investigación en Servicios Sanitarios, Institut Municipal d'Investigació Mèdica (IMIM), Barcelona, Spain. Department of Health Policy and Management Department, Johns Hopkins School of Public Health, Baltimore, Maryland, USA
b Unidad de Investigación en Servicios Sanitarios, Institut Municipal d'Investigació Mèdica (IMIM), Barcelona, Spain. Agència de Salut Pública de Barcelona (ASPB), Barcelona, Spain
c Agència d'Avaluació de Tecnologia i Recerca Mèdiques (AATRM), Barcelona, Spain
d Biblioteca Universidad Pompeu Fabra (UPF), Barcelona, Spain
e Unidad de Investigación en Servicios Sanitarios, Institut Municipal d'Investigació Mèdica (IMIM), Barcelona, Spain. Universitat Autònoma of Barcelona (UAB), Barcelona, Spain
This item has received
...
Visits
(Daily data update)
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Tables (4)
Figure 1. Performance evaluation indices for filters and search strategies.
TABLE 1. Selected Characteristics of the Studies Identified (n=84)
TABLE 2. Diagnostic Performance of the Electronic Strategies by Comparison With the Hand Search (Gold Standard)
ANNEXE 1. Use of the Geographical Filter
Show moreShow less
Introduction and objectives. The increasing size of the PubMed (Medline) database has made it necessary to develop specific tools for information retrieval (i.e., filters). The current configuration of the Limits option in PubMed does not enable searches to be limited according to geographic criteria. Our objectives, therefore, were a) to develop a geographic filter for the retrieval from PubMed of studies performed in the Spanish population, and b) to assess its performance. Methods. A second-generation filter was constructed by reviewing previous strategies. It involved the selection of three types of record that included: a) the term Spain in different languages; b) Spanish place names (i.e., mainly those of autonomous regions and provinces), and c) acronyms for health services in autonomous regions. The performance of the filter (i.e., its sensitivity and specificity) was assessed in a representative sample (i.e., clinical trials indexed in PubMed under the MeSH term Myocardial Infarction) by comparison with the results of a manual search (the gold standard). In addition, the filter's performance was also evaluated by comparing it with a simple search for the term Spain[ad]. Results. Use of the filter retrieved 74 (98.7%) of the relevant references. The sensitivity (88.1%) was higher than that of the simple search strategy (45.8%), and excellent specificity (100%) was achieved in both cases. Conclusions. A geographic filter has been developed for identifying studies in PubMed that involve the Spanish population.
Keywords:
Information storage standards
Information retrieval standards
Bibliometrics
Databases
Bibliographic standards
Organizational affiliation
Publication standards
Spain
Evidence-based medicine
Information systems
Introducción y objetivos. El volumen de PubMed (Medline) hace necesarias herramientas específicas para la recuperación de información (filtros). La actual configuración de la opción «Limits» en PubMed no permite restringir las búsquedas según criterios geográficos. Por ello, se plantearon como objetivos: a) la construcción de un filtro geográfico para la recuperación en PubMed de estudios realizados en la población española, y b) la evaluación de su rendimiento. Métodos. Se ha construido un filtro de segunda generación, basado en la revisión de estrategias previas, para la selección de 3 bloques: a) el término «España» y sus variantes en diversos idiomas; b) otros topónimos correspondientes principalmente a comunidades autónomas y provincias, y c) acrónimos correspondientes a los servicios autonómicos de salud. Se evaluó el rendimiento del filtro (sensibilidad y especificidad) en una muestra de conveniencia (ensayos clínicos indizados en PubMed bajo el término MeSH «Myocardial Infarction»), para lo cual se tomó como patrón de referencia la revisión manual de las referencias. También se comparó el rendimiento del filtro con el de la búsqueda simple «spain[ad]». Resultados. El filtro ha recuperado 74 (98,7%) de las referencias pertinentes y ha demostrado una sensibilidad (88,1%) muy superior a la de la búsqueda simple (45,8%), alcanzándose una especificidad excelente en uno y otro caso (100%). Conclusiones. Se dispone de un filtro eficiente para la identificación de estudios realizados en población española en PubMed.
Palabras clave:
Almacenamiento de información/estándares
Recuperación de información/estándares
Bibliometría
Bases de datos
Bibliografía/estándares
Filiación institucional
Publicación/estándares
España
Medicina basada en la evidencia
Sistemas de información
Full Text

INTRODUCTION

With nearly 15 million references, Medline, the biomedical database created and maintained by the US National Library of Medicine, is one of the most complete resources in the world.1 To a large extent, its success can be attributed to PubMed, its popular, freely available interface. Its vast size, which doubles every 5 years, makes it necessary to resort to specific information retrieval tools (search strategies). Some strategies, or certain parts of them, can serve general purposes and we call these "filters." Their use enables us to save time in constructing searches and increases their reproducibility, as well as achieving greater effectiveness if we take into account certain limitations in the indexing in PubMed.3

The appearance of the "evidence-based medicine" paradigm4,5 has stimulated the development of methods to identify references in biomedical databases and encouraged the construction of so-called methodological search filters. This term is derived from their purpose which is to identify studies with high internal validity,6 above all randomized clinical trials or systematic reviews, among others.3 Given their usefulness, some have even been incorporated into PubMed thru the "Clinical Queries" option.1

To date, the development of geographical filters, that is, filters oriented towards the identification of information with a specific geographical location, has not been tackled despite the fact that current PubMed options do not facilitate the identification of the geographical origin of populations studied. The principle alternatives available are to enter "Spain" as a restriction criteria either as the affiliation of the first author or as the place of publication, and to activate "Spanish" as the language option. Each of these has been demonstrated to be insufficient for our purpose. Restrictions by language or place of publication ignore numerous Spanish publications in international journals, whereas searches centered on author affiliation are limited as in PubMed this field only offers the affiliation of the first author. Moreover, the efficiency of this strategy is impaired by a lack of standardization. The first author affiliation field contains a wide range of information; frequently, when the country where the journal is published is the same as that of author affiliation, the latter is eliminated.7-10

Consequently, we set ourselves the following objectives: a) to develop a geographical filter to identify in PubMed those studies in which a Spanish population has participated, and b) to evaluate filter performance in terms of accepted methodological standards and by comparison with simple geographical search criteria.

METHOD

Development of the Geographical Filter

To develop the geographical filter we followed what is known as a second-generation procedure, in the terminology proposed by Jenkins.3 Filter construction is based on: a) applying expert knowledge of the database structure (PubMed) and of the thesaurus or controlled vocabulary (MeSH), and b) validating the strategy obtained by comparison with a gold standard.3

To identify relevant terms we reviewed search strategies used by other authors to identify studies conducted in Spain. These were mainly bibliometric studies aimed at evaluating Spanish scientific production. We contacted authors to obtain their search strategies.11-17 Based on components we identified in these strategies, we defined 3 syntactic structures of filters and their corresponding terms: a) the term "Spain" in the major PubMed languages (English, Spanish, German, French, Italian); b) place names of Spain's autonomous regions, provinces, provincial capitals and others locations of special interest, excluding any that are also found in other countries; and c) official acronyms of the Spanish autonomous regions' health services.

We specified the search for terms of the first type as free text or affiliation ([ad]). The PubMed search engine applied to free text an automatic mapping process of terms that localizes equivalences consecutively in tables of MeSH terms, journals and authors, in that order, and in which the field "affiliation" is not included.1 The remaining strategies were applied only to "affiliation." Terms were truncated to permit retrieval of names referring to inhabitants of the country or region. The different strategies were linked using the connector OR (Annex 1).

Performance Evaluation of the Filter

Performance evaluation of the geographical filter was conducted by comparison with a hand search (the gold standard) in an ad hoc validation sample.3

Identification of the Sample and Hand Search

To obtain the sample we selected all PubMed indexed references under MeSH keyword "myocardial infarction." In Spain, myocardial infarction accounted for 39 400 deaths in 2002 alone and its incidence is increasing.18,19 We only selected clinical trials (Limit: "Clinical Trial") and the default options (activate "Explode" and deactivate "Restrict to Major Subject Headings"). The format selected was "Medline," which details between 20 and 30 characteristics for each reference based on information presented in the publication (title, abstract, authors, affiliation, and MeSH terms among others).1 Validation samples were exported and analyzed using Reference Manager 10 software20 to guarantee the procedure was auditable.

Two reviewers (JM, JMV) the classified validation sample references by hand ("relevant" vs "non-relevant") applying the following operational definition: references were considered "relevant" when: a) they specifically indicated the study had been conducted in Spain; b) they mentioned the Spanish population as such, c) the study had been conducted by researchers affiliated to Spanish research centers. References connected with multicenter studies (whether international or not) in which a Spanish population participated were also classified as relevant. References failing to meet any of these criteria were considered "non-relevant." If information about author affiliation, where the intervention took place or the population was insufficient, full text articles were located for clarification. In some cases, reviewers needed to identify studies by the same authors or references outside of the sample to determine relevance.

This classification was compared with that obtained after applying the filter to the sample (using the connector "AND"). We included all references in PubMed from its launch in 1966 thru to August 1, 2005, when the electronic searches were performed.

Simple Strategy

To evaluate the relative usefulness of the filter we compared its performance with that of a much simpler strategy. Intuitively, we supposed this would correspond to the use of the free term "Spain." In fact, this is an inefficient term that could artificially increase the relative value of the geographical filter21 and therefore selected the strategy "Spain[ad]" which, in practice, retrieves three times as many references.1

Analysis

Evaluating PubMed searches is similar in operational terms to studying diagnostic test performance in that we apply the concepts of sensitivity (proportion of relevant articles in the validation sample retrieved by the filter) and specificity (proportion of non-relevant articles in the validation sample not retreived by the filter) (Figure).22,23 We calculated the sensitivity and specificity of applying the geographical filter and the simple strategy on the validation sample.

Figure 1. Performance evaluation indices for filters and search strategies.

We checked the stability of these calculations thru sensitivity analysis of a second, independent, validation sample. We chose a different clinical condition, but one that is also greatly concerns health services, and selected clinical trials indexed in PubMed under MeSH term "Pulmonary Disease, Chronic Obstructive."

RESULTS

Applying the criterion "Spain[ad]" to the entire database retrieved 109 344 references. Applying the filter produced 59 142 additional references giving a total of 168 486 (an increase of 54.1%). Applying the criterion "Spain" and its translations (free text and field [ad]) retrieved 141 961 (84.3%) references; using other place names retrieved an additional 26 499 references (15.7%); and health service acronyms, retrieved another 26 (0.1%).

Under MeSH term "Myocardial Infarction," we found 7312 clinical trial references. Only 2 of these corresponded to the same document, indexed once as a clinical trial and later as a review. The hand review of these references classified 84 (1.15%) as relevant and 7228 as non-relevant (98.85%). Just over half of these had been published in Spanish and in Spanish journals. Seven relevant references corresponded to international trials that included Spanish populations (Table 1)

The geographical filter recovered 75 references, 74 (98.76%) of which were identified as relevant in the hand search. The only reference that did not correspond to a trial conducted in Spain was a PubMed indexing error. In the abstract, the affiliation was given as "Niguarda Ca'Granda" a hospital in Milan, Italy. This had been incorrectly indexed as "Niguarda Ca'Granada, thus locating it in south-eastern Spain even though it was transcribed correctly in the original.24 The criterion "Spain" and its translations (free text and field [ad]) retrieved 53 relevant references (70.67%); that corresponding to autonomous regions, provinces, and cities retrieved 64 (86.47%); and that corresponding to health services, none. of these references, 31 (41.9%) were retrieved by only 1 of the 3 searches, most of them by using place names (n=21, 67.74%), or "Spain" (n=10, 32.26%). The remaining 43 (58.2%) were retrieved by more than 1 of the search criteria. The geographical filter failed to retrieve 10 relevant references. In 2 cases, this was due to the incomplete definition of filter terms (place names not included in the strategy): in 1 case, "Catalonia" was mentioned in the abstract as the place where the trial had taken place25; in another, author affiliation appeared as a hospital located in Cartagena.26 The other 8 lacked sufficient information: most (n=6) had been published between 1970 and 1990 and only reference information was available (title, authors, publication). All these had been classified as relevant in the hand review only after a secondary search for data on the trial and/or author affiliation thru references outside the validation sample.

Altogether, the filter showed sensitivity as 88.10% and specificity, 99.99% (Table 2). In contrast, the simple strategy "Spain[ad]" recovered only 38 relevant references; sensitivity was 45.8% and specificity 100% (Table 2). The effectiveness of estimating performance of the geographical filter in the second validation sample (478 references) was optimal (sensitivity and specificity 100%).

On completing the study, we reported all indexing errors identified to the US National Library of Medicine.

DISCUSSION

We have constructed an easy-to-use geographical filter to identify studies conducted in Spain on the Medline database (PubMed). Performance evaluation has shown excellent specificity and very high sensitivity, above that of the simple search criterion.

Strengths and Weaknesses of the Evaluation Procedure

Inconsistencies in indexing and the presence of transcription errors affect retrieval of references on Medline.27-31 Our filter validation procedure has been designed with precision bearing in mind the limitations of indexing and overcoming those inherent in methods employed to date. It is based on conducting a hand search of a single ad hoc validation sample, usually defined from references published in selected journals.3 This practice is controversial because the approach to selection introduces a degree of arbitrariness. Moreover, it uses references outside the database so indexing errors affect the estimation of filter performance.32 To take an extreme case, a reference not included in the database cannot be recovered by the filter under evaluation, or by any other, which makes it unreasonable to attribute this limitation to the search strategy. The evaluation procedure took the validation sample directly from PubMed which, moreover, meant the hand search was easily replicable, and guarantees validation samples can be accessed in Reference Manager file format. The construction of an unrelated supplementary sample based on a different clinical entity to determine the stability of the evaluation of filter performance also increases confidence in results.

We would point out that no standards for selection of comparison filters or of samples to evaluate or validate filters currently exist. The simple strategy was selected because it entailed an intuitive approach that trebles the performance of an even simpler strategy ("Spain" as free text), and was the strategy most often described in the literature. The interpretability of results has guided our selection of the validation samples.

Strengths and Weaknesses of the Proposed Filter

Terms have been included on the basis of a systematic procedure derived from the definition of explicit criteria: geographical, with geographical localizations of decreasing size; linguistic, with languages that cover >99.99% of Spanish scientific production33; and administrative, with the inclusion of the various health services. This same characteristic meant that 2 relevant references could not be recovered electronically because the criterion of localization did not coincide with either of the predefined criteria. Both terms have subsequently been incorporated into the filter (Annex 1), which will certainly continue to be enriched with the personal contributions and suggestions of users.

The geographical filter proposed includes a search for names of regional health services that, on analysis, neither increased nor reduced sensitivity or specificity in the samples used. Bearing in mind their lack of impact, that there are very few of them, that when used separately this criterion identified 1519 references at August 2005, and the authors' prior experience with these terms, we believe they should continue to be included. In any case, the modular construction of the filter means users can easily eliminate the criterion if they wish to.

Increasingly, databases tend to normalize data and have improved their indexing over time. This may lead to the improved performance of some simpler searches which, in any case, have already been inserted into the current filter.

Alternative search strategies based on criteria of language or place of publication, while useful in bibliometric studies, are barely applicable to achieve our objective of identifying studies conducted in the Spanish population. Limiting searches to entries in Spanish would ignore the vast volume of Spanish production written in English (66 855 documents alone in 1994-200233). On the other hand, more than half the references contained in PubMed in Spanish correspond to publication outside of Spain which, on the whole, present studies conducted on the American continent.

More and more, researchers publish their results in international journals (>45% in the present study).27 At the same time, some journals published in Spain have a markedly international character: Methods and Findings in Experimental, and Clinical Pharmacology, or Test, for example. Others, which were originally "local" publicatio ns, increasingly aspire to achieve recognition as international journals. This is the case of Revista Española de Cardiología, which has published more than a third of all studies. It seems clear that language and place of publication are unreliable criteria to identify studies conducted in Spain.

Applications of the Geographical Filter

Without doubt, one of the most useful applications of this geographical filter is in to enable us to identify those studies with greater external validity for the application of the data to the Spanish population, what we would call generalizability or proximal similarity.6,34-36 It is well known that the generalizability of findings depends on various aspects of the studies, ranging from the clinical, socio-demographic, and even genetic characteristics of the population studied to variables related to the process of attention, such as styles of clinical practice and care, and complementary treatments administered.6 If the previously indicated aspects could be of special relevance in studies related to cardiovascular risk (consider, for example, the controversial application of the different tables, and equations obtained in different populations), other aspects, such as the legislative or those related to health service organization, could be even more relevant in studies into giving up smoking37 (legislation and model of social behavior), or evaluating the effectiveness of interventions in the home (accessibility and model of attention).38 In both cases, geographical location can be considered an indirect (proxy) variable of many of these factors and may be of great use in evaluating external validity.

Specifically, the availability of a geographical filter of high sensitivity and specificity to identify in PubMed studies conducted in the Spanish population opens the door to its systematic application in clinical practice. Its application will permit quick, reliable identification of relevant information about the effects of procedures and treatments in conditions closer to our reality. Additionally, this information is especially relevant in defining clinical trajectories that justifiably arouse growing interest and in which considering local peculiarities of attention are decisive for effective implementation.39-42 From a cognitive point of view, retrieving relevant information facilitates learning, as well as improving clinical practice after applying the new information.43,44

Perhaps because of this, we are still surprised that clinical practice guidelines lack specific sections summarizing information available about using diagnostic and therapeutic procedures in the very contexts where they are intended to be applied45,46; information which geographical filters like that presented here could play an important role in providing.38 This contextualization of information, rather than contradicting the postulates of evidence-based medicine, confirms them, since considering the external validity of information identified is no less than one step prior to its application in clinical practice.4

Finally, this geographical filter offers a useful tool for bibliometric applications which have, to date, primarily centered on the Institute of Scientific Information of Philadelphia (ISI) database and that could now be complemented via PubMed.15,33 In fact, for this purpose one recent publication47 describes the independent use of a geographical filter similar to that presented in our article.

CONCLUSIONS

An efficient filter to identify studies conducted in the Spanish population in PubMed is available. Its application permits us to find those references with greater external validity for clinical practice in Spain and, if used systematically, it could be of great value in constructing clinical practice guidelines and defining clinical trajectories.

ACKNOWLEDGEMENTS

The authors wish to thank the Centro Cochrane Iberoamericano for advice received on filter structure, Dr Alberto Miguel-Dasit and Dr Luis Martí-Bonmatí for generously providing us with the search strategy they designed, and Olatz Garín for her invaluable comments.

See editorial on pages 1221-4

An earlier version of this study was presented at the 23rd Meeting of the Spanish Society of Epidemiology.

This study was part-financed by the Instituto de Salud Carlos III (Red de Investigación Cooperativa en Investigación de Resultados y Servicios Sanitarios [IRYSS] G03/202, Exp No. 3000118).


Correspondence: Dr. J.M. Valderas Martínez.
Unidad de Investigación en Servicios Sanitarios. Institut Municipal d'Investigació Mèdica (IMIM).
Dr. Aiguader, 88, edifici PRBB. 08003 Barcelona. España.
E-mail: jmvalderas@imim.es

Received August 29, 2005.
Accepted for publication August 2, 2006.

Bibliography
[1]
Medline [Internet]. Bethesda: National Library of Medicine (US); 2005.
[2]
Drazen JM, Curfman GD..
Public access to biomedical research..
N Engl J Med, 351 (2004), pp. 1343
[3]
Jenkins M..
Evaluation of methodological search filters-a review..
Health Info Libr J, 21 (2004), pp. 148-63
[4]
Users guides: essentials of evidence-based clinical practice. Chicago: American Medical Association; 2002.
[5]
Evidence-Based Medicine Working Group..
Evidence-based medicine. A new approach to teaching the practice of medicine..
JAMA, 268 (1992), pp. 2420-5
[6]
Juni P, Altman DG, Egger M..
Systematic reviews in health care: assessing the quality of controlled clinical trials..
BMJ, 323 (2001), pp. 42-6
[7]
Loria A, Arroyo P..
Language and country preponderance trends in MEDLINE and its causes..
J Med Libr Assoc, 93 (2005), pp. 381-5
[8]
Rosselli D..
Geography of biomedical publications..
[9]
Takei N..
Geography of biomedical publications..
[10]
Tutarel O..
Geographical distribution of publications in the field of medical education..
BMC Med Educ, 2 (2002), pp. 3
[11]
Baños JE, Bosch F, Bigorra J, Guardiola E..
Difusión internacional de los ensayos clínicos realizados en España: un análisis a través de su publicación en revistas científicas..
Med Clin (Barc), 102 (1994), pp. 441-5
[12]
Barenys M, Recasens MA, Camps J, Salas-Salvado J..
La producción científica española sobre alimentación y nutrición de 1987 a 1990..
Med Clin (Barc), 99 (1992), pp. 9-12
[13]
Guardiola E..
Algunas consideraciones sobre la búsqueda de estudios españoles en bases de datos internacionales..
Med Clin (Barc), 99 (1992), pp. 795-6
[14]
Identificaci??n y descripci??n de ensayos cl??nicos publicados en revistas espa??olas de medicina general y de medicina interna durante el per??odo 1971-1995. Med Clin (Barc). 1999;112 Supl 1:28-34.
[15]
Pestaña A..
El MedLine como fuente de información bibliométrica de la producción española en biomedicina y ciencias médicas. Comparación con el Science Citation Index..
Med Clin (Barc), 109 (1997), pp. 506-11
[16]
Ramos Rincon JM, Belinchon RI, Gutiérrez RF..
La producción científica española respecto a la infección por el virus de la inmunodeficiencia humana/sida. Un estudio a través de MedLine (1991-1999)..
Med Clin (Barc), 117 (2001), pp. 645-53
[17]
Miguel-Dasit A, Marti-Bonmati L, Aleixandre R, Sanfeliu P, Valderrama JC..
Producción española sobre diagnóstico por la imagen en cardiología y radiología (1994-1998)..
Rev Esp Cardiol, 57 (2004), pp. 806-14
[18]
Ministerio de Sanidad y Consumo. Mortalidad por causa de muerte 1991-2002. Espa??a y comunidades aut??nomas. Documento en l??nea [accedido 10 Ago 2005]. Disponible en: http://www.msc.es/Diseno/sns/ sns_sistemas_informacion.htm
[19]
Sans S, Puigdefabregas A, Paluzie G, Monterde D, Balaguer-Vintro I..
Increasing trends of acute myocardial infarction in Spain: the MONICA-Catalonia Study..
Eur Heart J, 26 (2005), pp. 505-15
[20]
Stamford: Thompson ResearchSoft; 2002.
[21]
Castellote Varona FJ..
Idioma y Medline..
An Med Interna, 19 (2002), pp. 152-3
[22]
Allison JJ, Kiefe CI, Weissman NW, Carter J, Centor RM..
The art and science of searching MEDLINE to answer clinical questions. Finding the right number of articles..
Int J Technol Assess Health Care, 15 (1999), pp. 281-96
[23]
Kagolovsky Y, Moehr JR..
Current status of the evaluation of information retrieval..
J Med Syst, 27 (2003), pp. 409-24
[24]
Merlini PA, Bauer KA, Oltrona L, Ardissino D, Spinola A, Cattaneo M, et al..
Thrombin generation and activity during thrombolysis and concomitant heparin therapy in patients with acute myocardial infarction..
J Am Coll Cardiol, 25 (1995), pp. 203-9
[25]
Grupo de Investigacion del estudio PREMISE..
Prevención secundaria del. infarto de miocardio y calidad de vida relacionada con la salud..
Med Clin (Barc), 119 (2002), pp. 9-12
[26]
Melgarejo-Moreno A, Galcera-Tomas J, García-Alberola A..
Prognostic significance of bundle-branch block in acute myocardial infarction: the importance of location and time of appearance..
Clin Cardiol, 24 (2001), pp. 371-6
[27]
Jiménez-Contreras E, Delgado López-Cozar E, Ruiz-Pérez R, Fernández VM..
Impact-factor rewards affect Spanish research..
Nature, 417 (2002), pp. 898
[28]
Bador P, Lafouge T..
Rédaction des adresses sur les publications. Un manque de rigueur défavorable aux universités françaises dans les classements internationaux..
Presse Med, 34 (2005), pp. 633-6
[29]
Fernandez E, García AM..
Accuracy of referencing of Spanish names in Medline..
[30]
Ruiz-Pérez R, Delgado López-Cózar E, Jiménez-Contreras E..
Spanish personal name variations in national and international biomedical databases: implications for information retrieval and bibliometric studies..
J Med Libr Assoc, 90 (2002), pp. 411-30
[31]
Sezer O, Bohme A..
Hazards of translation of non-English medical publications in Medline..
[32]
Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, et al..
An alternative to the hand searching gold standard: Validating methodological search filters using relative recall..
BMC Med Res Method, 6 (2006), pp. 33
[33]
Cami J, Suñén-Pinol E, Méndez-Vásquez R..
Mapa bibliométrico de España 1994-2002: biomedicina y ciencias de la salud..
Med Clin (Barc), 124 (2005), pp. 93-101
[34]
Relabeling internal and external validity for applied social scientists. En: Trochim W, editor. Advances in quasi-experimental design and analysis. San Francisco: Jossey-Bass; 1986. p. 67-77.
[35]
Campbell DT..
Relabeling internal and external validity for applied social scientists..
New Directions for Program Evaluation, 31 (1986), pp. 67-77
[36]
Modern epidemiology. 2nd ed. Philadelphia: Lippincot Williams & Wilkins; 1998.
[37]
Tappin DM, Lumsden MA, Gilmour WH, Crawford F, McIntyre D, Stone DH, et al..
Randomised controlled trial of home based motivational interviewing by midwives to help pregnant smokers quit or cut down..
[38]
Morcillo C, Valderas JM, Aguado O, Delas J, Sort D, Pujadas R, et al..
Evaluación de una intervención domicliaria en pacientes con insuficiencia cardíaca. Resultados de un estudio aleatorizado..
Rev Esp Cardiol, 58 (2005), pp. 618-25
[39]
Haynes RB, McKibbon KA, Wilczynski NL, Walter SD, Werre SR..
Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey..
[40]
Every NR, Hochman J, Becker R, Kopecky S, Cannon CP..
Critical pathways. A review..
Circulation, 101 (2000), pp. 461-5
[41]
Cardozo L, Aherns S..
Assessing the efficacy of a clinical pathway in the management of older patients hospitalized with congestive heart failure..
J Health Qual, 21 (1999), pp. 12-7
[42]
Cluzeau FA, Burgers JS, Brouwers M, Grol R, Mäkelä M, Littlejohns P, et al..
The AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project..
Quality Safety Health Care, 12 (2003), pp. 18-23
[43]
Grad R, Pluye P, Segal B..
Assessing the impact of clinical information-retrieval technology in a family practice residency..
J Eval Clin Pract, 11 (2005), pp. 576-86
[44]
Pluye P, Grad RM, Dunikowski LG, Stephenson R..
Impact of clinical information-retrieval technology on physicians: a literature review of quantitative, qualitative and mixed methods studies..
Int J Med Inform, 74 (2005), pp. 745-68
[45]
Schattner A, Fletcher RH..
Research evidence and the individual patient..
QJM, 96 (2003), pp. 1-5
[46]
Rothwell P..
External validity of randomised controlled trials:to whom do the results of this trial apply? Lancet, 365 (2005), pp. 82-93
[47]
Escudero C, Milán I, Posada de la Paz M..
Análisis de la producción científica española sobre enfermedades raras: 1990-2000..
Med Clin (Barc), 125 (2005), pp. 329-3
Idiomas
Revista Española de Cardiología (English Edition)

Subscribe to our newsletter

Article options
Tools
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

es en
Política de cookies Cookies policy
Utilizamos cookies propias y de terceros para mejorar nuestros servicios y mostrarle publicidad relacionada con sus preferencias mediante el análisis de sus hábitos de navegación. Si continua navegando, consideramos que acepta su uso. Puede cambiar la configuración u obtener más información aquí. To improve our services and products, we use "cookies" (own or third parties authorized) to show advertising related to client preferences through the analyses of navigation customer behavior. Continuing navigation will be considered as acceptance of this use. You can change the settings or obtain more information by clicking here.