Publish in this journal
Journal Information
Vol. 64. Issue 10.
Pages 869-875 (October 2011)
Share
Share
Download PDF
More article options
Vol. 64. Issue 10.
Pages 869-875 (October 2011)
DOI: 10.1016/j.rec.2011.04.011
Full text access
The Quality of Information Available on the Internet About Aortic Aneurysm and Its Endovascular Treatment
Calidad de la información disponible en internet sobre el aneurisma de aorta y su tratamiento endovascular
Visits
...
Enrique M. San Norbertoa,
Corresponding author
esannorberto@hotmail.com

Corresponding author: Servicio de Angiología y Cirugía Vascular, Hospital Clínico Universitario de Valladolid, Ramón y Cajal 3, 47005 Valladolid, Spain. esannorberto@hotmail.com
, James Taylora, Roberto Salvadora, Álvaro Revillaa, Borja Merinoa, Carlos Vaqueroa
a Servicio de Angiología y Cirugía Vascular, Hospital Clínico Universitario de Valladolid, Valladolid, Spain
Related content
Rev Esp Cardiol. 2011;64:849-5010.1016/j.rec.2011.06.014
Francisco Lupiáñez-Villanueva
This item has received
...
Visits
(Daily data update)
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (1)
Introduction and objectives

To evaluate the readability, accessibility, usability, and reliability of information available on the Internet in the Spanish language about aortic aneurysm and its endovascular treatment, and to determine whether this information source provides comprehensible material that will enable patients to participate in decisions regarding their condition.

Methods

In November 2010, information from the Internet was compiled by entering the terms “aneurisma aorta” (aortic aneurysm) and “endoprótesis aorta” (aortic endoprosthesis) in the most widely used search engines: Google, Yahoo, and MSN/Bing. The first 30 pages provided by each search engine were analyzed. The Inflesz software was used to calculate the readability of the information retrieved and the LIDA instrument, a validated tool to evaluate the quality of health-related Web sites, was used to assess accessibility, usability, and reliability.

Results

The results for Web pages containing the terms aneurisma aorta and endoprótesis aorta indicated that the readability of the material retrieved was “somewhat difficult” based on the Flesch index within Microsoft Word (48.3±11.42 and 50.11±9.33, respectively; P=.87), Flesch-Szigriszt index (52.69±8.86, 49.31±7.24; P=.87), Fernández-Huerta index (58.05±8.5, 54.44±7.19; P=.82), and Gunning-Fog index (22.03±2.05, 23.86±1.59; P=.83), as well as the Inflesz grading scale (2.39±0.7, 2.08±0.64; P=.28). The LIDA values for accessibility (82.28±14.14, 77.77±12.64; P=.98), usability (72.28±16.67, 72.28±26.61; P=.08), and reliability (46.17±28.69, 56.38±16.17; P=.06) and the total score (70.22±16.85, 72.15±14.93; P=.52), yielded an evaluation of “moderate”.

Conclusions

The Internet information on aortic aneurysms and its endovascular treatment with aortic endoprostheses was deficient with regard to accessibility, usability, and reliability, and had the added difficultly of complicated readability. Our results suggest that readability indexes should be incorporated in the creation and improvement of Web sites providing medical information related to cardiovascular disease.

Keywords:
Aorta
Aneurysm
Endoprosthesis
Readability
Accessibility
Usability
Reliability
Introducción y objetivos

Evaluar legibilidad, accesibilidad, utilidad y fiabilidad de la información existente en internet sobre el aneurisma de aorta y su tratamiento endovascular, para comprobar su función como vehículo de información comprensible que capacite al paciente para participar en decisiones respecto a su enfermedad.

Métodos

En noviembre de 2010 se recopiló información en internet sobre los términos «aneurisma aorta» y «endoprótesis de aorta» empleando los buscadores más extendidos: Google, Yahoo y MSN/Bing. Se analizaron las primeras 30 páginas ofrecidas por cada uno. Para calcular la legibilidad de los textos, se empleó el programa informático Inflesz y el instrumento LIDA como método validado de evaluación de páginas web relacionadas con la salud sobre accesibilidad, utilidad y fiabilidad.

Resultados

Los índices de correlación Word (48,3±11,42; 50,11±9,33; p=0,87), Flesch-Szigriszt (52,69±8,86; 49,31±7,24; p=0,87), Fernández-Huerta (58,05±8,5; 54,44±7,19; p=0,82) y Gunning-Fog (22,03±2,05; 23,86±1,59; p=0,83) y el grado de la escala Inflesz (2,39±0,7; 2,08±0,64; p=0,28) de las páginas web sobre «aneurisma de aorta» y «endoprótesis de aorta», respectivamente, reflejaron una legibilidad «algo difícil». Los valores de accesibilidad (82,28±14,14; 77,77±12,64; p=0,98), utilidad (72,28±16,67; 72,28±26,61; p=0,08), fiabilidad (46,17±28,69; 56,38±16,17; p=0,06) y valoración total LIDA (70,22±16,85; 72,15±14,93; p=0,52) resultaron «moderados».

Conclusiones

La información disponible en internet sobre el aneurisma de aorta y su terapéutica mediante implantación de endoprótesis presenta deficiencias en cuanto a accesibilidad, utilidad y fiabilidad, con la dificultad añadida de una complicada legibilidad. Creemos que los índices de legibilidad deber ser utilizados en la elaboración o la mejora de las páginas web sobre conocimientos médicos cardiovasculares.

Palabras clave:
Aorta
Aneurisma
Endoprótesis
Legibilidad
Accesibilidad
Utilidad
Fiabilidad
Full Text
INTRODUCTION

The Internet provides multiple information sources for patients, and knowledge obtained in this way can help patients understand their disease, but it is often incomplete, outdated, and even erroneous.1 Nonetheless, searching online is currently the most popular method for acquiring information related to health. It is estimated that more than 50% of patients consult the Internet to obtain medical information that can influence the decisions made regarding their health; moreover, the use of this resource has increased in recent years.2 In the specific case of data related to cardiovascular conditions, however, the information is often of poor quality and difficult to access, particularly data related to ischemia of the extremities.3

Internet users commonly apply one of two methods to search for information: they either follow the links to a known Web site or they use one of the available search engines. The 3 most commonly used search engines (Google, Yahoo, MSN/Bing) account for 50% of information-seeking activity. Although search engines offer a multitude of sources, the typical user only visits those appearing highest on the list retrieved.4 However, a position among the first Web pages appearing on a search engine does not guarantee that the information provided is relevant or accurate.

Patient capacity to comprehend the written information received can be significantly improved by adjusting the readability of the text to the patient's level.5 Readability, the relative ease with which the words of a text can be interpreted, is a decisive factor in rendering the information offered to readers comprehensible. The growing interest in promoting readability is linked to development of the idea that patients should have moral autonomy in making decisions regarding their health. Research in improving health-related texts directed to the general population is essential in the new model of clinical relationships, in which the patient has the leading role.6, 7

Aortic aneurysms are commonly encountered in medical practice. In the United States, the incidence of abdominal aortic aneurysm (AAA) in men older than 65 years ranges from 6% to 9%.8 Approximately 1% of all deaths in industrialized countries are attributed to AAA rupture.9 As to thoracic aortic aneurysms, the incidence of this condition is 10 cases per 100 000 persons/year10 and the current treatment of choice is endoprosthesis implantation, which is preferred to conventional open surgery in most cases. The changes in the management of aortic aneurysms began with the introduction of endoprosthesis treatment by Parodi in 1989.11 Now, more than half the elective procedures for AAA involve endoprosthesis insertion.12

The aim of this study is to determine the readability, accessibility, utility, and reliability of the information available on the Internet related to aortic aneurysm and its treatment by endoprosthesis insertion. In addition, the results obtained for both search terms used are compared to assess the implantation of this innovative, revolutionary treatment.

METHODS

This observational study began on 1 November 2010 with a search for information on the Internet. The following common topics in cardiovascular disease were selected for evaluation: aortic aneurysm and aortic endoprosthesis. The terms were written in each search engine as a phrase, and the search was limited to the Spanish language. The 3 search engines most widely used to identify the related Web sites were consulted: Google, Yahoo, and MSN/Bing. According to Golladay et al.,13 search engines are freely available programs that allow the user to search the Internet for Web pages related to a word or phrase entered in them. The selection of search engines was based on comScore2010, which showed that Google (62.6% of the total searches on the Internet), Yahoo (18.9%), and MSN/Bing (12.7%) were the Web search engines most commonly used in July 2010.14

The 3 authors who performed the search used the terms “aneurisma aorta” (aortic aneurysm) and “endoprótesis aorta” (aortic endoprosthesis). The results of the searches were not modified according to the importance of the Web sites or their contents. The first 30 Web pages retrieved by the search engines were compiled. Thus, 90 possible pages were analyzed per topic selected, making a total of 180 pages, excluding duplicated pages and those that contained irrelevant information. Each of the Web sites was classified into one of the following categories according to its creators: medical, academic, commercial, nonmedical, and nonspecific.12 Pages that did not contain any type of information on the subjects analyzed in relation to anatomy, pathophysiology, etiology, diagnosis, treatment, or prognosis were considered irrelevant.

The freeware version of the Inflesz program15 was used to calculate the readability of the texts selected. The following parameters were obtained: number of words, syllables, and sentences; mean number of syllables per word; mean number of words per sentence; and Flesch index (provided by Word), Flesch-Szigriszt index, Fernández-Huerta index, and Inflesz grade. The Gunning-Fog index was calculated using a freely accessed online utility.16 The LIDA instrument17 was used to assess the accessibility, utility, and reliability of the Web pages obtained.

Readability

Readability refers to a series of characteristics of a written text that enable one to read it and understand it with ease. There are several types of readability: linguistic, topographic, psychological, conceptual, structural, and pragmatic. Objective techniques are available to analyze linguistic readability, particularly for writing in the English language.18, 19 Various utilities, applications, and software programs can be used for this purpose, such as the Inflesz program, downloaded free of charge, which calculates the aforementioned parameters, among others, and the validation of the Szigriszt clarity scale, known as the Inflesz5 scale.

Flesch Index (Word Correlation Index)

In general, most readability formulas rely on the hypothesis that the shorter the words and sentences used in a text are, the easier the text is to read. R.F. Flesch has been the most influential person in developing formulas to analyze readability. His Reading Ease Score (RES) is based on the following equation:

in which WL is the number of syllables in a 100-word sample of text, and SL is the average length of the sentences in that 100-word sample. This is possibly the most well-known and influential formula in the history of readability. The interpretation of the results is carried out according to a scale proposed by the same author (Flesch Scale) with a range from 0 to 100, divided into 7 levels. Scores between 60 and 70 indicate moderate or standard difficulty for reading, and texts with lower scores are considered difficult to read. The closer the score is to 100, the easier the text is to read. Flesch-Szigriszt Index

The first formulas designed to analyze readability in the Spanish language appeared in the 1950s. Several attempts have been made to validate or adapt Flesch's original RES formula, such as the Fernández-Huerta readability formula and the Szigriszt-Pazos clarity formula. Without a doubt, the validation of the Flesch RES formula by Szigriszt-Pazos should be considered the current reference for the Spanish language. It is known as the Fórmula de Perspicuidad (Clarity Formula) or Índice de Legibilidad de Flesch-Szigriszt (IFSZ, Flesch-Szigriszt Readability Index):

As evaluated with this scale, the readability of a text with a score of 50 to 65 is considered average, and as the score approaches 0, where scientific literature is situated, texts become progressively more difficult.

Fernández-Huerta Index

The Fernández-Huerta index or reading ease formula is a modified version of the Flesch RES formula for Spanish texts, in which 0 is the score for greatest difficulty and 100 the score for greatest ease of reading. A text with a result of <30 is considered very difficult, whereas a score of 70 is considered appropriate for adults. The formula is as follows:

in which P is the number of syllables per 100 words and F the number of sentences per 100 words. Inflesz Scale Grade

As was reported in the study by Barrio-Cantalejo et al.20 in 2008, the Szigriszt Clarity Scale and the Flesch RES scale are not appropriate for the reading habits of the Spanish population. The authors of this study proposed the use of the new Inflesz scale, which is a modification of both these scales for a more appropriate assessment of texts in Spanish. On this scale, a score of 55 marks the cut-off between a text that is accessible or not to an average person. “Normal” is placed at a score of between 55 and 65, “very difficult”, between 0 and 40, and “somewhat difficult”, between 40 and 55. Among the higher scores, “quite easy” is indicated by a score of 65 to 80 and “very easy” by a score above 80.

Gunning-Fog Index

The Gunning-Fog index uses an algorithm based on the average number of words per sentence and the percentage of words containing 3 or more syllables. The specific formula is as follows: (average number of words per sentence+number of words with 3 or more syllables) × 0.4. The result is an index that indicates how many formal years of education a person would need to understand the content of the text. The lower the index is, the greater is the readability of the text. A Gunning-Fog index >17 is considered to be above the reading level of a high school student. Texts designed for general readers require an index <12. To reach nearly universal comprehension, a text should have an index <8.

Accessibility, Utility, and Reliability

The methods used to determine the quality of the information on Web sites vary, particularly for sites with medical content. Several tools are available for this purpose, but few have been tested to evaluate reliability. We chose the LIDA17 instrument because it has been validated in 3 important areas: accessibility, utility, and reliability. The LIDA results are considered high at >90%, moderate at between 50% and 90%, and low at <50%.

The LIDA instrument automatically provides results for accessibility. To obtain results for utility and reliability, a series of 9 questions are proposed to complete the validation. The 4 possible response options are “never”, “sometimes”, “mostly”, and “always”. To avoid possible bias and errors, two of the authors of this study responded to these questions.

Statistical Analysis

The mean±standard deviation were used to describe the quantitative variables. The Kolmogorov-Smirnov test was applied to verify that the variables followed a normal distribution (P=.047). In the analysis of differences between the various categories of the terms analyzed, the Student t test for independent samples was applied; a P value <.05 was considered significant. Determination of the degrees of interobserver agreement in the analysis of utility and reliability was performed by calculating the weighted Kappa with quadratic weights and the 95% confidence interval (CI), considering a value of ≥0.4 acceptable and a value of >0.75 excellent. Data were analyzed using SPSS 16.0 for Windows® (SPSS; Chicago, Illinois, United States).

RESULTS

Thousands of Web pages were obtained using the search terms of the study (Table 1). Among the first 90 pages obtained with the 3 search engines referring to the term “aneurisma aorta” (aortic aneurysm) (Table 2), 36 were repetitions (40%); hence 54 of the total (60%) were analyzed. In the search with the term “endoprótesis aorta” (aortic endoprosthesis), 33 (36.67%) of the first 90 Web pages obtained (Table 3) were repetitions, and 18 (20%) provided irrelevant information; hence 39 of the total (43.34%) were analyzed.

Table 1. List of Search Engines Used and Number of Results Provided.

Search engine Aneurisma aorta Endoprótesis aorta
Google 181 000 35 300
Yahoo 292 000 434 000
MSN/Bing 148 000 150 000

Table 2. First 10 Results Obtained With Each Search Engine After Entering the Term “Aneurisma Aorta”.

Google 1 http://www.clinicazurbano.com/es/blog-clinica-vascular-angiologia/47-bloq-vascular-angiologia/47-aneurismas-de-aorta.html
  2 http://www.tuotromedico.com/temas/aneurisma_de_aorta.htm
  3 http://es.wikipedia.org/wiki/Aneurisma_de_aorta
  4 http://www.cun.es/index.php?id=21404
  5 http://tratado.uninet.edu/c011301.html
  6 http://www.msd.es/publicaciones/mmerck_hogar/seccion_03/seccion_03_029.html
  7 http://www.portalesmedicos.com/publicaciones/articles/629/1/Patologia--aorta--toracica-Aneurisma-de-aorta-toracica-Apuntes-de-Cirugia-Cardiovascular-Apuntes-de-Patologia-Quirurgica-Apuntes-de-Medicina.html
  8 http://www.nlm.nih.gov/medlineplus/spanish/ency/article/001119.htm
  9 http://familydoctor.org/online/famdoces/home/articles/883.html
  10 http://www.mapfre.com/salud/es/cinformativo/aneurisma-de-aorta.shtml
Yahoo 1 http://es.wikipedia.org/wiki/Aneurisma_de_aorta
  2 http://es.wikipedia.org/wiki/Aneurisma
  3 http://www.tuotromedico.com/temas/aneurisma_de_aorta.htm
  4 http://www.mapfre.com/salud/es/cinformativo/aneurisma-de-aorta.shtml
  5 http://www.msd.es/publicaciones/mmerck_hogar/seccion_03/seccion_03_029.html
  6 http://www.medtronic.es/su-salud/aneurisma-aorta-abdominal/tratamiento/index.htm
  7 http://www.cdyte.com/index.php?option=com_content&view=article&id=194&Itemid=147
  8 http://tratado.uninet.edu/c011301.html
  9 http://www.saludalia.com/servlet/ServletConsultaDefinicion?idTermino=109&Termino=Aneurisma+de+aorta+
  10 http://www.clinicadam.com/salud/5/001119.html
MSN/Bing 1 http://es.wikipedia.org/wiki/Aneurisma_de_aorta
  2 http://www.tuotromedico.com/temas/aneurisma_de_aorta.htm
  3 http://www.aneurismadeaorta.com/
  4 http://es.wikipedia.org/wiki/Aneurisma
  5 http://tratado.uninet.edu/c011301.html
  6 http://www.msd.es/publicaciones/mmerck_hogar/seccion_03/seccion_03_029.html
  7 http://www.nlm.nih.gov/medlineplus/spanish/ency/article/001119.htm
  8 http://aneurismadeaortaabdominal.blogspot.com/
  9 http://www.mapfre.com/salud/es/cinformativo/aneurisma-de-aorta.shtml
  10 http://www.hemodinamiadelsur.com.ar/pacientes/articulo_002.asp

Table 3. First 10 Results Obtained With Each Search Engine After Entering the Term “Endoprótesis Aorta”.

Google 1 http://www.clinicazurbano.com/es/blog-clinica-vascular-angiologia/47-bloq-vascular-angiologia/47-aneurismas-de-aorta.html
  2 http://es.goremedical.com/excluder/patient/es_patientlanding?gclid=CJ30to74pKQCFUiU3wod608s6g
  3 http://www.teknon.es/consultorio/riambau/aneurismas.htm
  4 http://www.endovascularweb.com/
  5 http://escuela.med.puc.cl/deptos/cxvascular/TratamientoEndovascular.html
  6 http://www.symposium-cmi.org/casos_cas.htm
  7 http://www.medicrit.com/Revista/v3n5.06/35122.pdf
  8 http://www.medtronic.es/su-salud/aneurisma-aorta-abdominal/dispositivo/que-es/index.htm
  9 http://www.scielo.cl/scielo.php?pid=s0034-98872003000600005&script=sci_arttext
  10 http://www.cun.es/fileadmin/Servicios%20Generales/067.12-13.pdf
Yahoo 1 http://www.medtronic.es/su-salud/aneurisma-aorta-abdominal/dispositivo/que-es/index.htm
  2 http://www.medtronic.es/su-salud/aneurisma-aorta-abdominal/acceder-dispositivo/factores-considerar/index.htm
  3 http://www.diariomedico.com/2007/05/02/area-cientifica/especialidades/el-exito-de-la-endoprotesis-de-aorta-en-aneurisma-del-992
  4 http://www.cdyte.com/index.php?option=com_content&view=article&id=188&Itemid=141
  5 http://www.pharmaimage.tv/canal/clinica-universidad-de-navarra/trasplantado-con-exito-un-segmento-de-aorta-toracica-en-un-paciente-al-que-hubo-que-extraer
  6 http://www.scielo.org.ar/scielo.php?script=sci_arttext&pid=S0025-76802008000600006&lng=es&nrm=iso&tlng=es
  7 http://www.cun.es/nc/la-clinica/servicios-medicos/cardiologia/al-dia/noticias-del-departamento/noticia/back/27/actualidad/trasplantado-con-exito-un-segmento-de-aorta-toracica-en-un-paciente-al-que-hubo-que-extraer-una-dobl/
  8 http://www.enfermeriaencardiologia.com/revista/3303.pdf
  9 http://www.institutovascular.es/endoprotesis.html
  10 http://www.teknon.es/consultorio/riambau/aneurismas.htm
MSN/Bing 1 http://www.teknon.es/consultorio/riambau/aneurismas.htm
  2 http://www.endovascularweb.com/
  3 http://www.institutovascular.es/endoprotesis.html
  4 http://dialnet.unirioja.es/servlet/articulo?codigo=2331591
  5 http://www.medtronic.es/su-salud/aneurisma-aorta-abdominal/dispositivo/que-es/index.htm
  6 http://www.medtronic.es/su-salud/aneurisma-aorta-abdominal/dispositivo/que-es/index.htm
  7 http://www.enfermeriaencardiologia.com/revista/3303.pdf
  8 http://www.enfermeriaencardiologia.com/revista/res3303.htm
  9 http://www.sedar.es/restringido/2006/3/9.pdf
  10 http://www.registroanaconda.com/documents/protocolo_anaconda.pdf
Aortic Aneurysm

Most of the Web sites found were of a nonspecific type (39%), followed by medical (33%), academic (11%), commercial (11%), and nonmedical (6%) types. The mean number of syllables (3175.61±4.36), words (1367±1.82), and sentences (134±1.77) was high, as was the mean number of syllables per word (2.3±0.16) and words per sentence (10.48±3.41). Among the Web pages retrieved, the mean Flesch index score (Word correlation index) was 48.3±11.42, the Flesch-Szigriszt index was 52.96%±8.86%, corresponding to a “somewhat difficult” readability level, the Fernández-Huerta index was 58.05±8.5% (normal), and the Gunning-Fog index was 22.03±2.05, that is, much higher than the reading level of a high school student. In addition, the most frequent scores on the Inflesz scale (2.39±0.7) were “somewhat difficult” for 10 Web pages (55.56%) and “normal” for 6 (33.34%), with 1 “very difficult” (5.56%) and 1 “quite easy” (5.56%) (Table 4).

Table 4. Statistical Study of the Readability of Web Pages Containing the Term “Aneurisma Aorta”.

  Syllables Words Sentences s/w w/s Word correlation F-S index F-H index Inflesz grade G-F index
Mean 3175 1367 134 2.3 10.48 48.3 52.96 58.05 23.92 22.03
Median 1848 775 118 2.28 10.66 4.97 53.08 58.39 20.05 22.1
Standard deviation 2349.42 623.54 79.04 0.16 3.41 11.42 8.86 8.5 7.01 2.05
Minimum 661 285 25 2.08 3.55 29.9 26.02 32.5 10.03 15.96
Maximum 19 753 8221 820 2.85 17.12 75.98 69.38 74.01 40.01 26.9

F-H, Fernández-Huerta; F-S, Flesch-Szigriszt; G-F, Gunning-Fog; s/w, syllables per word; w/s, words per sentence.

The LIDA values for accessibility (82.28±14.14), utility (72.28±16.67), and reliability (46.17±28.69), as well as the overall LIDA value (70.22±16.85), indicated “moderate” results (Figure 1). Interobserver agreement for utility and reliability were κ=0.84 (0.73-0.91) and κ=0.76 (0-6.9-0.88), respectively.

Figure 1. Graphic representation of the accessibility, utility, and reliability of the Web pages encountered, obtained with the LIDA instrument.

Aortic Endoprosthesis

Among the 13 Web sites analyzed, 46% were of an academic type, 23% commercial, and 31% medical. The mean number of syllables (3657.69±2073.41), words (1570.46±895.15), and sentences (159.69±123.41), as well as the mean number of syllables per word (2.34±0.07) and words per sentence (12.07±4.9) were higher than the values found in the search for aortic aneurysm. The results for the Flesch Word correlation index (50.11±9.33), Flesch-Szigriszt index (49.31%±7.24%) and Fernández-Huerta index (54.44%±7.19%) yielded a “somewhat difficult” level of readability. In addition, the most frequent scores on the Inflesz scale (2.08±0.64) were also “somewhat difficult” for 8 Web pages (61.54%) and “normal” for 3 (23.08%), with 2 “very difficult” (15.8%). The Gunning-Fog index was slightly higher than that seen for Web pages on aortic aneurysm (23.86±1.59) (Table 5).

Table 5. Statistical Study of the Readability of Web Pages Containing the Term “Endoprótesis Aorta”.

  Syllables Words Sentences s/w w/s Word correlation F-S index F-H index Inflesz grade G-F index
Mean 3657 1570 159 2.34 12.07 50.11 49.31 54.44 20.86 23.86
Median 4428 1970 140 2.34 12.77 3.52 46.49 51.57 20.03 23.65
Standard deviation 2073.41 895.15 123.41 0.07 4.9 9.33 7.24 7.19 6.47 1.59
Minimum 853 380 27 2.23 4.86 13.59 38.68 43.66 10.02 21.49
Maximum 7483 3217 405 2.51 21.66 76.39 61.94 67.02 30.04 27.04

F-H, Fernández-Huerta; F-S, Flesch-Szigriszt; G-F, Gunning-Fog; s/w, syllables per word; w/s, words per sentence.

The LIDA study of accessibility, utility, and reliability (77.77±12.64, 72.28±26.61, and 56.38±16.17, respectively; total, 72.15±14.93) classified the Web pages related to the term endoprótesis de aorta as “moderate” (Figure 1). Interobserver agreement for utility and reliability were excellent (κ=0.82; 95% CI. 0.76-0.97 and κ=0.91; 95% CI, 0.84-0.93, respectively).

Comparison Between the Two Terms Analyzed

There were no statistically significant differences between the texts analyzed for the 2 search terms in the mean number of syllables (P=.59), words (P=.6), sentences (P=.87), mean syllables per word (P=.19), or words per sentence (P=.22). Nor were differences seen in any of the readability measures studied: Flesch (Word correlation) index (P=.87), Flesch-Szigriszt index (P=.87), Fernández-Huerta index (P=.82), Inflesz grade (P=.28) or Gunning-Fog index (P=.83).

With regard to accessibility (P=.98), utility (P=.08), reliability (P=.06) and the overall result obtained with the LIDA instrument (P=.52), no significant differences were seen between the pages obtained for the two search terms (Table 6).

Table 6. Statistical Study of the Accessibility, Utility, and Reliability of the Web Sites Found Containing the Terms “Aneurisma Aorta”(A) and “Endoprótesis Aorta” (B), Obtained With the LIDA Instrument.

  Accessibility Utility Reliability Total
  A B A B A B A B
Mean 82.28 77.77 72.28 72.28 46.17 56.38 70.22 72.15
Median 88 72 75 67 45 53 73 66
Standard deviation 14.14 12.64 16.67 26.61 28.69 16.17 16.85 14.93
Variance 199.86 159.69 277.74 708.09 823.09 281.09 284.07 222.81
Interval 48 35 65 90 107 67 62 54
Minimal 48 61 42 17 0 40 46 54
Maximum 96 96 107 107 107 107 108 108
DISCUSSION

The Internet has become the most extensively used information source. At the beginning of 1998, it was estimated that more than 720 million people worldwide would be using Internet at the end of 2005,13, 21 and a recent study reported that more than 28% of the population of our planet used Internet during 2010.22 In the last 3 months of 2010, 52.5% of the population of Spain consulted Internet services to seek information on health topics, most users being in the age range of 35 to 44 years (59.4%) and a somewhat smaller percentage in the range of 16 to 24 years (44.3%).23 More than half the population with Internet access seeks information related to health at least once a month.24 Although free access to the Internet has enabled the general public to obtain a great amount of medical information, there are few rules, guidelines, or governing bodies that examine the validity and reliability of the information provided.24, 25 This means that patients may acquire inadequate and inappropriate information about the topic of their concern,26, 27 and this fact can have a negative influence on their health-related decisions. Furthermore, and unfortunately, patient surveys have indicated that medical information on the Internet tends to be too complex and technical.27 Nonetheless, according to Eurostat, Spain is at the European average in this activity, with 34% of individuals between the age of 16 and 74 years using the Internet to seek information related to health.28

Our results show that patients and the general population who carry out a search for information on terms related to cardiovascular conditions, such as aortic aneurysm and aortic endoprosthesis, encounter moderate difficulty in the accessibility, utility, and reliability of the Web sites found, as well as a readability level classified as “somewhat difficult”. There were no statistically significant differences in readability between the two groups of pages related to the search terms; nonetheless, pages on aortic endoprosthesis obtained better results for utility and reliability.

When offering information directly to the general public, it is important not to hide comprehensibility behind a complex scientific vocabulary. Complex sentences containing long words and phrases can discourage and distract readers from the objective of learning about medical conditions. Educational material should obtain a balance between readability and learning.29 The Spanish language uses longer words and sentences than English does; hence, it is a good policy to use formulas that adapt the readability index to the characteristics of each language.16 In this study, the Flesch index was used because several studies have validated its use in Spanish and it is reported to be the most highly indicated system for evaluating medical information in general,5 being surpassed only by the Inflesz grade.20 The need to include medical terms in the texts analyzed may, in part, be the reason for the elevated results obtained, although in 2001 Berland et al.30 calculated the overestimation in medical texts at only 0.3. The United States Department of Health and Human Services has recommended several measures to improve the utility of health information.31 Additional ways to improve online information have also been proposed,32 such as enhancing the text with video and audio files and including interactive resources. Knowledge of other languages such as English can be a valuable factor when seeking high-quality information, since this increases the number of Web pages one can use.

Our study presents a series of limitations. Because of the dynamic nature of Web sites and the cross-sectional nature of our study, it is difficult to generalize the information obtained for future use. The changes in content produced with the passage of time are not represented. Second, although we used the most commonly consulted search engines and Web pages, the total of resources available on the Internet related to the terms analyzed were not investigated. Nonetheless, the results were obtained with the search strategy commonly used by the majority of the general population. For the same reason, we used very simple search expressions, although more sophisticated approaches may have yielded slightly different results. By changing terms used in the search algorithm, different Web pages may have been found. Furthermore, it is possible that some bias was derived from the authors’ work of assessing the utility and reliability of the Web pages studied. The most important limitation of this study, however, it that it is not a natural experiment (subjects from the general population did not perform the information search) and the resulting knowledge obtained was not evaluated. In addition, the research was limited to aortic aneurysms and aortic endoprostheses, and it cannot be generalized to other cardiovascular diseases. There may also be some bias from commercial pressure related to aortic endoprostheses, since 1 of every 4 Web sites with information on this subject belonged to a pharmaceutical company.

Health professions should recommend a series of Web sites on the Internet where patients can find easy to read, high-quality medical information, and should propose the best search terms and appropriate search strategies to locate other information.33 In addition, they should help their patients evaluate Web site credibility using the criteria proposed by various authors and governmental bodies.34 The currently available readability formulas do not consider whether the Web pages include drawings or multimedia information (eg, sound, videos, graphics). To obtain medical information, it would be very useful for scientific societies or qualified groups to develop Web sites, and if possible, unified ones, that are readable, accessible, useful, reliable, and positioned at the top of the ranking list on Internet search engines. Future studies investigating the impact of health information searching on the health professional-patient relationship and on patients managing their health problems would be of value.

CONCLUSIONS

This study shows that the information available on the Internet about terms related to cardiovascular disease, such as aortic aneurysm and its current treatment by endovascular prosthesis, is deficient in accessibility, utility, and reliability, and has the additional problem of difficult readability. The patient's capacity to understand the written information received can be significantly improved by adjusting the readability of the text to his or her reading level. This is essential for the Internet to be used as an accurate tool for disseminating scientific knowledge that will help healthcare users obtain high-quality, easily comprehensible medical information.

CONFLICTS OF INTEREST

None declared.

Received 4 December 2010
Accepted 17 April 2011

Corresponding author: Servicio de Angiología y Cirugía Vascular, Hospital Clínico Universitario de Valladolid, Ramón y Cajal 3, 47005 Valladolid, Spain. esannorberto@hotmail.com

Bibliography
[1]
Burkell J..
Health information seals of approval: What do they signify?..
Information, Communication & Society. , 27 (2004), pp. 491-509
[2]
Kummervold PE, Chronaki CE, Lausen B, Prokosch HU, Rasmussen J, Santana S, et al..
eHealth trends in Europe 2005-2007: a population-based survey..
J Med Internet Res. , 17 (2008), pp. e42
[3]
Soot LC, Moneta GL, Edwards JM..
Vascular surgery and the internet: a poor source of patient-oriented information..
J Vasc Surg. , 30 (1999), pp. 84-91
[4]
Eysenbach G, Kohler C..
How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests and in-depth interviews..
BMJ. , 324 (2002), pp. 573-577
[5]
Barrio I, Simón P..
Medición de la legibilidad de textos escritos. Correlación entre método manual de Flesch y métodos informáticos..
Aten Primaria. , 31 (2003), pp. 104-108
[6]
Organización Mundial de la Salud. Oficina regional Europea. Health for all policy framework for the WHO European region: 2005 update. Copenhagen: WHO; 2006..
[7]
Dedding C, Van Doorn R, Winkler L, Reis R..
How will e-health affect patient participation in the clinic? A review of e-health studies and the current evidence for changes in the relationship between medical professionals and patients..
Soc Sci Med. , 72 (2011), pp. 49-53
[8]
Thompson RW, Geraghty PJ, Lee JK..
Abdominal aortic aneurysms: Basic mechanisms and clinical implications..
Curr Probl Surg. , 39 (2002), pp. 110-230
[9]
Collin J, Araujo L, Walton J, Lindsell D..
Oxford screening programme for abdominal aortic aneurysm in men aged 65 to 74 years..
Lancet. , 332 (1988), pp. 612-615
[10]
Lee WA, Carter JW, Upchurch G, Seeger JM, Huber TS..
Perioperative outcomes after open and endovascular repair of intact abdominal aortic repair aneurysms in the United States during 2001..
J Vasc Surg. , 39 (2004), pp. 491-496
[11]
Parodi JC, Palmaz JC, Barone HD..
Transfemoral intraluminal graft implantation for abdominal aortic aneurysms..
Ann Vasc Surg. , 5 (1991), pp. 491-499
[12]
Bickerstaff LK, Pairolero PC, Hollier LH, Melton LJ, Van Peenen HJ, Cherry KJ, et al..
Thoracic aortic aneurysms: a population-based study..
Surgery. , 92 (1982), pp. 1103-1108
[13]
Mathur S, Shanti N, Brkaric M, Sood V, Kubeck J, Paulino C, et al..
Surfing for scoliosis: The quality of information available on the internet..
Spine. , 30 (2005), pp. 2695-2700
[14]
Top search providers for july 2010. Available at: http://searchenginewatch.com/3641266..
[15]
Programa INFLESZ. Available at: www.legibilidad.com..
[16]
Test document readability. Available at: www.online-utility.org..
[17]
The LIDA instrument (version 1.2). Available at: www.minervation.com..
[18]
Blanco A, Gutiérrez U..
Legibilidad de las páginas web sobre salud dirigidas a pacientes y lectores de la población general..
Rev Esp Salud Publica. , 76 (2002), pp. 321-331
[19]
Paasche-Orlow MK, Taylor HA, Brancanti FL..
Readability standards for informed-consent forms as compared with actual readability..
N Engl J Med. , 348 (2003), pp. 721-726
[20]
Barrio-Cantalejo IM, Simón-Lorda P, Melguizo M, Escalona I, Marijuán MI, Hernando P..
Validación de la escala INFLESZ para evaluar la legibilidad de los textos dirigidos a pacientes..
An Sist Sanit Navar. , 31 (2008), pp. 135-152
[21]
Murali NS, Murali HR, Auethavekiat P, Erwin PJ, Mandrekar JN, Manek NJ, et al..
Impact of FUTON and NAA bias on visibility of research..
Mayo Clin Proc. , 79 (2004), pp. 1001-1006
[22]
Mabrey JD..
Survey of patient-oriented total hip replacement information on the World Wide Web..
Clin Orthop Relat Res. , (2000), pp. 106-113
[23]
Encuesta sobre equipamiento y uso de tecnologías de la información y comunicación en los hogares 2010. Instituto Nacional de Estadística. Available at: http://www.ine.es/jaxi/menu.do?type=pcaxis&path=/t25/p450/a2010/&file=pcaxis..
[24]
Miniwatts Marketing Group. Internet usage statistics —The big picture. World Internet users and population stats 2006. Available at: http://www.internetworldstats.com/stats.htm..
[25]
Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J..
Evaluating the source and content of orthopaedic information on the Internet. The case of carpal tunnel syndrome..
J Bone Joint Surg Am. , 82 (2000), pp. 1540-1543
[26]
Winker MA, Flanagin A, Chi-Lum B, White J, Andrews K, Kennett RL, et al..
Guidelines for medical and health information sites on the internet: principles governing AMA web sites. American Medical Association..
JAMA. , 283 (2000), pp. 1600-1606
[27]
Bessell TL, McDonald S, Silagy CA, Anderson JN, Hiller JE, Sansom LN..
Do internet interventions for consumers cause more harm than good? A systematic review..
Health Expect. , 5 (2002), pp. 28-37
[28]
Individuals using the Internet for seeking health-related information. European Commission Eurostat. Available at: http://epp.eurostat.ec.europa.eu/portal/page/portal/product_details/dataset?p_product_code=TIN00101..
[29]
Wang SW, Capo JT, Orillaza N..
Readability and comprenhensibility of patient education material in hand-related web sites..
J Hand Surg. , 34 (2009), pp. 1308-1315
[30]
Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, et al..
Health information on the internet: accessibility, quality and readability in English and Spanish..
JAMA. , 285 (2001), pp. 2612-2621
[31]
U.S. Department of Education, Office of Educational Research and Improvement Adult literacy in America. Available at: http://www.nald.ca/full text/Report2/rep15-01.htm..
[32]
Gazmarian JA, Parker RM..
Overview of health literacy in health care..
Overview of health literacy in health care., pp. 1-2
[33]
Lawrence S, Giles CL..
Searching the world wide web..
Science. , 280 (1998), pp. 98-100
[34]
Boyer C, Selby M, Scherrer JR, Appel RD..
The Health On the Net Code of Conduct for medical and health Websites..
Comput Biol Med. , 28 (1998), pp. 603-610
Idiomas
Revista Española de Cardiología (English Edition)

Subscribe to our newsletter

Article options
Tools
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

es en
Política de cookies Cookies policy
Utilizamos cookies propias y de terceros para mejorar nuestros servicios y mostrarle publicidad relacionada con sus preferencias mediante el análisis de sus hábitos de navegación. Si continua navegando, consideramos que acepta su uso. Puede cambiar la configuración u obtener más información aquí. To improve our services and products, we use "cookies" (own or third parties authorized) to show advertising related to client preferences through the analyses of navigation customer behavior. Continuing navigation will be considered as acceptance of this use. You can change the settings or obtain more information by clicking here.