ABSTRACT

Background: As much as 80% of US adults search online for health related information. The value of the information is limited by the patients’ ability to comprehend it. Despite the recommendation by the American Medical Association (AMA) to provide all patient education materials (PEMs) at a 6^th grade reading level, many online sources do not conform. This study aims to evaluate the readability of PEMs from major online sources for anesthesiology. Methods: We determined the readability of PEMs from five major anesthesiology organizations and twelve brochures from the American Society of Anesthesiologists (ASA) using ten validated readability scales. PEMs from the various anesthesiology websites were compiled into a single word document for analysis. The twelve patient education brochures from the ASA were downloaded and analyzed separately. We then grouped the twelve documents into one sample and compared it to the other anesthesiology association websites. Results: All of the PEMs provided by the ASA were determined to be higher than a 6^th grade reading level with nine being at a 12^th grade reading level or above. The PEMs of the five major anesthesiology association websites were beyond the 6^th grade reading level with four out of five at greater than the 12^th grade reading level. Conclusion: The results showed that the readability of PEMs provided by five major anesthesiology associations and the ASA was beyond the 6^th grade. Therefore, the PEMs with improved readability in anesthesiology are needed.

1. Introduction

Research has shown that up to 80% of US adults search online for health-related information [1]. A study published in 2009 reported that 74% of all US adults use the internet, and 61% have looked for health or medical information online [2]. This number is growing exponentially with the increasing number of web users. Of those that reported using the internet for medical information, 60% say the information found online affected a medical decision, and 56% say it changed their overall approach to maintaining their health or the health of someone they help take care of [2]. This phenomenon will only increase with the growing popularity of smartphones and newer technology making internet access more expansive [3].

Although it is a standard of care to provide patients with a complete explanation of care, time constraints sometimes hinder the ability to fully do so [4]. This results in patients employing their own methods to obtain information. A major concern that arises is the comprehensibility of the materials patients may read. Nevertheless, studies have shown that the specific use of a computer website, as opposed to other forms of structured information, before scheduled surgery elicited a greater acquisition of knowledge of preoperative anesthesia [4].

Since the value of the information is limited by the patients’ ability to comprehend it, we must provide information to our patients with this in mind. The AMA has determined that an average adult reads at an eighth grade level [5]. Considering this determination, they recommend literature designed for patients at a 6^th grade reading level or below [5]. Medical practices that have a high prevalence of patients with limited literacy may want to consider lowering their goals accordingly, perhaps 3^rd to 5^th grade. One example of such a situation is encountered when considering patients with Medicaid which has an average reading skill for enrollees at the 5^th grade level [5]. Furthermore, it is important to recognize that an assigned reading grade level for ordinary literature may have little value when considering literature pertaining to more difficult topics, such as healthcare. Data from the National Assessment of Adult Literacy showed that only 12% of US adults had proficient health literacy [6]. Over a third of US adults experience difficulty with simple tasks relating to health care, such as following directions on a prescription drug label [6]. Furthermore there is evidence to suggest that although the prevalence of health literacy deficiency may be higher in some ethnic groups, it is an issue that is present regardless of race or ethnicity [6].

Most of the population uses the internet as a major resource for information including healthcare. Most websites are providing patient information written at a level too advanced to be of educational value [7-9]. The importance of our patients’ ability to understand their physicians is emphasized by the correlation between health outcome and health literacy. A report written by the ad hoc committee on health literacy in JAMA 1999 showed that health literacy is a stronger predictor of a person’s health than age, race, and socioeconomic standing [10]. Therefore, this study aims to evaluate the readability of patient education materials (PEMs) from major online sources for anesthesiology.

2. Methods

In July 2013, after institutional review board (IRB) approval, PEMs were retrieved from the web sites of five major anesthesiology associations (Figure 1): American Society of Anesthesiologists (ASA), American Society of Regional Anesthesia and Pain Medicine (ASRA), The Society for Pediatric Anesthesia (SPA), Society of Cardiovascular Anesthesiologists (SCA), and The Society for Ambulatory Anesthesia (SAMBA). For each web site, PEMs was compiled into a single Microsoft word document (Microsoft Corp., Redmond, WA). In addition, 12 patient education brochures from the ASA website were downloaded and analyzed separately (Figure 2). Material that was not related to the patient education text (e.g. hyperlinks, copyright notices, and disclaimers) was removed from the analysis. The documents were analyzed

using the Readability Studio Professional Edition Version 2012.1 (Oleander Software, Ltd, Vandalia, OH) and Microsoft Excel (Microsoft Corp).

For the analysis, ten commonly used readability scales, including specific health care-oriented scales, were selected to evaluate the web sites (Table 1). The scales used were as follows: Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), the New Fog Count Formula (NFC), Coleman-Liau Index (CLI), Gunning-Fog Index (GFI), SMOG (Simple Measure of Gobbledygook) Grading (SMOG), FORCAST formula, the New Dale-Chall Readability Formula (NDC), Raygor Readability Estimate, and the Fry Graph. The FRE scale rates the literature from 0 to 100, with 0 to 30 being very difficult; 30 to 50 being difficult; 51 to 60 being fairly difficult; 61 to 70 being standard; 71 - 80 being fairly easy, 81 - 90 being easy, and 91 - 100 being very easy. The score is based on reading ease and human interest [11]. The FKGL assigns a grade level to the literature being analyzed based on syllable count and sentence length [12]. NFC Formula assigns a grade level based on sentence length and the number of words containing

three or more syllables [12]. CLI evaluates literature according to sentence length and character count [13]. The GFI assigns a grade level based on sentence length and the number of words with 3 or more syllables, excluding proper nouns and words that are three syllables due to simple suffixes [14]. SMOG assigns a grade level based on sentence length and number of words with three or more syllables, without exclusions [15]. FORCAST uses a variety of variables to assign a grade level, but is most closely correlated to the number of one syllable words [16]. The NDC assigns a grade level based on the amount of “hard words”. Words are designated “hard”, if they do not appear on a list of 3000 familiar or “non-hard” words. This scale addresses the issue of skewed data arising from common words with 3 or more syllables [17]. RRE assigns grade level based on the number of sentences and words with 6 or more letters per any given 100 words in the material being analyzed [18]. The Fry Graph plots data to estimate readability according to the number of sentences and syllables in any given 100 words of the passage being analyzed [19].

3. Results

The readability of PEMs were evaluated and analyzed from the following websites: ASA, ASRA, SPA, SCA, and SAMBA.

Twelve patient brochures were downloaded from the ASA website and were analyzed individually using each of the chosen ten scales (Figure 2). Only two brochures (The Anesthesia Checklist and Smoking & Surgery) were evaluated to be within the sixth grade reading level on the NFC scale. Taking the mean grade level score of all scales with regards to each article demonstrates that all articles are well above the recommended 6^th grade reading level (Figure 1). The lowest overall grade level was assigned to the SPA on the NDC scale. The lowest average grade level assignment was also obtained by the SPA, evaluated at grade 11.

Concerning the differences in measurement of each readability scale in this analysis, the FRE scale had values ranging from 21 (very difficult) to 62 (standard), with a mean readability score of 38.1, in the difficult range. FKGL scored reading levels ranged from 8.3 to 15.6, with a mean grade level of 12.3. The NFC values ranged from 5.8 to 13.4, with a mean of 9.9. The CLI values ranged from 9.6 to 15.6, with a mean of 13.1. The GFI values ranged from 9.3 to 18.1, with a mean of 14.5. The SMOG values ranged from 10.9 to 17.2, with a mean value of 14.3. The FORCAST values ranged from 10.2 to 12.1, with a mean value of 11.3. The NDC reported a range of values from 7 to 16+, with 16 being the highest calculable grade level (corresponding to a university senior level or beyond). The NDC had the lowest mean readability value for the composite of all twelve patient articles. The FORCAST scale had the smallest range of values between the articles. The GFI had the largest range and assigned the highest overall grade level to the PEMs. The RRE reported a range from 9 to 17+ (maximum attainable score of the scale) (Figure 3). Three of the articles were evaluated at 17+, designating them suitable for post-graduate freshmen level or beyond. One article “What You Should Know About Herbal and Dietary Supplement Use and Anesthesia” failed to be effectively analyzed by the scale because it contained too many 6+ character words. The FRY graph reported values from 9 to 17+, with seven articles evaluated at 17+, corresponding to a post-graduate freshmen level or beyond (Figure 4).

The second evaluation compared the PEMs of five

major anesthesiology associations. None of the websites’ PEMs fell within the recommended grade level on any scale (Figure 1). The SPA received the lowest average grade level readability score and correlated with a grade level of 11.3. All the PEMs from the five major websites analyzed were all evaluated at a high school reading level or above.

With regards to the differences in the readability scales in the analysis of the PEM of the five anesthesiology association websites, the FRE values had a range of 34 to 51 with a mean of 41.2. The FKG values had a range of 11.1 to 15.3 with a mean of 12.9. The NFC values had a range of 9.9 to 16 with a mean of 12. The CLI values had a range of 11.2 to 13.1 with a mean of 12.4. The GFI values had a range of 13.5 to 17.8 with a mean of 15.3. The SMOG values had a range of 13.4 to 16.4 with a mean of 14.7. The FORCAST values had a range of 10.5 to 11.3 with a mean of 11. The NDC values had a range of 9.5 to 14. The FORCAST values assigned the smallest grade levels and also had the smallest range. GFI assigned the highest overall grade levels. The RRE values had a range of 11 to 13 with a mean of 12.3. The FRY Graph values had a range of 11 to 16 with a mean of 14.6. None of the PEMs reached the maximum grade level assigned by the scale (17+).

4. Discussion

Technology has pervaded all aspects of society and the medical community is no exception. The internet affects all aspects of healthcare right down to the physician patient relationship [20]. The effects are profound and may lead to serious changes in healthcare decision making amongst patients with access to online PEMs. In a study by Penson in 2002, 50% of patients report that the information they found on the internet led them to ask new questions of their doctor or to seek a second opinion, and more than 70% report that the information influenced a treatment decision [20]. The repercussions are clear on both a societal and personal level. Poor health literacy is estimated to contribute more than 73 billion dollars of additional burden to the US healthcare system [21]. Patients can become more active participants in their healthcare armed with valuable information presented in a manner that’s understandable to them. Moreover, it has been suggested that improving health literacy may improve self-help skills and reduce financial burden [22]. Despite this, much of the information available to patients online is of little value to our patients who can’t comprehend it. Readability is a major part of the quality considerations of online patient education materials. The lack of quality control during production can easily lead to a lack of readability [23]. Solutions have been considered, such as a systematized critical appraisal of health related information by third parties using a validated standard core vocabulary, but have yet to become common practice [23]. The major problem with this solution is that the internet is so dynamic and rapidly changing that it would be nearly impossible for third party reviews to keep up with all the literature available to patients.

Limitations of the Study

A number of studies [7-10,24-29] have shown that PEMs across numerous fields of medicine are written well above the AMA recommended 6^th grade reading level. However, this benchmark of a 6^th grade reading level may be misleading. The Institute of Medicine has suggested that even people with strong literary skills have trouble comprehending and processing health information [30]. Furthermore, basing the data on grade levels may be functionally flawed.  Functional illiteracy and marginal literacy can be equated at the 5^th grade or lower reading levels and 6^th to 8^th grade reading skills, respectively. Based on National Adult Literacy Surveys nearly half of the US population is either functionally illiterate or marginally literate. Around 25% of functionally illiterate individuals have graduated from high school. From these data it is evident that just because an individual has achieved a certain grade level does not automatically suggest that he or she is reading at that grade level [21]. There is a need for a unified approach in creating PEMs that would be comprehensible by all laypersons that may access these materials. The lack of a gold standard impedes this task [30]. 

Most of the current scales used to evaluate patient education materials rely on sentence and word length. A more accurate measure of readability should take into account medical vocabulary, cohesion, and style. The development of these scales is in process, but has yet to be completed and implemented [30]. Therefore, in order to ensure as much accuracy as possible, ten different readability scales were chosen. The results suggested the materials were written at grade levels beyond the recommended 6^th grade reading level.

Although the analysis in the this study shows the mean readability values of anesthesia PEMs to be much higher than the recommended grade level, additional factors must be considered. Multimedia videos, which are meant to supplement some of the text on the ASA website, may facilitate comprehension. Other factors, such as, graphics, layout, and typography may also facilitate readability, the extent to which is hard to say [21]. Another limitation is the nomenclature involved in the field of focus. The words anesthesia and anesthesiologist, themselves, are considered difficult words on all scales. These terms are unavoidable when considering the material at hand and cannot be replaced, just as the word “surgery” (also considered a difficult word and more frequent in some of the articles analyzed) could not be replaced. Despite these limitations, the need for improvement of readability of PEMs is evident, and should be attainable without losing the original intent of the message to our patients.

5. Conclusion

Patient education materials as provided by five major anesthesiology association websites and the ASA are written at greater than the recommended sixth grade reading level. Efforts must be made to improve the readability of PEMs in anesthesiology to promote health literacy in our patients.

REFERENCES