Creative Education 2013. Vol.4, No.11, 683-693 Published Online November 2013 in SciRes (http://www.scirp.org/journal/ce) http://dx.doi.org/10.4236/ce.2013.411097 Open Access 683 Assessment of an Interactive Internet Program to Educate Children Aged 7 - 9 about Science, the Brain and Drugs* Mary P. Metcalf Clinical Tools, Inc., Chapel Hill, USA Email: metcalf@clinicaltools.com Received October 4th, 2012; revised February 6th, 2013; accepted February 15th, 2013 Copyright © 2013 Mary P. Metcalf. This is an open access article distributed under the Creative Commons At- tribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. BrainTrain4Kids.com is an interactive science Internet program for children aged 7 - 9 developed with funding from the National Institute on Drug Abuse (NIDA). Based on NIDA’s classroom curriculum, Brain Power!, we adapted and expanded this material to optimize online media. The primary objective of the curriculum is to provide an early foundation for drug abuse prevention efforts by educating elemen- tary school-aged children about the brain and how alcohol, tobacco, and drugs can harm it. Secondary aims include improving knowledge, appreciation for scientific inquiry, and improving/sustaining a posi- tive attitude towards science. The program employs engaging and entertaining elements including Flash- based activities and educational games. Assessment of the program occurred using a two-group, random- ized case/control pre/post trial with a crossover design using a convenience sample. Child participants (N = 102) from control and case groups had similar knowledge and attitudes towards science at baseline. At post-intervention, there was a significant increase in knowledge scores for the case group; this increase was retained at the six-week follow-up. Case group attitudes towards science were more positive immedi- ately after post-intervention than at baseline, and at follow up than at baseline. BrainTrain4Kids can be an effective tool for educating children about science and drugs, and has the potential to positively impact at- titudes. It can be used as a part of a drug abuse prevention program either in schools or at home. Keywords: Neuroscience Education; Substance Abuse Prevention; Internet Based Education Introduction In society, concerns abound that the age of first use of drugs is decreasing. The National Parents’ Resource Institute for Drug Education (2002) reports that among 4th graders, 2.1% had smoked cigarettes in the past year, 5% had drunk beer, 2.7% had used inhalants, and 0.5% had used marijuana. An increase in all types of drug use was documented by the time children reached the 6th grade. DuRant, Smith, Kreiter, & Krowchuk (1999) found that among middle school children, 10.1% of smokers started at age nine or younger. Furthermore, the CDC (2007) reports that 24% of US students drank alco- hol for the first time prior to age 13. Experience with alcohol significantly increases between 4th - 6th grade (Donovan et al., 2004) and rises with each grade level, with boys more likely to have alcohol experience than girls. Although most school drug prevention programs are oriented to a middle school audience (Drug Strategies, 1999; e.g., NREPP, 2009; Winters, Fawkes, Fahnhorst, Botzet, & August, 2007), evidence indicates that drug prevention should actually begin in elementary school. One study found that the use of alcohol in elementary school increased the use of alcohol in middle school by nearly three times (Wilson, Battistich, Syme, & Boyce, 2002). Moreover, the same study found that early use of tobacco and marijuana in elementary school increased the use in middle school by five and four times, respectively. Hence, discouraging children from using drugs at an elementary school age could be beneficial in later years. There is also the problem of children falling behind in the sciences to consider. A study has documented that fourth grad- ers’ performance in math and science is decreasing relative to other countries (NCES, 2003). Moreover, in examining average science scores for fourth grade US students, 2007 scores were not measurably different compared to scores in 1995 (Gonzales et al., 2008). Hence, science education in the US is in need of improvement. The problem of drug abuse and erosion of knowledge in sci- ence can be addressed in tandem with a science-based drug prevention program. Focusing on early-age prevention, it is important to examine an effective mode for educating a young audience. Various health education programs have effectively used interactive technologies to teach children and youth in areas such as heart disease (e.g., Lindsay, Christie, Gaw, Pack- ard, & Shepherd, 1994), asthma (e.g., Lieberman, 2001), HIV (e.g., Schinke, Orlandi, Schilling, & Parms, 1992), and water quality (e.g., Yuan, Manderson, Tempongko, Wei, & Aiguo, 2000). Thus, we predict that a technology-based curriculum can help the efficacy of drug prevention programs. It was found that the use of animated cartoons in a multimedia intervention sig- nificantly increased elementary student’s science comprehen- *Conflicts of Interest: The author is employed by Clinical Tools, Inc. The development and assessment of the program were funded by a Small Busi- ness Innovation Research Award contract awarded to Clinical Tools, Inc. by the National Institutes of Health/National Institute on Drug Abuse.
M. P. METCALF sion and knowledge (Dalacosta, Kamariotaki-Paparrigopoulou, Palyvos, & Spyrellis, 2009), suggesting that certain technology features are engaged to a younger audience. Our rationale is that, via technology, we can help children learn and appreciate science. We wanted to explore the practicality of technological solutions to the delivery of drug abuse prevention programs, which leads to our main research question: will an interactive, multimedia, Internet-based version of a science and drug abuse prevention curriculum be effective in changing knowledge and attitude of children about science and drugs? To answer this, we had two aims: 1) create, and 2) evaluate such a program to investigate whether an Internet curriculum effects changes in knowledge and attitudes. We created a science education Internet curriculum, partly adapted from NIDA’s classroom curriculum entitled Brain- Power! The NIDA Junior Scientist Program, whose primary target audience is 2nd - 3rd graders. We enhanced this program to use the full potential of the Internet and designed it with features including animated cartoons, Flash movies, quizzes which provide immediate and customized feedback, 3D mod- eling with objects that can be manipulated, supplemented in some instances with audio. There are significant differences between the two curricula, which necessitated a separate study to determine the impact of BrainTrain4Kids on children in the target age range. To our knowledge, no interactive Internet program is designed solely for educating children aged 7 - 9 about science, neuroscience, and drugs for independent use. The primary goal is to lay the foundation for future drug abuse prevention efforts and scientific learning by educating children at an early age about the brain, the differences between helpful and harmful drugs, and how alcohol, tobacco and drugs affect the brain and body. Secondary objectives include improving knowledge and appreciation for science and scientific inquiry, and building or maintaining a positive attitude towards science. Evaluation of the curriculum involved: Instruments: the development of instruments to assess knowledge and science/drug attitudes which were created in consultation with education experts. Formative Analysis: a small pilot study of those instruments to determine the instruments’ viability to measure knowledge, attitude and satisfaction with the program. Summative Study: a final evaluation, to assess the impact of the curriculum in a randomized case-control study employing a test/retest methodology, and in which instrument/inter-item validity was examined. Early Age Drug Prevention Programs Multiple factors such as peers, media, siblings, and parents influence attitudes towards substances like alcohol. It is argued that since risk factors are present before initiation, prevention programs should start in elementary school and be reinforced in advanced schools (Bell, Kelley-Baker, Rider, &Ringwalt, 2005). NIDA (2003) outlines principles for school prevention plan- ning, recommending that prevention commence in preschool to address risk factors for drug abuse. They indicate that children with risk factors at age seven, such as poor academic perform- ance, are prone to engage in substance abuse by age 14 or 15. Subsequently, they suggest that delaying intervention until adolescence makes it increasingly harder to overcome risks, since children’s attitudes are well grounded by then. Addition- ally, the US Dept. of Education (1998) recommends that par- ents begin to lay the foundation for later drug abuse prevention as early as kindergarten. Numerous studies have shown significant knowledge reten- tion in children through brief educational interventions, indi- cating the desirability of school-based health educational pro- grams (Dressman & Hunter, 2002). Nearly 85% of children ages 12 - 13 attended drug education classes that included mul- tiple sessions, with 75.9% reporting having attended in the past year. Among 9 - 11-year-olds, the proportion of students at- tending similar drug education classes in the past year drops to 55.3 percent. Furthermore, only 12% of all youth ages 9 - 18 have ever attended such a class outside of school (NIDA, 2000). Which Drug Abuse Prevention Programs Are of Proven Effectiveness? The evaluation of the effectiveness of behavioral interven- tions such as drug use prevention programs involves quantita- tive and/or qualitative pre-and post-assessment of one or more of the core components of knowledge, attitudes and/or beliefs, and practice. A review by the National Registry of Evidence- based Programs and Practices (2002) found that there are a number of community-based or school-drug abuse prevention programs that were effective. However, Hallfors & Godette (2002) reported that a majority of schools were implementing drug abuse prevention programs which were neither evaluated for effectiveness nor exceeded limited effectiveness. They found that only 25% of teachers were using evidence-based programs, and that these programs appear to be more effective, partly as the teachers were specifically trained to teach them. The effectiveness of the popular and widely used DARE pro- gram was questioned; for example, one study conducted by the Research Triangle Institute for the Department of Justice failed to show evidence of reduced drug use in the schools and com- munities where the program has been implemented (Ennett, Tobler, Ringwalt, & Flewelling, 1994). There is some evidence to suggest greater impact where teachers, rather than law en- forcement officers conduct the program. There is also some evidence to suggest that involvement of a combined team of families, teachers and other adults in chil- dren’s drug abuse interventions may have a more positive im- pact than any one group alone, and that parents can have a posi- tive impact on drug education and attitudes as well as teachers (Hanson, Deere, Lee, Lewin, & Seval, 2001). Behavioral health intervention programs such as “Starting Early, Starting Smart” can provide effective approaches to delivering integrated be- havioral health services for preschool as well as older children (Hanson et al., 2001). In addition, a recent study by the Part- nership for a Drug-Free America suggests that, among teens, a willingness to discuss drug use “regularly” in the family was associated with a significant decrease in drug use. The US Dept. of Education has produced guidelines that summarize how par- ents can educate their children about drugs at each age level. Use of Computer Develivered Health Education for Youth In a study by Lieberman (2001), children and adolescents assume the role of main characters in a disease management video game, in which they must manage the character’s chronic Open Access 684
M. P. METCALF diabetes, like helping their character take appropriate amounts of insulin. Role-playing may help translate skills to real-life scenarios. After six months, children who used the video game experienced a 77% decline in diabetes-related urgent care and emergency room visits; also, this group improved in self-effi- cacy, diabetes self-care, and communication with parents about diabetes. The same study assessed a different video game about smoking prevention, found to be effective in increasing knowl- edge about the effects of smoking and attitudes towards not smoking. Anti-smoking attitudes and intentions at 10 - 12 years of age atrophy when children reach 13 - 14 years old, a time when many people pick up smoking. The use of computer based education has clear advantages in terms of accessibility for home and school use and low utiliza- tion cost when compared to teacher-, health educator- or com- munity-based interventions, relatively few health intervention programs have, to date, explored the potential of these media as tools for health behavior interventions. There is some evalua- tive research to support the view that this form of health educa- tion and behavioral intervention may be effective, although few studies have directly compared effectiveness with conventional approaches. In one study, the short-term impact on knowledge and interactive multimedia software on preventive alcohol education was positive in young adults (Reis, Riley, Lokman, & Baer, 2000). Shegog et al. (2001) found that a new educa- tional CD-ROM aimed at enhancing pediatric asthma education in children at ages of 9 - 13 was effective at enabling children to self-manage their disease. Programs delivered through the Internet, CD-ROM, and other interactive software can improve the efficacy of risk communication (Strecher, Greenwood, Wang & Dumont, 1999). According to Hornung et al. (2000), a multimedia interactive CD-ROM developed to teach elemen- tary age children about skin cancer was found to increase knowledge and awareness of skin cancer and in shaping posi- tive attitudes towards sun protection, after their use of the soft- ware. A computer programs format can encourage users to stay with the task longer and to absorb more material. Multimedia can also allow children to choose in a virtual safe environment (McPherson & Glazebrook, 2001). The multimedia’s recep- tiveness, partly occurs as technology can be incorporated and used as a game, far more interesting to children than plain text (Brown et al., 1997). Piaget’s theory suggests that interactivity prompts children to take an active vice passive role in learning, and attention theorists suggest that programs that employ ani- mation can keep children more alert (as cited in Hornung et al., 2000). Use of the Internet in Drug Prevention Programs The Internet is a knowledge resource widely used for drug prevention information, with both governmental and non-gov- ernmental, community, educational and consumer-based or- ganizations. An expanding range of novel interactive Internet or CD-ROM based tools is under-exploited for drug use preven- tion or in the general promotion of health behaviors among younger age groups. SAMHSA (2000) lists a number of pro- grams nationwide that it deems “model programs”. Of these, none offers an Internet component in order to be implemented and only two, the Too Good for Drugs (TGFD) program and the Protecting You/Protecting Me (PY/PM) program, integrate information about the effects of drugs and alcohol into a cur- riculum for 2nd and 3rd graders. To our knowledge, none have been converted to Internet-based applications. BrainTrain4Kids Program Description Based on NIDA’s classroom curriculum Brain Power! for 2nd - 3rd graders, we adapted material between 2004-2007 to make it more interactive. In doing so, we made the information more suitable to an Internet audience through our website. The value of an Internet-based program could increase access to, and aid in the dissemination of, such programs. Furthermore, we created additional material and all elements developed by us are copyrighted; this includes the games, many of the printables, the look and feel of the website, the assessment instruments, and all materials that were not taken directly from Brain Power!, and thus in the public domain. There were significant differ- ences between the two programs, encouraging our study to assess the impact of our Internet curriculum. First, our cur- riculum differs in its design for independent use by the student in a non-school setting, as in the home. Unlike Brain Power!, our program omits information on illicit drugs of abuse, and instead focuses on tobacco and alcohol. Since the potential of iatrogenic effect for any prevention program exists, we decided that it should solely focus on legal substances. We felt that whereas a classroom-based teacher can adapt a presentation to students and determine an apt presentation-level related to ille- gal drugs, an Internet-based program lacks this option. Both programs use six main lessons, but BrainTrain4Kids focuses and emphasizes the scientific method, the brain, and prevention of alcohol and tobacco use. Features unique to our program include Playground which contains arcade-style games, and subsequent links to outside sites with brain teasers, puzzles, and games, as well as the Grown-up Guide, with information on how to use the website, website content, educational objectives, all printable and hands-on activities, and additional resources. Also, children can complete the Junior Scientist Quiz at the program’s end. The website www.BrainTrain4Kids.com is fully developed and users can self-register free of charge via the Internet. Albeit the program is designed primarily for children aged 7 - 9, we tailored the site to parents, caregivers, and substance abuse workers. The program incorporates three minimum themes: science education, neuroscience, and drug abuse prevention. Albeit our program is designed for independent use, we do not propose that children attempt to use the site without paren- tal supervision. Children can navigate the program from any location at any time to allow them to work individually and to advance independently, rather than working under a class- room’s pace. There is no time limit on how quickly a user must complete an activity. Since all the material on the website is designed using Adobe Flash technology, the user can stop at any time and return to the activity when convenient. Users navigate at their own pace, and to explore subjects of interest in as often as desired. Three different methods of imparting information are used to engage the learner: the main program which presents and ex- plains the basic curriculum, interactive educational materials (online riddles, quizzes, and games that provide immediate feedback), and offline activities that further review and link information to real life via science experiments and hands-on activities. The program employs educational games, taken from ideas of game theory to reach educational objectives. As noted Open Access 685
M. P. METCALF in Cameron & Chudler (2003), in science teaching, there has been a shift away from didactic teaching, and adaptation to- wards using an inquiry-based method; this inquiry and hands- on approach stimulates engagement in reasoning and problem- solving. Our program begins by introducing the steps of scien- tific inquiry. BrainTrain4Kids follows the National Science Education Standards (1996) for levels K-4 in the following categories: 1) Science as Inquiry: Understanding the abilities necessary to do scientific inquiry 2) History and Nature of Science: Science as human en- deavor 3) Unifying Concepts and Processes: Systems, order, and organization 4) Physical Science Standards: Properties of objects and ma- terials 5) Life Science Standards: Characteristics of organisms 6) Science in Personal and Social Perspectives: Personal health Drug prevention programs in the U.S. have seen a rise in science-based prevention programs (Winters, Fawkes, Fahnhorst, Botzet, & August, 2007). For example, a program for high school students uses a multi-module curriculum, incorporating science-based information from areas such as forensic science, pharmacology, nutrition, and toxicology (Lennox & Cecchini, 2008). In a study by Sigelman et al. (2003), the scientific theory of drug action was used, by explaining the role of the brain in producing adverse effects due to cocaine and alcohol for 3rd - 6th graders. Additionally, an alcohol prevention program for 1st - 5th graders incorporates lessons and activities about the brain, its development, and the effects of alcohol on it, which in- creased knowledge and changed attitudes afterward (Bell, Kel- ley-Baker, Rider, & Ringwalt, 2005). Similar to these afore- mentioned programs, our curriculum uses science information to impart our drug abuse prevention message. In our research, we wanted application of facts, not just learning by rote. Underlying this approach is the constructivist theory, in which knowledge is best retained when there is active participation and a base of knowledge to build upon (Miller, Moreno, Willcockson, Smith, & Mayes, 2005). Our program is designed linearly so that each part of the curriculum builds on information presented earlier in the program. Additionally, we employ a narrative approach so the user is entertained; narra- tion occurs by a cerebral cortex, named Corty, who is the con- ductor of the Brain Train, and at some points, by Junior Scien- tists who travel with Corty as they learn about neuroscience. Learning can be enhanced through storytelling or a narrative (as cited in Miller et al., 2005). The program consists of six train stations that build a knowledge base encompassing science, the human brain, and how drugs affect the brain and body. Each station, in sequential order, focuses on a particular theme: Science Place (scientific inquiry), Brainville (brain function), Neurontown (how mes- sages travel through the nervous system), Drugopolis (helpful and harmful effects of drugs), The Smoke Stacks (risks associ- ated with smoking and what makes cigarettes addictive), and New Health City (what drugs do to your brain and body and how to stay healthy). Children can then learn how drugs and medicines affect the brain and body, using the knowledge gained in the previous stations about the brain and nervous system. This is unique, as few curricula teach the science be- hind drug addiction to elementary-school children. Each station has four buildings that children can visit, with each one focusing on a different topic of science education. Building 1: Welcome Center, introduces what children will learn in the station, and Buildings 2 and 3 contain the bulk of the station’s content while providing interactive lessons and activities. Lastly, Building 4: Brain Games, contains three categories of materials (Online Games, Printables, and Hands- on) that reinforce and supplement previous information pre- sented in the station. Each building has a unique design aligned with the station theme. For example, in Station 5 (The Smoke Stacks), educational objectives focus on preventing tobacco use; Building 1 resembles lungs and in Building 2 (Tobacco Fac- tory), the smoke stacks represent cigarettes. After completion of the six-station program, the user can click on the link to the final quiz at the end of Building 3 in Station 6 (New Health City) to earn a Junior Scientist Award and to print a Certifica- tion of Completion. Each station must initially be accessed in numerical order. The buildings within each station are password-protected to encourage children to complete the buildings in sequence; the password for each building is given after completion of the previous building. While the Internet lends itself to a child- directed pace of discovery, we believe that a logical order is needed to internalize the educational objectives, especially when the child is using the program independently. Such a design allows children to build upon this knowledge and to maximize their understanding of concepts. This design should compel children to visit them in order, allowing them to use knowledge gained in prior lessons to complete the following building’s activities and games. Methodology Formati ve A nalysis We created the BrainTrain4Kids website as well as new software for course development. This software enabled re- search staff to add content through simple, user-friendly editors. Among the range of research methods used by us is the testing of the system and product usability. During development, BrainTrain4Kids was assessed using a series of iterative, for- mative usability evaluations. The main goal was to gather qualitative information about the process of interaction between the user and the tested software application. By replicating the natural pattern of Internet use with a focus on qualitative be- havioral data, a relevant and objective assessment of a user’s experience is obtained. This is crucial for building successful web applications. Usability testing was conducted at two dif- ferent stages of the web application development. The general areas of research included the communication of the site’s pur- pose, the navigation, design, download time, performance of the application, and organization of the home page, the folders, and the separate pages (Nielson, 2000; Nielsen & Tahir, 2001). A total of seven usability tests were held with parent/child pairs, in which formalized evaluations solicited feedback from chil- dren (aged 7 - 9) and their parents. Concurrently, feedback was sought from professional teachers of this age group, and our education and content consultants. Child participants evaluated the website in a usability lab setting; testing was conducted on all stations and the overall site. After each round of formative analysis, we employed an iterative process to adapt the website in response to feedback; information from each round of testing led to changes that were incorporated into the program before Open Access 686
M. P. METCALF the next round of testing. Changes included navigation related issues, graphics/artwork, adding voice-over to all main program text, and similar interface changes. In addition, a small pilot study was conducted prior to the summative study to assess the instruments and the research protocol. The instruments were pretested with 30 parent/child pairs, to determine the instruments’ viability to measure knowledge, attitude and satisfaction with the program. Each pair completed a standardized informed consent process, and was assigned a user name and password. Each knowledge and attitude item included a follow-up question that asked the child if they thought they understood the question, or if it confused them. We also used a short instrument to gather feedback from the parents including topics such as level of language complex- ity in the questions themselves, the parent’s perception of the child’s understanding of the Likert-style answering scale, any technical problems with delivery, and the length of time that it took to complete the instruments. The responses to the knowledge test showed that there was neither a ceiling nor a floor effect for the knowledge instrument. Some questions were clearly more “difficult” as determined by the high number of incorrect answers. Children reported under- standing the format and the questions, even if they didn’t know the correct answer. The attitude scale did show a slight ceiling effect, as anticipated. Average scores for the positive questions ranged from 4.5 to 4.8 on a 5-point Likert scale, and “negative” questions such as “Smoking cigarettes is good for your health” had perfect disagreement. The only question which children and parents reported difficulty with was “People who exercise are cool” since it was pointed out that “exercise makes you hot”. No technical difficulties were found, and the online instru- ments passed the usability tests. As there were neither technical difficulties nor floor or ceiling effects, the knowledge instru- ment was not revised after the pilot test. We considered revis- ing the attitude instrument in order to reduce the ceiling effect that the data suggested. There were compelling arguments against this route. First, it was unlikely that we would be able to demonstrate a strong improvement in attitude towards science with this age group using a simple survey instrument, since children in this age range are well known to answer questions “the right way.” While it would be quite interesting to develop a more versatile instrument (e.g., using open ended questions, scenarios, or science stories), that was beyond the project’s scope. Second, the primary goal of the instrument was to check for any unintended negative changes. The potential for iatro- genic like effects with any drug use prevention is well estab- lished in the literature; the attitude survey’s main purpose in the final study was to assess the presence or absence of such effects. Findings from the pilot study indicated that the instrument items and format were usable and sufficiently clear to the target audience; hence, they were used at the beginning of a sentence. Summative Study Recruitment of subjects occurred via a variety of local papers and appropriate publications. Print materials included Carolina Parent Magazine and newspapers such as the Santa Fe New Mexican and the Albuquerque Journal. However, many re- cruitment advertisements were also posted online, in places such as Craig’s List, Postaroo, LookSeek classifieds, OK4Free.com, and online newspaper and news station classi- fieds in Texas, California, and North Carolina. These adver- tisements noted that parents must have children between the ages of seven and nine (or, in the second or third grade), be at least twenty-one years old, and have regular Internet access. Parents were asked to contact the researchers via email for more information about the study. Once we received a request for more information, we sent the parents a screening document. Information requested included the gender, ethnicity, race, age, and Internet use of both parents and child, as well as the child’s birth month and year, and school grade. The inclusion criteria were: interest in the project and willingness to participate. Ex- clusion criteria included an inability to use a computer. The screening document was also used to obtain adequate represen- tation from gender and minority groups. Distribution by race is shown in Table 1; distribution of participant ethnicity is shown in Table 2. Participants selected for the study were chosen to mirror the target enrollment as closely as possible. When the child par- ticipants (N = 102) were randomly assigned to either the con- trol or case group, some care was taken to divide the demo- graphic groups evenly. Evaluation Instru me nts We measured the effect of the intervention by developing two instruments. The outcome measures focused on changes in 1) knowledge and 2) attitude. Scales were developed to meas- ure these outcomes. Furthermore, satisfaction surveys were developed to assess components of the website. Knowledge areas are based upon the objectives of the Brain Power! curriculum and include: knowledge of basic compo- nents of scientific inquiry, names and functions of four major parts of the brain, basic information on what a nerve cell is and how nerves cells work, the differences between drugs and medicines, and the effects of nicotine on the brain and lungs. The knowledge instrument consisted of 21 total items; there were 16 multiple choice and 5 True/False (T/F) items. The T/F Table 1. Enrollment demographics by race. Racial Category Number of Enrollees Females Males Total% American Indian/Alaska Native2 3 5 4.2% Asian 3 3 6 5.10% Black or African American 13 5 18 15.3% Native Hawaiian/Other Pacific Islander 0 0 0 0% White 41 46 87 73.70% No Answer/Other 1 1 2 1.7% Total 60 58 118 100% Table 2. Enrollment demographics by ethnicity. Ethnic Category Number of Enrollees Females Males Total% Hispanic or Latino 6 6 12 10.2% NOT Hispanic or Latino 54 51 105 89% No Answer Given 0 1 1 0.8% Total 60 58 118 100% Open Access 687
M. P. METCALF Open Access 688 items included “I don’t know” as a possible response. The questions focused on recognizing the five main parts of the brain and functionality, understanding the steps of scientific inquiry, and negative health effects of tobacco use and alcohol misuse. Attitudinal/belief change goals include instilling in the target audience a general appreciation of science, an appreciation the brain and of the importance of keeping the brain healthy, and the belief that drugs can harm the brain. Attitudinal change is influenced by via interactive tools and features. The attitude scale was a 16-item instrument that assessed the child’s attitude towards science (6 items), health (5 items) and related health behavior issues (5 items). For example, items asked about science attitudes in general (“I think doing science is fun”), health/drug prevention (“Drinking alcohol can be harmful to your brain”), and health behavior (“I think exercise is fun”). Thirteen items were answered with a five-point Likert- style scale ranging from NO! to YES!, with the available option of “I’m not sure what this question means.” In addition, three items were related to the health behaviors of others, for exam- ple, “If someone I knew was smoking cigarettes, I would feel…” These questions had four possible answers: not at all worried about them, a little worried, very worried, or I don’t understand this sentence. Satisfaction scales were designed and measured components of the website including its utility, clarity, completeness, and ease-of-use. The measure also included global assessments including the user’s overall satisfaction, willingness to reuse the website and to recommend the website to other professionals/ individuals, interest in using Internet-based resource compared to paper-based, and perceived value. The scale is given after the curriculum intervention. Our standard for success is based on how many participants rated the item favorably. We deem any item that receives less than a 75% favorable rating (or a lower rating than other items) to indicate weakness. We then reevalu- ated the website to identify why that aspect was unsatisfactory. User self-report helped us answer: Will they recommend it to others, do they want to use it again, is it better than other means, and will it affect how they seek out additional resources? Study Desi g n Parent/child participant pairs were randomly assigned to ei- ther the control or case group, and assigned individual user- names and passwords. The study used a two-group, randomized case/control pre/post trial with a crossover design. Child par- ticipants were asked to complete assessment measures (knowl- edge and attitude) at three time points. As shown in Table 3, the control group completed the knowledge and attitude meas- ures at baseline, after a six-week wait period with no program intervention, and finally, a third time after “crossing over” to use the intervention. This crossover design allowed control group participants program access after the wait period. Case group participants were given knowledge and attitude measures at baseline, after using the intervention, and at a six-week fol- low-up. Administration of the pretest to the control group at baseline and after the six-week wait period provided informa- tion on the test-retest reliability of the measures. The follow-up test with the case group provided data on whether or not the effects on knowledge and attitudes were maintained. Partici- pants spent a maximum of thirteen weeks in the study, but some participants were unable to finish the study, and others were discontinued due to concerns over their participation (see Limitations). If a parent/child pair was selected for the summative study, the parent received consent and assent forms via mail. The parents were instructed to read and sign the consent forms, as well as to read the assent forms with their child before having the child sign them. Informed consent could also be completed by a brief phone conversation with two research members. Af- ter consent, parents received a link via email to the pretest (knowledge and attitude measures) instructions, which would ultimately provide us baseline measures. Child participants were asked to complete the pretest within one week of this initial email. After completion, parents were told whether their child was in the control or case group. Children in the control group were asked to wait six weeks without viewing the BrainTrain4Kids website. In contrast, par- ents of children in the case group were sent a link to the website and were instructed that their child was allowed to use the web- site at their own pace over the next six weeks. Six weeks after baseline in which both control and case groups took the pretest, all parents received an email asking their child to complete a second assessment, which contained the same questions as the pretest. These participants were given one week to complete this. If the participants did not complete the second assessment, they were sent a reminder email. Once the participants completed the second assessment, they received another email with instructions for the final step. Con- trol group participants were given a link to the website and were told that they had six weeks to view the entire site and finish the course. If the children did not complete the post-test at the end of six weeks, parents received a reminder email for their children. The case group was asked to wait six weeks, at the end of which they were sent an email with a link to the fol- low-up, and instructions to answer this within a week. Again, the third assessment for each group was the same as the base- line and second assessment taken by all participants. Results Factor analysis was deemed insufficient for this sample size and type of analysis. Both knowledge and attitude measures were tested for internal reliability, using Cronbach’s alpha. A result was deemed statistically significant when p < .001. Participants who did not complete the experience: Seventy-two Table 3. Summative evaluation timetable. Group Baseline (Week 1) Six-Week Interval (Weeks 1 - 6) 2nd Assessment (Week 7) Six-Week Interval (Weeks 7 - 12) 3rd Assessment (Week 13) Case Pretest BrainTrain4Kids Intervention Posttest Post-Intervention Interval Posttest 2 Control Pretest No Intervention Pretest 2 BrainTrain4Kids Intervention Posttest
M. P. METCALF of the 102 child participants completed all parts in the study time frame. An independent t-test analysis was performed to determine if the participants who dropped out, before com- pleting all three assessments, should be included. This analysis proved that there was no difference between the participants who completed all three assessments and those who dropped out after the first or second assessment (t(110) = −1.471, p = .144). This allowed us to include all available data for each knowledge and attitude measure. Knowled ge Assessment Sc oring and Item Analysis After the child participants completed the knowledge instru- ment, each answer was scored with either a 1 or a 0, for correct or incorrect answers, respectively. Skipped questions were treated as incorrect. Each child’s set of knowledge questions was scored for a possible total of 21. Cronbach’s alpha revealed that the T/F item 18 (“Some drugs are always helpful”) had a negative item-test correlation. Due to this finding, responses to item 18 were excluded. As the knowledge scores illustrate in Table 4, the average of correct answers at baseline from the control and case groups, were 53.75 and 52.67, respectively. Using a repeated measures ANOVA test, it was determined that there were no significant differences in the knowledge scores of the control and case participants at baseline (p = .484). After the second assessment, the knowledge scores of the case group increased, whereas those of the control group did not (see Table 4). At this point, a repeated measures ANOVA test showed that the difference in knowledge scores of the control and case participants was sta- tistically significant (p < .001). Additionally, between the first and second administration of the assessment in the control group, the knowledge scores did not change significantly (p = .057). After the third assessment, the scores of the control group increased, while the case group’s scores remained the same compared to their scores from the second assessment. A repeated measures ANOVA test was also used to compare the knowledge scores of boys versus girls as well as knowledge scores of children ages 7, 8 and 9. No significant differences existed among these groups. Attitude Assessment Scoring and Item Analysis Attitude scores were determined for the 13 questions an- swered with the Likert-style format by summing ratings for each statement. The statements were rated from 1 to 5; a score of 1 signified strong disagreement; a 5 meant strong agreement. Generally, agreement was considered a favorable response. However, some of the statements given were expected to elicit disagreement rather than agreement (“Smoking cigarettes can be helpful to your body”). The scores for these questions were reversed so that children who responded with a 1 received a score of 5, those who responded with a 2 received a 4, and so on. This allowed us to standardize the scores so that a high overall score was considered positive. Additionally, there were three attitude items regarding health behaviors of others that consisted of four possible answers: not at all worried about them, a little worried, very worried, or I don’t understand this sentence. The responses were given the following scores: 5 was given to the most favorable response, 3 to “A little worried about them”, and 1 was given to the least favorable response. It was found that all items had positive item-test correlations; the internal consistency reliability of the 16-item attitude outcome (across groups and administrations) was 0.711, indicating an acceptable level of balance between internal consistency and uniqueness for the items in the scale. As shown in Table 5, the average of correct answers for the attitude test at baseline from the control and case groups, were 85.31 and 86.58, respectively. Using a repeated measures ANOVA test, there was no statistically significant difference between the control and case groups at baseline (p = .519). However, after taking the second assessment, these two groups had significantly different scores (p = .012). The control group’s score also changed insignificantly from the pretest (p = .666), but the case group’s score increased significantly (p < .001). In the control group, scores on the second and third attitude assessments were compared using a paired samples t-test. The scores were found to be significantly different (p = .001), which suggests significant gains in the participants’ attitudes towards science and against drugs. In the case group, a paired samples t-test comparing the second and third assessments did not find a significant decrease in this group’s scores. However, the scores from the third assessment of the case group were insignificantly higher when compared to the pretest scores. A repeated measures ANOVA test was also used to compare the attitude scores of boys versus girls and the attitude scores of children ages 7, 8 and 9. Insignificant differences were found among the scores of these groups. Qualitative Feedback from Parents and Children Parents provided a variety of comments during the research study both voluntarily and via satisfaction surveys. Parents indicated that their children found use of the website fun, that Table 4. Knowledge scores for all child participants at three time points. Group N Minimum Maximum MeanStd Dev Control Baseline 52 20 85 53.7515.71 Admin #2 49 15 85 49.1813.67 After the Program37 30 100 70.9516.02 Case Baseline 60 25 80 52.6713.13 After the Program53 25 100 69.0617.92 Follow-Up 36 45 95 68.3314.83 Table 5. Attitude scores for all child participants at three time points. Group N Minimum Maximum MeanStd Dev Control Baseline 52 56.25 100 85.3111.17 Admin #2 49 65 100 85.879.53 After the Program36 75 100 91 6.65 Case Baseline 60 56.25 100 86.589.77 After the Program53 70.31 100 90.328.05 Follow-Up 36 70.31 100 88.608.13 Open Access 689
M. P. METCALF the information was useful, and that, while it contained a sig- nificant amount of information, it was valuable to expose their children to the vocabulary and served as a way to initiate family based conversations about both science and drug abuse and prevention. Several parents mentioned that while their children did not like “science”, the children did like playing the Brain- Train4Kids “game”. Discussion Since there was an insignificant difference in baseline scores between the control and case groups, the children in both groups were comparable in knowledge and attitudes about sci- ence and drugs. This allowed comparison of the two groups. We are confident that there was no measurable testing effect that could alter our results, because the control group did not show any significant differences in knowledge and attitude scores after completion of the second assessment. In the case group, we found that there was a significant in- crease in knowledge scores on the second assessment, as com- pared to their pretest scores and the control group’s second assessment scores. The control group’s knowledge scores in- creased significantly after the program intervention, which shows that this group learned from BrainTrain4Kids after use. The case group’s knowledge scores at follow-up did not sig- nificantly differ from scores in the second assessment. Thus, our findings demonstrate that children in the case group retained knowledge changes over a follow-up period of six weeks. After evaluating the impact of the program’s education ex- perience on children’s knowledge, our results support that chil- dren can and do learn material related to neuroscience. This underscores the inclusion of more specific neuroscience in cur- ricula related to health, science, and substance abuse prevention, even at early ages. As noted in Cameron & Chudler (2003), a knowledge base of the brain and how drugs affect the brain may help children understand other conditions, such as learning disabilities of students who have ADHD or dyslexia. Unfortunately, the attitude results were less dramatic. For the case group, there was a significant increase in average attitude scores on the second assessment, as compared to their pretest scores and the control group’s second assessment scores. Fur- thermore, compared to the second assessment, the average atti- tude score decreased insignificantly in the third assessment, to a point where it could not be considered different. Moreover, case group attitudes towards science were higher at follow-up than at baseline, but the difference is statistically insignificant within this sample. One possible explanation is that this is due to a ceiling effect since the children initially had high scores at baseline, subsequently making an increase in attitude difficult to observe. Noteworthy is that use of the program significantly improved attitudes for both control and case groups, and nega- tive effects were absent; iatrogenic effects were unseen. Atti- tudes towards science and drugs were influenced in the hoped- for direction. The attitude baseline scores were consistent with other re- search focused on elementary-aged children towards science, such as Sorge (2007) review of science attitudes from ages 9 - 14. We are unaware of other current research that discusses the attitudes of children in the U.S. as young as 7 and 8 towards science. Our finding, even with such a small sample size, that 7 and 8 year-old girls and boys share the positive attitude with their older counterparts (aged 9 - 11 in the Sorge study) is a contribution. One would expect that, in a broader sample, the overall positive attitude would be lower. Nonetheless, it is en- couraging that our attitude results align with other studies indi- cating that negative attitudes towards science are absent in the early elementary years. Based on a search of current elementary attitudes toward science, there appears to be research on this area in other countries, including places such as the UK (e.g., Jarvis & Pell, 2005; Osborne, 2003) but a dearth of current data on attitudes of US elementary-age students towards science still exists. Overall, the program is effective in increasing children’s knowledge of the brain and nervous system and positively im- pacts children’s attitudes on science and drugs regardless of race, gender, or age, assuming that all children are between 7 and 9 years old. Furthermore, parents and children reported high satisfaction with the program. Parents were concerned that their child might not retain the information if the child did not continue to use the site after the course, yet the brief follow-up for the case group indicates this is not necessarily the case. The real world use of the program will permit children to return to the website, as often as desired. Many of the interactive games and the offline activities are designed to attract repeat visits. Furthermore, it is possible that parents and children did not realize that they were welcome to return to the website after program completion. Many parents said that their child found the website engaging and that the information was helpful and well-presented. However, many parents also mentioned that the information was either too complicated or, in some cases, bor- ing. Another consistent theme included that the program was a useful starting point for family discussions of the issues. The overall anti-drug message was well-received, but the details about brain parts and nervous system functionality were diffi- cult to grasp for some children. It is interesting that the knowl- edge results indicated that the children as a group could grasp more material than the parents envisioned. Limitations An unanticipated finding is a cautionary note about Internet- based research. From our participation inquiries, we believe that as many as five “parent participants” lacked a child par- ticipant. For example, one “parent” gave different birth-dates for a child periodically, without correlation to earlier responses when queried. One of the dates would have excluded participa- tion by the “child.” Other “child users” completed online sur- vey forms after midnight in their stated time zones. Bearing in mind that the study period was during the typical academic year, and that the child participants were aged 7 - 9 years, we con- cluded that these were not actual participants, and hence ex- cluded the data. When queried by email regarding the time stamp of the survey forms, two parents did not reply, strength- ening our belief that there was not an actual child participant or they were apathetic. To circumvent these challenges, we developed a series of questions about the child participant during the informed con- sent interview, with Institutional Review Board (IRB) approval. This information was not verified, but the ability of the poten- tial participant to answer questions became an inclusion criteria. Questions were child-focused, related to current events, and tangent to general science learning. Since this was a screening interview to determine eligibility to participate, the questions were appropriate and not invasive; for example, Can you tell Open Access 690
M. P. METCALF me some other child friendly websites your child uses at home/ school? We noted that some individuals who presented them- selves as “parents” were unable to answer these questions or to provide appropriate answers. With a convenience sample, we opted to omit these potential participants. All together, fewer than 10 “potential” parent/child pairs were uninvited to partici- pate or continue with the study. Additionally, we dropped five participants who turned in surveys with times stamps between midnight - 6 am, times that would be unreasonable for a child to be awake to take a survey. Since this was a convenience sample, the parents were inter- ested in their child participating, and the child may be more interested in the topics of science or drug prevention interven- tion and more motivated to do well. Conversely, the child par- ticipants might resent their parents forcing this study and re- spond in an unintentional poor manner. Post hoc analysis de- termined that the instruments had strong validity, but the pri- mary measure used was face-validity. Children aged 7 - 9 often answer attitude questions the way “they think they should” rather than honestly, or may have insufficient background knowledge to be able to accurately self-assess. In a study by Krishna et al. (2003), an internet-enabled mul- timedia program was developed to educate children about asthma, by including animated lessons and real-life scenarios; in addition to increasing knowledge, the program decreased emergency visits and lowered morbidity. In comparison to this study, we did not directly assess behavior. It is unclear whether the knowledge and attitudes gained will ultimately influence or alter behavior. There was no longitudinal follow-up, and the incipient follow-up was brief. The program was also short in duration; frequent use of the program might yield more signifi- cant findings. Rideout (2001) reports that among young “health surfers” aged 15 - 24, 39% altered their behavior in response to observed online health information. Thus, knowledge of online information can cause people to act. The same rationale that people can alter behavior based on knowledge was used in our curriculum, in the effort that a young elementary-age audience also has the capacity to change attitude by providing them a knowledge base about science and drugs. Our program used a targeted approach to cater to children aged 7 - 9, but this age range may be considered broad. A more targeted approach to this age group might have been more ef- fective, or a focus on a high-risk child population. Also, due to a small sample size, we cannot generalize our results to chil- dren at-large; regardless, our results are promising. Conclusion Alcohol and tobacco use among youth prompts the need for drug abuse education. Delay in initiation of alcohol or tobacco use is a significant public health goal. We addressed the prob- lem of drug abuse by developing and evaluating an interactive program. We took a traditional educational intervention and transferred it to a new technology, in this case the Internet. Per- haps other programs with proven efficacy can be incorporated into a new technology. Our technology-based curriculum is fun and maximizes entertainment via games and animations to mo- tivate and engage the user. It is science-based, and introduced scientific inquiry to lay groundwork for later problem-solving while teaching the science behind drug abuse and addiction. The goal of BrainTrain4Kids was to lay a foundation for later drug abuse prevention programs and interventions by educating an elementary school-age audience. This gave children a know- ledge base for drug abuse prevention efforts by offering educa- tional games, interactive science lessons, and hands-on activi- ties that teach about science, the brain, and drugs. We conclude that our program was effective in increasing general science and scientific inquiry knowledge, and in instilling positive atti- tudes toward science. Currently, a decrease in positive attitudes towards science occurs toward the end of elementary school (Sorge, 2007). Further research of early elementary-age stu- dents towards science and health could raise understanding of these attitudes, especially as educators try to determine how to affect this decrease. Moreover, this area warrants research as an atrophy of science understanding may result in an inability for children and youth to make informed rational health choices. More research is also needed to assess if teaching the science behind drug abuse and addiction is effective, with a challenge to do so at an appropriate age level. While the conclusions are limited to this specific intervention and a small sample size, the positive results suggest that inter- active Internet-based materials such as game-style elements can be an effective way to deliver drug abuse prevention and to educate an elementary-age audience. Another implication is the use of the Internet to be used independently by children, and that it can be used as a medium to increase science and health knowledge, while improving attitude. Ergo, these results sug- gest that the Internet can be an effective tool for developing and implementing children’s educational tools for other curricula. However, more research is needed to assess if games can be effective with other teaching curricula and other age groups, as in continuing education courses for medical students and healthcare professionals. One study found that third-year medi- cal students who used a Jeopardy-style game, vice the lecture teaching method, to learn about ectopic pregnancy, had an in- significant difference in knowledge learned between the two groups. Albeit, students who used the game rated it signifi- cantly higher in helping to retain information, overall enjoy- ment, and stimulating faculty/student interaction while learning about ectopic pregnancies (O’Leary, Diepenhorst, Churley- Strom, & Magrane, 2005), suggesting that the game format offers additional advantages. A future challenge will be inves- tigating how games can be applied to different curricula and how they can be specifically applied to an Internet curriculum, designed for independent use. More research on how to maxi- mize the effectiveness of Internet programs in increasing knowledge is needed. As future educational resources become even more restric- tive, programs such as BrainTrain4Kids can expand the class- room teacher’s reach without straining limited time. As the Internet takes a role in health education, appropriate websites and educational programs can be of avail outside the classroom as educational tools. This study’s results, complemented by positive reviews of both parents and children, imply that BrainTrain4Kids is useful to enable parents and teachers to educate children about science and drugs. This program can be a growing resource of drug prevention education and preven- tion. Our study implies that our program can complement a drug abuse prevention program in schools or at home; likewise, it can be a great economical investment to achieve results. Acknowledgements This research was funded by a Small Business Innovation Open Access 691
M. P. METCALF Research contract, #HHSN271200455234C from NIDA. Brain- Power! is distributed by NIDA. Clinical Tools, Inc. distributes Brain-Train4Kids free of charge to users. Between 2003-2005, Elizabeth B. Sullivan was the Primary Investigator for this pro- ject, and we appreciate her contribution, as well as that of Meg- han Hennis, Bart Wodjiouski, Sean Sealy, Lyndsay Kelley, Kimberly Workman, and Ashley Crimmins. We also thank Sharyn Levine Rosenberg, who performed the majority of the statistical analyses. REFERENCES Baylor College of Medicine (2004). BrainLink. http://www.lifescied.org/content/5/2/118.full Bell, M. L., Kelley-Baker, T., Rider, R., & Ringwalt, C. (2005). Protecting you/protecting me: Effects of an alcohol prevention and vehicle safety program on elementary students. Journal of School Health, 75, 171-175. Brown, S. J., Lieberman, D. A., Germeny, B. A., Fan, Y. C., Wilson, D. M., & Pasta, D. J. (1997). Educational video game for juvenile diabetes: Results of a controlled trial. Medical Informatics (London), 22, 77-89. http://dx.doi.org/10.3109/14639239709089835 Cameron, W., & Chudler, E. (2003). A role for neuroscientists in engaging young minds. Nature Reviews Neuroscience, 4, 763-768. http://dx.doi.org/10.1038/nrn1200 Center for Disease Control and Prevention (2007). United States 2007: Percentage of students who drank alcohol (other than a few sips) for the first time before 13 years. http://www.cdc.gov/mmwr/preview/mmwrhtml/ss5704a1.htm Dalacosta, K., Kamariotaki-Paparrigopoulou, M., Palyvos, J. A., & Spyrellis, N. (2009). Multimedia application with animated cartoons for teaching science in elementary education. Computers & Edu- cation, 52, 741-748. http://dx.doi.org/10.1016/j.compedu.2008.11.018 Donovan, J. E., Leech, S. L., Zucker, R. A., Loveland-Cherry, C. J., Jester, J. M., Fitzgerald, H. E., Puttler, L. I., Wong, M. M., & Looman, W. S. (2004). Really underage drinkers: Alcohol use among elementary students. Alcoholism: Clinical and Experimental Re- search, 28, 341-349. http://dx.doi.org/10.1097/01.ALC.0000113922.77569.4E Dressman, L. A., & Hunter, J. (2002). Stroke awareness and knowledge retention in children: The brain child project. St r oke, 33, 623-625. http://dx.doi.org/10.1161/hs0202.102728 Drug Strategies (1999). Making the grade: A guide to school drug prevention programs. Washington, DC: Author. DuRant, R. H., Smith, J. A., Kreiter, S. R., & Krowchuk, D. P. (1999). The relationship between early age of onset of initial substance use and engaging in multiple health risk behaviors among young adole- scents. Archives of Pediatrics & Adolescent Medicine, 153, 286-291. http://dx.doi.org/10.1001/archpedi.153.3.286 Ennett, S. T., Tobler, N. S., Ringwalt, C. L., & Flewelling, R. L. (1994). How effective is drug abuse resistance education? A meta-analysis of Project DARE outcome evaluations. American Journal of Public Health, 84, 1394-1401. http://dx.doi.org/10.2105/AJPH.84.9.1394 Gonzales, P., Williams, T., Jocelyn, L., Roey, S., Kastberg, D., & Brenwald, S. (2008). Highlights from TIMSS 2007: Mathematics and science achievement of U.S. fourth- and eighth-grade students in an international context (NCES 2009-001 Revised). Washington, DC: US Department of Education, National Center for Education Sta- tistics, Institute of Education Sciences. Hallfors, D., & Godette, D. (2002). Will the “principles of effectiveness” improve prevention practice? Early findings from a diffusion study. Health Education Resea r c h, 17, 461-470. http://dx.doi.org/10.1093/her/17.4.461 Hanson, L., Deere, D., Lee, C., Lewin, A., & Seval, C. (2001). Key principles in providing integrated behavioral health services for young children and their families: The starting early starting smart experience. Washington, DC: US Department of Health and Human Services Hornung, R. L., Lennon, P. A., Garrett, J. M., DeVellis, R. F., Weinberg, P. D., & Stretcher, V. J. (2000). Interactive computer technology for skin cancer prevention targeting children. American Journal of Preventive Medicine, 18, 69-76. http://dx.doi.org/10.1016/S0749-3797(99)00115-4 Jarvis, T., & Pell, A. (2005). Factors influencing elementary school children's attitudes toward science, before, during, and after a visit to the UK National Space Centre. Journal of Research in Science Teaching, 42, 53-83. http://dx.doi.org/10.1002/tea.20045 Krishna, S., Francisco, B. D., Balas, E. A., König, P., Graff, G. R., & Madsen, R. W. (2003). Internet-enabled interactive multimedia asthma education program: A randomized trial. Pediatrics, 111, 503-510. http://dx.doi.org/10.1542/peds.111.3.503 Lennox, R. D., & Cecchini, M. A. (2008). The NARCONON drug education curriculum for high school students: A non-randomized, controlled prevention trial. Substance Abuse Treatment, Prevention, and Policy [electronic], 3, 8. http://www.substanceabusepolicy.com/content/3/1/8 http://dx.doi.org/10.1186/1747-597X-3-8 Lieberman, D. A. (2001). Management of chronic pediatric diseases with interactive health games: Theory and research findings. Journal of Ambulatory Care Management, 24, 26-38. http://dx.doi.org/10.1097/00004479-200101000-00004 Lindsay, G. M., Christie, J., Gaw, A., Packard, C. J., & Shepherd J. (1994). Educating children about heart disease. Nursing Standard, 8, 32-25. McPherson, A., & Glazebrook, C. (2001). Double click for health: The role of multimedia in asthma education. Archives of Disease in Childhood, 85, 447-449. http://dx.doi.org/10.1136/adc.85.6.447 Miller, L., Moreno, J., Willcockson, I., Smith, D., & Mayes, J. (2005). An online, interactive approach to teaching neuroscience to adole- scents.CBE-Life Sciences Education, 5, 137-143. http://dx.doi.org/10.1187/cbe.05-08-0115 National Center for Education Statistics (2003). Trends in international mathematics and science study (TIMSS), mathematics and science achievement of fourth-graders between 1995 and 2003. http://nces.ed.gov/Timss/results03_fourth95.asp National Institute on Drug Abuse (2000). Evaluation of the National Youth Anti-Drug Media Campaign (No. N01DA-8-5063). Rockville, MD: US Department of Health and Human Services, National Institute of Health. National Institute on Drug Abuse (2002). Brain power! The NIDA Junior Scientist Program. http://backtoschool.drugabuse.gov/curricula.html National Institute on Drug Abuse (2003). Preventing drug use among children and adolescents: A research-based guide for parents, educators, and community leaders (2nd ed.). Bethesda, MD: US Department of Education, National Center for Educational Statistics. National Parents’ Resource Institute for Drug Education (2002). Grades 4-6 pride surveys national summary (2001-02). http://www.pridesurveys.com/customercenter/01elemns.pdf National Registry of Evidence-Based Programs and Practices (2002, revised 2009). SAMHSA’s national registry of evidence-based pro- grams and practices. http://www.nrepp.samhsa.gov National Science Education Standards (1996). National science edu- cation standards. Washington, DC: National Academy Press Nielsen, J. (2000). Designing web usability: The practice of simplicity. Indianapolis, IN: New Riders Publishing. Nielsen, J., & Tahir, M. (2001). Homepage usability: 50 websites deconstructed. Indianapolis, IN: New Riders Publishing. O’Leary, S., Diepenhorst, L., Churley-Strom, R., & Magrane, D. (2005). Educational games in an obstetrics and gynecology core curriculum. American Journal of Obstetrics and Gynecology, 193, 1848-1851. http://dx.doi.org/10.1016/j.ajog.2005.07.059 Osborne, J. (2003). Attitude towards science: A review of the literature and its implications. International Journal of Science Education, 25, 1049-1079. http://dx.doi.org/10.1080/0950069032000032199 Reis, J., Riley, W., Lokman, L., & Baer, J. (2000). Interactive multi- Open Access 692
M. P. METCALF Open Access 693 media preventive alcohol education: A technology application in higher education. Journal of Drug Education, 3 0, 399-421. http://dx.doi.org/10.2190/LWMQ-9CQA-B78H-9MA7 Rideout, V. (2001). Generation Rx.com: How young people use the internet for health information. Menlo Park, CA: The Henry J. Kaiser Family Foundation. Schinke, S. P., Orlandi, M. A., Schilling, R. F., & Parms, C. (1992). Feasibility of interactive videodisc technology to teach minority youth about preventing HIV infection. Public Health Reports, 107, 323-330. Shegog R., Bartholomew, R. K., Parcel, G. S., Sockrider, M. M, Masse, L., & Abramson, S. L. (2001). Impact of a computer-assisted education program on factors related to asthma self-management behavior. Journa l of the American Medical Informatics, 8, 49-61. http://dx.doi.org/10.1136/jamia.2001.0080049 Sigelman, C. K., Bridges, L. J., Leach, D. B., Mack, K. L., Rinehart, C. S., Sorongon, A. G., Brewster, A. B., & Wirtz, P. W. (2003). The efficacy of an education program to teach children a scientific theory of how drugs affect behavior. Applied Developmental Psychology, 24, 573-593. http://dx.doi.org/10.1016/j.appdev.2003.08.001 Sorge, C. (2007). What happens? Relationship of age and gender with science attitudes from elementary to middle school. Science Educator, 16, 33-37. Strecher, V. J., Greenwood, T., Wang, C., & Dumont, D. (1999). Interactive multimedia and risk communication. Journal of the National Cancer Institute Monograp hs, 1999, 134-139. http://dx.doi.org/10.1093/oxfordjournals.jncimonographs.a024188 United States Department of Education. (1998). Growing up drug free: A parents’ guide to prevention. Washington, DC: US Department of Education, Office of Elementary and Secondary Education, Safe and Drug-Free Schools Program. Wilson, N., Battistich, V., Syme, S. L., & Boyce, W. T. (2002). Does elementary school alcohol, tobacco, and marijuana use increase middle school risk? Jo urnal of Adolescent Health, 30, 442-447. http://dx.doi.org/10.1016/S1054-139X(01)00416-5 Winters, K. C., Fawkes, T. Fahnhorst, A., Botzet, & August, G. (2007). A synthesis review of exemplary drug abuse prevention programs in the United States. Journal of Substance Abuse Treatment, 32, 371- 380. http://dx.doi.org/10.1016/j.jsat.2006.10.002 Young, N. K. (2000). TIP 38: Integrating substance abuse treatment and vocational services . Rockville, MD: Substance Abuse and Mental Health Services Administration. Yuan, L., Manderson, L., Tempongko, M. S., Wei, W., & Aiguo, P. (2000). The impact of educational videotapes on water contact be- havior of primary school students in the Dongting Lakes region, China. Tropical Medicine & International Health, 5, 538-544. http://dx.doi.org/10.1046/j.1365-3156.2000.00602.x List of Abbreviations NIDA—National Institute on Drug Abuse CDC—Center for Disease Control and Prevention
|