Creative Education 2012. Vol.3, No.2, 241-250 Published Online April 2012 in SciRes (http://www.SciRP.org/journal/ce) http://dx.doi.org/10.4236/ce.2012.32038 Copyright © 2012 SciRes. 241 A Preliminary Investigation into Critical Thinking Skills of Urban High School Students: Role of an IT/STEM Program Mesut Duran1, Serkan Şendağ2 1School of Education, University of Michigan-Dearborn, Dearborn, USA 2Faculty of Education, Mehmet Akif Ersoy University, Burdur, Turkey Email: mduran@umich.edu Received February 20th, 2012; revised March 18th, 2012; accepted March 31st, 2012 This paper reports the development of critical thinking of urban high school students in an IT/STEM pro- gram-using information technology (IT) within the context of science, technology, engineering, and mathematics (STEM). The study used a quasi-experimental time-series design, involving 47 initial par- ticipants in an eighteen-month intervention period. Data were collected from the Test of Everyday Rea- soning (TER), which provides an overall score on critical thinking skill (CTS) and five sub-scale scores (analysis, inference, evaluation, inductive reasoning, and deductive reasoning). Findings indicate that based on a mean score of 15.77, the average participant scored between the 16th and 19th percentiles at the beginning of the program compared to an aggregated national sample. Participants who completed the program and responded to all three time-series TER tests (14) significantly improved their critical think- ing skills throughout the program. Program completers’ overall post-program CTS test score was more homogeneous than the pre-program scores with a 20.07 mean score. In addition, data showed significant improvement in inductive reasoning skills of the program participants during the first nine months with continuing improvement in the second nine months. In contrast, data presented improved inference skills during the first nine months with significant gains during the second half of the program. The study esti- mates the relative effects of IT/STEM experiences with technology-enhanced, inquiry and design-based collaborative learning strategies on CTS of urban high school students. Keywords: Critical Thinking Skills; High School Students; Stem Education; Urban Education Introduction As the United States continues to compete in a global econ- omy that demands innovation, US educational institutions give even more emphasis on the 21st century skills, which include critical thinking and problem solving, communication, collabo- ration, and creativity and innovation (The Partnership for 21st Century Skills, n.d.). In this undertaking, there is a major chal- lenge: how are millions of students still struggling to acquire 19th-century skills in reading, writing, and mathematics sup- posed to learn 21st century skills? This is the question the au- thors of this study investigated in their recent IT/STEM project, which focused on using information technology (IT) within the context of science, technology, engineering, and mathematics (STEM) through technology-enhanced, inquiry and design-based collaborative learning experiences. In the last two decades, contemporary work environments required fundamental changes in the profiles of the work force, which basically stemmed from the rapid change and transfor- mation in the nature of information. For societies to survive in this competitive world, it is necessary to equip individuals with skills to conduct research, use and transform information, think critically and reflectively, and make higher order decisions. In addition, technological changes along with the changes in the workplace have made critical thinking abilities more important than ever. In this regard, learning experiences designed to assist students’ critical thinking focus on skills that are applicable across different domains of knowledge and the disposition to use these skills (Halpern, 1999). A Nation at Risk report of 1983 sounded an alarm regarding insufficient and ineffective attempts to foster higher order thinking skills in schools in- cluding critical thinking and problem solving (Richardson, 2003). Later in 1991, Richardson reports, the Secretary’s Com- mission on Achieving Necessary Skills regarded higher order thinking competencies as complementary for productive work- places including critical thinking, decision making, problem solving and reasoning. In short, Richardson argues, the ability to understand and use information is emphasized rather than merely possessing it. Critical Thinking Thinking is based on relating and drawing conclusions on notions and events, and involves a variety of different cognitive processes such as implicating, problem solving, examining, reflecting and criticizing. Thinking begins with a physical or psychological inconvenience stemming from lacking the solu- tion for a problem whose solution becomes the objective for an individual. Higher order thinking skills like critical thinking, and problem solving are considered necessary skills for 21st century individuals. Thus, it is necessary to examine these no- tions objectively, study on the contents of these skills, and
M. DURAN ET AL. elaborate on the ways to equip individuals with such skills. Problem solving involves cognitive, sensory, and psychomotor domains which help instructors to resort to a large variety of contexts and materials. Kalaycı (2001) argues that the most reliable way to equip individuals with problem solving skills is to integrate it with creative thinking and decision making. Critical thinking entails awareness of one’s own thinking and reflection on the thinking of the self and others (Kuhn & Dean, 2004). In this regard, metacognition is defined as thinking about thinking, a skill originating early in life when children first become aware of the mind. Metacognition does not always develop to the extent that we would like (Kuhn & Dean, 2004). The construct of metacognition can be examined within a de- velopmental framework during which the metacognitive skills become more powerful and effective gradually as they operate increasingly under individuals’ conscious control. Thus, im- proving metacognitive awareness of what to believe and how to know, and employing metacognitive control on information processing strategies are important developmental and educa- tional goals. Critical thinking involves critical implication and discussion, which has a crucial role in activating problem solving and deci- sion making processes (Chaffee, 1994). Critical thinking is a constructivist analysis process to examine what is going on in the environment. This analysis system can be used to define problems, take actions towards an aim, make decisions and conduct retrospective evaluations. In order to define, describe, measure and evaluate the critical thinking process, it is neces- sary to understand indicators of critical thinking skills (CTS). The literature has several classifications in this regard. For in- stance, the Watson-Glaser classification of CTS involves de- fining a problem, determining possible solutions and strong assumptions, drawing valid conclusions regarding the solution and evaluating these conclusions (Demirtaşlı-Çıkrıkçı, 1996; Kaya, 1997). Such skills include inference, recognition of as- sumptions, deduction, interpretation, and evaluation. In other words, critical thinking is a comprehensive thinking way in- volving analysis, synthesis, and interpretation. Critical thinking is a type of thinking that closely associates with reasoning, decision making, and problem solving (Wil- lingham, 2008). Educators recognize the importance of gaining the skill of thinking critically. However, the crucial question is how to teach critical thinking; directly for instance as a core course or indirectly such as structuring the teaching and learn- ing process in a way that those instructional activities enable the learners to employ their higher order cognitive processes. At high school level, Burkhart’s (2006) study provides evi- dences from an explicit critical thinking skill development pro- grams. However, a strong argument is in the domain of teach- ing critical thinking in an integrated way in different subject areas (Sendag & Odabaşı, 2009; Willingham, 2008). In terms of the impact of critical thinking-integrated instructional activities, the literature provides examples from solving authentic real-life problems within the context of a subject matter to focusing on authentic and ill-structured problem solving process and to constructivist strategies like problem-based learning and pro- ject-based learning (Jonassen, 1997; Jonassen, 2000; Savery & Dufy, 1996; Sendag & Odabaşı, 2009). Garrett (2009) compared teaching critical thinking directly versus within an integrated instruction, examining the relation- ship between critical thinking skills and nonperformance activi- ties in high school choral rehearsals. Nonperformance activities referred to cognitive domain skills (“remember, understand, and apply” as lower order thinking skills, “analyze, evaluate, and create” as critical thinking skills). Findings of the study indicated a significant, strong positive correlation between the amount of time spent in nonperformance activities and time spent engaged in critical thinking skills. There was no signifi- cant connection between the time spent in developing critical thinking and critical thinking skills. That is, the time spent in the context of subject matter where learners needed to employ higher order cognitive skills showed significant interaction with critical thinking, while activities that aim to directly develop critical thinking process had no significant effect on enhancing critical thinking skills. Appamaraka, Suksringarm, and Singseewo (2009) conducted a study with 9th grade students. The study results indicated that the treatment group who received instructional activities based on 5Es Learning Cycle model with metacognitve moves out- performed in critical thinking skills compared to those students receiving instructional activities based on teachers’ handbook approach within a six weeks period. In addition to structuring instructional activities intentionally for the improvement of critical thinking, sometimes the nature of the subject matter can be suitable for students to use their higher order thinking skills. A recent study by Hove (2011) highlighted that the high school students receiving inference strategies in novel reading in an English literature class had improved their critical thinking skills compared to those stu- dents who did not receive any reading strategies. Similarly, Ernst and Monroe (2006) found a significant positive effect of environment-based education on 9th and 12th grade high school students’ critical thinking skills and disposition toward critical thinking. Although it is suggested that a technologically-enriched so- cial interactive learning environment is essential to enable learner-learner, and learner-instructor interaction for developing critical thinking (Sendag & Odabaşı, 2009), critical thinking is something that can be built individually through the learners’ own higher order cognitive processes. In this context, Devi (2008) pointed out that high school students’ interaction with directed learning activities, which is also defined as “minds-on” activities by the author, in a chemistry class enhanced the stu- dents’ critical thinking skills. Other studies in improving high school students’ critical thinking seem to focus on hybrid, blended learning activities where the content of the course and technology integration go hand in hand rather than giving emphasis on the nature of the course, instructional strategies, and/or explicit critical thinking development. Bob’s (2009) study presents such a blended ap- proach. The study was conducted with 186 participants ranged from 8th to 9th grade and concluded that an online technol- ogy-based math course employing real-life problems as an in- structional activity can stimulate critical thinking and reasoning skills. Jonassen, Carr, and Yueh (1998) suggest that rather than using the computers to disseminate information, they should be used in all subject domains as tools for engaging learners in reflective critical thinking about the ideas they are studying. In this context, using information technologies within the context of classroom teaching in which the content requires the use of specific computer software like SPSS or Minitab in problem solving or using STELLA modeling software to create complex numerical models would stimulate student critical thinking. Other IT/STEM technologies such as robotics, Geographic Copyright © 2012 SciRes. 242
M. DURAN ET AL. Information System (GIS) mapping, and multimedia gaming are gradually emerging at the middle and high school levels, but literature provides only descriptive studies of how these technologies have been implemented in the classroom (Parker, Carlson, & Na’im, 2007). The research literature is even more limited in describing how IT/STEM experiences impact critical thinking skills of K-12 students. Several studies highlight that high school students in the US are not well-prepared for college and the work force (Bernas- coni, 2008; Burkhart, 2006; Pittman, 2010) since high schools failed to adapt their curricula to help students in improving a set of higher order thinking skills (Paul & Elder, 2008). Therefore, recent initiatives in support of innovation and capacity-building for STEM learning are currently focused on informal science learning outside formal school settings. The National Science Foundation (NSF) is the leading institution on this front pro- viding research funding through its Informal Science Education (ISE) and Innovative Technology Experiences for Students and Teachers (ITEST) programs. The underlying assumption of these programs is the possibility of supporting students to gain the 21st Century skills including critical thinking through ex- tracurricular activities. Research in this area is at too early a stage to document the impact of these programs on students’ critical thinking development. In sum, review of the literature on the development of high school student’s critical thinking skills highlights the following arguments: Courses designed to explicitly teach critical thinking skills may enhance that ability. Instructional activities such as problem-based learning, in- quiry-based learning within the context of subject matter teaching may improve critical thinking. Social interaction has a key role for building critical think- ing; the ability of critical thinking is an individual outcome. The nature of the course content could be an important fac- tor in improving critical thinking. A technology-enriched learning environment and technol- ogy-integrated instruction may help students in improving their ability to think critically. Extracurricular activities may play a role in improving cri- tical thinking skills. In accordance with the existing literature and in response to the growing need for understanding high school students’ criti- cal thinking skill development, this study aimed at studying the development of critical thinking skills of urban high school students in an IT/STEM program. The following specific re- search questions were examined in the study: 1) What were the initial CTS profiles of the study partici- pants? 2) Were there any differences in the profiles of the partici- pants who completed the program and who did not? 3) Was there any significant increase in the CTS test scores of participants throughout the program? 4) Was there any significant increase in the subscale (analy- sis, inference, evaluation, inductive reasoning, deductive rea- soning) test scores of the CTS test during program participa- tion? In the following sections, we first describe the IT/STEM program examined in this study and discuss the conceptual framework that provided the basis for the project design. We then describe the study that used a times-series design to deter- mine the changes in participating students’ CTS during the life of the program. In the subsequent section, we present the study findings accompanied with a discussion of those findings. In the conclusion section, we discuss ways in which IT/STEM experiences with technology-enhanced, inquiry and design- based collaborative learning strategies might have effects on CTS of high school students. Conceptual Framework The present study reports from the Fostering Interest in In- formation Technology (FI3T) project funded by the National Science Foundation (NSF) through Innovative Technology Experiences for Students and Teachers (ITEST) program. The FI3T project is designed to increase the opportunities for un- derrepresented and underserved high school students, particu- larly those from disadvantaged urban communities in South- eastern Michigan, to learn, experience, and more importantly use IT within the context of STEM. The FI3T project calls for the investment and robust participation of post-secondary col- leges and schools, area school districts, and the business, indus- try, and government sectors. The “Community of Designers” approach introduced by Mishra, Koehler, and Zhao (2006) pro- vides a framework for a collaborative partnership among a range of participants involved in the program. Community of Designers As Mishra et al. (2006) describe, the Community of Design- ers is an environment in which groups of individuals work col- laboratively to design and develop solutions to authentic prob- lems. Mishra et al. highlight the essence of this approach with four key words: community, design, products/solution, and au- thentic problems. Mishra et al. (2006) describe “community” as the social ar- rangement of the approach. Within the context of social con- structivism the design community lends itself to sustained in- quiry and revision of ideas. “Design” specifies the activity di- mension of the approach. Building upon ideas grounded in situated cognition theory, learning is contextualized in the process of doing-solving an authentic problem of practice. De- sign-based activities provide the rich context for learning, sus- tained inquiry, and revision and are well-suited to develop the deep understanding needed to apply knowledge in the complex real-world domains. Whereas “products/solution” stresses the goal-oriented psychological dimension, “authentic problems” addresses the motivational challenge, which become the driving force behind the work of the community. Authentic problems that project participants face and need to work on provide the connection between what they learn and what they actually do. Mishra et al. (2006) highlight that learners have to actively engage in practices of design, inquiry, and research in collabo- rative groups to design tangible, meaningful artifacts as end products of the learning process. The actual process-by-design is the anchor around which learning happens. This evolving artifact is also the test of the viability of individual and collec- tive understandings as participants test their and others concep- tions and ideas of the project. Mishra et al. also indicate that implementing a community of designers breaks down into four stages that each design team experiences over its lifecycle: identifying participants and problems, forming communities, providing leadership and support, and working on authentic problems. Copyright © 2012 SciRes. 243
M. DURAN ET AL. One would argue that in many ways community of designers framework parallels with the principals of “community of prac- tice” idea where Lave and Wegner (1991) describe learning through social engagement in which members share under- standings regarding what they are conducting and what that means in their daily life and for their community (cited in Parker, Malyn-Smith, Reynolds-Alpert, & Bredin, 2010). Fur- ther referencing Wenger, the authors highlight that “these com- munities foster mutual engagement among the members, while they work on a joint enterprise using shared repertories of ter- minology and skills” (p. 190). The FI3T Project Consistent with the aforementioned discussions, the FI3T project calls for the collaborative engagement of high school students, K-12 STEM teachers, Undergraduate/Graduate Stu- dent Assistants (U/GSAs), and STEM content experts from university, business, industry, and government sectors to create high-quality learning projects, strategies and curriculum models for use in after school, weekend, and summer settings through hands-on, inquiry-based activities with a strong emphasis on non-traditional approaches to learning and understanding. The project also distributes online learning activities using the pro- ject’s Web site and social media sites, and thus establishes a culture of collaboration and discourse that extends participation outside the confines of the formal scheduled events. The FI3T project concentrates on all four areas of STEM cre- ating four project-based design teams to address IT in science, engineering, technology, and mathematics. Each design team normally includes 10 high school students, one STEM area high school teacher from the participating school district, one U/GSA and one postsecondary STEM content expert from par- ticipating higher education colleges and schools. Each design team, therefore, consists of 13 collaborating members. One specialized member of the project leadership team leads each team. The science team concentrates on three different but related applications of IT in the sciences; measurement, modeling, and mapping. Participating students’ learning experiences for IT/ Science include making location measurements using GPS and integrating the measurements in a GIS system, using tempera- ture and light sensors in the sciences, and creating mathematical models using STELLA that incorporate measured quantities and make predictions. The technology team focuses on technological tools and lan- guages for designing and developing Web applications such as Web-based games and chat-rooms. Participants gain experi- ences with the basics of visual programming, familiarize them- selves with integrated development environments such as Vis- ual Studio and/or Alice, and practice designing and developing games. The engineering team emphasizes the basics of robotics and its applications as related to IT, including modeling robots, programming robots, and integrating robots into an application environment such as a manufacturing system or a medical ap- plication. Engineering related learning experiences involve using robotics simulation software packages such as IGRIP and ROBCAD, learning the basics of robotics such as modeling and programming robots, and learning integration of robots in an application environment such as manufacturing system or sur- gery operating room. The mathematics team focuses on statistical science with consideration of the two-sample comparison problem, the sim- ple regression/correlation problem, and the simple analysis of covariance problem taking examples and assignments from public health science, environmental science, and manufactur- ing reliability. Participants use Minitab to create comparative displays and regression displays and perform appropriate ana- lyses to test for and estimate effect sizes. The work of FI3T is accomplished over an eighteen-month cycle through summer and school-year activities. The first nine months of the program is primarily a time for capacity-building among students to increase IT knowledge and skills in STEM- related fields. The second part focuses on facilitating student activities in which they engage in designing inquiry-based au- thentic projects of science-fair quality using what was learned in the capacity-building phase of the program. The following sections describe the Capacity Building and Design activities in detail. Capacity Building. During this initial phase of the project, teachers, higher education faculty, graduate students, and busi- ness partners in their assigned STEM area collaborate to facili- tate school-year IT/STEM workshops (September to May) for participating high school students. Offered in two different levels, each STEM area offers a total of six different three-hour content-specific IT/STEM workshops on the campus of the participating postsecondary institution during weekend meet- ings. During the fall semester, each STEM area offers two Level 1 workshops to all participating students. These workshops con- sist of brief presentations followed by hands-on activities to provide students the opportunity to learn about IT toolsets within the context of STEM. A second purpose of the Level 1 workshops is to allow participating students to identify specific areas of interest within STEM fields. During this period, STEM area teachers, faculty members, and the project leadership team observe and survey students for their interest in specific STEM subject areas and assist them to narrow down their interest into two specific STEM related fields. During the winter semester, a set of small group in-depth Level 2 IT/STEM workshops are offered to participating stu- dents in their identified two STEM areas of interest. Each STEM area offers four three-hour content-specific IT work- shops, allowing each student to participate in a total of eight workshops related to their identified two STEM areas. Through- out these workshops, participating students have the opportu- nity to learn advanced use of IT toolsets. Level 2 workshops also allow students to narrow down their interest into one spe- cific STEM area and help them to join in a specific IT/STEM design team. Design Year. During the second phase of the project, par- ticipating STEM teachers continue to collaborate with higher education faculty, graduate students, and business partners to facilitate IT-supported STEM project activities for high school students assigned to their IT/STEM design team. The design year consists of a two-week summer camp and a series of site-based sessions for each individual design team. The over- arching task of each design team in this year is to develop in- quiry-based authentic projects that are of at least science fair quality using one or more content-specific IT tools explored during the previous capacity building year and stimulating ideas/experiences gained during the summer camp. Copyright © 2012 SciRes. 244
M. DURAN ET AL. Copyright © 2012 SciRes. 245 Two-week summer camps for each design team (Science, Technology, Engineering, Mathematics) take place at the be- ginning of the design year in which students and teachers meet and observe the work of scientists and professionals in IT/ STEM fields. Collaborating business, industry, government, and university sectors host these sessions. Summer camp also allows students to develop inquiry-based authentic project pro- posals and share their ideas with other members of their design team. Summer camp readies the students for the project devel- opment stage that occurs during the subsequent collaborating school year. shrinking population of students, financial instability, school closings, and a low high school graduation rate. The study was announced to all students in four different high schools of the participating school district at the beginning of the 2009-2010 school year and volunteers were asked to participate. These four schools were targeted because the FI3T program had participating teachers from these schools creating relatively easy access to the student population. Based on STEM area teacher recommendations, a total of 47 students were invited to participate in the program including 1 freshman, 26 sophomores, 19 juniors, and 1 senior. The Test of Everyday Reasoning (TER) test was adminis- tered to all of these students at the beginning of the program. Of those, 18 students completed the program, attending all project activities with almost perfect attendance. The FI3T project pro- vided these students a certification of program completion. For the purpose of this study, these students were identified as Cer- tificated Participants (CP) hereafter and those others as Non- Certificated Participants (NCP). Out of the 18 CP, 14 com- pleted all -pre, -mid, and -post program TER tests. Critical framework for the design year activities is built around the “cyclic inquiry” model’s 5 major steps (Bruce, 2003). Ask, Investigate, Create, Discuss, and Reflect. During the summer camp program, each design team facilitates col- laborative learning experiences where students learn how to design and conduct inquiry-based authentic IT/STEM projects, more specifically how to Ask and how to Investigate. Starting with the new school year, students engage in the remaining three stages of their project design (Create, Discuss, Reflect) iteratively. During these stages, each design team facilitates multiple site-based face-to-face sessions where students work on their projects and conduct ongoing discussions about design activities. The design year and student/teacher participation in the FI3T program ends with a showcase meeting where students present their projects and discuss their experiences with the community. Instrumentation In order to gather data related to participants’ CTS, the TER was used. The TER is a member of the California Critical Think- ing Skills Test (CCTST) Family of Critical Thinking Skills Tests offered by Insight Assessment (insightassessment. com). The TER reports an overall score on critical thinking skill (total score) and 5 sub-scale scores (analysis, inference, evaluation, inductive reasoning, and deductive reasoning) along with demographic info and descriptive statistics. The TER is de- signed for test takers in secondary school or the first two years of post secondary education, and for adults of all ages in the general population. The TER is a 35 item multiple choice test that is administered in 50 minutes. The Flesch-Kincaid Read- ability Level is 6th grade. No specialized content knowledge is required. Test questions engage the test-taker’s reasoning skills using familiar topics and contexts. Different questions progres- sively invite test-takers to analyze or to interpret information presented in text, charts, or images; to draw accurate and war- ranted inferences; to evaluate inferences and explain why they represent strong reasoning or weak reasoning; or to explain why a given evaluation of an inference is strong or weak. Method Research Design The study applied a quasi-experimental times-series research design. Fraenkel and Wallen (2006) describe that quasi-ex- perimental times-series research design is similar to one group pre-test and post-test research design but requires repeated measurements or observations over a period of time. Figure 1 illustrates the conceptual scheme of the study with an emphasis on the treatment procedures and measurements. Participants and Settin gs The school district involved in this study was a major urban school district located in Southeastern Michigan. It is the larg- est school district in the State serving nearly 66,000 students (mostly African American) throughout a major city area. The district has a long history of challenging issues including Data Collection and Analysis The data was collected through TER test, which also included CTS-T: Critical Thinking Skills Test Workshops (IT/STEM and Web 2.0) Communities of Designers (Community, Design, Products/Solution, and Authentic Problems) Cyclic Inquiry (IT/STEM Projects) Capacity Building Yea (9 months) Desi n Yea (9 months) CTS-T MID CTS-T POST CTS-T PRE Figure 1. Structure of the research design.
M. DURAN ET AL. questions about demographic info; gender, ethnicity, grade level. The TER was employed three times—beginning, middle, and end of the program as illustrated in Figure 1. The data was analyzed using SPSS 15.0 software. Descriptive statistics such as percentages, standard deviations (SD), and mean were used to analyze the initial profiles of the CP/NCP participants. The one way repeated measure ANOVA test was used to analyze the CTS progress of the CP participants throughout the program. The critical value (α) was set at .05. Limitations The present study has some limitations that need to be taken into account when considering the study and its contributions. The participants in this study were self selected based on their existing interest in IT/STEM fields. These participants may not be representative of urban high school students. A total of 47 participants in the study is characteristic of what might be con- sidered a pilot in preparation for a more extended study. Findings Initial CTS Profile of the Participants Data collected from the pre TER test were analyzed to un- derstand the initial CTS profile of the participants. As shown in Table 1, a total of 47 participants completed the pretest. Most of the participants were female (64%), 36% were male. The majority of the participants (83%) were identified themselves as African-American. The remaining participants checked either “Other” (8.5%) or didn’t provide information for the related section on the test (8.5%). Most of the participants started the program at their sophomore year (26) or junior year (19), a freshmen and a senior also included in the initial group. Based on overall mean score of 15.77, the average project participant at the beginning of the program scored between the 16th and 19th percentiles compared to an aggregated sample of aggregated sample for Total (see Figure 2, Insight Assessment, 2010a). In a later report, Insight Assessment (2010b) reported that based on their aggregated sample the high school national TER test mean score was 18.5. This indicates that the initial CTS profile of the project participants was lower than the ag- gregated national mean score comparing overall scores. Data also presented overall mean score differences between CP and NCP groups in which CP’s overall mean (18.17) was larger than NCP’s overall mean score (14.28). In order to un- derstand if this difference was statistically significant the au- thors conducted t-t test between initial total CTS test scores of CP and NCP. The data showed normal distribution while the homogeneity test of Levene showed that the variance of both groups were heterogeneous. Thus, the authors proceed with non-parametric counterpart of independent sample t-test, Mann- Whitney U. As shown in Table 2, the test results showed no significant difference between CP and NCP in relation to their total CTS test scores. In other words, the CP and NCP groups were similar at beginning of the program in terms of their over- all critical thinking skills. Table 1. Demographics and Initial CTS scores of CP and NCP. Gender Ethnicity Female Male African-AmericanOthers Not Provided CTS N % N % N % N % N % N % MeanSD CP 18 38 9 30 9 53 16 41 0 0 2 50 18.17 6.12 NCP 29 62 21 70 8 47 23 59 4 100 2 50 14.28 4.19 Overall 47 100 30 64 17 36 39 83 4 8.5 4 8.5 15.77 5.31 Figure 2. Comparison to high school students aggregated sample from insight assessment aggregated sample for total. Copyright © 2012 SciRes. 246
M. DURAN ET AL. In sum, the project participants started the program with similar level of critical thinking skills. Compared to an aggre- gated national data collected by Insight Assessment participat- ing high school students started the program with relatively lower overall CTS score. Changes in Critical Thinking Skills Data collected through -pre and -post TER tests were ana- lyzed to understand if there was a significant difference in CP’s total CTS scores throughout their participation in program. In order to employ repeated measure ANOVA test, the Shapiro- Wilk normality test was conducted to test the normality of CTS scores. As shown in Table 3, CP’s CTS scores showed no sig- nificant difference from standard normal distribution, allowing the use of one way repeated measure ANOVA test between pre and post CTS scores. As shown in Table 4, the results of the one-way repeated measure ANOVA test indicated a significant difference be- tween CP’s pre and post program total CTS scores (F(1 - 14) = 7.37; p < .05, partial η2 = .362). In other words, there was sig- nificant increase in the CP CTS scores throughout the program duration. In addition, standard deviation for the CP’s post-pro- gram CTS scores was considerably smaller than the standard deviation of the pre-program CTS scores. The results indicate that the post-program CTS test scores of the CP were more homogenous than the pre-program CTS scores. The TER test used in this study has five sub-scales; Analysis, Inference, Evaluation, Inductive Reasoning and Deductive Rea- soning. The one-way repeated measure ANOVA test showed significant increase in the level of CP’s inference and induction sub-scale scores. Test results showed no significant difference for deduction, analysis, and evaluation scores whereas total post-test scores for each of these sub-scales were greater than the pre-test scores, indicating a degree of increase. Table 5 shows the test results for Inference sub-scale. As shown in the table, CP significantly improved their inference scores throughout the program (F(2 - 12) = 4.135, p < .05, partial η2 = .408). As Table 6 presents, CP’s improvement of inference sub- scale was statistically significant between pre and post (Mean Dif = 1.929, p < .05), and mid and post (Mean Dif = 1.357; p < .05). Improvement between pre and mid test scores of infer- ence was not significantly different. Figure 3 illustrates the progress from pre to post program measures. Similarly, repeated measures ANOVA test showed a signifi- cant increase for Inductive Reasoning sub-scale scores of CP. As shown in Table 7, CP’s inductive reasoning test scores sig- nificantly increased from pre to post program (F(2 - 12) = 4.448, p < .05, partial η2 = .426). Table 8 shows the comparisons between pre, mid, and post- test mean scores related to CP’s inductive reasoning scores. As Table 8 illustrates, increase in inductive reasoning scores was significant between pre and post measures (Mean Dif = 1.786, p < .05), as well as pre and mid measures (Mean Dif = 1.214, p < .05). Improvement between mid and post test scores of induc- tive reasoning was not significantly different. Figure 4 depicts the progress from pre to post program measures. In sum, CP’s CTS test scores significantly improved during their participation in the program with post-program CTS test scores becoming more homogenous compared to the pre-pro- gram CTS scores. In addition, data presented significant im- provement in inductive reasoning skills of the PC during the first nine months of the program with continuing improvement in the second nine months. In contrast, data presented improved inference skills during the first nine months with significant gains during the second half of the program. Discussion and Conclusion The present study investigated the development of critical thinking of urban high school students. The new understanding gained from the study highlights possible impact of IT/STEM Table 2. Results of mann-whitney U test related to initial CTS scores. Groups N Mean RankSum of Ranks U Sig CP 18 28.83 519.00 174.00.06 NCP 29 21.00 609.00 Table 3. Test of normality for CTS Scores throughout FI3T program. CTS Tests of CP Groups Statistic Df Sig Pre-program CTS Scores 915 14 .189 Mid-Program CTS Scores 954 14 .625 Post-Program CTS Scores 956 14 .665 Table 4. Results of one-way repeated measure ANOVA test based on pre and post program CTS scores. CTS Scores Mean Std. Deviation N F df1 df2 Sig Partial Eta Squared Pre-program 17.07 5.59 14 7.37 1 14 .018 .362 Post-Program 20.07 3.73 14 Table 5. Results of one-way repeated measure ANOVA test based on inference scores. Inference Scores Mean Std. Deviation N F df1 df2 Sig Partial Eta Squared Pre-program 7.21 2.32 14 4.135 2 12 .043 .408 Mid-program 7.78 2.86 14 Post-program 9.14 2.53 14 Copyright © 2012 SciRes. 247
M. DURAN ET AL. Table 6. Pairwise comparisons for inference scores. Inference Mean Scores Pre Mid Post Pre (Mean = 7.21) - .571 1.929* Mid (Mean = 7.78) - - 1.357* Post (Mean = 9.14) - - - Figure 3. Progress of CP’s inference test scores. Figure 4. Progress of CP’s induction test scores. experiences on critical thinking development through technol- ogy-enhanced, inquiry and design-based collaborative teaching and learning strategies. The first objective of this study was to investigate the initial CTS profiles of the study participants who were coming from a major urban school district located in Southeastern Michigan. Majority of the participants (83%) were African-American and most of them were female (64%). Most of the students were at sophomores or junior level at the beginning of the program. Findings indicate that compared to an aggregated national data collected by Insight Assessment (Insight Assessment, 2010a; 2010b) participating high school students joined the program with relatively lower overall CTS test results (mean = 15.77), scoring between the 16th and 19th percentiles on the pre TER test. Extensive literature review failed to identify if the initial profile of the project participants was similar or different than typical high school population from different urban schools or high school population in general. Further studies in this area seem necessary. The authors’ second objective was to investigate if there were any differences in the CTS profiles of the participants who completed the program (CP) and who did not (NPC). Findings presented overall mean score differences between CP and NCP groups where CP’s overall mean (18.17) was larger than NCP’s overall mean score (14.28). However, further data analysis revealed that the difference between these two groups was not statistically significant. In other words, the CTS profiles of CP and NCP were similar at the beginning of the program. The third objective was to investigate if there was any sig- nificant increase in the CTS of project participants throughout the program. Findings indicate that CP significantly improved their CTS throughout their participation in the program. In ad- dition, participants’ post-program CTS scores were more ho- mogeneous than pre-program CTS scores. It appears that the IT/STEM experiences gained through the program significantly impacted program completers’ critical thinking skills. Without an experimental study with a control group, the program impact of this project on critical thinking skill development cannot be strongly argued. However, the literature provides implications for the study design and its components regarding development of critical thinking of high school students. The study design provides a unique collaborative environ- ment for high school students in which they work with a group of experts from K-12, university, and industry to design and develop solutions to authentic problems. Another main feature of the program is its emphasis on IT knowledge and skills within the context of STEM-related fields. In addition, over an extended period of time (18 months), the project provides par- ticipants access to year-round enrichment experiences through technology-enhanced, hands-on, inquiry and design-based ac- tivities around authentic projects with a strong emphasis on non-traditional approaches to learning and understanding. The program structure is closely aligned with the position of teaching critical thinking in an integrated way in different sub- ject areas, as argued by Sendag & Odabaşı (2009) and Willing- ham (2008). The study findings support the findings of previ- ous research in which focusing on authentic real-life problems within the context of a content area or focusing on problem solv- ing process through problem-based and project-based learning improves students’ critical thinking skills (see Appamaraka et al., 2009; Ernst & Monroe, 2006; Hove, 2011; Jonassen, 1997; Table 7. Results of one-way repeated measure ANOVA test based on inductive reasoning scores. Inference Scores Mean Std. Deviation N F df1 df2 Sig Partial Eta Squared Pre-program 7.857 2.597 14 4.448 2 12 .036 .426 Mid-program 9.071 2.813 Post-program 9.642 1.823 Copyright © 2012 SciRes. 248
M. DURAN ET AL. Table 8. Pairwise comparisons for inductive reasoning scores. Inference Mean Scores Pre Mid Post Pre (Mean = 7.857) - 1.214* 1.786* Mid (Mean = 9.071) - - .571 Post (Mean = 9.642) - - - Jonassen, 2000; Savery & Dufy, 1996; Sendag & Odabaşı, 2009). The final objective of the authors was to investigate if there was any significant increase in the five sub-scale scores of the CTS test used in the study, TER. The findings showed no sig- nificant difference for deduction, analysis, and evaluation scores of program completers from beginning to the end of the program. In contrast, there was significant improvement in inference and inductive reasoning skills. Inference includes the skills of “querying evidence”, “con- jecturing alternatives”, and “drawing conclusions” Facione (2011). In the context of the TER test, “inference skills are used to draw conclusions based on reasons and evidence” (Insight Assessment, n.d.). Findings indicate that the project participants significantly improved their inference skills throughout the program with significant gain recorded in the second phase (Design Year) of the project. During the design phase, the project facilitated IT-supported STEM project activities in which participating students col- laborated with K-12 STEM teachers, higher education faculty, graduate students, and business partners in their IT/STEM de- sign team. The overarching task of each design team in this year was to develop inquiry-based authentic projects that are of at least science fair quality using one or more content-specific IT tools explored during the previous capacity building year and stimulating ideas/experiences gained during the summer camp. Critical framework for the design year activities was built around the “cyclic inquiry” model’s 5 major steps (Bruce, 2003). Ask, Investigate, Create, Discuss and Reflect. It appears that inquiry and design-based collaborative learning activities had significant impact on project participants’ inference skills. The finding of this study is similar to previous research found in the literature. Citing several studies (i.e., Chiappetta & Russel, 1982; Saunders & Shepardson, 1987; Haury, 1993) Abdelraheem & Asan (2006) reports that inquiry-based learn- ing enhances students’ academic achievement, especially prob- lem solving skills, ability to explain data sets, thinking critically, and understanding of concepts in science teaching. Abdelra- heem and Asan further argue that by using processes like col- lecting and classifying data, stating hypotheses, making predic- tions, and interpreting results of experiments, students infer knowledge from the information provided. The authors add that this is essential skill because such a knowledge base is not di- rectly available in discovery situations and knowledge is to be inferred (Shute, Glaser, & Raghavan, 1989). Within the context of TER test, inductive reasoning is de- scribed as “drawing warranted probabilistic inferences regard- ing what is most likely true or most likely not true, given the information and the context at hand” (Insight Assessment, n.d). The study findings highlight that in contrast to the gain in in- ference skills during the second half of the program, significant improvement for inductive reasoning skills was recorded in the first nine months during Capacity Building year. During this initial phase of the project, high school students participated in school-year IT/STEM workshops offered jointly by K-12 STEM teachers and university faculty in their respective STEM area. Taught in two different levels, each STEM area offered a total of six different three-hour content-specific IT/STEM work- shops on the campus of the participating postsecondary institu- tion during weekend meetings. It appears that learning experiences related to IT/STEM technologies impacted participants’ inductive reasoning skills significantly. In the context of this study, specific IT/STEM technologies included GPS and GIS systems, STELLA model- ing software, Alice 3D programming for game design, robotics simulation software packages such as IGRIP and ROBCAD, and Minitab statistical software. Willingham (2008) argues that having adequate knowledge is essential for individuals to think critically. Heit (2000) describes inductive reasoning as consid- ered prediction of the unknown situations using known. Heit argues that problem solving, social interaction, and motor con- trol are important activities facilitating such a cognitive process. IT/STEM workshops provided during the Capacity Building year aimed at utilizing knowledge acquisition with “hands-on” learning experiences. Thus the instructional strategies used during the workshops along with the nature of the subject mat- ter taught (IT/STEM toolsets) may have triggered participants’ inductive reasoning skills. A great degree of social interaction among project participants was also evident during the first phase of the program through whole-group seminar meetings where students, teachers, content experts, project leaders, and parents collectively focused on identifying students’ specific areas of interest in STEM fields and planning for the design year. In conclusion, IT/STEM experiences supported through technology-enhanced, inquiry and design-based collaborative learning strategies seem to have significant impact on devel- opment of critical thinking of urban high school students. The improvement is specifically evident in inference and inductive reasoning areas of critical thinking. Previous research provides implications for the study design and its components in devel- oping critical thinking of high school students. Without an ex- perimental study with a control group, the program impact of this study should be considered as preliminary. As a next step, the authors of this study recommend a more rigorous study to fully investigate the impact of the design model and instruc- tional strategies implemented in the program. Acknowledgements The study reported in this article is based on a project being funded by the National Science Foundation (NSF) through Innovative Technology Experiences for Students and Teachers (ITEST) program. Co-author, Dr. Serkan Şendağ’s involvement in the study is supported by the Scientific and Technological Research Council of Turkey (TUBITAK). REFERENCES Abdelraheem, A., & Asan, A. (2006). The effectiveness of inquiry- based technology enhanced collaborative learning environment. In- ternational Journal of Technology in Teaching and Learning, 2, 65-87. Appamaraka, S., Suksringarm, P., & Singseewo, A. (2009). Effects of Copyright © 2012 SciRes. 249
M. DURAN ET AL. learning environmental education using the 5Es-learning cycle with the metacognitive moves and teacher’s handbook approach on learn- ing achievement, integrated science process skills and critical think- ing of high school (grade 9) students. Pakistan Journal of Social Science, 6, 287-291. Bernasconi, L. (2008). The jewels of ERWC instruction. California English, 14, 16-19. Bob, B. (2009). A problem based on-line mathematics course and its affect on critical thinking, reasoning skills and academic achieve- ment. Proceedings of the 31st Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathemat- ics Education. Atlanta, GA: Georgia State University. Bruce, B. C. (2003). Literacy in the information age: Inquiries into meaning making with new technologies. Newark, DE: International Reading Association. Burkhart, L. (2006). Thinking critically about critical thinking: Devel- oping thinking skills among high school students. Unpublished PhD dissertation, Claremont, CA: The Claremont Graduate University. Chaffee, J. (1994). Thinking critically. Boston: Houghton Mifflin. Chiappetta, E. L., & Russell, J. M. (1982). The relationship among logical thinking, problem solving instruction, and knowledge and ap- plication of earth science subject matter. Science Education, 66, 85-93. doi:10.1002/sce.3730660111 Demirtaşlı-Çıkrıkçı, N. (1996). Critical thinking: A scale and a study. 3. Ulusal psikolojik danişma ve rehberlik kongresi bildiri kitapçığı (pp. 208-216), Adana. Devi, P. K. (2008). D.A.R.TS using work sheets for developing process skills and critical thinking with pencil and paper tasks an experiment study in chemistry senior high school at “colligative properties con- cept”. VTE Research and Networking, 1. URL (last checked 07 March 2012). http://ojs.voctech.org/index.php/seavern/article/view/128/121. Ernst, J. A., & Monroe, M. (2006). The effects of environment-based education on students’ critical thinking skills and disposition toward critical thinking. Environmental Education Research, 12, 429-443. doi:10.1080/13504620600942998 Fraenkel, J. R., & Wallen, N. E. (2006). How to design and evaluate research in education (6th ed.). New York, NY: McGraw-Hill. Facione (2011). Critical thinking: What it is and why it counts. URL (last checked 07 March 2012). http://www.insightassessment.com/content/download/1176/7580/file/ what%26why2010.pdf Garrett, M. L. (2009). An examination of critical thinking skills in the high school choral rehearsal. Unpublished PhD dissertation, Talla- hassee, FL: The Florida State University College of Music. Halpern, D. F. (1999). Teaching for critical thinking: Helping college students develop the skills and dispositions of a critical thinker. New Directions for Teaching and Learning, 80, 69-74. doi:10.1002/tl.8005 Haury, D.L. (1993). Teaching science through inquiry. Eric Document Reproduction Service No. ED 359048. Heit, E. (2000). Properties of inductive reasoning. Psychonomic Bulle- tin & Review 2000, 7, 569-592. doi:10.3758/BF03212996 Hove, G. (2011). Developing critical thinking skills in the high school English classroom. Unpublished Master dissertation, Menominee, WI: The Graduate School University of Wisconsin-Stout. Insight Assessment (2010a). Report results, TER, The Test of Everyday Reasoning, Univ MI Dearborn Education. Dearborn, MI: Author. Insight Assessment (2010b). Cap score results. Dearborn, MI: Univer- sity of Michigan-Dearborn. Insight Assessment. (n.d.). Scales of the TER. URL (last checked 07 March 2012) http://www.insightassessment.com/Products/Critical-Thinking-Skills-T ests/Test-of-Everyday-Reasoning-TER/Scales-of-the-TER. Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem solving learning outcomes. Educational Technology Research and Development, 45, 65-94. doi:10.1007/BF02299613 Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 48, 63-85. doi:10.1007/BF02300500 Jonassen, D. H., Carr, C., & Yeuh, H. (1998). Computers as mindtools for engaging learners in critical thinking. TechTrends, 43, 24-32. doi:10.1007/BF02818172 Kalaycı, N. (2001). Problem solving and applications in social studies. Ankara: Gazi Kitapevi. Kaya, H. (1997). Critical thinking skills of the students of university. Unpublished PhD dissertation. Istanbul: Istanbul University Graduate School of Health Sciences. Kuhn, D., & Dean, D. (2004). Metacognition: A bridge between cogni- tive psychology and educational practice. Theory into Practice, 43, 268-273. doi:10.1207/s15430421tip4304_4 Lavy, J., & Wegner, E. (1991). Situated learning: Legitimate peripheral participation. In C. Parker, J. Malyn-Smith, S. Reynolds-Alpert, & S. Bredin (2010). The innovative technology experiences for students and teachers (ITEST) program: Teachers developing the next genera- tion of STEM talent. Journal of Technology and Teacher Education, 18, 187-201. Mishra, P., Koehler, M. J., & Zhao, Y. (2006). Communities of design- ers: A brief history and introduction. In P. Mishra, M. J. Koehler, & Y. Zhao (Eds.), Faculty development by design: Integrating tech- nology in higher education (pp. 1-26). Charlotte, NC: Information Age Publishing. Parker, C. E., Carlson, B., & Na’im, A. (2007). Building a framework for researching teacher change in ITEST projects. URL (last checked 07 March 2012) http://itestlrc.edc.org/sites/itestlrc.edc.org/files/Researching_Teacher_C hange_%20in_ITEST%20Projects.pdf. Parker, C., Malyn-Smith, J., Reynolds-Alpert, S. & Bredin, S. (2010). The innovative technology experiences for students and teachers (ITEST) program: Teachers developing the next generation of STEM talent. Journal of Technology and Teacher Education, 18, 187-201. Paul, R., & Elder, L. (2008). Critical thinking: The nuts and bolts of education. Optometric Education, 33, 88-91. Pittman, K. (2010). College and career readiness. School Administrator, 67, 10-14. Saunders, W. L., & Shepardson, D. (1987). A comparison of concrete and formal science instruction upon science achievement and rea- soning ability of sixth grade students. Journal of Research in Science Teaching, 24, 39-51. doi:10.1002/tea.3660240105 Savery, J., & Duffy, T. M. (1996). Problem based learning: An instruc- tional model and its constructivist framework. In B. G. Wilson (Eds.), Designing constructivist learning environments. Englewood Cliffs, NJ: Educational Technology Publications. Sendag, & Odabaşı, H. F. (2009). Effects of an online problem based learning course on content knowledge acquisition and critical think- ing skills. Computers & Education, 53, 132-142. Shute, V. J., Glaser, R., & Raghavan, K. (1989). Inference and discov- ery in an exploratory laboratory. In P. L. Ackerman, R. J. Sternberg, & R. Glaser (Eds.), Learning and individual differences: Advances in theory and research. New York: W. H. Freeman & Company. Richardson, V. (2003). Constructivist pedagogy. Teachers College Record, 105, 1623-1640. doi:10.1046/j.1467-9620.2003.00303.x The Partnership for 21st Century Skills (n.d.). A framework for 21st century learning. URL (last checked 07 March 2012). http://www.p21.org Willingham, D. T. (2008). Critical thinking: Why is it so hard to teach? Arts Education Policy Review, 109, 21-32. doi:10.3200/AEPR.109.4.21-32. Copyright © 2012 SciRes. 250
|