ABSTRACT

The way in which students are being educated is evolving; the traditional teaching paradigm, where formal and non-formal education are treated as a mutually exclusive, must be changed and turned into a new learning paradigm where formal and non-formal education are treated as one in order to achieve a meaningful and relevant learning. In Mexico, it is clear that educational level of students is barely improving in elementary school students, considering basic topics on Math, Science, and Spanish. There is a need to integrate more efficient mechanisms, like serious video games, in non-formal environment for students and increase learning opportunities beyond the classroom. Therefore, it is envisaged as a possibility, the large scale production of video games to address this problematic. But, a reason why it is not often carried out the large scale development of digital resources is the lack of well-defined development processes and documented experiences. This research will provide solution to the lack of well-defined development processes for large scale production of serious video games, presenting a proposal for a development process with fundamentals in software engineering. The overall findings of the research show that there is an improvement in the knowledge acquisition of the students.

Keywords:Game Design Process, Serious Video Games, Electronic Learning

1. Introduction

The way in which students are being educated is evolving; modern individuals require an ongoing learning due to the technological changes (UNESCO, 2009). The profile of the modern student has changed with the “information age”. Individuals evolve in an environment strongly influenced by the presence of diverse artificial and technological elements (McGinnis, Bustard, Black, & Charles, 2008).

The traditional teaching paradigm, where formal and non-formal education are treated as a mutually exclusive, must be changed and turned into a new learning paradigm where formal and non-formal education are treated as one in order to achieve a meaningful and relevant learning.

In Mexico, general basic education is oriented to contribute in achieving better levels of intelligence, feelings and personality in students. Aimed to complement these efforts, several strategies has been implemented to incorporate technology (equipment, educational software, Web 2.0 tools, among others) within both, public and private schools from elementary to higher education (SEP BCS, 2009).

However, and despite the efforts, it is clear—based on the ENLACE (by its acronym in Spanish, National Assessment of Academic Achievement in Schools) assessment results—that educational level of students is barely improving in elementary school students, considering basic topics on Math, Science, and Spanish (Secretaría de Educación Pública, 2012).

There is a need to integrate more efficient mechanisms in non-formal environment for students and increase learning opportunities beyond the classroom. Mechanisms with great potential are video games. One of the aspects that have to be solved is to produce content that “provide immersion, motivation, fun and high level of engagement” (Ibrahim & Jaafar, 2009).

Therefore, it is envisaged as a possibility, the large scale production of video games to address both production and mix of strategies (Barajas Saavedra, 2009).

This research is focused on large-scale production of games, the resolution of problems related to such production and present a solution to the lack of production processes for large-scale games. Then, in Section 2, the problematic of production of games is presented. Later, in Section 3, a state of the art studio for the characterization of the games (which establishes the foundation for the design of the process) is presented. In Section 4 some processes proposed by other authors are reviewed. Finally, in Sections 5 and 6, the process proposed in this research, its implementation and the results obtained of the case study are shown.

2. Is There Any Process?

“A large scale development of digital learning resources involves the creation of a large number of these educational resources with a time limit, usually with the aim of supporting one or more educational courses.” (Velázquez Amador et al., 2011). Some of the reasons why it is not often carried out the large scale development of digital resources is the difficulty of developing these resources, as they are resources with particular complexity by relying on pedagogical aspects.

Another aspect is the lack of well-defined development processes and documented experiences (Barajas Saavedra, 2009; Velázquez Amador et al., 2011). Lack of standard methodologies to guide and organize game design can result in longer and less predictable game production processes. Moreover, the need for interaction among domain experts (providing the instructional content) and game developers is a peculiar aspect of serious games that makes their development more difficult (Vidani & Chittaro, 2009). Game design in an educational environment has to face some severe restrictions in the development process compared to professional game development (Masuch & Rueger, 2005).

However, despite the existing difficulties, in video games lies a viable means to solve the current problems of education, creating materials that support the knowledge acquired in the classroom, extending the classroom beyond the physical limits of the educational institution and allowing students to have an improvement in the way of learning through the use of these resources.

Quality Aspects for Educational Digital Resources

In the particular case of educational resources studied in this research (Velázquez Amador et al., 2011), mentions that the quality of a digital educational resource covers various aspects of software development using an object-oriented paradigm, and issues related to pedagogy. Therefore, is identified the existence of technical and pedagogical aspects, and usability and content components, which are considered as aspects that determine the quality.

Technical aspects include reuse and adaptability, as well as those established by the software engineering and utility, reliability, among others.

Pedagogical aspects consider everything that facilitates the teaching-learning process, as are examples, assessments, self-assessments, feedback, and a pedagogical objective expressed under any taxonomy, to mention some, Bloom’s Taxonomy.

The relationship between teaching methods and quality of the resource depends on the learning style of the user, so that, it is recommended that the modalities of digital resources include auditory, visual and kinesthetic; video games cover this recommendation perfectly.

The aspects of usability of a digital resource concern the presentation of information (fonts, colors, sizes, etc.) and the disposition thereof (symmetrically, asymmetrically, using positive and negative space, etc.). From the point of view of software engineering usability is ease of use and learning of an object created by humans.

This research will provide solution to the lack of well-defined development processes for large scale production of serious video games, presenting a proposal for a development process with fundamentals in software engineering practices, the results of using this process for the production of various video games, and the results of usability testing of video games with students. Solving, in this way, the problems identified within this subject of investigation (Barajas Saavedra, 2009), that is to say: 1) the experts in contents have not been provided with simple and intuitive tools that automate the large scale production; 2) the game producers do not have the rationale that supports the structuring or design of the serious game, or the experience in the competencies in which the videogame applies; 3) for the production of a serious game neither a structured nor based on software engineering process exists that guarantees the consistency and standardization of the production to increase and to guarantee the quality of products.

3. Video Game Specification or State of the Art

3.1. Blended Learning (B-Learning)

Jeff Pankin et al. (Pankin, Roberts, & Savio, 2012) define b-Learning as structured opportunities to learn, which use more than one learning or training method, inside or outside the classroom. This definition includes different learning or instructional methods (lecture, discussion, guided practice, reading, games, case study, simulation), different delivery methods (live classroom or computer mediated), different scheduling (synchronous or asynchronous) and different levels of guidance (individual, instructor or expert led, or group/social learning).

Blended approaches reflect a combination of the following factors (Pankin, Roberts, & Savio, 2012) (Figure 1).

Some strategies have emerged such as the proposed in (Madeira, Silva, & Palma, 2012), which suggests combining traditional classroom learning with modern computer-mediated activities. The authors establish that “the use of Learning Management System (LMS) in a b-Learning setting facilitates the student-teacher interaction outside the classroom in a virtual classroom environment” available 24 × 7. They report an improvement in the engagement and failure, among others success aspects.

In other researches like in (McGinnis, Bustard, Black, & Charles, 2008) is mentioned that e-learning systems, despite their potential, “do not yet have the impact that many believe is possible”. Moreover, the gap seems to be increasing because of the greater expectations of the current generation who have grown up with modern technology. Some of the main problems of e-learning are (McGinnis, Bustard, Black, & Charles, 2008): lack of engagement in the learning material, lack of compelling content, lack of interaction with user, unchallenging learning material, finally, lack of contextualization in the e-Learning design.

One of the aspects that have to be solved is to produce content that “provide immersion, motivation, fun and high level of engagement” (Ibrahim & Jaafar, 2009). The e-Learning by itself does not achieve these objectives, making it clear that it requires a blended strategy to achieve the expected learning objectives.

3.2. Serous Video Games

Many authors (e.g. Aghabeigi, Calvert, El-Nasr, & Riedewa, 2012; Critelli, Schwartz, & Gold, 2012; Emam & Mostafa, 2012; Huang, Dong, & Liu, 2011; Hwang, Wu, Huang, & Huang, 2012; Jovanovic, Starcevic,

Minovic, & Stavlja, 2011; Mao, Yi, JianGang, & Guo-tao, 2010; Cai, Liu, & Liang, 2010; Chuang, You, & Duo, 2010), establish that video games are effective learning mechanisms that provide “immersion, motivation, fun and high level of engagement” (Ibrahim & Jaafar, 2009).

“Computer games are very effective in the just-in-time delivery of new competencies and knowledge”. Engagement and immersion is kept by the game system continually delivering optional, achievable, new challenges and/or experiences to the player (McGinnis, Bustard, Black, & Charles, 2008).

“Playing” is a close activity with “learning”. During “playing”, one can often learn many things to improve his/her “playing” with high motivation. However, the “playing” is usually excluded from “learning” from the educational point of view, because of the following two difficulties, the one is the difficulty in grasping individual learner’s playing and the other is the difficulty in providing adequate feedback for each playing. Such adaptive interaction is impossible to realize in usual classroom (Umetsu, Hirashima, & Takeuchi, 2002).

Several authors consider Serious games as the next generation of e-Learning tools because games stress visual and experiential learning; they require active involvement, strategizing, hypothesis testing, or problem-solving by student players while providing almost instantaneous feedback (Dondlinger, 2007; Cooper & Longstreet, 2012).

3.3. Serious Games for Learning

It is important to produce content that eliminate the border between formal and non-formal education. To accomplish this, it must be established, in concrete terms, how these games will be produced to successfully meet the desired competencies at each grade level for each subject. A proven successful way to accomplish this production is the decomposition of each subject into generic and specific competencies (Competency-based decomposition), such as that proposed in (Barajas Saavedra, 2009) and later carried out in the massive development of learning objects project “Business-Academia-Government Linkage Model for the Development of IT Capabilities of Human Resources”. All these contents are stored in the CAPACINET platform, available at http://capacinet.gob.mx/modpara/ (Velázquez Amador, et al., 2011). This approach allows to match a formal competency with a non-formal content, identifying the aspects and factors that should be implemented in the production of the game so that satisfactorily cover the expectation of the competency within a scholar grade.

Bearing in mind this approach, and the analyzed literature on video games and learning objects, a non-exhaustive set of basic features that represent a good starting point to achieve a usable and effective transfer of knowledge through serious games were identified: 1) Short and focused on a single competency, to enable the student to focus on a particular topic at a time; 2) Graphical user interface with aesthetic and minimalist design, friendly, and pedagogically evaluated; 3) Cases with formal reasoning; 4) Cases randomly generated to prevent the student memorize the answers to problems; 5) Challenging content and generating competition among students using the game, i.e., cases with different levels of difficulty.

4. Previous Approaches

Masuch establishes that a typical game design process consists of the following steps (Masuch & Rueger, 2005): 1) Developing the core idea; 2) Writing a game concept; 3) Producing the artwork; 4) Programming the game engine; 5) Game content production; 6) Play testing; and 7) Balancing and bug fixing Ibrahim proposes an Educational Game Design Model that indicates that the game production is divided into the main stages a) Game design, b) Pedagogy, and c) Learning content modeling. But this proposal do not clearly indicates how those stages interact and which are their inputs and outputs (Ibrahim & Jaafar, 2009).

Zin proposes an Educational game design that consists of four main elements, interaction, knowledge, engine and level. But they do not have a structured process that guides to the reader from a starting point in the process to the end where a game is a finished product (Zin & Yue, 2009).

5. Game Design Process

In Figure 2 can be seen the Game Design Process proposed by this research, which one is described in the next paragraphs. As the reader can see, the designed process has five stages, which are intended to capture all the elements of the game from the targeting and pedagogical requirements establishment, through design and development up to technical testing, usability testing and testing for absorption of knowledge, with a final step of continuous improvement.

This Game Design Process is founded in the traditional Software Engineering paradigms and complemented by previous efforts on large scale development of digital learning resources (Barajas Saavedra, 2009), and specific skills on Mathematics. Next, a brief description of each stage of the process is explained.

•   Requirements Stage. The objective is to set goals that will cover the game; to establish the pedagogic mechanisms, across which the knowledge will be transferred to the students; to determine the competences and the knowledge areas that must be covered; and to create storyboard and concept art.

•   Design Stage. Its objectives are to create all digital resources needed by the game engine for the creation of the video game. These digital resources includes: 2D illustrations, 3D models, Maps, Objects, Materials, surfaces, etc., Sounds and music; and to create game engine if needed.

•   Development Stage. The objective of this stage is to create the game including: Layout, Events, Shader, and AI; to design game play; and to integrate all the above elements with menus, options, etc.

•   Testing Stage. Its objectives are to test the videogame in the next aspects: Technical, Knowledge absorption, Usability, Usefulness; to obtain efficiency statistics; and to maintain game.

•   Postmortem Stage. The objective is to analyze all process and product information collected during development process to improve future developments.

Advantages of the Proposed Process

The game development process proposed provides developers and game designers with a process that will lead them clearly through the production of an serious video game, and in this way, have a map of the steps from conception of an idea to the release of the game, something that until now it was available only in internal documents of the major game development companies.

The game development process also provides a framework for the integration of experts from different disciplines to develop a serious video game, such as graphic designers, programmers, instructional designers, content developers, educators, project managers , project leaders, to name a few.

The game development process proposed has a unique feature against other proposals, is developed from the point of view of Software Engineering, which allows to implement the process in a transparent way because the game is considered as a software product, so that a company dedicated to software development can deploy it easily and efficiently. It is important to emphasize that the process is independent of the development platform to be used, the specific techniques and pedagogical models to be implemented in the game, in other words, the game development process was designed to be implemented independently the type of product to be developed.

Finally, the game development process also provides, at the stage of requirements, the ability to integrate efficiently the pedagogics and competencies to the video game requirement, so, developers are able to design video games that meet the need in the classroom (formal education) and the team is able to create products that tell teachers how to integrate the game within their classes, creating a bridge between formal and informal education.

6. Study Case

As a proof of concept of our approach the research team conduct a study case using as scenario “the development of serious games to cover all the official competencies for sixth grade for elementary school in Mexico”.

At first, the team identify a set of competencies for mathematics learning for sixth grade in elementary school in Mexico. This activity consisted in a deep review of syllabi and textbooks contents distributed by the Mexican Ministry of Public Education. After that, the team applied the Competency-based decomposition approach in order to establish the set of knowledge area, which should be covered by the developed serious games, see Table 1.

Competencies shown in Table 1 lead us to create a collection of 50 serious video games oriented to increase learning encouraging appropriation of specific math-competencies. An extract of this list is shown in Table 2.

After the initial production phase of serious video games, the team proceeded to test them in order to study the impact on the learning level of students exposed to this learning strategy. Participants consisted in a group of 29 students from sixth grade of elementary school from the “Federal Rural Cuauhtémoc Elementary School” located in La Paz, Ojuelos, Jalisco. Children studying in this school come from families just as scarce resources. This community has many needs, and to increase the use of IT access to information technology helps to alleviate some of them.

The process performed for the test was as follows (Hernández Sampieri, Fernández Collado, & Baptista Lucio, 2010): 1) Identify potential schools. 2) Tests were designed for initial and control evaluations. The tests were designed to evaluate knowledge level of students in the next knowledge areas: Areas, Handling of solid figures, Fractions, Shapes and polygons, and Crossed product. 3) School was selected. 4) Students group was selected. The group was divided into two parts; taking into account that in both groups, students’ average grade must be equally distributed, i.e., the group was divided according to the average grades of the students. 5) Initial evalua

tion was applied to all students. 6) The test group used video games in one-hour sessions twice a week for four weeks. 7) At the end of eight sessions, a second test was applied to identify the impact of video games use. 8) The collected data were analyzed with SPSS software.

6.1. Results

The team obtained linear regressions of each knowledge area by applying statistical analysis on collected data. This information allows the team to determine trends in scores comparing the results before and after serious video games use.

The overall findings of our study are graphically depicted in Figure 3 where diamonds-line displays the results obtained during initial examination. Squares-line displays the results of the evaluation performed after serious video games use.

The team was able to observe significant improvements in three knowledge areas (“Handling of solid figures”, “Areas”, and “Shapes and polygons”). Meanwhile, the area of knowledge “Fractions” has a slight rise in the scores. These enhancements are strongly related to the use of serious video games that helped both, decreasing the frequency of low scores, and increasing the frequency of higher scores. The bigger discrepancy was found in the scores from knowledge area “Crossed Product” where we observed a mild decrease in the scores.

6.2. Discussion

It is important to mention that the characteristics of the games used in this study were implemented in each of the produced resources, facilitating production and assured learning outcomes.

We think that the improvement in the scores achieved in knowledge areas “Handling of solid figures”, “Areas”, and “Shapes and polygons” shows that the use of video games help to improve the knowledge level of students who use them. The results mentioned above show a strong trend of improvement in the level of knowledge in the considered knowledge area.

Related to knowledge areas with low scores we observed some interesting aspects that certainly could have a negative influence in the scores. In the case of “Fractions” (Figure 4), the staff responsible for conducting the tests observed that the video game graphic design was unattractive to kids, resulting in little interest in using the game by the students.

On the other hand, the game related to the knowledge area “Crossed Product” (Figure 5) had a different condition, in this case the problems presented by the game were not randomly generated, but the game had a question bank which students were able to memorize. This specific situation allowed students have high scores when they played the game, but getting the opposite in the test.

7. Conclusions

This research work is focused on creating a process to successfully develop serious games for learning math and to determine the impact of using serious video games to convey specific Basic-Math competencies in elementary school. The proposed process enabled a team of developers to create a collection of 50 video games, explicitly defining its pedagogical characteristics and its implementation in the finished product. The developed serious video games are based on a competency-based approach oriented to improve appropriation of Math competencies by students.

We conducted a four-week study to determine the impact of using the developed serious video games to ensure that design guaranties the knowledge absorption. This study was performed at a public-rural elementary school participating 29 sixth-grade kids. The results obtained show that serious video games represents suitable resources for teaching Math in elementary schools, since children are very interested in their use. Similarly, we could observe that the use of video games increases the level of knowledge of students significantly in a short period of time. In this vain, it is possible to visualize higher learning levels in students if these strategies were applied in a continuous way by teachers and along the scholar year.

The results obtained of the application of the questionnaire of the System Usability Scale reflected the importance of ensuring well-designed serious video games, from internal code through the user interface, which directly impact on the interest of kids on the game having repercussions on the level of use. All details must be carefully considered, analyzed, developed, and evaluated, otherwise the generated products will not ensure student learning, resulting in low absorption of knowledge and poor performance by students, even worst, the videogame content could confuse kids and prejudice over helps.

There are several aspects to consider as future work. One of them is improve the current set of serious video games and increase the items of the list, improve and formalize the development process of serious video games bearing in mind the competency-based approach followed in this research.

Another important aspect to do is to conduct a study along the scholar-year period to evaluate the enhanced

set of serious video games. We also are working in identify and analyze usability aspects and user interface design in serious video games.

References