A new subject in postgraduate coursework at UNSW Australia at the Australian Defence Force Academy campus provided a curricula opportunity to allow students to do structured collaborative learning and to choose their own context for an investigative research. The student-centred investigation is necessary to reinforce formative knowledge of advanced optimi z ation and test techniques. The course developer knew from his own educational research [ 1 ] that such pedagogy should be inherently inclusive of diverse abilities, interests, learning styles, cultures, prior studies and genders; however, the positive effect on this inaugural course was well beyond the expectations of both the teachers and learners. This paper presents the research of just one of the 15 students to exemplify the inclusiveness of such pedagogy in tertiary curriculum development, in this case for gender-inclusion, however other students benefitted who had less ability for such tertiary concepts, and in one student’s case the pedagogy helped overcome significant cultural barriers. The context chosen by the showcased student’s work was to optimi z e the baking of cupcakes. Ever wondered how to bake the perfect cupcake? Ever wondered if the recipe you are following is actually correct? How did they come up with the temperature and time for cooking? Her normal work is in highly classified and compartmentali z ed electronic warfare and so she opted instead for a topic that could be shared with her military colleagues and the public; itself an inclusive and possibly reciprocating gesture. Many people who ordinarily do not associate with Science, Technology, Engineering and Mathematics (STEM) subjects, either through gender, academic ability or culture, can relate to trying to perfect baking of a recipe such as cupcakes. As such, this case study offers a timely and interesting reminder of the inclusive benefits of student-centred context in curricula development, particularly for STEM subjects, reinforcing previous research from the literature reviewed.
The article begins with three background sections: first a short general background on the University context that led to the course being developed; second a background on the course learning objectives, structure and the topics students chose freely for their individual research; and third a more traditional literature review of the inclusive pedagogy employed for the course. The research method used in this case study was direct observation of the students’ enthusiasm and success, made possible because the teacher was also an experienced educational researcher and the number of students was small.
The article proper then showcases the stages in the investigation of cupcake baking, starting with setting up the experiment and how to judge the cupcakes (measurement system analysis) which are shown in
Once the measurement system was suitable, the process screened the significant cupcake baking factors (i.e. oven temperature) from all the possible factors, and then involved a detailed modelling test and optimization of settings for the best cupcake baking. The article finishes with two conclusions, one for cupcake baking and the other for the benefits of inclusive curriculum design through
student-centred learning especially in STEM subjects. The work is only offered as an encouraging example of curricular techniques to try for greater inclusion.
The University of NSW has developed one of Australia’s largest and most popular higher degree programs in systems engineering and project management, leveraging a close relationship with the Australian Department of Defence (DoD) that formed initially around the delivery of undergraduate programs to the Australian Defence Force Academy in Canberra, but which has since evolved to all aspects of normal university offerings. Most of the higher-degree coursework students are Defence Force members or public servants studying part-time. As a consequence, the technological contexts used are often skewed heavily towards ships, aircraft, missiles, land vehicles and other warfare fields like cybersecurity. Also, the STEM trends in the U.S. DoD are always closely monitored and followed wherever possible, as these fields are relevant to DoD students because of the close alliance between the two countries and the fact that much of Australia’s DoD materiel is procured from the U.S. (Defence, 2014) . Notwithstanding these general contextual tendencies, the University has worked to broaden the appeal of its degrees, so as to help attract and retain non-traditional parts of society to STEM subjects and careers. Such appeal is considered critical because Australia has a low population and yet an enormous area to defend, and it has an aging population [2], meaning that the DoD must compete for a smaller work-ready population, especially the school-leaver population (Defence, 2016: 150-152) .
The University of NSW’s Master of Systems Engineering and Master of Project Management are popular coursework higher degrees that have as one of their core subjects an introduction to test and evaluation (T&E). Despite many years of growth in these courses, they did not have any advanced elective subjects in T&E techniques. Recent comparison of Australia’s rigour in T&E compared to the U.S. found that the U.S. DoD had prescribed and implemented new advanced scientific techniques and competencies [3]. As such, the UNSW Australia partnered with Air Academy Associates, LLC (AAA) in the U.S., who are a leading research and training firm in these advanced T&E techniques, to adapt one of their courses to the University’s Master programs. The resulting elective subject, known as ZEIT 8034 Advanced T&E Techniques, was developed in October 2015 to February 2016 and trialed for the first time in Semester One of 2016 (February to June) with 15 volunteer students (4 female, 11 male). The subject has successfully run twice since the inaugural trial course.
The Advanced T&E Techniques course aims to provide advanced test design and analysis techniques that perform the following three major functions:
screen what are the significant factors that influence system responses; model precisely enough the effects of those significant factors to optimize use of a system, particularly limiting variation within quality boundaries; and validate that both the model and system in the optimized state perform acceptably in representative conditions.
The methods used originate from the design of experiments work by Fisher (1971 [
Students do a one-week intensive course period where they are given the theory and use the new techniques in about eight scripted workshops of increasing complexity. Students “follow-along” the test design and analysis examples to get the solutions provided. They also work in collaborative groups over this week for about three hours each day to do very practical screening, modelling and performance validation on a toy instructional system known as a “statapult”. The Statapult® catapult [6] is a small wooden catapult that fires rubber balls over a short distance within the classroom as shown in
This learning device has many different factors that affect the distance the balls travel and how that varies between like firings, such that the system has a fairly significant performance variation (i.e. difficult to use accurately)unless it carefully understood and controlled. The teaching device therefore reinforces many of the key themes of the course, such as:
focus on understanding your measurement system before testing and ensure it is adequate; do a simplistic screening test to focus the later more detailed tests that will give accurate modelling; validating performance usually requires new test measures (i.e., timeliness and supply can become as important as accuracy); and there is nearly always multiple solutions in complex systems and understanding how to optimize from these with multiple input factors and multiple output responses requires multiple constraints and weighted solutions.
There is a 15-minute video compilation of the course at the following link that shows the Statapult® in use with student testimonials about what they learnt in various parts of the course:
https://www.unsw.adfa.edu.au/capability-systems-centre/advanced-test-and-evaluation-techniques#overlay-context.
At the end of the intensive week, students propose to the teacher and their peers a system that they will have access to and that they want to screen and model using their new test design and analysis techniques. Topics chosen by the students in this inaugural class are listed in
Such context choices are intended to meet the challenge of making tertiary education internationally inclusive by “developing curricular contexts that extend themselves meaningfully into the personal life-worlds (i.e. environment from the perspective of an individual) of students” (Rasi, Hautakangas, & Väyrynen, 2015: 134) . Oral presentation and group discussion of those personal choices to their peers allowed students to “connect the theories, concepts and issues being taught to their life-worlds” (p. 139) and thus be more inclusive in many different ways.
Students undertook their personal research over the following months at work or home with some mentoring at key times by the teacher. Students also undertake a knowledge quiz on the key concepts. Assessment was divided into: 35 percent for the collaborative assignment report concerning the Statapult® system, 15 percent for the computerized knowledge quiz and 50 percent for the individual research and report.
Gender | Reason | Topic to Screen, Model & Optimize |
---|---|---|
Male | Hobby | Three dimensional printing of casino dice |
Female | Hobby | Pizza base quality?changed later to a drawing robot |
Male | Interest | Supersonic transport aircraft design (historical data) |
Female | Work | Small commercial shop?stocking, transactions & customer loyalty |
Male | Test ease | Adhesive of blu-tack for wall hangings |
Male | Work | Nested simulation of electro-optical countermeasure models |
Male | Hobby | Ammunition loading for high caliber rifle |
Male | Work | Army handheld counter-improvised explosive device detectors |
Female | Hobby | Slot car race tracks and cars |
Male | Interest | Water in whiskey |
Male | PhD topic | Effect of Supply Chain disruption on Capital Project Success |
Male | Test ease | Comparison of three different kettles |
Male | Hobby | Quadcopter landing |
Male | Work | Army night-fighting illumination rounds |
Female | Hobby | Baking cupcakes |
The challenge in any new university course is articulated as follows by Tait (2009: 193-194) :
“It is fundamental to the tertiary educator’s role to foster the development of a vibrant and supportive learning community through their relationship with students. Students need to know that university lecturers care about them and their learning. This can be evidenced through well-organized courses and materials; interesting, exciting, and fun activities for diverse learners; deep seated knowledge of the unit concepts; and flexibility to accommodate emergent student learning needs.”
One of the most fun and exciting aspects of the collaborative part of the course was validating the performance of groups’ models and systems to penetrate an enemy castle within an acceptable overall time and number of rounds (i.e. balls fired to get a set number of hits). Foam play blocks were taped together to represent walls and under time pressure, students delighted in trying to hit the walls as often as possible, albeit all underpinned by some serious scientific results. A ball having just impacted a castle wall is shown in
The course lecturer had significant educational research experience twenty years ago [i]that explored the benefits in tertiary education of structured collaborative learning, multiple teaching and learning styles and variety in contexts. He had remained interested in inclusiveness through texts like Hyter and Turnock (2006) and Hyde, Carpenter and Conway (2013) and when given this curricula development opportunity ensured the pedagogy had the crucial social ele-
ment as outlined by Tait (2009) as follows, and known more academically as Vygotskian theory (Udvari-Solner & Thousand, 1996) :
“learning is primarily a social, cultural, and interpersonal process that is influenced as much by social, emotional, and cultural factors as by cognitive ones. … This concern for the social context of learning clearly needs to be added to the suggestion that the meaningful learning of complex material (in contrast to the acquisition of isolated information, which in certain cases is still necessary) may be characterized as being active, constructive, cumulative, self-regulated, and goal oriented …The learner-centered orientation inherent in modern views of learning has important implications for instruction at the tertiary level …” [p. 193]
Inclusive education principles were applied to this course design, where for example there is the diverse pedagogy of structured lectures, structured workshops, collaborative and unstructured workshops (i.e., Statapult®), a knowledge quiz, student presentation, and student-centered individual research with mentoring. The benefits of diversity in pedagogy can include more robust conception, greater enjoyment, higher grades than normal, and better inclusiveness. In Ashman’s (2010) research on inclusive curriculum for Australian university, he refers to this notion as curriculum differentiation:
“Curriculum differentiation refers to a flexible approach to teaching that addresses the different learning needs of students including learning interests, styles and rates within specific learning contexts. In general terms, the curriculum can target a range of outcomes by concentrating on content mastery (e.g. learning ideas and skills), concept mastery (e.g. systems of knowledge) and process mastery (e.g. research and information management skills).” [p. 670]
Ashman (2010: 677) found, inter alia, significant inclusive benefits in creating “work units that accommodated levels of skill, preferences, and interests”. Udvari-Solner and Thousand as early as 1996 (p. 182) found that “sound theoretical foundations and the use of learner-centered, process-oriented, and communication-based instructional approaches are … promising practices for designing a curriculum that is responsive to the needs of diverse learners.”
The research method used in this case study was direct observation of the students’ enthusiasm and success, made possible because the teacher was also an experienced educational researcher and the number of students was small. There was no baseline comparator for this course however the teacher and most students saw examples of all of the aforementioned benefits during this course, when compared to other postgraduate subjects they had taught or taken. Such benefits can be particularly potent when computer-based learning speeds the ability of students to explore concepts and multiple contexts in student-centered ways and these software packages do this well.
Of all of the possible benefits of diverse pedagogy, student inclusiveness is the most rewarding, because students of lower ability often interact more, sharing their conceptions in less confronting ways than a lecture, while students bring in gender-inclusive and culturally-inclusive examples that are well beyond what the teacher could envisage alone. Simply the act of collaborative learning and presenting back to class has been shown to make STEM subjects more gender-inclusive as outlined as follows by Wistedt (1998) :
“From a gender perspective, variations in ways of knowing a subject are considered to be crucial to the learning process, as are the many styles of understanding and ascribing meaning to the course content. The notion of negotiation (Voigt, 1994; Burton, 1995) directs attention towards the reciprocal nature of knowledge formation. Negotiation calls for two-way communication, for reflection upon the foundations of statements put forward, for the trying out and testing of assumptions, for exploration and synthesis.” [p. 144]
Just why the social aspects of learning are important to women learning has always been less important in education than using this knowledge to positive advantage. However, a study by Robichaud et al. (2003) into four contributing factors to worry among university students found “Women in the sample reported significantly higher levels of thought suppression and negative problem orientation” than men. Collaborative learning helps create interactions around problems and compensate for any one student’s lack of confidence and as such, might counteract these tendencies. Surprisingly no studies could be found linking the work on gender differences in worry with collaborative learning.
The gender-inclusive benefit of pedagogy that supports negotiation and interaction is supported by a more recent study by Koppi et al. (2010) , especially their
“A pedagogical approach that is inclusive would include the value and meaning of the technology in the broad context of its human application. Without lessening the technological content, the inclusive approach would relate the technology to everyday usage in society and the benefits afforded. … These considerations also relate to the greater call for relevancy and work-integrated learning that the great majority of survey respondents requested.”[pp. 278-279]
By allowing students to select topics important to them from work or hobbies is fundamentally relating what they learn to everyday usage, where they can engage with work colleagues, fellow hobbyist or families to relate at least the benefit of what they have learnt.
The greatest risk in such pedagogy is a potential loss of teaching structure, where some collaborative groups can become inefficient, really weak students can limit their classmates, teachers can become limited by “needy” students, student presentations can either run too long or collapse through a lack of confidence, and sometimes students will have to change topic or re-do large portions of work because they took a wrong direction and it is missed in the freedom of a task. All of these examples occurred in this course but were managed through either intervention or simply accepted by the participants as a reasonable tax for the freedom given to explore and express. Mentoring part-time students on their own work can require phone calls and e-mails on the only weekend they have set aside for their research, and because they chose the topic, the teacher has to ask more questions than they do. Such a reversal of knowledge provided the opportunity to show genuine teaching interest in individual student learning as outlined earlier from Tait (2009: 193-194) .
Having the Statapult® learning device for collaborative groups to explore, fundamentally deconstructs the classroom pedagogy from lecture-centered to student-centered, invoking more active participation. In collaboration students will represent their group’s views in ways they would not do alone and will volunteer ideas and ask questions amongst a group of four students that they might never do in front of the whole class. What makes such interaction inclusive is captured by Wistedt (1998: 152) following the examination of inclusive assessment methods to postgraduate mathematics, physics and statistics classes:
“Inclusive education appreciates such a variation in perspectives, individual as well as social and cultural. Furthermore, if the students’ appreciation of what is worth knowing, what counts as knowing and what characterizes knowing in an academic setting is inextricably linked to the norms expressed within the social setting in which learning takes place, critical reflection upon this setting is a necessary prerequisite for successful learning and teaching. Opening a dialogue with students who vary in backgrounds, interests and experiences, and with teachers who vary in perspectives and expertise, is one way of realizing an inclusive education.”
Students gain a lot of confidence to analyze a system of their own choice from the structured collaborative learning groups, as well as trust that the teacher is genuine in mentoring without direct control. The trust in mentoring is key, because students do their testing and analysis of a system of their own choice part-time once they return home and they need to feel confident to call the teacher and disclose knowledge forgotten or not understood, possible mistakes and so on. Students who are reluctant to take over their learning at such times, first do so in collaborative groups, building confidence from their peers and the students who are keener in this type of exploratory learning. What breaks down such barriers more than anything else once they are doing their own research context, is that they are the expert on their chosen system, and they enjoy teaching the teacher about their chosen system in the process of getting any help they might need.
Experimental design for a system process like cupcake baking begins with examining the system through process flows, cause and effect diagrams and classification of factors in accordance with the methods in the course texts (Schmidt & Launsby, 2005; Reagan & Kiemele, 2008) .
In the process of cooking a cupcake there are many factors that could contribute to the success of the baking process. This research only investigated the factors that affect the cooking process and did not look into the effects the ingredients or method of the batter preparation could have with baking the perfect cupcake.
“X” shows the factors that were deliberately altered as part of screening,
“C” are those factors kept constant, and
“N” are factors treated as experimental noise.
The experimental design model in
three response outputs for the system ready to do a screening test. Shown for the input factors are the high and low settings (i.e., two-level) such as ten to twenty minutes for cooking time and 140 to 180 degrees Celsius for oven temperature. The output responses chosen needed to be measured consistently by judges. The grading index shown in
Standard operating procedures (SOPs) were developed for each of the controlled variables so as to minimize unwanted variation; principal among these was the all-important recipe from Australian Good Taste (2016) and the one home oven as given in
The repeatability and reproducibility of the cupcake measurement was assessed using the measurement system analysis (MSA) techniques of the course texts to ensure these were accurate, precise and stable. Because the judging criteria is bi-
Moisture | |
---|---|
5 | Dry |
4 | Quite even but crumbly, lack of moisture |
3 | Evenness all the way through, perfect crumble and moist |
2 | Very moist and very dense |
1 | Overly gooey, fondant like texture |
Appearance | |
5 | Perfectly risen, symmetrical, homogeneous |
4 | Well risen but imperfect symmetry |
3 | Slightly risen |
2 | Only risen on edges |
1 | Not risen at all |
How-well cooked | |
5 | Burnt |
4 | Overdone-dry |
3 | Perfectly cooked |
2 | Slightly raw |
1 | Raw |
nary data according to the course texts the number of people multiplied by the number of parts must ideally be greater than or equal to 60. As such the MSA was conducted with 6 judges (operators) and 10 different bake settings (parts) and replicated twice to determine the variation. The raw MSA data and ANOVA analysis of the MSA results for “appearance” are shown as an example in
The precision-to-total of the ANOVA for the MSA of the “appearance” grading lies between 0.10 and 0.30 and is therefore sufficient to proceed to testing in accordance with the course texts. Also the resolution is greater than five (6.8) and is therefore adequate to proceed. The operator-to-part interaction is high (96%) showing some bias towards certain cupcakes dependent on their personal preference. The consistency in judges’ ratings is shown in
Judge 1 | Judge 2 | Judge 3 | Judge 4 | Judge 5 | Judge 6 | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Bake # | Rep 1 | Rep 2 | Rep 1 | Rep 2 | Rep 1 | Rep 2 | Rep 1 | Rep 2 | Rep 1 | Rep 2 | Rep 1 | Rep 2 |
1 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 |
2 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
3 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
4 | 1 | 1 | 1 | 1 | 1 | 1 | 2 | 1 | 1 | 1 | 1 | 1 |
5 | 2 | 2 | 2 | 2 | 1 | 1 | 2 | 1 | 1 | 1 | 2 | 2 |
6 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 |
7 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
8 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
9 | 4 | 4 | 5 | 5 | 5 | 4 | 4 | 5 | 4 | 4 | 4 | 4 |
10 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
Variance | Standard Deviation | % Contribution | ||
---|---|---|---|---|
Source | 0.095 | 0.3082207 | 4.16% | p Value |
Total Measurement (Gage) | 0.03333333 | 0.182574186 | 1.46% | |
Repeatability | 0.06166667 | 0.24832774 | 2.70% | |
Reproducibility | 0.00685185 | 0.082775913 | 0.30% | |
Operator | 0.05481481 | 0.234125639 | 2.40% | |
Oper * Part Interaction | 2.19055556 | 1.480052552 | 95.84% | 0.0000 |
Product (Part-to-Part) | 2.28555556 | 1.511805396 | 100.00% | |
Precision to Total Ratio | 0.20387591 | |||
Resolution | 6.8 |
bakes, illustrating that judges mainly rated differently in Baking 1 and Baking 2 but consistently thereafter.
The other rating scales were also found to be sufficiently accurate and consistent to proceed to the screening test:
“moisture” had a precision-to-total of 0.18 (<0.3) and resolution of 7.6 (>5), and “how-well cooked” had a precision-to-total of 0.26 (<0.3) and resolution of 5.3 (>5).
Using the mixed-method flowchart of the course text screening design selection is the Taguchi L12 test design for evaluating eight factors, each at two levels (i.e., high and low values only) with four or more repetitions. This screening method is capable of dealing with up to 11 factors without increasing the number of test runs. The final test design is shown in
The two-level design for screening captures the linear effects of each factor independent of the other factors. This is adequate for a screening test design since the investigation is to determine the factors that have a significant effect on the perfect cupcake. The insignificant factors can then be screened out and the significant factors can be investigated more thoroughly using a three-level test design to determine any quadratic and interaction effects. Most of the factors for this test are discrete and qualitative except for temperature, fill-of-pan and cooking time which are continuous.
Four tools are predominately used from DOE PRO XL® to screen from the results: marginal means plots, multiple response regression, Pareto of regression coefficients and a multiple response optimizer. Since there are three output res-
Factor | A | B | C | D | E | F | G | H | Moisture | |||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Row # | Preheat oven | Temperature | Oven Setting | Fill of pan | Size of pan | Cooking Time | Preheat Pan | Position in oven | Y1 | Y2 | Y3 | Y4 |
1 | 0 | 140 | 0 | 0.5 | 0 | 10 | 0 | 0 | 3 | 3 | 3 | 3 |
2 | 0 | 140 | 0 | 0.5 | 0 | 20 | 1 | 1 | 4 | 4 | 4 | 4 |
3 | 0 | 140 | 1 | 1 | 1 | 10 | 0 | 0 | 1 | 1 | 1 | 1 |
4 | 0 | 180 | 0 | 1 | 1 | 10 | 1 | 1 | 1.167 | 1 | 1 | 1 |
5 | 0 | 180 | 1 | 0.5 | 1 | 20 | 0 | 1 | 1.667 | 1.5 | 2 | 2 |
6 | 0 | 180 | 1 | 1 | 0 | 20 | 1 | 0 | 3 | 3 | 3 | 3 |
7 | 1 | 140 | 1 | 1 | 0 | 10 | 1 | 1 | 1 | 1 | 1 | 1 |
8 | 1 | 140 | 1 | 0.5 | 1 | 20 | 1 | 0 | 1 | 1 | 1 | 1 |
9 | 1 | 140 | 0 | 1 | 1 | 20 | 0 | 1 | 2 | 2 | 2 | 2 |
10 | 1 | 180 | 1 | 0.5 | 0 | 10 | 0 | 1 | 1 | 1 | 1 | 1 |
11 | 1 | 180 | 0 | 1 | 0 | 20 | 0 | 0 | 4.333 | 4.333 | 4 | 4 |
12 | 1 | 180 | 0 | 0.5 | 1 | 10 | 1 | 0 | 1 | 1 | 1 | 1 |
ponses and both the average and variation in each distribution has to be considered, some 24 analyses occurred―hence only an example of each is shown here.
The marginal means plots of the absolute rating value of “moisture” is shown in
The multiple response regression analysis for the absolute values of each output cupcake rating is shown in part in
The coded regression table (
“Moisture” Rating:
y ^ = 1.910 − 0.611 A − 1.069 C − 1.389 E + 0.132 F − 0.319 G − 0.389 H
“Appearance” Rating:
y ^ = 0.979 − 0.819 A + 0.007 B − 1.403 C + 1.028 D − 1.403 E + 0.121 F − 0.792 H
Y-hat Model | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Factor | Name | Moisture | Appearance | How well cooked | ||||||
Coeff | P(2 Tail) | Tol | Coeff | P(2 Tail) | Tol | Coeff | P(2 Tail) | Tol | ||
Const | 2.000 | 0.000 | − | 2.410 | 0.000 | - | 2.069 | 0.000 | - | |
A | Preheat oven | −0.306 | 0.000 | 1 | -0.410 | 0.000 | 1 | -0.257 | 0.000 | 1 |
B | Temperature | − | − | − | 0.132 | 0.064 | 1 | 0.160 | 0.001 | 1 |
C | Oven Setting | −0.535 | 0.000 | 1 | -0.701 | 0.000 | 1 | -0.611 | 0.000 | 1 |
D | Fill of pan | − | − | − | 0.257 | 0.001 | 1 | 0.201 | 0.000 | 1 |
E | Size of pan | −0.694 | 0.000 | 1 | -0.701 | 0.000 | 1 | -0.653 | 0.000 | 1 |
F | Cooking Time | 0.660 | 0.000 | 1 | 0.604 | 0.000 | 1 | 0.625 | 0.000 | 1 |
G | Preheat Pan | −0.160 | 0.000 | 1 | - | - | - | -0.188 | 0.000 | 1 |
H | Position in oven | −0.194 | 0.000 | 1 | -0.396 | 0.000 | 1 | -0.229 | 0.000 | 1 |
R2 | 0.975 | 0.901 | 0.948 | |||||||
Adj R2 | 0.972 | 0.884 | 0.938 | |||||||
Std Error | 0.201 | 0.480 | 0.308 | |||||||
F | 268.787 | 52.253 | 89.137 | |||||||
Sig F | 0.000 | 0.000 | 0.000 |
“How-well cooked ” Rating:
y ^ = 0.250 − 0.514 A + 0.008 B − 1.222 C + 0.806 D − 1.306 E + 0.125 F − 0.375 G − 0.458 H
As shown in
The multiple response regression table for variation in each of the responses is not shown but revealed the following regarding the most likely “spread shifters”:
Oven Temperature is likely significant on variation of “Moisture” rating.
Preheat Oven and Fill-of-Pan are significant on variation of “Appearance” rating.
Oven Temperature is significant and Oven Position is likely significant on variation of “How-well-cooked ” rating.
The final tool used was the DOE PRO XL® optimizer tool which involves setting desired constraints, weighted as necessary to examine ideal settings, noting that at this stage the model is only linear. The optimizer allows the ideal ratings and a minimization of variance, however for simplicity here, only the optimization results of the optimum ratings are shown in
Optimized Settings & Weighting | ||||
---|---|---|---|---|
Variables | Moisture | Appearance | How-Well Cooked | Appearance Only |
Y = 3, W = 50% | Y = 5, W = 50% | Y = 3, W = 50% | Y = 5, W = 100% | |
Preheat Oven | 0 | 0 | ||
Temperature | 140 | 173 | ||
Oven Setting | 0 | 0 | ||
Fill of Pan | 0.833 | 0.833 | ||
Size of Pan | 0 | 0 | ||
Cooking Time | 10.68 | 16.75 | ||
Preheat Pan | 1 | 1 | ||
Position in Oven | 0 | 0 |
There were significant shifts if consistency in the ratings was desirable:
When variance is not as important as the rating values themselves, a lower cooking temperature is the preferred setting; however, when variance is controlled (set to zero) a higher temperature is recommended.
When the baker is only interested in the appearance of the cupcake then a mid-range and a 80% fill is the recommended method; however, if all variances are aimed to be set at a minimum then the fill should be lower (approximately 60%).
Collectively it was determined that the factors required for detailed modelling are Oven Temperature, Pan Fill and Cook Time, while the Position in the Oven, Oven Setting, and Size of Pan can be set to a constant based on their best value. The Preheat Oven and Preheat Pan results were conflicting, therefore these factors are required for modelling in order to appreciate the system accurately. It was decided to combine the Preheat Pan and Preheat Oven into one factor; “Preheat” where if Preheat = 1 then the oven and pan will be preheated at the cooking temperature for 10 minutes prior to baking, if Preheat = 0.5, the pan and oven are heated at 90 degrees Celsius for 10 minutes before baking and Preheat = 0, is starting the baking with the oven and pan cold.. These four factors will be investigated further in modelling.
The settings of the constants were determined from the marginal means plots and the consistent responses in the optimizations to be:
Position in oven―top;
Oven setting―fan forced;
Size of pan―small.
A four-factor, 3-level modelling design was required to model the system which according to the course text leads to the Box Behnken modelling design. The benefits of the Box Behnken is that there is a reduced test demand compared to a full factorial test, however, it still provides information on the main, two-way and quadratic interactions. A disadvantage of this method is that it is unable to show the three-way interactions. The setup of the four-factor Box Behnken can be seen below in
As recommended by the texts the origin point is repeated three times to help with orthogonality, at test cases 9, 18 and 27, and the repeats are used to help check the consistency of the testing. Once again, similar to the screening, five
Factor | A | B | C | D | Moisture | |||
---|---|---|---|---|---|---|---|---|
Row # | Preheat | Temperature | Time | Fill | Y1 | Y2 | Y3 | |
1 | 0 | 160 | 15 | 0.75 | 3.4 | 3 | 3.2 | |
2 | 0 | 180 | 15 | 0.75 | 2.9 | 3.2 | 3.05 | |
3 | 1 | 160 | 15 | 0.75 | 3.4 | 3.7 | 3.5 | |
4 | 1 | 180 | 15 | 0.75 | 4.4 | 4.2 | 4.3 | |
5 | 0.5 | 170 | 10 | 0.5 | 3.3 | 2.8 | 3 | |
6 | 0.5 | 170 | 10 | 1 | 2.4 | 2.4 | 2.4 | |
7 | 0.5 | 170 | 20 | 0.5 | 5 | 5 | 5 | |
8 | 0.5 | 170 | 20 | 1 | 4.5 | 4.8 | 4.7 | |
9 | 0.5 | 170 | 15 | 0.75 | 3.9 | 3.8 | 3.8 | |
10 | 0 | 170 | 15 | 0.5 | 3 | 3.4 | 3.2 | |
11 | 0 | 170 | 15 | 1 | 3.2 | 2.4 | 2.8 | |
12 | 1 | 170 | 15 | 0.5 | 5 | 3.8 | 4.4 | |
13 | 1 | 170 | 15 | 1 | 3.5 | 3.1 | 3.3 | |
14 | 0.5 | 160 | 10 | 0.75 | 2.2 | 2.4 | 2.2 | |
15 | 0.5 | 160 | 20 | 0.75 | 4 | 4.1 | 4.1 | |
16 | 0.5 | 180 | 10 | 0.75 | 2.9 | 2.6 | 2.8 | |
17 | 0.5 | 180 | 20 | 0.75 | 5 | 5 | 5 | |
18 | 0.5 | 170 | 15 | 0.75 | 4.1 | 4.2 | 4.15 | |
19 | 0 | 170 | 10 | 0.75 | 1.1 | 1.1 | 1 | |
20 | 0 | 170 | 20 | 0.75 | 3.8 | 3.6 | 3.85 | |
21 | 1 | 170 | 10 | 0.75 | 3.6 | 3.3 | 3.5 | |
22 | 1 | 170 | 20 | 0.75 | 4.8 | 4.6 | 4.7 | |
23 | 0.5 | 160 | 15 | 0.5 | 3.5 | 3.6 | 3.5 | |
24 | 0.5 | 160 | 15 | 1 | 3.2 | 3.2 | 3.1 | |
25 | 0.5 | 180 | 15 | 0.5 | 4.8 | 4.1 | 4.4 | |
26 | 0.5 | 180 | 15 | 1 | 4.4 | 3.6 | 4.1 | |
27 | 0.5 | 170 | 15 | 0.75 | 3.6 | 3.7 | 3.6 |
judges were used to taste the cupcakes and record their outputs for the “appearance”, “moisture” and “how-well cooked ” for each repetition of each cupcake. This time a 0.5 mark was included between each grading to allow for better distinction between similar cupcakes.
So in total the number of cupcakes baked for the detailed modelling was 27 by three or 81 bakes, with five scores for each bake making 405 test points―once again a lot of free tasting and advertising of the test rigour process within the workplace!
The multiple-response regression model from the test results is shown in… with the insignificant factors progressively removed for each response output.
The tolerance in
The uncoded equations for the three output responses are as follows:
“Moisture”:
y ^ = − 4.227 − 1.963 ⋅ Preheat + 0.009 ⋅ Tem p ' + 0.512 ⋅ Time − 0.378 ⋅ Fill + Preheat [ 0.046 ⋅ Temp p ' − 0.145 ⋅ Time − 1.400 ⋅ Fill ] − 1.498 ⋅ Preheat 2 − 0.008 ⋅ Fill 2
Y-hat Model | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Factor | Name | Moisture | Appearance | How well cooked | ||||||
Coeff | P(2 Tail) | Tol | Coeff | P(2 Tail) | Tol | Coeff | P(2 Tail) | Tol | ||
Const | 3.838 | 0.000 | − | 3.459 | 0.000 | - | 3.833 | 0.000 | - | |
A | Preheat | 0.553 | 0.000 | 1 | 0.747 | 0.000 | 1 | 0.475 | 0.000 | 1 |
B | Temperature | 0.318 | 0.000 | 1 | −0.019 | 0.805 | 1 | 0.324 | 0.000 | 1 |
C | Time | 1.015 | 0.000 | 1 | 0.375 | 0.000 | 1 | 0.885 | 0.000 | 1 |
D | Fill | −0.269 | 0.000 | 1 | −0.347 | 0.000 | 1 | -0.106 | 0.020 | 1 |
AB | 0.229 | 0.007 | 1 | - | - | - | 0.158 | 0.043 | 1 | |
AC | −0.363 | 0.000 | 1 | -−0.958 | 0.000 | 1 | -0.163 | 0.038 | 1 | |
AD | −0.175 | 0.036 | 1 | 0.333 | 0.017 | 1 | - | - | - | |
CD | − | − | − | - | - | - | 0.183 | 0.020 | 1 | |
AA | Preheat2 | −0.374 | 0.000 | 0.96 | 0.478 | 0.000 | 0.9 | −0.449 | 0.000 | 0.96 |
BB | Temperature2 | − | − | − | −0.264 | 0.020 | 0.9 | - | - | - |
CC | Time2 | −0.197 | 0.003 | 0.96 | - | - | - | −0.276 | 0.000 | 0.96 |
DD | Fill2 | − | − | − | -0.722 | 0.000 | 0.9 | - | - | - |
R2 | 0.913 | 0.794 | 0.902 | |||||||
Adj R2 | 0.902 | 0.768 | 0.889 | |||||||
Std Error | 0.283 | 0.470 | 0.266 | |||||||
F | 83.037 | 30.375 | 72.419 | |||||||
Sig F | 0.000 | 0.000 | 0.000 |
“Appearance”:
y ^ = − 81.202 + 3.333 ⋅ Preheat + 0.895 ⋅ Tem p ' + 0.267 ⋅ Time + 14.611 ⋅ Fill − Preheat [ 0.383 ⋅ Time − 2.667 ⋅ Fill ] + 1.911 ⋅ Preheat 2 − 0.003 ⋅ Temp 2 − 11.556 ⋅ Fill 2
“How-well cooked”:
y ^ = − 3.562 − 1.662 ⋅ Preheat + 0.017 ⋅ Tem p ' + 0.431 ⋅ Time − 2.622 ⋅ Fill + Preheat [ 0.032 ⋅ Tem p ' − 0.065 ⋅ Time ] + 0.147 ⋅ Time ⋅ Fill − 1.797 ⋅ Preheat 2 − 0.011 ⋅ Time 2
These equations show three different four-dimensional spaces that are hard to envisage. Fortunately DOE PRO XLTM provide some impressive tools to explore the effect space that has been modelled. By this stage in their course students usually know their test domain and the tools and they go competently to the optimizer and run “what if ” cases. Before showing that though, it is worth showing two graphs that illustrate two aspects of the cupcake baking model. The first in
A second graph in
heating was almost discarded in screening to try to get to a three factor test and the resultant model justifies the decision to keep it, as it is complex across all three cupcake ratings.
The optimization tool of DOE PRO XLTM was used to determine the optimal settings for cupcake baking for five cases as shown in
From
When variance isn’t important (Case 1) then Preheat and a short Cook Time is best, however when variance is to be minimized as well (Case 2) then no Preheat, a slightly lower Pan Fill and a long Cook Time is better.
If the baker is only concerned with the “appearance” of the cupcake, for example for a window display, then higher Oven Settings (Cases 4 & 5) are better than when “moisture” and “how-well cooked ” matter as well (Cases 1-3).
When variance isn’t important for “appearance” (Case 4), the same trend identified earlier of Preheat and a short Cook Time is evident compared to when consistency in appearance is to be minimized as well (Case 5) where no Preheat, a slightly lower Pan Fill and a long Cook Time is better.
Case 3 demonstrates the settings to enable the best compromise between all three outputs while also considering the consistency (variance) of all of the outputs. The predicted multiple-response confidence intervals for the Case 3 settings are in
If the cupcakes are to be iced then “appearance” is not important and only “moisture” and “how-well cooked” plus the consistency of those two measures needs to be optimized. The ideal settings are shown in Case Six, which are much the same as Case 3 but with a much lower Pan Fill and very good confidence intervals as shown in
optimized setting & weighting | |||||||
---|---|---|---|---|---|---|---|
(moisture = 3, Appearance = 5, How-well cooked = 3, variance = 0 when used) | |||||||
variables | case | 1 | 2 | 3 | 4 | 5 | 6 |
weighting of outputs | yes all equal | yes all 6 equal | yes all equal | appearance | appearance both mean & variance equally | moisture & how-well cooked equal | |
weighting of variances | no | yes all equal but ½ outputs | no | yes both equal but½outputs | |||
Preheat Pan? | yes | no | no | yes | no | no | |
temperature (degC) | 167.1 | 160.7 | 161.4 | 170.6 | 180 | 162.1 | |
Cooking Time(min) | 11.5 | 16.2 | 16.1 | 10.6 | 20 | 15.3 | |
Fill of Pan | 0.83 | 0.7 | 0.7 | 0.87 | 0.72 | 0.59 |
Multiple Response Prediction | 99% Confidence Interval | |||
---|---|---|---|---|
Y-hat | S-hat | Lower Bound | Upper Bound | |
Moisture | 3.150 | 0.108 | 2.825 | 3.475 |
Appearance | 3.442 | 0.155 | 2.976 | 3.908 |
How-well cooked | 3.000 | 0.000 | 3.000 | 3.000 |
Multiple Response Prediction | 99% Confidence Interval | |||
---|---|---|---|---|
Y-hat | S-hat | Lower Bound | Upper Bound | |
Moisture | 3.000 | 0.111 | 2.666 | 3.334 |
Appearance | 3.283 | 0.170 | 2.774 | 3.791 |
How-Well Cooked | 2.911 | 0.000 | 2.911 | 2.911 |
of the variety in the different baking settings. From general observations, the mushroom like muffin top was a result of a full Pan Fill (i.e., 1) and a low temperature oven. The cupcakes with the burnt edges were a result of the longer cook time and the spherical top to some was often a result of a hot, preheated cooking temperature. Note also that there is some variance between cupcakes in each row despite the cupcakes in each row being baked under the same settings.
A new post-graduate tertiary course in advanced test and evaluation techniques (experimental design) provided an opportunity for a more inclusive curriculum through structured collaborative learning on a fun learning device, followed by students having an open choice of a system from their work or personal interests to analyze themselves over the following months with mentoring from their teacher. This extended the curriculum to students’ “life-worlds” as proposed by Rasi et al. (2015) . Presenting on that choice to their peers and instructor further empowered students to share their “life-worlds” in ways that leveraged and enhanced the social aspect of learning and formed greater trust with the teacher for the part-time mentoring phase. Because students go on to also share their analyses with their work colleagues, hobby friends and family, their new knowledge is reinforced in personal ways entirely consistent with both Constructivist and Vygotsky educational theory (Udvari-Solner & Thousand, 1996) . The research method used in this case study was direct observation of the students’ enthusiasm and success, made possible because the teacher was also an experienced educational researcher and the number of students was small. The work is only offered as an encouraging example of curricular techniques to try for greater inclusion.
Using the showcased student’s work on cupcake baking, a female electronics engineer, after the fun collaborative learning, was able to bring her passion for cooking into the class and then, over the following months, and conducting 139 individual bakings, 753 judgings, and obtain 2259 judge ratings amongst her work and friends. This enabled her to share her new knowledge of advanced test techniques in a very personal way, which undoubtedly will have robust and enduring conceptions which she can use to benefit her future test work. Her example was not the only ones, another female engineer shared her passion for toy slot cars, and another aspiring female researcher brought her commercial business knowledge into the learning, helping breakdown difficulties in English-as-a second-language. This case study has reinforced that if STEM subjects are to appeal to non-traditional sources of students, then such structured fun learning and open contextualization are key. In this case, a common cooking effort has been analyzed with advanced test techniques and this should appeal to several non-traditional STEM markets. The social aspect of learning was not only beneficial for females, several fairly reclusive male students blossomed when bringing their hobbies into the class and then their classwork to their hobbies.
There are also other educational aspects at work in the new course as showcased in this article. The ability to explore complex systems with relatively easy-to-learn statistical and experimental design packages involving multiple visual analysis tools is highly effective computer-assisted learning for engineers and project managers, very analogous to the burgeoning use of finite-element modelling packages in research and teaching in the 1980’s and 90’s. As such, the inclusivity of the course is likely to extend to students of lower ability or who are more visual learners.
This case study in new curriculum for a complex STEM subject found the student-centred learning of collaboration, computer-based analysis, and an open student choice of personal research interests, to be highly inclusive in the ways proposed by the literature reviewed (Tait, 2009; Ashman, 2010; Koppi et al., 2010) , especially for gender (Wistedt, 1998) .
Joiner, K., & Brewster, A. (2017). Cooking Perfect Cupcakes: Freeing Curricula Context Gives Student-Centred Pedagogy for Course on Experimental Design. Creative Education, 8, 1833-1859. https://doi.org/10.4236/ce.2017.812126