Psychology
2012. Vol.3, No.10, 923-933
Published Online October 2012 in SciRes (http://www.SciRP.org/journal/psych) http://dx.doi.org/10.4236/psych.2012.310139
Construct-Validity of the Engagement with Challenge
Measure for Adolescents: Structural- and
Criterion-Validity Evidence
E. Whitney G. Moore1, David Hansen2
1Health, Sport & Exercise Sciences Department, University of Kansas, Lawrence, USA
2Psychology & Research in Education Department, University of Kansas, Lawrence, USA
Received July 18th, 2012; revised August 19th, 2012; accepted September 12th, 2012
For adolescents, engaging with challenge is a key developmental task, hypothesized to support develop-
ment of adult-like competencies (e.g., agency and self-direction; Larson, 2000). This study aimed to as-
sess the construct-validity (structural- and concurrent-validity) of a new self-report measure assessing
adolescents’ engagement with challenge to help researchers understand how different settings and the
conditions in these settings support adolescents’ development. The sample consisted of 337 adolescents in
10 FFA programs along with the adult advisors in each program. Adolescents completed a questionnaire,
which included the Engagement with Challenge measure and the following criterion variables: number of
contests completed, participation frequency, and leadership roles. In addition to the self-reported criterion
variables, the adult advisor evaluated Engagement with Challenge for each FFA student member in that
program using a single item. The findings of this study provided strong evidence for the structural-valid-
ity of the engagement with challenge construct measured by the new scale, including having passed con-
firmatory factor analysis configural, weak, and strong invariance tests across four grade groupings. The
findings also provided further evidence of criterion-validity, as Engagement with Challenge correlated in
the a priori hypothesized direction and magnitude. Suggestions for analysis with the new measure and for
future research are presented.
Keywords: Adolescent; Engagement; Challenge; Cognitive Development; Out-of-School Time
Introduction
There is a dearth of instruments assessing key adolescent
learned competencies related to self-initiated or self-governed
thought and behavior. As a form of intrinsic motivation (Csik-
szentmihalyi, 1990; Larson & Rusk, in press; Ryan & Deci,
2000), learning to become engaged with challenges is hypothe-
sized to be a key indicator of progress toward becoming an
agentic adult (Larson, 2000)—one who has developed a pro-
pensity to enjoy and become motivated by and absorbed in
challenges that emerge while working on goals or projects over
time. Learning to engage with challenge has also been hypothe-
sized by some scholars to promote enjoyment and thriving in
adulthood (Csikszentmihalyi & Rathunde, 1993). Relatively
recent neuroscience findings on brain develop suggests adoles-
cents may be primed to learn certain competencies, such as
engagement with challenge, although its development is not a
spontaneous or guaranteed outcome of adolescence. Engage-
ment with challenge, we suggest, is an increasingly valuable
commodity in contemporary economies, as suggested, for ex-
ample, by the Secretary’s Commission on Achieving Necessary
Skills (SCANS; 1991) report. Thus, having a valid measure of
engagement with challenge is an important step that can sup-
port research into how different experiences and settings of
adolescents’ lives support the development of this competency,
as well as others. The aim of this research, then, is to evaluate
the construct-validity of a new self-report measure of adoles-
cents’ engagement with challenge, focusing on evidence for the
structural-validity and criterion-validity of the measure.
Theoretical Foundation for Engagement with
Challenge
The theoretical roots for the engagement with challenge con-
struct stem from the literature on intrinsic motivation (Csik-
szentmihalyi, 1990, Larson, 2000; Ryan & Deci, 2000). At its
most basal level, humans possess an incentivizing system (e.g.,
Steinberg et al., 2006) for self-initiated behavior and thought to
attend to novel stimuli and explore their environment (Richards,
Reynolds, & Courage, 2010). When individuals experience
enjoyment or pleasure (reward) as a result of novelty seeking or
exploration they are more likely to be motivated (incentivized)
to repeat similar behaviors or thoughts. Theoretically, novelty
seeking and exploration has considerable adaptive utility when
it is linked to enjoyment, as it can propel learning and contrib-
ute to a sense of personal “control” of one’s environment (e.g.,
ability to predict and cause behaviors and events that lead to
goal attainment; Csikszentmihalyi , 1990; Hidi, 2006; Larson &
Rusk, in press; Panksepp, 1998; Ryan & Deci, 2009). Ryan and
Deci (2000) define intrinsic motivation as an “inherent ten-
dency to seek out novelty and challenges” (p. 70). Although
intrinsic motivation is couched in terminology suggesting it is
an innate or “hardwired” predisposition, we suggest it is more
consistent with recent neuroscience findings to view intrinsic

E. W. G. MOORE, D. HANSEN
motivation as developing from an interaction of the brain’s
incentivizing (e.g., positive emotion, enjoyment) and cognitive
regulating (e.g., future planning, goals, challenges) systems that
simultaneously encode experiences, which results over time in
enduring patterns of thought, emotion, and behavior consistent
with someone we describe as intrinsically motivated. Concep-
tualized this way, intrinsic motivation serves the evolutionary
function of promoting the widest range of learning for adapting,
surviving, and thriving within an environment that is in flux.
The engagement with challenge construct represents a par-
ticular form of intrinsic motivation, which Csikszentmihalyi
(1990) and Larson (2000) posit is a commonly shared charac-
teristic among those who develop skill and expertise in a par-
ticular domain, such as music or sport. Research suggests that
individuals who become self-motivated by challenges, com-
pared to those who are motivated by more extrinsic reasons,
tend to sustain and expend effort on tasks longer when faced
with difficulties, select more challenging tasks, learn from ex-
perience more proficiently, and find more creative solutions to
problems (Deci & Ryan, 2000; Larson & Rusk, in press). Addi-
tionally, scholars have suggested that learning to become
self-motivated by challenges can serve as a source of enduring
enjoyment, satisfaction, and thriving throughout adulthood
(Csikszentmihalyi & Rathunde, 1993; Ryan, 1995). Beyond the
personal benefits, contemporary work and society highly value
and reward individuals who demonstrate competency for en-
gaging with challenge, as it can lead to innovation, novel solu-
tions to problems, etc. (e.g., Secretary’s Commission on
Achieving Necessary Skills, 1991; SCANS). We speculate that
this learned competency was less integral to successful func-
tioning in past industrialized economies that relied on a work-
force with discrete skill sets that applied to a fixed set of tasks
and problems, but that today’s “high-performance” economies
has created a greater demand for individuals with competencies
for engaging with challenge. As espoused here then, engage-
ment with challenge is a learned competency consisting of per-
sistent linkages between enjoyment and challenges that arise in
pursuit of a goal or outcome; it is evidenced as an inclination or
propensity to enjoy and become motivated by and absorbed in
tasks’ challenges rather than motivated primarily by the extrin-
sic rewards of preforming a task (e.g., praise, public recogni-
tion).
Adolescence and Engagement with Challenge
While the functional importance of becoming skilled at en-
gaging with challenge in society may vary over time and cul-
ture, relatively recent findings from the cognitive and neuro-
science literature suggest adolescents may be “neuro-develop-
mentally primed” to learn such a competency. With the onset of
puberty, adolescents’ brains begin to undergo a major reor-
ganization or restructuring, which ultimately leads to the full
integration of emotion-motivation (e.g., incentivizing system)
and cognitive regulation (e.g., executive control) systems
(Keating, 2004; Luna et al., 2001; Spear, 2000; Steinberg et al.,
2006). The emotion-motivation system experiences a period of
heightened arousal, especially during early adolescence, result-
ing in greater emotional and motivational awareness, arousal,
and “excitatory tone” (Spear, 2000; Steinberg et al., 2006). The
cognitive regulation system itself (e.g., the PFC), which is
linked to higher-order thinking skills—such as developing
strategies for accomplishing complex work or managing one’s
time and effort—experiences a pruning of extraneous neurons
and increased myelination of axons (Griedd, 2004; Keating,
2004; Spear, 2000). (Myelination supports greater efficiency
and speed of synaptic firing and is an indication of learning that
has occurred over time). Perhaps most poignant for becoming
accomplished at engaging with challenge, the adolescent brain
develops new, more comprehensive axonal pathways between
the emotion-motivation and higher-order cognitive regulation
systems (Goldman-Rakic, 1988; Luna et al., 2001). These mye-
linated pathways represent an emerging functional integration
of two systems—each system able to contribute its own spe-
cialized learned functions and skills—that results in greater
self-governance over thought, motivation, behavior, and learn-
ing. We suggest, then, that adolescents’ brains are primed to
learn complex competencies, including engagement with chal-
lenge, which reflect the integration among these two systems.
Becoming skilled at and garnering enjoyment from engaging
with challenges, however, should not be viewed as a spontane-
ous outcome of contemporary adolescence. Such a process is
not an automatic developmental outcome of “normal” adoles-
cence. Experience is an essential, yet surprisingly overlooked,
influencer of brain development and functioning (Greenough,
Black, & Wallace, 1987; Markham & Greenough, 2004). In
laboratory settings, for example, stimulating or enriching envi-
ronments induce morphological changes in the brains of animals
(e.g., greater density of neural connections), which are associ-
ated with greater learning or brain plasticity (Markham &
Greenough, 2004). While a direct association between adoles-
cent brain development and experiences that support learning to
engage with challenge—or other positive competencies (e.g.,
strategic planning/thinking) for that matter—have not yet been
established, research on adolescents’ experiences in structured,
out-of-school youth programs suggest this is a setting that sup-
ports development.
A Setting That Supports Learning to Engage with
Challenges
One early study identified organized, out-of-school youth
programs as a setting in which adolescents reported experience-
ing both high levels of challenge and high levels of enjoyment
(Csikszentmihalyi & Larson, 1984). Subsequent research has
consistently pointed to these youth programs as a prominent
setting in adolescents’ lives—compared to other settings such
as class or hanging out with friends—in which they report
higher rates of a wide range of developmental experiences,
including challenge and enjoyment (Hansen, Larson, Dworkin,
2003; Hansen & Larson, 2007; Larson, Hansen, & Moneta,
2006). Qualitative research has also found that, over a course of
time between three to eight months, depending on the particular
youth program, adolescents describe becoming motivated to
take on greater challenges that emerged while working on pro-
jects, particularly projects that produced “real-world,” tangible
products, such as creating murals for public display (Larson &
Angus, 2011; Larson, Hansen, & Walker, 2005; Larson &
Walker, 2006).
There is no single activity or program that would support
every adolescent’s learning to engage with challenge, however.
Adolescents’ interests in activities, for example, change and
diversify over time (Hofer, 2011; Krapp, 1999). Thus, activities
or programs that may support learning to engage with challenge
for one adolescent may not be same for another. Similarly,
924

E. W. G. MOORE, D. HANSEN
engaging with challenge implies the tendency to enjoy and to
be motivated by increasingly more difficult challenges. Given
Western adolescents have considerable discretion over partici-
pation in activities and programs—some for example simulta-
neously participate in multiple programs (Larson, Hansen, &
Moneta, 2006)—they can select activities or programs that fit
with their current desire to experience a particular level of
challenge. Some programs offer progressively more difficult or
skilled activities, which could support an individual adoles-
cent’s increasing desire for greater challenge. Research, then,
suggests that organized youth programs tend to offer opportuni-
ties for adolescents to experience engagement with challenge,
and thus provide a reasonable setting in which to evaluate a
new quantitative measure of engagement with challenge.
Methods
Data come from a larger study of rural adolescents’ devel-
opment in FFA.1 The FFA is an agriculturally-based, national
youth program that offers high school students opportunities to
participate in different competitions (e.g., electrical wiring),
hold official elected offices (e.g., Chapter president), work with
peers and adults on projects, socially interact with adults and
other youth, as well as design and implement long-term agri-
culture-related projects (e.g., Supervised Agricultural Experi-
ence). Throughout the year, students have opportunities to plan
and implement events/projects (e.g., community service pro-
jects), set personal and group goals, and receive feedback on
their work. Participation in FFA activities is voluntary, and
adolescents are free to participate as much or as little as they
choose.
The FFA was selected for this study because it provides an
extensive array of activities that can appeal to a broad range of
adolescent interests and because activities vary in the degree of
training and challenge needed, which we reasoned could pro-
vide progressive opportunities to engage with challenge for a
wide adolescent population. FFA Chapters are located in high
schools and have at least one adult “advisor”, typically the ag-
ricultural teacher. FFA Chapter activities occur after school
hours, on weekends, and often during the summer months. Be-
cause the FFA curriculum and activity options are consistent
across Chapters, the focus only on the FFA program helped
reduce one potential source of systematic variance in the meas-
ured variables that could result from differences in the types of
youth programs (e.g., sports and arts).
Procedures
Convenience sampling, with purposive selection of FFA
Chapters, was used to select 10 out of a potential of 58 FFA
Chapters within a three hour driving distance from the univer-
sity in which the co-author was employed at the time.2 The
selection procedures were based on criteria on Chapters’ per-
formance on the “program standards” and “quality indicators”
collected by the overarching regulatory FFA body, Facilitating
Coordination in Agricultural Education (FCAE). All FFA
Chapters report on these indicators to the FCAE. We chose 28
indicators that reflected the degree a particular Chapter fol-
lowed the aims and curriculum of the FFA, and reflected the
degree to which a particular Chapter was engaged and active in
FFA: eight indicators of classroom instruction, four of the su-
pervised agricultural experience (SAE), and 16 of the FFA.
Collectively we refer to these indicators as Chapter “fidelity”.
A score for each chapter was computed (range 0 - 28) based
upon the number of standards and indicators a Chapter met, one
point for each indicator (1 = “met the standard”). Based on the
computed scores, the 58 chapters were divided into three
groups: Chapters with 20 or more points, 15 - 19 points, and
less than 15 points. Chapters scoring less than 15 points were
excluded from selection because we reasoned that they would
not provide sufficient enough opportunities for adolescents to
experience engagement with challenge, and because the pur-
pose of the larger study was to examine the impact of students’
active participation (e.g., greater dosage) in the program. Fi-
nally, only those Chapters with (a) at least one agriculture in-
structor who had been teaching for three years or more in the
current school and (b) had a Chapter membership of least 40
members were included. These two final criteria ensured advi-
sors had sufficient experience leading an FFA Chapter and that
there were sufficient student participants for statistical analysis.
After applying the criteria, 16 chapters were eligible for selec-
tion: eight had 20 or more points and eight had 15 - 19 points.
We then randomly selected five chapters from each of the two
groups of eight for this study.
Data were collected from both the student FFA members and
the adult advisors of each Chapter; questionnaires were given to
students and advisors during the spring of 2006 (Time 1) and
2007 (Time 2). Two weeks prior to the data collection, an in-
formational flyer was sent to parents of FFA members of a
given Chapter along with instructions for declining a child’s
consent if a parent so desired. Adolescent assent was obtained
on the day of the questionnaire administration and FFA advi-
sors provided their consent. While this study focuses solely on
evaluating the construct-validity of the new Engagement with
Challenge measure at one point in time, data from both collec-
tion times were used to best inform the multiple imputation
process (Little, in press; Wu, Lang, & Little, 2009). Evaluating
the measure longitudinally in this study, however, was deemed
premature and was reserved for a future study. In addition,
simultaneously including longitudinal analysis in this study
would expand its scope and length and address a fundamentally
different research question, the longitudinal invariance of the
Engagement with Challenge construct, which requires cross-
sectional structural-validity. Thus, this analysis of the con-
struct-validity of the measure, focusing on structural- and crite-
rion-validity evidence, analyzed only data from Time 1.
Participants
Students
The sample consisted of 337 high school students (67% male)
from 10 FFA Chapters. The mean age of participants was 16.21
(range = 14 to 19). There were 121 freshman, 60 sophomores,
54 juniors, and 102 seniors. The majority of students, 60.7%,
lived in a rural area (“on a farm in the country” or “not on a
farm but in the country”), 37.7% reported living “in a small
town or city (less than 10,000 people),” and 1.6% of students
reported living in a “medium size city (between 10,000 and
1FFA stands for Future Farmers of America but the national FFA organization
no longer include the full title, preferring simply the FFA.
2Lead author was employed at the University of Illinois at the time the larger
study was conducted.

E. W. G. MOORE, D. HANSEN
200,000 people).” No students reported living in a “large city
(more than 200,000 people).” The sample was 85.8% White,
2.7% Native American, 0.3% Hispanic, 6.0% reported multiple
ethnicities or “other”, and 5.2% chose not to respond. Students
self-reported this demographic information.
Adult Advisors
Eleven adult advisors participated in this study; one school
had two advisors. Within each FFA Chapter, the adult advisor
rated each youth’s engagement with challenge. In the one
Chapter with two advisors, each advisor rated half of the youth.
Ten of the 11 advisors were male and all were White. The av-
erage number of years these advisors had been teaching was
28.82, with a range from 5 to 37.
Measures
Engagement with Challenge
The engagement with challenge scale was developed from
the qualitative research of Larson and colleagues (Larson &
Hansen, 2005; Larson, Hansen, & Walker, 2005; Pearce &
Larson, 2006). In that research, organized youth programs and
their adolescent participants and adult leaders were studied over
a naturally occurring cycle of the program (e.g., academic year).
Observational, interview, and survey data were collected. From
the emerging grounded-theory of that qualitative work, items
for the engagement with challenge scale were generated. The
aim was to create a concise measure that not only captured the
theoretical, higher-order linkages between intrinsic motivation
and challenge but also to use adolescents’ language and expres-
sions that reflected the construct; for this latter aim, items drew
directly on youths’ words from the interview data when possi-
ble. Nine items for the scale were originally generated from the
qualitative work. In a different study with a group of 20 ado-
lescents attending a conference at a university, we asked par-
ticipants to complete the scale and comment on any items that
seemed awkward or unclear. Once completed, a researcher met
with participants in groups of 4 - 5 students to elicit their feed-
back about the items. Based on the students’ feedback, three of
the nine items were deemed to be confusing or vague and thus
were dropped. As a result of this vetting process, the final scale
comprised six items, three that were positively worded and
three that were negatively worded.
The engagement with challenge scale is a self-report measure
of a form of intrinsic motivation, assessing the adolescents’
enjoyment of the challenges that occur while working on a goal
or project. Students were given the following instructions,
“Please read each statement and circle the number that is most
correct about your participation in FFA in the last 12 months”.
The six items in the scale are: 1) There are always things I’m
trying to work on and achieve in this program; 2) I feel chal-
lenged in a good way in this program; 3) The activities in this
program are boring; 4) I’m not working toward anything in this
program; 5) What we do in this program is both difficult and
enjoyable; 6) The goals people are working on in this program
are not important to me. Items 3, 4, and 6 are reverse coded.
Students’ indicated agreement with each statement on a 6-point
scale: 1 = “Strongly Disagree”, 2 = “Disagree”, 3 = “Slightly
Disagree”, 4 = “Slightly Agree”, 5 = “Agree”, 6 = “Strongly
Agree”.
Criterion Va riables
Student-Reported Lead Role
The FFA offers multiple opportunities for students to serve
in a leading capacity at the Chapter, state, and national level
(e.g., Chapter president, district representative). Holding a lead
role represents a substantial investment of time and effort, as
well as a commitment to the FFA culture. As such, holding a
lead role should also correlate with student’s level of engage-
ment with challenge. Students were asked to indicate, “yes” (1)
or “no” (0), if they had a lead role in the FFA Chapter. Holding
a lead role was hypothesized to be moderately and positively
correlated with scores on engagement with challenge.
Student-Reported Freque ncy of Participation
Conceptually, a students’ level of engagement with challenge
in the FFA Chapter should be reflected in the amount of time
they spend in the activities. One indicator of time is the fre-
quency of students’ participation in the FFA Chapter. Fre-
quency of participation was assessed with one item: “How of-
ten did you participate in FFA in the last 12 months?” Response
categories were, 1 = “Less than once a month”, 2 = “Once a
month”, 3 = “A few times a month”, 4 = “A couple of days a
week”, and 5 = “Almost every day”. More frequent participa-
tion in the activities of the FFA Chapter was expected to mod-
erately correlate with greater engagement with challenge.
Student-Reported Participation in Competitions
The FFA offers competitions in which students demonstrate
skills and knowledge, such as electrical wiring, at the local,
regional, state, and national levels. Because of the additional
time commitment required to participate in FFA contests, it was
expected to positively and moderately correlate with engage-
ment with challenge. Students reported on each FFA contest in
which they participated (1 = participated, 0 = did not partici-
pate). The total number of contests was computed by summing
across each student’s reported contest participation to make a
single sum variable.
Advisor Ratings of Each Student’s Engagement with
Challenge
Advisors reported their perception of each student’s engage-
ment with challenge compared to other same-age peers. Each
advisor rated every student in the chapter using a single item.
Given the number of FFA members in most Chapters, it was
not reasonable to ask the advisor to rate every student member
in a manner equivalent to the student-report measure with six
items. The advisor was given the following definition: “En-
gagement with challenge refers to how motivated and engaged
a student has been in challenging activities of FFA. For en-
gagement with challenge, rate how each youth compares to
other youth their age using the scale below.” Each student’s
name was listed by the scale and the advisor ranked each stu-
dent by deciles from 1 = “0% - 10%”, the lowest decile, to 10 =
“91% - 100%”, the highest decile. A student ranked in the 50th
percentile, for example, would indicate that the advisor consid-
ered the student to demonstrate an average level of engagement
with challenge compared to other students his or her age. This
response scale was used to provide the advisor with as objective
reference points as possible and to increase the likelihood of
approximating a normal distribution with a single item. While
not an equivalent measure to students’ reports on the engage-
926

E. W. G. MOORE, D. HANSEN
ment with challenge measure, advisors’ ratings were expected
to have a moderately positive correlation with students’ re-
ports.
Results
All analyses were completed within a structural equation
modeling (SEM) framework. Preliminary analyses were first
conducted in order to analyze the data’s tenability for meeting
parametric analysis assumptions and to compute descriptive
statistics. Confirmatory factor analysis (CFA) for engagement
with challenge followed the preliminary analysis to evaluate the
structural-validity (Clark & Watson, 1995; Loevinger, 1957) of
the scale—testing the general fit of the hypothesized construct’s
measurement model and if the instrument measured the same
construct across grade groups. SEM analyses were then con-
ducted to examine the concurrent-validity of the engagement
with challenge scale by assessing the construct’s correlations
with the criterion variables.
Preliminary Analysis
The preliminary data evaluation indicated that the data had
1% missingness and that the highest fraction missing for any
individual item was .053. Given this high recoverability of the
missing data, modern missing data techniques were employed
(Little, 2010). Specifically, multiple imputation was conducted
using Amelia (R Development Core Team, 2010). Multiple
imputation was selected in order to retain the maximum sample
N (e.g., no listwise deletion of cases), while producing more
accurate and stable estimates of standard errors—that are nei-
ther inflated nor left out of the estimation process (Little,
2010)—even compared to the full information maximum like-
lihood method (Olinsky, Chen, & Harlow, 2003). One set of
sufficient statistics were then calculated across the 100 imputed
datasets to use in all subsequent SEM analyses (Little, 2010),
which were completed using Mplus, Version 5.21 (Muthen &
Muthen, 2009). Table 1 displays the descriptive statistics used
in the SEM analyses. Stronger positive correlations existed
between the three positively worded engagement with chal-
lenge items than between these three items and the negatively
worded engagement with challenge items, and vice versa (See
Table 2).
Measurement Invariance (CFA)
The purpose of this important step—establishing measure-
ment invariance—is to confirm that the latent construct’s
measurable manifestations (e.g., scores on the items) were ef-
fectively measuring the same construct across the four grade
groups (i.e., 9th, 10th, 11th, and 12th grade). Thus, before SEM
analyses can be conducted to examine differences in variances,
covariances or means, the measurement model must be estab-
lished to ensure subsequent comparisons across groups are
accurately based on comparing the same construct of interest.
As the final model attained from the CFA is then used for the
SEM analysis, the criterion variables were included throughout
the CFA process. However, as the criterion variables were all
single-indicators, their measurement invariance was not tested;
their inclusion was solely in preparation for evaluating the cri-
terion validity of the engagement with challenge scale after the
CFA was complete. Therefore, the emphasis and results of the
invariance testing is on the findings for the engagement with
challenge construct across the four grade groupings.
Configural Model
The initial CFA step was to establish the appropriate meas-
urement or configural model. The configural model is the hy-
pothesized model for the measure of interest (i.e., engagement
with challenge). If an initially hypothesized configural model
does not have adequate model fit, then either alternatively hy-
pothesized models or post-hoc adjustments that align with the-
ory should be made. The configural model included the three
positively worded and the three negatively worded engagement
with challenge items (e.g., items 3, 4, and 6) after being reverse
coded to positively load on the engagement with challenge
construct. The measurement model for a single latent construct
demonstrated (with all items coded in the same direction) in-
adequate fit (RMSEA = 0.128, 90%CI: .111 - .146; TLI = .859;
Table 1.
Means and standard deviations by grade.
9th Grade 10th Grade 11th Grade 12th Grade
1Engagement with Challenge (EWC) Items M SD M SD M SD M SD
Item 1 4.17 1.24 3.90 1.34 3.94 1.33 3.86 1.30
Item 2 4.32 1.34 4.06 1.36 4.08 1.24 4.30 1.17
Item 3 4.51 1.54 4.54 1.41 4.43 1.42 4.75 1.31
Item 4 4.82 1.37 4.66 1.46 4.55 1.40 4.50 1.36
Item 5 4.34 1.31 4.22 1.31 4.39 1.10 4.41 1.13
Item 6 4.79 1.35 4.66 1.35 4.61 1.34 4.71 1.18
Frequency of Participation 2.65 1.30 2.82 1.15 2.82 1.23 3.04 1.32
2Advisor Rating of Student’s EWC 5.44 2.48 5.66 2.49 6.20 2.43 7.13* 2.30
Lead Role 5.05 2.37 5.36 2.42 5.88 2.43 6.98* 2.33
Contest Number 1.55 2.29 1.97 2.67 1.93 2.21 2.75* 2.78
Note: 1Response on a 1 - 6 scale; 2Response on a 1 - 10 scale; *The 12th grade latent means were significantly different than the 9th, 10th, and 11th grade latent means.

E. W. G. MOORE, D. HANSEN
Table 2.
Bi-variate correlations among study variables across all grades (N = 337).
1 2 3 4 5 6 7 8 9 10 11
1) Grade --
2) Eng1 (EwC) −.16** --
3) Eng2 (EwC) −.05** −.62** --
4) Eng3+ −.06** −.29** −.39** --
5) Eng4+ −.10** −.46** −.42** .55** --
6) Eng5 (EwC) −.01** −.47** −.61** −.46** −.41** --
7) Eng6+ −.02** −.34** −.31** .51** −.53** −.41** --
8) Frequency −.09** −.42** −.27** −.14** −.30** −.20** −.17** --
9) Advisor −.19** −.24** −.22** −.21** −.23** −.29** −.18** .26** --
10) Contests −.07** −.22** −.12** −.08** −.19** −.18** −.03** .26** .31** --
11) Leadership −.38** −.26** −.17** −.15** −.20** −.19** −.16** .39** .42** .31** --
Note: +Identifies the eventual non-engaged with challenge construct’s items. *p < .05, **p < 001.
CFI = .909). A second model was then fit with the negatively
worded items’ residual variances correlated to account for the
methodological similarity of these three items. This decision is
a supported approach for handling “method variance” shared
among specific indicators. Additionally, the negatively worded
items greater correlation with each other than with the posi-
tively worded items provided evidence that this methodological
specific variance existed. The updated measurement model re-
sulted in improved model fit (RMSEA = .054, 90%CI: .021 - .079;
TLI = .975; CFI = .986). Despite this improved model fit and
the non-significant p-value of the Chi-square model value, ex-
amination of the model parameters revealed that item 6 consis-
tently had a factor loading below .5 across the four grade
groupings and a large residual variance. By comparison, the
rest of the factor loadings were .62 and above, which sug-
gested that item 6 did not add much to the measurement of the
latent construct (i.e., was not an effective manifestation of the
construct). Therefore, a configural measurement model was fit
with only items 1 - 5 loading onto the latent construct. Results
with this revised model improved, albeit slightly, model fit
(RMSEA = .054, 90%CI: .013 - .082; TLI = .978; CFI = .988)
and, more importantly, it resulted in consistent strong loadings
for all five items and across all four groupings (Table 3, Panel
A provides the fit indices for each of these configural models).
This final five-indicator configural model, then, was used for
all subsequent invariance testing.
Weak and Strong Invariance
Invariance testing was conducted to assess the consistency of
the engagement with challenge’s measurement across the four
grade groupings. Weak invariance tests the tenability of equat-
ing (i.e., constraining) the construct loadings across multiple
groups, and strong invariance tests the tenability of equating the
construct intercepts (e.g., item means) across multiple groups.
The quality of these constraints is best assessed by the change
in CFI being less than .01 compared to the prior, less con-
strained model’s CFI (∆CFI < .01; Cheung & Rensvold, 2002).
The invariance tests were completed across the four grade-
levels (9th, 10th, 11th, and 12th grade), with grade 9 as the
reference group (see Table 3, Panel B for model fit indices).
The weak invariance model indicated that enforcing the
equality constraint of the item loadings did not significantly
diminish the model’s fit (∆CFI = .004), that is, the engagement
with challenge’s factor loadings were equivalent across grade
groups. There was sufficient evidence, then, supporting the
tenability of weak invariance—constraining each of the en-
gagement with challenge’s five items’ loading across the four
grade groupings. Test of the scale’s strong measurement in-
variance—equating the item intercepts (i.e., individual item’s
means) across the four grade-groupings—was conducted next
(Table 3, Panel B and Figure 1). Results indicated the strong
invariance model’s constraints were also tenable based upon the
change in CFI (∆CFI = .006). Therefore, no systematic differ-
ence on item means between grade groups was evidenced.
Overall, engagement with challenge passed these CFA invari-
ance tests, providing support that the latent variable was meas-
ured consistently across grade groupings.
At this point in the analysis, the reliability of the five-item
engagement with challenge scale was calculated using equa-
tions that take the CFA factor loadings and residual variances
into account. Two assessments used for examining a latent
construct’s internal reliability are the average variance ex-
tracted (minimum criterion value is .5; Zinbarg, Revelle, Yovel,
& Li, 2005) and the composite reliability (minimum criterion
value is .6; Zinbarg, Revelle, Yovel, & Li, 2005). The strong
invariance model’s standardized loading and residual variance
values for the engagement with challenge’s five items were
used in calculating both of these reliability test values. The
average variance extracted passed with a .55 value, and the
composite reliability passed with a .85 value. Thus, both tests’
values provided additional support for the good reliability of
the engagement with challenge measure.
Testing for Homogeneity of Variance/Covariance
Prior to evaluating the criterion-validity of the engagement
with challenge measure—i.e., correlating the measure with the
criterion variables—the variances and covariances of the engage-
ment with challenge construct acoss the four grade groupings r
928

E. W. G. MOORE, D. HANSEN
Table 3.
Measurement model invariance and homogeneity testing.
Model χ2 (df) p ∆χ2 (∆df)p RMSEA90%CI TLI CFI ∆CFI
Maintain
Fit, Tenable
Constraint?
Panel A—Configural Models
Null Model, 6 items 2284.02
(180) 0 -- -- .342 .330 - .3550 0 --
Configural Invariance 6 items 306.71
(116) 0 -- -- .128 .111 - .146.859 .909 -- Not adequate
Configural Invariance 6 items, negcorr 134.25
(104) .025 -- -- .054 .021 - .079.975 .986 +.077 Yes
Configural Invariance, 5 items, negcorr 103.58
(80) .039 -- -- .054 .013 - .082.978 .988 +.002 Yes
Configural Invariance, 5 items, no corr 216.25
(84) 0 -- -- .126 .105 - .146.882 .931 -.057 No
Panel B—Measurement Invariance & Homogeneity Summary
Weak Invariance, 5 items with 1 negcorr 134.02
(104) .025 30.44 (24) .17 .054 .020 - .079.978 .984 -- Yes
Strong Invariance, 5 items with 1 negcorr; eng
items constrained only
157.29
(116) .007 23.27 (12) .03 .060 .033 - .092.973 .978 -- Yes
HomogenietyVar&Cov Omnibus Test 189.90
(149) .013 32.62 (33) .49 .052 .025 - .074.979 .979 -- Yes
Homogeneity of Latent Means: Engagement
(9 - 12), Frequency (9 - 12), Leadership (9 - 11),
Advisor Ratings (9 - 11) & Contests (9 - 12)
216.36
(162) .003 8.90 (3) .03 .058 .035 - .077.975 .972 -- Yes
Figure 1.
Strong invariance model for engagement with challenge. Note: This
model shows the standardized values from the strong invariance model
for the engagement with challenge construct. Constraining the loadings
and the intercepts across the grade groups was tenable; therefore only
one value for each of these parameters is shown.
needed to be tested. If the variances were found to not be ho-
mogeneous, then the variability of the participants’ report of
engagement with challenge was being moderated by their grade.
That is, there would be systematic variance in the engagement
with challenge construct associated with grade.
The tenability of equatable variances and covariances of the
engagement with challenge construct across all four grade
groupings was assessed with an omnibus test of the chi-square
difference for nested models. This omnibus test was used be-
cause it is the most conservative test of model fit difference
available and is the most appropriate test for latent space sig-
nificance testing since the latent parameter estimates are unbi-
ased (Kline, 2012). The Chi-square difference test criterion was
with an alpha value of .001. Results of this omnibus test indi-
cated that constraining the variances and covariances did not
lead to a significant change in model fit ( = 32.62, p
= .49). Therefore, this more parsimonious model was supported,
providing evidence for the homogeneity of the engagement
with challenge construct’s variances/covariances across grades,
which allows for the criterion-validity tests.
2
(33)

Criterion-Validity
Criterion-validity was tested by evaluating the correlations
between the engagement with challenge construct and the fol-
lowing criterion variables: 1) FFA student leadership role held;
2) student’s frequency of participation; 3) number of contests
the student entered that year; and 4) the FFA Advisor’s percep-
tion of engagement with challenge score for each individual
student. The direction and magnitude of theses correlations was
hypo thesi zed a priori. As hypothesized, results indicated that
the engagement with challenge construct was significantly cor-
related with each of the criterion variables and was in the ex-
pected direction and magnitude. For example, frequency of
participation was moderately and positively correlated with the
latent engagement with challenge construct (r = .490; 2
(1)

 =
89.0, p < .001). The criterion variables were also significantly
correlated with each other. Figure 2 depicts the correlations
among the latent engagement with challenge construct and the
criterion-variables. These correlaons then offer support for the ti

E. W. G. MOORE, D. HANSEN
.20
9
Figure 1.
Engagement with challenge & criterion validity variables correlation model.
criterion-validity of the engagement with challenge measure.
The final set of analyses evaluated the homogeneity of the
model’s means across grades. The final model from the previ-
ous step was used to evaluate (still using a nested model ap-
proach) if the means across grade groupings for both the latent
engagement with challenge mean and the means for the crite-
rion variables were equatable. The homogeneity of the means
model was found to fit the data significantly worse than the
parent model (homogeneous variance-covariance model;
( = 42.61, p < .001). Follow-up tests, however, indicated
there were significant mean-level differences by grade only for
two of the single-indicator criterion variables and not for dif-
ferences between latent means by grade for the engagement
with challenge construct.
2
(14)

Discussion
The purpose of this study was to evaluate the construct-va-
lidity of a new self-report instrument of Engagement with
Challenge for adolescents. This study evaluated two types of
construct validity: structural and criterion. The theoretical focus
of the engagement with challenge construct is on an adoles-
cent’s intrapersonal synchronous pairing of motivation with
challenges, within the context of working on tasks, projects, or
goals. Overall, the findings provided good initial evidence for
the construct-validity of this five-item Engagement with Chal-
lenge measure.
Structural-Validity
Evidence for the structural-validity of the Engagement with
Challenge measure was demonstrated through the fit of the
measurement model and the achievement of measurement in-
variance across grade groupings. In trying to fit the hypothe-
sized measurement model with the original six items to the data,
the analyses indicated inadequate fit. There were two sources of
this misfit.
Analysis revealed that the first source of misfit was attribut-
able to the inclusion of negatively and positively worded items
(i.e., “method effects”). There is a growing research literature
suggesting that wording half of the items in a scale in the oppo-
site way of the remaining items (i.e., negatively worded) may
result in statistical findings inappropriately indicating two latent
factors when they were a priori conceptualized as single,
unidimensional factor (Chen, Rendina-Gobioff, & Dedrick,
2020; DiStefano& Motl, 2006; Marsh, 1996; Schriesheim,
Eisenbach, & Hill, 1991). Analysis to specifically examine this
possibility revealed the presence of method effects from inclu-
sion of negatively worded items. Removing this source of
variance by allowing the residuals for the negatively worded
930

E. W. G. MOORE, D. HANSEN
items to correlate resulted in a good fit of the hypothesized
model to the data.
However, examining the model parameters suggested an ad-
ditional source of model misfit also be tested. Specifically, the
model estimated parameter values indicated item six was po-
tentially distinct from the other items (e.g., large residual vari-
ance). Inspection of the wording of this item provided insight as
to why it was not a reasonable indicator of the construct. The
item reads, “The goals people are working on in this program
are not important to me.” The focus of this item, then, is on
importance of other people’s goals in relation to self rather than
on an individual’s pairing of his or her own motivation to chal-
lenges that emerge in pursuit of a program’s goal or project.
The latter of which is the theoretical basis of the construct.
Removal of item six had minimal impact on the model’s fit
statistics supporting that it was not informing the estimation of
the engagement with challenge construct. Thus, the five-item
engagement with challenge construct provided a strong and
parsimonious explanation of the data, offering initial evidence
for structural-validity of the measure.
An important further step in establishing the structural-va-
lidity was to determine if the Engagement with Challenge in-
strument measured the same latent construct for each of the
four grade groupings (i.e., invariance). The CFA findings pro-
vided good evidence for the invariance of the construct across
grade groupings—there were equivalent indicator loadings and
intercepts. This provided evidence that the Engagement with
Challenge instrument measured the same latent construct for
each of the four grade groupings. Overall then, these initial
findings provided strong evidence for the structural-validity of
the five-item Engagement with Challenge measure. These CFA
invariance results provide important support for the measure-
ment model of a construct that should always be reported be-
fore the SEM analysis can be conducted and the results inter-
preted with confidence.
Criterion-Validity
The results of this study also provided evidence for the crite-
rion-validity of the Engagement with Challenge measure. The
latent engagement with challenge factor was significantly cor-
related in the hypothesized (a priori) direction and magnitude
with the criterion-variables, r = .31 - .56; or sharing 10% - 32%
of the variance. While the correlations between three of the four
criterion variables were self-report, the correlation between the
adult advisor’s ratings of a student’s engagement with chal-
lenge supports the interpretation that the criterion finding are
not attributable to self-report biases. In addition, the three self-
report criterion variables were based on tangible (as possible)
behaviors, such as holding a lead role in the program, which
were less likely to be influenced by attitudes or ability to accu-
rately report. A particular challenge faced in this study, how-
ever, was that the criterion variables were all single indicators,
which limited the ability to evaluate their construct validity.
It is important to note the strong correlations between the ad-
visor’s reports of each student’s engagement with challenge and
students’ self-report of holding a lead role (r = .86 - .91). Fur-
ther, the correlation between students report of engagement
with challenge and lead roles was much smaller (r = .38 - .40).
This pattern suggests that advisors may rely on a student hold-
ing a lead role as a “proxy” or indicator of his or her engage-
ment with challenge (the correlation between advisor’s ratings
and the other criterion variables is comparatively much smaller).
Given the intrapersonal (psychological) nature of engagement
with challenge, which is beyond the direct observation by oth-
ers, this pattern suggests that the individual adolescent may
provide a more direct assessment of this learned disposition.
Construct validity, evidenced in many forms (e.g., structural,
criterion), is built over time, and no single study is sufficient to
infer validity (Clark & Watson, 1995). Evidence in this study
for the structural-validity of the new Engagement with Chal-
lenge measure is promising. There is, however, additional re-
search is needed with the measure in order to expand the sup-
port for its structural- and criterion-validity. For reasons out-
lined in the methods section, this was a homogeneous sample of
rural high school students in one type of youth program.
Therefore, an obvious need is to replicate the structure of the
measure and the patterns of associations with other additional
variables using a more diverse sample of adolescents. In theory,
the engagement with challenge construct should be salient for
adolescents across socio-economic and geographic regions. It
will also be important to evaluate the construct-validity with
younger adolescents, for example in grades 6 - 8, to understand
if the instrument measures the same latent construct or if there
are developmental limits and differences.
Research with the Engagement with Challenge
Measure
Early evidence from this study of construct-validity of the
Engagement with Challenge measure suggests it has utility for
future research. In theory, engagement with challenge is an
important developmental task of adolescence (Larson, 2000).
Quantitatively understanding its developmental course, then, is
an important research question to address. Research on adoles-
cent brain development suggests that early adolescents experi-
ence a heightened period of motivational-emotional arousal
(Keating, 2004; Spear, 2000), which could provide an impetus
for wanting experiences that satisfy this state. Do younger ado-
lescents, then, experience greater engagement with challenge
compared to older adolescents; and are they more likely to ex-
perience it across multiple settings or activities? We reason that
this motivational-emotional biological “push” could serve to
increase the chances that early adolescents will learn to engage
with challenge across settings, which could allow them to ex-
plore a range of interests and activities through which they can
develop, over-time, more specific pursuits. A related question is
if engagement with challenge is a general competency that is
transferable across settings of individuals’ lives, or if it is a
state that is setting dependent. Research could evaluate adoles-
cents’ engagement with challenge across multiple settings or
activities, which could offer insight its stability.
Because, in theory, engagement with challenge results from
experiences in different settings, future research could also
examine how the particular environment or climate affects its
development over time. Research literature on intrinsic motive-
tion (Deci & Ryan, 1985; Larson & Rusk, in press; Ryan &
Deci, 2000; Vallerand, 1997) suggests that autonomy suppor-
tive environments (e.g., where individuals have control and
choice over activities) could promote the development of en-
gagement with challenge. Similarly, a rich research line on the
climate of sports teams and academic settings suggests that a
mastery/task-involving climate can enhance intrinsic motive-
tion and enjoyment (Cury, Elliot, Sarrazin, Da Fonseca, & Rufo,

E. W. G. MOORE, D. HANSEN
2002; Elliot, Cury, Fryer, & Huguet, 2006; Morris & Kavus-
sanu, 2009; Nicholls, 1989). We would expect then, that a
highly mastery/task-involving climate would support the de-
velopment of engagement with challenge.
This study’s findings offer evidence for construct-validity
and suggest that future research evaluate the antecedents, cor-
relates, and consequences of engagement with challenge. Al-
though important in its own right, engagement with challenge
has conceptual ties to other skills or competencies. We suggest
sustained engagement with challenge in a setting could grant
adolescents access to other experiences for building skills. In a
qualitative study of the development of adolescents strategic
planning skills, Larson and Hansen (2005) reported that, as
high school adolescents in a civic activism program became
empowered and motivated to overcome the challenges of in-
fluencing the Chicago School board, they began to develop
strategic planning skills (e.g., thinking that involves “system
processes”) for how to accomplish the goals of their project.
Thus, future research could explore the longitudinal relation-
ships between engagement with challenge and other skills, such
as strategic planning.
Practical Implications and Limitations
Overall, the findings of this study provide promising early
evidence for the construct-validity of the Engagement with
Challenge measure. As with all cross-sectional data, however,
the longitudinal invariance of the construct still needs to be
assessed. In addition, this study offers little guidance on what
should be the appropriate time interval between measurement
points when using this instrument. The measurement time in-
terval that is best for capturing the influence of engagement
with challenge on adolescent’s higher-order skills is an impor-
tant research question to answer in order to be able to more
accurately increase our understanding of how these skills are
learned.
Finally, there are at least two practical implications of this
study. The method effect attributed to the inclusion of nega-
tively and positively word items of the same construct implies
that researchers using this measure need to statistically account
for the correlated residuals among the two negatively worded
items in order for the latent construct to retain its intended
meaning. Fortunately, this can be more easily accomplished
with modern statistical approaches, such as SEM used in this
study, but it poses problems for statistical methods (e.g., stan-
dard linear regression) that are unable to account for this rela-
tion. A practical advantage of this new measure is that it is
short, only five items, which increases its ease of use for re-
searchers and practitioners with minimal risk of fatiguing ado-
lescent participants. Overall then, the findings of this study
suggest that the utility of this measure is promising for ad-
dressing salient research questions related to adolescents’ nor-
mative development.
Conclusion
This study addressed a specific quantitative measurement
need faced by researchers: assessing salient developmental
tasks of adolescence. The findings provided support for the
construct validity of Engagement with Challenge. The meaning
and impact of engaging with challenge can now be further in-
vestigated to understand how characteristics of different set-
tings in adolescents’ lives support or hamper its development.
Acknowledgements
We are grateful to colleagues and reviewers who provided
comments and suggestions for improvement. Special acknowl-
edgement is given to one reviewer’s comments regarding test-
ing for method effects.
REFERENCES
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit
indexes for testing measurement invariance. Structural Equation
Modeling, 9, 233-255. doi:10.1207/S15328007SEM0902_5
Clark, L. & Watson, D. (1995). Constructing validity: Basic issues in
objective scale development. Psychologica l Assessment, 7, 309-319.
doi:10.1037/1040-3590.7.3.309
Csikszentmihalyi, M., & Larson, R. W. (1984). Being adolescent. New
York: Basic Books.
Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experi-
ence. New York: Harper Perennial.
Cury, F., Elliot, A. J., Sarrazin, P., DaFonseca, D., & Rufo, M. (2002).
The trichotomous achievement goal model and intrinsic motivation:
A sequential mediational analysis. Journal of Experimental Social
Psychology, 38, 473-381. doi:10.1016/S0022-1031(02)00017-3
Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-de-
termination in human behavior. New York: Plenum.
Eccles, J., Barber, B., Stone, M., & Hunt, J. (2003). Extracurricular
activities in adolescent development. Journal of Social Issues, 59,
865-889. doi:10.1046/j.0022-4537.2003.00095.x
Eccles, J., & Gootman, J. A. (Eds.) (2002). Community programs to
promote youth development. Committee on community-level pro-
grams for youth. Washington: National Academy Press.
Elliot, A. J., Cury, F., Fryer, J. W., & Huguet, P. (2006). Achievement
goals, self-handicapping, and performance attainment: A mediational
analysis. Journal of Sport a nd Exercise Psychology, 28, 344-361.
Jay, G. (2009) The teen brain: Primed to learn, primed to take risks.
Cerebrum. URL (last checked 26 February 2009).
http://bit.ly/GieddPrimed
Giedd, J. N., Blumenthal, J., Jeffries, N. O., Castellanos, F. X., Liu, H.,
Zijdenbos, A., Paus, T., Evans, A. C., & Rapoport, J. L. (1999).
Brain development during childhood and adolescence: A longitudinal
MRI study. Nature Neuroscience, 2, 861-863. doi:10.1038/13158
Goldberg, E. (2009). The new executive brain: Frontal lobes in a com-
plex world. Oxford: Oxford University Press.
Gottfried, A. E., Fleming, J. S., & Gotfried, A. W. (1998). Role of
cognitively stimulating home environment in children’s academic in-
trinsic motivation: A longitudinal study. Child Development, 69,
1148-1460. doi:10.2307/1132277
Guay, F., Boggiano, A. K., & Vallerand, R. J. (2001). Autonomy sup-
port, intrinsic motivation, and perceived competence: Conceptual and
empirical linkages. Personality and Social Psychology Bulletin, 6,
643-650. doi:10.1177/0146167201276001
Hansen, D. M., & Larson, R. (2007). Amplifiers of developmental and
negative experiences in organized activities: Dosage, motivation,
lead roles, and adult-youth ratios. Journal of Applied Developmental
Psychology, 28, 360-374. doi:10.1016/j.appdev.2007.04.006
Hansen, D. M., Larson, R., & Dworkin, J. (2003). What adolescents
learn in organized youth activities: A survey of self-reported devel-
opmental experiences. Journal of Research on Adolescence, 13, 25-
55. doi:10.1111/1532-7795.1301006
Keating, D. (2004). Cognitive and brain development. In R. Lerner, &
L. Steinberg (Eds.), Handbook of Adolescent Psychology (2nd ed., pp.
45-84). Hoboken, NJ: Wiley & Sons.
Kuhn, D. & Franklin, S. (2006). The second decade: What develops
and how. Handbook of Child Psychology (6th ed., pp. 953-993), New
York: Wiley.
Larson, R. W., & Angus, R. M. (2011). Adolescents’ development of
skills for agency in youth programs: Learning to think strategically.
932

E. W. G. MOORE, D. HANSEN
Child Development, 82 , 277-294.
doi:10.1111/j.1467-8624.2010.01555.x
Larson, R., & Hansen, D. (2005). The development of strategic think-
ing: Learning to impact human systems in a Youth Activism program.
Human Development, 48, 327-349. doi:10.1159/000088251
Larson, R. (2000). Towards a psychology of positive youth develop-
ment. American Psychologist, 55, 170-183.
doi:10.1037/0003-066X.55.1.170
Larson, R., Hansen, D., & Moneta, G. (2006). Differing profiles of
developmental experiences in across types of organized youth activi-
ties. Developmental Psychology, 42, 849-863.
doi:10.1037/0012-1649.42.5.849
Larson, R., Hansen, D., & Walker, K. (2005). Everybody’s gotta give:
Adolescents’ development of initiative within a youth program. In J.
Mahoney, R. Larson, & J. Eccles (Eds.), Organized activities as con-
texts of development: Extracurricular activities, after-school and
community programs. Hillsdale, NJ: Erlbaum.
Larson, R. W., & Rusk, N. (in press) Intrinsic motivation and positive
development. In R. M. Lerner, J. V. Lerner, & J. B. Benson (Eds.),
Advances in Child Development and Behavior: Positive Youth De-
velopment. Oxford: Elsevier.
Little, T. D. (2010). Structural equation modeling: Foundations and
extended applications. Lawrence, KS: University of Kansas.
Little, T. D. (in press). Longitudinal structural equation modeling:
Individual difference panel models. New York: Guilford Press.
Loevinger, J. (1954). The attenuation paradox in test theory. Psycho-
logical Bulletin, 51, 493-504. doi:10.1037/h0058543
Markham, J. & Greenbough, W. (2004). Experience driven brain plas-
ticity: Beyond the synapse. Neuron Glia Biology, 1, 351-363.
doi:10.1017/S1740925X05000219
Mascolo, M. F., Fisher, K. W., & Neimeyer, R. (1999). The dynamic
co-development of intentionality, self, and social relations. In J.
Brandstädter, & R. M. Lerner (Eds.), Action and Self-Development:
Theory and Research through the Life Span. Thousand Oaks, CA:
Sage. doi:10.4135/9781452204802.n5
Morris, R. L., & Kayussanu, M. (2009). Approach-avoidance achieve-
ment goals in sport: Psychological correlates and a comparison with
the dichotomous model. International Journal of Sport and Exercise
Psychology, 9, 185-202. doi:10.1080/1612197X.2009.9671899
Muthen, L. K., & Muthen, B. O. (1998-2009). Mplus (Version 5.21)
[Base Program (32-bit)]. Los Angeles, CA: Muthen & Muthen.
National Research Council & Institute of Medicine (2002). Community
programs to promote youth development. Washington, DC: National
Academy Press.
Olinsky, A., Chen, S., & Harlow, L. (2003). The comparative efficacy
of imputation methods for missing data in structural equation mod-
eling. European Journal of Ope rat ion Research, 151, 53-79.
doi:10.1016/S0377-2217(02)00578-7
Pearce, N., & Larson, R. (2006). The process of motivational change in
a civic activism organization. Applied Developmental Science, 10,
121-131. doi:10.1207/s1532480xads1003_2
R Development Core Team (2010). R: A language and environment for
statistical computing. Vienna: R Foundation for Statistical Comput-
ing.
Richards, J. E., Reynolds, G. D., & Courage, M. L. (2010). The neural
bases of infant attention. Current Directions in Psychological Sci-
ence, 19, 41-46. doi:10.1177/0963721409360003
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the
facilitation of intrinsic motivation, social development, and well-
being. American Psychologist, 55 , 68-78.
doi:10.1037/0003-066X.55.1.68
Spear, L. P. (2000). The adolescent brain and age-related behavioral
manifestations. Neuroscience and Biobehavioral Reviews, 24, 417-
463. doi:10.1016/S0149-7634(00)00014-2
Vallerand, R. J. (1997). Toward a hierarchical model of intrinsic and
extrinsic motivation. In M. P. Zanna (Ed.), Advances in Experimental
Social Psychology (pp. 271-360). New York: Academic Press.
Wu, W., Lang, K. M., & Little, T. D. (2009, October). A simple method
of assessing fit and significance in multiply imputed data. Annual
Meeting of the Societ y of Multivariate Experimental Psychology.
Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach’s,
Revelle’s, and McDonald’s: Their relations with each other and two
alternative conceptualizations of reliability. Psychometrika, 70, 123-
133. doi:10.1007/s11336-003-0974-7