Open Journal of Social Sciences, 2015, 3, 81-90
Published Online March 2015 in SciRes. http://www.scirp.org/journal/jss
http://dx.doi.org/10.4236/jss.2015.33015
How to cite this paper: Castle, M.A., Tan, N. and LaGro, J.A. (2015) Evaluating Capacity Building to Foster Climate Change
Adaptation. Open Journal of Social Sciences, 3, 81-90. http://dx.doi.org/10.4236/jss.2015.33015
Evaluating Capacity Building to Foster
Climate Change Adaptation
Mary Ann Castle1, Norma Tan1, James A. LaGro2
1Cora Group, Inc., New York, NY, USA
2Department of Urban and Regional Planning, University of Wisconsin-Madison, Madison, USA
Email: ntan@coragroupinc.com
Received 28 November 2014; accepte d 6 March 2015; published 11 March 2015
Copyright © 2015 by authors and Scientific Research Publishing Inc.
This work is licensed under the Creative Commons Attribution International License (CC BY).
http://creativecommons.org/licenses/by/4.0/
Abstract
This paper describes an evaluation of a cap acity-bui lding approach for promoting locally-d riven
climate change adaptation through local action. Usin g a leadership development strategy, a US-
based NGO convened teams of dedicated sustainability practitioners from 15 localitie s for peer
learning, team-bui ldin g, and access to expert resources. The evaluation strengthened the NGO’s
theoretical framework and methods for understanding the capacity-buildin g contribution of the
inter ve nti on to climat e change adaptatio n. It demonstrated the use of stakeholder data to test and
refine assumption s about 1) how i ntend ed changes are expected to occu r and 2) priori ti zin g the
use of capacity-building resources. It also underscored the necessity of evaluation partnerships
between the NGO and committed teams of change agents to sustain capacity-buildin g effects while
allowing data gathering over time to continuously refine the Theory of Change (ToC) and to g uide
local efforts to achieve climate cha ng e adaptati on outcome s. The evaluatio n data sho wed that the
initial ToC was not su fficien tly robust to identify necessary conditions to be embedded in specific
local situational contexts to increase the likelihood of success. The evaluators recommended en-
hancemen t of the ToC to consider team “readiness”, whi le offerin g a logic model framework and
capacity-building process metrics for progress and outcome tracking.
Keywords
Building Capaci ty, Climate Chan ge Adaptation, Local Action
1. Background & Context
1.1. Awareness and Policy Action: The Current Landscape
The United States’ contribution per capita to global warming is the highest of any country in the world [1].
M. A. Castle et al.
82
While China has surpassed the US as the world’s biggest carbon emitter overall, the US is cumulatively respon-
sible for almost 27% of all global carbon dioxide emissions [2]. A large body of evidence exists about the reality
of global climate change and its environmental consequences—increased weather-related mortality, increased
salinity of freshwater, radically altered crop yields and arable land, changed spatial distribution of infectious and
respiratory disease, and the like [3]. In response, the Federal Environmental Protection Agency (EPA) has de-
veloped regulatory strategies to address some of these issues, e.g., improving fuel economy standards for cars [4]
[5] and implementing new standards in 2015 to decrease methane emissions from natural gas and the fracking
industry1. Despite President Obama’s recent speech about climate change at the UN, powerful, well-financed
corporate and conservative political forces with vested interests in fossil fuel have created and support “active
denial” among large segments of the population about US vulnerability to the threats posed by climate change.
The US Congress has often blocked legislation to reduce the US contribu tion to global warming. It is, therefore,
urgent and necessary to support local community-based efforts for climate change adaptation, resilience, and
mitigation in the US. Through effective evaluation and ways to support current adaptation practice, we can learn
how best to adapt to climate change.
1.2. Fostering Local Level Climate Change Adaptation, Mitigation and Resilience through a
Capacity-Building Initiative
This paper describes an evaluation of one US-based non-profit organization’s (NGO) capacity-building process
to promote locally-driven climate change adaptation policy and action. Using a civic development strategy, the
NGO convened teams of dedicated sustainability practitioners through a series of meetings focused on know-
ledge exchange and team building for climate change adaptationwhat we will here refer to as “adaptation
planning sessions”. These agents of change, drawn from 15 diverse cities and regions across the US, came to-
gether in teams to mitigate the causes and consequences of climate change within their local contexts. At the
Adaptation Planning Sessions, local teams strengthened their relationships as a team and with other teams and
faculty experts. The underlying premise was that through peer learning, team building, and access to the latest
climate-related resources, participan ts would be better able to identify, promote, and implement promising local
and regional climate change adaptation strategies.
1.3. Evaluating Capacity-Building for Climate Change Adaptation, Mitigation and
Resilience
The NGO engaged the authors to conduct an external evaluation of the adaptation planning intervention. Broad-
ly speaking, the NGO wished to te s t their Theory of Change (ToC) using a systematic evaluation methodology
to determine whether, and how the adaptation planning session intervention contributed to locally intended pol-
icy and program outcomes. Underlying this request was the NGO’s stated hope that the evaluation methodology
could identify specific climate change adaptation indicators or measures of environmental impact that could be
directly attributable to the adaptation planning session intervention.
2. Capacity Building
2.1. What Is Capacity-Building?
Capacity building is a process that builds upon and strengthens the existing capacities and potential of individu-
als, groups, organizations and collaborations to create change. Capacity building for sustainable climate change
adaptation can encompass a host of behaviors and activities: providing exposure to new ideas, models and skills;
1The new fuel economy standards are p robably the most significant regu latory action taken so far to red uce actual US GHG emission s. By
the 2025 model year, all auto manufacturers selling cars and light-duty trucks in the US must meet an average fuel economy standa
rd of 54.5
miles per gallon, nearly double today’s standard. It requires the industry to capture methane and other emissions when drilli
ng new wells.
The EPA has also promulgated CO2 standards for new coal-fired power p lants (the single largest source of G
HG emissions). The standards
for new plants a mount to a de f acto con struction ban, b ecause the pollutio n control techn ology necessar y to meet the standards (carbon ca
p-
ture and sequestration ) is not yet avai lable on an ind ustrial scale. So , no ne w coal p owe
r plants may be built in t he US at th is ti me. The E PA
is currently working on stan d ards for ex istin g co al p lants. All of these reg ulato ry actio ns are t aken with in th e exis ting f ra
mework of Sections
111 and 202 of the federal Clean Air Act. These provisions require EPA to regulate pollutants (including CO2
) that are found to cause or
significantly contribute to air pollution which may reasonably be anticipated to endanger public health or welfare. EPA made this “enda
n-
germent finding” with respect to CO2 and other GHGs in December 2009, following the landmark Supreme Court case Massachusetts
v.
EPA. http://www.epa.gov/airquality/oilandgas/pdfs/20120417fs.pdf
M. A. Castle et al.
83
fostering relationships and networking; effectively engaging in political dialogue; tailoring public communica-
tion messages; providing climate change leadership development skills and inspiration; and expanding resources.
Good capacity-building practice requires a commitment to the multiple processes through which people can bet-
ter influence the social forces that affect the change they want to create. It also requires a broad and long-term
strategic vision that can guide concentrated activity on specific topics and contexts to produce capacity-building
outcomes and, ultimately, intended environmental impact [3].
Climate change adaptation initiatives ar e challenging to evalua te [6 ]-[8]. Reasons include the enormity and
complexity of the phenomenon and its far-ranging effects on natural and built environments and on human
health and well-being. Beyond this, capacity building to enhance communities’ ability to devise, organize and
deliver effective climate change strategies is also difficult to evaluate. For example, programs may lack baseline
data and have a long time lag between the climate change adaptation intervention and a measureable outcome [9]
[10]. Or, many different interventions may be implemented without agreement on indicators or definitions of
success. Climate change adaptation also requires collaboration across multiple sectors. And strategies used to
support adaptation can vary widely in different local contexts, with different key leaders, and in different sectors.
These factors make evaluation of capacity building for climate change adaptation esp ecially challenging.
Consequently, capacity-building effort and their evaluation require 1) a deep understanding of strengths of
community collabor a tions, 2) the degree of readiness to change by community leaders, policy-makers, busi-
nesses and other stakeholders, and 3) the social, economic, political and natural environments in which c limat e
change adaptation is supposed to take place.
2.2. Capacity-Building through Cross-Sector Teams, Knowledge Transfer, and Local Action
Planning
Fifteen teams from cities and regions throughout the US participated in the adaptation planning sessions. T eams
were comprised of senior municipal and regional officials, civic and community lead ers, private sector repre-
sentatives, advocates, and other local stakeholders. At these in-person meetings, team members from different
local communities translated their commitment to reducing climate pollution into effective plans of action. They
did this by sharing promising practices; developing relationships with peers from other cities and regions and
with leading climate change and adaptation national experts; and expanding their knowledge of how to access
and use funding sources. The NGO’s intervention, thus, gave them 1) easier access to the best available tools
and resources and 2) the opportunity to build a collaborative network with local, regional, state and federal
counterparts.
2.3. Evaluating Capacity Building
Most change agent teams were at early stages of formation at the time they attended an adaptation planning ses-
sion. Consequently, the evaluation used existing and new data to develop a framework to delineate the steps re-
quired for the NGO’s capacity-building effort to strengthen local climate change adaptation programming. Spe-
cifically, we:
Reviewed project documentation and extant evaluative data, including NGO on-line resources;
Reviewed the literature for good practice in climate change adaptation, including evaluation metrics for spe-
cific interventions;
Assessed the NGO’s Theory of Change (ToC);
Developed a Logic Model Framework and demonstrated how to use it to operationalize the ToC by applying
data from one case;
Conducted an on-line survey of recent adaptation planning session participants (n = 292) from 15 cities/re-
gions, with findings based on a response rate of 32% (n = 94);
Conducted in-depth interviews with change agents (25) from 3 (of 15) city/regional teams, i.e., Low Carbon
Transportation, a Regional Climate Adaptation Planning Alliance, and Green Job initiative s .
3. Testing the Theory of Change
3.1. Using Extant Data and Stakeholder Feedback to Enhance the Theory of Change
The ToC postulated the contributory factors and processes by which the knowledge and skills gained by adapta-
M. A. Castle et al.
84
tion planning session participants would lead to climate change leadership in their local communities, subse-
quent to attending the adaptation planning sessions. Our evaluation focused on strengthening the NGO’s theo-
retical framework and methodologies for understanding its capacity-building contribution to climate change
adaptation interventions. Our goa l was to better elucid a te the processes by which participants—as individuals
and in team formations—benefited from and applied what they learned at the adaptation planning sessions.
Specifically, we reviewed and refined the ToC as well as the tools and methods that the NGO had been using
to track outcomes associated with their capacity-building interventions. What were the underlying assumptions
of the ToC? Did the ToC delineate the necessary conditions for establishing effective local teams to meet cli-
mate change adaptation goals? Did it specify the factors associated with the intervention that might be shown,
together or separately, to contribute to individual and/or team learning and subsequent efficacy? What metrics
and indicators did the ToC suggest, if any, by which to track participan ts ’ learning and to evaluate the possible
impact of participation on subsequent activity to foster resilience?
We also recommended the use of a logic model framework to operationalize the theory of change and to use
evaluation data to test and refine assumptions with respect to how: 1) intended changes are expected to occur,
and 2) capacity-building resources are to be prioritized.
A ToC that guides any initiative must initially be confirmed by stakeholders and then re-examined through
evaluation. The NGO and teams expected us to develop a system of quantitative metrics to evaluate their climate
change adaptation interventions and outcomes, believing this to be the best way to meet funder expectations.
However, relying on quantitative metrics at the outset—without descriptiv e information on the dynamic so-
cio-cultural context within which climate change adaptation is to occurwould fail to reveal dynamic power
structures that teams themselves saw as among the most formidable barriers to successful climate change adap-
tation planning and implementation. Over-emphasis on metrics when teams were at beginning stages of readi-
ness risked overriding the more critical need to engage people in making joint commitments to collaborative
work toward common goals.
The evaluation data showed that the initial ToC was not sufficiently robust to identify necessary conditions
for increasing the likelihood of adaptation planning session success. The findings, thus, led us to enhance the
ToC to include consideration of the team’s degree of “readiness” to work together when selecting teams for par-
ticipation. Including team selection criteria in the ToC was important, considering its implica tions for both: 1)
the design of the leadership development experience, and 2) follow-up after the adaptation planning sessions.
The assessment of the ToC identified how a more robust theory would lead to an expansion in the NGO’s ca-
pacity-building role and activities. It also underscored the necessity of evaluation partnerships between the
NGO and committed teams to sustain capacity-building efforts. By sharing responsibility for both implementa-
tion and data-gathering, the NGO and local teams could most effectively facilitate leadership networks, hone
climate change adaptation plans, and guide collaborative so cial actio n to achieve climate change adaptation
outcomes while also demonstrating results.
3.2. Developing a Logic Model Framework to Operationalize the Program Evaluation
We developed a “ logic map” to help the NGO conceptualize how the adaptation planning session can operatio-
nalize, and evaluate, its theory of change. This construct, viewed as a living tool that would evolve with the pro-
gram and evaluation methods, would be useful to the NGO in communicating goals and strategies; coordinating
data gathering, analysis and progress briefings; and reporting to, and by, stakeholders (including funders).
The logic map provides a structure to specify all assumptions underlying the program design. It defines re-
source “inputs” and implementation “outputs”—the unique program elements that are core to the program’s
value and efficacy. The logic map stabilizes stakeholders’ understanding of the necessary components of the
program, which must be recognizable even when implemented under varying conditions, times and situations. In
each instance, the relative presence or absence of these elements provides the basis for determining the level of
implementation of the intervention being evaluated.
We used one case study to illustrat e how the logic map could serve as an evaluation framework for speci-
fying, and then observing, outcomes over time. The log ic map demonstrated the use of real data from a specific
team’s (observed and expected) outcomes. The segmentation of outcomes into short-, medium- and long-term
indicators and outcome measures underscored the developmental nature of the capacity-building process. In this
way, the logic map deepened understandings of necessary and sufficient conditions for effective capacity building.
M. A. Castle et al.
85
3.3. Distinguishing Process vs. Climate Change Outcome Metrics
We specified sample metrics drawn directly from the evaluation data in the climate change literature and from
the US Environmental Protection Agency guidelines. This showed how different types of evidence of capacity-
building at the local and regional levels could be mapped onto the NGO’s theoretical framework. Case study and
survey data revealed that adaptation planning session participants could describe the processes necessary to put
their sustainability actio n plans into practice. Building on their input, we identified process benchmarks that
could serve as common measures of the quality of partnerships [11]. Exhibit 1 illustrates partnership measures
that emerged from the data. While not an exhaustive list of indicators, the exhibit shows the kinds of measures
that teams generated to assess the quality of their collabor ative capacity for meaningful social action.
Exhibit 1. Process Indicators Related to Partnership Measures
Short-Term Measures
Identify and have all the “right” people who can make decisions and implement plans;
Establish workin g relationship with
mayor, city council, workforce development, public or private utility,
appropriate city agencies, union, advocacy group, etc.;
Establish planning sessions in departments/agencies for specific plans with strategies, suggested financing
and inter-departmental discussions;
Convene routine meetings;
Establish and use on-going co mmunic a t i on mechanisms;
Write environment-specific plans;
Obtain city council/mayoral approval for plans;
Design and initiate advocacy strategies for legislative/city actions, e.g., tax referenda, court resolutions,
city resolutions;
Initiate legislative city actions completion of climate adaptation plan with strategies;
Identify community stakeholders for priority action & initiate public educatio n;
Develop initial set of outcome measures or indicators;
Establish annual stra tegic meeting to review plans, goals, successes to dates, challenges and solutions.
Mid-Term Measures
Complete all plans with strategies and sugg ested financing;
Initiate community wide education and educational programs for students;
Continue advocating with legislators and identify opportunities for integration ;
Integrate plans into general plan, emergency operations plan, other city planning & decision making, e.g.,
include climate and energy strategies in transportation and land use plans;
Plan and initiate resear ch: e.g., risk analysis, appropriate mapping, inventories, assessments with intent to
understand opportunities and challenges and to develop benchmarks;
Identify longer term indicators;
Establish data collectio n procedures.
Longer-Term Measures
Continued citizen support and city approval of programs;
Regional engagement in developing, initiating plans;
Implement pilot projects in cities in the region;
Continuation of strong inter-jurisdiction al coord ination;
Integrate/coordinate city pla ns with regional plan and associated funding;
Secure public-private partnerships for infrastructure investment.
4. Outcomes and Lessons Learned from the Evaluation
4.1. Early Success Indic ators
After attending the adaptation planning session activities, almost two-thirds (61.7%) of survey respondents re-
ported having adjusted or created relevant policies and programs; promoted the importance of climate change
issues more intensively; engaged in action planning with others; built teams to do the work of climate change
M. A. Castle et al.
86
adaptation. About three fourths (71.9%) of respondents reported mobilizing their organizations to get results and
to include more stakeholders in action planning (e.g., community organizations, advocates, union representatives,
businesses). We are cautious in interpreting the self-reported survey data provided by the self-selected sample of
respondents. However, it is consistent with feedback from interviews with adaptation planning session partici-
pants suggesting that some teams changed or enhanced their climate change adaptation planning processes di-
rectly as a result of information received from experts and peers at the adaptation planning sessions. Three ex-
amples follow:
Team X. Some Team X participants reported that they held a conference call with NYC on how to calculate
the GHG baseline a nd ben chmark measures. For example, the Transit Department began to collect data to
measure improvements, e.g., increased transit ridership, transit trips per capita, air quality analysis, etc. After the
adaptation planning sessions, there were important and new collaborations, such as between the Seaport and
Expressway Authority. One of the ideas that emerged from this peer exchange was to analyze the flow of traffic
and to consider a Stop and Stand project that might incentivize truck drivers to work off peak to minimize con-
gestion. Thus, Team X adopted a strategy that had worked well for another city.
Team Y initiated an entry-level 18-month green job training program with a local trade school, businesses,
and unions. Members from this team reported that prior to attending the ada ptation planning sessions, they had
the support of a strongly motivated senior city official with a commitment to developing Green Jobs for resi-
dents. This official, energized to take direct action by newly inspired team members who returned from the
adaptation planning sessions, helped the team:
Invest public funds of $14 million to build retrofit jobs;
Seek private investment;
Design a pre-civil service certification program, adapting a model curriculum from a different team that at-
tended the same adaptation planning session (452 completed training with 225 employed in retrofit projects
in the city at the time of the evaluation);
Pass an ordinance that all city buildings must be retrofitted (15 city buildings were retrofitted or almost
completed at the time of the evaluation);
Create more rigorous standards for privately owned buildings;
Agree on, and set, long term targets for the city.
Team Y had included a representative from an advocacy organization. This was the first time the union and
the department of water and power worked collaboratively with an outside entity to advance a climate change
agenda. When individually interviewed, all team members reported that the advocate had been instrumental in
creating a less defensive political environment and garnering public support. Their concern was how to scale up
their weatherization and solar power programs by finding peers in other states/cities with successful models.
Team Z gained knowledge about low carbon transportation at the adaptation planning session. Afterwards,
Team Z initiated a teleconference with another adaptation planning session team during which members learned
how to calculate the greenhouse carbon or gas baseline and benchmark measures, e.g. their city transit depart-
ment could collect data to measure improvements, such as increased transit ridership, transit trips per capita, air
quality analysis and the like. They were, then, better able to create a strategy for designing better bus service
through rider and community engagement.
4.2. Social Influence
Team members represented different sectors and cultural perspectives, professions, and organizational contexts;
they also had varying access to decision-making power. The adaptation planning sessions provided a valuable
opportunity for cross-sector teams to problem-solve on climate change adaptation issues that required working
across individual and organizational boundaries to gain support for climate change adaptation plans. To be ef-
fective, each member had to build upon and develop their capacity to work collaborativ ely to pr oduce results.
Collaboration is complex. Team members must be open to exchanging information and sharing resources, al-
tering their activities, and enhancing one another’s competencies for mutual benefit and a common purpose.
Team members must build trust to be able to share risks and responsibilities, and to weather both failure and
success [12]. In the US, as elsewhere, power to transform society is concentrated in elites. Often these are the
very people who fail to support climate change adaptation, especially when their focus is on wealth creation ra-
ther than on human and environmental sustainability. The ada ptation planning sessions provided team members
M. A. Castle et al.
87
the opportunity to be in a “safe place” to establish a common vision, mutual respect and a basis for trusting one
another. This was a key aspect of the inte rvention: team members knew that they might have to take professional
job risks to work collaboratively toward agreed upon climate change adaptation goals. Exhibit 2 lists some of
the complex challenges described by teams.
Exhibit 2. Challenges Faced by Change Agent Teams across Cities and Regions
Financial instability;
Inter-jurisdictional complexity and variations;
Scalability of projects;
Climate change denial by decision makers/politicians;
Lack of comprehensive, multiple sector team composition, including legislative authority;
Need for Pre and Post adaptation planning session on-going capacity-building assistance;
Need to build strong trust, respect and process for collaborative decision making, especially when at an
early developmental stage.
5. Recommendations
The evaluation of the NGO’s climate leadership development program showed a broad range of climate change
adaptation strategies a nd approaches by different teams throughout the US. The findings highlighted the
processes necessary for climate change leadership development and capacity-building to have a measurable im-
pact. Reading across the findings for each team, a number of key lessons and priorities can be identified.
1) Conceptualizing a Theory of Change. Our evaluation assisted the NG O to re-conceptualize its ToC to
more accurately portray the adaptation planning session intervention as a first step in a longer-term capaci-
ty-building process. Through our work, the NGO modified its initial desire to demonstrate to their funders a di-
rect causal relationship between their climate leadership development strategy and specific environmental indi-
cators (e.g. reduction in fossil fuel emissions). We helped the NGO to identify those elements of the adaptation
planning session intervention that were especially critical for increasing the likelihood that sessions would re-
sult in teams returning to their home cities/regions and planning and implementing sustainable local climate
change adaptation capacity. Using evidence from our evaluation, we helped the NGO enhance its ToC. We also
re-focused the NGO on using process measures over time to assess: a) initial team readiness, b) composition of
teams, c) level of expertise and follow-up support needed, and d) degree of success of different teams in meeting
their specific climate change adaptation objectives over time. The approach we recommended (and illustrated
using case study data), required ongoing collaboration between the NGO and teams to plan, implement, an d also
evaluate locally-specific teamwork outcomes.
2) Developing and Communica ting a Local Theory of Change. Team members appeared to be unaware of
the value of the NGO’s Theory of Change in providing “the building blocks and relationships between them”
that could lead to accomplishing the longer term climate change adaptation goals [8]. In the absence of a sus-
tained approach to tracking collaborative team activity after the adaptation planning sessions, neither the NGO
nor the teams would be able to offer convincing evidence of a connection between the session intervention and
collaborative social action, much le s s show the relationship between such collaborative social action and mea-
surable environmental change. Future adaptation planning sessions might focus more on educating participants
on the benefits of developing a local ToC to facilitate persuasive communication with stakeholders. When
stakeholders are asked to participate in concerted action, they naturally want, and need, to understand the ratio-
nale for adopting a particular change strategy. A logic model framework would also help stakeholders to appre-
ciate the change strategy as a series of logical steps with resource and activity requirements which, when im-
plemented, could result in desired climate change adaptation outcomes.
3) Requiring Pre-Adapta tion Planning Session Activities. The evaluators recommended that the selection
criteria for team participation in the adaptation planning sessions—as well as pre-session assessments and prep-
aration—be more inclusive to achieve greater acceptance and impact. An important recommendation was to en-
hance the NGO’s preparatory component by: a) recruiting based on team readiness, b) advising pre-session
preparation, with emphasis on inclusive team membership and team meetings prior to participation in the adap-
tation planning sessions. The NGO could advise teams applying to attend an adaptation planning session on
team composition, encouraging cross-sectoral, inter-jurisdictional representation in advance. This would go far
M. A. Castle et al.
88
in helping team members persuade their municipalities to recognize the importance of addressing climate
change adaptation and including key stakeholders and decision-makers in the adaptation planning session. Those
likely to be most affected by climate change adaptation changes as well as individuals and groups with power to
generate, or obstruct, change (i.e . community residents, labor union representatives, advocates, workers) should
be involved in initial team formation. Inclusive teams would have a “collaborative advantage” in moving for-
ward to actualize their goals. The NGO could also assist teams to assess their readiness to do joint planning and
to participate in knowledge sharing with teams from other localities. Preparatory work would also help the NGO
work most effectively with selected teams to adapt promising strategies to fit the ir unique context.
4) Facilitating Knowledge Sharing across Networks. We encouraged the NGO to collect cross-discipline
and cross-jurisdictional data to understand specific organizing needs of localities and regions. For example, the
NGO might consider using network analysis to track the influence of its leadership development strategy on
broad-based coalition-building to accomplish climate change adaptation aims. In line with th is, we proposed
process benchmarks by which to measure team commitment and effectiveness, hypothesizing that team sustai-
nability (at least for some duration) would be a crucial intervening factor linking program participation and cli-
mate change adaptation actions and outcomes. By partnering with multiple teams, the NGO could collect and
compare data across different climate change adaptation issues, action strategies, and team/situational contexts.
Systematic evidence of “what works” could be gathered over the time necessary for teams’ a c tions to have an
effect on policy and implementation processes. Such data could serve to connect the NGO’s intervention to cli-
mate change adaptation outco mes, perhaps even along “hard” measures of environmental impact.
5) Continuing Capacity-Building Activities Post-Session. The NGO’s capacity-building services have and
can continue to assist municipa lities and jurisdictions in the process of planning, developing, implementing and
maintaining city- or region-wide climate change activities. Many, if not most, teams would need further help to
strengthen their climate chang e adaptation plans and activities over an extended period of time. The NGO can
make an important contribution by offering follow-up support to teams in their local environments. Such support
would help teams stay together after the sessions, improving the likelihood of their garnering support for im-
plementation of their climate change adaptation plans. Follow-up capacity-building assistance would strengthen
specific teams’ commitment and confidence in a common issue and approach. It would also give teams the op-
portunity to solidify joint goals and success criteria, while identifying implementation challenges likely to arise
along the way. Teams would also have support for monitoring the attainment of their targeted outcomes. Finally,
post-session technical assistance could further support th e emergence of a network of local, regional and nation-
al level sustainability practitioners
6) Collaborative Fundra ising. F unding, of course, is a significant consideration. The NGO, working with
localities, might be able to coordinate fundraising efforts to attract support for follow-up technical assistance to
several teams and/or to maximize the impact of smaller grants procured through currently available sources of
support. Funders have an interest in seeing that their investment in convening teams leads to future results. The
case could be made for resources to support a continued relationship between specific teams and the NGO to
reinforce team building and application of knowledge in those localities showing readiness to engage as advo-
cates with influential individuals and diverse stakeholder groups after the adaptation planning session. The NGO
might also seek support to facilitate communication across localities. Funding for virtual conferencing would
allow teams to help one another address challenges that they confront through in-depth information exchange
and analysis, as well as sharing of solutions and success stories. With funding for continued knowledge ex-
change and targeted technical assistance, teams would be better able to overcome barriers that they face within
their local contexts.
7) Incorporating Evaluation into Strategic Thinking for Future Planning. Evaluation designs should be
integrated into climate ch an ge adaptation planning during early stages of development. The NGO can facilitate
this by dedicating time at the adaptation planning session to evaluation. The NGO might also work with selected
cities or regions to track progress, using both qualitative and quantitative metrics. Technical support might be
prioritized for teams that agree to participate in outcome evaluation. These teams should receive support to
document their progress using process benchmarks as well as implementation milestones. Ultimately, successful
social action should result in measurable environmental change, taking a long term view. We also suggested,
and partially illustrated, techniques that the NGO could use to track, enhance and evaluate its contribution to city
and regional climate change efforts to promote environmental adaptation and sustainability over the long term.
Many respondents emphasized the need to conduct research and develop strong baseline information upon
M. A. Castle et al.
89
which to formulate their plans. Such data could provide information about current sustainability actions, policies,
programs and/or interventions that are missing or needed, and the benefits, risks and costs of change. Assistance
provided to team members with data collection and analysis would create opportunities to share and compare
results, methods and lessons learned. Through this process, team members could identify and test effective solu-
tions, sharing their evidence with other teams at future adaptation planning sessions.
6. Summary and Conclusions
Our evaluation aimed to guide this unique strategic lead ersh ip development effort toward greater impact. It did
so by equipping the NGO with a realistic evaluation framework focused on tracking progress in building capac-
ity—the necessary prerequisite to effect environmental change.
It also identified critical factors for successful implementation of climate change adaptation strategies, based
on a review of secondary literature, post-session survey outreach to participants, as well as case study interviews
with three selected teams. It also elicited and organized participants’ thinking with respect to metrics planning.
The NGO’s intervention posited that through participatory group learning, community leaders and change
agents would be better able to identify, prioritize and implement appropriate climate change adaptation activi ties,
drawing on newly acquired knowledge, models, and strategies. Our findings strongly suggested that the adapta-
tion planning session intervention model enhanced the facilitative role of the NGO both before and after the
adaptation planning sessions. Beyond this, the NGO can play a more direct role advising teams on their compo-
sition and preparation for the sessions, a nd on formal team selection criteria. These criteria should take inclu-
siveness of the team into account, with an emphasis on cross-sector representation (including outside agencies
and advocates) and the support of influential stakeholders (ideally, with a strong “inside” champion on the team).
Team readiness, as demonstrated through pre-convening team building and planning, also increases the likelih-
ood of sustained team commitmen t and follow-up.
We also observed a great need for consistent and sustain ed support to teams after the adaptation planning ses-
sion intervention. We, thus, strongly emphasized the need for ongoing collaboration between the NGO and
teams, as well as the funding for follow-up technical assistance to teams. In line with this, we stressed collabora-
tive follow-up to collect necessary and su fficient data on local capacity-building processes and outcomes to
evaluate the contribution of the adaptation planning session intervention to climate change adaptation action and
results. The adaptation planning sessions potentially created a network of “change agents” within and across ci-
ties and regions to further support climate change adaptation. Technical assistance to teams would permit rou-
tine, on-going cross-team communication and knowledge sharing, and effective messaging for specific popula-
tions on specific topics, such as how to successfully work with resistant legislators/politicians. Moreover, robust
local efforts to disseminate the change strategy using a ToC and logic model framework could help promote
stakeholder involvement in operationalizing the change strategy in complex, challenging environments.
As illustrated in this evaluation, agents of change in cities and regions across the US benefited from the ca-
pacity building offered by the NGO, using a climate change leadership development strategy. However, our
findings highlighted a major challenge for the NGOnamely, to be able to procure and commit the resources
necessary to facilitate long-term capacity-building and systematic evaluation. Additional funder and NGO in-
vestments toward locally appropriate technical assistance to increase and maximize the use and impact of li-
mited resources would go far in helping local teams to meet their climate change adaptation goals.
Acknowledgment
The authors gratefully acknowledge Mr. Erik Laby, Project Associate, for his important contributions to the
work described in this article.
References
[1] Scientific AmericanTM (2012) China Greenhouse Gas Emissions Set to Rise Well Past US.
http://www.scientificamerican.com/article/china-greenhouse-gas-emissions-rise-past-us/
[2] Boden, T.A., Marland, G. and Andres, R.J. (2013) Global, Regional, and National Fossil-Fuel CO2 Emissions. Carbon
Dioxide Information Analysis Center, Oak Ridge National Laboratory, US Department of Energy, Oak Ridge, Tenn.,
USA.
[3] Eade, D. (1997) Capacity-Building. An Oxfam Publication, UK and Ireland.
M. A. Castle et al.
90
[4] Worldwatch.org. (2013) US Public Still Unconvinced on Climate Change.
http://www.worldwatch.org/node/6300#.U0WdRyO4-pY.gmail
[5] New York Times (2014) White House Unveils Plans to Cut Methane Emissions. New York Times, 27 March 2014,
A12.
[6] Prowse, M. and Snilstveit, B. (2010) Impact Evaluation and Interventions to Address Climate Change: A Scoping
Study. International Initiative for Impact Evaluation, Working Paper 7, 6.
[7] Ayers, J. and Forsyth , T. (2009) Community Based Adaptation to Climate Change: Strengthening Resilience through
Development. Environment, 51, 22-31. http://dx.doi.org/10.3200/ENV.51.4.22-31
Institute of Development Studies (2008) Desk Review: Evaluation of Adaptation to Climate Change from a Develop-
ment Perspective. Institute of Development Studies, UK.
[8] Bours, D., M cGin n, C. and Pringle, P. (2014) Guidance Note 1: Twelve Reasons Why Climate Change Adaptation
M&E Is Challenging. UKCID: SeaChange.
http://www.ukcip.org.uk/wordpress/wp-content/PDFs/MandE-Guidance-Note1.pdf
[9] Hedger, M.M., et al. (2008) Evaluation of Adaptation to Climate Change from a Development Perspective. Institute of
Development Studies (IDS)/AEA Group. www.seachangeco.org/node/128
[10] Bours, D., McGin n, C. and Pringle, P. (2014) Guidance Note 3: Theory of Change Approach to Climate Change
Adaptation Programming. UKCID: SeaChange. http://www.seachangecop.org/node/2933
[11] Environmental Protection Agency (EwPA) http://www.epa.gov/cleanpowerpla
[12] Himmelman, A.T. (1996) On the Theory and Practice of Transformational Collaboration; Form Social Service to So-
cial Justi ce . In: Huxham, C., Ed., Creating Collaborative Advantage, Sage Publications, London, 18f.