From Theory to Practice: The Student Experience Evaluating Development Projects Focused on Nature-Based Solutions

Abstract

Graduate students often seek hands-on experiences in the international development field. Given that Multilateral Development Banks (MDBs) provide hundreds of billions of dollars in aid each year, we expected that reviewing the design, implementation, and outcomes of their environmental projects would provide valuable learning outcomes for students. This novel study on Nature-based Solutions (NbS) in the Global Environment Facility (GEF) gave students the opportunity to engage directly with practitioners in the review of 50 environmental projects across 45 countries. A team of professionals from the Scientific and Technical Advisory Panel (STAP) of the GEF and eight students from the University of Michigan School for Environment and Sustainability and the University of Maryland School of Public Policy developed lessons learned from reviewing the GEF portfolio over a twenty-year time span. When screening projects for enabling conditions including theory of change, climate risk screening, multi-stakeholder engagement, and adaptive management, most had stronger explanations of the environmental than the social outcomes sought, and only more recent ones incorporated climate risk screening. The process and findings associated with this educational experience contributed to students’ climate change leadership development; for example, by learning about the tradeoffs and possible co-benefits of improving both environmental conditions and livelihoods in less developed countries. Our research led to practice advice for the design of future GEF projects, as well as ideas for future coursework to further bridge the gap between theory and practice in academia, which we believe to be essential to preparing the next generation of climate leaders.

1. Introduction

Many graduate students seek careers tackling climate change in the international practitioner domain. However, there is little scholarship on using development agencies’ projects as a learning experience for students. The Global Environment Facility (GEF) requested a review of 50 completed and ongoing GEF projects that included “Nature-based Solutions” (NbS) (NbS are defined as “actions to protect, sustainably manage and restore natural or modified ecosystems that address societal challenges effectively and adaptively, simultaneously, providing human well-being and biodiversity benefits” [1,2]) to determine what lessons could be learned to inform future investments. The projects reviewed represented an investment of USD 377 million, and an additional USD 2.7 billion in co-financing across 45 countries.

The GEF serves as a “financial mechanism” to five multilateral environmental treaties and conventions: the Convention on Biological Diversity, the United Nations Framework Convention on Climate Change (UNFCCC), the Stockholm Convention on Persistent Organic Pollutants, the UN Convention to Combat Desertification, and the Minamata Convention on Mercury (for more on the GEF, please see https://www.thegef.org/partners/conventions (accessed on 1 March 2021)). With expenditures of about USD 1 billion per year, the GEF seeks to amplify actions to address the multiple drivers of global climate change more coherently in their projects as greenhouse gas emissions, biodiversity loss, land degradation, and waste are increasing, aquifers are being depleted, fisheries are being overfished and ocean pollution is becoming pervasive. Projects that can find synergies across these global issues are in urgent need [3].

An independent advisory body, the Scientific and Technical Advisory Panel (STAP), chaired by one of the authors (Bierbaum), conducted a review of 50 NbS projects implemented by five development agencies with eight graduate students—four from the University of Michigan School for Environment and Sustainability (UMich-SEAS) and four from the University of Maryland School of Public Policy (UMd-SPP) (for more on STAP, please see https://stapgef.org/ (accessed on 1 March 2021)). The overall effort was coordinated by two STAP Secretariat professionals, the STAP chair, and a STAP undergraduate intern.

The detailed analysis of NbS projects revealed that more recently designed projects tended to include elements related to climate risk screening and more comprehensive stakeholder dialogue, in addition to nature-based goals. But there was still little focus on behavioral change, tradeoffs of choosing different options, and identification of desired social and economic outcomes.

The initial review, however, resulted in more than just lessons learned from projects using NbS; it resulted in a novel educational experience that integrated real-world and classroom learning, providing students an opportunity to gain the skills necessary for becoming prominent climate change practitioners. In this paper, we assess the educational benefits for students who participated in the NbS study to determine its value as a model for developing climate change leaders. Two research questions guided our study:

• To what extent do collaborations across universities and across types of schools (policy and environment and sustainability) enhance the educational experience and provide students with skills to become better global change leaders?
• What are the benefits of a non-traditional learning experience working directly with practitioners and graduate students?

To examine the unique learning experience of participating in the NbS project, both the students and STAP leaders were interviewed to understand how the review of the 50 GEF NbS projects affected their collective understanding of the links between sustainability theory and practice. In addition, students were questioned about the overseeing and mentoring from the STAP professionals and whether this suitably enabled them to distill best practices and recommendations from complex and detailed project documents. We also sought to learn how the students’ views of the work that development professionals conducted, and what it encompassed, evolved. Lastly, the benefits of working with peers across two graduate programs and universities was explored.

Overall, our findings show that our interviewees gained several skills and insights from engaging in this project. These skills and insights fell into several topics: general insights, career skills, life skills, research skills, insights from working with students from another university, and insights from professionals working with students. Generally, students noted that our initial study brought them greater awareness of different types of development projects and solutions people are proposing around the world to tackle environmental issues and climate change. Working weekly with professionals and participating in a virtual international NbS conference gave students a better understanding of the operating procedures in development institutions, the breadth of the organizations working on these types of issues, and access to experts in the field. Every student gained skills in analyzing and understanding project documents, which are complex, nuanced, and often over 200 pages long. From reading these documents, students valued the lessons they learned in having to distill necessary information to complete their analytic tasks and categorizations and communicate effectively with professionals. These skills were particularly valued by students in helping them gain confidence in communicating science by using the language of the field. Students involved in the NVIVO analysis portion of the project also learned critical skills in conducting qualitative data analysis. Overall, this non-formal educational experience was deemed valuable as it afforded students deep insight into a multilateral trust fund, how projects are designed to implement NbS interventions, and an appreciation of ancillary issues that can augment or diminish the success of achieving global environmental and social benefits. Professionals valued the fresh eyes the students brought to the NbS analysis, and the willingness to question established project norms in design, implementation, and evaluation. Working across two universities and two different types of schools was deemed positive, as students initially approached the analysis from either a policy or environmental angle. However, the short (but intense) 12-week duration of the project, the necessity of conducting meetings over Zoom because of COVID-19, and the variance in students’ schedules did not allow as much interaction among students as would have been possible through a full-semester or year-long class.

2. Literature Review

Peer-reviewed literature on the need for integrating sustainability concepts into education is burgeoning, but how practical sustainability learning can be implemented and evaluated in higher education is less common. Integrating knowledge from researchers and practitioners requires finding common ground and combining the theoretical with the pragmatic. Working together across these worlds, coordinating ongoing data collection efforts, and paying more attention to the analysis of combined datasets can likely lead to a better understanding of what makes development projects succeed or not.

Based on our prior work focused on developing appropriate indictors of sustainability for agro-ecosystem projects (Web of Science identified 602 papers from which we screened the subset meeting four criteria: (1) the article dealt with commodity agriculture; (2) assessed sustainability aspects of production; (3) documented possible indicators; (4) focused on tropical and sub-tropical countries), we found that there may be only limited engagement between scholars and practitioners, or even across different groups within scholarly and practitioner communities. Further, interviews with development agencies investing in sustainable commodity agriculture revealed that the most proposed indicators for sustainability in this arena in the academic literature do not overlap with the central aspects that practitioners seek to monitor [4]. From 85 papers reviewed, we extracted 294 unique proposed indicators of agroecosystem sustainability, indicating little convergence in the academic world on what to measure to assess needed development agency outcomes. While almost all articles considered the environmental aspects of sustainability, economic and social aspects of sustainability were less commonly considered (43% and 25%, respectively). Thus, academics are not meeting development practitioners’ needs to identify metrics to simultaneously improve both the environment and lift people out of poverty. To do so, a systems approach is needed that recognizes connections across issues and sectors and seeks to produce multiple benefits while decreasing further damages. For example, conservation and land management actions are highly determined by political, societal, and economic interests which can determine success or failure in attaining biodiversity goals [5].

In addition, the pedagogical literature is increasingly promoting learning through “real-world” practice, as this project sought to provide. As a recent article authored by Motzer et al. in this journal concluded: “While much has been done to identify this need, inadequate Leadership Training for graduate students in Sustainability (LTS) continues to plague even the most highly-resourced institutions [6]”. Competencies that the literature encourages—systems thinking, anticipatory thinking, values thinking, and strategic thinking—are not traditionally learned in typical university settings [7]. Equally important, what constitutes knowledge varies by culture, and many practitioners lack indigenous perspectives. The needed composite approach is one of integrated problem solving recognizing the iterative, multidisciplinary, and multi-sectoral attributes of advancing sustainability.

To have the necessary transformational impact on the multiple environmental crises of biodiversity loss, climate change, and over-consumption of natural resources, sustainability practice should become more mainstream in today’s educational systems [8]. Teamwork is particularly relevant for working in the sustainability space, given the transdisciplinary nature of development practice. Good communication skills and translation of technical material into useable information are key competencies students need. This type of education should include understanding motivations, values, and behaviors embedded in different communities—an area often overlooked by focusing more on easily analyzed factors such as changes in technology, policies, and demography [9].

Looking across the sustainability pedagogy literature, project/problem-based learning in an organization/community was ranked as fostering the broadest range of sustainability competencies [10,11]. However, as Lozano et al. noted, “there has been limited research on the connection between how courses are delivered (pedagogical approaches) and how they may affect sustainability competencies” [12]. The NbS project request of the GEF offered a chance to gain additional insights into the benefits of project/problem-based learning as related to sustainability leadership development. To analyze NbS projects in the GEF, students needed to think in an integrated way across geographies and issues, and work as a team to evaluate outcomes that advanced both livelihoods and environmental sustainability.

The framework for NbS emerged from the Ecosystem Approach of the Convention on Biological Diversity and recognizes that biodiversity conservation and human well-being are not mutually exclusive from function and resilient natural ecosystems. There are therefore a variety of definitions of NbS; the Inter-American Development Bank defined NbS as “ecosystem-related approaches to address societal challenges” and “activities associated with the protection, management, enhancement and restoration of natural capital to develop climate-resilient infrastructure” [13] and the European Commission defined NbS as “solutions for addressing societal challenges (such as risk management) that are ‘inspired by, supported by or copied from nature’ and ‘simultaneously provide environmental, social and economic benefits and help build resilience’” [14,15].

The most common definition comes from the International Union for Conservation of Nature (IUCN) which defined NbS as “actions to protect, sustainably manage and restore natural or modified ecosystems that address societal challenges effectively and adaptively, simultaneously, providing human well-being and biodiversity benefits” [14,15]. The IUCN also points out NbS encompass several approaches to nature and biodiversity conservation: area-based conservation, climate adaptation services, ecological engineering, ecological restoration, ecosystem-based adaptation, ecosystem-based disaster risk reduction, ecosystem-based management, ecosystem-based mitigation, forest landscape restoration, green infrastructure, and natural infrastructure. STAP has used the IUCN definition of NbS, that is, including both joint environmental and societal benefit from GEF projects designed to address climate change, biodiversity loss, land degradation, etc. [16].

Figure 1 visualizes the definition and principles of NbS as reaching across the broad sustainability space, and across both societal and environmental outcomes. Figure 2 shows the ‘systems thinking’ employed in STAP’s approach to conceptualizing optimum outcomes of NbS projects. Students found the real-world outcomes to lie more often in the bottom right quadrant, demonstrating environmental benefits, with a less clear focus and outcomes on the human well-being quadrant as a simultaneous goal.

3. Materials and Methods

3.1. Student Involvement in the Review of GEF NbS Projects

The review of the NbS projects covered projects in 45 countries implemented by five development agencies (World Bank, United Nations Development Program, United Nations Environment Program, International Union for Conservation of Nature, and International Fund for Agricultural Development). These projects were analyzed to answer the GEF/STAP’s research questions:

• ‘What lessons can be learned from projects employing NbS over two decades that can inform future project design?’;
• How strongly did these NbS projects include additional ‘enabling’ elements—climate risk screening, behavior change, multi-stakeholder dialogue, adaptive management, and biodiversity mainstreaming—that can enhance project durability?

To achieve these goals, this study was conducted in two parts: (1) project reviews, and (2) data analysis and synthesis into a presentation. All eight students participated in the first part, and a subset of students participated in the second part as the semester had ended and many students had summer plans that limited them from participating fully during that time.

For part one, students were assigned a project from the GEF portfolio to evaluate weekly. For the first two rounds of review, student researchers worked in pairs to standardize methods and then worked individually on the next five reviews. This model was used to accommodate differing class schedules and to allow each student a chance to serve as the lead reviewer and presenter in the weekly meetings. Students could request countries or regions of study, or choose projects in foreign languages; otherwise, STAP professionals assigned weekly reviews. Often, students had insights into projects of other students based on their own review of similar country projects or on related topics.

We began by reviewing all available documents for each GEF project, including the initial approval, CEO Endorsement stage, Mid-Term Evaluations (MTE), Project Implementation Reviews (PIR), and Terminal Evaluations (TE) (project documents for each project listed in

Table 1. NbS Projects Reviewed by Region and GEF ID.

Region	GEF ID
Asuncion	9127
Bangladesh	5636
Bhutan	4975
Bhutan	9199
Brazil	9617
Bulgaria	1123
Burkina Faso	9141
Burundi	9178
Cambodia	3635
Cameroon	9519
Caribbean	1254
Central African Republic	9514
China	2740
China	3265
China	2483
Colombia	3574
Congo	9515
Ethiopia	2794
Ethiopia	9135
Gabon	5264
Ghana	9340
Good Growth Partnership	9617
Guinea-Bissau	9521
Indonesia	3188
Kazakhstan	1244
Kenya	9139
LAC	2505
LAC	5681
Mexico	4763
Niger	9136
Nigeria	9143
Paraguay	9180
Peru	3627
Philippines	5826
Regional Africa	2701
Restoration Initiative
Rwanda	4952
Senegal	9134
Somalia	5592
South Africa	123
South Africa	2615
South Africa	9073
Sudan	5703
Tajikistan	3129
Tanzania	9524
Thailand	3445
Timor-Leste	5671
Turkey	4469
Uganda	5204
Vietnam	9484

, can be searched by GEF ID at https://www.thegef.org/projects-operations/database (accessed on 1 March 2021)). Information gleaned from this detailed review was then recorded in an Excel spreadsheet. The STAP professionals designed the initial spreadsheet, and then our team worked together throughout the study to amend the spreadsheet to ensure it was designed to best capture useful information. Students spent between 5 and 10 h a week reviewing GEF project documents.

At the end of the term, following the completion of the project reviews, a subset of the students continued to work with the STAP staff team on the second part of the project, conducting both quantitative and qualitative data analysis. An NVivo codebook was developed collaboratively in consultation with experts from Clark University. The codebook included sections on project characteristics, project objectives, NbS type, and project design. After solidifying the codebook, we worked asynchronously to code projects. For this analysis, we removed projects with incomplete project documentation. We were left with 30 projects to code. Following the coding, all NVivo data files were merged which was then used to analyze results. We designed mind maps and queries, a basic function of NVivo, to showcase associations between project components. We then used Microsoft Excel to create graphs to depict statistics drawn from NVivo surrounding word and coding frequencies.

3.2. Students’ Insights from the NbS Review

In light of their reviews, students were able to follow the evolution of project components, design, and priorities over time. More recent projects tended to include more ‘enabling’ elements in addition to NbS, but even newer projects were generally stronger in describing the environmental components of NbS than the societal challenges, such as gender. Monitoring and evaluation were more commonly addressed in recent NbS projects, but there was relatively little information about why projects did or did not succeed and adaptive management was usually not mentioned. There were consistent gaps in including important enabling conditions that STAP has encouraged [17]: climate risk was recognized by many projects, but few had fully screened for the risk, and/or identified mitigation measures. All projects referred to some type of multi-stakeholder dialogue, but these were sometimes ‘top-heavy’, with government and other official bodies seeming to dominate, and it was not always clear what the roles and responsibilities of different stakeholders were. Involvement from the ‘bottom up’ and involvement of indigenous groups were less consistent. All projects expected behavior change outcomes, either at the individual level or at the institutional level by a government or national system. However, this was usually implicit, and not often stated as an explicit project objective. The behavioral outcome being sought was clear, as was whose behavior needed to change, but how this was to be achieved was typically not explained. Durability and scalability were often mentioned as desirable, but sometimes without much information about how these were to be achieved. (For more on the results, please see https://www.stapgef.org/resources/advisory-documents/nature-based-solutions-and-gef (accessed on 1 July 2021)).

Findings were shared during a virtual workshop hosted by STAP in concert with the Wildlife Conservation Society (WCS) and The Gordon and Betty Moore Foundation. The project review phase of the study officially concluded at the end of the Virtual Workshop on 20 May 2020.

Building on the preliminary and informal feedback from student researchers during the NbS study, we conducted interviews to answer our educational research questions and more clearly understand how our initial study prepared students to be better global change leaders. Data were analyzed using NVivo.

3.3. Interview Guide Development

We developed two interview guides: one for student researchers and one for STAP professionals (

Table 2. Interview guide questions for students and professionals.

Questions for Students
1	What was your perception of climate change education at your graduate institution before engaging in this research?
2	To what extent did the unique elements of this experience give you new skills or insights you would not have otherwise gained [in a traditional classroom setting’?
3	What are these new skills/insights?
4	Why are these skills/insights valuable and relevant to your future career?
5	How did Climate Risk Screening and Adaptive Management contribute to the successful outcomes of the development project?
6	How did participating as an equal in the international workshop affect you?
7	Do you think your experience helped prepare you to be an effective change agent?
8	How could the experience be improved?
9	Do you have any advice for professors to enable students to have more ‘hands on’ experiences at the intersection of theory and practice?
Questions for Professionals
1	What was the value of student participation in achieving the goals of the review?
2	How did student participation in this project impose/alter the pace and ultimate outcomes and analysis?
3	Did you develop any new insights based on the students’ experiences?
4	Do you recommend future student/professional interactions to extract lessons learned from development projects? Why or why not?
5	How would you improve the student experience?

). As the STAP intern was still a student and functioned as both a student and member of staff in varying capacities, we decided to not create a third guide for this individual but to ask them questions from the respective two guides. The final interview guide included nine questions for students and five questions for professionals.

3.4. Site and Sample

We conducted a total of eight interviews—three with students from UMich-SEAS, two with students from UMd-SPP, and three with STAP professionals and the intern. Two students from UMd-SPP were unavailable as they graduated between the end of the NbS and this study. The authors, i.e., the STAP Chair and the fourth student from UMich-SEAS, were not interviewed.

Each interview, conducted remotely via Zoom, lasted for up to an hour and was guided by the interview guide. Prompts were used to ask follow-up questions as relevant. We also asked students questions about which projects they reviewed and their roles in the study. We used the Zoom transcript and recording functions to facilitate analysis. Following each interview, we converted the Zoom transcript files into Microsoft Word documents to make them compatible with the most recent version of NVivo. At that time, we also reviewed the recordings to make sure that the transcripts were complete, and spelling errors were fixed.

3.5. Interview Data Analysis

The first step of the interview analysis was to develop the NVivo codebook. We developed the codebook so that the nodes reflected the interview guide questions to ensure that comparisons could be made across questions. The codebook was then refined through test coding. The final codebook, pictured in Figure 3, included nodes on demographic information and project logistics as well as nodes that correlated to the interview questions. This allowed for comparisons across interviewee type.

When the codebook was completed, we uploaded the interview transcripts into NVivo and coded them into the nodes. Lastly, we used mind-maps and queries, a basic function of NVivo, to showcase associations across interview questions and across interviewee type.

4. Results

At the completion of the formal NbS analysis and group zoom calls with STAP professionals, all students reported three initial common insights during a reflection zoom call:

• Working with students from another graduate institution helped build team skills and a better understanding of how this work is approached from a public policy and environment perspective;
• Not having a set class time, and conducting weekly conference calls to discuss findings, led to an important lesson in time management as deadlines were not always weekly or even on the same day of the week as they are in formal classroom settings;
• Working with a large development organization helped students learn about the importance of the hierarchies and the complex processes they have in place.

The subsequent study sought to provide deeper answers to answer two specific research questions:

• Do collaborations across universities and across types of Schools (policy and environment and sustainability) enhance the educational experience and provide students with skills to become better global change leaders?
• What are the benefits of a non-traditional learning experience working directly with practitioners and graduate students?

4.1. Benefits of a Non-Traditional Project-Based Learning Experience

4.1.1. Previous Perceptions of Climate Change Education at Graduate Institutions

There was a consensus that climate change education at UMich-SEAS is an underlying concept discussed or built into the framework of almost every course, consistent with the school’s climate and energy theme, but that there are few interdisciplinary climate change courses. The majority addressed selected sub-topics or disciplines such as adaptation, the UNFCCC, and climate economics and policy. Moreover, these courses are distributed across MS area specializations (i.e., Behavior Education and Communication; Environmental Justice; Environmental Policy and Planning, Ecosystem Science and Management; Geospatial Data Sciences; Sustainability and Development; and Sustainable Systems). All of the UMich-SEAS students said they primarily found climate courses on their own, sometimes serendipitously.

UMich-SEAS students felt as though there was often a disconnect between their classroom learning and what was happening in practice; they attributed this to faculty primarily with limited practice experience. One student, for example, emphasized that they had little exposure to frameworks such as the UN Sustainable Development Goals or use of the actual terminology used in the field that would better prepare students for their professional roles. They said, “[professors did not explicitly point out] here are the UN Sustainable Development Goals, but we were working within that framework and do not define it or discuss it”. Because of the perceived lack of connection to practitioners, the UMich-SEAS students interviewed felt as though they were more often gaining theoretical knowledge of global change than practical knowledge on how to engage in the field or with development agencies.

Students from UMd-SPP did not feel as though climate change was currently an important part of their education in the School of Public Policy. Unlike UMich-SEAS, UMd-SPP is a public policy program with specializations in a variety of types of policy including environmental and energy policy, international development, international security, nonprofit management, health policy, and education policy. All students have general requirements and if they choose a specialization, they take that specialization’s additional courses. The UMd-SPP students did not feel as though climate change, or its impacts on other environmental issues, were well integrated into the policy program even though it is a global issue intersecting UMd-SPP specializations. One student noted that they dropped their environmental policy specialization because the courses on climate change and environmental justice were not “robust enough” and because “there’s not a big incentive to specialize if you don’t feel strongly about it or if in my case, you feel strongly that the specialization is not strong enough”; she is now taking courses on climate change in the department of anthropology. With recent changes to the ‘capstone’ requirements, UMd-SPP students are no longer required to take a course in which they work with practitioners (the course that resulted in the students who participated in the NbS and this study). Lastly, there is a consensus that there is a lack of coordinated climate change education in the department.

It should be noted that since the NbS study concluded in 2020, University of Michigan has approved a Climate Solutions graduate certificate program (https://clasp.engin.umich.edu/academics/graduate-studies/climate-change-solutions-certificate/ (accessed on 1 February 2022)) that includes coverage of the role of development agencies in mitigation and adaptation, as well as classes on science, technology, and impacts. The University of Maryland, School of Public Policy is currently also considering a graduate climate certificate to integrate coursework.

4.1.2. Skills and Insights Learned

Overall, our findings show that our interviewees gained several skills and insights from engaging in this project. Generally, students noted that the NbS study raised their awareness of different types of development projects and solutions individuals are proposing around the world to tackle environmental issues like climate change. In addition, it gave them an understanding of how development professionals work to design, implement, and review project outcomes, and the terminology they use. As a result, they learned new terminology and about enabling conditions. For example, investigating multi-stakeholder dialogues showed students the nuances of creating systems for dialogue among project stakeholders and partners, as many were top-down and did not involve collaboration with local communities or indigenous peoples. Because students recognized climate change as the critical issue of our time, they were surprised to learn that climate risk screening was not better integrated into these projects. Students noted that climate and/or climate risk screening was mentioned in almost every project but that few elaborated on the topic or had conducted a thorough climate risk screening process. (At this time, all current and future GEF projects must explicitly screen for climate risk following STAP’s guidance—https://www.thegef.org/council-meeting-documents/stap-guidance-climate-risk-screening (accessed on 1 March 2021)). This helped students recognize that although climate might be an underlying driver, many development projects are aimed at addressing specific, often more immediate, environmental issues, and that results are expected to have synergistic benefits that help combat climate change. In addition, working with professionals and attending the NbS conference gave students a better understanding of the operating procedures in development institutions and the breadth of the organizations working on these types of issues. They also appreciated seeing several world leaders present in the plenaries and participating in breakout sessions as equals.

Career skills were the most often coded node in this section of the codebook. Every student reported gaining skills in reading and comprehending project documents, which are complex, nuanced, and often over 200 pages long. From reading these documents, students valued the lessons they learned in having to distill the information necessary to complete their tasks and communicate effectively with professionals and fellow students. These skills were particularly appreciated as they helped students gain confidence in communicating science by using the field’s terminology. Students who read projects in foreign languages also noted that they felt more confident in communicating science in their foreign language, an asset they noted as many development agencies also work in French and Spanish. Students involved in the analysis portion of the project also learned what they perceived as critical skills in conducting qualitative data analysis and using Nvivo software. While these are also research skills, students noted that they expect to professionally conduct similar analyses and that they therefore viewed those skills as important for their careers.

One student also found that this project improved their decision-making skills and now feels more equipped to engage in developing sustainable development projects. They said, “I think [this study] would make me a better decision-maker when I’m designing my own project. If I’m designing my own project, I’m like well there better be a theory of change, so I can go through and pinpoint where things go wrong, and I learned that from this project. There should be behavior change components of a good climate funded project… I know what needs to be included in a good project”.

Lastly, students also shared that they gained valuable life and research skills. For example, students felt that the communication and critical thinking skills they gained in English and foreign languages were not just important for working in the field but for any endeavor in life. The research skills students gained using Nvivo, reading projects, or learning about tools like the Resilience, Adaptation Pathways and Transformation Approach (RAPTA) [18], were identified as important for their future work as these particular skills can be used in a variety of research contexts, not just within the development field.

4.1.3. Value of Bridging Theory and Practice

Many students felt as though bridging the gap between theory and practice exposed them to new potential career paths that they might not have considered or known about before the project. One student said, “Honestly, this is kind of a big statement but [this study] exposed me to thinking more about in terms of careers working at a high level. Whereas before I’ve always kind of considered myself as interested in working directly on the ground at the project level”.

Students also explained that this experience bridging theory and practice allowed them to directly, through observation and collaboration, learn how professionals are truly engaging in their work. To one student “the benefit of having access to those tables, to those rooms, even virtually is an incredible opportunity for a student… even virtually, it gives you an opportunity to engage with experts in the field, and also get a better understanding of what experts in the field are saying about NbS. That’s a really incredible opportunity”.

Lastly, there was consensus among students that engaging in this type of study, bridging theory and practice, was valuable not just because of the skills and insight gained, but because of the experience of developing recommendations that will contribute to positive, real-life development outcomes. Especially since some of the development projects reviewed were still ongoing, students felt that they were helping to meaningfully improve projects, particularly with regard to ensuring attention to climate change adaptation. Such opportunities, they felt, were not available through traditional classroom assignments.

4.2. Collaborations across Universities and with Professionals

Students thought the collaboration between the two universities’ programs could have been stronger. Although the respective groups of students worked together in meetings, their project review teams did not comprise members from both universities. Half of the interviewees shared that working with students from another university and program was a unique way to learn about different perspectives and methods for tackling the same problems. One UMich-SEAS student explained that even though her background was not necessarily in the natural sciences, she was coming into the project reviews with a more science-oriented thought process than the policy-oriented UMd-SPP students. She said that during our team meetings she was often wondering, “what are you looking at because they did have some insights that I thought were really interesting that I hadn’t seen”. The perspectives she gained helped her to think differently when analyzing projects. Two students who worked on the NVIVO analysis also noted the value of learning how to use the method and tool from a colleague their own age and felt that they might not have learned similar technical skills at their home university. In sum, students’ research experience was strengthened by the involvement of students from more than one university and two kinds of programs.

As a result of collaborating with students, professionals reported that they gained fresh perspectives on social issues, such as equity, that they felt have not yet fully penetrated international development agencies’ sometimes ‘siloed’ projects. Professionals also appreciated the opportunity to encourage students to develop a healthy skepticism. For example, they were proud of helping students go beyond their initial acceptance of projects’ claims, including with regard to NbS, by examining assertions more critically. Professionals shared that they were “refreshed”, as one professional put it, by students’ enthusiasm and by what they were able to contribute to the project. One professional ended their interview by stating, “it’s a great idea and I wish I had [worked with practitioners] while I was in college or grad school!”.

Project Improvements and Advice

It was clear that the students interviewed had thought carefully about how the NbS project study could be improved in the future. Most prominently, interviewees were concerned about the timeline and wished that the NVivo analysis could have been less rushed. Ideas for improvements to the timeline included extending it a full semester, reviewing fewer projects, engaging more students in the data analysis, creating a master spreadsheet or grid typology to improve tracking, or coding directly from the project documents. There was consensus that initially, more time focused on learning terminology and identifying project goals and desired outputs would have been helpful. Students also stressed that they would have liked more opportunities to interact with their peers from the other institution, and more time to form relationships with professionals, which would have helped some students feel more comfortable asking questions. The Zoom medium did not allow more informal conversations in hallways or personal meetings in offices. Students believed that strengthening these inter-relationships would have helped create a more structured and inclusive feedback mechanism. Lastly, professionals emphasized that coordinating the study was time-consuming, leaving insufficient time for engaging with the students on data and results.

5. Limitations

The intense but short 12-week period for the review of the NbS projects limited students’ cross-university interactions that could have been enriching. The group of students and professionals involved in the NbS study and subsequent interviews was relatively small, limiting the potential applicability of the results across other student-practitioner collaborative learning experiences. Because interviews were completed several months after the initial NbS project ended, it was difficult for some participants to remember detailed specifics of their experiences during the original project reviews. Lastly, the multi-agency and NbS focus of the study was unique, which could also limit the applicability of the results to other student-practitioner partnership experiences that are not related to international development and the environment.

6. Discussion

Our initial NbS study provided a unique opportunity to bridge the gap between theory and practice for students to engage with large NbS development projects in multiple countries, as well as examine how climate risks and adaptation were or were not included in project design. Although the student educational experience highlighted at the end of the first part of the study only focused on three insights (i.e., team skills and working across perspectives, working expectations and timelines, and complex processes of development organizations), upon reflection by participants, we found that the impacts of this study on student researchers were more complex and nuanced. These deeper insights presented in the Results section may be because the intended results of the initial study were focused on NbS and development project components rather than the student researchers’ experiences, and the time that passed until they were interviewed gave students the opportunity to reflect on their experiences.

Breaking down our results based on the initial feedback, we found that only two of three original themes were re-identified during the interviews. Initial feedback on the relationship between the University of Michigan and University of Maryland students was both upheld and contradicted. While three students felt strongly that they gained skills in collaborating across universities and disciplines, two students shared that the short time frame, the nature of Zoom interactions, and full schedules did not allow sufficient opportunity to create relationships with peers at the other institution. As stressed in the literature [5,7,8,9], change agents are enabled by strong collaboration with others. While feedback on the quality of cross-university student collaborations were mixed, interviewed students agreed that they formed productive relationships with professionals.

The second preliminary result from the initial NbS regarding time management and deadlines was not mentioned during the interviews. Interviewees focused mainly on skills and insights gained from working with professionals from a large development agency, most closely aligning with the third piece of initial feedback on the complex processes of large development organizations. Students were particularly excited about these skills and insights because they could directly correlate these competencies to ones needed for positions they were applying for and allowed them to network and learn how to engage effectively with professionals. Having had the opportunity to review projects from 45 countries, funded by five major development agencies, was deemed a particularly positive addition to students’ resumes.

6.1. The Impact of the Project on Students by Background

When examining student outcomes by background, it was clear that they experienced the initial study differently and accordingly, developed different feelings of competence toward becoming a global change leader.

For example, there were differences in response between students at UMich-SEAS based on their disciplinary background. Although two of the three students did not come in with a background in environmental sciences, only one student felt that they had learned about conservation techniques as a result of participating in the study. In contrast, all three agreed that they had learned important terminology and norms for practice in the international development field. The two first-year students who had not yet finished their core courses felt that they gained more from the experience than the second-year student who had already completed these requirements and their Master’s capstone project. The latter student emphasized that they had already gained collaboration and other professional skills as a result of their Master’s capstone project. They did, however, stress that the study had been especially important for developing their science communication skills that they had not gained as part of other curriculum experiences. This perspective was reinforced in our pedagogy review, e.g., [9]. In contrast, the UMd-SPP students felt they gained the same skills as their first year UMich-SEAS peers.

We also found a significant shift between students who reviewed projects in a foreign language and those who did not, with students who read foreign language projects feeling an added sense of accomplishment and ability to make positive impacts on climate change adaptation in non-English speaking countries. Because we originally did not intend to review projects written in French or Spanish, this was an added benefit that we had not anticipated based on the literature review. Given that climate change is a global issue that spans beyond country borders, effective change agents should be able to work beyond their national borders.

The most significant difference in students’ response was for those who participated in the project’s NVivo analysis and those who did not. Students who engaged in this analysis displayed more in-depth knowledge and indicated that they were more comfortable collaborating with the project’s professionals. These students were most at ease with asking questions and sharing their skepticism about the study’s results. Students who did not engage in the NVivo analysis indicated that they gained tangible skills and insights but felt removed from the results and the ultimate ‘systems’ value of the study. This is consistent with findings in the literature [6].

Lastly, all students were invited to attend the virtual STAP/WCS/Moore NbS Workshop and were impressed by being able to work with and learn from the expert participants, including the Head of the Biodiversity Convention, the Canadian Ambassador for Climate, the VP of Conservation International, Director of Environment at The Moore Foundation, the CEO of the Wildlife Conservation Society, etc. [19]. These students were excited to have participated in this event, along with scientists, practitioners, diplomats, and NGOs. In light of the virtual format, however, they did not feel as though they had the same networking opportunities as if the workshop had been in person.

6.2. The Impact of COVID-19 on the Student Experience

The COVID-19 pandemic did not significantly alter our methods for the initial study which would have taken place both synchronously and asynchronously over a remote platform as collaborators were already in three different locations: Ann Arbor, MI, College Park, MD, and Washington, DC. In addition, COVID-19 did not result in a change of plans to have students review projects with a peer partner. Unfortunately, however, the extra challenge of the pandemic prevented partners, even at the same institution, from working in the same room. In addition, class schedules between the two university programs often meant only a subset of the eight students could meet on Zoom at one time. Importantly, the pandemic also made it difficult for students to meet informally with professionals outside of the STAP Secretariat during the workshop; thus, the kind of personal networking that can often build lasting connections when there are opportunities to meet during coffee breaks and meals were not possible. While the intention was for more and stronger collaborations, the changes as a result of the pandemic may have resulted in strengthening other important climate change leadership skills such as adaptability and self-motivation.

7. Implications

7.1. Implications for Global Change Leadership

Future global change leaders will be working to both mitigate climate change, and adapt to changes underway, while lifting communities out of poverty, pursuing equity, sustainable livelihoods, and solving a host of environmental and societal problems. This will require leaders to think in ‘systems ways’, to ensure that climate progress also brings co-benefits in those requisite areas of concern.

As a result of having the opportunity to review development projects and engaging with professionals from these agencies, students agreed that they had a unique opportunity to learn climate-related terminology, risk evaluation, and progressive adaptation strategies over 20-year timelines. Ultimately, students learned to evaluate successes and failures in nature-based and climate change adaptation solutions to build ‘lessons learned’ and ‘best practices’ for informing future projects—as Redman and Wiek note, sustainability education at the university level helps students evaluate problems and solutions, but it does not fully prepare graduates to ‘do’ sustainability. They suggest we need to help students gain ‘implementation competence’ as well—“a largely unexplored space for university programs” [20]. Evaluating projects from the early proposal stage from mid-term evaluation (and often needed adaptive management to achieve goals) through to terminal evaluations five years later, allowed students to engage in the ‘doing’, including examining the tradeoffs across social and environmental outcomes. Having students approach these projects initially from their environmental (UMich-SEAS) or policy perspectives (UMd-SPP) allowed for interesting discussions about equity, empowerment, governance, and regional capacity. The ready availability of development professionals allowed questions to be answered and procedures explained quickly. Frisk and Larson argue that “the transition towards sustainability will require action and change that is guided by an understanding of the complexities that arise within an interconnected system, as well as the ability to collaborate with people from diverse backgrounds” [21]. This study afforded that opportunity with projects across global regions, development agencies, and with partners from different institutions and thus helped prepare students to be better global change leaders in their future lives.

The pace and growing impacts of environmental degradation are such that we need to move quickly to solve complex inter-related intractable global problems. That requires transformational change, not incremental change. Glasser argues that tomorrow’s environmental leaders must also have “a better understanding of human capacities, capabilities, and competencies for facilitating societal transformation toward sustainability” [22]. Several students felt that their involvement in the NbS study did just that and as a result, helped them to obtain employment post-graduation. For one student working on developing energy efficiency pilot projects, participating in the NbS study helped them develop needed skills in reviewing project documents, from ideation and design to final evaluation. Although these energy projects are not NbS, they felt that the skills were transferable and essential to their success. Another student explained that participation in the NbS study enabled them to learn about NbS project components and best practices, in ways that allowed them to create a niche for themselves in their work at an environmental think tank and in the US Department of Housing and Urban Development. What one student and the co-author of this article learned about the GEF helped them obtain a position in the organization’s evaluation office. For this individual, skills gained on evaluation protocol development, project review, and qualitative statistical analysis were invaluable to evaluating other portfolios of climate projects.

7.2. Implications for Global Change Leadership Teaching

Classroom instruction can support mastery of climate change core competencies, but combining theory-based insights with dynamic, practical project-based analyses with practitioners are likely to be necessary for developing skills that allow students to become change agents. With climate change taught primarily in disciplinary courses at UMich-SEAS and in one specialization at UMd-SPP, students felt that the NbS study, which was conducted like an independent study rather than as a course, provided them with a framework for a “Project Skills” course that could fill the gaps and bridge the respective institutions’ curricula. The proposed full semester course, ideally taught by a professor with support from a professional, would combine traditional classroom learning, including readings, discussion, and lectures, with an overarching project engaging practitioner clients. UMich-SEAS students suggested that the proposed course could follow similar course models as in the program, in which students prepare hypothetical products such as an interview guide for use in the field. However, the final products would be commissioned by a practitioner and used in the practitioner’s future work. In case of a project similar to that of the NbS study, students in the class could review projects each week and their reviews would be accompanied by classroom lessons on the various components of the projects, including theory of change, multi-stakeholder dialogue, behavior change, climate risk screening, monitoring evaluation and learning, and on tools used in the field, such as NVivo. In other words, class time would be used to focus on necessary tangible skills for completing the work at hand. At the end of the class, students would ideally be able to present their results to the professional client(s), to make a meaningful contribution to their work. The students felt that the proposed class could be a good extension to already existing courses and bridge the gap between disciplinary and more theoretical courses on climate change economics, adaptation, and mitigation. Such a course could also be scaled to other programs across the country.

8. Conclusions

Experiential learning is an important complement to traditional classroom learning [7,11,12,20,21]. The GEF NbS study afforded students from two universities—an environment school (Michigan) and a public policy school (Maryland)—the opportunity to work together with experienced development professionals from five agencies and analyze real-world projects in 45 countries, representing a total USD 377 million investment.

The complexity of development project design, implementation, review, and evaluation would likely be difficult for students to fully comprehend solely through traditional classroom instruction. Facilitating “hands-on” collaborations with development practitioners to help students review projects from the proposal stage to full approval stage (a year later), from mid-term evaluation (and possibly adaptive management to improve results and/or refine the original ‘theory of change’) to the outcome stage, provided participating students with a clear view of the project cycle. For example, students learned how to seek improvements in both social and environmental outcomes simultaneously, even if a given GEF project had the main purpose of addressing climate change, or biodiversity, or land degradation. They also learned how a set of intended interventions may not play out due to funding issues, serendipity, war, or lack of involvement of the requisite stakeholders to design culturally appropriate strategies.

The fact that global issues need to be addressed simultaneously to move us toward sustainable development brings to the fore the difficulty of monitoring and measuring success along multiple axes. Climate change has the potential to erode the gains that have or could be made on social and environmental fronts, and students could see how well, or poorly, future climate impacts were incorporated into actual development projects.

A global change leader is versed in and can communicate science, knows the terminology and trends in the field, understands how climate change impacts all aspects of life, and values the need for solutions that strengthen both the environment and society [23]. Engaging in the NbS study not only provided students the opportunity to gain these skills but to do so alongside global change leaders. Experiential learning opportunities like this have the potential to strengthen the ability of the students to act as change agents as they enter the field and for the rest of their careers.

Development agencies spend hundreds of billions of dollars each year to advance environmental goals and livelihoods. Analyzing these projects for lessons learned, best practices, and ways to integrate across sectors and multiple goals can serve as a valuable educational experience and help link theory to practice in graduate programs. Future global change leaders can learn a great deal from such analyses, enhance their systems thinking, and be better prepared to transform society toward sustainability after completing their academic training.