Integrated Outreach: Increasing Engagement in Computer Science and Cybersecurity

Abstract

Research on innovative, integrated outreach programs guided three separate week-long outreach camps held across two summers (2018 and 2019). These camps introduced computer science through real-world applications and hands-on activities, each dealing with cybersecurity principles. The camps utilized low-cost hardware and free software to provide a total of 84 students (aged 10 to 18 years) a unique learning experience. Based on feedback from the 2018 camp, a new pre/post survey was developed to assess changes in participant knowledge and interest. Student participants in the 2019 iteration showed drastic changes in their cybersecurity content recall (33% pre vs. 96% post), cybersecurity concept identification within real-world scenarios, and exhibited an increased ability to recognize potential cybersecurity threats in their every-day lives (22% pre vs. 69% post). Finally, students’ self-reported interest-level before and after the camp show a positive increase across all student participants, with the number of students who where highly interested in cybersecurity more than doubling from 31% pre-camp to 65% post-camp. Implications for educators are large as these activities and experiences can be interwoven into traditional schooling as well as less formal camps as pure computer science or through integrated STEM.

1. Introduction and Context

Computer science topics have spread into many science, technology, engineering, and mathematics (STEM) fields as related skills have become a prerequisite for solving technical problems. In fact, by 2029, employment in computer and information technology occupations is projected to grow 11% while creating over half a million new jobs [1]. This has prompted a growing demand for computer science literacy, creating a gap between current education offerings and industry/labor demands. While many educators equate coding and computer science, and while coding is fundamental to many computer science courses, computer science also involves computational thinking, algorithm and program design, and problem-solving processes [2]. Simply stated, computer science and programming are not synonymous [3]. Many current courses focus solely on programming instead of the broader and more transferable concepts and content found within computer science and computation thinking spaces. In fact, there is little agreement of what belongs in pre-collegiate computer science curricula and what does not [4], making curricula generation difficult for even experienced educators.

Furthermore, rapidly advancing technology has introduced numerous cybersecurity vulnerabilities, underscoring a need for cybersecurity professionals and, by extension, substantial cybersecurity education, a broad topic well situated within the blossoming arena of computer science. Many schools struggle, for a variety of reasons, to offer quality education in these subjects [5]. Outreach opportunities, after school or in the summer, may help to support traditional education efforts, while providing unique experiences for students of all ages. This research study specifically focused on how a cybersecurity outreach program, implemented as three free “open to all” week-long summer camps, impacted students’ (aged 10–18 years) content learning and interest in cybersecurity.

In addition to a lack of basic computer science and cybersecurity content knowledge, soft skills which include effective communication, teamwork, and conflict resolution are important for students as future STEM professionals [6,7]. Information and technology advances have changed how people communicate, socialize, and co-exist [8,9] and soft-skills are as desirable in working STEM professionals as core technical skills [10]. Currently, many college graduates lack leadership, organizational, communication, and critical thinking skills [11]; however, prior work has shown that, with explicit instruction and guidance, students can understand and apply soft skills [6]. There is an opportunity for university students to engage in outreach events to expand their opportunities in professional skill sets of communication, leadership through mentoring, and gaining confidence in their field of study [12,13,14]. There should be a focus on these skill sets in STEM professions such as flexibility, multidisciplinary problem solving, teamwork, conflict resolution, and communication [9].

With all of these considerations, educators have different ideas of what STEM education should include [15]. Computer science can (and the authors argue should be) integrated into the STEM fields and courses to prepare students in developing solutions to today’s interdisciplinary problems (e.g., tracking infectious diseases, modeling animal behavior, investigating water quality, developing new materials). The STEM disciplines can intertwine with each other, strengthening each other as one large field rather than four distinct fields by interacting with real-world, authentic problems and by demonstrating the connections of ideas and skills in all of the disciplines [16,17]. Although not the focus of this work, utilizing an integrated STEM framework [16,18] could offer insight into using real world applications, building skill sets, and engaging collaborative teams. Computer science offers a space for these skills focused on real-world problems as well as intelligent solutions to issues facing the STEM community. Previous research has shown the importance of integrating computer science into STEM [17] and that computer science concepts can be taught embedded within traditional STEM lessons in pre-collegiate settings [3]. Students should be able to recall/apply what they have learned to solve real-world problems [19] and utilize problem solving exercises while synthesizing and developing core concepts [20]. These types of skills are enhanced with hands-on student-centered learning that increases conceptual understanding rather than educator-centered lectures which focus on recall [21]. Female students in particular benefit from student-centered learning and hands-on activities [22,23,24,25]. Often, student experiences with integrated STEM subject matter promotes engagement [16]. The outreach camps created, implemented, and studied by authors Burrows and Borowczak in 2018 and 2019 build on this research, capitalizing on computer science integration, teaching students to solve problems in creative ways, and showcasing STEM as one interdisciplinary field where problems span multiple disciplines.

The three outreach camps provided week-long computer science exposure through educator activities focused on effective teaching methods, and student activities full of hands-on labs and real-world applications. This type of outreach experience provides computer science exposure to students who may not have the opportunity to learn effective computer science skills otherwise. Availability of these types of outreach activities gives students the opportunity to start building skills that can be applied daily in evolving science and engineering fields. For example, data scientists are growing in demand as Industry 4.0 [26] advancements require large amounts of data to be analyzed. Additionally, cybersecurity landscapes have evolved to include more complex adversaries which necessitates cybersecurity specialists for system protection. The summer camps described in this article introduced pre-collegiate students, age 10–18, to necessary problem-solving skills required across various fields, and sparked increased interest in continuing with a cybersecurity career trajectory.

2. Literature Review

Following a review of the current literature regarding STEM outreach, STEM education, and computer science/cybersecurity teaching methods, three themes emerged that are discussed in more detail after they are initially presented:

• The Need for Outreach Programs and Open-Source Activities: Computer science literacy is lacking or lagging behind and many educators are struggling to provide quality computer science education, making outreach programs and quality open-source activities crucial for meeting the growing demand for computer science professionals.
• Outreach Experiences Support Continued Learning: Outreach programs support continued learning through instilling confidence and motivation, employing hands-on and student-centered activities, and promoting technical skills desirable by future employers.
• Sensible Learning Experiences for Diverse Students: Constructive outreach promotes sensible, novel, and cost-effective activities that fit the needs of many diverse students and promote their engagement.

2.1. The Need for Outreach Programs and Open-Source Activities

Regardless of the pressing need for computer science literacy and integrated STEM education as described earlier, countries around the world are struggling to provide adequate opportunities for students to master crucial skills in these areas. For example, a study of Australian educators found that only 37% of educators felt they could promote a career in computer science and only 26% of educators reported that they were able to host industry visitors to promote computer science [27]. While experienced STEM university faculty could offer assistance and create broader outreach efforts for pre-collegiate students, these faculty activities are often regarded as an inferior mission, subsidiary to university teaching and research obligations, and thus they are often not explored in depth [28].

In the United States (U.S.), the same trends are true, as computer science literacy is lacking and outreach activities/programs which increase motivation could offer a potential solution until school infrastructure has been revisited [9]. Increasing student interest and motivation is critical for fulfilling the growing demand for computer science professionals. In 2016, the U.S. Bureau of Labor Statistics predicted that one of every two STEM jobs would require computer science and three of four require some computer science skills [29]. Only a year later in 2017, computer occupations already made up 45% of STEM jobs [30], nearly reaching the previous five year prediction in a single year. By 2018, the U.S. Bureau of Labor statistics found that computer and mathematical occupations accounted for six of the top 30 fastest growing occupations and, of those six occupations, information security analysts was projected to grow the fastest by 2028 [31]. A report in 2019 found that computer and information scientists are specifically projected to see a 16% increase in employment by 2028, which is significantly higher than the average growth rate across all occupations of 5% [32]. Furthermore, although interest in a few STEM fields has grown, interest in computing and related fields has declined [33], regardless of the growing industry demand. Thus, at least a temporary intervention is required to promote computer science with all students, as outreach opportunities for students must be provided to combat the decreased interest in STEM [4]. Reassuringly, this focus has spurred outreach activities in the U.S. as well as other countries [34], and more are needed.

Educators are asked to do an extraordinary number of tasks, and learning a content areas outside of their realm of expertise is often not possible. However, just searching the internet for help is not enough. Research shows that, when educators with limited computer science content knowledge search for resources, their search phrases are too generic and then yield unproductive results [2]. As an extension, students do not get that additional support at home either, as parents searching for computer science opportunities for their children are generally unsuccessful [35].

2.2. Outreach Experiences Support Continued Learning

Computer science education enables students to solve problems creatively, recognize opportunities for technical innovations, and forge career paths in numerous different fields [29]. Outreach activities make this continued involvement possible. For example, professional development focused on simple computer programming approaches can propel pre-collegiate educators to integrate new, challenging computer-controlled robotics into their instruction [36]. Importantly, all educators can access lessons with computer science or cybersecurity components (e.g., TeachEngineering.org, csunplugged.org) and find professional development and workshop opportunities (e.g., TeachEngineering.org) [37], but finding the time to invest in learning these concepts and activities is difficult.

Outreach activities support continued student achievement through increased motivation [24], especially since many students who participate in outreach activities do not have much background in STEM fields. In fact, low motivation has been linked to low achievement in individuals with non-science backgrounds attempting to learn science topics [24,38]. There is a universal need to feel connected to others and a desire to experience integration and freedom [39], particularly among young women [40]. This implies that students are motivated by a sense of belonging to a community as well as a sense of agency over their learning. Outreach programs, including summer camps, are one potential avenue to achieve this level of connection. Keeping students motivated requires building their confidence and providing them with a more realistic baseline when assessing their own self-efficacy [41]. Outreach programs can foster achievement, confidence, and self-efficacy, through the creation of inspiring learning environments that demonstrate the need for computer science and encourage continuous learning [42]. Furthermore, establishing long-term student interest requires support from educators [43,44], and educators should be involved in maintaining the learning environment that houses outreach events that spark motivation [34].

Student commitment to learning increases when they have a sense of respect from instructors, such as expressed admiration or using examples that students identify with [45]. This allows for outreach staff to serve as role models for students [45,46] and helps students form practical perceptions of future professions. Additionally, student commitment is increased as images of professionals become more accurate and attainable [45,47]. Furthermore, outreach camps can provide student spaces to break free of stereotypical images of STEM professionals that can broaden student involvement, supporting diversity.

2.3. Sensible Learning Experiences for Diverse Students

Outreach activities should be sensible and reasonable. This means that the materials and methods chosen should properly support learning. In STEM fields, particularly computer science and cybersecurity, this means considering current trends in a quickly changing technology landscape and creating novel approaches for teaching new concepts. Many STEM fields benefit from the use of relatively expensive materials; regardless, cost-effective approaches should be taken to provide outreach activities. This ensures that they are maintainable, repeatable, and extensible. In addition to monetary restrictions, outreach availability is largely constrained by activities that are only offered in a single location [48]. Some outreach opportunities are established through university programs, and they are often limited to the university’s city. While outreach activities are vital to supporting local interest in STEM, they do little to reach many diverse students.

Outreach programs should also create conducive learning environments for diverse students [4]. Simply exposing students to STEM topics does not guarantee an impact on the diversity of STEM fields [49]. Therefore, students’ interests and motivating factors must be considered to provide conducive learning environments. For example, activities should be suitable for a wide age range, maintaining flexibility while ensuring core concepts are conveyed. By gathering students across wide age ranges, outreach opportunities provide unique experiences for students who are normally surrounded by others close to their age. Furthermore, studies have shown that students develop attitudes towards STEM professions early in their lives. Young girls start making decisions about future careers before entering even middle school (around age 11) [20,50,51]. Always being surrounded by similar classmates limits the opportunity for students with different backgrounds to learn from each other. For example, older students are robbed of the benefits of leading younger students and, more broadly, establishing early leadership roles. Therefore, outreach opportunities have implications in supporting diversity in STEM fields. In particular, women are underrepresented in STEM fields and care should be taken to research gender differences in learning patterns [49] before making important decisions about the activities included in an outreach program. For example, research shows that hands-on activities are particularly effective for female students [24].

2.4. Research Questions and Instruments

The identification of the lack of pre-collegiate computer science instruction, along with possible outreach solutions, led the authors of this article to address current gaps in understanding pre-collegiate students’ interest, awareness, and knowledge of cybersecurity. The researchers investigated three research questions during the week long summer camps:

• How does a week-long summer outreach program impact student interest in cybersecurity?
• How do summer outreach activities influence student security awareness and privacy practices?
• How do hands-on activities and student-centered labs impact student content knowledge after voluntary cybersecurity summer camps?

3. Methods and Context

During the summers of 2018 and 2019, three free summer camps were offered for pre-collegiate students in the Rocky Mountain region of the U.S. These camps provided an introduction to computer science with a particular focus on cybersecurity. Topics were presented to student participants through real-world applications, cross-disciplinary approaches, and hands-on activities. All of the camps were led by by experts in computer science, cybersecurity, and education as well as collegiate camp staff. These camp staff members were undergraduates majoring in computer science and graduate students with expertise in computer science and cybersecurity.

3.1. Methods

The authors of this article utilized field notes (qualitative methodology) and surveys (quantitative methodology) for the three summer camps. After the first two summer camps in 2018 [52], the authors used the field notes to evaluate the strengths and weaknesses of each and improvements were made for the third summer camp in 2019. One of the most relevant assessment changes for 2019 was the creation of a new 10 question pre/post survey (see Appendix A.1 with nine content questions and one student interest question) which the team utilized in conjunction with demographic survey data to answer the research questions presented earlier. Furthermore, the 10 question survey instrument captured a high-level assessment of students’ cybersecurity interests, practices, and knowledge. Beyond gauging baseline interest in the field, the survey asked students to identify important cybersecurity concepts, addressed security and privacy practices through real-world scenarios, and tested for specific content knowledge, all in a series of multi-select and ordering questions. Thus, in 2019, two student surveys (demographic and 10 question content/interest) and field notes (taken by the article’s authors) were the main data collection methods.

The demographic survey was completed by the middle and high school students’ parents during the initial registration process, while the 10 question survey was completed by the student participants once before the week-long camp and again during the final hour of the week-long camp. On the reverse side of the 2019 pre-camp survey (before camp activities began), the students wrote a question about computer science or cybersecurity. Throughout the camp, instructors touched on each of these questions/topics through conversations, real-world examples, and hands-on activities. At the end of the camp, the students were given the 2019 post-camp survey with the same content questions. On the back of the post-camp survey (end of the week), the students rated the outreach week experience from one to five (five being the best) and they wrote a comment explaining why they gave the camp that rating. There were a total of 24 paired surveys in which the students completed both the pre-camp survey and the post-camp survey.

3.2. Demographics and Context

The 2018 and 2019 summer camp demographics (n = 84), summarized in

Table 1. Gender breakdown of student camp participants.

	# Participants	# Females	# Males
Camp 1	23	5	18
Camp 2	31	13	18
Camp 3	30	5	25
Total	84	23	61

and

Table 2. Grade and age breakdown of student camp participants.

U.S. Grade Level	5th	6th	7th	8th	9th	10th	11th	12th
Age Range	10–11	11–12	12–13	13–14	14–15	15–16	16–17	17–18
Camp 1	0	2	4	3	3	2	7	2
Camp 2	1	4	11	4	4	1	3	3
Camp 3	0	0	10	9	5	4	1	1
Total	1	6	25	16	12	7	11	6

, show an average female to male gender breakdown of 27% to 73% and an average age of about 13 years old, with 49% of the students in grades 7 and 8 (ages 12–14). While the advertised age range for the camps was set at 13–18, several younger students (n = 7) ultimately participated as well. In addition, though the percentage of females participation in the camps does not reach parity with male participation, the percentage of female participation is 10% higher than the average U.S. computer science program.

These camps utilized a set of best-known practices, grounded in literature, and activities for informal education tailored to participant demographics. These practices included the use of a structured and predictable schedule which featured hands-on unplugged activities in the morning and engaged inquiry and computer-based programming lab activities in the afternoon (see Appendix B). The unplugged activities served to introduce basic computer science and cybersecurity concepts which scaffolded student knowledge in order for them to be successful during the hands-on programming of Micro:Bit devices. There were multiple opportunities for collaboration daily and throughout the week. For example, during the programming labs, student participants were grouped into smaller staff led/supported problem solving teams. As students worked towards solving real-world cybersecurity problems (through the application of computer science concepts in the programming of Micro:Bits), the camp staff supported the student teams in their problem solving activities by asking questions, providing advice, offering potential solutions, and sharing insight into potential security vulnerabilities of solutions.

4. Results

The results from the three camps were categorized into themes from field note observations and survey results. The first result set explains the field note observations that were made during the camps, how each successive camp was improved based on these findings, and how those improvements affected future camps. The second result set details the effectiveness of the camps based on the survey (given before the 2019 camp and again after the week-long 2019 camp).

4.1. Qualitative Results: Field Note Observations and Improvements by Theme for 2018 Summer Camps

4.1.1. Theme 1: Adjusting for Initial Learning Curves

Many of the students attending the 2018 summer camps had never been exposed to programming or computer science basics. This created a bit of a learning curve during the first few activities since the students were introduced to new hardware, new software, programming, and computer science topics all at once. While some students could liken the Micro:Bit to a Raspberry Pi or Arduino, other students were not familiar with similar technology. For students without this prior exposure, the bridge between programming and actually implementing code on hardware was a completely new concept. Furthermore, since programming benefits from logical yet creative thinking, some students struggled with the initial code. When creativity outweighed logical reasoning, the student had many ideas on how to solve certain real-world problems (e.g., how to advance their robot), but struggled with implementing those solutions programmatically. When logical reasoning outweighed creativity, students were able to create new functionality for their devices, but many of these upgrades were not particularly useful in solving the given problems.

While the introductory activities in the 2018 summer camps taught students how to use the programming environment, it did little to sharpen their problem solving skills. Therefore, introductory Micro:Bit 101 activities were introduced in the summer 2019 camp to address the issues regarding baseline programming knowledge. Although the Micro:Bit website has online tutorials that work with students to create interesting projects, the Micro:Bit 101 activities designed by the researchers were more beneficial to the students than the online tutorials since they were built towards specific learning goals (called level-up guides). Specifically, the tutorials showed the students how to program the devices (e.g., showing which blocks go where in block programming). This approach gives a quick answer and can appease students, but stifles creativity, does not help the students build problem solving skills, and robs them of the opportunity to discuss their solution differences with others. Furthermore, while the tutorials helped students achieve functionality on their Micro:Bits, it did not help students understand the logistics of a given problem, internalize a solution, and synthesize that solution into a program. Since all of the answers were functionally the same, students were unable to use soft skills and communicate to others what made their code unique. Conversely, by providing students with the hand crafted Micro:Bit 101 exercises in the summer 2019 camp, students were able to build their problem solving skills (while still learning the programming environment and Micro:Bit basics).

4.1.2. Theme 2: Maintaining Attention and Engagement

The unplugged activities in the 2018 camps had less engagement than the programming activities. To raise this engagement in 2019, methods were used such as likening activities to real-world applications and keeping the students actively engaged. The students were grouped into two sections based on their self-reported experience level (either novice/intermediate or intermediate/advanced). Each group was instructed simultaneously by a different instructor in a separate room, and then the groups switched rooms. This raised student attentiveness by requiring students to focus for shorter periods of time, provided less competition for open discussions, and allowed the outreach staff to interact with them in more specialized conversations.

Furthermore, although the basic curriculum remained the same for the unplugged activities, it became more interactive in 2019 to stimulate engagement. For example, in 2018 camps, the “telephone game” was played in order to demonstrate how a message could degrade as it was passed from one person to the next, showing a lack of integrity (a principle of computer science information theory). This game was well received as many students had played the game before, and it was a familiar concept to them (pass the message from the front of the line to the back of the line). This introduced an important cybersecurity concept: What if a message is deliberately changed? While opening the door to conversations about a “man-in-the-middle” attack, the conversation was cut short. In 2019, this activity was lengthened and given more focus. Additionally, a lesson on Wi-Fi was developed to further solidify the concepts and relate them back to students’ daily lives.

4.1.3. Theme 3: Providing Explicit Real-World Applications

The lessons and activities were all planned around common cybersecurity concepts and computer science challenges facing professionals today, with specific care taken towards relating the topics to student experiences. However, in the 2018 labs, while there were strong parallels between topics and students, the burden for drawing those parallels rested in the hands of the students. For example, one activity involved beacons that sent messages over radio waves and receivers that read those messages, mimicking current beacon technology (used largely for advertising and marketing). While this parallel existed, students were not able to draw this connection on their own. Without properly forming these connections to the real-world, the activity lost quality as students struggled to find ways to make sense of why these beacons degrade security. To solve this issue, in the 2019 labs, camp staff made specific connections to the students’ previous knowledge. While the students were all familiar with robotics in general, they were unaware of many of the security concerns associated with this technology. Simply explaining the security vulnerabilities was not sufficient in grounding their knowledge in real-world applications. To fully relate robotics security, the staff explicitly drew comparisons to the real-world. Instead of likening the activity to robotics in general, the staff gave explanations on precisely how they are similar. For instance, the students quickly learned that it was not particularly difficult to control their friend’s robot. This demonstrated a common attack vector in unmanned aerial vehicles (UAV) technology or self-driving cars. However, rather than trusting the students to draw these comparisons, specifics were given that directly mapped the activity to these types of issues. Questions were posed such as “How does this relate to self-driving cars?” and “What would happen if this was an actual car and not just a robot?”

Being explicit about the real-world implications of the labs solidified student learning as well as gave students a place to connect the lesson to their life. In other words, clearly explaining connections between lessons and the real-world gave students familiar topics with which to associate the new knowledge. Lastly, consider password strengths. Many people today do not understand why their password should be long and contain different types of characters. To these people, a five character password made up of lowercase letters should be hard to crack since there are ${26}^5=11,881,376$ total possible passwords and 11,881,376 is a big number. However, 11,881,376 is actually quite a small number considering how fast a computer can generate and test the passwords. Accordingly, when the lessons were grounded in real-world applications (like password strengths), not only was the concept clearly linked to students’ existing knowledge, but the proper scaffolding was included to ensure retention.

4.1.4. Theme 4: Enhancing Incentives and Motivation

During all three camps, the students were able to track their progress using their individual badge, and the badges were fit with a ring of light emitting diodes (LEDs) that demonstrated their achievements. This gave the students incentive to work through the labs and stretch to achieve goals that might have seemed out of reach. The students were not required to complete the most difficult levels of the labs or create the best code; instead, they were motivated to solve problems to the best of their ability.

While the badges provided a low-level of motivation in the 2018 camps, the potential incentive was stifled by the inconsistency of the reward. In 2018, the camp staff leveled-up the badges (or increased the score) when they felt the students had demonstrated mastery over a new concept. In 2019, this system was standardized such that activities had specific checkpoints (or level-up guides) that, when reached, granted a student to level-up their badge. This created a direct expectation for levels and better motivated the students by giving them clear goals tied to specific incentives. Furthermore, in the 2018 labs, the students worked to level-up their badges throughout the week (no daily differentiation). However, in 2019, each day marked a new opportunity to level-up their badge. At the end of each day, the students’ level was reset to zero meaning that each day they had the opportunity for success regardless of what happened previously in the camp. This better motivated students because they were not as discouraged by earlier mistakes.

4.2. Quantitative Results: Descriptive Statistics for the 2019 Summer Camp

The three research questions presented earlier form the basis for the 10 question pre/post survey (see Appendix A.1) that was administered to all the students. For the purpose of this study, the survey is segmented into four distinct areas of “Confidentiality, Integrity and Availability” recall (Q1) and applications (Q2–4), followed by online security (Q9), and finally camp interest (Q10).

4.2.1. Q1: CIA Recall

Student participants gained knowledge and confidence in being able to correctly identify the components of the CIA triad—namely confidentiality, integrity, and availability. The CIA triad is integral to cybersecurity and drives many of the decisions made in the field. The CIA triad was the first concept taught to the students and each lesson related back to these three core principles. Likewise, the first question on the survey was based on recalling this most basic cybersecurity lesson and asked: When talking about information security, what do the following letters represent, C, I, and A? The question asked the students to “select all that apply”. When asked this question before the camp, almost half of the students selected that C, I, and A stood for Central Intelligence Agency and only eight students (33%) selected the correct answer (Confidentiality, Integrity, and Availability). After the camp, however, the wrong answer was only selected three times and the correct answer was selected 23 times (96%). These results can be seen graphically in Figure 1—with only two students answering more than one answer (in both the pre-camp survey and the post-camp survey).

4.2.2. Q2–4: CIA Applications

Student participants identified real-world applications for each component of the CIA triad—with drastic improvement in understanding availability and integrity. Additionally, students generally understood applications of confidentiality and availability at higher rates than the concept of integrity. Questions 2–4 of the survey focus on the application of the CIA concepts. During the camp, each of these concepts were related to real-world issues. For example, confidentiality was linked to media streaming platforms. Many people share their passwords to media streaming platforms. Average users feel safe doing this because they trust the person with whom they are sharing the password. However, many people also use similar passwords across all of their accounts. This means that they are trusting that person to use that password appropriately, as well as trusting them to keep that password a secret at the risk of multiple accounts being compromised. This underscores a significant breach in confidentiality because, while people may share passwords with someone they trust, it becomes difficult to maintain privacy. Students understood this example since they use passwords on a daily basis and are familiar with media streaming platforms. To relate integrity to the real-world, the camp staff asked the students if they had social media accounts and what they would think if they logged on one day to find their account filled with random information. This demonstrated the importance of protecting data from being changed by unwelcome parties. This was also solidified through a lab activity involving the Micro:bots. The students were not told to pick a unique channel for their robot to receive control messages on. This meant that the students were sending control commands to other students’ robots and the robots acted erratically. The students learned quickly that ensuring valid data was integral to controlling their robot, and in turn they learned how integrity impacts their day-to-day lives. To relate availability to the students, camp staff asked the students if they played games online, a common pastime for most students. Online games require a connection to a server. If the device (e.g., computer, gaming console) cannot connect to the Internet or the server cannot handle all of the traffic, the game becomes unavailable. Games that are continuously unavailable, or devices that consistently lack functionality, lose value over time and the companies providing them are less likely to be successful. While students did not seem to particularly care about the business aspect, they were able to identify that when services they want are not available, they go somewhere else (e.g., they pick a different game).

To determine the effectiveness of relating the CIA triad to real-world examples, the surveys included three questions that gave an example of a security failure, then asked the students to identify the lapse in security. They were given five multiple choice options and were asked to “select all that apply.” The questions were also shuffled such that the answers were not in order with the CIA acronym. If the student selected the correct answer (even if they also selected other answers), the answer was categorized as correct. If the student did not select the correct answer, their answer was classified as incorrect. The results are shown in Figure 2.

The students demonstrated the greatest improvement in the availability question with 10 (42%) answering correctly before the camp and 22 (92%) answering correctly after the camp. In other words, the number of students who were able to identify availability as a security concern (when given a real-world example) more than doubled during the camp. The number of students who answered the integrity question correctly also increased dramatically. Although only 15 (63%) students selected the correct answer after the camp, only three (13%) students answered correctly before the camp, meaning five times as many students answered correctly. The difference in the pre-camp surveys and post-camp surveys for availability and integrity both increased by 12 correct answers; however, since so few had answered the integrity question before the camp, the students showed more class-wide improvement on the integrity question, but a higher level of class-wide mastery on the availability question. The question concerning confidentiality, a less foreign concept to the students, demonstrated the least improvement between the pre-camp and post-camp surveys. Specifically, 18 (75%) of the students answered correctly before the camp and 20 (83%) answered correctly after. Finally, notice that more students answered the availability question correctly after the camp than the confidentiality question, even though the confidentiality question had much better pre-camp results. This likely means that the activities relating to availability were particularly relatable to the students.

While the first question indicated that students were able to recall the CIA triad and what each letter stands for, the next three questions showed that the students were also able to apply those concepts to real-world examples.

4.2.3. Q9: Online Security

Student participants recognized the implications of sharing information online, with a majority successfully identifying the risks associated with certain seemly benign social media activities. All students determined that sharing information has an impact. Other topics beyond the basic CIA triad concepts were also considered. During one activity, the students were shown a common “What _____ character are you?” meme. These memes assign popular characters to people based on some identifying information. For example, it may determine what popular movie character each person would be based on their birthday, the first letter of their middle name, and the name of their first pet. The students reviewed the memes and, when asked what the point of the meme was, most answered that the meme was supposed to be funny. However, these memes are usually created using common security questions and other personal information. When someone posts their character, they are posting the answers to their security questions and breaking confidentiality best practices. During the pre-camp and post-camp surveys, students were asked: Online surveys and “What _____ character are you?” memes are... (See Appendix A.1 for potential answers). Before the camp, the most popular answer was that the memes were funny with 10 (42%) students selecting that answer and seven (30%) students selecting each of the other answers. After the camp, the most popular answer was the correct answer that the memes contain security questions, with 18 (75%) students answering correctly. These results can be seen in Figure 3. This shows that the students understood that the memes contained security questions, but also understood that they are not meant to be funny (they are meant to discover personal information) and that the memes do have an impact on unsuspecting people.

4.2.4. Q10: Cybersecurity Interest

Overall, student interest level increased by the end of the week long outreach camp, with all students having a medium to high interest level and with the percentage of students highly interested doubling. Recall that one of the main goals of the outreach activities was to increase interest in computer science and cybersecurity, and the students were asked how interested they were in computer science and cybersecurity both before and after the camp. Before the camp, four (17%) students said that they were only minimally interested in cybersecurity and indicated that they were unsure of cybersecurity. Eleven (46%) students indicated that they were moderately interested in cybersecurity and seven (30%) students reported that they were highly interested in computer science/cybersecurity, finding it fun and exciting. This showed that before the camps, the students were generally unsure of what computer science and cybersecurity entails, but, after the camp, none of the students reported being minimally interested in computer science and cybersecurity. The number of students highly interested in cybersecurity increased to 15 (63%), more than doubling, and seven were moderately (30%) interested. In total, 22 students (92%) were interested in cybersecurity. These results indicate that the camps increased the students interest in cybersecurity and computer science and they are shown in Figure 4.

4.2.5. Camp Rating

During their post-camp survey, in addition to indicating their interest in cybersecurity and computer science, the students were asked to rate their camp experience and write a comment explaining their camp rating. None of the students ranked the camp as a “one” out of five or “two” out of five, indicating that all of the students enjoyed the camp to some extent. Only three of the students (13%) ranked the camp as a “three” out of five. Of these students, one student noted that they would still recommend the camp to others. Six students (25%) rated the camp as a “four” out of five and 15 students (63%) rated the camp as a “five” out of five. These results indicate that the outreach week was engaging and the activities held the interest of the students. They also indicate that some modifications could be made to improve camps in the future for some students. These results are also shown in Figure 5 and the comments are listed in

Table 3. Comments from the students along with the rating they gave the camp.

Rating	Comment (Spelling Errors Corrected)
3	Longer lunch.
	Too much hand drawings. Not enough programming antivirus or firewalls.
	I don’t think that I would do another one of these camps but it was definitely an experience that I would recommend to others if they haven’t done something like it already.
4	I enjoyed the camp but felt there was a lack of focus on firewalls. What they are and how they work, and viruses, how to avoid them, and how to remove them before a complete wipe.
	This camp was really interesting and fun and I really liked it but there were some boring parts like the hand, but the programming and hacking were fun.
	It was fun.
	This camp was very fun.
	Because it was fun but some minor adjustments would be better.
	It was fun hacking and programming.
5	The camp was interesting, the activities were mostly really fun. I also gained knowledge and skill that will help me in the real-world.
	I chose this due to some lack of explanation on somethings ex ciphers, overall it was great.
	Because I would want to come back to this camp if I had the opportunity to and it teaches you about online services and how to protect your accounts.
	Because it was fun to hack other peoples microbots.
	I had a lot of fun and met friends.
	It was exciting and I would totally do this again and it boosted my interest in cybersecurity.
	I had amazing leaders who helped me and encouraged me.
	Because I now am more interested and knowledgeable than I was before I started this camp.
	I want to do this camp again to gain more knowledge.
	Fun and interesting.
	very fun/interesting and would like to do it again.
	It was a blast and I learned so much. I learned a lot but it did not feel like school and I enjoyed it. I liked programming the Micro:Bits and the robots. Learning how to code was a ton of fun and it was very enjoyable. I would like to go to another camp like this because it had the best activities and was a nice way to spend a week in the summer.
	We learned important information about safety and security, explored coding and hacking, and were informed about potential employment.
	I loved this camp, it was super fun and informative. It had great projects.
	It was interesting to me because I like computers.

.

The comments provided a qualitative context in support of the quantitative assessment, and they clarified which components of the camps were well received and which ones were not (from themes discovered in the 2018 camps’ field notes).

5. Discussion

The results, implications, and limitations are discussed in this section in relation to the three main research questions. The authors highlight connections to current literature and propose future research questions. A list of suggestions are also provided for those creating student outreach opportunities.

5.1. Research Question Discussion

Related work in this field demonstrated a need for outreach programs and open-source activities in response to growing demands for computer science and cybersecurity professionals [32]. Outreach programs can increase interest in these fields and motivate students to become not only good programmers, but good computer scientists [9]. This central issue guided the first research question: How does a week-long outreach program impact student interest in cybersecurity? The responses from the 10 question interest survey revealed that the number of students who reported being highly interested in cybersecurity more than doubled between the start and end of the camp. This indicates that week-long outreach programs can greatly increase interest in cybersecurity. In addition, all of the students who indicated that they were only minimally interested in cybersecurity at the start of the camps reported being more interested by the end of the camp. This demonstrated that the camps did not just make the already-interested students more interested, but also increased interest across the group regardless of their ability level.

Recalling that educators should aim to teach more than just programming to their students when teaching computer science [2], they should teach their students through real-world problems that apply to their everyday lives [19]. This idea guided the investigation into the second research question: How do summer outreach activities influence student awareness of their security and privacy practices? One survey question asked the students about online memes that can compromise the privacy of unsuspecting users. While most of the students did not realize the security implications of some of their actions online before the camp, 18 of the 24 students recognized the given situation as a potential breach of their confidentiality. This indicated that the camp activities focused on teaching students to be mindful of their online actions were successful. This result also shows that outreach activities can influence how students think about their regular behavior, conditioning them to be cognizant of the information they share.

Lastly, it is well known that outreach programs should provide hands-on, student-centered activities [19,20,21,22,23,24]. Students responded positively to the hands-on activities and laboratory projects created for this outreach program. This gave students the opportunity to discover cybersecurity lessons through exercises led by university camp staff as well as industry professionals. This engagement enabled the team to address the third research question: How do hands-on activities and student-centered labs impact content knowledge in voluntary summer camps? Data from the first four questions in the survey confirm that students were able to recall important security information as well as apply basic security concepts to the real-world. When asked about the CIA concepts after the camps, 23 of 24 students were able to recall what the acronym stood for, 20 were able to identify a breach of confidentiality, 15 were able to identify a breach in integrity, and 22 were able to identify a breach in availability. This indicates that the hands-on activities and student-centered labs were successful in teaching the students basic security concepts and, by extension, that hands-on activities and student-centered labs had a positive impact on information retention in the voluntary summer camps.

5.2. Implications

Educators and those creating outreach experiences for students can glean insight from the authors’ lessons learned during the three, week-long summer camps. Recall that related work in the field also showed that educators are currently struggling to find helpful resources for teaching computer science and cybersecurity [2,35]. The camp staff focused on creating quality activities that used computer science problem solving and programming to solve real-world problems. To support the growth of usable activities, each of the Micro:Bit activities was made available to educators along with the “level-up guides” at http://www.wycyberstars.com. This supports educators and parents looking to provide students with cybersecurity lessons. Each of the activities has flexible time constraints to fit both short classes as well as longer classes. In addition, since the outreach program welcomed students across a wide age range, the activities were built to suit students with varying backgrounds. The authors created level-up guides with activities (or milestones) that “grow incrementally” from step to step. The authors believe that while novice/intermediate students learned more by going through the first one or two level-up stages, the intermediate/advanced students were able to expand their learning and make meaningful connections throughout the levels while sometimes engaging in extensions beyond the level-up guide.

Other related works stressed the importance of student motivation [24,38] and strong leadership from outreach staff [41,42,45,46]. The camps focused on motivating and engaging students through the badge leveling system. In addition, the camp staff expressed genuine excitement for students when their solutions worked and supported the students when their solutions did not work as students learned through failure. The camp staff shared their experiences in computer science and cybersecurity with the students, modeling that professions in STEM fields are attainable. These mentors came from many different backgrounds including educational professionals, computer science and cybersecurity educators, professors with industry experience, and collegiate camp staff. By providing a variety of models (e.g., female doctoral student, Hispanic female masters student, African-American male undergraduates student, female high school student), students were able to benefit from the mentors in a variety of ways.

Related work also shows that outreach programs can promote the growth of soft skills such as teamwork and communication [8,9,53]. In these outreach camps, students were encouraged to communicate their thoughts and ideas to each other as they worked together in groups to create solutions to open problems. The intermediate/advanced students were often paired up with novice/intermediate students and given the opportunity to lead and mentor them. This created a chain of support for the students. The intermediate/advanced student participants supported the novice/intermediate student participants, while the high school, undergraduate, and graduate students supported all of the student participants while the camp leaders supported the whole team. This promoted leadership across the entire group as well as cooperation and teamwork.

Additional related works were concerned with the sensibility and effectiveness of outreach activities [4,49]. Constantly improving lessons and upgrading laboratory activities helped maintain the camps, keeping them up-to-date with current cybersecurity issues facing the field. The lessons were based on these real-world issues and camp staff explicitly tied the lab back to these applications.

Related works were also concerned about the accessibility of outreach programs [48]. The low cost of the lab materials supported the maintainability of these outreach activities as well as encouraged the use of the materials by educators looking for cost-effective ways to include computer science and cybersecurity in their classrooms. Furthermore, these camps were held in numerous rural locations to provide outreach opportunities to a wide range of students.

Lastly, these camps aimed to align with other related works which highlighted the need to be flexible for a diverse set of students [20,24,49,50,51]. The camps focused on creating lessons that could be taught in multiple ways, making it more efficient for camp staff to pivot when students lost focus or needed assistance in approaching the problem in a different way. In addition, because the 2019 camps provided space for smaller groups of students, camp staff were able to better tailor lessons and methods to students learning needs. The considerations made in the pursuit of providing an effective week-long summer outreach program were successful. This is demonstrated through field notes showing high student engagement and in the high ratings the students gave the camp during their post-camp survey (15 students giving the camps a “five” out of five rating and another six giving a “four” out of five rating).

5.3. Limitations

Despite the improvements made to the outreach experience labs, they were limited by the amount of time students had to work on some projects. To provide a relative structure to the day, regular breaks were built in to accommodate the students’ attention spans. Some of the breaks came at inopportune times throughout the day and disrupted a project’s flow. Some activities held student attention much longer than anticipated, forcing hard decisions about how to structure the rest of the day to provide enough time for each activity. This also inhibited deeper dives into subjects with some students becoming restless (and some students becoming more engaged). In addition, these camps may not be indicative of outreach opportunities and are thus not generalizable, and the scope of the results is limited to week-long, free, voluntary, student summer camps. Many of the decisions made during these particular outreach camps influenced the results since the camp was only a week long. Furthermore, these results only give overall perceptions of the students’ content knowledge, interest level, and cybersecurity awareness. While these perceptions give an impression of how the outreach activities influenced the students, they do not prove that specific activities or outreach decisions were responsible for the discussed findings.

6. Conclusions

This research study was situated at the crossroads of three challenges currently facing computer science educators: (1) need for outreach programs and open-source activities as educators report being under-prepared to provide quality computer science education to meet the growing demand for computer science professionals; (2) need for outreach considerations that support continued learning and include instilling confidence and motivation, employing hands-on and student-centered activities, and promoting technical skills desirable by future employers; and (3) need for sensible learning experiences for diverse students that can be promoted through novel, cost-effective outreach activities.

These camps created many open-source activities with staged levels of difficulty (level-up guides) to provide a wide-range of educator resources for computer science classrooms. In addition, these activities are integrated with other subject-matters, connecting different lessons through common themes, although they are focused on cybersecurity. To support continued learning, high school students and collegiate camp staff were employed to create working relationships and give student participants the chance to work with others and engage in an array of complementary strengths. Additionally, unplugged activities gave students a platform for computational thinking and practice for formulating solutions beyond their current programming abilities. Finally, these outreach activities focused on novel approaches to real-world problems through cost-effective labs (i.e., Micro:bits) that educators can easily integrate into their existing curriculum.

Observational field notes were made to promote improvements for the summer camps. Then, the effectiveness of the outreach camps was assessed through the use of pre-camp and post-camp surveys. Finally, the field note observations and results were discussed in the context of the previous research, explicitly outlining the contribution to the field as well as implications and limitations. The camp surveys showed that outreach programs can positively impact student interest in cybersecurity, raising both awareness and curiosity for cybersecurity. The results indicated that the number of students who reported being highly interested in cybersecurity more than doubled from the start to the end of the camp. Furthermore, the results did not solely indicate that students who were already interested in cybersecurity became more interested; instead, all of the students who were only minimally interested in cybersecurity at the beginning of the camp reported being more interested after the camp. In addition, students showed increased awareness for the implications of their online behavior, recognizing the potential vulnerabilities associated with common online activities. Enforcing cyber-safe practices (e.g., being cognizant of sharing personal information) equips students to protect themselves in an advancing technological world. Lastly, these camp surveys indicated that the hands-on activities and student-centered labs were successful in teaching basic cybersecurity concepts. Specifically, students were able to identify breaches in confidentiality, integrity, and availability posed in real-world scenarios. Holistically, this research concludes that outreach programs centered around cybersecurity can have a positive impact on student perceptions through the use of hands-on labs and student-centered activities, teaching them the importance of cybersecurity and cultivating curiosity in exploring cybersecurity topics.

Based on the results of these three week-long camps, suggestions for educators include:

• Set the stage with baseline information in short (less than 20 minutes) of the topic;
• Connect the topic to the real-world and give explicit and varied examples;
• Have students create a real-world question (from the problem) to investigate;
• Allow students flexibility to change their questions as evidence is collected;
• Provide opportunities for students to practice and fail in a safe space and continue changing their question and collecting evidence;
• Encourage students to use the data collected and analyze it in different ways;
• Promote students’ ideas for improving designs, data collection, and analysis;
• Provide a space for students to share outcomes, lessons learned, potential impacts on society, career connections to the work, applications to other fields, and more;
• Give students choices of topics and sample questions as well as non-examples if they get stuck;
• Promote student engagement by allowing students to find a place of connection with the computer science (or any other) work;
• When possible and appropriate, foster student engagement in real-world scenarios through connections with experts in the field (such as industry professionals;
• Provide computer science activities that rely on computational thinking rather than only programming and stress the importance of responsible code use (from shared libraries, code banks, and widespread tool-kits);
• Be explicit about the real-world implications of each activity to solidify learning and encourage trying again or trying differently when student failure occurs.

The authors are planning future student outreach activities and educator professional development opportunities for upcoming summers. Activities are extended and new activities will be introduced based on the collected field notes and survey results. The future camps reach out to new rural areas through in-person and virtual scenarios. In addition, new research studies could focus on current issues facing STEM fields and cybersecurity professionals, as well as current methods being used to provide outreach opportunities. Research into the industry might reveal new concepts and problems to pose to students, while the research into other outreach offerings might drive enhancements on the outreach logistics. Future research is needed concerning the current status of computer science, cybersecurity, and their relation to integrated STEM education, sustaining interest in STEM fields, and employing outreach programs like summer camps.