A Serious Game for Mediated Education on Traffic Behavior and Safety Awareness

Abstract

Computer games are considered a useful tool for educational purposes. Alternative media applications such as serious games combine edification with challenge and entertainment. Thus, learning becomes enjoyable, more comfortable, and more efficient. The paper presents the implementation of an educational computer game regarding traffic behavior awareness through the main stages of analysis, design, development, and evaluation, aiming at investigating the contribution of gamification in traffic safety. The game was developed as an advanced media education approach in Unreal Engine, encompassing various adventures. The game hero’s tasks are to move into the virtual city to complete a mission, follow road safety rules, and experience the adventures either as a pedestrian or as a vehicle driver. Research hypotheses/questions are tested concerning the gaming impact and the audience engagement through first-person storytelling to communicate and perceive traffic regulations. The results reveal that a properly developed educational game could become more engaging, amusing, and efficient. It could also enhance traffic awareness through experiential and mediated learning, also fostering social responsibility.

1. Introduction

Media can be a component of active learning strategies; instructors move away from the traditional lecture method and facilitate innovative learning interventions to support students, embodying multiple media and gaming components [1]. Computer games are linked to entertainment and are associated with various social aspects as well. Therefore, interest in digital game technology and its use for purposes other than entertainment is constantly growing. Moreover, studies concerning positive and negative game effects on humans and various aspects of society are steadily increasing and becoming ever more prominent [1,2,3]. Most researchers seem to agree that gaming benefits are substantial, even in pure entertaining contexts, as long as the time spent is controlled and care is taken to prevent addiction [2,4]. From this perspective, it is rather pointless to deter kids from playing computer games entirely, with releases that incorporate an edutainment character to be considered a fruitful alternative or settlement [3,4,5]. In most cases, game users spend little time reading instructions. This feature makes gaming and gamification approaches ideal for familiarizing individuals with demanding and/or time-consuming technological topics, such as digital media literacy needs (i.e., borrowing essential gaming elements into practice, like scoring, levels, etc.) [4,5,6,7,8]. Hence, a sub-category of digital games has emerged, the so-called “serious games,” which do not focus exclusively on having fun. Serious games contribute to the achievement of a defined purpose other than pure entertainment, including education, training, health care, marketing, etc. The idea is to provide new ways of interactions for people to learn, analyze, and explore [1,4,5,6,7]. Furthermore, there is a relatively strong relationship between digital media advances and the evolved mediated communication power, which even non-professional developers can deploy in immersive scenarios of virtual worlds and first-person storytelling experiences [4,5,6]. These trends also align with the modern genres of multimedia and immersive journalism, pursuing audience engagement and game-like interactive narratives, enhanced with virtual and/or augmented reality elements [8,9,10,11].

Extending the above, while featured serious games and gamification strategies have been around for more than two decades, new technologies allow the implementation of even more appealing environments to suit various learning needs, including traffic education [12,13,14]. Many benefits are offered from such rich media/playing experience outside the original scope of gaming, such as the indirect effects on several everyday life situations (e.g., traffic safety and behavior, as well as overall social responsibility experiential learning) [13]. Experiential learning refers to knowledge acquisition derived from specific experiences, i.e., generated by actions like observation, reflection, knowledge application, abstract conceptualization, and active experimentation [15]. Since it is not always feasible to perform these experiments in physical surroundings, serious games can provide an excellent learning tool, offering virtual worlds similar to the real ones. Thus, they allow the reproduction of original contexts or even create training situations that only occur in specific circumstances, controlled in more safe and educative ways [16]. In this perspective, highly interactive and immersive environments form advanced media services that can be very productive towards digital literacy and other demanding life-long learning tasks, utilizing media to inform and educate people in more pleasant and effective ways [6,7,8,9].

Each year, traffic accidents cost a million human lives worldwide. According to a World Health Organization report, the number of road traffic deaths steadily rises, reaching 1.35 million in 2016 [17]. The percentage of children is significantly high. It is important to note that 6 to 10-year-old children’s traffic accident rate is four times higher than those among adults [18]. Therefore, it is necessary to consider education programs aiming at positive traffic behavior, one of the main road safety priorities. These strategies have been internationally called the “four E” (4E: Education, Enforcement, Engineering, Emergency Systems) [19]. Traffic education involves a set of rules, which pedestrians, passengers, and drivers should follow while interacting within a broader social context. For this reason, the allocation of attention, the responsible attitude, and the behavior on the streets by pedestrians and drivers are skills that can be acquired within coordinated and well-targeted training [19,20]. There are several methods of implementing a road safety education program for children worldwide, which are considered of paramount importance. Most of them rely on guiding manuals and employ computer technologies while promoting children’s educational programs with practical traffic safety teaching. In many European countries, the different perspectives are of great importance, leading to various participants’ roles, i.e., a pedestrian, a bicycle driver, vehicle passenger, etc. It seems that each country follows a variety of programs and methods to ensure that children develop proper traffic behavior [21]. Still, we are not sure if these specific approaches are the most appropriate or sufficient.

In Greece, where the current initiative has been deployed, the lack of traffic education is demonstrated by the annual rates of road accidents and the high death toll [22]. Despite long-term efforts, an effective traffic program for children is yet to be applied. The content and objectives of related courses should not be general but rather specific and clear, as required for the systematic training to obtain proper knowledge and behaviors. However, the demanded skills are pursued almost exclusively by instructions on safety rules, while exercising in traffic education parks cannot be efficiently implemented [23]. Therefore, it is essential to deploy a compound methodology that combines theory with reaction practicing in a simulated road environment. Hence, testing scenarios that cannot be deployed in real/physical settings and are difficult to comprehend through conventional teaching can be simulated within game worlds, offering virtual navigation. Such initiatives started about two decades ago [24], but, to the best of our knowledge, neither a broader continuation nor the anticipated progress has been achieved so far.

The main hypothesis is that advances in human-computer communication and media-driven education have not been fully productive yet to shape the needed traffic safety and awareness culture. Related experience on game-based learning in primary education has demonstrated that quality content enables practice through stimulating activities, offering fun and pleasurable experiences [25]. The projection of related gaming simulations to real-world driving has also been tested as part of traffic education on adults [26,27], pointing into knowledge gains, even without incorporating all the currently available potentials that dedicated campaigns could exhibit. We argue that traffic awareness should be part of children’s schooling activities, not only within a class but also in infotainment and edutainment time, i.e., through first-person/role-playing games. Interactive applications (mobile [28,29], web [30,31]), multimedia services, and audiovisual material [32,33,34], including educational TV programs and featured documentaries, have played a significant role in this aspect (not only in traffic-related topics but on broader domains as well). The current research aims at elaborating on the above perspectives through the processes of logical user-centered interactive design (LUCID) [28,30,31,32], seeking formative evaluation and feedback to value audience engagement and the anticipated impact of the envisioned traffic game. Overall, a proper virtual interaction environment is targeted to help young people realize that actions on the street directly affect fellow human beings and, therefore, the whole of society.

The rest of the paper is organized as follows. Next section discusses related work, reviewing important works on the field. Material and methods are then presented, describing the phases of analysis, design, development, and evaluation. Experimental results and discussion illustrate the implementation outcomes and their corresponding assessment, concerning the stated hypotheses and questions, followed by summary and conclusions.

2. Related Work and Targeted Innovation

Over the past years, a growing multidisciplinary interest in alternative educational approaches has been observed. Serious games form a vital tool for influencing cognitive evaluation [1,2,3]. Relevant studies report that the practices of gamification and serious games are mutually bound, as both attempt to leverage gaming aspects to achieve something beyond playfulness [3,4,5,6]. Gamification frameworks have been deployed for informal training by providing visual media engagement and attractive storytelling. The integration of game elements, i.e., points, levels/stages, badges, leaderboards, prices and rewards, progress bars, feedback, and role-playing, facilitate participation, motivation, enjoyment, and productive learning [7,35,36,37]. Regarding the traffic safety topic, gaming approaches could improve the ability to recognize and properly manage dangerous situations, providing a safe environment where users can practice traffic rules correctly [27].

Several studies have been conducted regarding traffic safety with positive outcomes and benefits in users’ behavior after engaging in serious games. In [12], Mobility, a 2000s game that simulates city development, is put to the test, reviewing subsequent studies on its educational impact, which has been proven positive concerning traffic learning perspectives as well. Elaborating further, [13] studied how drivers’ skills and reactions can be improved through game-based simulations, using a real car panel as the game joystick/interface. A drive simulation game was constructed in [13] to prove the existence of learning results, projecting a positive correlation of playing with car races, action, and sports computer games into real-life driving. Zare, Niknami, Heidarnia, and Fallah (2019) confirmed the positive effects of active learning with parental involvement on promoting safe street-crossing behavior among school-aged children engaged in visual stories and toys (dolls and cars) [18]. Studies [19,20] demonstrated and compared the so-called good practices on road safety and traffic education. Assailly (2017) presented a workshop and its positive effects on young drivers who have just obtained a driver’s license [19]. The next report appraises the use of visual and interactive material offered through an e-learning module package [20]. This 2005 guide highlights training methods separately for each road user category (i.e., car passengers, pedestrians, cyclists, public and school transport, moped users, pre-drivers). Effective traffic education practices also include driving simulation tasks, advertising and promoting social norms messages, collaborative learning, student-centered discussions, interactive lessons, role-plays, prevention videos, and presentations by safety experts [19,20]. These sessions can be supplemented by real roadside training, discussion panels/fora, media assets and videotaped/filmed material related to traffic rules/signs, and visits to public transport enterprises, supported by leaflets and brochures. Lee and Al-Mansour (2020) concentrated on printed visual aids, proving that cartoons, graphics, and pictures can effectively contribute to children’s traffic behavior [21]. Li (2015;2020) showed that game-based learning could increase players’ knowledge of highway code and road safety [26,27]. Participating students were able to link the game-acquired knowledge to associated experiences of real traffic conditions, keeping this information in their memory for a long time after playing.

Based on the above, it is vital for traffic safety awareness to be more thoroughly demonstrated and communicated within society. Media and news organizations have an essential informing (and infotainment) role in this direction, which should play through mainstream news-reporting and newfangled digital journalism services [8,9,10,11]. More specifically, rich-media interactive environments, featured documentaries, social media campaigns, etc., can be deployed to raise public attention and consciousness on this sensitive matter. Hence, interactive multimedia and immersive storytelling form media advances that should be considered today’s innovative concerns in that direction, pursuing audience engagement, informing, and edification. Practicing and developing skills while simulating on a “digital road” offers additional benefits, i.e., to avoid actual traffic conditions that might be dangerous. Players can comprehend essential rules, attitudes, and behaviors more easily, enjoyably, and effectively. In this context, the proposed serious game policy combines the advantages of easy, experiential, and entertaining informing, while also promoting media literacy and knowhow of all the involved parties, in alignment with the recent trends of multimedia, mobile, and immersive journalism [6,7,8,9].

According to the presented research findings, game-based simulations can enhance various learning aspects in driving education. Relevant computer gaming approaches improve children’s alertness and traffic behavior [13]. However, there are critical perspectives that are covered in the current state-of-the-art. For instance, both driving and pedestrian mediated scenarios, offered within the safe digital world, are equally essential to cultivate traffic awareness steadily and systematically in early and middle childhood, before grown-up children are solely involved in real-world traffic situations. The envisioned game story should incorporate appealing virtual navigation with realistic missions and playing roles, matching up with the standards of first-person storytelling techniques encountered in immersive and highly interactive environments, to promote and augment enjoyable edutainment services. Simultaneously, the game should be easily installed and deployed on present-day personal computers without specialized hardware or software components, favoring broad utilization and experience sharing among the users. Likewise, multiple gaming modes and missions with further leveling and scoring options were considered imperative to attract the targeted audience. A modular architecture was suggested to anticipate social responsibility extensions at various levels. To the best of our knowledge, the proposed solution attempts a holistic approach that neither the already presented driving simulations [13,14] nor the associated practicing and teaching activities [12,18,19,20,21,26,27] have attempted. In this direction, initial requirements are tested through the deployed user experience (UX) design practices and innovations concerning serious games and gamification components. Hence, different perspectives are covered concerning media advances and education, which can be elaborated to support the continuous training and digital literacy demands in the rapidly transforming media landscape [6,7,8].

3. Research Aims and Project Motivation

As already implied, the paper explores the various factors that drove our decision to develop an educational application. Project motivation was stemmed at the initial investigative stages from the confidence that serious games can offer new experiences and capabilities. The main goal of digital games is to generate pleasant moments/feelings and infotainment values, increasing motivation, the interest in learning, and improving engagement in the implicated tasks. Hence, most studies consider the results of gamified processes or activities not only educative but also entertaining. Serious games and gamification approaches are considered promising future educational methods, especially for children. Moreover, a serious game focusing specifically on traffic safety may serve as an excellent complementary learning approach used by students themselves or in combination with the teacher’s interaction [1,5,20].

The methodological framework of the educational game puts targeted users at the center of the design (human-centered model/LUCID) and usability assessment is employed at almost all the phases of project evolution [28,29,30,31,32].

Consequently, based on the above analysis, the research hypotheses (RH) in the conducted work are stated, as follows:

Hypotheses 1 (H1).

Individuals do not have adequate education on traffic behavior and safety.

Hypotheses 2 (H2).

If featured topics were selected and adapted to appropriate gaming scenarios, experiential learning through gamification could become more engaging and amusing, thus much more efficient.

Hypotheses 3 (H3).

A properly developed educational game could improve not only students’ but also adults’ traffic awareness.

Based on the above hypotheses, the implicated research questions (RQ) that are investigated are the following:

RQ1: What would be the audience response on the proposed game for traffic education?

To what extent would targeted users show interest and eagerness to get involved in such a gaming experience?

RQ2: Is it possible to quantify learning results and gains through gaming/gamification on traffic behavior?

RQ3: Are digital competency and computer skills promoted and enhanced through the engagement with the proposed serious game?

RQ4: Could further research, knowledge, and literacy be stimulated through serious gaming experiences, making it productive in advanced media education concepts?

To address these RHs/RQs, continuous communication, review, and reinforcement processes were conducted during the multimedia production phases, including the formative and final evaluation of the developed gaming environment.

4. Materials and Methods

This work involves the methodological and technological framework of an educational game design. Prior to actual implementation, it was necessary to acquire knowledge in combining the learning procedure with stimulating interest and entertainment. As already stated, the adopted implementation model follows the principles of human-centered design. A spiral methodology was applied, containing the initial idea, the phases of analysis, design, development, and evaluation, deployed in conjunction with the agile development principles. Evaluation procedures took place in every iteration cycle to implement, elaborate, and refine requirements and details until the targeted criteria were finally met [4,5,6,31,32,33].

4.1. Analysis

In this first phase, the multidisciplinary production team determined the focus of the project. The whole process specified further (and refined when needed) targeted users/audience, objectives, constraints, problems, and possible solutions. The analysis led to a more thorough problem definition, forming an initial plan proposal to guide the design.

Three main procedures were deployed in this phase. The first process was the exploration of the basic traffic education concepts and the examination of related teaching procedures in different countries. Greece’s particularities were further examined, investigating related efforts and today’s needs, along with the respective school settings. The second important task was the review of related traffic safety and awareness applications. Production team members (including authors of this article) inspected currently available educational environments and associated gaming approaches. Audience analysis was deployed next through interviews, focus groups, and online questionnaires to determine the main objectives and aims of the anticipated game in relation to the various users’ profiles, indicated by the collected demographic and psychographic data.

The initial analysis resulted in the description of some basic requirements and guidelines to be used in the remaining design iterations. Therefore, a catalog with related topics was formed to highlight the interesting ideas and possible best scenarios to adopt. A broader list of themes was created after an in-depth review of the relevant literature (

Table 1. The main topics/items for the traffic education gaming software [21].

Listed Topics/Items for the Traffic Game Scenarios
1.	Pedestrian safety
2.	Road signs
3.	Risk of fatigue while driving
4.	Illegal use of mobile phone while driving
5.	Exceeding the speed limits
6.	Reactions in case of a traffic accident
7.	Road safety in urban areas
8.	The impact of weather conditions
9.	Priority of emergency vehicles
10.	Use of seatbelt
11.	Driving under the influence of alcohol
12.	Observation of correct distances between vehicles
13.	Ramps and parking
14.	Traffic safety at night
15.	Bus lane
16.	Emergency lane
17.	Familiarity with the Highway Code
18.	Crossing the street
19.	Choosing the safest route

) [21]. In this process, some of the listed items were preferred to prioritize the more prominent issues, also taking into consideration the difficulty of the requested development and the associated playing experience. The main gaming components (tasks, leveling, scoring, leaderboards, etc.) were carefully tested, validated, and reshaped during the overall development and evaluation stages, resulting in the elucidated list of topics, as presented in Table 2 of the Design sub-section that follows. The rest of the items remain available for future/extended releases to elaborate on game leveling and difficulty perspectives.

4.2. Design

The initial analysis resulted in the description of basic requirements and guidelines to be used in the remaining design iterations. In this stage, block diagrams were created to depict data flow within the application. Storyboards were also designed to provide rough sketches of the envisions screens, the associated educational scenarios, and the navigation structure. Attention was paid to the issues raised while playing. At that moment, it was essential to specify the plot of the game and its elemental composition.

The idea of having a robotic traffic assistant involved in the process was decided, driven by related research on educational applications, which appraised positively such game-play helpers [26]. According to [38], feedback on players’ mistakes can have a positive impact, as they devote time to correct them and enhance their knowledge on the relevant topic. Constructive feedback helps decrease frustration and promotes users’ ability to retain information and deeper cognitive processing. At that time, searching for an appropriate well-targeted assistant’s name began. As the next section explains, the Greek name was a play on words from the global term RoboCop and the Greek acronym for the Highway Code [5], which was adapted as RoboRoad in the English version.

Overall, the analysis outcomes led us to finetune the structure of the game. Four main scenarios with a similar pattern were created. In this way, comprehensive yet appealing game-playing was attempted, tailored to the broader users’ interests. The user chooses the desired stage and starts to explore the city when an accident suddenly occurs (for instance, a building is on fire). At this moment, the assistant (RoboRoad) announces the mission, and the countdown begins. The game hero (driver or pedestrian) needs to arrive at the destination (e.g., the fire department) in-time while following the traffic rules. If players manage to accomplish the mission, the stage is completed, and RoboRoad rewards them. Otherwise, a failure message is displayed with a “try again” option, supplemented by returning to the main menu link, i.e., to select a different mission. During exploration, several events happen in the street/screen asking users to make decisions (for example, the cell phone is ringing while driving, what should they do? Answer it or reject it). Depending on the choices, RoboRoad provides feedback concerning traffic rules and safety protocols [26]. The main scenarios include a wide variety of topics in traffic safety education as listed in Table 2 below.

Table 2. The scenarios of the digital game and the relevant topics depending on the game play choice (pedestrian or driver).

Stage	Topics
	Pedestrian	Driver
Tickets	🗸 Street crossing 🗸 Road signs	🗸 Use of seatbelt 🗸 Stop at a red traffic light
Fire	🗸 Proper use of the horn 🗸 Proper use of the crosswalk	🗸 Proper use of the horn 🗸 Driving carefully even in case of emergency
Night	🗸 Risk recognition 🗸 Potentially dangerous contacts	🗸 Rejecting cell phone use while driving 🗸 Traffic policeman and traffic lights 🗸 Amber light and turn
Wallet	🗸 Problems at crossing from parked vehicle	🗸 Priority of emergency vehicles 🗸 Parking places

Table 2. The scenarios of the digital game and the relevant topics depending on the game play choice (pedestrian or driver).

Stage	Topics
	Pedestrian	Driver
Tickets	🗸 Street crossing 🗸 Road signs	🗸 Use of seatbelt 🗸 Stop at a red traffic light
Fire	🗸 Proper use of the horn 🗸 Proper use of the crosswalk	🗸 Proper use of the horn 🗸 Driving carefully even in case of emergency
Night	🗸 Risk recognition 🗸 Potentially dangerous contacts	🗸 Rejecting cell phone use while driving 🗸 Traffic policeman and traffic lights 🗸 Amber light and turn
Wallet	🗸 Problems at crossing from parked vehicle	🗸 Priority of emergency vehicles 🗸 Parking places

Furthermore, the production team members concluded that it is crucial for the player to be able to choose the game mode. More specifically, the choice of more active involvement of the player from both sides of the street was intended. In this settlement, the player has the chance to explore the city as a pedestrian or a driver. By experiencing both scenarios, users will better realize the street risks and dangers caused by others.

Taking the deliverables of the analysis as the inputs, low-fidelity prototyping iterations were deployed during this phase (mock-up design). The main screens of the initial menu and the directions for stage selection with the associated instructions were designed in Unreal Engine 4.19, while variant RoboRoad logos were sketched (Figure 1). All the created material mentioned above was refined during the successive spiral design. Color pallets, chromatic codes, and screen patterns were chosen in response to users’ preferences, as decided by the production team. Processes and outcomes were systematically discussed and evaluated with the participating users, providing formative validation feedback before reaching the final decisions.

4.3. Development

In this phase, authoring takes over to convert a design plan (i.e., storyboard, low-fidelity prototypes, navigation structure, etc.) into a multimedia project. As already mentioned, the design and development processes co-exist within the adopted spiral method. Hence, the already presented plan elaborates and adapts to the emerged requirements. However, some strategic decisions have to be clarified in this section, as well [1,32].

Concerning RoboRoad development, the fundamental design requirements advocated in the creation of a first-person educational game within a 3D environment, compatible with the majority of computing devices. Thus, it was decided to develop the project in Unreal Engine 4.19.3. Unreal Engine is an open and advanced, real-time 3D modeling environment developed by Epic Games. The Unreal Engine possesses significant progress over time, with detailed documentation and a large community for users to be informed about various topics. Additional assets, models, sounds, environments, and other features are also provided for purchase or free of charge at the Marketplace (the digital storefront). Continuously evolving, it gives users today the freedom and control to create and deliver virtual worlds by using a visual programming method known as blueprints. Thus, Unreal is considered a user-friendly tool that can be deployed without requiring coding skills. This feature is very important given the multidisciplinary nature of the multimedia production team and the associates involved in educational/instructional design and formative evaluation procedures [4,5,6,28,29,30,31,32].

Specifically, the RoboRoad application is structurally split into two components, the blueprints and the widgets. The Blueprints Visual Scripting is a complete gameplay scripting system in Unreal Engine, relying on the concept of using a node-based interface to create gameplay elements from within Unreal Editor. Widgets, on the other hand, are the screens being shown during the gameplay (for instance, the initial menu or events that the user needs to make a decision). Indeed, they are not only simple images, but they are also characterized by visual scripting and function due to widgets’ blueprints [5]. Hence, if the contained buttons are clicked/activated, they can lead to another screen with functionality or back to the game (Figure 2).

4.4. Evaluation

As already pointed out, assessment is conducted throughout all the development phases within the LUCID model, proven to be a key element in multimedia production, especially if a human-centered design is involved (as in the current work). Overall, the main scope of the evaluation is to provide useful feedback for rectification regarding imperfect behavior or unjustified deviations from prior phases. It can have the character of formative or final valuation, expressed in qualitative and/or quantitative terms [1,32]. This plan was also followed during the RoboRoad assessment procedures concerning the validation of analysis, design, and development derivatives, through qualitative feedback (comments and suggestions), measurable metrics, and key performance indicators.

The formative assessment process took place in two stages. A respective small panel of experts was formed in the first phase, consisting of two individuals from the multimedia/web authoring and UX design sectors, supplemented by an educator and a game developer. Specifically, the first two hold a diploma degree in electrical and computing engineering and a Ph.D. degree in media technologies and audiovisual semantics, respectively, with expertise in multimedia/web authoring (Expert #1) and in UX design concerning mobile applications and web interfaces (Expert #2). The third specialist holds a diploma degree in music, currently serving as an elementary school teacher, with digital media and sound design experience in related audiovisual and multimedia productions (Expert #3). The last person (Expert #4) holds a bachelor’s degree in software engineering and has also been involved in game development projects. All four participants had related backgrounds in the field as gamers.

Experimental/inspection sessions and discussion groups were organized to identify the weaknesses or failings of the multimedia product. The “Five Es of usability” (effectiveness, efficiency, engagement, error tolerance, ease of learning) combined with the Nielsen metrics were used for testing and validation procedures. Base measures were adapted to the characteristics of this multimedia production, its functionalities, and the targeted audience/end-users [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42]. At the end of this stage, the experts participated in the relevant questionnaire. The production team proceeded to all the necessary changes based on the received qualitative evaluation reports (presented in the Results section). During the second stage of the formative evaluation, post-graduate students in a “Multimedia Production Workshop” class (seven in total) participated in the assessment procedure by playing the game and answering the relevant questionnaire (

Table 3. The analysis and usability evaluation questionnaire.

Question (Indicative Answers—Range)
A	Are you familiar with digital games (1–5)
B	What do you think as the main objective of the RoboRoad game? (entertainment; infotainment; traffic education; traffic safety awareness)
C	Were the objectives of the stages clear and easily understood? (1–5)
D	Did you manage to complete the stages easily? (1–5)
E	Which stage was the most difficult? (tickets, fire, night, wallet)
F	Were the stages entertaining? (1–5)
G	Does the game become repetitive and boring? (1–5)
H	What do you think about the two game mode options (pedestrian/driver)? (useful; repetitive)
I	Are you satisfied with the knowledge obtained from the game? (1–5)
J	Does the game reflect your interests? (1–5)
K	Did something stimulate your interest in the beginning of the game? (1–5)
L	Do you prefer learning through gaming, instead of other common methods? (1–5)
M	Did you find it more difficult to complete the stages as a pedestrian or as a driver? (pedestrian; driver)
N	After playing as a driver, can you realize the drivers’ mistakes in real life? (1–5)
O	Would you adhere to the same traffic rules in real life? (1–5)
P	Do you believe that you would improve your traffic behavior after this game? (1–5)
Q	Would you be more aware of other drivers/pedestrians after this game? (1–5)
R	Was it easy and simple to move around in the game? (1–5)
S	Did you comprehend the instructions easily? (1–5)
T	Are you satisfied with the virtual environment and the 3D graphics of the game? (1–5)
U	Do you believe that your computer skills were improved after this game? (1–5)

). After the formative evaluation and the changes made, the final version of RoboRoad was released and uploaded to be assessed by the targeted users.

Table 3 depicts the analytic structure of the formed questionnaire. During the survey preparation, all procedures and rules suggested by the “Committee on Research Ethics and Conduct” of the Aristotle University of Thessaloniki were followed. The declaration of Helsinki and MDPI directions for the case of pure observatory studies were also taken into account. More specifically, the questions were shaped/hosted online to be fully anonymized, explaining to candidate subjects that they would agree to the stated terms upon sending their final answers. They were also informed that they have the option of quitting anytime without submitting any data. In case of underage people, the above directions were presented both to the children and the escorting parents, who gave their full consent.

The final assessment session was scheduled to be employed at project completion among primary school students. In a first step, a pilot assessment procedure was set up at the Laboratory of Electronic Media of the Aristotle University of Thessaloniki. Schoolkids, family members of associates and friends, were recruited for this pivotal validation session to provide initial feedback and establish best practices for repeating the process. Two groups of ten people were formed for conducting two successive classes. Τhis sample of twenty (20) persons in total consisted of (11) eleven girls and nine (9) boys, the ages of which were varying between eight and fourteen years old.

A traffic education workshop was conducted with the help of the game, with a computer being reserved for a pair of students playing the game in succession (i.e., one was handling RoboRoad while the other was watching and vice versa). This approach was adopted to speed up the process but, mostly, to check if playing/controlling difficulties would affect the whole experience (driven by related comments received during formative evaluation). Moreover, it was arranged for all users to experience both the pedestrian and the driver modes. Subjects were directed/supported while playing and overtly observed by a production team member to examine their reactions, interactions, approaches, and behaviors. The monitoring was focused on the participants’ “virtual” traffic behavior, representing qualitative data collected for further analysis, i.e., to combine them with the quantitative results and the estimated impact on real life. The average duration of this procedure was estimated at one hour, including the preparatory settlement phase and the initial directions. A quick break was scheduled, allowing those who would not desire to continue with the evaluation to leave. The remaining participants (all in our case) were asked to answer the online questionnaire concerning their gaming familiarity and impressions regarding fun, excitement, and clarity of objectives. The process was distantly/discreetly supervised, without supervisors having direct sight on their monitors, to avoid discomforting users, therefore, to eliminate biasing their responses. Players were also asked to judge the game tasks and their effectiveness in improving their traffic behavior and awareness. Demographic variables were recorded along with technology familiarity metrics to extract the correlation outcomes and meaningful conclusions. Most of the questions were structured in a categorical form of potential answers, with five-point Likert scales (1–5, from “Not much” to “Very much”). Binary values were also involved (i.e., stages), as Table 3 depicts.

It is worth noting that the initial plan was to extend the evaluation to additional representative users. However, the coronavirus disease 2019 (COVID-19) pandemic situation did not allow to run focus and assessment groups within elementary school classes (i.e., in dedicated computer labs where the software could be massive installed and executed). As a substitute, alternative participation was considered, i.e., through conferencing rooms and virtual classes, discarding direct/physical monitoring and supervision. The same pairing test (one playing, one watching) was attempted. Again, the recruitment was made through friends and families due to restrictions and the lock-down measures caused by the pandemic situation. Specifically, seven additional pairs-of-users (fourteen users in total, not necessarily belonging to the target audience) also participated in the reviewing procedure by playing the game remotely, with a production team member operating the teleconference to overtly observe the users’ visual reactions and progress. Finally, eight additional sole users were randomly invited to a distant evaluation interview after presenting the gaming experience through recorded videos and subsequently playing the game in a one-to-one teleconference session. In all these cases, participants were appropriately informed (as previously presented) and gave their consent (parental consent for children) before they were asked to answer the relevant questionnaire. These mediated communication sessions with the twenty-two (22) total remote users were deployed to seek further evaluation insights/feedback and check statistical consistency of the results. However, these samples were not counted in quantitative evaluation results, given that share screen degradation and delay deficits deteriorated the assessment transparency (at least, compared to the physical one). Summing up, the initial sample of thirty-one (31) participants involved in the physical meetings (11 in the formative and 20 in the final evaluation) is considered entirely adequate for the implicated pilot validation process [1,30,32].

5. Experimental Results and Discussion

Elaborating on the previous paragraph, this section presents the qualitative and quantitative results, which include the implemented RoboRoad (http://m3c.web.auth.gr/roboroad/, accessed on 15 March 2021) serious game, its educational impact, the analysis of the associated usability assessment outcomes, and the answers to the stated RH and RQ.

5.1. Implemented RoboRoad Gameboard and Offered Educational Services

The delivered multimedia production managed to combine both educational and entertainment elements regarding the specific topic of traffic education. The starting screen (Figure 3) contains a short welcome text, accompanied by a corresponding funny robot sound, linked to the main menu (start, instruction, exit). By choosing the start button, the four main scenarios are shown on the screen, and the user can select the desired stage and mode, i.e., either as a pedestrian or as a vehicle driver. Selection-adapted instructions are displayed along with a relevant video. When the game begins, the player starts exploring the city, searching for the yellow arrow.

Once reaching this point, a new screen accompanied by the same distinct sound is displayed, and RoboRoad assigns a mission, depending on the chosen stage. The player starts looking for the red arrow that denotes the final destination (Figure 4). While wandering around, several events and accidents occur, for which the user should make crucial decisions. A countdown timer is also shown at the top-right corner, while featured music shapes the mood and the atmosphere.

If the player manages to reach the destination on time, by following the rules, a congratulatory screen is displayed, and RoboRoad prompts to the next stage. It is worth mentioning that the player needs to accomplish the appointed mission in a specific time interval, following the circulation rules. Otherwise, a “game over” screen is displayed, accompanied by essential consolidated feedback explaining the type of infringement, thus helping users to realize wrong behavior and errors. At that point, “try again” and “main menu” options are offered. Based on the qualitative evaluation outcomes that we will subsequently analyze, a related video was added to this screen (Figure 5).

5.2. Analysis and Usability Pilot Evaluation

As mentioned above, the evaluation was conducted during all the multimedia production phases of the project (analysis, design, and development) in both qualitative and quantitative terms. The assessment procedure was carried out, on the one hand, to continuously control the achievement of the initial objectives (design and development specifications), and on the other hand, to reveal key characteristics for subsequent development. It is known that for the creation of a user-friendly application, the combination of formative and inferential reviews is considered crucial. In this case, the process involved the evaluation of the application within the production team, with the help of a multidisciplinary experts’ group, and through indicative/selected users of the targeted audience.

5.2.1. Formative Evaluation Results and Associated Updates

The comments, remarks, and modifications proposed by the experts’ team were focused on the gameplay/content and the graphical user interface (GUI). Regarding the game content, positive feedback was provided during the formative evaluation phase. Moreover, supplementary remarks were noted, identifying usability and GUI weaknesses.

Table 4. A summary of the changes made to the original RoboRoad version, after the qualitative evaluation by experts, which led to the final game release.

Qualitative Evaluation Feedback and Respective Game Updates
1.	Limit the range of motion of the camera and modify the vehicle-camera distance to serve users’ convenience (Expert #1, #2, #4).
2.	Move the speedometer at the bottom right corner of the screen according to the “five Es of usability” (Expert #2, #3).
3.	Add pause menu according to the “five Es of usability” (Expert #3).
4.	Convert the start key of the mission from an arrow to “GO” according to the “five Es of usability” (Expert #2).
5.	Differentiate the background music of each stage, depending on the mission, to eliminate repetition and potential boredom (Expert #4).
6.	Add video guides to the instruction screens for effectively supplementing the existing text (Expert #4).
7.	Reduce the size of the vehicle to facilitate its control (Expert #2, #4).
8.	Change the “exit” option into “main menu” on the Game Over screen to serve users’ convenience (Expert #1, #2, #3).
9.	Change the “try again” option into “continue” on the pop-up screens to allow margins of error (Expert #1, #2).
10.	Provide three chances (“3 lives status”) for “driving at the sidewalk” and “wrong way driving” violations to allow margins of error (Expert #2, #4).

presents the experts’ feedback, including specific updates that led to the final version of the RoboRoad game. Below, several crucial modifications are selectively analyzed.

In the first game version, every traffic violation, minor or major, led the player directly to a “Game Over” screen and back to the starting point of the same stage compulsorily. A pop-up window was displayed on the screen, and the player faced a dilemma; the wrong choice led to a “Game Over” screen and forced the player to repeat the same stage. This context was considered quite strict and likely to result in tediousness and annoyance for the player. Furthermore, it was noted that, while playing as a driver, a person not familiar with digital games might have difficulty controlling the (digital) vehicle. Taking into consideration both the above comments, it seemed necessary to allow margins of errors. More specifically, when a wrong choice is made, crucial feedback about the correct answer is now given, and the “try again” option has been changed into “continue”.

In the same context, a second and a third chance (“3 lives status”) for specific traffic violations was additionally provided in driver modes (i.e., driving at the sidewalk and wrong way driving). Furthermore, it was decided to differentiate the music background in each stage by choosing a well-fitted track, i.e., adequately motivational and, at the same time, relevant to each scenario. In this way, the stage repetitiveness impression could be avoided. Moreover, the textual instructions before each stage were deemed quite conventional and tedious for the user; hence, a relevant video guide was added to enrich the media experience by vividly and effectively complementing the existing text. According to the “five Es of usability”, additional key buttons were placed on the selection screens to serve users’ convenience. Likewise, some buttons were modified to change the stage selection path (e.g., the “exit” option at the “Game Over” screen was changed into “main menu”).

Furthermore, as it was mentioned above, during the formative evaluation, four (4) experts and seven (7) post-graduate students in a “Multimedia Production Workshop” class played the game and answered the relevant questionnaire. The quantitative data acquired at this stage were analyzed separately, i.e., not included in the final sample, while the participants were not considered targeted users (within the targeted age intervals). However, the formative evaluation outcomes are considered supplemental, helping to check the consistency of the final assessment results and, overall, to validate the usability and the educational character of RoboRoad. Figure 6 provides graph statistics for some critical questions, mostly regarding estimations of the educative impact on real-life after playing the game (namely, #I, #J, #L, #O, #P,#Q, and #U).

5.2.2. Final Evaluation Results

As already mentioned in the Evaluation subsection, the final assessment procedure was accomplished in two steps. In the first one (pilot validation in the Laboratory of Electronic Multimedia), twenty (20) targeted users participated. In a second step, twenty-two (22) individuals were also involved in the reviewing process (by actually playing and sharing the game screen during a teleconference or watching related video-recorded sessions). As already explained, this latest sample was formed to increase the number of participants and check the consistency of the results (given that physical testing on primary school premises was not feasible at that time). Indeed, it was observed that the statistic moments of mean and standard deviation were rather minimally affected (roughly < 0.4 ± 0.3) and remained in alignment with the formative evaluation estimates of Figure 5. However, the final assessment data do not count this latest sample of remote participants, considering that the teleconferencing process might have deteriorated the transparency of the whole approach. Moreover, this analysis is linked to typical alpha and beta testing, i.e., standard validation procedures of software and multimedia engineering tasks, and does not refer to a social science survey seeking to extrapolate the results (especially given the pilot character of this initiative). Hence, it was decided that the final evaluation sample would include only the twenty (20) indicative users participating with their physical presence in the primary testing, who were also considered among the targeted audience.

Overall, the users considered the interface of the game functional. For instance, eighteen (18) of the twenty (20) participants admitted that they managed to complete the stages without facing any difficulties (Likert scales: 3–5, 90%), while only two (2) found it somewhat challenging (10%). RoboRoad usability was proven high, as indicated through the positive answers to many relevant questions. Likewise, positive feedback was also given in questions, attributing comprehensiveness of the instructions and general clarity of the objectives.

Questions concerning real-life impacts after playing the game (i.e., the adherence to traffic rules, the improvement of the subject’s traffic behavior, and their awareness of other street users) could not be verified but only estimated because players could not be considered real drivers. Nevertheless, some of the results are worth mentioning. The participants’ majority answered positively in questions concerning the advancement of their traffic behavior. A 65% also considered that their engagement with the RoboRoad game could raise their street safety awareness, while 90% deemed that they could better now realize drivers’ mistakes and real-world traffic violations. Furthermore, 80% of the evaluators expressed their willingness to adhere to the traffic rules in real-life effectively, exploiting the knowledge produced through playing. Similarly, 95% of the users expressed their preference to learn through game-playing instead of other conventional methods. They also considered it very useful to play both game modes, pedestrians and drivers. Finally, 18 out of 20 individuals stated that they were very satisfied with the knowledge obtained by the gaming experience (Likert scales: 4–5); therefore, the educational impact of RoboRoad is confirmed. Figure 7 provides graph statistics for some vital questions (namely, #I, #J, #L,#O,#P,#Q, and #U), which seem to be aligned with the corresponding results of the formative evaluation (Figure 6). Again, these values provide a rough estimate of the educative impacts on real-life after playing the game.

Regarding the virtual environment and the 3D graphics, eighteen (18) users declared that they are satisfied (Likert scales: 4–5), while only two (2) reacted neutrally. Positive feedback was also given in queries related to the entertaining and stimulating content of the game. Another significant issue is linked to computer skills improvement. Eighteen (18) out of twenty (20) people supported that playing RoboRoad helped them improve their computer literacy (Likert scales: 3–5). It is also worth mentioning that 85% of the participants found it more challenging to complete the stages as a driver rather than as a pedestrian. Hence, the actions taken during the formative evaluation stage regarding the margin of error have proven necessary, effective, and targeted.

Furthermore, a correlation analysis between the questionnaire parameters was also conducted, extracting even more interesting results on how a serious game affects players’ behavior. Even if these reactions refer to a virtual world, no one can disregard that important messages could pass through, which is the main goal of the current approach. Chi-squared tests were conducted at a significance level α and further supported below (Figure 8).

It should also be mentioned that the project was twice presented in the Greek national television network and related sites with educational content. While this point demonstrates the relation of the topic with media-driven education and featured informing campaigns, it also increased the game downloads and the summarized feedback received by the production team. In all cases, the already presented results were very thoroughly validated and strengthened, so that it was decided not to seek extra evaluation reviews, by more participants.

5.3. Discussion and Answers to the Stated Research Hypotheses (RH) and Questions (RQ)

Summing all the received feedback during the conducted formative and final evaluation, essential conclusions can be drawn concerning the validity of the design, which could provide some initial answers to the stated hypotheses and questions. A significant relationship between gaming familiarity and the comprehension of both the instructions and the objectives of the stages is established. The above statistical analysis also demonstrated that the game mode (pedestrian/driver) is related to the belief that playing RoboRoad could improve the engaged subjects’ emulated traffic behavior, implying the validity of all the three RHs. More specifically, 95% of individuals considered the player’s involvement from both sides of the street (pedestrian and driver) quite useful. Furthermore, 84% of them answered positively (4–5) regarding the belief of improving their traffic behavior after the game. Hence, the indicated headroom on traffic behavior/safety education improvements indirectly validates RH1 (at least to some degree).

The interest and engagement that the vast majority showed about RoboRoad verify RH2 and RQ1, which are also strengthened by assuming that changing roles (pedestrian/driver) while playing could have a positive impact. Likewise, the quantitative analysis outcomes confirm the interest and the learning gains on traffic behavior through gamification (RH3, RQ2). Furthermore, a significant correlation is observed between the entertaining game content and the knowledge obtained through playing; namely, learning through gamification could become, apart from efficient, more engaging, and amusing (RH2). Finally, another interesting and noteworthy result is the relationship between the ease of moving around the city and the raised awareness of the other street users after playing RoboRoad. Overall, it was revealed that the more manageable the control of the character (pedestrian/vehicle) is in the virtual city, the more the awareness of other street users in real-life is expected. Hence, the relevant RH3 and RQ3 regarding students’ attention and behavior are confirmed.

Returning to RH1, some of the results are reasonably expected based on common sense/knowledge that led to the initial idea and the conducted literature review, which is considered a significant part of this research. Extending the above, qualitative data derived from the overt observation during the summative evaluation procedure was supplemented to confirm the inadequate traffic safety education in Greek reality (RH1). Specifically, subjects were observed to exhibit wrong traffic behavior models while playing, especially at the beginning of their gaming experience. Typical such examples include answering mobile phone-calls or violating stop signs in empty crossroads while driving, using car horns practically for no reason, ignoring red traffic lights as pedestrians, etc. After familiarizing themselves with the game concept and being alerted for their illegal/socially irresponsible attitude, they responded by conforming to the rules. Apart from RH1, this finding also verified RH2, indicating how essential alternative learning approaches would be. Experiential learning through gamification becomes more attractive, amusing, and much more effective (RH2). Simultaneously, the game content and flow impact students’ traffic behavior and awareness as future road users (RH3).

Extending the above, the analysis showed that the audience is interested in such approaches (RQ1), which result both in traffic safety and awareness learning skills (RQ2), as well as in the promotion of digital competency and computer skills (RQ3). As already discussed in the Related Work section, serious games and first-person navigation/playing experience are closely related to the recent genres of multimedia and immersive journalism. Hence, the current effort can be aligned with these advanced media concepts and their training dynamics. Apart from the contemporary mediated education perspectives, news and media organizations can use gaming and gamification strategies to inform and enlighten the audience on sensitive social matters, such as traffic awareness and safety. Furthermore, methods and best practices deployed in this work can be further adapted and elaborated to serve specific needs of digital and media literacy. The fact that the multimedia production team involved in this project did not contain experienced software engineers and developers points to the way that selected models, and specific technical solutions could match the multidisciplinary needs for continuous media training and support. All these remarks converge that further research, knowledge, and literacy can be stimulated through serious gaming experiences, making it productive in advanced media education concepts (RQ4).

6. Summary and Conclusions

The present study aimed to determine the effects of an educational computer game on elementary school students’ traffic behavior. In this context, an attempt was made to unveil useful elements regarding the gaming experience and its educational impacts on real-life. The conducted qualitative and quantitative assessments with their subsequent analysis have convincingly confirmed all the stated hypotheses (RHs). Significant outcomes and relationships between the playing experience and the traffic behavior that users emulate in the virtual world were established. Hence, the initials goals of the targeted game development have been accomplished, confidently answering the associated RQs. Adverse gaming effects on traffic behavior skills were not found or indicated. Overall, RoboRoad was positively evaluated by targeted users regarding its educational, amusing, adventurous, engaging, and useful content, along with its GUI usability and simplicity. The results of the evaluation showed that users prefer to strengthen traffic awareness through an alternative teaching way, like a serious game (>70%) (RQ1). Playing with RoboRoad can offer traffic safety education benefits (>85%), with the user representing both pedestrian and car driver roles. Hence, experiences in the virtual environment can systematically turn into practical, real-life knowledge, thus steadily cultivating traffic awareness and responsible behavior. Moreover, game-based learning can be more attractive, amusing, engaging, and effective (RQ2). According to the evaluation outcomes, RoboRoad can enhance the users’ digital skills, as it features an entertaining, immersive experience into a 3D virtual world, making the gameplay and the underlying learning procedure intuitive (>80%) (RQ3). In conclusion, the conducted research showed that alternative educational methods (i.e., serious games/gamification) could enhance the learning procedure, either in the current case of propelling safer traffic behavior or in broader/advanced media education concepts (RQ4).

The conducted research constitutes a pilot study with preliminary yet solid results. The examination of relevant topics is in no way exhaustive. Apart from the visual programming conveniences, the adoption of the Unreal Engine allowed the use of artificial intelligence (AI) technology for the movement of the surrounding road items (pedestrians/vehicles). This specific technology attaches natural behavior to all automatically moving objects of the virtual world, making the game seem more real. The utilization of the AI adds to the perception of realistic traffic. The clear and positive relations between gaming and traffic behavior prove that games can be used further to enhance traffic skills and develop an improved traffic safety culture, thus being of great value for society. Game products have also started to be part of the media industry, offering a rich storytelling experience while informing the audience. The outcomes of this paper can be a trigger for further research into an ever-evolving area of study. Finally, the whole approach and its pilot results can create interesting challenges for serious game developers.