In the following subsection, we present and analyze our results.
3.1. The Gamification Model in Production
For implementing the model, we used cloud architecture that provides a set of flexible interaction consoles, data monitoring and storage, and data privacy management. The gamified web system was developed in Python using the Django framework. The main objective was to make the weekly goals more transparent to the entire site team and to display the teams’ performance and weekly service progress on a worksite monitor. The context of a soccer championship was used as an engagement strategy (narrative), because approximately 60% of professionals enjoy and participate in this sport.
Site supervisors are usually responsible for monitoring the services as well as individual and team performance. They can enter the information manually during supervision using the existing interface or integrate their management systems with the gamified system so that the data are entered automatically. In addition, the worker can receive feedback on their performance based on the data collected during service monitoring via a private link sent via WhatsApp.
For the system to function properly, rules, challenges, and missions should be developed in collaboration with the site engineers during the implementation process, so that the gamified system can display the performance of teams and workers. To motivate employees, the gamification strategy proposed here employs points, challenges, missions, avatars, badges, and positive messages in the form of a soccer championship. This work is based on the Empowerment model, which promotes autonomy, collaboration, and cooperation in groups through narratives, missions, challenges, and discoveries [
34].
‘Motivational factors for employees, such as identifying top performing employees and displaying their performance on display boards and making them visible among other employees that encourage others to perform well’, according to Subhav, were recommended [
35] (p. 1161). However, the display in this case will be for teams, and workers will only have access to their own performance for ethical reasons; additionally, tracking the worker on the construction site using sensors or cameras is not possible, so individual feedback will not be visible in real time.
The Brazilian Soccer Championship inspired the development of gamification. In comparison, the clubs in the Brazilian Championship correspond to labor teams; the season, which runs from April to December each year, has timeframes that can be configured at the discretion of the site management. The rounds in both correspond to one week, which corresponds to the project’s short-term programming.
Because there are team changes and new services, the duration of the championship is configurable depending on the stage of the project. There is no relegation in this championship, and even if the team does not score a goal, it can still score points.
Because it is important not to distract the workers who perform manual labor on the construction site, the gamification results are modified based on the data entered by the supervisor of the services performed by the teams throughout the week. Similarly to the Brazilian Championship, the games are played on Sundays or at night, once a week, when the public can watch, and the championship outcome changes with each game.
The services completed, in progress, or just begun will be visualized on the construction site monitor or on the worker’s smartphone, using the colors green, yellow, and red to represent good, reasonable, and bad, which is the symbology already known to all.
Figure 2 and
Figure 3 depict illustrative system screens that show the weekly goals and performance of each team.
The following progress levels were assigned in the example shown (
Figure 2): green—indicating that the team has completed more than 90% of the service; yellow—indicating that the team has completed more than 50% of the job; and red—indicating that the team has completed less than 50% of the job.
Figure 3 depicts the teams’ performance screens, which use icons of the same colors (green, yellow, and red) to indicate not only progress, but also the following scores: 20 points for a ball in the net, 12 points for a ball at the post, and 4 points for a ball out.
Weekly, the total score is displayed in points. The number of points earned determines who wins the trophy for the week.
There is also the goal difference, which accumulates how many goals (ball in the net) the team has scored each week throughout the championship. The goal difference determines who wins the championship. Each team is represented by an avatar, in this case, the clubs of the Brazilian Championship, but nothing prevents them from using other symbols to identify themselves.
Goals with the names Safety, Quality, Deadline, and Production were created here, which are related to the service provided by the team and can assume the statuses represented by the icons.
When planning the gamification process, the goals, challenges, and missions will be developed in collaboration with the engineers. The challenges are timed tasks that must be completed throughout the day by a specific deadline. An example of an initial challenge that can be implemented is to select a song to play every time the team wins the trophy of the week, that is, to achieve the most goals. These celebrations can take place in the cafeteria or during the site’s weekly meeting. Another example is a team selecting an image to represent itself (avatar). Always keep in mind that the challenge is something that can be completed quickly and with a limited amount of time.
The missions, on the other hand, can be completed at any time during the week. A mission could be something like this: some companies provide school to their employees outside of office hours, or computer courses to those who already have a basic level. As a result, enrolling at least one team member in these courses would be a mission.
In addition, the system includes Detail and Overview functions. Short training videos can be posted on the construction site using the Detail function. Consider the explanation of the shortest path to minimize transportation losses in construction processes [
36], as well as the use of simulations in BIM 4D to present the most appropriate metal form assembly planning method, resulting in cycle time reduction [
37] as examples of these short trainings. The overview feature displays important images for the stage of the work, such as floor plans of the floor being executed and team location plans.
To demonstrate individual feedback, we created a fictitious worker named Philip who received his private link via WhatsApp to access information about his individual performance, as depicted in
Figure 4.
To improve communication, the worker can see individual attitudes (return tools, wear mask, wash hands, etc.) represented by pictures on this screen. His performance was represented by green, yellow, and red emoji.
Figure 4 depicts a profile picture, a picture of the soccer idol, an emblem with the number of points earned during the week, a ball with the number 20, and a positive message (quote from a soccer star).
This model’s information is fictitious and was added for testing and evaluation purposes.
3.2. Evaluation of the Gamification Model in Production
The following requirements must be met before this gamified system can be implemented on the construction site:
The company must have a well-structured weekly planning process in place, as well as daily production information for its services.
Install a 32-inch monitor in a strategic location on the job site to display system information.
Have a computer connected to a monitor from which the web system will be accessed.
Having enough capacity for Wi-Fi that can be accessed from anywhere on site.
In short, the Production Gamification system presents the services and goals to be developed and achieved during the week using a soccer championship metaphor. On the first screen, the system displays the services, quantities, and teams that will develop them. Throughout the week, the ‘Shots on goal’ field changes color, becoming red for services that are less than 50% complete, yellow for those that are more than 50% complete, and green for those that are more than 90% complete. The second screen displays the teams’ performance in four goals, such as safety and production, as well as a challenge and mission that will be presented at the start of the week.
The team with the most points wins the weekly trophy.
The third screen, accessed via a private link, displays the performance of each individual worker.
The second and third screens provide feedback on how the services were developed to the teams and workers.
Figure 5 depicts how visual management and gamification should be used on the construction site to improve information flow and suggests scenarios in which functions should be used.
Each subject of this study evaluated the model in production using data generated by the instruments outlined in the methodology. The model representing not only the gamified system, but also the entire environment created by its use, was considered for evaluation. Engineers, workers, and users evaluated the model’s functionality and usability.
3.2.1. Analysis of the Gamified Model from the Workers’ Answers
The model was presented to the worker, demonstrating the tool’s functionalities as well as how team and individual feedback work. Following the presentation, a form was used to collect responses on the model. This form is broken down into the following sections: worker profile, consent for research participation, UEQ-S tool, NPS, and questions about the worker’s preferences.
Figure 6 and
Figure 7 show the outcomes.
As a result, as illustrated in
Figure 6, 60% of the workers are young (less than 36 years old), 50% are servants, and 50% have an incomplete middle school education.
An amount of 90 percent of them enjoy playing, with 90 percent preferring team games, and 50 percent preferring electronic games. They use social media, and 90 percent of them have WhatsApp.
Table 1 summarizes the workers’ answers to the UEQ-S.
All scales were made up of a series of seven-point items ranging from −3 (extremely bad) to +3 (extremely good). Values between −0.8 and 0.8 indicate a neutral evaluation of the corresponding scale, values greater than 0.8 indicate a positive evaluation, and values less than 0.8 indicate a negative evaluation [
30].
Figure 8 shows that the model has positive characteristics in the perception of workers, such as being interesting to receive feedback through gamification, reaching values above 2.0 (two), indicating that they approved of the model’s quality and believe it will contribute to optimizing their daily work. For workers, pragmatic quality (efficiency, ease of use, and dependability) is more important than hedonic quality (originality, stimulation), both of which received high ratings.
A benchmarking database is included with the UEQ-S tool. The averages of the measured scale are defined in relation to the existing values of a benchmark dataset. This dataset includes data from 14,056 people from 280 studies on various products (business software, web pages, web stores, social networks). When the evaluated product’s results are compared to the benchmark data, conclusions about the relative quality of the evaluated product in comparison to other products can be drawn. These findings are shown in
Table 2.
The NPS was evaluated using only one question: Would you recommend this model for use on the site where you work? (
Figure 9).
Nine of the ten workers said ‘yes’, with only one saying ‘maybe’, indicating that the model is in the Zone of Excellence (76–100) for this audience.
3.2.2. Analysis of the Gamified Model Based on the Engineers’ Answers
Analyzing the profiles of the 15 engineers who responded, 67 percent of the sample is made up of young engineers, between the ages of 26 and 35, and 35.3 percent of engineers are over the age of 56, indicating a diverse range of experience, as shown in
Figure 10.
After the engineers’ evaluation of the model, a summary of the answers is presented in
Table 3.
Table 3 demonstrates that the model has positive characteristics, as measured by values greater than 1.8, in the engineers’ opinion. Engineers rate pragmatic quality (efficiency, ease of use, dependability) lower than hedonic quality (originality, stimulation), indicating how open they are to gamification on the construction site. According to the evaluations for general quality above 2.0, they believe that the solution will motivate workers to achieve their goals, as shown in
Figure 11.
When the evaluated product’s results are compared to the reference data, conclusions about the evaluated product’s relative quality can be drawn.
Table 4 summarizes these findings.
Regarding NPS, all engineers would recommend the gamified model, which demonstrates that the model is in the Zone of Excellence (76–100) for engineers, as shown in
Figure 12.
3.2.3. Analysis of the Gamified Model from the Responses of Off-Site Users
Figure 13 depicts responses from people of varying ages and education levels, with a focus on young people.
The first section of the questionnaire refers to the System Usability Scale tool, which assesses the system’s acceptability and is composed of 10 statements on which the respondent must indicate their level of agreement on a scale of 0 to 4, as explained in the methodology. As a result of using this tool, the averages were calculated, and a value of 61 was obtained, indicating acceptability as an average (50–70).
Because they were not a specific construction production audience, the external user group had some difficulty understanding the system, indicating that care should be taken in selecting the images representing the weekly goals and that training workers in the system are essential for their understanding.
To evaluate the gamified model based on the information displayed on the worker screen, external users were asked to rate it using the full UEQ Tool, with flag values ranging from 1 (most negative) to 7 (most positive) for each of the attributes.
Table 5 contains a summary of the responses.
Figure 14 shows that the external users’ results are all positive and greater than 0.8.
The UEQ tool also allows us to compare the model’s attractiveness and quality. For external users, we observed that the model’s Pragmatic and Hedonic Quality are very close positive values, well above 0.8, which the tool’s authors consider the neutral range limit [
30]. They assigned a higher value to Attractiveness than to Pragmatic and Hedonic Quality (
Figure 15), implying that the model is attractive, ‘good’ in task-related quality aspects, and interesting in non-task-related quality aspects.
As a result, external users rated the model as ‘Good’ in all measured indexes: attractiveness, transparency, efficiency, control, stimulation, innovation, pragmatic quality, and hedonic. The results of the evaluated product were compared to the benchmark data, as in previous analyses.
Table 6 displays these findings.
In terms of the NPS, we also attempted to determine whether this audience would recommend the model, as shown in
Figure 16.
As a result, only one of the external users indicated they ‘might’ recommend the gamified model, with the result that 90 percent would, indicating that the model is also in the Zone of Excellence (76–100).
To maintain the attractiveness of the Gamification system in production, an icon in the shape of a ‘soccer ball’ will be created for the worker’s profile, which will be at a visible point on the screen in the event of a long duration. This icon serves the same purpose as the Facebook ‘like’ button. The worker will have access to five screens and will be able to ‘kick the ball’ from any of them. When the icon is active, it will change color (from gray and white to black and white).
The ‘kick’ will remain active until the date changes, which means the worker can only give five ‘kicks’ per day, enough time for them to consider goals, team performance, and so on. As a result, a single click activates the ‘kick’, whereas two clicks on the same day deactivate it.
On the Weekly Goals screen, next to each team’s avatar, there will be a counter that will increase with each ‘kick’ of your team. As the number of team kicks increases, this counter will change color, becoming more intense. Every week, the counter will be reset to zero.
A counter will be incremented for each ‘kick’ taken on the worker feedback screen, next to the week’s points badge. As the number of ‘kicks’ increases, this counter will also change color, becoming more intense. Every week, the counter will be reset to zero.
The audience, who includes people who are not members of the gamification teams, is also welcomed to participate. The audience will be able to react by logging into the system with the ‘public profile’: next to each team’s avatar on the performance screen, they will be able to react, as on Facebook, by selecting the images: ‘clapping’, ‘jumping’, and ‘fireworks.’ Throughout the season, the reactions will be accumulated next to each avatar.
The existence of these icons should not be mentioned to workers in training; the logic should be discovered by them.
These counters will serve as system indicators, measuring both the workers’ long-term interest and the workers’ participation.