The Development of an Educational Outdoor Adventure Mobile App

Abstract

This article focuses on the development of an educational outdoor adventure mobile app while presenting findings that were derived from various case studies that we conducted using it. The mobile application, called RouteQuizer, is complemented by a web application and a database, forming a system that enables teachers to create educational treasure hunt activities for their students and monitor their performance. The aim of the research was to create a system that would exploit all possible Outdoor Adventure Education (OAE) and treasure hunt benefits while excluding possible smartphone use negative consequences. The development of the system took place in Greece and began in December 2017, by conducting a nation-wide research examining Greek secondary teachers’ Information and Communication Technology (ICT) literacy and perceptions on smartphone use and outdoor activities. By June 2018, 700 questionnaires were collected. In order to test the system, in March 2018, we conducted a pilot case study in Lesvos island Greece and between July 2018 and February 2020, we conducted four additional case studies and a teacher training program, all of which took place in Lesvos island Greece. During the development process of the mobile application, we focused on the participatory aspect of the process, paying special attention to the teacher and student evaluation during the design and prototyping phases. Considering that the system is educational we research whether the mobile application provided effective learning outcomes and whether it benefited students’ social and physical skills. The results that we collected suggest that the mobile application is an effective learning tool while mobile learning and treasure hunt benefits have been repeatedly confirmed during the case studies. Greek teachers and students also proved to be capable smartphone and computer users, and reported being willing to participate in similar activities in the future.

1. Introduction

Outdoor adventure education (OAE) has the ability to contribute to the personal and social development (PSD) of the participants [1]. Increased self-efficacy, self-confidence, self-regulation, and problem-solving skills as well as group-related outcomes like social cohesion, communication, and team functioning [2] are the most common benefits that derive from OAE, according to our literature review, while similar social, physical and educational benefits have been observed in location-based games [3].

Conversely, social isolation, psychosomatic symptoms [4], and significant negative influence in daily lifestyle behaviors as well as physical and mental health problems [5,6] are attributed to extensive smartphone use.

Nevertheless, smartphones are closely linked to both mobile learning and location-based games. Most of the mobile devices that were used during m-learning interventions, such as PDAs (personal digital assistants), mp3 players, flash drives and e-book readers, are now replaced by smartphones that can offer more than just being attractive to students. Activities that could benefit the students in many different ways, such as treasure hunt and geocaching that are greatly benefited by the GPS and touch screen capabilities of smartphones, can be implemented by teachers, while also training the students’ social, physical, and learning skills. Maps, GPS devices, and photographs were the main means of conducting such activities, and teachers used to experiment with several different location-based games, a lot of which are reviewed in [7].

In this paper we present an effort to develop an educational outdoor adventure mobile application and study its effects on students’ perceptions, performance, and skills. The research was conducted in Lesvos island, Greece, from December 2017 until March 2020.

In the rest of the introduction, we present an overview of location-based games and their benefits, smartphone use challenges as well as the aim of the research.

1.1. Location Based Games

Modern mobile devices and especially smartphones have provided a big boost in location-based games. The most recent and well-known example is “Pokemon GO”, a game that holds five Guinness World Records because of its phenomenal appeal to players worldwide [8]. Indicative of how popular a mobile game can be is that 130 million people downloaded the application within one month of its launch, on 6 July 2016. “Pokemon GO” and other similar games require physical activity in order to be played, they take advantage of the cellular network connectivity and GPS features of the smartphones, and although their purpose is recreational rather than educational, they do prove that location-based games are very attractive to young people. An educational mobile game, based on the same basic principle (location awareness), could be a great way to implement mobile learning activities.

The popularity of treasure hunt led to many variations of the game some of which also require electronic mobile devices. Photo treasure hunt requires a camera, while participants must collect pictures of targets instead of finding the hidden treasure. Letterboxing, an activity that dates to 1854, combines puzzle and orienting in a search of a hidden small box that contains a rubber stamp and a notebook. Clues as to where the box is hidden are shared via the internet or mouth to mouth and letterboxers leave their personal signature on the boxes’ logbook.

Another very popular modern variation is geocaching. On 1 May 2000, thousands of GPS receivers around the world were nearly ten times more accurate due to the abolition of “Selective Availability” [9], an intentional degradation of public GPS signals, which came as a very pleasant surprise, to everyone who worked with GPS technology. In order to celebrate that, on 3 May 2000, Dave Ulmer, a computer consultant and GPS enthusiast, placed a black bucket containing a logbook, a pencil, and various gift items in the woods of Beavercreek in Oregon. He called the game “GPS Stash Hunt”, and shared its coordinates to the USENET newsgroup sci.geo.satellite-nav, encouraging people to discover it, and “take some stuff, leave some stuff”. It only took a day for the first stash to be found, and within a month new stashes had been hidden around the world. As the game continued to grow, new websites emerged, containing lists of stashes worldwide, and soon, after replacing the word “stash” with “cache”, due to negative connotations of the first, the game became known as geocaching. Until a few years ago, geocachers used GPS devices, maps, photographs, and compasses in order to locate the hidden caches, while nowadays the most popular geocaching device is the smartphone [10].

The benefits of treasure hunt games such as geocaching are multifaceted, and apart from the obvious associated with the natural state of the participant ones, they mostly favor young people that are still developing their social skills. Taylor [3] distinguished geocaching’s benefits in three main categories; physical, social, and educational.

The physical benefits we can perceive are:

• The ability to adjust the level of fitness required, by carefully selecting the location of the treasure/geocache.
• Development of the sense of orientation.
• The process of exploration gives a high level of interest.
• The challenge and the subsequent feeling of success when a treasure is found, psychologically benefits the participant
  The social skills that can be acquired.
• The pleasant feeling of achieving a goal. In particular, students that are not very successful in sports can build in this way their self-esteem.
• When participants are separated in groups, the sense of togetherness, as a target of a group with a common purpose.
• Socialization with other participants.
• Develops the cohesion and the collaboration of a group, along with the encouragement of communication.

The educational benefits are multiple. The selection of the participants’ targets can help them acquire information about the sounding area or the specific point. That information can be related either to the geography of the area or the history behind a plague or a statue, especially considering that the curiosity about an area that someone visits is a natural human tendency.

1.2. Smartphone Use Educational Challenges

The most popular list of using smartphones disadvantages in education mainly consists of device related characteristics such as small screen, limited storage capacity, limited battery life, robustness, no printing support, and the fact that mobile devices become out of date quickly [11,12].

Technology has solved many of the limitations described in older articles by constantly improving the devices. Most researchers tend to compare mobile devices with laptops and desktop computers, ignoring the fact that mobile devices are proposed as a means to conduct mobile learning activities and not to replace desktop computers and laptops. Mobile applications are designed to operate with mobile devices, e.g., smartphones, exploiting their hardware and software characteristics. These applications will not work as they should on a laptop, while computer applications will not work on smartphones. Based on the above, mobile devices and laptops/personal computers are not comparable, even though, a closer look to the technical characteristics of the last five years’ smartphones and tablets would reveal that those devices have dealt with all the aforementioned limitations, while at the time, in some respects they end up being superior to laptops (e.g., battery life).

In contrast, excluding device limitation factors, mobile devices can negatively affect students. It is of prime importance to prevent irrational use of the mobile devices by the students as they can lead to unethical behavior [13] student distraction [14], and physical health compromise [15]. Tutors must implement mobile learning activities in a way that phenomena such as cyber bulling, distraction, and digital divide [16] issues will be excluded.

Teachers’ digital skills are required. The most significant barrier to successful integration of Information and Communication Technologies (ICT) in education is teachers’ lack of confidence, experience, and pedagogical understanding in mobilizing the potential of digital technologies [17]. Advanced knowledge on how to use mobile devices and application is required.

The lack of guidelines for appropriate use [18] can be a challenge for teachers that wish to implement mobile learning activities while affecting the method’s effectiveness. While many countries such as Canada, Australia, Denmark, Sweden, Spain, Romania, and Estonia, encourage students to bring their own devices (BYOD system), smartphones are banned from schools in other countries such as Greece and many states of the U.S. [19,20].

Digital exclusion is also a big risk when introducing ICT in the educational process as not everywhere and not everyone can use smartphones and internet connectivity. Also, less advanced users may feel intimidated by the technology and stay out or feel isolated during an activity [21].

Not all mobile applications are ideal for learning. Poor design issues are existent in many educational mobile applications so teachers must be very careful on choosing an application.

1.3. Motivation and Research Aim

Smartphone use in outdoor recreation settings is controversial [22]. Smartphone use has been accused of increasing participant carelessness due to the false sense of security their smartphone provides [23]; discouraging the development of outdoor navigation, as participants rely on the device [24]; and distraction, as social media, games, music and movies are one tap away [25] and disturbance of fellow participants that want to completely disconnect form civilization [22]. Contrarywise, many instructors use mobile technologies in outdoor settings for multiple reasons [26,27]. Map, compass, location information, text, audio, video [25], but also more advanced features such as heart rate monitors that modern smartphones provide have been proven to be very useful during outdoor adventures.

The lack of a common attitude towards the subject is also reflected on bibliography. Most treasure hunt related applications in Google Play, are entertainment games not developed for educational purposes. After browsing the related bibliography, we identified only two mobile applications designed to enhance outdoor adventure education using smartphones. Kohen-Vacs et. al. [28] presented Treasure-HIT, a mobile treasure hunt game for outdoor learning. Treasure-HIT provides clues that students have to follow in order to reach a destination point. Upon arrival, they are asked to perform specific tasks on site. The application also provides activity results that refer to each activity. More specifically, Treasure-HIT informs the teacher that a group made a certain amount or mistakes while conducting the activity’s tasks although it does not provide data about each question. Also, except clues, it does not provide information about the points visited, which in our opinion is essential to enhance the educational aspect of the activity. Mobilogue [29] is also a similar application but due to getting no response from its servers it has not been possible to further evaluate it.

Contradictory attitudes and views on smartphone use in OAE result in unclarity on whether smartphones can benefit outdoor adventure education. Also, the small number of educational outdoor activity mobile applications suggests that this sector is not particularly developed making the development of such software imperative for a detailed analysis of the subject.

2. Methodology

Considering the multiple potential benefits that location-based games can offer, we decided to develop a mobile learning system that would exploit all possible OAE and treasure hunt advantages, while excluding possible smartphone use negative consequences. The RouteQuizer system consists of a database, an Android application, and a web application and it enables teachers to create educational treasure hunt activities and monitor their students’ progress.

The development of the system was based on the MADLC [30] and Agile [31] software development models, while we focused more on the participatory aspect of the process. During the first stage of the development, the assimilation of modern technology by teachers was investigated, as ICT literate teachers are an essential factor [17]. Specifically, in order to identify the ability and willingness of teachers to adopt such a learning tool in the educational process, we conducted a nationwide survey on the digital literacy and the perceptions on mobile learning and treasure hunting of teachers in secondary education, presented in Section 2.1.

The subsequent design and implementation stages led to the first RouteQuizer prototype, presented in Section 2.2. The prototype was further improved through an iterative testing and re-implementation process. Each iteration included an experimental use of the system in real conditions (case study). The feedback of these case studies determined the required modifications of the system. In more detail, the prototype was initially tested by conducting a pilot case study with students at a STEM school program (presented in Section 2.3) and by organizing a teacher training program (presented in Section 2.4). After improvements integration, the system was tested again in four different case studies with different objectives each, as presented in Section 2.5.

The observations deriving from conducting RouteQuizer activities would help us answer the following research questions:

Question 1: To what extent do students like treasure hunting using a smartphone for educational purposes?

Question 2: How distracting is the mobile application to participants during outdoor activities?

Question 3: How beneficial to student’s social and physical skills is treasure hunting and m-learning?

Question 4: To what extent is knowledge acquisition possible using a mobile app?

2.1. Researching Teachers’ Skills and Perceptions

Instead of relying on our hypotheses about the teachers’ technology related skills and perceptions on mobile learning and geocaching, we conducted a nation-wide survey that was conducted from December 2017 until June 2018.

2.1.1. Participants

The collection of the questionnaires was based on random sampling and all responses were anonymous. Based on data we received from the Greek Ministry of Education, the total population of secondary education teachers in Greece (2017) was 68,139, across 75 directorates of secondary education throughout the country. In order to get a less than 2% error margin we calculated that the required sample of teachers was 544. Also, in order to ensure the representativeness of the research outcomes, we calculated the exact number of responses required by every region and continued accepting responses until all regions were equally represented.

The total sample of our research, excluding the responses that failed the consistency tests we conducted, as well as the ones that were mostly empty, consisted of 700 secondary school teachers teaching in 283 schools from every region of Greece [20]. Of these, 57.53% were aged between 36 and 50 years old, 37.16% more than 51 and only 5.31% of the participants were aged between 23 and 35. Most of the teachers, 33.8%, taught language and philological sciences, 28.2% mathematical and physical sciences, 18.4% information and computer science courses, 4.0% engineering, and 15.6% other courses. 51.8% of the participants taught in junior high schools, 32.7% in high schools, and 15.5% in vocational high schools.

2.1.2. Research Tools

The instrument used was a questionnaire that consisted of four parts. The first part recorded personal data. The second part was titled “familiarity with personal computers” and contained seven questions. Two multiple choice type questions recorded whether their school provides PC and internet access. The remaining five questions were four-scale Likert type, recording how often they use personal computers and the internet for personal use, whether and how often they use PCs for teaching, and how familiar they are with various web applications and PC use skills. The third part was titled “familiarity with smartphones and tablets” and contained five questions. Using four Likert four-scale questions, it recorded how often they use smartphones and various types of smartphone applications, whether and how often they use smartphones for their teaching needs and how willing they are to implement an educational mobile application during an educational activity. Another multiple-choice type question recorded the reasons why they would not in case they answered so. The fourth and last part of the questionnaire was titled “Geocaching” and contained five questions. Four out of five were multiple choice type questions and one was 1–10 scale. During this part, the teachers were asked whether they know what geocaching is and if not, whether they would like to be informed about a mobile treasure hunt application and under which conditions. They were also asked whether they would organize such an activity and which parameters affect the implementation of such an activity. Finally, in case they knew what geocaching is, in a scale of 1 to 10 they recorded whether they believe that geocaching and similar activities can benefit the students. The outcomes of the two middle parts of the questionnaire provided us with skill-specific data about various personal computer and smartphone tasks. Those findings enabled us to complete the design phase of the system’s development process.

The internal consistency reliability of the questionnaire was measured by calculating the Cronbach’s alpha value of all 28 items that were measured in a 4-point Likert scale (never, rarely, frequently, daily). The calculated value was 0.896 > 0.7.

Prior to deploying the questionnaire, we tested its content validity by conducting a pilot research. The questionnaire was completed by 51 teachers, teaching in secondary education schools in Lesvos island. The questionnaire was very similar to the one used in the nationwide survey but contained more open-ended questions. The teachers’ responses in all open-ended questions, were analyzed and the most frequent responses were implemented in the final version of the questionnaire. In particular, open ended questions such as “which web applications do you use?”, “which smartphone applications do you use?” and “what are the deterrents in using mobile apps in educational activities?”, were transformed in multiple choice type questions in the final version of the questionnaire containing the most frequent answers the teachers provided.

We also used a criterion validity test. There were five set of questions used as validity switches. In case the responses to these questions were inconsistent the questionnaire was discarded. As a result of this test 2 questionnaires were rejected.

2.1.3. Results

The percentages of personal computer, internet and smartphone usage were very high. 84.2% of the participant use a computer daily, 13.2% frequently, and 89.1% of the computer users use the internet daily. Also, 62.75% use smartphones daily and 18.77% frequently. Those high usage percentages suggested that teachers would not face many difficulties in using the system but more specific data about their ICT skills were also collected.

More specifically, the participants among other questions were asked how frequently they use various web applications (containing some geocaching related applications as well) and how familiar they are with different types of web and mobile application skills. The most popular web applications were Google Drive (14.0% daily and 46.4% frequent use), Google Forms (31.2% frequent use), Google Earth (29.7% frequent use), and photodentro, a Greek learning object repository (54.4% frequent use). When asked how familiar they were with different web application skills, teachers reported being very familiar with all of them and the same applies to different smartphone application types, such as map and GPS using applications, as well as educational apps. In particular, regarding the skillset required in using the RouteQuizer system, the outcomes were encouraging: 82.5% reported fully or very familiar with register and login functions using web applications and 65.2% fully or very familiar with web applications using maps. Furthermore, 67.8% reported fully or very accustomed with mobile applications that use maps and 71.3% with educational smartphone apps.

When asked whether they would use a smartphone educational use in the context of an educational activity, 23.6% replied negatively, 27.8% would implement such activities 1 to 3 times per semester, 30.1% 3 to 10 times per semester and 18.4% more than 10 times per semester. At this point, it must be noted that 66.3% of the teachers that replied negatively, did so because “smartphone use is banned” (although this is not very accurate as smartphone use in Greece is allowed when the equipment used is available to students by the school they attend, during the teaching process and the educational process in general, and only under teacher’s supervision).

Although geocaching proved to be not too popular, as only 31.9% of the participants knew what it is and only 8.1% of them had previously participated in such an activity, 68.6% were interested in being furtherly informed about mobile treasure hunt activities, and 16.7% possibly. Lastly, when teachers with previous geocaching experience were asked to rate in a scale of 1 to 10 how much geocaching benefits students, the average score was 6.93/10.

We used one-way analysis of variance (ANOVA) in order to determine whether there were any statistically significant differences between the means of different independent groups. Between the course the participants teach (independent variable, IV) and whether they use smartphones for teaching interventions (dependent variable, DV), there was a statistically significant difference between groups as determined by one-way ANOVA (F(4.672) = 5.415, p = 0.000270). Post hoc comparisons using the Tukey HSD test indicated that the mean score for the “Information and Computer Sciences” condition (M = 2.00, SD = 0.871) was significantly different than the “Languages and Philological Sciences” condition (M = 1.61, SD = 0.797). The “Mathematical and Physical Sciences” (M = 1.70, SD = 0.745), “Engineering Sciences” (M = 1.79, SD = 0.738) and “Other courses” (M = 1.63, SD = 0.764) conditions did not significantly differ from the “Languages and Philological Sciences” condition. Taken together, these results suggest that Information and Computer science teachers are more frequent smartphone users for teaching purposes (Figure 1).

A one-way between subjects ANOVA was also conducted to compare the effect of teachers’ age (IV) on how beneficial they believe geocaching is to students (DV) in 23 < teacher’s age < 35, 36 < teacher’s age < 50, and teacher’s age < 51 conditions. There was a significant effect for the three conditions (F(2.217) = 3.389, p = 0.036). Younger teachers (M = 8.22, SD = 1.787) appear to evaluate geocaching benefits with higher score than age groups 36–50 (M = 7.06, SD = 2.008) and 51+ (M = 6.50, SD = 2.391) (Figure 2).

A multiple linear regression was calculated to predict whether teachers would use a mobile learning application during an educational activity (DV) based on how experienced smartphones users they are (IV1), and how often they use educational mobile apps for their teaching (IV2). A significant regression equation was found (F(2.685) = 82.726, p = 6.5918 × 10⁻³³), with an R² of 0.195. The participants predicted that their willingness to adopt smartphone application in educational activities is equal to 1.134 + 0.507 (mobile educational app use) + 0.126 (smartphone experience), where mobile educational app use is measured as 1 = never, 2 = Rarely, 3 = Often, 4 = Daily, and smartphone experience is also measured as 1 = never, 2 = Rarely, 3 = Often, 4 = Daily.

A Spearman’s rank-order correlation was run to determine the relationship between frequency of smartphone use and willingness to use smartphone applications in educational activities. There was a strong, positive correlation between the two variables, which was statistically significant (r_s = 0.241, p = 1.4146 × 10⁻¹⁰). Statistically significant Spearman’s rank-order correlations were also identified between the “familiarity with mobile educational apps” and “willingness to use smartphone applications in educational activities” variables (r_s = 0.190, p = 5.8614 × 10⁻⁷).

The above findings confirm studies suggesting that the higher the teachers’ competence of using these technologies the higher is their willingness to adopt ICT for teaching and learning [32,33]. Also, the participants’ age-related findings are in line with studies that show that “older teachers tend to regard the use of ICT for different purposes as less useful, and potential problems and obstacles for teaching and learning” [34,35].

Taking into account the high percentages of smartphone and PC familiarity and experience, the questionnaire’s findings suggested that a system such as RouteQuizer would be welcomed by Greek secondary education teachers, as the participants proved willing and capable.

2.2. The RouteQuizer Prototype

The RouteQuizer system operates in three steps; teachers create activities using the system’s web application, students participate in the activities using the mobile application on the field and finally teachers can check their students’ performance using the web application.

During an activity, the students must visit certain locations one after the other. Without knowing the exact location of each destination, the students use hints provided in the application by the teacher, a google map with a compass that is loaded in the main screen, and a distance meter that indicates their current distance to the destination. A Help button is also existent in case the students get lost. By tapping it, a marker is displayed on the map showing them the exact location they must visit and the route they must follow to get there. Upon their arrival at each destination, the application automatically notifies them that they have arrived, and presents additional information about the location and a multiple-choice type question. All answers are recorded in the system’s database and can be accessed by the teacher using the web application.

2.2.1. Requirements

In order to create a system that would satisfy the research objectives, during the design phase the following basic requirements where set:

• The mobile application should be Android OS compatible.

Between Apple’s iOS and Android, we chose the second based on device cost. Android powered devices’ cost ranges from 50 € to 1000 € while iPhones cost closer to the top end Android ones, making the purchase of several devices much more costly. Another important factor was Android devices are much more widespread than iOS device, thus reducing possible digital exclusion phenomena.

• The system should be simple to use.

Both the smartphone and web applications’ interface and design had to be as simple as possible. Complex menus and operational difficulties would discourage teachers and students from using the systems apps.

• The system should provide effective learning outcomes

Although the system is not designed to replace traditional formal teaching methods, as a supplementary tool it should also help students acquire knowledge effectively other than being a fun activity.

2.2.2. The Web Application

The system’s web application (Figure 3) was developed using HTML5 and CSS language and is mobile friendly (easily browsed using mobile devices). During its development we paid a lot of attention in creating a simple user interface while also providing detailed instructions on how the website works.

2.2.3. Creating and Managing Activities

Teachers are the only users of our system that need to create a free account to be able to browse, edit or delete activities.

The creation of an activity is done by filling a simple form (Figure 4). In essence, an activity is a table that consists of the following fields.

• Code. The tutor defines the “Activity Name”.
• Point Order. The tutor enters the order in which each point will appear to students. In each new activity that a tutor creates, the first point must have the point order value of 1, the second a value of 2, and so on. In this way the teacher can largely control the route that the students will follow, during the use of the activity.
• Latitude. In this field, he enters the first part of the coordinates, i.e., latitude.
• Longitude. In this field, he enters the second part of the coordinates, i.e., longitude.
• Information. Information about the point to be visited.
• Question. The question to be displayed to the students as soon as they arrive at the point.
• Answer 1. The first possible answer.
• Answer 2. The second possible answer.
• Answer 3. The third possible answer.
• Answer 4. The fourth possible answer.
• Right Answer. The tutor provides the right answer.
• Distance. This field is filled by a number, corresponding to the maximum distance in meters, in which the students must approach, for the question to be displayed.

After all the above information about a certain point is inserted, the tutor repeats the process for each point. In this stage, the new activity is ready to be used by the students. In order to register a different activity, the teacher defines a different activity name and restarts the “Point order” value.

2.2.4. Browsing the Results of an Activity

As mentioned before, the tutor can overview their students’ performance. After logging in the application’s website and selecting the “view results of an activity” button, they are asked to submit an activity’s name. As a result, a table containing the following columns appears.

• User
  Contains the student’s username.
• Activity name
• Point
  1 in case it was the first visited point, 2 if it was the second etc.
• Result
  Contains either “Correct” or “Wrong”.
• Help
  Contains either “Used Help” or “Did not use help”, that indicates whether the user used the help button or not.
• Date and Time
  The date and time when the students answered the question.

2.2.5. The Mobile Application

RouteQuizer’s application for Android devices is compatible with all Android OS versions, and during its beta testing it has been extensively tested using many smartphone devices of different prices (ranging from 50€ to 700€), screen sizes (4 inches to 6 inches), and computational power, in order to ensure that no problems will occur regardless of the device used.

Before using the application, the user must make sure that mobile data usage and the GPS function of the device are enabled. After the application is launched, an introductory screen appears for 3seconds, and afterwards the student/group of students is/are asked to write their name/team’s name. Right after typing their name, the application asks an activity name, which should be provided beforehand by the teacher. By typing the activity’s name, the application loads all data concerning the first destination point.

The way the application handles an activity’s name favors teachers in the following ways:

• Only authorized users that know the activity’s name can have access.
• It ensures the uniqueness of each activity.
• Sharing the same activity with other teachers, is easy, simply by sharing its name.
• The application, as is, is usable by different teachers and for different classes.
• The teacher can easily disable an activity if he wants to, by changing its title.

2.2.6. Main Screen

The main screen is a map that can be zoomed in and out, rotated and centered. The map displays a blue dot that indicates the current position of the user. To make RouteQuizer more appealing and fun to the students, the position of each destination is not acknowledged. They can get hints as to where it is located, through the information provided by the tutor. The main tool the students have is a distance meter although to prevent the users from getting lost, there is also a help button.

More specifically, the main screen of the application consists of four buttons and a distance meter.

• Center map button.
  Upon selection, the screen is centered in the user’s position.
• Map type selection button.
  The user can choose between four different map types, a road map, a satellite map, terrain map and a hybrid map.
• Help button.
  A red marker indicating the position of the destination, as well as the shortest route to get there, appear on the screen, preventing the user from getting lost. Also, the tutor is informed whether the help button has been used or not.
• Information button.
  A window containing all the information provided by the tutor appears, helping the user locate the destination point but also to get informed about it. In case the text is long, the window contains a scroll bar.
• Distance meter.
  The distance meter represents the distance in meters between the user and the destination. That way, the user knows whether he is heading to the right direction or not.

Using the above user interface (Figure 5) and taking advantage of the distance meter and the available information, the students must reach one point at the time, within a certain distance. That, and also the fact that (unlike all other geocaching applications) only one point is loaded at a time, are a powerful tool in the hands of tutors, because it makes it easy for them to plan a trip exactly as they want to, and to be sure that the students did really visit the points.

Upon arrival at their destination, the smartphone automatically vibrates to notify the students that they arrived, and a window containing a question and four possible answers appears. Along with the multiple-choice question, a new button is also added to the screen (while the help, and distance meter disappear). The students select their answer and press the “Check button” to submit their answer. At this point, a message informs them if they were right or wrong, in case they were wrong it displays the right answer, and also at the same time, their result, the time and date they answered, as well as whether they used help or not, are submitted in the according table of the online database, enabling the tutor to overview students’ performance.

Afterwards, the second point data loads. The students keep visiting their destinations and answering questions, until they answer the final point’s question. The application automatically perceives that there are no more points to load, and a game over screen appears, congratulating the students, for discovering all points.

A very significant characteristic of the application is the lack of menus. Each screen contains only the most important elements, without confusing the user. That simplicity translates into a fluent and easy user experience, enabling people that have no previous smartphone experience to use the application.

2.3. Students’ Evaluation—Pilot Case Study

The first case study was conducted as part of a STEM summer school program that took place in a high school campus in Mytilene, Lesvos island, Greece in March 2018. The 34 participating students were aged between 12 to 15. Following an in-class presentation of how the GPS system works the students were separated into 4 groups of 8 to 9 students each. Each group accompanied by a researcher participated in a different activity in terms of points’ location but the four GPS related questions they had to answer were the same.

Being the first case of the system’s mobile application, we focused solely on recording our observations on:

• how the application performed technically,
• the students’ spontaneous comments, and
• the student’s behavior and concentration during the activity.

The activity proved to be a pleasant break from the summer school’s routine. Although the students were not classmates the communication levels between them were high. The existence of a hidden box containing prizes at the end of each activity seemed to be of great importance to the students and the rally to finish the activity before the other groups were common in all four groups. It was clear that the hidden box added much needed adventurous aspect on the activity.

In terms of functionality the application worked better in three out of four devices that were used although the devices used were the same. The GPS signal inaccuracy we faced in one team made reaching point no 3 impossible, something that really annoyed the team members. With further investigation we found out that the specific device was faulty, and it was never used in a case study again.

That first case study helped us reach to multiple conclusions in order to improve not only the application itself but also the organizing and planning part of a RouteQuizer activity. The distance meter of the app helped in retaining the students’ interest in high levels although the school campus area was not the most exciting place to be. In order to increase interest levels in future activities we replaced the “Information” button with the “Hints” button. Instead of providing all information about a certain point before reaching it, we decided that all information should be presented afterwards. By providing hints and riddles as to where the students should go, we can increase the difficulty levels of an activity while also providing a more adventurous experience.

The number of students consisting the groups in future case studies had to be reduced to a maximum of 5. It is impossible for 9 students to get their hands on the device within an activity of 4 points. Also, it is much easier for a teacher to accompany a group of 4 to 5 students especially when the activity takes place in urban places.

Moreover, changes to the application were applied during the development phase in order to improve the user experience. Specifically, in low-end smartphones (the type of smartphones that will most likely be used in real world implementations) we noticed increased loading time in calculating the user’s distance to the destination that had not been identified during the beta testing of the app, as well as lagging in acquiring the user’s position. By completely revising the corresponding code parts, we managed to greatly reduce loading times.

2.4. Teachers’ Evaluation—Teacher Training Program

Teacher feedback was also crucial for the further development of the application. For that reason, in collaboration with the directorate of secondary education of Lesvos island we organized a teacher training program that would help us evaluate the application according to the teachers’ perspective.

The training program lasted 2 days and was attended by 11 secondary education teachers. During the first day the teachers were instructed on how to use the web application. They were also separated in 3 groups of 3 to 4 each and were asked to create an activity that would take place in Mytilene city’s center. The meeting point of the second day was the starting point of each of the activities. Each team conducted one of the activities that their fellow teachers had created during the first day of the training program. After completing the activity all teachers were gathered indoors were the evaluation of the system as well as the training program took place. At the end of the day, they were also asked to fill a questionnaire evaluating both the web and the mobile application as well as writing down their suggestions.

All participating teachers were experienced PC and smartphone users. Most of the participants (55.5%) reported that they have participated in a treasure hunting activity in the past although not using a smartphone. Both web and mobile applications were highly rated (Figure 6 and Figure 7) and all teachers were willing to adopt the system in their teaching. Moreover, all of them believed that their students would respond positively in the implementation of such a teaching method. Responding to whether they believe that the application can benefit the educational process on a scale of 1 to 5, the application averaged 4.33/5.

Based on the teachers’ remarks the following suggestions were adopted:

• Web application:
  • The location of the points should be selected on a map.
  • Open ended questions should also be supported.
  • Teachers should be able to copy an activity they have created and just change the order of the points.
• Mobile application:
  • The information window should not only contain text, but also photos, videos, and sound recordings.
  • Help button should not be easily used. A confirmation window should also appear.

2.5. Case Studies

After improving the system based on the feedback we received during the case study and the teacher training program presented above, we conducted four more activities (

Table 1. Case studies.

Case Study	Participants	Number of Participants	Participants’ Age	Location	Learning Subject
1	1st junior high school students	27	12–15	Surroundings of the University of the Aegean campus	Analysis and Perception of the landscape
2	University of the Aegean Geography students	25	18–21	Surroundings of the University of the Aegean campus	Analysis and Perception of the landscape
3	Mantamados junior high school students	18	12–15	Mantamados village	The village’s history
4	Dyslexic junior high school students	12	12–15	Mytilene city centre	Practising orientation skills visiting the city’s monuments

) with a total of 70 students participating. Each one differed in terms of location, participating students, subject, and objective, although the way the activities were planned and executed was similar. The students were separated in groups of 3 to 5 and were accompanied by a teacher during the activity. After each activity was finished, all students filled a questionnaire, containing questions about the learning subject. Furthermore, data about the mobile application and the experience of treasure hunting using a smartphone were collected. All smartphone devices that were used during the activities were identical, and they, as well as internet connectivity, were provided by the researchers in order to exclude digital exclusion phenomena. Also, during each activity, researchers made sure that every student on each group would hold the device, and actively participate in the process.

2.5.1. Case Study no1—First Junior High School

The case study was conducted in the surroundings of the University of the Aegean campus. The 27 participating students aged 12–15 years old, were separated in groups of 4 to 5, and were asked to visit 7 destination points during the activity. The total distance they covered was approximately 2 km. The activity’s learning subject was “Analysis and Perception of the Landscape”.

Our objective was to study the children’s reactions and behavior during the activity and to evaluate whether the distance of 2 km was too small or too big.

2.5.2. Case Study no2—University of the Aegean Geography Students

This case study was conducted within 2 weeks of the first Junior High School one. The learning subject, destination points, questions and location were exactly the same, in order to record the differences in students’ overall reactions and behavior.

Furthermore, the main objective of this case study was to identify whether the activity could benefit the students educationally [36].

2.5.3. Case Study no3—Mantamados Junior High School Students

Mantamados is a small village located in the northern part of Lesvos island, Greece, and its population is 3210 people. After studying an old book about the history of the village, full of old localities and stories, we created an activity that would help local students learn more about their village’s history.

This case study’s objective, other than monitoring the students’ reactions, was to evaluate whether the students would cooperate with the other team members in order to reach each destination. The nature of the activity was ideal to study the social interaction of the groups, as not all but surely not every student would know which coffee shop was the oldest building of the village, or where the 30s police station was located.

2.5.4. Case Study no4—Dyslexic Junior High School Students

Dyslexia, one of the most distinctive learning difficulties, has been proved to cause topographic disorders as well as orientation problems [37]. In order to examine whether the RouteQuizer application could benefits dyslexic students to orientate their selves, we conducted an activity with 12 dyslexic students participating [38]. The research was conducted in Mytilene city center on Lesvos island, Greece, in January 2020. Before the two teams started, they were instructed about the use of the application. There was a 15-min delay between the two teams as the routes they had to follow where the same (in order to collect comparable data). Each team was accompanied by two of the researchers.

The in-app questions that the students had to answer required good understanding of the four points of the horizon, the capability to estimate distances covered and route to be followed.

3. Results

The four case studies we conducted (Table 1), provided us with valuable feedback not only about the perceptions of the students towards field trips using smartphones, and the RouteQuizer app’s performance but also with specific findings from each. By collecting questionnaires after each activity and closely observing every team that participated in those activities, we managed to answer all four research questions.

Regarding research question 1, the answer derived from the student’s responses in the questionnaire as well as the observation process.

Two of the common questions that were asked in all four case studies were:

• How much did you enjoy the treasure hunt experience?
• Would you like to repeat such an activity in the future?

Most of the students reported that they enjoyed the treasure hunt experience (45.6% of the students reported they enjoyed the treasure hunt experience a lot, 49.1% fairly, and 5.3% a little) and 57.9% would like to participate in a similar activity in the future a lot and 33.3% fairly.

As far as the mobile application is concerned, the students were asked to rate it in a scale of 1 to 10 in terms of functionality, usability, loading speed and content (Figure 8). Furthermore, most students reported that the application helped them in answering the learning subject related questions (32.75% a lot, 56.9% fairly).

Also, all 4 case studies followed a similar pattern in the way they were conducted. Groups consisted with less than 6 students, and each group was accompanied by either one or two researchers. Every researcher’s observation was recorded and compared. The main characteristics of all four case studies were the following:

• Winning was important [39].
• Competitiveness levels were high [40].
• Collaboration within the members of each team always occurred.
• Students had a great time [41].

Case study no2 contributed to answering both research questions 2 and 4. During the case study, the students were divided into two groups: half of them participated in the field exercises without using the application but accompanied by the course professor who was also lecturing during the activity. The second group used the RouteQuizer mobile app exclusively, without supervising or instructing by the professor, relying solely to the information provided from the app. After grading and rating the students’ answers, the first group of students that did not use the system’s application managed a 56.25% score, while the second group scored 76.92% [36]. Based on our observation we justified that 20.67% difference in three main reasons:

• Not all students of the first group were paying attention,
• The application encouraged collaborative learning
• No questions were left unanswered while using the mobile application.

Regarding the students’ social and physical skills (research question 3), case studies no 3 and no4 provided crucial findings. The “Mantamados” case study (no3) proved to be the most physically demanding one, as the village is densely structured something that lead students to many deadlocks. Most of the students were not familiar with any of the toponyms they were asked to visit which led to extensive collaboration within the members of the groups. Furthermore, when extra help was needed, 3 of the 4 groups asked elder people passing by for directions, which although most of the times were not very helpful, lead to the students hearing even more stories from the past. All members of the 4 groups were continuously active, and their dedication to complete the tasks impressive.

During case study no4 all four researchers reported that the students had significant difficulties in distinguishing the four points of the horizon [42,43], even though a compass was built in the application’s map. After failing to answer the first couple of questions, and better realizing their location and movements on the map, students performed better at the last set of questions [36]. Another interesting finding occurred when during the activity the teams met. Instead of decreasing the interest levels (as one team would simply follow the other one in order to reach the next station) the competition went up sharply, and even the shyer children seemed to be more interested in the activity as well as the evaluation of their performance.

4. Discussion

During the development and evaluation process of the system we managed to provide answers to our research objectives by conducting four different activities while also collecting valuable data from Greek teachers.

It was clear during the research that the use of smartphones is a pleasant experience for students. The questionnaires suggest that 45.6% of the students reported they enjoyed the treasure hunt experience a lot, 49.1% fairly, and just 5.3% a little. Furthermore 57.9% would like to participate in a similar activity in the future a lot and 33.3% fairly. The same finding can also be found in [39,40,41].

During case study no 2, we compared the performance of students using the smartphone app with those participating in a traditional outdoor activity. The better outcomes of the first group can easily be explained when observing both groups during the activity but can also be explained when considering the size of the participating groups. The 12 students that were not using the application had to follow the tutor and pay attention to what he said. Although 12 students are not a big group, it is much more difficult to keep all students committed to the lecture while being on the field. Alternatively, the much smaller groups of 4 students were much easier to keep interested, as they had to cooperate in order to proceed and answer the questions correctly. The smartphone in this occasion not only did not distract the students, but it was the factor that kept them focused throughout the activity [14].

Regarding social and physical skill development using the system’s application, we confirmed all social and physical skill presented by Taylor [3] and Palmárová & Lovászová [44]. In all case studies, not only those with student participants, but also the three activities conducted by secondary education teachers, the collaboration levels among team members were very high. Prior to answering each question, a small meeting between the members of each group was held, and students were sharing their views and opinions, before picking their answer [36]. An example of the increased levels of collaboration and communication can be viewed in case study no 3, during which the members of three of the teams not only collaborated with each other, but also with passing by people in order to get directions.

As far as physical skills are concerned, many of the activities although not too demanding required at least 2 hours of walking time. Also, the orientation related skills of all students that participated were clearly developed as they were getting closer to the last destination point [45]. The above finding became more apparent during the case study no 4, during which dyslexic student were greatly benefited by the map, compass, and information that the application supplied, confirming that also dyslexic students can greatly be benefited by ICT in terms of geospatial abilities [46].

During case study no 2 it was demonstrated that the mobile application can improve student outcomes and help effectively in achieving the educational goals [36]. Although the sample of the case study is small, which is a limitation that does not enable us to generalize the finding, it is consistent with the theoretical framework [47,48].

A product that is intended to be used by young people, and especially when it concerns their education must come with instructions that will ensure the best possible use of the product. In our case, taking into account that not all teachers are familiar with how to organize a mobile treasure hunt activity, we attempt to accompany our system with some recommendations, in order to overcome as many mobile learning challenges as possible while also taking advantage of all mobile learning and treasure hunt game benefits without of course depriving from teachers the right to use the system as they eventually decide.

• Students should work in groups.
  That way students develop their social skills (sense of togetherness, socialization, development of the cohesion and the collaboration of the group, encouragement of communication). At the same time, the number of devices needed is significantly decrease, resulting to a huge cost saving in case the devices are provided by the school.
• Schools should provide the devices to be used.
  The cost of a “low-end” android smartphone ranges from 50USD to 100USD a cost that would not significantly affect the yearly budget of an education institute/school. The students will not be required to either own an android smartphone or bring their device to school, something that possibly could also be banned in many schools. In addition, the students will be using the device only for as long as the activity lasts and will all be using the same device type encouraging equality and avoiding digital exclusion.
• Smartphone devices should be “locked”.
  In the occasion the school provides the mobile devices we suggest all other applications and features of the device to be locked, meaning that the only application accessible to students will be RouteQuizer. This ensures that students will not get distracted by other applications installed in the device such as camera, web browser, etc. Technically, this can be easily achieved by using one of many free applications available on Google Play that enable users to hide all applications installed in a device.
• Teachers must ensure the activity is conducted safely.
  Although RouteQuizer provides all possible functions to prevent students from getting lost, it is the teacher’s sole responsibility to ensure the safe conduct of the activity. We recommend that activities should take place in a controlled environment, especially in the occasion the students are children.

5. Conclusions

RouteQuizer’s developing process differs from other systems because of its testers. Greek teachers and students become part of the developing team by frequently testing and evaluating the system, to create an as beneficial as possible mobile learning system ready to be inducted in the teaching process. In its present form, RouteQuizer is fully operational and has reached every technical and educational milestone we had set at the beginning of the project.

From a technical aspect the android application is supported by all Android operating system versions (Android 2.1 to Android 10.0), and is executable in all Android smartphone models, regardless of their manufacturer, screen size, resolution, and computational power. To achieve that many tests and optimizations where needed such as creating all graphics in different resolutions. The application also takes advantage of every new feature that a smartphone can offer to education. Portability, connectivity, and pleasant user experience are features that not only characterize smartphones but also the RouteQuizer application. In addition to the apparent portability, connectivity is one of the main characteristics of the application, as it communicates in real time with the database without requiring any special action from the user. The pleasant feeling that smartphone users enjoy is still existent, as the lack of menus, settings, advertisements and any other annoying elements are absent while at the same time every interaction between the application and the users is simple.

In terms of educational benefits, the system complies with all standards set by UNESCO and EU while offering all mobile learning and treasure hunt games’ benefits. It enables students and teachers with no previous smartphone and web application experience, respectively, to use it because of the simple design both the android and the web application provide. The way the application handles each destination point, loading one point at the time, and providing information as well as the distance to be covered, not only adds a certain degree of difficulty but also makes the hunt more entertaining and interesting for the students. The system’s capability to inform teachers about the students’ answers on every point is a unique feature that we did not meet to any other similar application that we tested, and we believe it can be very handful to teachers that do not only use the system for its entertaining features but also for the educational benefits it provides.

RouteQuizer’s purpose is to encourage teachers adopt mobile learning in their teaching and benefit students in every possible way. That is the reason why it is accompanied by propositions that will make RouteQuizer more effective on pursuing its purpose.