1. Introduction
Due to the spread of mobile information terminals, such as smartphones and tablet PCs, in the current advanced information and communications society in recent years, anyone can easily access the internet and transmit and obtain information. However, as a plethora of information is available on the internet, each user must efficiently search the information they seek by themselves. Additionally, because a wide variety and a large amount of information are available, it may take time to search and obtain the necessary information. Therefore, information literacy is required for users to appropriately determine what information is necessary and how to effectively use it. The above can also be applied to sightseeing. Although tourists can search for the necessary information on the internet while sightseeing, it takes effort and is inconvenient to obtain the available information related to specific sightseeing spots among the copious amount of information online. Therefore, it is necessary to develop a method to efficiently and appropriately present sightseeing information to tourists. In response to this necessity, Dinis et al. (2019) [
1] proposed a methodology that uses the big data from Google Trends (GT) tool to develop composite indicators to measure the public online search interest by tourist destinations. Additionally, Wise et al. (2019) [
2] described the significance of the Internet of Things (IoT) and its potential for smart cities and provided practical foundations for destination organizers and stakeholders by integrating social media, content marketing, and wearable IoT devices into the emerging smart tourism paradigm.
On the other hand, augmented reality (AR) is a technology that visualizes information by displaying digital content, such as images and videos, in the actual world by means of wearable terminals, including mobile information terminals and smart glasses (Konoike, 2019) [
3]. There are various types of AR, such as location-based AR, which use the location information obtained using global positioning systems (GPS), image-recognition AR, which recognizes images and markers, and markerless AR, which recognizes flat surfaces and spaces. Information technology developed using AR is not only used for sightseeing and disaster prevention (GrapeCity, 2019) [
4] but is also used for Pokémon GO and other applications for various purposes that are used in the actual world by combining the technology with GPS. In this way, AR has become more common even to the general public. Additionally, AR is used in many various ways and situations and can also be used as a method to provide sightseeing information for the general public. The usefulness of providing sightseeing information using AR is evident from the results of the preceding studies (Fujita et al. 2016 and Zhou et al. 2016) [
5,
6] that developed the navigation system for normal conditions and for disaster situations, as well as the system to provide sightseeing spot information.
Meanwhile, since the start of the “Visit Japan Campaign” in 2003, Japan experienced 10 million foreign visitors by 2013. Additionally, due to the Tokyo Olympic and Paralympic Games that will be held in 2020, the number of foreign tourists is expected to increase. However, it has been pointed out that the most serious complaint among foreign tourists towards the Japanese sightseeing industry is the lack of foreign language services. Therefore, this issue must be resolved. As a solution to this issue, Yamamoto (2018) [
7] and Abe et al. (2019) [
8] proposed sightseeing support systems using English and other nonlinguistic information, including pictograms (symbols and marks).
Therefore, since foreign tourists come from various countries, and all of them may not necessarily completely understand Japanese or English, it is essential to develop a sightseeing support system that effectively uses nonlinguistic information. Additionally, it is desirable to adopt AR into the system that provides guidance and information concerning sightseeing spots. However, as described in the next session, such systems that meet these demands have not yet been developed. Based on the social and academic background described above, the present study aims to target urban tourist areas in Japan to develop a system that can provide guidance and information concerning sightseeing spots by integrating location-based AR and object-recognition AR and uses pictograms. Specifically, by using these two types of AR, the system can appropriately provide guidance for users. Additionally, using location-based AR and pictograms, this approach can also efficiently provide information concerning sightseeing spots for both Japanese and foreign users. In Japanese smart cities, as Wise et al. (2019) [
2] described, it is expected that everyone would be able to fully enjoy sightseeing using the system as one of smart tourism tools, including mobile applications, AR and near-field communications (NFC). In other words, one component of the system is mobile applications that adopt the two types of AR that are powerful tools for NFC. This method will be detailed in the next section. Although the present study will develop a system that will operate within urban tourist areas in Japan, it is possible to apply it to similar areas in other countries. Because the system does not depend on any language, it is necessary to elaborate the display of nonlinguistic information, including pictograms.
2. Related Work
The present study is related to four study fields, including (1) studies related to navigation using location-based AR, (2) studies related to navigation using image-recognition AR, (3) studies related to information services using AR, and (4) studies related to information services using pictograms. In (1), using the AR smart glasses, Fujita et al. (2016) [
5] developed a navigation system that can be used during sightseeing and disasters, while Zhou et al. (2016) [
6] developed a recommendation system for sightseeing spots, and Rehman et al. (2017) [
9] proposed an indoor navigation system. Morozumi et al. (2016) [
10] and Chung et al. (2016) [
11] developed a navigation system to navigate, users to their destinations. Amirian et al. (2016) [
12] developed a landmark-based pedestrian navigation system using machine learning in addition to AR. Gerstweiler et al. (2016) [
13] presented a hybrid tracking system specifically designed for complex indoor spaces that runs on mobile information terminals.
In (2), Mata et al. (2011) [
14] developed an experimental virtual museum using paintings. Mulloni et al. (2011) [
15], Okada et al. (2011) [
16], Möller et al. (2012) [
17], and Neges et al. (2017) [
18] proposed a system that points an arrow in the direction of the destination by recognizing the user’s present location using images. Kurihara et al. (2014) [
19] developed a marker-based indoor navigation system that allows location information to be shared. Wang et al. (2018) [
20] proposed a mobile navigation system, adopting a multitarget AR recognition mechanism and polygon approximation-based data acquisition. Zhou et al. (2019) [
21] designed a tourist attraction guide system combining image-recognition technology and AR technology.
In (3), Fukada et al. (2011) [
22] developed a system that provides sightseeing information using image-recognition AR, while Han et al. (2014) [
23] developed a mobile sightseeing application that provides information related to urban heritage using location-based AR. Komoda et al. (2013) [
24] developed a system that provides information related to regional movies using images. Jung et al. (2015) [
25] used a quality model to test user satisfaction and intention to recommend marker-based augmented reality applications in the case of a theme park. Ma et al. (2018) [
26] developed a smart sightseeing application system combining AR technology and a paper sightseeing map for emergency response. Blanco-Ponsa et al. (2019) [
27] presented an application that provides information related to cultural heritage. Makino et al. (2019) [
28] developed a system that visualizes spatiotemporal information in both real and virtual spaces, integrating social networking services (SNS), Web GIS, mixed reality (MR) and the original gallery system, as well as Wikitude and connecting external social media. Both virtual reality (VR) and AR are integrated into MR that is used in the system for area-based learning and sightseeing.
In (4), Kusano et al. (2013, 2015) [
29,
30] developed a disaster information sharing system, while Yamamoto (2018) [
7] and Abe et al. (2019) [
8] developed sightseeing support systems using nonlinguistic information, including pictograms, as mentioned in the previous section. Sándor (2017) [
31] examined the impact of weather-related warning message signs on traffic in adverse weather circumstances on motorways. Hayashi et al. (2019) [
32] proposed an information-providing method to express train service situations in central urban areas by combining multiple pictograms, such as sign–logo images. Kołodziejczak (2019) [
33] analyzed the information systems on internet tourist portals and suggested the uses of various symbols for accessible sightseeing for people with disabilities.
From the results of (1) and (2), i.e., the studies related to navigation using location-based AR and image-recognition AR, in smart cities, it is evident that AR is one of the powerful smart tourism tools to appropriately navigate the general public to their destinations. However, the AR smart glasses, which were used in Fujita et al. (2016) [
5], Zhou et al. (2016) [
6], and Rehman et al. (2017) [
9], have not been popular all over the world. Moreover, considering the limitations of Kurihara et al. (2014) [
19] and Zhou et al. (2019) [
21], in the case of outdoor navigation, object-recognition AR is more suitable than image-recognition AR to show the information related to specific spots. The results of (3), i.e., the studies related to information services using AR, show that the general public, in addition to scientists and technicians, can efficiently obtain various information by means of AR technologies. Additionally, the results of (4), i.e., the studies related to information services using pictograms, indicate it is possible to develop a user-friendly system of universal design for both Japanese and foreigners by adopting nonlinguistic information.
In reference to the aim of the present study, as mentioned in
Section 1, it is essential to integrate location-based AR and object-recognition AR and use pictograms in a unique system, despite these methods being separately adopted in the above 4 study fields. Therefore, in comparison with the abovementioned studies, the present study demonstrates originality by developing a unique system that can provide guidance and information concerning sightseeing spots by integrating location-based AR and object-recognition AR as smart tourism tools and by using pictograms as nonlinguistic information. Specifically, considering the findings and limitations based on the results of the above preceding studies, the present study demonstrates originality by navigating users to sightseeing spots, as well as by providing sightseeing spot information, using both location-based AR and object-recognition AR, in addition to visualizing and providing information about facilities near sightseeing spots on mobile information terminals using location-based AR and pictograms. Therefore, by means of the system, it is expected that even those who are not good at reading maps in either the digital or paper format or at using Google Maps and who have limited knowledge or sense of locality would be able to fully enjoy sightseeing.
3. System Design
3.1. System Characteristics
As shown in
Figure 1, the system in the present study is composed of the website and the original mobile application, and the latter is uniquely developed by integrating location-based AR and object-recognition AR and by using pictograms. To use the system, users must first access the website on their mobile information terminals, such as smartphones and tablet PCs, and install the original mobile application that has been uploaded onto the online storage. Then, users can start using the system by activating the application and registering their information, including email addresses and passwords.
Using images, such as pictograms, that are provided by location-based AR, the system can display the directions to sightseeing spots and nearby facilities on the screens of mobile information terminals and provide the navigation to each destination. Furthermore, as the system displays images on mobile information terminals in a way that overlaps with the actual world using location-based AR, users do not need to walk around with pamphlets or guidebooks. Additionally, as users can easily obtain the relevant information by pointing the mobile information terminal in the direction of the sightseeing spot and using object-recognition AR, the system contributes to raising efficiency while sightseeing. Therefore, with a system that makes obtaining sightseeing information efficient, tourists who are not familiar with the local area can enjoy sightseeing without relying heavily on information found on the internet.
3.2. Usefulness of the System
The following three points explain the usefulness of the system.
(1) Guidance and information services using AR
The system displays images for sightseeing spots, as well as pictograms for facilities near sightseeing spots, which enables users to know the directions of their destinations. As the images are displayed using AR, those who are not skilled at reading maps can also grasp the directions through the screen of their mobile information terminals. Additionally, users can obtain the related information without having to go online to research the sightseeing spots they wish to visit by pointing their mobile information terminals to sightseeing spots or their images.
(2) Information services using pictograms
The system uses nine types of pictograms (i.e., stations, convenience stores, public restrooms, lodging facilities, restaurants, parking lots, police stations, bus terminals, and rental cars) provided by the Foundation for Promoting Personal Mobility and Ecological Transportation [
34]. Therefore, users can easily understand what facilities are indicated by the pictograms displayed on the screens of their mobile information terminals.
(3) Dynamic and real-time services
The distances from users’ present locations to all sightseeing spots are constantly updated in real-time. Therefore, the users’ location information is updated by GPS, and the distance to each sightseeing spot can be provided to them on a real-time basis. Additionally, the above nine types of pictograms that are nearest to the users’ present locations will be displayed each time the distance information is updated, which helps users to see what facilities are available nearby.
3.3. Target Information Terminals and Operating Environment
The system is meant to be used mainly on mobile information terminals both indoors and outdoors. As the mobile application used on mobile information terminals is set to be an Android application, the OS is also required to be Android (5.0 or higher). Due to the system requirement of Wikitude SDK used in application development, mobile information terminals must have a compass, network positioning, an accelerometer, high resolution, high-performance CPU and OpenGL (2.0 or higher), in addition to a camera and GPS function.
3.4. Overview of the System Design
As mentioned in
Section 3.1, the system in the present study is composed of the website and original mobile application, and the latter is uniquely developed by integrating location-based AR and object-recognition AR and by using pictograms. The application can be downloaded from the website. Additionally, the images used for object-recognition AR and user’s manual can be viewed. The following will provide an explanation concerning the application, as well as location-based AR and object-recognition AR, that makes up the application.
3.4.1. Mobile Application
In the system, the mobile application that is uniquely developed by integrating location-based AR and object-recognition AR, and by using pictograms in an Android application. In developing the application, Android Studio, which is an integrated development environment for mobile applications, as well as Wikitude SDK, which is the SDK for AR application development that can be installed into Android Studio, were used. Android Studio can not only develop an application layout but can also confirm the camera and GPS permissions installed into Android devices as well as load the location information of the mobile information terminals. Since Wikitude SDK uses web technologies, such as HTML and JavaScript, the screen layout for the application was developed using xml files for Android Studio, HTML files, and CSS files.
For Android Studio, Java was used to set the camera and location information of the mobile information terminals, while JavaScript was used in Wikitude SDK to develop location-based AR and object-recognition AR. The screen layout was designed using HTML files and CSS files, while the user interface was designed using jQuery Mobile.
3.4.2. Location-Based AR
As the basic function of location-based AR, in addition to the marker images indicating the directions of sightseeing spots, the pictograms indicating facilities, such as stations and public restrooms, are displayed on the screens of mobile information terminals, and the directions are provided to sightseeing spots and nearby facilities. Furthermore, users can tap the image on the screens to see the name of each sightseeing spot and the distance to it and tap the marker to refer to each sightseeing spot information.
When starting the mobile application, the location-based AR screen will always appear. When this screen comes up, the user’s present location will be determined using GPS, and the images of sightseeing spots and nearby facilities will be allocated in each specified part of the screen. Then, the distance from the user’s present location to each sightseeing spot will be calculated. While the location-based AR screen is up, the present location will be constantly updated on a real-time basis using GPS, and the distance to sightseeing spots and nearby facilities displayed as images on the screen will be recalculated. Therefore, the system can provide users with real-time distances to each sightseeing spot.
As the displayed image is allocated based on the location information (latitude and longitude) of sightseeing spots and nearby facilities, the location information of such places is researched using Google Maps. The file, along with the names of such places, was added into the application in the JSON format. Regarding pictograms, the location information and numbers according to the type of facility were added, and the pictogram that corresponds to each number was set to be displayed. The process mentioned above is shown in
Figure 2.
3.4.3. Object-Recognition AR
As the basic function of object-recognition AR, sightseeing spots can be recognized by pointing a mobile information terminal at actual sightseeing spots or the images of such places, and the related information can be displayed on the screen. Users can switch between the screens of object-recognition AR and location-based AR.
Object-recognition AR uses a web tool called Wikitude Target Manager provided by Wikitude to register 20 to 30 images of one sightseeing spot that is recognized. After the number of characteristics for the objects used in the recognition of registered images were extracted, files containing the characteristic amount for every sightseeing spot were downloaded and installed in the mobile application. Additionally, the images displayed for sightseeing spots were also installed into the application, and the location settings of each image were determined. The process mentioned above is shown in
Figure 3.
7. Conclusions
The present study designed and developed a system (
Section 3 and
Section 4), conducted a test of the system operation (
Section 5), and evaluated and extracted improvement measures (
Section 6). The present study can be summarized in the following three points:
(1) To provide guidance to sightseeing spots and nearby facilities using location-based AR and pictograms and to provide information on the sightseeing spots using object-recognition AR, a system that is composed of the website and original mobile application was designed and developed. The application was uniquely developed by integrating location-based AR and object-recognition AR and by using pictograms. As the system enables users to efficiently obtain the directions to sightseeing spots and nearby facilities within urban tourist areas and sightseeing spot information, it is a method that can help users who are not good at reading maps or searching necessary information to enjoy sightseeing. Additionally, the city of Chofu in the metropolis of Tokyo was selected as the operation target area for the system, and an evaluation of the operation of the system was conducted.
(2) The operation of the system was conducted over a one month period, targeting those inside and outside the operation target area, and a web questionnaire survey was conducted with a total number of 50 users. Based on the results of the web questionnaire survey, the usefulness of the original functions of integrating location-based AR and object-recognition AR and of using pictograms, as well as of the entire system, was highly rated, indicating that efficient sightseeing support for users can be expected. Although most users were not familiar with the application that used AR, they highly rated the functions using location-based AR and object-recognition AR.
(3) From the results of the access analysis of users’ log data, the transition of the number of active users revealed that the system was continuously used during the operation. By continuously using the system, it is expected that users will further utilize each function. Based on the results of the user engagement rate, the usage time was 67% for location-based AR and 29% for object-recognition AR, indicating that the former was used more often because it is easier for users to use.
As future study projects, the improvement of the system based on the results in
Section 6.3, as well as the enhancement of the significance of using the system by gaining more data from other urban tourist areas inside and outside Japan, can be raised. Additionally, it is desirable to develop a unique mobile application in accordance with the above improved system.