A Multi-Criteria Evaluation of Applications Supporting Public Transport Users

Abstract

Reducing the energy consumption of transport in urban areas is possible if appropriate measures are taken to make public transport more attractive. These include all kinds of journey planners that are part of the passenger information system. Various applications available on the market allow passengers to evaluate their usability. This paper presents and compares nine of the most popular journey planners available to iOS and Android users travelling in Krakow. The comparison took into account all the information obtained from the surveys. In addition, using a multi-criteria approach, the final ranking of the set of journey planners was developed. The assessment was made on the basis of a set of nine criteria indicated by travellers as the most important ones. The obtained results showed disproportions in the functionality of particular solutions. They also indicated the apps that are most frequently and willingly used by local (urban city/agglomeration) travellers.

1. Introduction

1.1. Background

In the life of every medium or large city dweller, urban public transport plays a key role in reducing traffic congestion that is particularly troublesome during rush hour. Efficient public transport allows the user, on the one hand, to reduce travel time but, on the other hand, to reduce the energy consumption of transport per passenger, which is nowadays highly emphasised. According to Perez-Martinez and Sorba [1], the energy consumption for cars ranges from 1.9 to 1.58 MJ pkm⁻¹ (per passenger-kilometre), while for journeys by public transport these values range from 0.25 to 1.01 MJ pkm⁻¹. In turn, the International Energy Agency (IEA) [2] reports that for cars these values currently vary in different countries between 0.8 and 2.9 MJ pkm⁻¹, for public means of transport between 0.1 and 1.1 MJ pkm⁻¹. Taking even the average values into account, it can be concluded that the energy intensity of a journey (per passenger-kilometre) made by a passenger car is always several times higher than that of a journey made, for example, by a regional train, bus, tram or underground. This ultimately translates into the total energy intensity of transport, which in the United States, for example, is estimated at 28% of the total energy consumption [3]. However, it should be stressed that 58% of this energy is used by light vehicles (passenger cars), motorbikes and trucks. Only 3% is estimated to be used by public transport such as buses, trams, railways and underground trains. [3]. Obviously, in other countries these proportions differ, e.g., the number of cars per capita, distances travelled by car, different energy consumptions of cars per km, the mobility of public transport per capita. In Poland [4], in 2019, the energy consumption of transport had increased by more than 100% in less than 20 years. In comparison, over the same timeframe, in Germany, the Netherlands or France a decline in energy consumption was recorded [4].

Among the elements that indirectly translate into the lower energy intensity of passenger transport, we should mention the efforts made by the organisers and operators of public transport aimed at the effective use of their resources [5,6,7]. These efforts include, among others [5,6,8,9]: timetable development, vehicle and crew scheduling, route choice and assignment, network (routes) design, etc. In order to achieve the policy goals of sustainable transport development, it is also necessary to design and implement passenger information systems (PIS), which should be adapted to the needs of the passengers using the system of public transport [10,11,12]. Consequently, in the era of rapid development of mobile technologies, the process of moving around urban areas has become much less complicated from the passenger’s point of view. In addition, such a system also allows the formation of travel behaviour [13,14,15,16,17,18,19], that takes into account general traffic flow management [20], predictive solutions [21] and solutions/technologies in a smart city [22,23]. Thanks to various travel planning applications, public transport users are not limited to traditional timetables. The so-called journey planners allow them to decide on the key elements of their journey, from the choice of the means of transport to the distance they need to cover to reach the chosen stop or transfer point. It is these applications that largely encourage people to travel by sustainable transport; in particular they break down the barrier of ignorance about the transport on offer in a given area. By way of example, people travelling for the first time in a new city may not be familiar with its topography or the functioning of its transport system, and with the help of appropriate applications, the traveller obtains the necessary information on getting around from one place to another.

However, it is certain that any information and functionalities provided in journey planners should be tailored to the needs of the specific customer—i.e., the public transport user in the area. Therefore, the evaluation of applications should be carried out primarily among the stakeholders in a specific area and should include the applications allowing for travel planning in a specific location.

In this article, the region of the City of Krakow, the second largest city in Poland, is defined as the area of application use. According to a study verifying transport behaviour in 2018, the proportion of residents’ journeys by public transport was 30%, while by private car it amounted to 40%. Every day in Krakow, about 940,000 inhabitants need to transfer mainly to their place of work or study. About 230,000 journeys are made from the neighbouring regions to the city [24]. In 2016, the City Council of Krakow established a new transport policy [25] whose goals were to:

• Increase passenger transport efficiency,
• Reduce the level of energy consumption of passenger transport.

With the above research problem presented by the authors of this publication in mind, the following questions can be posed:

• To what extent and which local travel planning apps are used by travellers?
• What elements do travellers look for when choosing a local travel application?
• How does the popularity of applications developed for the local market compare with those used in other countries?

In view of the above, the authors believe that the implementation of the above objectives calls for the adaption of journey planners to the stakeholders’ requirements. Therefore, a holistic evaluation of the applications available in Krakow by those using them both daily and occasionally should be the beacon for app developers in the future.

1.2. Literature Review

For some time now, there has been growing interest among researchers in studying the development and usability of mobile travel applications tools [26,27,28,29,30,31,32,33,34,35,36,37]. The various journey planner solutions have a number of different attributes (functionalities), which translate into less information for the traveller to use. This is related, among other things, to geographical coverage of the route planners [33]. For instance, interurban–urban planners provide less information than the other groups (urban, interurban–global and global) [33]. Here, it needs to be stressed that identification of user behaviour is key in journey planning. Boratto et al. [34] showed that users speaking different languages behave differently and point to the fact that English speakers cover longer distances when travelling. The authors of this publication believe that this group must use different journey planners to those travelling shorter distances. Due to the diversity of traveller behaviour, in order to meet their specific transport needs, it becomes necessary to carry out an evaluation of the travel planner solutions that are designed, implemented and then used. For example, in paper [28], the authors deal with the usability evaluation of an intermodal PIS, which emphasizes guiding passengers in intermodal scenarios throughout the whole journey. A prototype of an intermodal PIS is investigated in a usability evaluation and tested in comparison to the leading mobility application in Germany—DB Navigator. Both iOS apps were evaluated by means of a questionnaire using the system usability scale in a lab setting and in a field test. The approach presented by Noertjahyana et al. [29] includes a prototype solution intended for the city of Surabaya in Indonesia and its public transport provided by cars. Here, the authors demonstrated the usefulness of the application, but stressed the need to also consider public transport. In contrast, a passenger system supporting the use of small buses and minibuses in South Indian cities was proposed by Beul-Leusmann et al. [30]. In this case, the application was Android based, and in addition to the current location of the means of transport, it also provided information on the current occupancy of the vehicles. The developers’ main objective was to encourage passengers to use public transport so as to reduce congestion and air pollution in cities.

However, it should be stressed that, due to the diversity of stakeholders, it is advisable to carry out an evaluation of the travel sites using multiple criteria aid (MCA) methods [38,39,40,41]. The approach presented by Esztergár-Kiss and Csiszár [35,37] and Esztergár-Kiss [36] may serve as an example thereof. Thus, in the case of [35], the authors evaluated 20 different journey planners, taking into account such aspects as:

• Spatial coverage (urban, regional, international).
• The transport modes used, i.e., one mode (rail, air), more public transport modes (bus and tram and rail); combined (individual cars and public transport).
• Dependency on service provider if it belongs to one service provider (dependent), independent.

The authors elaborated an evaluation method to decide which of several popular and typical journey planners were qualified and compared. Based on this, journey planner applications were ranked. The very same approach was taken by those authors in [35]. In this case, the MCA was used to analyse the journey planners of bus transport operators (Volán) in Hungary. In the latter of the papers considered here, Esztergár-Kiss [36] evaluated 14 different web-based journey planners.

The fact that the said applications also needed to be evaluated at the city or agglomeration level is evidenced by research conducted by Javid et al. [18] in Lahore. Those authors concluded that their performance is of great local importance, due to the fact that the city of Lahore has different social, cultural, and transportation system characteristics in comparison with other regions. Nevertheless, it was stressed that the findings of that study helped in drafting the policies for the development and improvements of app-based demand-responsive public transport services.

To sum up, it should be emphasised that the interest in applications is constantly growing, and thus the users’ expectations have also increased. Developers are constantly improving their solutions, e.g., by integrating information that, on the one hand, can be communicated to the traveller and, on the other, is relevant to transport organisers. Contemporary travel planners have several common features, which include the following:

• Taking into account the actual departure time of the means of transport;
• Indicating the geographical location of public transport stops;
• Indicating the GPS vehicle location;
• Presenting travel routes on a map overlay;
• Using Internet communication for acquiring timetable data.

The multitude of travel planner apps, some of which are described in Section 2 of this paper, indicates that the additional functionalities that can be introduced are varied. For example, the applications are equipped with the following features and systems:

• Information on actual delays or speed-ups of public transport vehicles as well as on possible changes in routes of individual lines due to random events;
• Purchasing single tickets, sectional tickets or season tickets;
• e-Payments;
• Information about bicycle, motorbike, scooter and car sharing points.

2. Available Travel Planning Applications

Nowadays, there are many available applications supporting travelling around the city, but how does one choose the ones that will be the most useful and convenient for daily trips to destinations such as work, study or leisure. The applications will provide information on how to get from one place to another, what means of transport to use, how long the journey will take, what the travel cost will be, when the tram, bus, metro or train will arrive, the current location of the means of transport, etc. Some applications also offer the possibility of buying tickets, renting a car, a moped or an electric scooter. The most commonly used travel planning applications include [32]: E-podróżnik, Google Maps, Jakdojade, Kiedyprzyjedzie.pl, Mobile MPK, Moovit, myBus online, Transportoid. In the next part of the paper, the above-mentioned applications are synthetically characterised. They were selected based on the literature review and internet rankings.

2.1. E-Podróżnik.pl

E-podróżnik.pl. is an application that makes it possible to combine public and intercity transport, it also allows the purchase of tickets for several means of transport such as minibus, rail, bus and city/local. The ticket offer of the e-podróżnik.pl application is limited to national and international tickets, but does not offer the option to purchase city transport tickets. The application has a single desktop, where the connection search engine and links to other application tabs are located (Figure 1a–d). The planner is equipped with a trip filter which allows the passenger to specify their trip by setting a number of parameters. The user also has the possibility of searching only for the journeys for which tickets are sold online. This application is equipped with a function “My tickets”, which stores tickets that are pending, active, archived and cancelled. In addition, the application allows the user to enter basic data in advance (the Account Data tab). For those searching for a transfer route, E-podróżnik.pl provides such information as total travel duration, duration of particular stages of the journey, waiting time for a transfer vehicle, the distance to the bus stop from the starting point, the distance between transfer points, time needed to get to the bus stop, the route including the stops of a given transport line and information on the carrier. Another interesting feature of the application is the “Money box”, which allows the user to top up a virtual account, from which the user can then pay for the purchased tickets. The application is free of charge. It can present the timetables of public transport in more than 30 cities in Poland. Moreover, the user has the possibility of checking public transport timetables abroad in such countries as Belarus, Czech Republic, Estonia, France, Germany, Great Britain, Italy, Latvia, Lithuania, Russia, Spain, Ukraine—the total of 1500 carriers (https://mobile.e-podroznik.pl (accessed on 20 January 2021)).

2.2. Google Maps

Google Maps is a mobile application created by Google. Initially, it was designed for people travelling on foot or by private transport. Relatively recently, it has also been addressed to public transport users thanks to an extension called Google Transit. Available for Android and iOS smartphones, the app allows the user to define public transport options such as preferred modes, route options and connections to public transport (Figure 1a). As soon as the user enters a starting point and the destination point, the app automatically generates a route for the different modes of public transport and presents all the possible options for the route and the estimated travel time (Figure 2b).

Mode selection preferences can be adjusted at any time. The application also has a route filter, which displays only those public transport vehicles that are adapted to the needs of disabled users. Google Maps displays the connections with changes, navigates the traveller from the starting point to the stop, presents the total journey time and the time of each journey stage, displays all stops of a given transport line, and additionally indicates the distance to the stop or between stops that the traveller needs to cover. The app allows users to combine journeys by public transport with the use of city bikes. The application is available at no charge. The user can check the accessibility of public transport in more than 15,000 cities in the world (https://www.google.com/maps (accessed on 16 December 2020)).

2.3. Jakdojade

Jakdojade is a Polish application which is often chosen by the users of larger cities or agglomerations to support travel planning. It was designed by CITY-NAV Spółka z o.o. Available for Android and iOS, the app has several configurable tabs. If the GPS location is not activated, the user selects the location where the journey is to be made from a closed list. Only then is it possible to select the starting point and the destination (Figure 3a). From here, in the Options tab, it is possible to indicate the type of route, the minimum transfer time, the type of vehicle (e.g., low-floor vehicles, which is important for disabled people in wheelchairs) as well the public transport operator (Figure 3b). In addition, the user defines the time parameters of their journey (time and day of departure and arrival). Once the key data have been defined, the user is presented with route alternatives (Figure 3c). After selecting one of these, detailed information about the journey is provided including journey times, number of stops and waiting time (Figure 3d). At this point it is possible to purchase tickets for the selected journey. The basic version of the application is free of charge. The premium version of the app allows for real-time tracking of the location of public transport vehicles. The application features not only the function of planning journeys in urban transport, but also in rail transport and it offers access to the timetables of nearly 50 cities in Poland.

2.4. KiedyPrzyjedzie

KiedyPrzyjedzie is an official application of Operibus Sp. z o.o. combining the functions of a dynamic passenger information system and a tool to manage the fleet of public transport vehicles. It is compatible with Android and iOS. The application is dedicated in particular to smaller and medium-sized towns. KiedyPrzyjedzie only informs passengers about the public transport vehicles that will arrive at a given stop within 4 h. However, the passenger has no possibility of tracking the location of the vehicle on the map in real time (online). The application’s interface is not very sophisticated, as it is only possible to select the agency of public transport (Figure 4a). In the Starting point and Destination tab (Figure 4b), the user can select the desired stop (Figure 4c). In the next step, the app redirects the user to Google Maps. In contrast to other apps, this one does not require any logging-in, does not have any route filtering options or the option of ticket purchasing. KiedyPrzyjedzie is available for 81 cities in Poland and several cities in Austria and the Czech Republic.

2.5. Krakow Pod Reka

An application developed by independent authors, designed to facilitate travel by various means of public transport; this application uses Google Maps and information from external services including the location of buses, trams, rental cars (Traficar) and bicycles (Wavelo). In the case of public transport, the user receives information about the scheduled and actual departure time, as well as information on delays. The solution is characterised by a relatively low level of complexity. The user can set the display of Cracow’s public transport stops (Figure 5a—KMK stops), or car locations (Figure 5a—Traficar). Moreover, it is possible to filter the display of public transport lines (Figure 5b,c). If a specific line is indicated, the user is informed about the departures from the consecutive stops and whether the means of transport is accessible to disabled passengers (Figure 5d). The presented solution is dedicated only to the city of Krakow.

2.6. mobileMPK (BusNavi)

mobileMPK is a Polish mobile application developed by the MobiCORE company. In other languages than Polish it is known as BusNavi. It is available for Android smartphones. Just like the Jakdojade application, mobileMPK contains three main tabs: (1) favourites (Figure 6a), (2) redirections to connections, timetable from a given stop, available city bike stations in the vicinity, address finder, maps and the option for sending the coordinates of where the user is located (Figure 6b), (3) defining the departure and arrival points and setting the parameters of the journey (Figure 6c). mobileMPK is available in two versions: a free light version and a paid version Pro. In the case of the latter, no advertisements are displayed to the user. In addition, it is also possible to add the timetable of your own preferred lines. The application is additionally extended with a function of displaying delays and positions of buses in real time in the cities where such a solution is implemented. In the case of connections with transfers, mobileMPK provides the time for each stage of the journey, including the time for arrival at the stop and the waiting time for the means of transport at the designated stop (Figure 6d). The app allows the user to search for low-floor vehicles by selecting the appropriate route filtering option. Thanks to the location function, the traveller can also check the available city bike stations. The application uses Google maps to visualise the route. mobileMPK allows users to view public transport timetables for over 60 cities in Poland, the UK, Ireland and the US.

2.7. Moovit

Moovit is an Israeli mobile application using OpenStreetMap maps to support public transport trip planning developed by the company of the same name—one of the world leaders in mobile applications dedicated to public transport. The application is available for Android and iOS mobile phones. Similar to the previously described applications, it also has three main tabs: (1) “Directions” which is a connections search engine, featuring favourite destinations (suggested: home, work) and an option that allows the user to order Uber (Figure 7a), (2) “Stations” where stops are presented on the map (Figure 7b), (3) “Lines” containing timetables for the following lines: bus, tram and train (Figure 7c). The proposed route is displayed as a straight line with a list of stops (Figure 7d). It should be noted at this point that the application makes it possible to monitor a journey in such a way that the user, having previously set a destination, will be notified to get off before the final stop. After the trip, it is possible to rate the quality of the public transport service. In addition, the passenger using the application receives information about detours or changes to the timetable. Moovit has a journey-plan function which allows the user to filter the routes, search for a connection with the minimum number of transfers and the minimum walking distance to the stop. The app also gives the user the option to choose their preferred mode of transport, with a choice of bus, train, tram, bike and electric scooter. The application is free of charge. In its database, it has passenger information from 3400 cities and 112 countries all over the world—about 7500 public transport operators. The Moovit application can be used in 32 Polish cities (https://moovit.com (accessed on 10 January 2022)).

2.8. myBus

myBus is an application which is embedded in the passenger information system (SIP) of the MUNICOM.premium software developed by PZI TARAN Spółka z o.o. The application is available for both Android and iOS smartphones. In contrast to other applications, myBus online is adapted for the visually impaired, the function works independently on the basis of continuous reading of the current GPS position, which is used to find the nearest bus stop. The application informs the visually impaired user about the distance they have to travel from their location to the bus stop. Moreover, if the user is already at a bus stop, the application reads out the lines of public transport that depart from it and the current time. The main advantage of myBus Online is the GPS tracking of public transport vehicles. Thanks to this function, the user obtains a more precise time of arrival of a means of public transport at a given stop, i.e., the actual departures including the time of delays or speed-ups of vehicles which are not included in the timetable. The application’s main screen displays a closed list of cities where it is possible to search for a route (Figure 8a). In the next stage it is possible to use a list of functions of the application (Figure 8b). As in other applications, after defining the starting and destination points and the number of transfers, the user is presented with proposed routes (Figure 8c,d). myBus uses Google maps to visualise the route. The application is available in an online and offline mode. In contrast to the offline mode, the online mode keeps the timetables up to date and provides access to the nearest departure boards. Importantly, the programme uses QR codes which can be found on the stops. The application enables journeys in 48 cities, of which 45 are in Poland.

2.9. Transportoid

Transportoid is FTL Software’s timetable application for Android and iOS smartphones. Its interface consists of four main tabs: (1) “Favourites”, where the user can add favourite routes (Figure 9a), (2) “Timetables”, containing timetables for each stop in the city (Figure 9b), (3) “Lines” containing timetables for each public transport mode (Figure 9c) and (4) “Direction”, a connection search engine (Figure 9d).

Transportoid allows the user to search for the connections with a minimum transfer time, a maximum transfer time and a minimum distance to the stop that the traveller has to cover. The application has a function of indicating low-floor vehicles. Transportoid users are kept up to date with all changes in the functioning of public transport including timetables, each upgrade is updated in real time. The basic version of the application is free of charge, those who want to use more advanced functions such as displaying selected timetables on the desktop, grouping stops and GPS functions, have to pay a fee. The application allows the user to set individual reminders for the purchase of season tickets, and it also offers them the offline option. As one of the described solutions, it has an interface in Polish only. The application offers access to the timetables of 71 cities in Poland (as of 15 September 2021).

3. Methodological Framework

3.1. The General Concept of Methodology

In this paper, the evaluation of the travel planning applications was carried out in stages. A diagram of the procedure is shown in Figure 10.

The basis for the final evaluation of the applications is stage 1, i.e., the available travel planning applications, followed by identification of available functions per app. Subsequent stages (3.1–3.4, 4 and 5.1–5.4) provide the final ranking of the evaluated travel planners. As can be seen in the case of stage 3.1, internet sites or resources and tests of apps constitute the basis of the evaluation. While stages 3.2 and 4 were based on surveys, 3.4 was based on the data reported by Google Play shop.

3.2. MCDM/MCDA

This section provides general information about multiple criteria decision making (MCDM)/multiple criteria decision aid (MCDA) and presents one of its methods (compensation-conjunctive method), which was used to assess the applications supporting public transport users. Furthermore, the methodology and survey results used in the MCDM/MCDA analysis were presented.

MCDM/MCDA is a scientific field originating from operations research, involving the solutions to complex decision-making problems where multiple, often opposing, points of view are considered. In general, we can distinguish the following multi-criteria problems [38,40,41]:

• Choice, in which one option is selected from among many that belong to a finite set.
• Order (ranking), in which the set of options is ordered from the best to the worst (or vice versa).
• Classification, in which the decision-maker allocates the options to predefined classes.

The problem considered in this paper is formulated as a multi-criteria problem of ranking a finite number of variants of public transport travel applications [38]. In the literature [42], one may find a number of approaches (methods) used for solving transport-related problems. In this paper, the compensation-conjunctive evaluation method was used to evaluate different travel planning applications in combination with social surveys of a selected group of respondents. [8]

The compensation-conjunctive method is used for multi-criteria evaluation of variants by means of multiple criteria. It consists of the following steps [8]:

• Determination of a consistent family of criteria evaluating a finite number of variants.
• Assignment of weights to the criteria.
• Determination of threshold criteria.
• Assessment of the degree to which the criteria are met by the considered variant and determination of the desired minimum for the threshold criteria.
• Elimination of solutions that do not meet the threshold criteria.
• Aggregation of partial evaluations into a global assessment.
• Ranking of the solutions according to the value of the global assessment index. In this case, the computations were based on the following formula:

$$S_j={\displaystyle \sum _{i=1}^n{w}_i\cdot s_{ij}},,$$

(1)

where:

$w_i$—the weight of the i-th criterium of evaluation of variants (of the app),

$s_{ij}$—the degree to which the i-th criterion is met for application variant j (the value may be provided in%),

$n$—the number of criteria to be taken into account when assessing the variants.

The sub-weights of the criteria are standardised, i.e.,

$${\displaystyle \sum _{i=1}^n{w}_i}=1.$$

(2)

The global option assessment index $S_j$ expresses the global weighted average degree of fulfilment of all criteria in the decision problem under consideration,

As far as the surveys are concerned, it was assumed that they would be used to:

• Identify a coherent family of criteria for evaluating individual alternatives.
• Determine (calculate) the weights of individual criteria.

As with many other social surveys, they are based to a large extent on the results of surveys of travellers or other stakeholders [43].

Nowadays, a large majority of such studies in the field of transport are conducted online using the so-called computer-assisted web interview [43]. The respondent is asked to fill in an electronic questionnaire. It is considered the most economical way to collect survey data, because there is no need for interviewers, devices or extra tools [44,45,46].

The authors assumed that:

• The target group (the app’s stakeholders) would include residents of the Krakow agglomeration, travelling and using planners to varying degrees while travelling to and from Krakow.
• The research would be carried out within 14 consecutive days of one month.
• The evaluation would apply to the apps used in the area of the Krakow agglomeration, i.e., V1, V2, V3, V5, V6, V8 and V9.

4. Comparison of Selected Travel Planner Applications

4.1. Usability Evaluation of Applications Based on Analyses of Web Resources

All applications supporting urban public transport travel described in Section 2 were compared in terms of functionality. A summary of the labels assigned to each app is presented in

Table 1. List of assessed applications with the adopted labels.

Code of Variant	Name of Application
V1	E-podróżnik
V2	Google Maps
V3	Jakdojade
V4	Kiedyprzyjedzie.pl
V5	Kraków pod ręką
V6	mobileMPK
V7	Moovit
V8	myBus
V9	Transportoid

.

In the web-based assessment, the 25 most important elements taken into account by travellers were considered.

Table 2. Overview of available functions per application.

No.	Utility of Applications	V1	V2	V3	V4	V5	V6	V7	V8	V9	Sum (+)
1	Public transport timetable	+	+	+	+	+	+	+	+	+	9
2	Map of the entire route network for a given mode of transport	+	+	+	−	−	+	+	+	+	7
3	Option of mapping the route from departure to arrival point	+	+	+	−	−	+	+	+	+	7
4	Option of mapping the route from stop to stop	+	+	+	−	−	+	+	+	+	7
5	Information on total travel time	+	+	+	−	−	+	+	+	+	7
6	Automatic location detection	+	+	+	+	+	+	+	+	+	9
7	Option of using GPS	+	+	+	−	−	+	+	−	−	5
8	Information on the actual arrival/departure time	−	−	+	+	+	+	−	+	−	5
9	Offline operation option	−	−	−	−	−	+	−	+	+	3
10	Information on integration with various modes of public transport	+	+	+	−	−	+	+	−	−	5
11	Adjustment for the visually impaired	−	−	−	−	−	−	−	+	−	1
12	Option to save the route	−	−	+	−	−	+	+	+	+	5
13	Ticket purchase option	+	−	+	−	−	−	−	−	−	2
14	Tutorial module	−	+	−	−	−	+	−	−	−	2
15	Information on the time of individual journey stages	+	+	+	−	−	+	+	+	+	7
16	Option of searching for connections outside Poland	+	+	−	−	−	−	+	−	−	3
17	Possibility of switching the language (from Polish to English)	+	+	+	+	−	+	+	+ *	-	7
18	Avoidance of selected means of public transport	−	+	+	−	−	+	+	−	−	4
19	Avoidance of selected lines	−	−	+	−	−	+	−	−	−	2
20	Use of low-floor vehicles only	−	+	+	−	−	+	−	+	−	4
21	Route without or with a minimum number of transfers	−	+	+	−	−	+	+	+	+	6
22	Minimum transfer time	−	−	+	−	−	+	−	−	−	2
23	Distance between transfer locations	−	+	−	−	−	+	+	−	−	3
24	Installable on Android devices	+	+	+	+	+	+	+	+	+	9
25	Installable on iOS devices	−	+	+	+	+	−	+	+	+	7
	Total of (+)	13	18	20	6	5	21	17	16	12

(*)—most elements translated into English.

summarises all the compared functions, where the “+” sign indicates an application that has a particular function and the “−” sign when the application is not equipped with a particular function. All of the analysed applications have access to timetables, automatically detect the user’s location and can be installed on Android devices. At this point, it should be emphasised that nearly 85% of page views are made from Android phones [47].

Thus, in terms of functionality, the best rated travel planner was the V6 application (mobileMPK), which has 21 of the 25 functions compared. It can be used in many Polish cities. The application is primarily characterised by a very large number of options (variants) for filtering routes, which allows travel preferences to be refined. A great advantage of the application is the possibility of working in the offline mode since, apart from updating the application itself, the app automatically downloads timetables for the selected city when it is first launched. This solution is particularly important for those users who do not have access to the Internet while travelling. This may apply to people who, for example, limit the amount of data transmitted over the Internet due to roaming costs. In addition, the application has a tutorial module, so that people who are not fluent in operating a mobile phone can easily cope with this type of travel planner. The real-time arrivals and departures of the means of public transport option is one of the most sought-after options. It is becoming increasingly popular with travellers.

In terms of functionality, the V3 application (Jakdojade) came second. Unlike its predecessor (V6—mobileMPK), Jakdojade has a smaller range of route filtering. The main advantage of the application, apart from the real departure/arrival time of public transport, is the possibility of buying tickets. This function enables the user to purchase their ticket online. The lowest rated applications in terms of functionality are V4 (Kiedyprzyjedzie.pl) and (V5) (Krakow pod ręką). They have respectively 6 and 5 out of the 25 considered functionalities. They function as virtual information boards. Application V4 is available for medium and small cities, while V5 covers only one city. Both indicate the actual departure/arrival times of public transport. A big handicap of these applications is that they do not show the current location of the public transport vehicles on the map, but delays or speed-ups are included in the departure time of the respective stop.

4.2. Evaluation of Supporting Applications Based on Surveys and Using the Ranking Method Pertaining to the Stop

4.2.1. Survey-Based Evaluation

The survey was finally conducted in November 2019, i.e., before the first cases of the COVID-19 pandemic were registered in Poland. One hundred and eighty properly completed questionnaires were obtained (with an estimation error of 6% for a confidence level of 1 − α = 0.90, $u_{\alpha }$ = 1.64). In terms of the structure of the respondents, 62.8% were women and 36.2% were men, with the largest group of respondents (72.9%) aged between 18 and 34 years. In every group except for the respondents under 18 years old, there was a prevalence of women. While the employed constituted the dominant group of the respondents (51.1%), pupils/students also made up a representative group (37.8%). Pensioners (7.2%) and the unemployed (3.9%) were the least numerous group, which is most likely due to the fact that these groups make far fewer trips than the other groups [48,49]. Nevertheless, the authors of this paper are aware of the fact that the mobility of city dwellers has undergone considerable change in Polish cities over the years [50].

As far as the popularity of the apps is concerned, as can be seen in Figure 11, by far the largest number, nearly 50% of those surveyed, indicated Jakdojade as the one they use. Among the less popular ones were Google Maps, mobileMPK and E-podróżnik. The fewest respondents indicated such applications as Transportoid, myBus and Kraków pod ręką.

Considering the frequency of using any application, the biggest group (almost 40% of the respondents) was those who use the apps several times a week. Slightly fewer, 31.1% of the respondents, indicated that they use the app daily. Only 7.2% of the respondents indicated that they used the application occasionally, i.e., several times a year.

Based on the conducted research, it was observed that almost all respondents (97.2%) evaluated the used travel planner positively (good or very good). Only a small percentage of respondents rated the used application as sufficienat (2.2%) or bad (1.1%). Moreover, the respondents pointed out the most important advantages the application should have. The respondents had a choice of 24 elements concerning the travel planner. The most frequently indicated element was its intuitiveness (13.4%) and the information about the departures of the given lines from a specific stop (12.6%). As can be seen in Figure 12, there was a large disproportion in the number of elements that the respondents highlighted.

In the survey, the respondents (stakeholders) also evaluated the performance of individual elements (usabilities) in the application on a scale from 1 to 5 (where 1 meant that the given aspect/element was functioning very poorly or that there was no such option in the application, while 5 meant that its performance was very good). In

Table 3. Average scores for individual travel applications.

No.	Utility of Applications	V1	V2	V3	V5	V6	V8	V9	Average Value	Median
1	Ease of use, intuitiveness of the app	4.0	4.3	4.4	4.5	4.4	3.6	4.0	4.2	4.35
2	Possibility of mapping the route from A to B (in relation to addresses, POI)	4.0	4.2	4.3	5.0	3.7	4.0	4.5	4.2	4.25
3	Search for alternative connections with the minimum number of transfers, and information on integration with different means of public and individual means of transport such as bus, tram, railway, metro, car, bicycle	3.7	3.9	3.8	3.5	3.5	3.6	3.0	3.6	3.60
4	Information on ticket prices, cost of the entire journey, travel time	3.8	3.5	3.8	3.5	3.0	2.0	3.5	3.3	3.50
5	Option to purchase tickets	3.6	1.0	3.7	1.0	1.0	1.0	1.0	1.8	1.00
6	Information on actual departure/arrival times for the respective means of transport	1.0	1.0	3.4	2.5	3.9	3.6	1.0	2.3	1.75
7	Detailed description of the route (understood as “leading by the hand”)	3.6	4.1	3.9	5.0	3.5	4.3	4.0	4.1	3.95
8	Automatic location detection	3.9	4.2	4.1	5.0	3.7	4.0	4.0	4.1	4.05
9	Information on vehicles adapted for people with reduced mobility	3.6	3.4	3.2	3.0	2.9	3.6	3.0	3.2	3.10
10	Access to timetables in many cities in Poland	3.7	3.9	4.0	4.0	4.2	4.0	3.5	3.9	3.95
11	Option to work offline, without the need for an Internet connection	3.7	2.9	2.6	2.0	3.7	4.0	2.5	3.1	2.75
	Average value	3.5	3.3	3.8	3.6	3.1	3.4	3.4	3.43

Explanation: the red colour indicates values below the average and the green colour indicates values above the average.

, the average rating of particular elements of the travel planner obtained from the questionnaires is presented. The red colour indicates the values of the app ratings that are lower than the average, while the green colour indicates the values that are higher than the average for a given usability. As can be seen, the best rated application was V3 (Jakdojade), which received an average score of 3.7, while the lowest rated application was V9 (Transportoid, value 3.1).

In the next question, the respondents were asked to indicate any changes that affected the performance of the app they were using. The most frequent answer was that the application fully met their requirements. Among the remaining answers, the respondents suggested that the applications should include the following elements:

• Notification of the departure time from the current location to the arrival point of the vehicle, up to 5 min in advance.
• Actual departure/arrival schedule based on the location of public transport vehicles.
• Application should be adapted to older versions of smartphones.
• No additional elements associated with advertising.
• Offline availability of the app.
• Possibility of using the app in more than one city.
• Possibility of purchasing tickets.

The last question was addressed to people who do not use travel apps to provide the reasons thereof. The most common reply given by those surveyed: (1) the traditional timetable is fully sufficient, (2) no need to use the app, (3) good knowledge of the city topography, (4) difficulty in using a mobile phone.

4.2.2. Ranking Method Evaluation

According to the methodology described in Section 3, one of the multi-criteria methods for ranking a finite number of alternatives was used to assess the application, i.e., the compensation-conjunctive evaluation method [8]. The nine highest rated criteria, in terms of importance for the users of the application (obtained from the surveys), were used as evaluation criteria. These criteria were normalised, and the obtained summary results are presented in

Table 4. Criteria for the assessment of variants to be considered for ranking of the apps [Adapted with permission from Ref. [31]. 2020, copyright Cholewa.

Criterion Code	Name of Criterion	Normalised Value of the Criterion [-]
C1	Ease of use, intuitive operation	0.20
C2	Mapping a route from point A to point B (relative to addresses, POIs)	0.17
C3	Mapping a route from stop to stop	0.13
C4	Searching for alternative connections	0.12
C5	Information on total travel time	0.10
C6	Ticket purchase option	0.09
C7	Finding connections with the least number of stops	0.07
C8	Automatic location detection	0.06
C9	Information on the actual departure/arrival time of a particular means	0.06
	Total	1.00

.

One of the most important criteria (based on the results of the survey)—criterion C1 (ease of use and intuitiveness)—was adopted as the threshold criterion. The threshold value was set at 65% of the degree to which a criterion was met by a given variant. The actual degree to which each criterion was met by a given variant was then determined. This degree was determined as a percentage on a scale from 0% to 100%, where 0% meant that the criterion was not met at all, and 100% meant that the criterion was fully met by the given variant. The results obtained from the surveys were used to determine the degree of fulfilment and the values are presented in

Table 5. Assessment of the fulfilment degree of the criteria by each variant.

Criterion Code	Fulfilment Degree of the Criteria [%]
	V1	V2	V3	V5	V6	V8	V9
C1	77	82	85	88	86	67	75
C2	75	79	82	100	69	75	88
C3	75	79	82	100	69	75	88
C4	68	73	70	63	63	67	50
C5	70	63	69	63	50	25	63
C6	66	0	67	0	0	0	0
C7	68	73	70	63	63	67	50
C8	73	80	78	100	69	75	75
C9	0	0	59	38	74	67	0

. As can be seen all the options met the threshold criterion condition. Therefore, none of the applications was rejected.

In the next step the global assessment of each variant was calculated according to the algorithm of the compensation-conjunctive method as the sum of products of weights of particular criteria and the degree of meeting the criterion in the given variant. The results presented in

Table 6. Global assessment of the variant obtained using the compensation-conjunctive method.

Global Assessment of the Variant [%]
V1	V2	V3	V5	V6	V8	V9
68	65	76	74	63	60	62

and the final ranking presented in

Table 7. Comparison of the final application rankings obtained using different methods.

Place in the Ranking	R1	R2	R3	R4 **
1	mobileMPK/V6/21 *	Jakdojade/V3/3.75	Jakdojade/V3/76%	mobileMPK/V6/4.6
2	Jakdojade/V3/20	Kraków pod ręką/V5/3.55	Kraków pod ręką/V5/74%	Kraków pod ręką/V5/4.5 and Moovit/V7/4.5
3	Google Maps/V2/18	E-podróżnik/V1/3.51	E-podróżnik/V1/68%	Jakdojade/V3/4.4
4	Moovit/V7/17	myBus/V8/3.42	Google Maps/V2/65%	Kiedyprzyjedzie.pl/V4/4.1
5	myBus/V8/16	mobileMPK/V6/3.41	mobileMPK/V6/63%	E-podróżnik/V1/4.0
6	E-podróżnik/V1/13	Google Maps/V2/3.31	Transportoid/V9/62%	Google Maps/V2/3.8 and myBus/V8/3.8
7	Transportoid/V9/12	Transportoid/V9/3.09	myBus/V8/60%	Transportoid/V9/2.9
8	Kiedyprzyjedzie.pl/V4/6
9	Kraków pod ręką/V5/5

Explanations: (*) name of variant/code of variant/evaluation of alternative, (**)—data as of 20 December 2020 based on Google Play.

(R3 ranking) show that the best variant in terms of the analysed assessment criteria was V3—(Jakdojade), whose global assessment value amounted to 76%. The second position was occupied by Variant V5 (Krakow pod ręką), which received the global evaluation value of 74%. The difference between these two options was only 2 percentage points. The remaining variants received a global rating below 70% (from 68% to 60%). The lowest global evaluation value of 60% was achieved by variant V8 (myBus), which took the last place in the ranking. The winning variant V3 (Jakdojade) received high values of degree of performance for the criteria for C1 (ease of use, intuitive operation), C2 (mapping a route from point A to point B) and C3 (mapping a route from stop to stop). It should be noted that these criteria were characterised by high values of weights (Table 4). Criterion C9 (information on the actual departure/arrival time of a particular means) received the lowest level of performance for this option. However, this criterion did not significantly affect the final result, as it was characterised by a low value of weight in comparison with the other criteria. For the obtained values of the global evaluation, the basic statistical measures were also determined, i.e., the arithmetic mean which amounted to 66.85, the standard deviation from the sample: 6.12, median 65 and the gap between the highest and lowest score, i.e., between variant V3 and V1, which amounted to 16 percentage points.

Table 7 presents the final rankings of journey planner applications:

• Obtained by evaluating the publicly available information (Section 4.1)—ranking R1 Table 7;
• Using surveys of a group of respondents (Section 4.2.1)—ranking R2 Table 7;
• Using the compensation-conjunctive evaluation method (Section 4.2.2)—ranking R3 Table 7;
• Based on the data reported by Google Play shop (ranking R4 Table 7) as of 20 December 2020.

In the ranking R1, created by the assessment based on publicly available information (Section 4.1), variants V6 (mobileMPK) and V3 (Jakdojade), took the highest position in the ranking. The difference between these options is negligible, so they can be considered to be equivalent. On the other hand, the lowest ranked variants were V4 (Kiedyprzyjedzie.pl) and V5 (Krakow pod ręką), which may also be considered as equivalent. The difference between the best and the worst variant is 16. Analysing the R2 ranking based on the survey results (Section 4.2.1), V3 (Jakdojade) ranked the highest as in R3 and V9 (Transportoid) ranked the lowest.

When analysing the final ranking R4, V6 (mobileMPK) ranked highest as in ranking R1, while V ranked lowest (Transportoid).

5. Discussion

This article discusses the assessment of journey planner applications using one of the multicriteria decision support methods, namely, the compensation-conjunctive method along with a set of criteria proposed by research based on a survey (the criteria indicated by travellers as the most important ones). The assessment carried out in this way allows for a comprehensive analysis of various criteria related to the functioning and utility of the application as well as its user-friendliness. The assessment included such criteria as the ease of use, mapping a route from point A to point B, mapping a route from stop to stop, searching for alternative connections, information on total travel time, ticket purchase options, finding connections with the least number of stops, automatic location detection, information on the actual departure/arrival time of a particular means. The assessment carried out in this way also allowed the identification of the journey planner elements yet to be improved. Such information may turn out to be useful when creating new apps as well as improving the existing journey planner applications. As a result, the ranking of travel planners from the best to the worst in relation to the considered evaluation criteria was obtained. In addition, rankings obtained by evaluating the publicly available information, using surveys of a group of respondents and based on the data reported by Google Play shop were created. All four rankings were then compared with each other. As can be seen, the applications which most frequently ranked in the top three are Krakow at hand (V5) and Jakdojade (V3). V5 was reported by fewest respondents as the one they used for travelling whereas V3 was the most frequently used app. Considering the usability of individual solutions, in the case of the Krakow pod ręką application, it is characterised by great simplicity, which is probably its main advantage. Jakdojade (V3), on the other hand, is a travel planner with a wide range of configuration possibilities, which can be used in many Polish cities. In terms of functionality, the application meets high user requirements. In comparison to other applications, the creators of Jakdojade put a lot of emphasis on marketing and promoting their product, e.g., the coupons included in the application for use when purchasing tickets. In addition, the application is promoted on a number of social networking sites. The creators of the application encourage the users to evaluate the application and to write reviews, which is reflected by a significantly higher number of opinions in comparison to, for example, Krakow pod ręką (V5). Besides, Jakdojade (V3) was downloaded to Android devices over 5 million times, whereas Krakow pod ręką (V5) over 50 thousand times. This is primarily due to the local nature of this solution.

In the case of the mobileMPK application (V6), which received the best score in the first comparison (R1), it leaves much to be desired as far as marketing and customer awareness are concerned. In terms of functionality, the application itself is very well developed; it can be said that it matches the Jakdojade application (V3), but in the case of assessment on the basis of surveys, mobileMPK was rated low. It is worth mentioning that in the survey the most important element of the application was “Ease of use, intuitive use”. This is an important aspect concerning the functionality of the application, which affects the workload required for effective and efficient performance of tasks by the user and largely influences the assessment of the quality of the application.

The authors paid particular attention to the ranking of the R3 application obtained by the compensation-conjunctive method, where the best variant in terms of the analysed evaluation criteria was V3—(Jakdojade), whose global evaluation value amounted to 76%. The second place was taken by Variant V5 (Krakow pod ręką), whose global assessment value was 74%. The high position of these variants in the ranking resulted in the first place from the obtained high degree of compliance with criteria C1 (ease of use, intuitive operation), C2 (mapping a route from point A to point B), C3 (mapping a route from stop to stop) in case of variant V3 and C2 (mapping a route from point A to point B), C3 (mapping a route from stop to stop) and C8 (automatic location detection) in case of variant V5. It should be noted that these criteria were characterised by high weight values. Although the final result of the study was the ranking of the journey planner applications, the individual stages of the analysis provide important information, i.e., the importance of the specific element of the journey planner applications (weighting the evaluation criteria) and to what extent the analysed journey planner applications meet the requirements in terms of the degree of compliance of the variant with specific criteria. In addition, this type of analysis also provides information about the strengths and weaknesses of the analysed journey planner applications which facilitates the identification of the elements to be improved. Regarding the importance of the criteria, the analysis pointed to the fact that the most important criteria were C1 (ease of use, intuitive operation) and C2 (mapping a route from point A to point B). The highest values of the degree of compliance with criterion C1 were obtained by variants V5, V6 and V3 and with the criterion C2 by variants V5, V9 and V3. In order to improve the performance and usability of the evaluated travel planning applications, the main focus should be on improving the criteria (application elements) that received low compliance rate scores. The lowest compliance rate values were observed for criterion C6 (ticket purchase option). Only two variants fulfilled this criterion at the level of over 50% (variant V1 and variant V3). The presented rankings, obtained using different methods and referred to data from Google Play, indicate that despite the general popularity of the Moovit application, travellers in the Kraków agglomeration use local applications more often, as they are more popular in the analysed area. This situation may be due to the low awareness of particular applications resulting from the lack of their proper promotion. In the article, attention should be drawn to the original approach of the authors to the evaluation of the functioning applications. The approach based on the comparison of four rankings resulting from the evaluation of four completely different approaches was not found in previous studies. This approach gives a broader view of the functioning applications for travel planning. It is worth noting that despite using different evaluation methods, the first two places in the rankings are occupied by the Jakdojade, Krakow pod ręką and mobileMPK applications. Therefore, it can be concluded that these applications are best accepted by users and most frequently used in planning local journeys in the Krakow agglomeration. However, some shortcomings can also be identified in this type of approach. One of them is the evaluation against different or partly repeated criteria and a different number of criteria. On the other hand, by analysing the criteria for which an app performs less well than others in each ranking, many of the shortcomings and inadequacies in the apps can be identified, which provides direction for those involved in improving the existing apps and developing new ones.

Mobile applications have enormously facilitated the process of getting around the city. Their general availability and ease of use encourage growing numbers of people to use them and, consequently, to use public transport. Mobile applications should be useful products. Many researchers stress that in order to achieve this, it is important to know the needs and preferences of their users [18,33]. There is an increasing focus on the appearance of the interface itself, the user-friendliness and the relevance of the information provided to users. The interfaces should help users to obtain information intuitively, without much concentration or memory involvement. Based on the presented results of the surveys, in which the individual criteria for evaluating applications were evaluated in terms of importance for the user, the authors of this article show exactly which travel planner applications users expect. For the evaluation of the travel planner applications, a set of nine criteria were used that received the highest values of importance of the criteria from the surveys. Some of these criteria were also indicated as important by researchers working on similar topics. Esztergár-Kiss [36] and Esztergár-Kiss and Csiszár [35,37], as well as the authors of this article applied a multi-criteria evaluation of the applications against the criteria including the aspects of route planning services, booking and payment, data handling, and supplementary information. Zhang et al. [51], however, described a framework of a personalised multimodal traveller information system, which includes real-time information, personal preferences, and multimodal route planning. Gohar and Nencioni [22], in their research, show the need to develop user-friendly applications, to create smart cities with a strong focus on connecting individual vehicles through the development of cooperative intelligent transport systems, which can make cities smart and the automated transport systems safer and more efficient than the existing transport networks. This also helps the public transport system to cope with key transport problems in large cities, including congestion, pollution and collisions. Based on the above discussion, it may be concluded that the authors of this paper have raised important issues concerning travel planners. In particular, it is important to note the issues of the evaluation criteria that show the strengths and weaknesses of the evaluated applications. In this paper, the authors have shown that global-level travel planning applications are not always popular at the local or national level, as shown by the results presented in this paper. So far, the evaluation of travel planner applications for stakeholders at the local level (cities, agglomerations) has been carried out extremely rarely. On the other hand, the juxtaposition of local solutions (developed and used, e.g., in one country or city) with international applications is, according to the authors, an original approach to the multi-criteria evaluation of journey planners.

6. Conclusions

This paper deals with the issue of adapting journey planners to the needs of their users. The method applied in the research differs from the methods used in other studies. MCDA was used for reviewing several journey planner applications against various criteria and developing final rankings on that basis. The application of the compensation-conjunctive method provided for compensation on non-compliance with one criterion with a high compliance level with another. The unique nature of this study involved creating and comparing four rankings, based on the evaluation of four different approaches. At the same time, the authors compared the applications used locally, nationally as well as those used abroad.

The available applications supporting urban mobility by public transport should be easily accessible and adapted to all traffic users, including people with reduced mobility. Although equipped with many functions, the existing apps have a number of shortcomings. Many of them offer the option to select a means of transport adapted to people with disabilities, but not all of them display, for example, routes to and from bus stops or interchanges adapted to the needs of people with reduced mobility. For many public transport users, it is important that public transport travel apps also include other facilities such as information on public toilets, information on ramps or lifts to facilitate movement at interchanges. Further development of travel support applications in the Krakow area is necessary to achieve the assumed 75% optimal share of soft-travel and public transport in Krakow [24]. Consequently, it will be possible to reduce transport energy consumption. Taking into account the different evaluation methods, it may be concluded that the potential passenger should have access to solutions with varied usability. Each city should offer both simple and easy-to-use applications allowing travellers to obtain basic information about the planned journeys as well as those urban planners that, besides the basic elements, also provide additional functionalities. In addition, it is recommended that journey planners also include:

• Information on electric buses;
• Information on energy consumption of travel;
• Information on CO₂ emissions;
• Trip planning options using the means that use most renewable energy;
• Information about the dependence of the ticket price, and consequently the route, on the environmental performance of the means of transport.

The results of the research on the importance of different criteria can provide valuable suggestions for policymakers and those who develop journey planners as this information should help them to understand user expectations and predict the most important factors contributing to user satisfaction. Ultimately, this should make it easier to adapt the application to the changing expectations of travel planners.

In order to deepen the analyses related to the assessment of the applications, it is proposed to carry out a sensitivity analysis in further studies, which will allow us to determine the impact of the change in the importance of individual elements on the obtained results (final ranking). The starting point for such analysis may be a change in the values of individual evaluation criteria and observation of how the final rankings change. Moreover, in order to deepen the analysis, the evaluation of applications is suggested from, for instance:

• People with reduced mobility;
• The city authorities;
• Groups of people whose numbers have suddenly increased in a given area (e.g., due to migration of people from war zones).

In conclusion, the passenger-friendly applications play a significant role in increasing the attractiveness and competitiveness of public transport and they may serve as the showcase for the local transport system.