Beyond Limitations of Current Behaviour Change Apps for Sustainable Mobility: Insights from a User-Centered Design and Evaluation Process

1

Papeer   is a good way of evaluating and a survey of current apps.  However the   conclusion is crystal clear: you are presentencing a commercial one, GoECO.   You must rewrite the conclusions as bullets, one per each claim with respect   to the state of the art.

We thank the reviewer for this suggestion.

We reworked the Conclusions section, in order to better relate it to   the twofold focus of the paper (designing    a persuasive  app  overcoming    limitations usually characterizing current persuasive apps, particularly   the lack of grounding in a behaviour change theory, and the over-reliance on   "one size fits all" point-based reward systems; testing the app in   real-life settings and identifying additional shortcomings and   recommendations for future works, from the perspective of its direct users)   and to offer more general value recommendations.

We hope we managed to better highlight the lessons we can draw from   the project experience, to the benefit of possible future persuasive apps in   the field of mobility. In particular, we hope we more clearly highlighted our   suggestions to better meet the recommendations for effective persuasive apps,   that had emerged during the literature review (summarized in the “Introduction”   section) and that we referred to throughout the “Discussion” section.

Line 687 – entire section rewritten

In  this  paper    we  presented  an    attempt  to  overcome    the  main  limitations    that  were  found    to affect behaviour change apps targeting more sustainable individual   mobility patterns.  Particularly,   we  focused  on    the  lack  of    grounding  in  a    behaviour  change  theory    and  on  the    over-reliance  on   one-size-fits-all, point-based reward mechanics, that typically occurs when   gamification approaches are exploited.    To this purpose, we developed an app, named GoEco!, and opted for grounding it in the Transtheoretical Model   for behaviour change.

Instead of primarily relying on a standardized point-based rewarding   system, the app’s features and components were designed with the specific aim   of assisting individual progress from one stage of the behaviour change   process to the next one, through activation of the proper process(es) of   change at each stage. Namely,    specific  app  features    were  designed  for    each  stage,  from pre-contemplation   of  behaviour  change    to maintenance of  a    new,  more  sustainable,  behaviour,    and,  to  increase    their effectiveness, a user-centered approach was adopted.

The app effectiveness was then field-tested by 47 voluntary users in a   three-month experiment in two Swiss regions.    After the experiment, we ran a survey and interviews to get insights   on the app’s features and components, directly from its users. In   general,  the users expressed the   desire for further customisation    (regarding  potential  for    change,  available  alternatives,  goals,    challenges and notifications),    as well as further simplification concerning the number of offered   features. Based on this experience, we therefore recommend that future   persuasive apps better follow the process of behaviour change, by releasing   the app features in stages as well:  at   registration, a simple question such as the one proposed by Bamberg [79]   could allow to classify the users’ behaviour change initial stage.  Then, users could only be offered the app   features specifically designed for their stage.  For example, a user in the pre-contemplation stage should only   receive feedback on each travelled route and on her baseline mobility   patterns, while a user in the action   phase should have all the app features enabled (in both cases with a proper   on-boarding training period).  Then,   the classifying question should be periodically repeated, and new features   should be released when progress to the next stage is detected.

More specifically, this experience provides us with practical   suggestions to further improve the recommendations for effective persuasive   apps that were identified in previous works:

·           Provide  information:    improve  the automatic  detection    capability  of  both    travelled  routes  and transport  modes,    in  order  to    reduce  the  need    for  manual  validation    of  the  transport    mode by  the  users.     Also  endow  the    app  with  a    multi-modal  travel  planning    system,  capable  of actively supporting users in their daily   travel needs, especially for non-systematic trips. Such a component would not   only provide them with additional practical information useful in their   process of change, but would also increase the frequency of their interaction   with the app, thus helping them to remain committed to their goals and   challenges;

·           Provide goal setting opportunities:    allow for as much customisation and dynamism as possible, in  both    goals  and  challenges,    by  developing  a    timely  notification  system    based  on  user performances and by explicitly   referring to specific mobility patterns of other app users in the design of   goals and challenges themselves;

·           Provide feedback:    make individual impact more intuitive and immediately understandable   (for example, by exploiting visualization techniques instead of purely   numerical values), as well as more diversified (for instance, by offering   information on health and monetary impacts, besides the current energy and   climate ones).

·           Provide rewards or punishments:    provide tangible rewards,  both   at the individual level (mainly targeting individuals in the pre-contemplation stage, as a lure to   raise their interest in the app, and thus to activate their process of   change), and at the community level (to keep the interest of those who are   already in the next stages of behaviour change);

·           Provide occasions for social   comparison:  better exploit the power of social   interactions.  Instead of limiting  comparisons    to  behavioral  aspects,    which  is  difficult    to  perform  in    a  fair  way    and risks not to be trusted,    let the app include features and components aimed at increasing the perception   of social support and the feeling of belonging to a community of similar   people, engaged together towards the same goal for change. Namely, move the   focus from a competitive setting to helping relationships. To this purpose,   include features allowing users to share activities and performances, such as   for example comments, questions, travelled routes, average mobility patterns,   challenges, and so on.

Even  though  these    suggestions  entirely  originate    from  a  process    developed  in  Switzerland, one  of    the  top-three  countries    worldwide  according  to    the  Human  Development    Index,  we believe  they    are  only  marginally    influenced  by  the    Swiss  high  standards    of  living. In  lower income  countries    we  expect  even    higher  interest  for    tangible  rewards  at    the  individual  level,    and suppose recommendations on the other aspects would maintain their   effectiveness.  A fundamental   prerequisite for the effectiveness of a persuasive app such as GoEco!, however, is the accessibility   to realistic alternatives to individual car use, such as timely and frequent   public transport and safe and widespread cycling and walking paths. In lower   income countries, where these options might be lacking, this would be a major   constraint precluding effectiveness of similar persuasive approaches, no   matter whether the above recommendations have been put into practice.

Implementing  all  the    above  suggestions  will    require  further  research    efforts  in  different disciplines,  from    artificial  intelligence  for    automatic  transport  mode    detection,  travel  planning information  and    better  customisation  (alternatives,  potentials,    notification,  etc.),  to    user  interface design  (visualization  of    the  impact),  and    social  and  behavioural    sciences  (improvement  of    social support and a feeling of community). The route for effective   persuasive apps in the mobility sector is therefore still long. We hope   however, with this work, to have identified in which direction the steps   forward need to be taken.

2

You   must consider this idea along all paper.

The “Results” section presents the app we developed to address the   project’s research goals, showing how we designed the app’s features and   components in order to comply with the Transtheoretical model for behaviour   change – namely, this section deals with the first key limitation emerging   from literature review (the lack of grounding on a behaviour change theory).   Such a section also describes the design choices we made in order to avoid a   point-based, one-size-fits-all persuasive system – namely, it also shows how   we addressed the second key limitation emerging from literature review. 

Finally, the whole “Discussion” section is already organized with the   aim of discussing the users’ viewpoints, with respect to the “Recommendations   for persuasive apps” that emerged from literature analysis and were summarized   in the “Introduction” section.

Therefore, unless we   misunderstood this suggestion by the reviewer, we believe the current structure   of the manuscript already shows and discusses how the GoEco! app addressed the claims emerging from the review of the   state of the art, and did not introduce specific revisions. 

---

3

State   of the art: it is worthy of reading, congratulations.

We thank the reviewer for this general appraisal of our work.

---

4

In   short, make clearer what is new knowledge and what is an apps.

We thank the reviewer for this suggestion. As indicate above, we reworked   the “Conclusions” section in order to better highlight how the project   outcomes and the recommendations for persuasive apps could be beneficial for   future similar works.

Please, see changes already reported for comment #1

1

The authors present a very   interesting research aimed at evaluating the potential of behaviour change   apps for sustainable mobility. Through the development and testing of an   actual app they evaluated the effectiveness of multiple research goals thanks   to the involvement of test users. The paper is well written, the methodology   is described with accuracy and multiple results are presented, supported by   the opinion of the users.

We thank the reviewer for this general appraisal of our work.

---

2

The only aspects that I found   lacking, although not at the center of the research questions, is a   description of the quantitative parameters used for the evaluation of the   impacts in terms of energy consumption and CO₂ emissions of   different travel modes.

We introduced a description of the emissions and consumption factors   we used to provide app users with feedback on their energy and climate   impact.

Line 259 – new paragraph added

The   impacts are expressed in terms of primary energy consumption (kWh) and CO₂   equivalent

emissions   and are based on the Mobitool consumption and emission factors [89], which   depend on the mode of transport, refer to a single kilometer traveled in   Switzerland and take into account the consumption and emissions of the full   life-cycle, by consider an average vehicle occupancy. The specific values  are    presented  in  [88].  For    electricity-fuelled    transport  modes,  such    as  trains,  these    factors heavily depend on the involved power generation systems (e.g.,   power from renewable sources leads to much fewer CO₂ emissions   than power generated by fossil-fuel power plants).   As such,    these factors and the resulting energy consumption and CO₂

emission   values are specific to Switzerland. In order to apply the framework to other   countries, their power generation systems have to be analysed and taken into   account accordingly.

Regarding  cars,    the  Mobitool  values    reflect  the  powertrain    composition  of  the    Swiss  vehicle fleet  of    passenger  cars. To get a   meaningful feedback, GoEco! app  users    can  enter the average fuel consumption   value of their car (expressed in fuel liters per 100 kilometers), otherwise GoEco! automatically considers the   Mobitool default values.

3

Moreover, it should be described   if different types of energy have been weighted differently or not (e.g.   electricity and gasoline, which require the use of very different specific   primary energy consumption). The numbers that have been used, together with   the sources should be presented to give to the readers a complete figure,   giving the technical background of most readers of sustainability.

We introduced a description of the consumption and emission factors we   relied on. Since in another publication, clearly referenced in the   manuscript, we already presented their specific values, here we opted for not   including such values. However, we added a general comment highlighting their   limitations and how to interpret them. We hope therefore this meets the   expectations of the reviewer. 

4

Moreover, with reference to the   previous remark, some further aspects may be included in future development   of similar apps: the choice of private cars based on alternative technologies   (including electric or hybrid vehicles) may have a strong effect on the   travel impact. Another crucial aspect for the users may be the positive   effect of sharing car trips with other passengers, instead of driving a   private car alone. This possibility should be included to provide to the   users this additional information, with a consequent effect on their impacts.

We added two comments, about the difficulty of automatic detection

-            of the   use of an electric vehicle;

-            of the   number of people travelling in the same car (carpooling/ridesharing).

We also highlighted if and how we managed to address these issues in GoEco! and we hope this meets the   reviewer’s expectations.

Line 272 – new paragraph added

Electric   cars are not automatically identified by the system, since GoEco! has no elements to distinguish   them from internal combustion engine cars; however, whenever anyone uses an   electric car, they can manually select it when validating the transport mode,   therefore GoEco! can account for   the related energy consumption and CO₂ emission factors.

Line 276 – new paragraph added

Equally,  the systems is not capable of automatically   detecting the number of users travelling in    the  same  car,    therefore  ridesharing  or    car-pooling  routes  cannot    be  accounted  for    by  the  app.

Acknowledging such limitations, therefore,   the feedback on one’s own mobility impact is meant as a reference, for   comparison with the other transport modes.

5

Finally, it should be briefly   discussed whether the authors believe that a similar application in other   countries may have different outcomes, i.e. the potential biases and   limitations of using a very high income country for such an analysis.

This comment was addressed together with the following one, since they   are in close connection.

We thank the reviewer for highlighting this aspect, which we shortly   referred to in the “Introduction” and then addressed both in Section 4.4   “Providing rewards or punishments” and in the “Conclusions”.

Line 128 – better specification introduced

We conclude by summarizing key suggestions and recommendations for   future apps aimed at persuading more sustainable  mobility    patterns  in  similar,  wealthy    countries,  and  commenting    on  remaining  open research challenges (Section 5).

Line 581 – new paragraph added

Also,  we are aware that such a   low interest for individual tangible prizes could be due to the fact that   Switzerland, ranked second worldwide in the Human Development Index in 2018   [99], is a wealthy country, where on average people enjoy very high standards   of living. Previous research at the international level  summarized  by  [100]  reports    in  fact  that    building  a  long-term    commitment  between  an    ICT-based system and its users requires a proper economic reward.  Therefore, in future research activities   one could exploit individual, tangible prizes, to start engaging individuals   with low intrinsic motivation towards change (individuals in the pre-contemplation   stage), and gradually replace prizes by other motivational factors, as long   as the individuals progress through the stages of behaviour change.

Line 743 – new paragraph added

Even though these suggestions entirely originate from a process   developed in Switzerland, one of the top-three countries worldwide according   to the Human Development Index, we believe they are only marginally   influenced by the Swiss high standards of living. In lower income countries   we expect even higher interest for tangible rewards at the individual level,   and suppose recommendations on the other aspects would maintain their   effectiveness. A fundamental prerequisite for the effectiveness of a   persuasive app such as GoEco!,   however, is the accessibility to realistic alternatives to individual car   use, such as timely and frequent public transport and safe and widespread   cycling and walking paths. In lower income countries, where these options   might be lacking, this would be a major constraint precluding effectiveness   of similar persuasive approaches, no matter whether the above recommendations   have been put into practice.

6

Lines 534-536: Do you think that   other countries with lower well-being than Switzerland could have a different   attitude towards monetary prizes?

7

Lines 166-168: Are there potential   biases in this selection of voluntary individuals? They should be briefly   discussed.

We thank the reviewer for highlighting this problem. We addressed it   both in the “Methodology” section and in the “Discussion” section, by adding   new material.

Line 175  - new paragraph added

Such self-selection  recruitment  processes    are  typically  affected    by  a volunteer selection   bias  [81,82], which in turn might   affect the possibility to generalize the project results. It is in fact   well-known that opt-in frameworks such as the one adopted in GoEco! tend to raise the interest of   already motivated subgroups of the general population, thus tending to engage   individuals with higher than average environmental awareness and   pro-environmental attitude. Nonetheless, in interventions  such    as GoEco!,  which    require  active  and    conscious  engagement  with    a  smartphone app, no   obligations can be put into force and no opt-out strategies can be   implemented. Therefore, a self-selection of participants can barely be   avoided and a related bias has to be taken into account.

Line 429 - new paragraph added

According to their answers to the questionnaire, reported in Table 3,   they have a medium to high pro-environmental attitude, and also share the   feeling of a personal responsibility to control pollution and climate change.   Since no direct comparison is    possible  with  corresponding figures for the average   population, we cannot state whether these figures indicate the presence of a   self-selection bias. However, even in case it occurs, this does not disallow   investigating the user’s opinions and viewpoints with respect to the app’s   features and contents.

Line 437 -  new Table added   (Table 3)

8

Lines 173-174: Which are the main   reasons for this drop-out? Was it all on the beginning or at different stages   of the test?

We thank the reviewer for this comment. We addressed it in the   “Methodology” section.

Line 183 - new paragraph added

Similar  tendencies  might    also  have occurred  throughout    the  app  field    test,  with  higher    drop-out rates by individuals with lower environmental awareness and attitude.   A questionnaire targeting all initial project participants attributed to the   treatment group (145 individuals) showed that the two major reasons for   leaving the project were related to the need for confirmation of the mobility   data automatically  tracked  (validation, see Section 3.1)  and    technical  problems  precluding,    from  time  to    time,  the  app from fully working.

9

Line 234: It is not clear if these   algorithms are able to detect a specific transport mode (and if yes, it   should be described in more detail the method they use), or if the app relies   on the definition of the transport mode by each user.

The algorithms we developed on purpose for the automatic   identification of the transport mode are presented in details in another   publication [Bucher et al., 2019]. Here therefore we opted for providing a basic   introduction to such elements and giving more visibility to the reference to   our already published work.

Line 244 -  minor revision   introduced

To provide such feedback, GoEco!   tracks individuals’ mobility data. For this purpose, it exploits the   Application Programming Interface (API) of the commercial, free fitness   tracker app Moves [85] (discontinued since July 2018), that records users’   positions, segments travelled paths into "routes" and   "activities", and automatically determines   whether they are walking, running, cycling, or taking another mode of   "transport". Building on this information, new algorithms developed   specifically for GoEco! further   classify the generic   "transport" activities identified by Moves, so that also bus, train   and tram mode activities are automatically detected.

Line 250 – new paragraph added

To produce the needed fine-grained distinction between these different   transport modes, a classifier based on a naïve Bayes algorithm [87] was built,   which takes into account several route characteristics, such as travel speed,   acceleration, or spatio-temporal dependencies between visited points and the   public transport network (stops and lines). The whole algorithm is presented   in details in [88].  As Moves data gets updated at unknown points   in time, GoEco! is not capable of   providing real-time feedback. Instead, users are encouraged to interact with   it once a day, with the purpose of checking and validating the automatically detected transport mode for every   activity tracked on the previous day (Figure 2-a,b).  While performing this validation, they   receive the feedback on both transport-related indicators and related   impacts.