Next Article in Journal
Numerical Study of a Francis Turbine over Wide Operating Range: Some Practical Aspects of Verification
Previous Article in Journal
Local Perceptions of Fires Risk and Policy Implications in the Hills of Valparaíso, Chile
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 12
Issue 10
10.3390/su12104299
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Alternative Metrics for Assessing the Social Impact of Tourism Research

by
Alba Viana Lora
* and
Marta Gemma Nel-lo Andreu
Department of Geography, Rovira i Virgili University, 43480 Vila-seca, Spain
*
Author to whom correspondence should be addressed.
Sustainability 2020, 12(10), 4299; https://doi.org/10.3390/su12104299
Submission received: 22 April 2020 / Revised: 8 May 2020 / Accepted: 18 May 2020 / Published: 25 May 2020
(This article belongs to the Section Economic and Business Aspects of Sustainability)
Browse Figures
Versions Notes

Abstract

:
Alternative metrics are increasingly used to measure the social impact of research. This article seeks to analyze the social impact of research in the field of tourism. For this purpose, we will determine the extent to which the articles in this field reach society by examining the scores they achieve on social media and studying the correlation between scientific impact and social impact. Altmetric information will be used for data extraction and analysis. The results show a low correlation between citations and the Altmetric Attention Score (AAS), as well as a presence that is not captured by most publications in the field of study. Interestingly, publications with higher AASs are concentrated in the same journals. The article concludes by determining that alternative metrics can be used to complement academic impact but cannot be a substitute for it. Further progress is needed in the development of a framework that unifies both impacts.

1. Introduction

Social impact is gaining strength in funding calls and in research in general [1,2,3]. This is due, on the one hand, to the fact that there are complex social problems that require science to be understood [4]. On the other hand, it is due to the fact that entities that finance science seek results that show evidence, visualizing research as an investment that should benefit society [5]. This is a way to improve the relevance of research [6]. There are calls for proposals that already include a section on social impact, as is the case when dealing with vulnerable populations, and benefits must be demonstrated in order to obtain funding [7].
The aim of strengthening the social impact of research is to take into account not only scientific articles but also the changes and improvements that can be produced in society [8], going beyond academic impact. However, it is a reality that researchers have to demonstrate high scientific impact. Fulfilling both objectives becomes complex, since sometimes, the fulfilment of one implies the renunciation of the other. This occurs when you decide to publish your study in a journal in order to reach a wider part of society, thereby renouncing publication in a high impact journal. These objectives also come into conflict when the needs to collaborate for common benefit or competition for funding are considered [9].
Social impact is such a complex phenomenon that prediction would be impossible without a framework analysis to help organize and focus research [10]. However, evaluation is complex because the systems are either non-existent or very costly [4] and are mainly based on conventional methods such as local surveys, secondary data analysis or key informant interviews [11]. For this reason, it is necessary to continue advancing and developing methodologies that are able to measure them. Approaches that combine scientific and social impact assessment under one applicable framework are needed. This study aims to test the Altmetric tool to determine whether it can be used as an additional methodology or an instrument to be incorporated into social impact assessment. Altmetric collects and analyses information from scientific publications on social media and creates a score based on the scope of the research [12].
This article is original because there is no study that seeks to measure social impact using the Altmetric tool in the field of tourism. Therefore, this article aims to carry out analysis with alternative metrics that measure the social impact of research in tourism and to answer the following research questions (RQ):
RQ1: What is the presence rate of the tourism articles with the greatest impact (Journal Citation Reports (JCR) Q1 and Scopus Q1) in Altmetric? What is the status of the Altmetric Attention Score for tourism items with the greatest impact (JCR Q1 and Scopus Q1)?
RQ2: Is there a significant statistical relationship between the citation index of the tourism items with the greatest impact (JCR Q1 and Scopus Q1) and the Altmetric Attention Score?
RQ3: Which articles and journals in the field of tourism have the highest Altmetric Attention Score?
RQ4: Could the impact measured by Altmetric complement or replace traditional bibliometrics in the future?
The document will be organized as follows. Section 2 contains a review of the literature on the social impact of research and alternative metrics to scientific impact. Section 3 explains the methodology and methods used to extract information. The data analysis and results are shown in Section 4. Section 5 presents the study′s conclusions and future research.

2. Literature Review

2.1. Social Impact of Tourism Research

The social impact of research is understood as the changes in behavior that have occurred due to specific research [13]. These behavioral changes, which are consequences experienced by the population, are caused by public or private actions that alter their daily lives [14]. For Vanclay [15,16], social impact reflects the changes experienced by humans within their ways of life, cultures, communities, political systems, environments, health and well-being, personal and property rights, and fears and aspirations.
Flecha [17] determines that social impact occurs when scientific knowledge that has been produced, published and transferred to society and its institutions has a positive effect on change. If this knowledge is not shared, despite the fact that we are certain the results would have an impact, we would be talking about potential social impact [18].
The definition of the social impact of research provided by Reale et al. [19] relates to the translation of research results into policies that produce improvements in society. This conceptualization provokes a reflection on the relationship of impact and the application of research to problems previously defined by political actors. However, science must take into account real social problems when proposing research, avoiding being influenced by political interests [20] and seeking commitment to social improvement as a basis for research.
A new model for an interactive approach based on the relationship between researchers and stakeholders has been developed in response to this need, changing the traditional model in which science was only a source of knowledge flowing into society [21]. This model encourages the participation of all stakeholders, requiring them to recognize, value and raise their interests while their involvement is necessary [22]. This interaction, at the same time, is a way of promoting the social impact of research [23], since social problems are considered in advance. It is a way of visualizing what we want to change in the system in order to alter it [24].
There are research projects that incorporate tools designed to involve society in science. An example is RRI (Responsible Research and Innovation), which engages local stakeholders and researchers in a constructive discussion that helps to identify responsible strategies and improve the impact of research [25].
The participation of all agents involved is fundamental for sustainable development. One option is to propose research in compliance with one of the 17 United Nations Sustainable Development Goals (SDGs). In this way, science translates these goals into practical agendas while promoting sustainability. The multidisciplinary approach to sustainability requires the merging of different disciplines to solve society′s problems [26]. In 2002, the United Nations Environment Programme (UNEP) underscored the need to emphasize social impacts [27], since the social factor is one of the pillars of sustainable development [28].
Studying the social impact of research in tourism deserves special attention because it is one of the most important sectors in the world. According to the World Tourism Organization (UNWTO) [29], tourism contributes to the achievement of the SDGs through various actions such as reinvesting profits in health care, investing in education and integrating women into the labor market. The social benefits produced by tourism, such as preserving local culture or restoring historical buildings, increasing parks and recreational areas and improving transport infrastructure and other public facilities [30], are another reason for promoting research in this field.
The scientific impact of this area has a recognized historical evolution and has been gaining importance over the years. This impact, as measured by bibliometrics, has traditionally been based on publication citations to measure quality [31]. In the JCR 2018 ranking, the category Hospitality, Leisure, Sport and Tourism ranked 98th out of 236 by average impact factor [32]. The social impact of tourism research, however, has not received the same interest as the scientific impact. This is due to the consideration of scientific impact as the only useful tool for measuring research, since it was assumed that higher-level research could offer greater benefit to society [33].

2.2. Alternative Metrics and Their Relationship with Social Impact

The digital revolution is an opportunity in the field of research, allowing researchers to participate and share their knowledge about studies and results [34], while enabling people to access science more easily.
Social networks have become one of the main communication channels for involving different stakeholders [35]. All the actors involved in the process can access the network and interact to develop valuable knowledge for society, improving the probability of producing social impact [36]. Social networks provide a critical number of data that enable the creation of solid databases for researchers [37] and increase the scope of scientific production beyond the academic community [38]; it is a way to increase public attention to science.
The publication of results on networks allows the rapid dissemination of research and can call attention to other academics who generate knowledge on the subject [39] or detect social problems that need broader research for their solution. Thanks to this digital evolution, alternative metrics to the impact factor were developed, known as “altmetrics”, which are based on the number of times an article is mentioned, commented on and/or shared on the Web [40]. This is a new way of measuring the social impact of research, where it follows that an article that has received a lot of attention online may have produced an impact on society [41]. Social impact assessment will, therefore, be based on the analysis of interactions between social media and academia [42].
Pulido et al. [43] demonstrates the visibility of the social impact of the research on well-being using social networks by analyzing 1402 tweets and 157 Facebook posts from 10 well-being projects.
Among the alternative metrics we find the Altmetric tool, which has been used in this article because the data it provides allow us to evaluate which documents have an impact on society. In this way, Altmetric plays an important role in processing data to evaluate the social impact of research [44]. In Altmetric, data are measured in real time and instantly after publication it is open, fast, and applied to non-traditional formats and multiple sources and is easy to use [45]. Unlike other social network analysis tools, Altmetric cleans and standardizes the data collected by extracting a maximum of one citation per person/source [46]. Analyzing only Facebook and Twitter can be considered weak as we cannot prove the relationship of influence in a heavily commented publication; extending the analysis to the content of blogs and media, as is done by Altmetric, makes the study more consistent [47].
Quantitative metrics are gaining weight in the evaluation of the social impact of research due to the easy collection, the transparency of the analysis process and the fact that the information extracted can be quickly verified and used for comparative studies [39]. There are some quantitative indicators that have been created to measure this impact, such as those designed by the European Commission to monitor and evaluate the research they fund, some of which are quotes or mentions in the midst of debate, the number of participants in public conferences or the new policies developed [48]. Horizon 2020 also includes start-ups, prototypes or patents created within these indicators [49]. Smith [50] described some indicators, such as citations of scientific publications in policy documents, textbooks or teaching materials, which could be used for assessing the social impact of research.
To measure the social impact of the research on social media, the indicator developed by Altmetric can be used, based on the Altmetric Attention Score (AAS), which is calculated with an automatic algorithm that considers the volume, source and author of information extracted from Twitter, Facebook, Google+, policy documents, mainstream media, blogs, Mendeley, CiteULike, PubPeer, Publons, Reddit, Wikipedia, sites running Stack Exchange (Q&A), reviews on F1000, and YouTube. The Altmetric graphic, shown in Figure 1, is represented by a colored doughnut with the weighted score in the center. Each color reflects the source of information used [12]. There are several ways to access Altmetric information. In this study, we used the Dimensions database, which is free to access on request for researchers. Both tools are owned by the same company, Digital Science.
Recent studies see Altmetric as a new tool for assessing the social impact of research; Sedighi′s study [51] measures the social impact of research in 1738 articles in two scientometric journals over five years. Its low correlation index with respect to scientific impact determines the use of this tool as a complementary one, a conclusion also adopted by Garcovich and Adobes Martin [43], after demonstrating the low correlation between the JCR citation and the AAS in a total of 1080 publications from four journals in the field of pediatric dentistry from 2014 to 2017. It does allow a better view of the possible social impact of research results on society.
Bornmann et al. [52] analyzed case studies of the Research Excellence Framework (REF), the body responsible for demonstrating the non-academic impact of UK research, using Altmetric. They compared the AAS with traditional citations from two types of publications: those sent as case study results and those that have referenced those case studies. The correlation was close to zero or negative in the case studies that present social impact according to the REF, so this article highlights the divergence in the construction of social impact that both systems have.
Dardas et al. [53] established the 100 articles indexed in the Journal Citation Report in the nursing category with the highest activity and discussion in the networks using the AAS. In this study, it was stated that there was no representative difference between the Altmetric scores of articles catalogued in journals of different quartiles, unlike in the analysis performed by Delli et al. [46] of the dental field, in which the AAS was significantly higher in articles from Q2 journals. In this same field of research, Kolahi and Khazaei [54] used Altmetric to present the 50 articles with the highest AAS. Kolahi and Khazaei [55] also evaluated the social impact of articles on the topic in PubMed and collected publications from 150 different journals to determine the journal with the highest number of impact publications, the British Dental Journal, stressing the importance of giving visibility to this type of tool.
Cho [56] analyzed the social impact of Korean research in all fields using Altmetric in 383 articles. She compared it with bibliometrics to highlight a positive correlation between the articles stored in Mendeley and the citations. As research fields, he highlighted the medical sciences with the most presence on Twitter and the social sciences in Mendeley.
As can be seen, there are fields of research that have received numerous studies regarding alternative metrics and the use of Altmetric, as in the case of the dental field. In Tourism, this analysis is non-existent, which is why this article will analyze data extracted with Altmetric to measure the social impact of research in this field. The following section explains the methodology used and the process of data extraction.

3. Materials and Methods

This article presents applied research using the Dimensions database (app.dimensions.ai), which gathers all of the information from Altmetric. The study sample deals with articles classified within the tourism research category over three years (2017–2019). To extract them, we first accessed Dimensions and filtered by year and by research category and then we downloaded all available publications on 18 March 2020. These was filtered by research category and not by tourism journals to avoid excluding articles that deal directly or indirectly with tourism but have been published in journals catalogued in other fields of study. This tool gathers different information from each publication such as the title, DOI, name of the journal, date of publication, volume, number, pages, type of publication, authors, affiliation, citations and AAS and other alternative data. We used this information to design the database in Excel. Filtering was performed again to extract information from the articles with the highest scientific impact. For this purpose, the journal of the first quartile of journals of JCR and Scopus were selected using the Spanish Foundation for Science and Technology (FECYT) database and the Scimago Journal and Country Rank (SJR) for each year of reference. An analysis of the extracted data was performed and is explained in the following section.

4. Results

This section contains findings of the analysis carried out. The extraction of the publications catalogued in the tourism field during the three years of study shows a total of 16,453 publications, but for this research, we selected those with the highest scientific impact. For this purpose, the publications framed in JCR Q1 and Scopus Q1 journals were selected: a total of 5307 publications distributed in 65 journals. There were 2123 publications shared at least once on social networks over the three years.
Answering the first question of the research, the presence of tourism articles with the highest scientific impact, catalogued in JCR Q1 and Scopus Q1 journals, during the three years is 40%. In 2018, the highest proportion of articles mentioned at least once on social networks is found to be 45.14%, compared to 43.74% in 2017 and 36.49% in 2019 (Table 1). In absolute terms, we see an increase in the number of tourism publications every year.
The average number of citations shows similar results for all articles that have the Altmetric metric and those that do not. Articles with the AAS have averages of 2, 8 and 15 citations per year compared to the averages of 2, 7 and 13 citations per year (2017–2019) for articles without Altmetrics. This result shows that there is no significant difference in citation capacity between articles with and without Altmetric (Table 1).
To continue answering question one, an analysis of the AAS was performed. The maximum score obtained for a tourism article is 707 points in 2019, 381 points in 2018 and 423 points in 2017. The average score is around 8–13 points per article in the period analyzed, but its standard deviation shows the dispersion of the values spread over the score range. The mode calculation shows 1 as the most repeated score value for the three years. The median is 2 for the three years, and therefore, half of the articles are between 1 and 2 of the AAS (Table 2).
This article presents a correlation analysis to determine whether there is a relationship between the number of citations and the AAS in tourism research, using data extracted using Dimensions and answering question three. The correlation coefficient, although positive in all three years, shows data very close to zero. In 2019, it is 0.1027; in 2018, it is 0.0180; and for 2017, it is 0.0326. To better understand this information, Figure 2 shows the dispersion of these data. The blue, red and green colors show the citations and AASs in 2019, 2018 and 2017, respectively.
To answer question four of the research, Table 3 shows the ten research articles in tourism with the highest social impact according to the AAS, and Table 4 shows the ten journals with the highest AASs in tourism publications. The article in first place, "Vacation Posts on Facebook: A Model for Incidental Vicarious Travel Consumption", has a score of 707 points with only seven citations. On the other hand, the article in last place, with a score of 297 points, is the most cited in this top ten, with 51 citations.
The study includes tourism publications from 65 journals classified within JCR Q1 and Scopus Q1. As mentioned above, the filtering was done using tourism as the field of research, which leaves 29 journals that are within the category of tourism in the JCR and Scopus rankings and 36 journals that belong to other categories such as Marketing. Analyzing the publications with an AAS, one journal in the category of tourism and 13 in other categories disappear; that is, there are 14 journals of high academic impact whose tourism publications have not presented any social impact in social media. Of the 51 total journals that present an AAS in some articles, Table 4 includes the ten with the highest score. The ten articles in Table 3 are concentrated in six journals, which also appear within the ten journals with the highest AASs.
The provenance of the authors of tourism publications with AASs is another interesting aspect to analyze because it can be linked to the effort of governments to reflect the social impact of funded research. In fact, the three countries highlighted in this study are countries with programs that seek evidence of the social impact of research. This is the case with the National Science Foundation in the United States, the Research Excellence Framework (REF) in the United Kingdom and the Engagement and Impact Assessment (EI) in Australia [17].
With the results obtained and in response to question five, Altmetric metrics are considered complementary, providing additional information on research and its social impact on social networks, but not as a replacement for traditional metrics. It is a useful tool that provides immediate insights into scientific research in social media but does not have a valid system to replace the traditional method of scientific impact.

5. Discussion

This is the first study to use the Altmetric tool in tourism research to measure the extent of the social impact of tourism research. The article tries to verify whether there is a direct relationship between the scientific impact and social impact of the research. The Altmetric scores were analyzed, and the most relevant articles and journals at a social level in tourism were highlighted. This section contains the conclusions obtained from this research. When using the Altmetric tool, this research has found a limitation in terms of the absence of the analysis of ResearchGate, an academic social network used to give visibility to research.
If the tendency to use social networks as a means of generating social impact from research were to increase, a greater proportion of the tourism articles present in the last year should be reflected, and this is not the case. If it is true that the articles with the highest scores are found in the last year, this is due to the fact that the articles appearing in social networks have generated more conversation in recent times, perhaps because the intervention of the non-academic public extends the circle of impact.
The fact that the most repeated score value is one may be due to the fact that this mention in the social network has been produced by the author or the journal. In spite of having a mention in the social networks, these investigations lack social impact since they have not had much reach. The calculation of the median establishes that half of the articles analyzed have an AAS around 2 points; we are talking about a large number of articles with little social impact. On the contrary, the articles with the highest scores obtained in Altmetric are concentrated in certain journals, as can be seen in Table 3, which are also the journals that obtain the highest scores for the total set in this study, reinforcing Tonetti′s theory [5], which proposes that journals should be responsible for the social dissemination of articles with scientific quality. Another fact to bear in mind when dealing with high scores is whether these scores are due to negative comments; therefore, an analysis of the content for each publication in social media would need to be carried out. The AAS can be considered more as a measure of influence than quality [57].
There was no significant correlation between the citation index and the Altmetric score. The highest correlation was found in 2019, which would support the theory of the increased use of social media for research sharing. It should be borne in mind that the scientific impact of an article may increase over the years, with more and more citations accumulating, whereas this is unlikely to be the case with a publication on social networks in previous years, so the social impact of a particular study would not be very variable over time, except for older articles that have dealt with a very socially relevant topic at present, such as the new coronavirus disease (COVID-19). The correlation would then be higher in 2019 because it has not yet achieved the maximum number of citations but perhaps already has the maximum Altmetric score, these scores being more related now than if we analyzed the same articles a year later. In addition, a correlation close to one could determine more disclosable but not more scientifically interesting articles.
This research is the starting point that highlights the importance of social impact in tourism research. It is an additional instrument to consider; it serves to have greater criteria when developing a policy or prioritizing public or private investment. A future article may define concrete criteria and indicators that favor investments in projects with greater social impact.
It is necessary to develop a methodology that unifies scientific and social impact under a single framework. One opportunity for future research could be studying the relationship between scientifically irrelevant articles and Altmetric data, since in this study, only the articles with a high academic impact were selected. This could test whether in some cases, social networks pump out information from scientific publications without any academic value.
Another research opportunity is the gap in measuring social impact, since Altmetric does not measure the social impact of the research. This could be addressed by designing and testing a clear, measurable and universal system of indicators that could be applied at all stages of research. At the present time, there are no widely accepted indicators, such as citations or the H index, that can identify social production. There have not yet been established guidelines for the information to be collected and the monitoring of data [13].

Author Contributions

Conceptualization: A.V.L. and M.G.N.-l.A.; Method and data analysis: A.V.L. and M.G.N.-l.A.; Writing: A.V.L. and M.G.N.-l.A.; Revision and editing, A.V.L. and M.G.N.-l.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Spanish Ministry of Science, Innovation and Universities [“Analysing the role of public policy in managing tourism destinations in the mobilities era (POLITUR)”/CSO2017-82156-R], the AEI/FEDER, UE and the Department of Research and Universities of the Catalan Government [GRATET-2017SGR22].

Acknowledgments

The authors would like to acknowledge Altmetric and Dimensions for the free and open source of the data.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Holbrook, J.B.; Frodeman, R. Peer review and the ex ante assessment of societal impacts. Res. Eval. 2011, 20, 239–246. [Google Scholar] [CrossRef]
  2. Molas-Gallart, J. Research evaluation and the assessment of public value. Arts Humanit. High. Educ. 2015, 14, 111–126. [Google Scholar] [CrossRef]
  3. Muhonen, R.; Benneworth, P.; Olmos-Peñuela, J. From productive interactions to impact pathways: Understanding the key dimensions in developing SSH research societal impact. Res. Eval. 2020, 29, 34–47. [Google Scholar]
  4. Esko, T.; Miettinen, R. Scholarly understanding, mediating artefacts and the social impact of research in the educational sciences. Res. Eval. 2019, 28, 295–303. [Google Scholar] [CrossRef]
  5. Tonetti, M.S. Leadership in publishing. J. Dent. 2019, 87, 28–31. [Google Scholar] [CrossRef]
  6. Rau, H.; Goggins, G.; Fahy, F. From invisibility to impact: Recognising the scientific and societal relevance of interdisciplinary sustainability research. Res. Policy 2018, 47, 266–276. [Google Scholar] [CrossRef]
  7. Gomez, A.; Padrós, M.; Rios-Gonzalez, O.; Catalin-Mara, L.; Pukepuke, T. Reaching Social Impact through the Communicative Methodology. Researching With Rather than On Vulnerable Populations: The Roma Case. Front. Educ. 2019, 4. [Google Scholar] [CrossRef] [Green Version]
  8. Toledo, E.G. Research assessment in Humanities and Social Sciences in review. Rev. Española Doc. Científica 2018, 41, 1–14. [Google Scholar]
  9. Doyle, J. Reconceptualising research impact: Reflections on the real-world impact of research in an Australian context. High. Educ. Res. Dev. 2018, 37, 1366–1379. [Google Scholar] [CrossRef]
  10. Branch, K. Guide to Social Impact Assessment: A Framework for Assessing Social Change; Routledge: New York, NY, USA, 2018. [Google Scholar]
  11. Sherren, K.; Parkins, J.R.; Smit, M.; Holmlund, M.; Chen, Y. Digital archives, big data and image-based culturomics for social impact assessment: Opportunities and challenges. Environ. Impact Assess. Rev. 2017, 67, 23–30. [Google Scholar] [CrossRef]
  12. Altmetric Blog. Available online: https://www.altmetric.com/blog/scoreanddonut/ (accessed on 9 March 2020).
  13. Spaapen, J.; Van Drooge, L. Introducing ‘productive interactions’ in social impact assessment. Res. Eval. 2011, 20, 211–218. [Google Scholar] [CrossRef] [Green Version]
  14. Assessment, S.I. Guidelines and principles for social impact assessment. Environ. Impact Assess. Rev. 1995, 15, 11–43. [Google Scholar]
  15. Vanclay, F. Conceptualising social impacts. Environ. Impact Assess. Rev. 2002, 22, 183–211. [Google Scholar] [CrossRef]
  16. Vanclay, F. International principles for social impact assessment. Impact Assess. Proj. Apprais. 2003, 21, 5–12. [Google Scholar] [CrossRef]
  17. Flecha, G.R. Evaluación del impacto social de la investigación. Rev. Fom. Soc. 2018, 485–502. [Google Scholar] [CrossRef] [Green Version]
  18. Pulido, C.M.; Redondo-Sama, G.; Sordé-Martí, T.; Flecha, R. Social impact in social media: A new method to evaluate the social impact of research. PLoS ONE 2018, 13. [Google Scholar] [CrossRef]
  19. Reale, E.; Avramov, D.; Canhial, K.; Donovan, C.; Flecha, R.; Holm, P.; Primeri, E. A review of literature on evaluating the scientific, social and political impact of social sciences and humanities research. Res. Eval. 2017, 27, 298–308. [Google Scholar] [CrossRef] [Green Version]
  20. Mills, T.; Massoumi, N.; Miller, D. The ethics of researching ‘terrorism’and political violence: A sociological approach. Contemp. Soc. Sci. 2019. [Google Scholar] [CrossRef] [Green Version]
  21. Esko, T.; Tuunainen, J. Achieving the social impact of science: An analysis of public press debate on urban development. Sci. Public Policy 2019, 46, 404–414. [Google Scholar] [CrossRef]
  22. Nguyen, T.Q.T.; Young, T.; Johnson, P.; Wearing, S. Conceptualising networks in sustainable tourism development. Tour. Manag. Perspect. 2019, 32, 100575. [Google Scholar] [CrossRef]
  23. Álvarez-Bornstein, B.; Montesi, M. Who is interacting with researchers on Twitter? A survey in the field of Information Science. JLIS It 2019, 10, 87–106. [Google Scholar]
  24. Glover, T.D. Leisure research for social impact. J. Leis. Res. 2015, 47, 1–14. [Google Scholar] [CrossRef]
  25. Tricarico, L.; Galimberti, A.; Campanaro, A.; Magoni, C.; Labra, M. Experimenting with RRI tools to Drive Sustainable Agri-Food Research: The SASS Case Study from Sub-Saharan Africa. Sustainability 2020, 12, 827. [Google Scholar] [CrossRef] [Green Version]
  26. Salvia, A.L.; Leal Filho, W.; Brandli, L.L.; Griebeler, J.S. Assessing research trends related to Sustainable Development Goals: Local and global issues. J. Clean. Prod. 2019, 208, 841–849. [Google Scholar] [CrossRef] [Green Version]
  27. International Institute for Sustainable Development. Available online: https://www.iisd.org/learning/eia/es/eia-essentials/timeline/ (accessed on 4 March 2019).
  28. Cuthill, M. Strengthening the ‘social’ in sustainable development: Developing a conceptual framework for social sustainability in a rapid urban growth region in Australia. Sustain. Dev. 2010, 18, 362–373. [Google Scholar] [CrossRef]
  29. Tourism and the Sustainable Development Goals. Available online: https://www.e-unwto.org/doi/pdf/10.18111/9789284417254 (accessed on 6 March 2019).
  30. Gursoy, D.; Rutherford, D.G. Host attitudes toward tourism: An improved structural model. Ann. Tour. Res. 2004, 31, 495–516. [Google Scholar] [CrossRef]
  31. Butler, J.S.; Kaye, I.D.; Sebastian, A.S.; Wagner, S.C.; Morrissey, P.B.; Schroeder, G.D.; Kepler, C.K.; Vaccaro, A.R. The evolution of current research impact metrics: From bibliometrics to altmetrics? Clin. Spine Surg. 2017, 30, 226–228. [Google Scholar] [CrossRef]
  32. ISI Journal Citation Reports. Available online: Jcr.fecyt.es (accessed on 2 March 2020).
  33. Bornmann, L. What is societal impact of research and how can it be assessed? A literature survey. J. Am. Soc. Inf. Sci. Technol. 2013, 64, 217–233. [Google Scholar] [CrossRef]
  34. Chen, W.M.; Bukhari, M.; Cockshull, F.; Galloway, J. The relationship between citations, downloads and alternative metrics in rheumatology publications: A bibliometric study. Rheumatology 2020, 59, 277–280. [Google Scholar] [CrossRef]
  35. Manetti, G.; Bellucci, M. The use of social media for engaging stakeholders in sustainability reporting. Account. Audit. Account. J. 2016, 29, 985–1011. [Google Scholar] [CrossRef]
  36. De Jong, S.; Barker, K.; Cox, D.; Sveinsdottir, T.; Van den Besselaar, P. Understanding societal impact through productive interactions: ICT research as a case. Res. Eval. 2014, 23, 89–102. [Google Scholar] [CrossRef]
  37. Barros, B.; Fernández-Zubieta, A.; Fidalgo-Merino, R.; Triguero, F. Scientific knowledge percolation process and social impact: A case study on the biotechnology and microbiology perceptions on Twitter. Sci. Public Policy 2018, 45, 804–814. [Google Scholar] [CrossRef]
  38. Travieso-Rodríguez, C.; Araújo, R.F.D. Altmetrics and citation indicators applied to scientific production in ScienceOpen: Descriptive analysis for Brazil, Spain and Portugal: Brapci 2.0. Bibliotecas. An. Investig. 2018, 14, 124–137. [Google Scholar]
  39. De Silva, P.U.; Vance, C.K. Assessing the societal impact of scientific research. In Scientific Scholarly Communication; De Silva, P.U., Vance, C.K., Eds.; Springer: Cham, Switzerland, 2017; pp. 117–132. [Google Scholar]
  40. Priem, J.; Hemminger, B.H. Scientometrics 2.0: New metrics of scholarly impact on the social Web. First Monday 2010, 15. [Google Scholar] [CrossRef]
  41. Jamali, H.R.; Alimohammadi, D. Blog citations as indicators of the societal impact of research: Content analysis of social sciences blogs. Int. J. Knowl. Content Dev. Technol. 2015, 5, 15–32. [Google Scholar] [CrossRef] [Green Version]
  42. Lyu, X.; Costas, R. How do academic topics shift across altmetric sources? A case study of the research area of Big Data. arXiv 2020, arXiv:2003.10508. [Google Scholar] [CrossRef] [Green Version]
  43. Pulido, C.M.; Mara, L.C.; Ionescu, V.; Sordé-Martí, T. Social Impact of Psychological Research on Well-Being Shared in Social Media. Front. Psychol. 2020, 11, 135. [Google Scholar] [CrossRef]
  44. Garcovich, D.; Adobes Martin, M. Measuring the social impact of research in Paediatric Dentistry: An Altmetric study. Int. J. Paediatr. Dent. 2020, 30, 66–74. [Google Scholar] [CrossRef]
  45. Alonso, A.J.; Cordón-García, J.A.; Maltrás, B.B. Altmetrics: Medición de la influencia de los medios en el impacto social de la investigación. Cuad. Doc. Multimed. 2016, 27, 75–101. [Google Scholar] [CrossRef] [Green Version]
  46. Delli, K.; Livas, C.; Spijkervet, F.K.L.; Vissink, A. Measuring the social impact of dental research: An insight into the most influential articles on the Web. Oral Dis. 2017, 23, 1155–1161. [Google Scholar] [CrossRef] [Green Version]
  47. Timilsina, M.; Khawaja, W.; Davis, B.; Taylor, M.; Hayes, C. Social impact assessment of scientist from mainstream news and weblogs. Soc. Netw. Anal. Min. 2017, 7, 48. [Google Scholar] [CrossRef] [Green Version]
  48. Besselaar, P.V.D.; Flecha, R.; Radauer, A. Monitoring the Impact of EU Framework Programmes; Publications Office of the EU: Brussels, Belgium, 2018; pp. 1–171.
  49. Directorate-General for Research and Innovation. Horizon 2020 indicators. In Assessing the Results and Impact of Horizon; Publications Office of the EU: Brussels, Belgium, 2015; pp. 1–36. [Google Scholar]
  50. Smith, R. Measuring the social impact of research: Difficult but necessary. BMJ 2001, 323, 528. [Google Scholar] [CrossRef] [PubMed]
  51. Sedighi, M. Evaluating the impact of research using the altmetrics approach (case study: The field of scientometrics). Glob. Knowl. Mem. Commun. 2020. [Google Scholar] [CrossRef]
  52. Bornmann, L.; Haunschild, R.; Adams, J. Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF). J. Informetr. 2019, 13, 325–340. [Google Scholar] [CrossRef] [Green Version]
  53. Dardas, L.A.; Woodward, A.; Scott, J.; Xu, H.; Sawair, F.A. Measuring the social impact of nursing research: An insight into altmetrics. J. Adv. Nurs. 2019, 75, 1394–1405. [Google Scholar] [CrossRef] [PubMed]
  54. Kolahi, J.; Khazaei, S. Altmetric: Top 50 dental articles in 2014. Br. Dent. J. 2016, 220, 569. [Google Scholar] [CrossRef] [PubMed]
  55. Kolahi, J.; Khazaei, S. Altmetric analysis of contemporary dental literature. Br. Dent. J. 2018, 225, 68–72. [Google Scholar] [CrossRef]
  56. Cho, J. A comparative study of the impact of Korean research articles in four academic fields using altmetrics. Perform. Meas. Metr. 2017, 18, 38–51. [Google Scholar] [CrossRef]
  57. Mullins, C.H.; Boyd, C.J.; Corey, B.L. Examining the Correlation between Altmetric Score and Citations in the General Surgery Literature. J. Surg. Res. 2020, 248, 159–164. [Google Scholar] [CrossRef]
Sustainability 12 04299 g001 550
Figure 1. The doughnut and Altmetric Attention Score (AAS).
Figure 1. The doughnut and Altmetric Attention Score (AAS).
Sustainability 12 04299 g001
Sustainability 12 04299 g002 550
Figure 2. Correlation between the AAS and citations in tourism research publications.
Figure 2. Correlation between the AAS and citations in tourism research publications.
Sustainability 12 04299 g002
Table
Table 1. Presence of tourism research publications at Altmetric.
Table 1. Presence of tourism research publications at Altmetric.
Publication Year Total PublicationPublications with the AAS
Number of PublicationsNumber of CitationsAverage CitationsNumber of PublicationsPercentage of Publications with the AASNumber of CitationsAverage Citations
2019298270542.37108836.49%31222.87
2018128796687.5158145.14%47268.13
2017103813,83013.3245443.74%715115.75
Total530730,5525.76212340.00%14,9997.07
Table
Table 2. The status of the AAS in tourism research publications.
Table 2. The status of the AAS in tourism research publications.
Publication YearPublications with the AASTotal AASRank of the AASAverage of the AASStandard DeviationMedian
2019108810,4581–7079.6138.762
201858147431–3818.1624.702
201745461541–42313.5443.042
Table
Table 3. Tourism research publications with the highest AASs.
Table 3. Tourism research publications with the highest AASs.
Article TitleAuthorsPublication YearThe AASCitationsJournal Title
Vacation Post on Facebook: A Model for Incidental Vicarious Travel ConsumptionMarder, Ben; Archer-Brown, Chris; Colliander, Jonas; Lambert, Aliette20197077Journal of Travel Research
Turning It Off: Emotions in Digital-Free TravelCai, Wenjie; McKenna, Brad; Waizenegger, Lena20195811Journal of Travel Research
Keeping the Memory but Not the Possession: Memory Preservation Mitigates Identity Loss from Product DispositionWinterich, Karen Page; Reczek, Rebecca Walker; Irwin, Julie R.20174237Journal of Marketing
Uncertainty Increases the Reliance on Affect in DecisionsRad, Ali Faraji; Pham, Michel Tuan20174226Journal of Consumer Research
Celebrities, air travel, and social normsGössling, Stefan20193880Annals of Tourism Research
Decreasing vaccine coverage rates lead to increased vulnerability to the importation of vaccine-preventable diseases in BrazilFujita, Dennis Minoru; Salvador, Felipe Scassi; Nali, Luiz Henrique da Silva; Luna, Expedito José de Albuquerque20183811Journal of Travel Medicine
Autonomous vehicles and the future of urban tourismCohen, Scott A.; Hopkins, Debbie201937111Annals of Tourism Research
When an Hour Feels Shorter: Future Boundary Tasks Alter Consumption by Contracting TimeTonietto, Gabriela N; Malkoc, Selin A; Nowlis, Stephen M20193012Journal of Consumer Research
Sounds like a healthy retail atmospheric strategy: Effects of ambient music and background noise on food salesBiswas, Dipayan; Lund, Kaisa; Szocs, Courtney201929719Journal of the Academy of Marketing Science
Selling Pain to the Saturated SelfScott, Rebecca; Cayla, Julien; Cova, Bernard201729751Journal of Consumer Research
Table
Table 4. Journals with more AASs in tourism research publications.
Table 4. Journals with more AASs in tourism research publications.
JournalTotal AASNo. of Publications with an AAS
Journal of Consumer Research6002147
Journal of Marketing194462
Journal of Travel Research177190
Journal of Sustainable Tourism1695185
Journal of the Academy of Marketing Science1342109
Annals of Tourism Research1323137
Annals of Leisure Research859190
Journal of Travel Medicine82882
Current Issues in Tourism683123
International Journal of Contemporary Hospitality Management62590

Share and Cite

MDPI and ACS Style

Viana Lora, A.; Nel-lo Andreu, M.G. Alternative Metrics for Assessing the Social Impact of Tourism Research. Sustainability 2020, 12, 4299. https://doi.org/10.3390/su12104299

AMA Style

Viana Lora A, Nel-lo Andreu MG. Alternative Metrics for Assessing the Social Impact of Tourism Research. Sustainability. 2020; 12(10):4299. https://doi.org/10.3390/su12104299

Chicago/Turabian Style

Viana Lora, Alba, and Marta Gemma Nel-lo Andreu. 2020. "Alternative Metrics for Assessing the Social Impact of Tourism Research" Sustainability 12, no. 10: 4299. https://doi.org/10.3390/su12104299

APA Style

Viana Lora, A., & Nel-lo Andreu, M. G. (2020). Alternative Metrics for Assessing the Social Impact of Tourism Research. Sustainability, 12(10), 4299. https://doi.org/10.3390/su12104299

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop