Social Media Engagement for Urology Journals—A Correlation Analysis of Traditional and Social Media Metrics

Abstract

Introduction: The growing adoption of social media (SoMe) by the scientific community has cemented the role of SoMe in information dissemination and engagement of academic work. The objective of this study is to evaluate the relationship between traditional and alternative SoMe metrics of urology journals. Methods: Urology journals listed on the SCImago Journal & Country Rank (SJR) electronic portal were selected and data pertaining to traditional metrics were collected. Official SoMe platforms of eligible journals were identified and indicators of online activity were recorded. Correlations between traditional metrics (SJR, h-index, and Scopus CiteScore) and social metrics were performed via Spearman rank-order correlation. Results: Of 107 journals, 54.2% of journals had at least one form of SoMe presence. The median SJR (0.535 versus 0.334, p = 0.005), h-index (34 versus 20, p = 0.001), and Scopus CiteScore (3.25 versus 2.20, p = 0.014) were significantly higher among journals with SoMe networks. All 3 traditional indicators demonstrated strong global correlations with various Twitter-based metrics (r_s = 0.428 to 0.571). In particular, SoMe journals with more than 3000 citations in the previous 3 years also displayed very strong correlations between all 3 traditional metrics and alternative social metrics (r_s = 0.714 to 0.821). Conclusions: Journals with SoMe presence had significantly higher traditional metric values (SJR, h-index, and CiteScore) compared to journals without SoMe presence. Strong, positive correlations between citation-based and alternative social metrics were also observed. Alternative social metrics may be harnessed as supplemental indicators of a journal’s scientific impact.

Introduction

Today’s currency of communication has largely turned digital in nature after the advent of social media (SoMe) heralded a paradigm shift in human interaction. SoMe, broadly defined as any internet-based web application that empowers real-time electronic communication between users, transcends geographical boundaries and enables instant dissemination of information across the online community. Within the realms of health care communication, SoMe has been said to enhance both intra- and inter-professional interactions, promote health literacy and education, establish peer support, and design a constructive space for healthy discourse about healthcare practices[1].

Traditionally, the impact of surgical research has always been quantitatively measured with bibliometrics. It takes into consideration numbers and indices that are objectively easier to compare as opposed to qualitative inputs[2]. Examples include impact factor (IF)[3] based on the Science Citation Index, the SCImago Journal Ranking Indicator (SJR)[4], h-index[5], and Scopus CiteScore[6,7]. Unfortunately, these parameters are purely academic and fail to consider the social influence of articles across a wider audience. In addition, the accruement of article citations requires time, purporting a “lag time” before a journal’s impact may be accurately assessed[8,9]. In attempts to circumvent these limitations, newer counterparts such as Alternative Metrics (also known as Altmetrics)[10] have been established to specifically track the online presence of a research article across various SoMe platforms, culminating in an algorithmically derived figure that reflects the overall weight of article mentions online.

A prior study evaluated whether article Altmetric scores correlated with urological journal IF and citation counts in 2013 and 2016 respectively, concluding weak correlations between newer and traditional metrics[11]. Since then, SoMe adoption within urology has accelerated by leaps and bounds, with over 53% of journals harboring some form of online presence. Of the 4 major social networking platforms (Twitter, Facebook, Instagram, YouTube), it is worth noting that Twitter has drawn the greatest extent of attention and participation of the scientific community, establishing itself as the most active platform for academic discourse[12,13]. Our study aims to analyze the use of SoMe by urological journals, hypothesizing that there is a significant correlation between traditional journal (SJR, h-index, and Scopus CiteScore) and SoMe platform metrics while placing a relatively greater emphasis on Twitter-based indicators.

Methods

Data collection

We included all urology journals indexed in the electronic portal SCImago Journal & Country Rank (https://www.scimagojr.com/journalrank.php?area=2700&category=2748)[4]. SCImago is a research group affiliated with Consejo Superior de Investigaciones Científicas [CSIC], of the universities from Granada, Extremadura, Carlos III (Madrid), and Alcala de Henares. The rankings consider region or country of origin, subject area (27 major thematic areas), and subject category (309 specific subject categories). Data are retrieved from over 34 100 titles from more than 5000 international publishers across 239 countries. The SJR score of each journal is unique compared to IF, in that it is independent of self-citations, includes non-citable documents such as commentaries and letters to the editor as part of calculating the total number of documents published by a journal, and has a relatively larger geographic and language coverage[14]. Moreover, it utilizes the reputable PageRank algorithm, which is famously incorporated by Google search engine into its web search[15]. SJR uses it for the assessment of citation quality. The rankings are publicly retrievable alongside other specified indicators for each journal. Data from all journals filtered under the subset “medicine” and search term “urology” were collected. The following variables were recorded from the SCImago database—SJR index, h-index, latest impact factor (2021), Scopus CiteScore, total number of articles published in the previous 3 years, total citations in the previous 3 years, quartile, open access rights, and region of publication (Europe, North America, Asia, Latin America, Africa, and others). Referencing the same list, SoMe presence of journals on any of the 4 main social networking platforms (Twitter, Facebook, Instagram, YouTube) was evaluated. The following indicators were obtained: date of social media account creation; total number of followers; number of tweets (Twitter); number of videos, subscribers, and views (YouTube); and number of publications (Instagram). The results were time adjusted by considering the duration of account presence on social media to accurately determine audience growth. To ensure that the SoMe accounts sourced were valid and official, only links provided on the official page of the journal were used. SoMe accounts of journal publishing groups were excluded from analysis. Data procurement and collection were performed on September 14, 2022.

Statistical analysis

The Shapiro-Wilk test was used to determine normality of data points. Parametric continuous variables were presented as mean ± standard deviation (SD). Nonparametric continuous variables were reported as median and range. Chi-square tests (or Fisher exact test, wherever applicable) were used to compare categorical variables. The Mann-Whitney U test was used to compare nonparametric continuous variables.

Relationships between traditional academic (SJR, h-index, and CiteScore) and alternative SoMe metrics were investigated using Spearman rank-order correlation coefficient, using the coefficient of determination (r_s) as a measure of the goodness of fit. Subgroup correlations were performed wherever appropriate. The strength of bivariate correlations was interpreted according to ranges defined by Dancey and Reidy[16]: r_s ≥ 0.70 (very strong), r_s 0.40 to 0.69 (strong), r_s 0.30 to 0.39 (moderate), r_s 0.20 to 0.29 (weak), r_s 0.01 to 0.19 (no or negligible). Statistical significance in this study was determined as P < 0.05. All reported P-values were 2-sided, and analyses were performed with SPSS Version 26.0 (IBM Corp., Armonk, NY).

Journal characteristics

A total of 107 urology journals sourced from the SCImago Journal & Country Rank portal were included for analysis (Supplementary Table S1). Fifty-eight journals (54.2%) had social media presence on at least 1 major social networking site. When traditional academic metrics were compared across journals with and without SoMe networks, the median SJR (0.535 versus 0.334, P = 0.005), h-index (34 versus 20, P = 0.001), and Scopus CiteScore (3.25 versus 2.20, P = 0.014) were significantly higher among journals with online presence (Table 1). Across quartiles, there were significantly more quartile-1 journals within the social media group compared with those without social media (36.2% versus 12.2%, P = 0.002). A significantly higher proportion of journals with SoMe presence had more than 3000 citations over the previous 3 years compared to journals without social networks (12.1% versus 0%, P = 0.021). Across both groups, most journal publishers were based in Europe and North America. Otherwise, there were no other notable differences in baseline characteristics, such as the number of open access journals.

Table 2 depicts the baseline information of journals with social media presence. Fifty-four of 58 journals had a social media account on Twitter; 59.3% of them had at least 1000 followers, accruing a median count of 295 followers per year. The median number of tweets generated across these journals was 797 tweets. Twenty-six of 58 journals had a presence on Facebook, 7 of 58 journals were present on YouTube, and 9 of 58 journals had an Instagram account. To encompass a holistic correlation between metrics, our analysis focused mainly on Twitter- and Facebook-derived data. This is in view of the disproportionately low presence of journals on the remainder of the SoMe platforms.

Journals with SoMe presence and a publisher base in Europe observed significantly stronger correlations between their traditional and social metrics (Table 3). The SJR indicator demonstrated strong global correlations with various alternative social metrics, such as number of Twitter followers (r_s = 0.538), number of Twitter followers per year (r_s = 0.503), number of tweets (r_s = 0.520), and number of Facebook followers (r_s = 0.438). In particular, quartile-1 journals (r_s = 0.702) and journals with more than 3000 citations (r_s = 0.713) demonstrated very strong correlations between SJR and number of Twitter followers.

Table 1. Traditional academic metrics of journals.

Table 1. Traditional academic metrics of journals.

Table 2. Journal activity on social media.

Table 2. Journal activity on social media.

Table 3. Correlation between traditional and alternative social media metrics.

Table 3. Correlation between traditional and alternative social media metrics.

Likewise, h-index correlated strongly with various social media metrics such as the number of Twitter followers (r_s = 0.571), number of Twitter followers per year (r_s = 0.570), and number of tweets (r_s = 0.506). Journals with more than 500 publications in the previous 3 years displayed relatively strong correlations between h-index and number of Twitter followers (r_s = 0.625), number of Twitter followers per year (r_s = 0.518), and number of tweets (r_s = 0.749). Journals with more than 3000 citations in the previous 3 years also displayed very strong correlations between h-index and number of Twitter followers (r_s = 0.821) / number of tweets (r_s = 0.893).

CiteScore metrics also correlated strongly with various alternative social metrics, such as number of Twitter followers (r_s = 0.487), number of Twitter followers per year (r_s = 0.428), number of tweets (r_s = 0.474), and number of Facebook followers (r_s = 0.405). Journals with more than 500 publications in the previous 3 years displayed relatively strong correlations between CiteScore and number of Twitter followers (r_s = 0.578), while journals with more than 3000 citations in the previous 3 years displayed very strong correlations between CiteScore and number of Twitter followers (r_s = 0.714) / number of tweets (r_s = 0.750).

Discussion

SoMe has revolutionized the way we communicate on a day-to-day basis and interact both professionally and academically. The use of SoMe by the urology community has increased drastically during the past few years[17]. In the present study, we evaluated the correlation between traditional index metrics and SoMe platforms among SCImago-indexed urology journals. Interestingly, we not only uncovered the prominence of SoMe usage by urological journals but also provided updated insights regarding the utility of SoMe in research dissemination. In addition, we demonstrated that journals with SoMe presence had significantly higher traditional metric values (SJR, h-index, and CiteScore) compared to journals without SoMe presence. Strong, positive correlations between citation-based and alternative social metrics were also observed.

The omnipresent World Wide Web has not only reshaped how information is shared across the internet but also paved a new way for communication by introducing social networking platforms whose hallmarks are of the ability to provide a common yet integrated space where users can interact with the community and like-minded members of the society on personal and professional levels. It was found that 74% of urologists engage in some form of SoMe presence[18] and this has paved the way for consistent scientific discussion. This finding is potentiated by the highly accessible and portable nature of SoMe, where convenience and time efficacy are optimized. Instantaneous procurement of information is made possible in the context of a urologist’s hectic schedule, enabling bite-sized but relevant pieces of research to be tailored to an individual’s social feed. SoMe has provided an equally constructive platform akin to that of a physical journal club, but with all the virtual benefits—maintaining the same extent of productive discourse about a research topic through mentions and tweets[19]. In the same vein, the inception of #UroSoMe in December 2018, a Twitter hashtag specific to urology, as well as a dedicated account (@so_uro), has revolutionized the social landscape of urology as never before[17,20]. Initially set up to promote public awareness of urological conditions and professional academic discussions, the community has culminated in international, multicenter collaborations on research work[21,22]. With official guidelines already in place to regulate SoMe use by professionals[23], much is eagerly awaited by the #UroSoMe working group to further expand its outreach.

The potential of SoMe has also been tapped into by other fields in medicine, such as surgery[12], radiology[24], otolaryngology[25], pulmonology[26], and pediatric surgery[27]. The recurring theme in these studies that similarly evaluated the utility of alternative social metrics is that SoMe indicators should be harnessed as adjuncts alongside traditional metrics to holistically evaluate academic impact. A randomized study by Luc et al.[28] evaluated the impact of tweets on thoracic surgery research articles and found that articles that were tweeted on attained significantly greater increase in citation scores at 1 year (tweeted +3.1 ± 2.4 versus non-tweeted +0.7 ± 1.3, P < 0.001). Moreover, exposure to a larger number of Twitter followers was determined to be an independent predictor of citation count. All these collectively contribute to the plausibility of SoMe platforms as reliable media of scholarly activity, allowing journals to build a wider academic audience and be exposed to peer recognition as well.

Twitter has evolved as a primary player for information dissemination in research. It prides itself as being a platform that allows for information to be shared in multiple modes: text, photographs, videos, and weblinks, and in the most compact form possible (≤ 140 characters). Such content flexibility has given rise to creative outlets for circulating research findings and achieving outreach. Infographics and videographics are revolutionary new ways in which information is shared on SoMe platforms such as Twitter and have been shown not only to attract attention to the topic they advocate but also to display an artistic flair. This captivates readership and encourages interactive discussion about the topic[29,30]. On the other hand, the development of conference-specific hashtags (#SIU22, EAU22, #AUA22, #BAUS22) has also served as a surrogate marker of outreach beyond physical means, augmenting the conference experience for a wider audience. For instance, the American Urological Association’s Annual Meeting in 2013 garnered more than 8.6 million impressions and 4663 tweets in total across the peri-conference period[31], enhancing publicity for the subsequent year’s meeting.

Our study is not devoid of limitations. We excluded correlations with impact factors derived from Journal Citation Reports, as IF has certain inherent disadvantages that preclude holistic representation of all journals[32]. More robust counterpart metrics such as the SJR indicator consider self-citation articles and include a wider geographical and language scope. Despite establishing a strong association between traditional and alternative social metrics, causality cannot be identified—the impact of research dissemination on SoMe remains controversial, with a mixed bag of opinions regarding its academic utility. A relevant caveat of note here is the need to distinguish between journals that are highly cited versus those that are highly active on SoMe. Although there is a strong positive correlation between the 2, it is imperative for academics to discern the difference between these 2 groups, for they are not directly interchangeable. A journal that receives significant SoMe growth may not necessarily translate to direct academic impact—SoMe outreach is primarily aimed at instantaneous, bite-sized, and palatable dissemination of article content. While it does increase the viewership of said journal article, it does not always encourage greater citation counts. This can be for reasons as simple as only garnering a minority of their target academic audience interested in the same topic. For instance, Hayon et al. concluded that citation counts are positively associated with the number of citations an article accrues only after 3 years in publication[33]. In fact, a novel “Twitter impact factor” derived from SoMe metrics of urology journals was also trialled to determine its correlation with the traditional impact factor[34]. However, on the other hand, established alternative metrics such as the Altmetric score have failed to demonstrate strong correlations with article citation counts within urological literature[11]. Other confounding factors such as financial capacity can determine how dedicated the parent publisher of the journal is in promoting SoMe engagement. Journals with respectable traditional metrics may have more resources to begin with for publicity, which can substantially affect the extent of outreach compared to low-impact journals. Future prospective studies are necessary to elucidate any underlying causal relationships between these variables.

Conclusions

Our study clearly demonstrates that journals with social media presence have significantly higher values of traditional metrics than those without, and these metrics (SJR, h-index, and Scopus CiteScore) correlate well with journal activity on SoMe platforms such as Twitter. Given the immediacy of SoMe metrics, indicators of SoMe presence should be actively considered as an adjunct to traditional measurements of scientific impact, generating information on both short- and long-term journal outreach and publicity[35].