Eugene Garfield’s Ideas and Legacy and Their Impact on the Culture of Research
Round 1
Reviewer 1 Report
In this review article, Baykoucheva presents a review of several of the key information science accomplishments and products developed by Eugene Garfield along with several hypotheses for how these accomplishments have influenced the scientific information landscape more broadly.
I found the article interesting, insightful, and easy to read. Baykoucheva is highly familiar with Eugene Garfield’s work, having interviewed him twice and participating in a recent ACS Symposium session dedicated to Eugene Garfield. I have several recommendation for the author to consider that I think will improve the manuscript.
The abstract writing felt redundant to me. Much of the information in the abstract was restated in the introduction. I recommend a much shorter abstract.
2. Could the Author cite the figures from Eugene Garfield’s personal archive (if they are available to others)?
3. Figure 2 would be greatly improved if adapted into a table describing each of these products in 1-2 sentences.
4. Regarding the Essays of an Information Scientist work, I’m not sure if these are indexed in WoS, but it might be useful to point out if these Essays have been cited or if viewing data is available. This data would further the argument of the global impact.
5. Regarding the ResearchID, I think this has just been incorporated and rebranded into Publons and, as such, the ResearcherID discussion might need to be updated.
6. The conclusions introduced new arguments for how impact factor and SCI has influenced new metric and information products. I would recommend expanding this discussion into a separate section prior to the conclusions.
Author Response
SB: I would like to thank the reviewer for commenting on my manuscript and for useful recommendations.
Comments and Suggestions for Authors
In this review article, Baykoucheva presents a review of several of the key information science accomplishments and products developed by Eugene Garfield along with several hypotheses for how these accomplishments have influenced the scientific information landscape more broadly.
I found the article interesting, insightful, and easy to read. Baykoucheva is highly familiar with Eugene Garfield’s work, having interviewed him twice and participating in a recent ACS Symposium session dedicated to Eugene Garfield. I have several recommendation for the author to consider that I think will improve the manuscript.
The abstract writing felt redundant to me. Much of the information in the abstract was restated in the introduction. I recommend a much shorter abstract.
SB: The abstract has been shortened.
2. Could the Author cite the figures from Eugene Garfield’s personal archive (if they are available to others)?
SB: These pictures were included in my two interviews with Dr. Garfield. These interviews were cited in the manuscript as references #1 and # 12.
3. Figure 2 would be greatly improved if adapted into a table describing each of these products in 1-2 sentences.
SB: Thank you for suggesting this. I have created such table.
4. Regarding the Essays of an Information Scientist work, I’m not sure if these are indexed in WoS, but it might be useful to point out if these Essays have been cited or if viewing data is available. This data would further the argument of the global impact.
SB: The essays were included in the print Current Contents. It is at that period when they were read widely by researchers. Even if they had been digitized, the stats would not be relevant, as they would not account for that previous period.
5. Regarding the ResearchID, I think this has just been incorporated and rebranded into Publons and, as such, the ResearcherID discussion might need to be updated.
SB: ResearcherID still exists as a separate site and serves as a unique author identifier. The screen captures included in this manuscript are from my current personal ResearcherID site:
ResercherID: http://www.researcherid.com/rid/H-1969-2011
6. The conclusions introduced new arguments for how impact factor and SCI has influenced new metric and information products. I would recommend expanding this discussion into a separate section prior to the conclusions.
SB: I decided to focus only on the direct connection between Garfield’s ideas and the culture of research. While the new metrics are largely a result of IF and citations, they were not created by him and, thus, did not pass the test for inclusion. I think that mentioning them in the Conclusion section was appropriate. As a rule, this section should point to some new and future developments beyond those discussed in the rest of the text. Another reason not to devote a section to the topic of new metrics was that the manuscript is even now quite long. Citing a chapter from my book (Managing scientific information and research data, 2015), which is devoted to the new metrics, would allow readers to learn more about this topic. The chapter was cited as reference #34. I have now added a sentence in the Conclusion section to show that the new metrics were developed using Garfield’s ideas. This is how the paragraph looks now (the highlighted sentence is the added one):
“For decades, the IF has been the most widely used and accepted tool for measuring the quality of journals and the impact of research. It may not be perfect, but it is still considered an important metric [8], and no other tool has replaced it for evaluation of impact in academic institutions. Other indicators that have emerged more recently for measuring academic impact and the quality of journals include the h-Index, the Eingen Factor, Scopus Journal Analyzer, SCImago Journal Rank (SJR), Source Normalized Impact per Paper (SNIP), and Impact per Publication (IPP) [34]. It should be acknowledged that they were developed in reaction to the IF and the citation data developed by Garfield.”
Reviewer 2 Report
The manuscript by Svetla Baykoucheva has a dual goal: present the work of Eugene Garfield and how it transformed science. Furthermore, it shows that essential developments in the ‘60s could turn to be mostly negative (misused) a few decades later. The manuscript showcases the positive sides (indexing, searchable, content alert, etc.) as well as the negative ones (gaming IF, personal evaluations based on it, gaming journal metrics).
The topic is very important, not just the problem with scientific life as such, but reminding the younger generations, that information retrieval was not always as easy as typing a few words into Google and getting the results. Furthermore, we can praise or condemn the new metrics (h-index, SJR, etc.), but it needs to be acknowledged that they were developed in reaction to IF/citation data developed by Garfield. This historical perspective might be missing from many of the discussion on the employment/usage/value of metrics.
I recommend the manuscript for publication in Publications.
Minor comment on the scientific content
P2L45: I would add the date of founding SCI (1964 if wiki is right). It is enlightening that it was founded well before the internet.
P2L49: I would list scientometrics and infometrics first, as webometrics could have developed only after the internet expanded, so the ’90s.
P7: There is a chapter describing how different citation behavior manipulate IF. I would add to this list two things. (1) Some journals, especially high-ranking print journals, limit the number of references per papers. Usually there are a lot of papers to be cited, but when pressed to curb the list one might choose to include reference to articles in high-ranking journals and omit ones in lower-tier ones (referees tend to comment on missing citations even if they know that there is a cap on the number). (2) Nowadays a lot of reference to primary literature (data as opposed to reviews) are buried in supplementary information (SI), and they do not count in citation metrics. As more and more of the primary data and methods are relegated into SIs, reviews get an even higher chunk of the citations.
P7 Bibliographic coupling is not only a way to spot plagiarism, it is also a vehicle for marginalizing some researchers/papers in the field in favor of the others. As mentioned in the manuscript, people tend not to cite their competitors or scientist from certain countries. Thus, if someone discovers the field based on some selected articles (for example written by one’s supervisor) it might be a distorted view of the field already, which then can perpetuate.
P7: The citation count for individual papers and the IF of the journal has some correlation, as people tend to read papers from higher-ranking journals.
Minor comments on style/spelling
P2L52 “when they writing” -> „when they were writing”
P3L93: Space between “love.He”
P5L167: “writier” -> “writer”
P5L171: “technologies ,Nobel” -> “technologies, Nobel”
P8L270: “48, 000” -> “48,000”
Caption of figure 8. There seems to be a repetition here.
P9L316 Space missing “Scientific Babel”[28]
Author Response
SB: I would like to thank the reviewer for analyzing my manuscript and making very useful suggestions.
The manuscript by Svetla Baykoucheva has a dual goal: present the work of Eugene Garfield and how it transformed science. Furthermore, it shows that essential developments in the ‘60s could turn to be mostly negative (misused) a few decades later. The manuscript showcases the positive sides (indexing, searchable, content alert, etc.) as well as the negative ones (gaming IF, personal evaluations based on it, gaming journal metrics).
The topic is very important, not just the problem with scientific life as such, but reminding the younger generations, that information retrieval was not always as easy as typing a few words into Google and getting the results. Furthermore, we can praise or condemn the new metrics (h-index, SJR, etc.), but it needs to be acknowledged that they were developed in reaction to IF/citation data developed by Garfield. This historical perspective might be missing from many of the discussion on the employment/usage/value of metrics.
SB: Thank you for pointing out that the new metrics emerged as a result from Garfield’s work on IF and citation data. I have included the following sentence in the Conclusion section, to emphasize this point:
361-363: “It should be acknowledged that they were developed in reaction to citation metrics developed by Garfield.”
I recommend the manuscript for publication in Publications.
Minor comment on the scientific content
P2L45: I would add the date of founding SCI (1964 if wiki is right). It is enlightening that it was founded well before the internet.
SB: The year was added:
“In 1964, Garfield created the SCI and introduced it as a “new dimension in indexing”
P2L49: I would list scientometrics and infometrics first, as webometrics could have developed only after the internet expanded, so the ’90s.
SB: The order was changed as recommended: scientometrics, infometrics, and webometrics
P7: There is a chapter describing how different citation behavior manipulate IF. I would add to this list two things. (1) Some journals, especially high-ranking print journals, limit the number of references per papers. Usually there are a lot of papers to be cited, but when pressed to curb the list one might choose to include reference to articles in high-ranking journals and omit ones in lower-tier ones (referees tend to comment on missing citations even if they know that there is a cap on the number). (2) Nowadays a lot of reference to primary literature (data as opposed to reviews) are buried in supplementary information (SI), and they do not count in citation metrics. As more and more of the primary data and methods are relegated into SIs, reviews get an even higher chunk of the citations.
SB: These considerations were included in the manuscript (highlighted in the text below):
223-232:
There are also other factors that could influence citation counts such as the online availability of articles or continued citing of articles retracted from journals. Some journals may limit the number of references included in a paper. When there are many papers that are important, authors may decide to (1) cite a review paper rather than original primary research articles, (2) mention the original paper only in the text but omit it from the list of references, which will not count the original paper as a citation, or (3) cite papers published in highly-ranked journals rather than ones published in lower-tier journals. Some authors may intentionally cite review papers, to avoid citing original research papers of competitors. As more and more of the primary research data and methods are buried in the supplementary sections of papers and are not counted as citations, reviews get an even bigger share of the citations.
P7 Bibliographic coupling is not only a way to spot plagiarism, it is also a vehicle for marginalizing some researchers/papers in the field in favor of the others. As mentioned in the manuscript, people tend not to cite their competitors or scientist from certain countries. Thus, if someone discovers the field based on some selected articles (for example written by one’s supervisor) it might be a distorted view of the field already, which then can perpetuate.
SB: The following was added to include the suggested role of bibliographic coupling:
220-222:
Bibliographic coupling could lead to marginalization of some papers in the field in favor of others. Biased citations, if copied by others, could perpetuate a distorted view of a field.
P7: The citation count for individual papers and the IF of the journal has some correlation, as people tend to read papers from higher-ranking journals.
SB: This sentence was added:
213-214: “As people tend to read papers from higher-ranking journals, such papers are more likely to be cited, which will also increase the IF of these journals.”
Minor comments on style/spelling
P2L52 “when they writing” -> „when they were writing”
SB: Corrected
P3L93: Space between “love.He”
SB: Corrected
P5L167: “writier” -> “writer”
SB: Corrected
P5L171: “technologies ,Nobel” -> “technologies, Nobel”
SB: Corrected
P8L270: “48, 000” -> “48,000”
SB: Corrected
Caption of figure 8. There seems to be a repetition here.
SB: This was corrected: “The French reacted by launching a new multilingual journal.”
P9L316 Space missing “Scientific Babel”[28]
SB: The space was inserted.
Reviewer 3 Report
I have listed the article manuscript line #s on the left and highlighted my suggestions with **
22 designed to select journals for the SCI, has served [inappropriately] for measuring the impact of
61 impact. Originally designed to select journals for the SCI, it has been used [inappropriately] for
Since publication in a highly cited journal is an accomplishment ...
**both should read: [largely inappropriately]
--------------------------------
158 called Current Contents (CC). Pocket-size and printed on thin paper (to reduce mailing costs), this
159 portable journal had series for Agricultural, Biological, and Environmental Sciences; Arts and
160 Humanities; Clinical Practice; Engineering, Technology, and Applied Sciences; Life Science;
161 Physical Chemical and Earth Sciences; Social & Behavioral Sciences.
**Reference could be made to the Wikipedia article on Current Contents [ https://en.wikipedia.org/wiki/Current_Contents ]
----------------------------
164 world. An author index allowed them to send requests for article reprints— an invaluable resource
165 for researchers in countries that could not afford subscriptions to these journals
**add: or who didn't wish to read or photocopy the article in the library.
-------------------
211 journals—those with higher IF are often considered to be of higher quality than journals with lower
212 IF.
**This statement should be qualified ... add: 'within their subject areas'
------------------------------------------------------------------------------------------------
234 and B have something in common. If the two papers have 100 percent bibliographic coupling, then it
235 is a clear case of plagiarism [21].
**This statement is too harsh ... replace 'is a clear case' with strongly suggests
--------------------------------------------------------------------------------------------------------------------------------------
270 about improving society, not chasing academic kudos”)[26] was shared with more than 48, 000
**note typo: should read: 48,000
-------------------------------------------------
299 reacted by launching a new chemistry journal-in French launch a new chemistry journal-in French.
**Need to replace 'new chemistry journal-in French launch a new chemistry journal-in French.' with: multi-language chemistry journal
**I searched SciFinder and found:
Journal "Nouveau journal de chimie" > references (1178)
refine "French" (319) > refine "German" (17)
Web of Science
NOUVEAU JOURNAL DE CHIMIE-NEW JOURNAL OF CHEMISTRY - references 1197
| French - 351 German - 15 |
---------------------------------------------------------------------
417 note spacing typo: should read:
**6. Brin, S.; Page, L. The anatomy of a search engine.
--------------------------------------------------------------
Author Response
Comments and Suggestions for Authors
SB: I would like to thank the reviewer for commenting on my manuscript and making valuable suggestions for improving it.
I have listed the article manuscript line #s on the left and highlighted my suggestions with **
22 designed to select journals for the SCI, has served [inappropriately] for measuring the impact of
SB: This was mentioned in the Abstract and has been deleted from there.
61 impact. Originally designed to select journals for the SCI, it has been used [inappropriately] for
SB: This was corrected as suggested.
Since publication in a highly cited journal is an accomplishment ...
**both should read: [largely inappropriately]
--------------------------------
158 called Current Contents (CC). Pocket-size and printed on thin paper (to reduce mailing costs), this
159 portable journal had series for Agricultural, Biological, and Environmental Sciences; Arts and
160 Humanities; Clinical Practice; Engineering, Technology, and Applied Sciences; Life Science;
161 Physical Chemical and Earth Sciences; Social & Behavioral Sciences.
**Reference could be made to the Wikipedia article on Current Contents [ https://en.wikipedia.org/wiki/Current_Contents ]
SB: This Wikipedia page is incomplete, and it says the following: “This article needs additional citations for verification.” I would not cite it in the manuscript.
----------------------------
164 world. An author index allowed them to send requests for article reprints— an invaluable resource
165 for researchers in countries that could not afford subscriptions to these journals
**add: or who didn't wish to read or photocopy the article in the library.
SB: I disagree about adding this, because most of the researchers who requested reprints were those who did not have access to the journal, not those who had access to the article but were lazy to make a copy of it.
-------------------
211 journals—those with higher IF are often considered to be of higher quality than journals with lower
212 IF.
**This statement should be qualified ... add: 'within their subject areas'
SB: Inserted 'within their subject areas'
------------------------------------------------------------------------------------------------
234 and B have something in common. If the two papers have 100 percent bibliographic coupling, then it
235 is a clear case of plagiarism [21].
**This statement is too harsh ... replace 'is a clear case' with strongly suggests
SB: corrected (221)
---------------------------------------------------------------------------------------------------------------------
270 about improving society, not chasing academic kudos”)[26] was shared with more than 48, 000
**note typo: should read: 48,000
SB: corrected
299 reacted by launching a new chemistry journal-in French launch a new chemistry journal-in French.
**Need to replace ‘new chemistry journal-in French launch a new chemistry journal-in French.’ With: multi-language chemistry journal
**I searched SciFinder and found:
Journal “Nouveau journal de chimie” > references (1178)
refine “French” (319) > refine “German” (17)
Web of Science
NOUVEAU JOURNAL DE CHIMIE-NEW JOURNAL OF CHEMISTRY – references 1197
French – 351 German – 15 | ||||||
SB: corrected |
---------------------------------------------------------------------
417 note spacing typo: should read:
**6. Brin, S.; Page, L. The anatomy of a search engine.
SB: corrected
