Next Article in Journal
Effects of Core Resource and Competence Characteristics of Sharing Economy Business on Shared Value, Distinctive Competitive Advantage, and Behavior Intention
Previous Article in Journal
An Incremental and Philosophically Different Approach to Measuring Raster Patch Porosity
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 10
Issue 10
10.3390/su10103414
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Global Research on Syndromic Surveillance from 1993 to 2017: Bibliometric Analysis and Visualization

by
Ibrahim Musa
1,†,
Hyun Woo Park
1,†,
Lkhagvadorj Munkhdalai
1 and
Keun Ho Ryu
1,2,*
1
Database/Bioinformatics Laboratory, Chungbuk National University, Cheongju 28644, Korea
2
Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City 700000, Vietnam
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Sustainability 2018, 10(10), 3414; https://doi.org/10.3390/su10103414
Submission received: 16 August 2018 / Revised: 12 September 2018 / Accepted: 19 September 2018 / Published: 25 September 2018
Browse Figures
Versions Notes

Abstract

:
Syndromic Surveillance aims at analyzing medical data to detect clusters of illness or forecast disease outbreaks. Although the research in this field is flourishing in terms of publications, an insight of the global research output has been overlooked. This paper aims at analyzing the global scientific output of the research from 1993 to 2017. To this end, the paper uses bibliometric analysis and visualization to achieve its goal. Particularly, a data processing framework was proposed based on citation datasets collected from Scopus and Clarivate Analytics’ Web of Science Core Collection (WoSCC). The bibliometric method and Citespace were used to analyze the institutions, countries, and research areas as well as the current hotspots and trends. The preprocessed dataset includes 14,680 citation records. The analysis uncovered USA, England, Canada, France and Australia as the top five most productive countries publishing about Syndromic Surveillance. On the other hand, at the Pinnacle of academic institutions are the US Centers for Disease Control and Prevention (CDC). The reference co-citation analysis uncovered the common research venues and further analysis of the keyword cooccurrence revealed the most trending topics. The findings of this research will help in enriching the field with a comprehensive view of the status and future trends of the research on Syndromic Surveillance.

1. Introduction

Syndromic Surveillance, part of public health surveillance [1], is defined as the analysis of medical data to detect clusters of illness or forecast disease outbreaks [2]. Depending on the data collection, surveillance can be broadly classified into passive surveillance where medical data are collected in a routine manner, and an active surveillance where health data are actively gathered during outbreaks [3]. Syndromic surveillance is, therefore, crucial to the safety of the population. As such, a number of emergency department-based syndromic surveillance systems and early warning systems for early detection of adverse disease events have been implemented since 2000. Thereafter, numerous countries started developing their own systems. Korea Centers for Disease Control and Prevention (KCDC) have also implemented an emergency department-based syndromic surveillance system [4,5]. Up to now, numerous journals have published articles on syndromic surveillance; however, to the best of the authors’ knowledge, a comprehensive qualitative, and quantitate evaluation of the research output has not been done. Bibliometric analysis was performed based on a novel framework to map the research in terms of collaborations, publications, and trends.

2. Materials and Methods

This paper is based on two main steps, namely, data collection and preprocessing, which gathers citation datasets from major online database, and bibliometric analysis based on the resulting data.

2.1. Data Collection and Preprocessing

The work in this paper is based on citation datasets collected from prestigious publications that appear in ISI indexed journals. The data were gathered by querying both Clarivate Analytics’ Web of Science Core Collection (WoSCC) and Scopus. The search query was composed of two parts. The first query (Q1) was “Syndromic Surveillance”, while the second query (Q2) was “Syndromic Surveillance Methods and Algorithms”. Those queries resemble the first stage of the data collection and preprocessing framework shown in Figure 1. The second stage of the framework is the data collection. When querying both Scopus and WoSCC, the results of the search query as well as the extended dataset represented by the citing articles were retrieved. A record refers to information about one published article. The information included in a record consists of the title of the paper, a list of its authors, the affiliations of the authors, the abstract, keywords, information about the journal and a list of references cited by that paper. Detailed information for a typical record is reported here: https://images.webofknowledge.com/images/help/WOS/hs_wos_fieldtags.html.
When performing the search, the results can be divided into two: the main result, which includes records about articles related to the keyword used in the search process, and extended records or citing articles, which resemble articles that cite the ones reported in the main results.
In Figure 1, the search results of Q1 in WoSCC were 3805 citation records and 14,397 extra records as extended datasets. Equally, the results of Q2 in Scopus were 118 records and 1177 citing articles. Since some of the journals might be indexed in WoSCC and Scopus, a preprocessing step was needed. The purpose of the preprocessing step, which is the last stage of the framework shown in Figure 1, is to remove duplicate citation records, merge different datasets and handle the inconsistencies. For example, the datasets from Scopus were in RIS format while the data from WoSCC were in plain text format. Therefore, Scopus data were converted into plain text format for consistency. Thereafter, the two datasets were merged into one final dataset, which was considered for the analysis step. The size of this dataset is 14,680 records.

2.2. Bibliometric Analysis and Tools

Bibliometric analysis is used to map research areas according to researchers, publications, institutions and trends. It has been applied to analyze research output in different fields such as Middle East Respiratory Syndrome [6], Bacterial Meningitis [7], T-cell [8] and Medical Big Data [9].
Various tools are available for Bibliometric analysis. In this research, CiteSpace [10] was used for the analysis, especially for creating networks of document co-citation, terms, country, and institutional collaborations. In addition, aside from the geographic distribution map, which was created by StatPlanet [11] software, CiteSpace generated all other visualizations. The impact factors of the journals were retrieved according to the 2016 list.

3. Results

The major types of bibliometric analysis were performed, namely, collaboration networks between countries and institutions, reference co-citation and keyword co-occurrence analysis. Additionally, a basic exploratory analysis was performed to understand the distribution of the citation dataset. The citation data are shown in Figure 2. The graph shows a steady increase in articles published in the field of syndromic surveillance since 2005. This increase of high-quality publications shows not only the importance of the topic for the safety and security of nations, but also the necessity to analyze the data and get quantitative and qualitative insights of the research in this field.

3.1. Countries and Institutions

The 14,680 articles considered for analysis in this paper were produced by 73 countries. Country collaboration network was created using CiteSpace. The network data were used to further map the global distribution of these countries. As a result, StatPlanet tool was used to generate the global view of the countries publishing about syndromic surveillance along with the proportion of the publications from 1993 to 2017. The color-coded map is visualized in Figure 3. The network data were also used to rank the countries according to their publications. Table 1 shows the top ten countries. USA, England, Canada, France, and Australia were the top five of the list, in that order. Table 1 also shows a list of ten institutions ranked according to their publication output. Institutions located in the United States of America dominated the top of the list. Namely, The US Centers for Disease Control and Prevention (CDC), Harvard University and Johns Hopkins University were the top three research institutions followed by the Canadian University of Toronto, and the British London School of Hygiene and Tropical Medicine, representing the top five institutions, sequentially. Table 2, on the other hand, represents the top ten most cited publications produced by these institutions.
The colors in Figure 3 represent absolute counts of papers. For instance, >25 refers to countries that have published 25 or more articles in the specified period of our dataset. Consequently, the black circles indicate the exact number of publications. The size of the circle represents the number of articles.

3.2. Reference Co-Citation Analysis

To get an overview of the publication landscape, a CiteSpace based document co-citation network was generated. This analysis can reveal the most influential papers, an intellectual base from which most publications were generated. CiteSpace summarizes the co-citation relationship between documents in terms of nodes and edges. In networking terminology, nodes are defined as dots and lines connecting these dots are called edges. In line with the previous definition, CiteSpace generates a co-citation network which is composed of dots that represent documents (i.e., papers) referred to as nodes, and two nodes are connected by a line (i.e., edge) if the two documents have been cited together in another paper. The decomposition of the resulting network into groups of strongly connected components form a cluster. CiteSpace decomposes the network into clusters using a Silhoute measure that quantifies the extent to which nodes represented in a strongly connected component are actually homogeneous. Afterwards, such clusters were summarized with a selected term that frequently occurs within these documents. This task is accomplished by the measures of Information retrieval and text mining, such as Term-Frequency–Inverse Document Frequency (TF*IDF), Mutual Information (MI) and Log-Likelihood (LLR) [21]. According to Chen [10], labelling using LLR is preferred over the other two measures. The resulting network consisted of 708 nodes and 2252 edges. A timeline visualization of the network is depicted in Figure 4. Cluster analysis was applied to the network where the network was grouped into various clusters. The clusters are labeled using the terms appearing in the titles of the citing articles according to three measures, i.e., TF*IDF, MI and LLR. Figure 4 also shows the labels of the clusters.
The figure is composed of three parts: an ID, which shows the identifier of the cluster; a label, which represents the label given to the clusters; and the graphical representation of the clusters showing their evolution over time. For instance, the first cluster with ID 0 is labeled as: “Olympic Winter Games” [22], indicating the phrase was common in the papers included in this cluster. The second cluster was labeled “ESSENCE II”, which stands for Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE II), a regional system that supports advanced surveillance within the National Capital Region (NCR) developed by Johns Hopkins University Applied Physics Laboratory (JHU/APL) and the Division of Preventive Medicine at the Walter Reed Army Institute of Research [23]. The third cluster with ID 2 was labeled “Google Flu Trend” which is basically concerned with articles that use google trends as a source for conducting disease surveillance [15]. Following that, “Pandemic Influenza” [24] is the major topic discussed by the articles of cluster with ID 4. The next cluster is concerned with research regarding “Returning Traveler” [12]. In essence, the major concern is this research is returning ill travelers who might be seen as seeds to spread an epidemic. The last three clusters, with IDs 12, 18, 19, and 30, discuss issues of “Notifiable Disease” [25], “Public Health Response” [16,26], “Syndromic Surveillance” [2,4,19,27] and “Local Perspective” [28], respectively.
A summary of the three largest clusters is shown in Table 3, where the largest cluster (ID 0) has 98 members and a silhouette value of 0.827. It is labeled as Olympic Winter Game by LLR, Syndromic Surveillance by TFIDF, and Disease Surveillance by MI. The most active citer to the cluster is Gesteland’s paper [27]. The timeline view shows this clusters as the oldest. The second largest cluster (ID 1) has 48 members and a silhouette value of 0.902. It is labeled as ESSENCE II by LLR, Syndromic Surveillance by TFIDF, and EWMA Control Chart by MI. The most active citer to the cluster is Abrams’ article [29]. The third largest cluster (ID 2) has 42 members and a silhouette value of 0.965. It is labeled as Google Flu Trend by LLR, Social Media by TFIDF, and Crowd-Sourced by MI. Milinovich’s paper [25] was the most active citer to the references in this cluster. In the paper, the authors reviewed studies that have exploited Internet use and search trends to monitor two particular diseases: Influenza and Dengue. This cluster is also the newest cluster, indicating it is an active research area. The document co-citation network was used to find the most cited references. The top ten most cited articles are listed in Table 2. The most cited article is a paper authored by Freedman DO [12] which is published by the New England Journal of Medicine. In the paper, the authors discussed the ten most cited articles. The second paper by Ginsberg J [13] is published in Nature. The third was a paper by Kulldorff M [30], published in PLoS Medicine. The fourth by Eysenbach G [14] is published in the Journal of Medical Internet Research. The fifth article by Carneiro HA [15] is published in the Journal of Clinical Infectious Diseases. All of the articles listed in Table 2 have made significant contributions in advancing the global research in syndromic surveillance. For further insights into the field, articles with citation bursts have also been identified, i.e., articles that have excessive citations within a particular period. Such articles indicate the focus of the research within that period. The CiteSpace generated top 25 articles with citation bursts are listed in Table 4. The table also shows the strength of the citation (Column 4) along with the start and ending years of the citations burst.

3.3. Keyword Cooccurrence Analysis

Although a network of keyword cooccurrence can be used to track the evolution of a particular research, in this paper, it is used to highlight hot and trending research issues.
A network of 469 nodes and 1905 edges was created from the noun phrases occuring in the citation datasets. A list of the burst phrases was generated from the network (Table 5). At the top of the list with the highest citation burst is the term Zika Virus. Since its appearance in 2015, the Zika virus (ZIKV) epidemic has motivated the Americas to enhance their surveillance systems [54]. Table 5 shows that this term has had a citation burst from 2016 until now, suggesting it is a trending research topic. Other phrases such as Big Data [55], Social Media [56], and Google Trends [15] were also among the top keywords with recent citation bursts. Therefore, the proliferation of social media along with the massive amount of real-time data associated with it, and the increasing advancement in big data technologies suggest an emerging direction towards new surveillance systems that can use Google trend in detecting and forecasting outbreaks.

4. Discussion

In this paper, a bibliometric analysis of the global research on Syndromic Surveillance between the period of 1993 and 2017 has been presented. The citation datasetswere collected from two major sources: Scopus, and Web of Science. The analysis was performed to evaluate the volumes of publications, the major venues, the active institutions, countries, and authors, as well as to assess the intellectual bases, and highlight the research fronts. Some findings are as follows:
(1)
The publications follow almost linear increase from 2005 onwards. While this pattern has been observed in numerous bibliometric analyses of well-established research fields [8,9,57], it indicates the global efforts to combat epidemics [58,59,60] and pandemics [61] as they occur.
(2)
The top three productive countries were USA, England, and Canada, respectively. Table 1 and Figure 3 show the complete list.
(3)
The study has uncovered three major clusters of research, namely Disease Surveillance, EWMA Control Charts, and Crowd-Sourced. The papers within these clusters resemble the intellectual base of the subfield which can be labeled as the cluster label. For example, a careful investigation of Table 6 shows that the second largest cluster (i.e., research field) has 48 papers as its intellectual base. This cluster is concerned with the applications of the Statistical Process Control (SPC) methods for the purpose of disease surveillance. The papers which cite elements of this cluster can be viewed as research fronts. For example, the work in [62] can be considered as a current research front which builds on the intellectual base of Crowd-Sourced methods for disease surveillance.
This study has numerous strong points: (1) It can serve as a guide to new researchers in the field as well as help policy makers direct the research. (2) It is not only the first bibliometric review applied to research output in the field of Syndromic Surveillance, but also considered two major databases: Scopus, and Web of Science’s Core Collections. (3) Although the framework was applied to the research output in Syndromic and Disease surveillance here, it can be generalized to other fields. (4) To the best of the authors’ knowledge, it is the first study to put forward not only the major clusters of publications, but also to reveal their associated intellectual bases, and research fronts, as shown in Table 6.
Despite its major contributions, the study has some limitations: (1) although this study has combined datasets from two of the major sources, it might have missed papers not indexed in Scopus, and Web of Science. It has also ignored research papers written in other languages as well as technical reports. (2) Similar to the majority of bibliometric analysis tools, CiteSpace relies on the citation count as a measure of importance and impact. Thus, it might have overlooked recent articles of greater impact beyond the citation count. (3) Similar to some other works in bibliometric analysis, the clusters and their associated labels have not been validated by subject matter experts.

5. Conclusions

This paper has provided a first of its kind qualitative and quantitate view of the global research output on syndromic surveillance. In essence, the paper made the following contributions. It provides a general overview of the research landscape in the last 24 years (1993–2017) where the most productive countries and institutions with most publications have been identified. The paper also identified a list of the most significant articles. Furthermore, the current research themes and the emerging trends of research have been identified. In general, this article provides not only a bird’ s-eye view to the researchers and policy makers working on the field of syndromic surveillance, but also serves as a technical guidance for systematic reviews.

Author Contributions

Conceptualization, I.M. and H.W.P.; Data curation, I.M., H.W.P. and L.M.; Formal analysis, I.M. and L.M.; Funding acquisition, K.H.R.; Methodology, I.M. and H.W.P.; Project administration, H.W.P.; Resources, I.M. and H.W.P.; Software, I.M.; Supervision, K.H.R.; Visualization, I.M. and H.W.P.; Writing—original draft, I.M. and H.W.P.; and Writing—review and editing, I.M., H.W.P. and L.M.

Funding

This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT and Future Planning (No. 2017R1A2B4010826), and MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-2013-1-00881) supervised by the IITP (Institute for Information and Communications Technology Promotion).

Acknowledgments

The authors would like to thank Yang Eunjo for her suggestions to improve the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Teutsch, S.M.; Churchill, R.E. Principles and Practice of Public Health Surveillance; Oxford University Press: New York, NY, USA, 2000. [Google Scholar]
  2. Henning, K.J. What is syndromic surveillance? Morb. Mortal. Wkly. Rep. 2004, 53, 7–11. [Google Scholar]
  3. Vogt, R.L.; LaRue, D.; Klaucke, D.N.; Jillson, D.A. Comparison of an active and passive surveillance system of primary care providers for hepatitis, measles, rubella, and salmonellosis in Vermont. Am. J. Public Health 1983, 73, 795–797. [Google Scholar] [CrossRef] [PubMed]
  4. Yang, E.; Cho, S. The status of enhanced syndromic surveillance system in South Korea. Public Health Wkly. Rep. 2015, 8, 1255–1258. [Google Scholar]
  5. Yang, E.; Park, H.W.; Choi, Y.H.; Kim, J.; Munkhdalai, L.; Musa, I.; Ryu, K.H. A simulation-based study on the comparison of statistical and time series forecasting methods for early detection of infectious disease outbreaks. Int. J. Environ. Res. Public Health 2018, 15, 966. [Google Scholar] [CrossRef] [PubMed]
  6. Wang, Z.; Chen, Y.; Cai, G.; Jiang, Z.; Liu, K.; Chen, B.; Jiang, J.; Gu, H. A bibliometric analysis of pubmed literature on Middle East respiratory syndrome. Int. J. Environ. Res. Public Health 2016, 13, 583. [Google Scholar] [CrossRef] [PubMed]
  7. Pleger, N.; Kloft, B.; Quarcoo, D.; Zitnik, S.; Mache, S.; Klingelhoefer, D.; Groneberg, D.A. Bacterial meningitis: A density-equalizing mapping analysis of the global research architecture. Int. J. Environ. Res. Public Health 2014, 11, 10202–10214. [Google Scholar] [CrossRef] [PubMed]
  8. Zongyi, Y.; Dongying, C.; Baifeng, L. Global regulatory T-cell research from 2000 to 2015: A bibliometric analysis. PLoS ONE 2016, 11, e0162099. [Google Scholar] [CrossRef] [PubMed]
  9. Liao, H.; Tang, M.; Luo, L.; Li, C.; Chiclana, F.; Zeng, X.J. A bibliometric analysis and visualization of medical big data research. Sustainability 2018, 10, 166. [Google Scholar] [CrossRef]
  10. Chen, C. How to Use CiteSpace; Leanpub: Victoria, BC, Canada, 2016. [Google Scholar]
  11. Van Cappelle, F. StatPlanet User´s Guide; UNESCO-IIEP: Paris, France, 2009. [Google Scholar]
  12. Freedman, D.O.; Weld, L.H.; Kozarsky, P.E.; Fisk, T.; Robins, R.; von Sonnenburg, F.; Keystone, J.S.; Pandey, P.; Cetron, M.S. Spectrum of disease and relation to place of exposure among ill returned travelers. N. Engl. J. Med. 2006, 354, 119–130. [Google Scholar] [CrossRef] [PubMed]
  13. Ginsberg, J.; Mohebbi, M.H.; Patel, R.S.; Brammer, L.; Smolinski, M.S.; Brilliant, L. Detecting influenza epidemics using search engine query data. Nature 2009, 457, 1012. [Google Scholar] [CrossRef] [PubMed]
  14. Eysenbach, G. Infodemiology and infoveillance: Framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J. Med. Internet Res. 2009, 11, e11. [Google Scholar] [CrossRef] [PubMed]
  15. Carneiro, H.A.; Mylonakis, E. Google trends: A web-based tool for real-time surveillance of disease outbreaks. Clin. Infect. Dis. 2009, 49, 1557–1564. [Google Scholar] [CrossRef] [PubMed]
  16. Frumkin, H.; Hess, J.; Luber, G.; Malilay, J.; McGeehin, M. Climate change: The public health response. Am. J. Public Health 2008, 98, 435–445. [Google Scholar] [CrossRef] [PubMed]
  17. Heffernan, R.; Mostashari, F.; Das, D.; Karpati, A.; Kulldorff, M.; Weiss, D. Syndromic surveillance in public health practice, New York City. Emerg. Infect. Dis. 2004, 10, 858–864. [Google Scholar] [CrossRef] [PubMed]
  18. East, J.E.; Saunders, B.P.; Jass, J.R. Sporadic and syndromic hyperplastic polyps and serrated adenomas of the colon: Classification, molecular genetics, natural history, and clinical management. Gastroenterol. Clin. N. Am. 2008, 37, 25–46. [Google Scholar] [CrossRef] [PubMed]
  19. Mandl, K.D.; Overhage, J.M.; Wagner, M.M.; Lober, W.B.; Sebastiani, P.; Mostashari, F.; Pavlin, J.A.; Gesteland, P.H.; Treadwell, T.; Koski, E.; et al. Implementing syndromic surveillance: A practical guide informed by the early experience. J. Am. Med. Inform. Assoc. 2004, 11, 141–150. [Google Scholar] [CrossRef] [PubMed]
  20. Tarpey, P.S.; Raymond, F.L.; Nguyen, L.S.; Rodriguez, J.; Hackett, A.; Vandeleur, L.; Smith, R.; Shoubridge, C.; Edkins, S.; Stevens, C.; et al. Mutations in UPF3B, a member of the nonsense-mediated mRNA decay complex, cause syndromic and nonsyndromic mental retardation. Nat. Genet. 2007, 39, 1127. [Google Scholar] [CrossRef] [PubMed]
  21. Yu, D.; Xu, Z.; Pedrycz, W.; Wang, W. Information Sciences 1968–2016: A retrospective analysis with text mining and bibliometric. Inf. Sci. 2017, 418, 619–634. [Google Scholar] [CrossRef]
  22. Gesteland, P.H.; Wagner, M.M.; Chapman, W.W.; Espino, J.U.; Tsui, F.C.; Gardner, R.M.; Rolfs, R.T.; Dato, V.; James, B.C.; Haug, P.J. Rapid deployment of an electronic disease surveillance system in the state of Utah for the 2002 Olympic Winter Games. In Proceedings of the American Medical Informatics Association (AMIA) Symposium, San Antonio, TX, USA, 9–13 November 2002; pp. 285–289. [Google Scholar]
  23. Lombardo, J.S.; Burkom, H.; Pavlin, J. ESSENCE II and the framework for evaluating syndromic surveillance systems. Morb. Mortal. Wkly. Rep. 2004, 53, 159–165. [Google Scholar]
  24. Silk, B.J.; Berkelman, R.L. A review of strategies for enhancing the completeness of notifiable disease reporting. J. Public Health Manag. Pract. 2005, 11, 191–200. [Google Scholar] [CrossRef] [PubMed]
  25. Milinovich, G.J.; Williams, G.M.; Clements, A.C.; Hu, W. Internet-based surveillance systems for monitoring emerging infectious diseases. Lancet Infect. Dis. 2014, 14, 160–168. [Google Scholar] [CrossRef]
  26. Wu, T.S.J.; Shih, F.Y.F.; Yen, M.Y.; Wu, J.S.J.; Lu, S.W.; Chang, K.C.M.; Hsiung, C.; Chou, J.H.; Chu, Y.T.; Chang, H.; et al. Establishing a nationwide emergency department-based syndromic surveillance system for better public health responses in Taiwan. BMC Public Health 2008, 8, 18. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Gesteland, P.H.; Gardner, R.M.; Tsui, F.C.; Espino, J.U.; Rolfs, R.T.; James, B.C.; Chapman, W.W.; Moore, A.W.; Wagner, M.M. Automated syndromic surveillance for the 2002 Winter Olympics. J. Am. Med. Inform. Assoc. 2003, 10, 547–554. [Google Scholar] [CrossRef] [PubMed]
  28. Mostashari, F.; Hartman, J. Syndromic surveillance: A local perspective. J. Urban Health 2003, 80, i1. [Google Scholar] [CrossRef] [PubMed]
  29. Abrams, A.M.; Kleinman, K.; Kulldorff, M. Gumbel based p-value approximations for spatial scan statistics. Int. J. Health Geogr. 2010, 9, 61. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Kulldorff, M.; Heffernan, R.; Hartman, J.; Assunçao, R.; Mostashari, F. A space–time permutation scan statistic for disease outbreak detection. PLoS Med. 2005, 2, e59. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  31. Tsui, F.C.; Espino, J.U.; Dato, V.M.; Gesteland, P.H.; Hutman, J.; Wagner, M.M. Technical description of RODS: A real-time public health surveillance system. J. Am. Med. Inform. Assoc. 2003, 10, 399–408. [Google Scholar] [CrossRef] [PubMed]
  32. Lazarus, R.; Kleinman, K.P.; Dashevsky, I.; DeMaria, A.; Platt, R. Using automated medical records for rapid identification of illness syndromes (syndromic surveillance): The example of lower respiratory infection. BMC Public Health 2001, 1, 9. [Google Scholar] [CrossRef] [Green Version]
  33. Lober, W.B.; Karras, B.T.; Wagner, M.M.; Overhage, J.M.; Davidson, A.J.; Fraser, H.; Trigg, L.J.; Mandl, K.D.; Espino, J.U.; Tsui, F.C. Roundtable on bioterrorism detection: Information system–based surveillance. J. Am. Med. Inform. Assoc. 2002, 9, 105–115. [Google Scholar] [CrossRef] [PubMed]
  34. Harrison, A.; Wilkinson, D.; Lurie, M.; Connolly, A.M.; Karim, S.A. Improving quality of sexually transmitted disease case management in rural South Africa. Aids 1998, 12, 2329–2335. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Jernigan, J.A.; Stephens, D.S.; Ashford, D.A.; Omenaca, C.; Topiel, M.S.; Galbraith, M.; Tapper, M.; Fisk, T.L.; Zaki, S.; Popovic, T.; et al. Bioterrorism-related inhalational anthrax: The first 10 cases reported in the United States. Emerg. Infect. Dis. 2001, 7, 933. [Google Scholar] [CrossRef] [PubMed]
  36. Wagner, M.M.; Tsui, F.C.; Espino, J.U.; Dato, V.M.; Sittig, D.F.; Caruana, R.A.; McGinnis, L.F.; Deerfield, D.W.; Druzdzel, M.J.; Fridsma, D.B. The emerging science of very early detection of disease outbreaks. J. Public Health Manag. Pract. 2001, 7, 51–59. [Google Scholar] [CrossRef] [PubMed]
  37. Grosskurth, H.; Todd, J.; Mwijarubi, E.; Mayaud, P.; Nicoll, A.; Newell, J.; Mabey, D.; Hayes, R.; Mosha, F.; Senkoro, K.; et al. Impact of improved treatment of sexually transmitted diseases on HIV infection in rural Tanzania: Randomised controlled trial. Lancet 1995, 346, 530–536. [Google Scholar] [CrossRef]
  38. Clericuzio, C.; Johnson, C. Screening for Wilms tumor in high-risk individuals. Hematol. Oncol. Clin. N. Am. 1995, 9, 1253–1265. [Google Scholar] [CrossRef]
  39. Davidson, B.J.; Kulkarny, V.; Delacure, M.D.; Shah, J.P. Posterior triangle metastases of squamous cell carcinoma of the upper aerodigestive tract. Am. J. Surg. 1993, 166, 395–398. [Google Scholar] [CrossRef]
  40. Rasmussen, S.A.; Moore, C.A.; Khoury, M.J.; Cordero, J.F. Descriptive epidemiology of holoprosencephaly and arhinencephaly in metropolitan Atlanta, 1968–1992. Am. J. Med. Genet. 1996, 66, 320–333. [Google Scholar] [CrossRef]
  41. Ivanov, O.; Wagner, M.M.; Chapman, W.W.; Olszewski, R.T. Accuracy of three classifiers of acute gastrointestinal syndrome for syndromic surveillance. In Proceedings of the American Medical Informatics Association (AMIA) Symposium, San Antonio, TX, USA, 9–13 November 2002; p. 345. [Google Scholar]
  42. Vergis, E.N.; Akbas, E.; Victor, L.Y. Legionella as a cause of severe pneumonia. Semin. Respir. Crit. Care Med. 2000, 21, 295–304. [Google Scholar] [CrossRef] [PubMed]
  43. Croen, L.A.; Shaw, G.M.; Lammer, E.J. Holoprosencephaly: Epidemiologic and clinical characteristics of a California population. Am. J. Med. Genet. 1996, 64, 465–472. [Google Scholar] [CrossRef]
  44. Kura, M.M.; Hira, S.; Kohli, M.; Dalal, P.J.; Ramnani, V.; Jagtap, M. High occurrence of HBV among STD clinic attenders in Bombay, India. Int. J. STD AIDS 1998, 9, 231–233. [Google Scholar] [CrossRef] [PubMed]
  45. Wilkinson, D.; Connolly, A.M.; Harrison, A.; Lurie, M.; Karim, S.A. Sexually transmitted disease syndromes in rural South Africa: Results from health facility surveillance. Sex. Transm. Dis. 1998, 25, 20–23. [Google Scholar] [CrossRef] [PubMed]
  46. Wong, W.K.; Moore, A.; Cooper, G.; Wagner, M. WSARE: What´s strange about recent events? J. Urban Health 2003, 80, i66–i75. [Google Scholar] [PubMed]
  47. Kulldorff, M. A spatial scan statistic. Commun. Stat. Theory Methods 1997, 26, 1481–1496. [Google Scholar] [CrossRef]
  48. Harrison, A.; Karim, S.A.; Floyd, K.; Lombard, C.; Lurie, M.; Ntuli, N.; Wilkinson, D. Syndrome packets and health worker training improve sexually transmitted disease case management in rural South Africa: Randomized controlled trial. Aids 2000, 14, 2769–2779. [Google Scholar] [CrossRef] [PubMed]
  49. Espino, J.U.; Wagner, M.M. Accuracy of ICD-9-coded chief complaints and diagnoses for the detection of acute respiratory illness. In Proceedings of the American Medical Informatics Association (AMIA) Symposium, Washington, DC, USA, 3–7 November 2001; p. 164. [Google Scholar]
  50. Pollock, D.A.; Adams, D.L.; Bernardo, L.M.; Bradley, V.; Brandt, M.D.; Davis, T.E.; Garrison, H.G.; Iseke, R.M.; Johnson, S.; Kaufmann, C.R.; et al. Data elements for emergency department systems, Release 1.0 (DEEDS): A summary report. J. Emerg. Nurs. 1998, 31, 264–273. [Google Scholar] [CrossRef]
  51. Mayaud, P.; Mosha, F.; Todd, J.; Balira, R.; Mgara, J.; West, B.; Rusizoka, M.; Mwijarubi, E.; Gabone, R.; Gavyole, A.; et al. Improved treatment services significantly reduce the prevalence of sexually transmitted diseases in rural Tanzania: Results of a randomized controlled trial. Aids 1997, 11, 1873–1880. [Google Scholar] [CrossRef] [PubMed]
  52. Rotz, L.D.; Khan, A.S.; Lillibridge, S.R.; Ostroff, S.M.; Hughes, J.M. Public health assessment of potential biological terrorism agents. Emerg. Infect. Dis. 2002, 8, 225. [Google Scholar] [CrossRef] [PubMed]
  53. Garcia, P.J.; Gotuzzo, E.; Hughes, J.P.; Holmes, K.K. Syndromic management of STDs in pharmacies: Evaluation and randomised intervention trial. Sex. Transm. Infect. 1998, 74, S153–158. [Google Scholar] [PubMed]
  54. Russell, S.; Ryff, K.; Gould, C.; Martin, S.; Johansson, M. Detecting local Zika virus transmission in the continental United States: A comparison of surveillance strategies. PLOS Curr. Outbreaks 2017. [Google Scholar] [CrossRef] [PubMed]
  55. Brownstein, J.S.; Freifeld, C.C.; Madoff, L.C. Digital disease detection—Harnessing the Web for public health surveillance. N. Engl. J. Med. 2009, 360, 2153–2157. [Google Scholar] [CrossRef] [PubMed]
  56. Santillana, M.; Nguyen, A.T.; Dredze, M.; Paul, M.J.; Nsoesie, E.O.; Brownstein, J.S. Combining search, social media, and traditional data sources to improve influenza surveillance. PLoS Comput. Biol. 2015, 11, e1004513. [Google Scholar] [CrossRef] [PubMed]
  57. Sweileh, W.M.; Wickramage, K.; Pottie, K.; Hui, C.; Roberts, B.; Sawalha, A.F.; Zyoud, S.H. Bibliometric analysis of global migration health research in peer-reviewed literature (2000–2016). BMC Public Health 2018, 18, 777. [Google Scholar] [CrossRef] [PubMed]
  58. Buckeridge, D.L.; Burkom, H.; Moore, A.; Pavlin, J.; Cutchis, P.; Hogan, W. Evaluation of syndromic surveillance systems—Design of an epidemic simulation model. Morb. Mortal. Wkly. Rep. 2004, 53, 137–143. [Google Scholar]
  59. Broniatowski, D.A.; Paul, M.J.; Dredze, M. National and local influenza surveillance through twitter: An analysis of the 2012–2013 influenza epidemic. PLoS ONE 2013, 8, e83672. [Google Scholar] [CrossRef] [PubMed]
  60. Yuan, Q.; Nsoesie, E.O.; Lv, B.; Peng, G.; Chunara, R.; Brownstein, J.S. Monitoring influenza epidemics in China with search query from Baidu. PLoS ONE 2013, 8, e64323. [Google Scholar] [CrossRef] [PubMed]
  61. Singh, B.K.; Savill, N.J.; Ferguson, N.M.; Robertson, C.; Woolhouse, M.E. Rapid detection of pandemic influenza in the presence of seasonal influenza. BMC Public Health 2010, 10, 726. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  62. Xu, Q.; Gel, Y.R.; Ramirez, L.L.; Nezafati, K.; Zhang, Q.; Tsui, K.L. Forecasting influenza in Hong Kong with Google search queries and statistical model fusion. PLoS ONE 2017, 12, e0176690. [Google Scholar] [CrossRef] [PubMed]
  63. Bravata, D.M.; McDonald, K.M.; Smith, W.M.; Rydzak, C.; Szeto, H.; Buckeridge, D.L.; Haberland, C.; Owens, D.K. Systematic review: Surveillance systems for early detection of bioterrorism-related diseases. Ann. Intern. Med. 2004, 140, 910–922. [Google Scholar] [CrossRef] [PubMed]
  64. Buehler, J.W. Review of the 2003 National Syndromic Surveillance Conference? lessons learned and questions to be answered. Morb. Mortal. Wkly. Rep. 2004, 24, 18–22. [Google Scholar]
  65. Marsden-Haug, N.; Foster, V.B.; Gould, P.L.; Elbert, E.; Wang, H.; Pavlin, J.A. Code-based syndromic surveillance for influenzalike illness by International Classification of Diseases, Ninth Revision. Emerg. Infect. Dis. 2007, 13, 207. [Google Scholar] [CrossRef] [PubMed]
  66. Buckeridge, D.L. Outbreak detection through automated surveillance: A review of the determinants of detection. J. Biomed. Inform. 2007, 40, 370–379. [Google Scholar] [CrossRef] [PubMed]
  67. Buehler, J.W.; Berkelman, R.L.; Hartley, D.M.; Peters, C.J. Syndromic surveillance and bioterrorism-related epidemics. Emerg. Infect. Dis. 2003, 9, 1197. [Google Scholar] [CrossRef] [PubMed]
  68. Lewis, M.D.; Pavlin, J.A.; Mansfield, J.L.; O’Brien, S.; Boomsma, L.G.; Elbert, Y.; Kelley, P.W. Disease outbreak detection system using syndromic data in the greater Washington DC area1. Am. J. Prev. Med. 2002, 23, 180–186. [Google Scholar] [CrossRef]
  69. Lazarus, R.; Kleinman, K.; Dashevsky, I.; Adams, C.; Kludt, P.; DeMaria, A., Jr.; Platt, R. Use of automated ambulatory-care encounter records for detection of acute illness clusters, including potential bioterrorism events. Emerg. Infect. Dis. 2002, 8, 753. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  70. Irvin, C.B.; Nouhan, P.P.; Rice, K. Syndromic analysis of computerized emergency department patients’ chief complaints: An opportunity for bioterrorism and influenza surveillance. Ann. Emerg. Med. 2003, 41, 447–452. [Google Scholar] [CrossRef] [PubMed]
  71. Reis, B.Y.; Mandl, K.D. Time series modeling for syndromic surveillance. BMC Med. Inf. Decis. Mak. 2003, 3, 2. [Google Scholar] [CrossRef] [Green Version]
  72. Lombardo, J.; Burkom, H.; Elbert, E.; Magruder, S.; Lewis, S.H.; Loschen, W.; Sari, J.; Sniegoski, C.; Wojcik, R.; Pavlin, J. A systems overview of the electronic surveillance system for the early notification of community-based epidemics (ESSENCE II). J. Urban Health 2003, 80, i32–i42. [Google Scholar] [PubMed]
  73. Muscatello, D.J.; Churches, T.; Kaldor, J.; Zheng, W.; Chiu, C.; Correll, P.; Jorm, L. An automated, broad-based, near real-time public health surveillance system using presentations to hospital Emergency Departments in New South Wales, Australia. BMC Public Health 2005, 5, 141. [Google Scholar] [CrossRef] [PubMed]
  74. Reis, B.Y.; Pagano, M.; Mandl, K.D. Using temporal context to improve biosurveillance. Proc. Natl. Acad. Sci. USA 2003, 100, 1961–1965. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  75. Hutwagner, L.; Thompson, W.; Seeman, G.M.; Treadwell, T. The bioterrorism preparedness and response early aberration reporting system (EARS). J. Urban Health 2003, 80, i89–i96. [Google Scholar] [PubMed]
  76. Chapman, W.W.; Christensen, L.M.; Wagner, M.M.; Haug, P.J.; Ivanov, O.; Dowling, J.N.; Olszewski, R.T. Classifying free-text triage chief complaints into syndromic categories with natural language processing. Artif. Intell. Med. 2005, 33, 31–40. [Google Scholar] [CrossRef] [PubMed]
  77. Goldenberg, A.; Shmueli, G.; Caruana, R.A.; Fienberg, S.E. Early statistical detection of anthrax outbreaks by tracking over-the-counter medication sales. Proc. Natl. Acad. Sci. USA 2002, 99, 5237–5240. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  78. Begier, E.M.; Sockwell, D.; Branch, L.M.; Davies-Cole, J.O.; Jones, L.H.; Edwards, L.; Casani, J.A.; Blythe, D. The national capitol regions emergency department syndromic surveillance system: Do chief complaint and discharge diagnosis yield different results? Emerg. Infect. Dis. 2003, 9, 393. [Google Scholar] [CrossRef] [PubMed]
  79. Yih, W.K.; Caldwell, B.; Harmon, R.; Kleinman, K.; Lazarus, R.; Nelson, A.; Nordin, J.; Rehm, B.; Richter, B.; Ritzwoller, D.; et al. National bioterrorism syndromic surveillance demonstration program. Morb. Mortal. Wkly. Rep. 2004, 53, 43–46. [Google Scholar]
  80. Tsui, F.C.; Wagner, M.M.; Dato, V.; Chang, C. Value of ICD-9 coded chief complaints for detection of epidemics. In Proceedings of the American Medical Informatics Association (AMIA) Symposium, Washington, DC, USA, 3–7 November 2001; p. 711. [Google Scholar]
  81. Beitel, A.J.; Olson, K.L.; Reis, B.Y.; Mandl, K.D. Use of emergency department chief complaint and diagnostic codes for identifying respiratory illness in a pediatric population. Pediatr. Emerg. Care 2004, 2, 355–360. [Google Scholar] [CrossRef]
  82. Miller, B.; Kassenborg, H.; Dunsmuir, W.; Griffith, J.; Hadidi, M.; Nordin, J.D.; Danila, R. Syndromic surveillance for influenza like illness in ambulatory care setting. Emerg. Infect. Dis. 2004, 10, 1806–1811. [Google Scholar] [CrossRef] [PubMed]
  83. Reingold, A. If syndromic surveillance is the answer, what is the question? Biosecur. Bioterror. 2003, 1, 77–81. [Google Scholar] [CrossRef] [PubMed]
  84. Chapman, W.W.; Dowling, J.N.; Wagner, M.M. Classification of emergency department chief complaints into 7 syndromes: A retrospective analysis of 527,228 patients. Ann. Emerg. Med. 2005, 46, 445–455. [Google Scholar] [CrossRef] [PubMed]
  85. Stoto, M.A.; Schonlau, M.; Mariano, L.T. Syndromic surveillance: Is it worth the effort? Chance 2004, 17, 19–24. [Google Scholar] [CrossRef]
  86. Hutwagner, L.; Browne, T.; Seeman, G.M.; Fleischauer, A.T. Comparing Aberration Detection Methods with Simulated Data. Emerg. Infect. Dis. 2005, 11, 314–316. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  87. Kaufmann, A. The economic impact of a bioterrorist attack: Are prevention and postattack intervention programs justifiable? Emerg. Infect. Dis. 1997, 3, 83–94. [Google Scholar] [CrossRef] [PubMed]
  88. Hogan, W.R.; Tsui, F.C.; Ivanov, O.; Gesteland, P.H.; Grannis, S.; Overhage, J.M.; Robinson, J.M. Detection of pediatric respiratory and diarrheal outbreaks from sales of over-the-counter electrolyte products. J. Am. Med. Inform. Assoc. 2003, 10, 555–562. [Google Scholar] [CrossRef] [PubMed]
  89. Wein, L.M.; Craft, D.L.; Kaplan, E.H. Emergency response to an anthrax attack. Proc. Natl. Acad. Sci. USA 2003, 100, 4346–4351. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  90. Lober, W.B.; Trigg, L.J.; Karras, B.T.; Bliss, D.; Ciliberti, J.; Stewart, L.; Duchin, J.S. Syndromic surveillance using automated collection of computerized discharge diagnoses. J. Urban Health 2003, 80, i97–i106. [Google Scholar] [PubMed]
  91. Morse, S.S. Public Health Surveillance and Infectious Disease Detection. Biosecur. Bioterror. 2012, 10, 6–16. [Google Scholar] [CrossRef] [PubMed]
  92. Loonsk, J. BioSense—A National Initiative for Early Detection and Quantification of Public Health Emergencies. Morb. Mortal. Wkly. Rep. 2004, 53, 53–55. [Google Scholar]
  93. Bourgeois, F.T.; Olson, K.L.; Brownstein, J.S.; McAdam, A.J.; Mandl, K.D. Validation of syndromic surveillance for respiratory infections. Ann. Emerg. Med. 2006, 47, 265. [Google Scholar] [CrossRef] [PubMed]
  94. Barthell, E.N.; Cordell, W.H.; Moorhead, J.C.; Handler, J.; Feied, C.; Smith, M.S.; Cochrane, D.G.; Felton, C.W.; Collins, M.A. The Frontlines of Medicine Project: A proposal for the standardized communication of emergency department data for public health uses including syndromic surveillance for biological and chemical terrorism. Ann. Emerg. Med. 2002, 39, 422–429. [Google Scholar] [CrossRef] [PubMed]
  95. Zheng, W.; Aitken, R.; Muscatello, D.J.; Churches, T. Potential for early warning of viral influenza activity in the community by monitoring clinical diagnoses of influenza in hospital emergency departments. BMC Public Health 2007, 7, 250. [Google Scholar] [CrossRef] [PubMed]
  96. Inglesby, T.V.; Henderson, D.A.; Bartlett, J.G.; Ascher, M.S.; Eitzen, E.; Friedlander, A.M.; Hauer, J.; McDade, J.; Osterholm, M.T.; O’Toole, T.; et al. Anthrax as a biological weapon. JAMA 1999, 281, 1735. [Google Scholar] [CrossRef] [PubMed]
  97. Wagner, M.M.; Robinson, J.M.; Tsui, F.C.; Espino, J.U.; Hogan, W.R. Design of a national retail data monitor for public health surveillance. J. Am. Med. Inform. Assoc. 2003, 10, 409–418. [Google Scholar] [CrossRef] [PubMed]
  98. Rotz, L.D.; Hughes, J.M. Advances in detecting and responding to threats from bioterrorism and emerging infectious disease. Nat. Med. 2004, 10, S130. [Google Scholar] [CrossRef] [PubMed]
  99. Heffernan, R.; Mostashari, F.; Das, D.; Besculides, M.; Rodriguez, C.; Greenko, J. New York City syndromic surveillance systems. Morb. Mortal. Wkly. Rep. 2004, 53, 25–27. [Google Scholar]
  100. Sosin, D.M. Syndromic surveillance: The case for skillful investment. Biosecur. Bioterror. 2003, 1, 247–253. [Google Scholar] [CrossRef] [PubMed]
  101. Hope, K.; Durrheim, D.N.; d’Espaignet, E.T.; Dalton, C. Syndromic surveillance: Is it a useful tool for local outbreak detection? J. Epidemiol. Commun. Health 2006, 60, 374. [Google Scholar] [CrossRef]
  102. Hutwagner, L.C.; Thompson, W.W.; Seeman, G.M.; Treadwell, T. A simulation model for assessing aberration detection methods used in public health surveillance for systems with limited baselines. Stat. Med. 2005, 24, 543–550. [Google Scholar] [CrossRef] [PubMed]
  103. Travers, D.A.; Haas, S.W. Evaluation of emergency medical text processor, a system for cleaning chief complaint text data. Acad. Emerg. Med. 2004, 11, 1170–1176. [Google Scholar] [CrossRef] [PubMed]
  104. Sosin, D.M.; Thomasis, J.D. Evaluation challenges for syndromic surveillance—Making incremental progress. Morb. Mortal. Wkly. Rep. 2004, 53, 125–129. [Google Scholar]
  105. Kleinman, K.; Abrams, A.; Yih, W.K.; Platt, R.; Kulldorff, M. Evaluating spatial surveillance: Detection of known outbreaks in real data. Stat. Med. 2006, 25, 755–769. [Google Scholar] [CrossRef] [PubMed]
  106. Mocny, M. A comparison of two methods for biosurveillance of respiratory disease in the emergency department: Chief complaint vs icd9 diagnosis code. Acad. Emerg. Med. 2003, 10, 513. [Google Scholar] [CrossRef]
  107. Burkom, H.S.; Murphy, S.P.; Shmueli, G. Automated time series forecasting for biosurveillance. Stat. Med. 2007, 26, 4202–4218. [Google Scholar] [CrossRef] [PubMed]
  108. Rolland, E.; Moore, K.M.; Robinson, V.A.; McGuinness, D. Using Ontario’s “Telehealth” health telephone helpline as an early-warning system: A study protocol. BMC Health Serv. Res. 2006, 6, 10. [Google Scholar] [CrossRef] [PubMed]
  109. Widdowson, M.A.; Bosman, A.; van Straten, E.; Tinga, M.; Chaves, S.; van Eerden, L.; van Pelt, W. Automated, laboratory-based system using the internet for disease outbreak detection, the Netherlands. Emerg. Infect. Dis. 2003, 9, 1046–1052. [Google Scholar] [CrossRef] [PubMed]
  110. Zelicoff, A.; Brillman, J.; Forslund, D.W.; George, J.E.; Zink, S.; Koenig, S.; Staab, T.; Simpson, G.; Umland, E.; Bersell, K. The rapid syndrome validation project (RSVP). In Proceedings of the American Medical Informatics Association (AMIA) Symposium, Washington, DC, USA, 3–7 November 2001; pp. 771–775. [Google Scholar]
  111. Miller, J.R.; Mikol, Y. Sur veillance for diarrheal disease in New York City. J. Urban Health 1999, 76, 388–390. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  112. Meltzer, M.; Damon, I.; LeDuc, J.W.; Millar, J.D. Modeling potential responses to smallpox as a bioterrorist weapon. Emerg. Infect. Dis. 2001, 7, 959–969. [Google Scholar] [CrossRef] [PubMed]
  113. Green, M.S.; Kaufman, Z. Surveillance for early detection and monitoring of infectious disease outbreaks associated with bioterrorism. Isr. Med. Assoc. J. 2002, 4, 503–506. [Google Scholar] [PubMed]
  114. Roush, S.; Birkhead, G.; Koo, D.; Cobb, A.; Fleming, D. Mandatory reporting of diseases and conditions by health care professionals and laboratories. JAMA 1999, 282, 164–170. [Google Scholar] [PubMed]
  115. Hripcsak, G.; Wilcox, A. Reference standards, judges, and comparison subjects: Roles for experts in evaluating system performance. J. Am. Med. Inform. Assoc. 2002, 9, 1–15. [Google Scholar] [CrossRef] [PubMed]
  116. Mitchell, T. Machine Learning; McGraw Hill: Burr Ridge, IL, USA, 1997. [Google Scholar]
  117. Kuehnert, M.J.; Doyle, T.J.; Hill, H.A.; Bridges, C.B.; Jernigan, J.A.; Dull, P.M.; Reissman, D.B.; Ashford, D.A.; Jernigan, D.B. Clinical features that discriminate inhalational anthrax from other acute respiratory illnesses. Clin. Infect. Dis. 2003, 36, 328–336. [Google Scholar] [CrossRef] [PubMed]
  118. Quénel, P.; Dab, W. Influenza A and B epidemic criteria based on time-series analysis of health services surveillance data. Eur. J. Epidemiol. 1998, 14, 275–285. [Google Scholar] [CrossRef] [PubMed]
  119. Broome, C.V.; Pinner, R.W.; Sosin, D.M.; Treadwell, T.A. On the threshold. Am. J. Prev. Med. 2002, 23, 229–230. [Google Scholar] [CrossRef]
  120. Jorm, L.R. Watching the Games: Public health surveillance for the Sydney 2000 Olympic Games. J. Epidemiol. Community Health 2003, 57, 102–108. [Google Scholar] [CrossRef] [PubMed]
  121. Rodman, J.S.; Frost, F. Using nurse hot line calls for disease surveillance. Emerg. Infect. Dis. 1998, 4, 329–332. [Google Scholar] [CrossRef] [PubMed]
  122. Chapman, W.W.; Dowling, J.N.; Wagner, M.M. Generating a reliable reference standard set for syndromic case classification. J. Am. Med. Inform. Assoc. 2005, 12, 618–629. [Google Scholar] [CrossRef] [PubMed]
  123. Pavlin, J.A.; Mostashari, F.; Kortepeter, M.G.; Hynes, N.A.; Chotani, R.A.; Mikol, Y.B.; Ryan, M.A.; Neville, J.S.; Gantz, D.T.; Writer, J.V.; et al. Innovative surveillance methods for rapid detection of disease outbreaks and bioterrorism: Results of an interagency workshop on health indicator surveillance. Am. J. Public Health 2003, 93, 1230–1235. [Google Scholar] [CrossRef] [PubMed]
  124. Wagner, M.M.; Dato, V.; Dowling, J.N.; Allswede, M. Representative threats for research in public health surveillance. J. Biomed. Inform. 2003, 36, 177–188. [Google Scholar] [CrossRef]
  125. Greenko, J.; Mostashari, F.; Fine, A.; Layton, M. Clinical evaluation of the Emergency Medical Services (EMS) ambulance dispatch-based syndromic surveillance system, New York City. J. Urban Health 2003, 80, i50–i56. [Google Scholar] [PubMed]
  126. Pavlin, J.A. Investigation of disease outbreaks detected by syndromic´ surveillance systems. J. Urban Health 2003, 80, i107–i114. [Google Scholar] [PubMed]
  127. Cassa, C.A.; Grannis, S.J.; Overhage, J.M.; Mandl, K.D. A context-sensitive approach to anonymizing spatial surveillance data: Impact on outbreak detection. J. Am. Med. Inform. Assoc. 2006, 13, 160–165. [Google Scholar] [CrossRef] [PubMed]
  128. Waller, L.A.; Gotway, C.A. Applied Spatial Statistics for Public Health Data; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2004. [Google Scholar]
  129. Ryan, A.G.; Woodall, W.H. Control charts for poisson count data with varying sample sizes. J. Qual. Technol. 2010, 42, 260–275. [Google Scholar] [CrossRef]
  130. Sonesson, C.; Bock, D. A review and discussion of prospective statistical surveillance in public health. J. R. Stat. Soc. Ser. A Stat. Soc. 2003, 166, 5–21. [Google Scholar] [CrossRef]
  131. Patil, G.P.; Taillie, C. Geographic and network surveillance via scan statistics for critical area detection. Stat. Sci. 2003, 18, 457–465. [Google Scholar] [CrossRef]
  132. Mei, Y.; Han, S.W.; Tsui, K.L. Early detection of a change in Poisson rate after accounting for population size effects. Stat. Sin. 2011, 21, 597–624. [Google Scholar] [CrossRef]
  133. Sonesson, C. A CUSUM framework for detection of space–time disease clusters using scan statistics. Stat. Med. 2007, 26, 4770–4789. [Google Scholar] [CrossRef] [PubMed]
  134. Sweeney, L. Guaranteeing anonymity when sharing medical data, the Datafly System. In Proceedings of the Conference of the American Medical Informatics Association (AMIA) Annual Fall Symposium, Nashville, TN, USA, 25–29 October 1997; pp. 51–55. [Google Scholar]
  135. Franz, D.R.; Jahrling, P.B.; Friedlander, A.M.; McClain, D.J.; Hoover, D.L.; Bryne, W.R.; Pavlin, J.A.; Christopher, G.W.; Eitzen, E.M. Clinical recognition and management of patients exposed to biological warfare agents. J. Am. Med. Assoc. 1997, 278, 399–411. [Google Scholar] [CrossRef]
  136. Buckeridge, D.L.; Musen, M.A.; Switzer, P.; Crubézy, M. An analytic framework fo space-time aberrancy detection in public health surveillance data. In Proceedings of the Conference of the American Medical Informatics Association (AMIA) Annual Fall Symposium, Washington, DC, USA, 8–12 November 2003; pp. 120–124. [Google Scholar]
  137. Dórea, F.C.; Sanchez, J.; Revie, C.W. Veterinary syndromic surveillance: Current initiatives and potential for development. Prev. Vet. Med. 2011, 101, 1–17. [Google Scholar] [CrossRef] [PubMed]
  138. Hutwagner, L.C.; Maloney, E.K.; Bean, N.H.; Slutsker, L.; Martin, S.M. Using laboratory-based surveillance data for prevention: An algorithm for detecting salmonella outbreaks. Emerg. Infect. Dis. 1997, 3, 395–400. [Google Scholar] [CrossRef] [PubMed]
  139. Ozonoff, A.; Jeffery, C.; Manjourides, J.; White, L.F.; Pagano, M. Effect of spatial resolution on cluster detection: A simulation study. Int. J. Health Geogr. 2007, 6, 52. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  140. Duczmal, L.; Assunção, R. A simulated annealing strategy for the detection of arbitrarily shaped spatial clusters. Comput. Stat. Data Anal. 2004, 45, 269–286. [Google Scholar] [CrossRef]
  141. Buckeridge, D.L.; Burkom, H.; Campbell, M.; Hogan, W.R.; Moore, A.W. Algorithms for rapid outbreak detection: A research synthesis. J. Biomed. Inform. 2005, 38, 99–113. [Google Scholar] [CrossRef] [PubMed]
  142. Kulldorff, M.; Zhang, Z.; Hartman, J.; Heffernan, R.; Huang, L.; Mostashari, F. Benchmark Data and Power Calculations for Evaluating Disease Outbreak Detection Methods. Morb. Mortal. Wkly. Rep. 2004, 53, 144–151. [Google Scholar]
  143. Kleinman, K.P.; Abrams, A.M.; Kulldorff, M.; Platt, R. A model-adjusted space–time scan statistic with an application to syndromic surveillance. Epidemiol. Infect. 2005, 133, 409–419. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  144. Lawson, A.B.; Kleinman, K. (Eds.) Spatial and Syndromic Surveillance for Public Health; John Wiley & Sons, Ltd.: Hoboken, NJ, USA, 2005. [Google Scholar]
  145. Kulldorff, M. Prospective time periodic geographical disease surveillance using a scan statistic. J. R. Stat. Soc. Ser. A Stat. Soc. 2001, 164, 61–72. [Google Scholar] [CrossRef]
  146. Lawson, A.B. Large scale: Surveillance. In Statistical Methods in Spatial Epidemiology; John Wiley & Sons, Ltd.: Hoboken, NJ, USA, 2013; pp. 293–312. [Google Scholar]
  147. Kleinman, K.; Lazarus, R.; Platt, R. A generalized linear mixed models approach for detecting incident clusters of disease in small areas, with an application to biological terrorism. Am. J. Epidemiol. 2004, 159, 217–224. [Google Scholar] [CrossRef] [PubMed]
  148. Jensen, W.A.; Jones-Farmer, L.A.; Champ, C.W.; Woodall, W.H. Effects of parameter estimation on control chart properties: A literature review. J. Qual. Technol. 2006, 38, 349–364. [Google Scholar] [CrossRef]
  149. Han, S.W.; Tsui, K.L.; Ariyajunya, B.; Kim, S.B. A comparison of CUSUM, EWMA, and temporal scan statistics for detection of increases in poisson rates. Qual. Reliab. Eng. Int. 2010, 26, 279–289. [Google Scholar] [CrossRef]
  150. Kulldorff, M.; Mostashari, F.; Duczmal, L.; Yih, W.K.; Kleinman, K.; Platt, R. Multivariate scan statistics for disease surveillance. Stat. Med. 2007, 26, 1824–1833. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  151. Gordis, L. Ethical and professional issues in the changing practice of epidemiology. J. Clin. Epidemiol. 1991, 44, 9–13. [Google Scholar] [CrossRef]
  152. Kulldorff, M.; Tango, T.; Park, P.J. Power comparisons for disease clustering tests. Comput. Stat. Data Anal. 2003, 42, 665–684. [Google Scholar] [CrossRef]
  153. Assunção, R.; Costa, M.; Tavares, A.; Ferreira, S. Fast detection of arbitrarily shaped disease clusters. Stat. Med. 2006, 25, 723–742. [Google Scholar] [CrossRef] [PubMed]
  154. Williamson, G.D.; Weatherby Hudson, G. A monitoring system for detecting aberrations in public health surveillance reports. Stat. Med. 1999, 18, 3283–3298. [Google Scholar] [CrossRef]
  155. Rogerson, P.A.; Yamada, I. Approaches to syndromic surveillance when data consist of small regional counts. Morb. Mortal. Wkly. Rep. 2004, 53, 79–85. [Google Scholar]
  156. Cowling, B.J.; Wong, I.O.L.; Ho, L.M.; Riley, S.; Leung, G.M. Methods for monitoring influenza surveillance data. Int. J. Epidemiol. 2006, 35, 1314–1321. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  157. Rolka, H.; Burkom, H.; Cooper, G.F.; Kulldorff, M.; Madigan, D.; Wong, W.K. Issues in applied statistics for public health bioterrorism surveillance using multiple data streams: Research needs. Stat. Med. 2007, 26, 1834–1856. [Google Scholar] [CrossRef] [PubMed]
  158. Mostashari, F.; Kulldorff, M.; Hartman, J.J.; Miller, J.R.; Kulasekera, V. Dead bird clusters as an early warning system for west nile virus activity. Emerg. Infect. Dis. 2003, 9, 641–646. [Google Scholar] [CrossRef] [PubMed]
  159. Kulldorff, M.; Nagarwalla, N. Spatial disease clusters: Detection and inference. Stat. Med. 1995, 14, 799–810. [Google Scholar] [CrossRef] [PubMed]
  160. Shmueli, G.; Burkom, H. Statistical challenges facing early outbreak detection in biosurveillance. Technometrics 2010, 52, 39–51. [Google Scholar] [CrossRef]
  161. Patil, G.P.; Taillie, C. Upper level set scan statistic for detecting arbitrarily shaped hotspots. Environ. Ecol. Stat. 2004, 11, 183–197. [Google Scholar] [CrossRef] [Green Version]
  162. Fienberg, S.E.; Shmueli, G. Statistical issues and challenges associated with rapid detection of bio-terrorist attacks. Stat. Med. 2005, 24, 513–529. [Google Scholar] [CrossRef] [PubMed]
  163. Jackson, M.L.; Baer, A.; Painter, I.; Duchin, J. A simulation study comparing aberration detection algorithms for syndromic surveillance. BMC Med. Inf. Decis. Mak. 2007, 7, 6. [Google Scholar] [CrossRef] [PubMed]
  164. Dafni, U.G.; Tsiodras, S.; Panagiotakos, D.; Gkolfinopoulou, K.; Kouvatscas, G.; Tsourti, Z. Algorithm for statistical detection of peaks—Syndromic surveillance system for the Athens 2004 Olympic Games. Morb. Mortal. Wkly. Rep. 2004, 53, 86–94. [Google Scholar]
  165. Unkel, S.; Farrington, C.P.; Garthwaite, P.H.; Robertson, C.; Andrews, N. Statistical methods for the prospective detection of infectious disease outbreaks: A review. J. R. Stat. Soc. Ser. A Stat. Soc. 2012, 175, 49–82. [Google Scholar] [CrossRef]
  166. Brookmeyer, R.; Johnson, E.; Bollinger, R. Public health vaccination policies for containing an anthrax outbreak. Nature 2004, 432, 901–904. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  167. Cançado, A.L.; Duarte, A.R.; Duczmal, L.H.; Ferreira, S.J.; Fonseca, C.M.; Gontijo, E.C. Penalized likelihood and multi-objective spatial scans for the detection and inference of irregular clusters. Int. J. Health Geogr. 2010, 9, 55. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  168. Neill, D.B.; Cooper, G.F. A multivariate Bayesian scan statistic for early event detection and characterization. Mach. Learn. 2009, 79, 261–282. [Google Scholar] [CrossRef] [Green Version]
  169. Duarte, A.R.; Duczmal, L.; Ferreira, S.J.; Cançado, A.L. Internal cohesion and geometric shape of spatial clusters. Environ. Ecol. Stat. 2010, 17, 203–229. [Google Scholar] [CrossRef]
  170. Duczmal, L.; Tavares, R.; Patil, G.; Cançado, A.L. Testing spatial cluster occurrence in maps equipped with environmentally defined structures. Environ. Ecol. Stat. 2010, 17, 183–202. [Google Scholar] [CrossRef]
  171. Gangnon, R.E. A model for space-time cluster detection using spatial clusters with flexible temporal risk patterns. Stat. Med. 2010, 29, 2325–2337. [Google Scholar] [CrossRef] [PubMed]
  172. Gómez-Rubio, V.; López-Quílez, A. Statistical methods for the geographical analysis of rare diseases. In Rare Diseases Epidemiology; Springer: Dordrecht, The Netherlands, 2010; pp. 151–171. [Google Scholar]
  173. Chimka, J.R. Gamma regressive individuals control charts for influenza activity. Qual. Eng. 2009, 21, 182–189. [Google Scholar] [CrossRef]
  174. Rakitzis, A.C.; Castagliola, P.; Maravelakis, P.E. A new memory-type monitoring technique for count data. Comput. Ind. Eng. 2015, 85, 235–247. [Google Scholar] [CrossRef]
  175. Han, S.W.; Jiang, W.; Shu, L.; Tsui, K.L. A comparison of likelihood-based spatiotemporal monitoring methods under non-homogenous population size. Commun. Stat. Simul. Comput. 2014, 44, 14–39. [Google Scholar] [CrossRef]
  176. Han, S.W.; Zhong, H. A comparison of MCUSUM-based and MEWMA-based spatiotemporal surveillance under non-homogeneous populations. Qual. Reliab. Eng. Int. 2014, 31, 1449–1472. [Google Scholar] [CrossRef]
  177. Han, S.W.; Lee, K.J.; Zhong, H.; Kim, S.B. A comparison of spatiotemporal surveillance methods for nonhomogeneous change size. Commun. Stat. Simul. Comput. 2014, 44, 2714–2730. [Google Scholar] [CrossRef]
  178. Saghir, A.; Lin, Z. Control charts for dispersed count data: An overview. Qual. Reliab. Eng. Int. 2014, 31, 725–739. [Google Scholar] [CrossRef]
  179. Shu, L.; Jiang, W.; Tsui, K.L. A comparison of weighted CUSUM procedures that account for monotone changes in population size. Stat. Med. 2010, 30, 725–741. [Google Scholar] [CrossRef] [PubMed]
  180. Wilson, J.G.; Ballou, J.; Yan, C.; Fisher-Hoch, S.P.; Reininger, B.; Gay, J.; Salinas, J.; Sanchez, P.; Salinas, Y.; Calvillo, F.; et al. Utilizing spatiotemporal analysis of influenza-like illness and rapid tests to focus swine-origin influenza virus intervention. Health Place 2010, 16, 1230–1239. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  181. Zhao, H.; Shu, L.; Tsui, K.L. A window-limited generalized likelihood ratio test for monitoring Poisson processes with linear drifts. J. Stat. Comput. Simul. 2014, 85, 2975–2988. [Google Scholar] [CrossRef]
  182. Das, D.; Weiss, D.; Mostashari, F.; Treadwell, T.; McQuiston, J.; Hutwagner, L.; Karpati, A.; Bornschlegel, K.; Seeman, M.; Turcios, R.; et al. Enhanced drop-in syndromic surveillance in New York City following September 11, 2001. J. Urban Health 2003, 80, i76–i88. [Google Scholar] [PubMed]
  183. Khan, A.S.; Fleischauer, A.; Casani, J.; Groseclose, S.L. The next public health revolution: Public health information fusion and social networks. Am. J. Public Health 2010, 100, 1237–1242. [Google Scholar] [CrossRef] [PubMed]
  184. Scherm, H.; Thomas, C.; Garrett, K.; Olsen, J. Meta-analysis and other approaches for synthesizing structured and unstructured data in plant pathology. Annu. Rev. Phytopathol. 2014, 52, 453–476. [Google Scholar] [CrossRef] [PubMed]
  185. Thompson, L.H.; Malik, M.T.; Gumel, A.; Strome, T.; Mahmud, S.M. Emergency department and ‘Google flu trends’ data as syndromic surveillance indicators for seasonal influenza. Epidemiol. Infect. 2014, 142, 2397–2405. [Google Scholar] [CrossRef] [PubMed]
  186. Turbelin, C.; Boelle, P.Y. Open data in public health surveillance systems: A case study using the French Sentinelles network. Int. J. Med. Inform. 2013, 82, 1012–1021. [Google Scholar] [CrossRef] [PubMed]
  187. Salathé, M.; Khandelwal, S. Assessing Vaccination Sentiments with Online Social Media: Implications for Infectious Disease Dynamics and Control. PLoS Comput. Biol. 2011, 7, e1002199. [Google Scholar] [CrossRef] [PubMed]
  188. Chretien, J.P.; George, D.; Shaman, J.; Chitale, R.A.; McKenzie, F.E. Influenza forecasting in Human populations: A scoping review. PLoS ONE 2014, 9, e94130. [Google Scholar] [CrossRef] [PubMed]
  189. Kang, M.; Zhong, H.; He, J.; Rutherford, S.; Yang, F. Using google trends for influenza surveillance in South China. PLoS ONE 2013, 8, e55205. [Google Scholar] [CrossRef] [PubMed]
  190. Butler, D. When google got flu wrong. Nature 2013, 494, 155–156. [Google Scholar] [CrossRef] [PubMed]
  191. Eysenbach, G. Infodemiology and Infoveillance. Am. J. Prev. Med. 2011, 40, S154–S158. [Google Scholar] [CrossRef] [PubMed]
  192. McIver, D.J.; Brownstein, J.S. Wikipedia usage estimates prevalence of influenza-like illness in the United States in near real-time. PLoS Comput. Biol. 2014, 10, e1003581. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  193. Calain, P. From the field side of the binoculars: A different view on global public health surveillance. Health Policy Plan. 2007, 22, 13–20. [Google Scholar] [CrossRef] [PubMed]
  194. Shaman, J.; Karspeck, A. Forecasting seasonal outbreaks of influenza. Proc. Natl. Acad. Sci. USA 2012, 109, 20425–20430. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  195. Dugas, A.F.; Hsieh, Y.H.; Levin, S.R.; Pines, J.M.; Mareiniss, D.P.; Mohareb, A.; Gaydos, C.A.; Perl, T.M.; Rothman, R.E. Google flu trends: Correlation with emergency department influenza rates and crowding metrics. Clin. Infect. Dis. 2012, 54, 463–469. [Google Scholar] [CrossRef] [PubMed]
  196. Ayers, J.W.; Althouse, B.M.; Allem, J.P.; Rosenquist, J.N.; Ford, D.E. Seasonality in seeking mental health information on Google. Am. J. Prev. Med. 2013, 44, 520–525. [Google Scholar] [CrossRef] [PubMed]
  197. Santillana, M.; Zhang, D.W.; Althouse, B.M.; Ayers, J.W. What can digital disease detection learn from (an external revision to) google flu trends? Am. J. Prev. Med. 2014, 47, 341–347. [Google Scholar] [CrossRef] [PubMed]
  198. Chew, C.; Eysenbach, G. Pandemics in the age of twitter: Content analysis of tweets during the 2009 h1n1 outbreak. PLoS ONE 2010, 5, e14118. [Google Scholar] [CrossRef] [PubMed]
  199. Freifeld, C.C.; Mandl, K.D.; Reis, B.Y.; Brownstein, J.S. HealthMap: Global infectious disease monitoring through automated classification and visualization of internet media reports. J. Am. Med. Inform. Assoc. 2008, 15, 150–157. [Google Scholar] [CrossRef] [PubMed]
  200. Lazer, D.; Kennedy, R.; King, G.; Vespignani, A. The parable of google flu: Traps in big data analysis. Science 2014, 343, 1203–1205. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  201. Ortiz, J.R.; Zhou, H.; Shay, D.K.; Neuzil, K.M.; Fowlkes, A.L.; Goss, C.H. Monitoring influenza activity in the United States: A comparison of traditional surveillance systems with google flu trends. PLoS ONE 2011, 6, e18687. [Google Scholar] [CrossRef] [PubMed]
  202. Polgreen, P.M.; Chen, Y.; Pennock, D.M.; Nelson, F.D. Using internet searches for influenza surveillance. Clin. Infect. Dis. 2008, 47, 1443–1448. [Google Scholar] [CrossRef] [PubMed]
  203. Brownstein, J.S.; Freifeld, C.C.; Reis, B.Y.; Mandl, K.D. Surveillance sans frontières: Internet-based emerging infectious disease intelligence and the healthmap project. PLoS Med. 2008, 5, e151. [Google Scholar] [CrossRef] [PubMed]
  204. Salathé, M.; Bengtsson, L.; Bodnar, T.J.; Brewer, D.D.; Brownstein, J.S.; Buckee, C.; Campbell, E.M.; Cattuto, C.; Khandelwal, S.; Mabry, P.L.; et al. Digital epidemiology. PLoS Comput. Biol. 2012, 8, e1002616. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  205. Hulth, A.; Rydevik, G.; Linde, A. Web queries as a source for syndromic surveillance. PLoS ONE 2009, 4, e4378. [Google Scholar] [CrossRef] [PubMed]
  206. Jones, K.E.; Patel, N.G.; Levy, M.A.; Storeygard, A.; Balk, D.; Gittleman, J.L.; Daszak, P. Global trends in emerging infectious diseases. Nature 2008, 451, 990–993. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  207. Signorini, A.; Segre, A.M.; Polgreen, P.M. The use of twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic. PLoS ONE 2011, 6, e19467. [Google Scholar] [CrossRef] [PubMed]
  208. Chunara, R.; Andrews, J.R.; Brownstein, J.S. Social and news media enable estimation of epidemiological patterns early in the 2010 Haitian cholera outbreak. Am. J. Trop. Med. Hyg. 2012, 86, 39–45. [Google Scholar] [CrossRef] [PubMed]
  209. Dugas, A.F.; Jalalpour, M.; Gel, Y.; Levin, S.; Torcaso, F.; Igusa, T.; Rothman, R.E. Influenza forecasting with google flu trends. PLoS ONE 2013, 8, e56176. [Google Scholar] [CrossRef] [PubMed]
  210. Seifter, A.; Schwarzwalder, A.; Geis, K.; Aucott, J. The utility of “Google Trends” for epidemiological research: Lyme disease as an example. Geospat. Health 2010, 4, 135. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  211. Cook, S.; Conrad, C.; Fowlkes, A.L.; Mohebbi, M.H. Assessing google flu trends performance in the United States during the 2009 influenza virus A (H1N1) pandemic. PLoS ONE 2011, 6, e23610. [Google Scholar] [CrossRef] [PubMed]
  212. Flahault, A.; Blanchon, T.; Dorléans, Y.; Toubiana, L.; Vibert, J.F.; Valleron, A.J. Virtual surveillance of communicable diseases: A 20-year experience in France. Stat. Methods Med. Res. 2006, 15, 413–421. [Google Scholar] [CrossRef] [PubMed]
  213. Pelat, C.; Turbelin, C.; Bar-Hen, A.; Flahault, A.; Valleron, A.J. More diseases tracked by using google trends. Emerg. Infect. Dis. 2009, 15, 1327–1328. [Google Scholar] [CrossRef] [PubMed]
  214. Shaman, J.; Karspeck, A.; Yang, W.; Tamerius, J.; Lipsitch, M. Real-time influenza forecasts during the 2012–2013 season. Nat. Commun. 2013, 4, 2837. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  215. Wilson, K.; Brownstein, J.S. Early detection of disease outbreaks using the internet. Can. Med. Assoc. J. 2009, 180, 829–831. [Google Scholar] [CrossRef]
  216. Olson, D.R.; Konty, K.J.; Paladini, M.; Viboud, C.; Simonsen, L. Reassessing google flu trends data for detection of seasonal and pandemic influenza: A comparative epidemiological study at three geographic scales. PLoS Comput. Biol. 2013, 9, e1003256. [Google Scholar] [CrossRef] [PubMed]
  217. Eysenbach, G. Infodemiology: Tracking flu-related searches on the web for syndromic surveillance. In Proceedings of the Conference of the American Medical Informatics Association (AMIA) Annual Fall Symposium, Washington, DC, USA, 2006; pp. 244–248. [Google Scholar]
  218. Olson, K.L.; Grannis, S.J.; Mandl, K.D. Privacy protection versus cluster detection in spatial epidemiology. Am. J. Public Health 2006, 96, 2002–2008. [Google Scholar] [CrossRef] [PubMed]
  219. Kulldorff, M.; Huang, L.; Pickle, L.; Duczmal, L. An elliptic spatial scan statistic. Stat. Med. 2006, 25, 3929–3943. [Google Scholar] [CrossRef] [PubMed]
  220. Generous, N.; Fairchild, G.; Deshpande, A.; Valle, S.Y.D.; Priedhorsky, R. Global disease monitoring and forecasting with wikipedia. PLoS Comput. Biol. 2014, 10, e1003892. [Google Scholar] [CrossRef] [PubMed]
  221. Martin, L.J.; Xu, B.; Yasui, Y. Improving google flu trends estimates for the united states through transformation. PLoS ONE 2014, 9, e109209. [Google Scholar] [CrossRef] [PubMed]
  222. Nagar, R.; Yuan, Q.; Freifeld, C.C.; Santillana, M.; Nojima, A.; Chunara, R.; Brownstein, J.S. A case study of the new york city 2012–2013 influenza season with daily geocoded twitter data from temporal and spatiotemporal perspectives. J. Med. Internet Res. 2014, 16, e236. [Google Scholar] [CrossRef] [PubMed]
  223. Al-Tawfiq, J.A.; Zumla, A.; Gautret, P.; Gray, G.C.; Hui, D.S.; Al-Rabeeah, A.A.; Memish, Z.A. Surveillance for emerging respiratory viruses. Lancet Infect. Dis. 2014, 14, 992–1000. [Google Scholar] [CrossRef]
  224. Araz, O.M.; Bentley, D.; Muelleman, R.L. Using Google Flu Trends data in forecasting influenza-like–illness related ED visits in Omaha, Nebraska. Am. J. Emerg. Med. 2014, 32, 1016–1023. [Google Scholar] [CrossRef] [PubMed]
  225. Debin, M.; Turbelin, C.; Blanchon, T.; Bonmarin, I.; Falchi, A.; Hanslik, T.; Levy-Bruhl, D.; Poletto, C.; Colizza, V. Evaluating the feasibility and participants’ representativeness of an online nationwide surveillance system for influenza in france. PLoS ONE 2013, 8, e73675. [Google Scholar] [CrossRef] [PubMed]
  226. Domnich, A.; Panatto, D.; Signori, A.; Lai, P.L.; Gasparini, R.; Amicizia, D. Age-related differences in the accuracy of web query-based predictions of influenza-like illness. PLoS ONE 2015, 10, e0127754. [Google Scholar] [CrossRef] [PubMed]
  227. Nuti, S.V.; Wayda, B.; Ranasinghe, I.; Wang, S.; Dreyer, R.P.; Chen, S.I.; Murugiah, K. The use of google trends in health care research: A systematic review. PLoS ONE 2014, 9, e109583. [Google Scholar] [CrossRef] [PubMed]
  228. Timpka, T.; Spreco, A.; Dahlstrom, O.; Eriksson, O.; Gursky, E.; Ekberg, J.; Blomqvist, E.; Stromgren, M.; Karlsson, D.; Eriksson, H.; et al. Performance of ehealth data sources in local influenza surveillance: A 5-year open cohort study. J. Med. Internet Res. 2014, 16, e116.236. [Google Scholar] [CrossRef] [PubMed]
  229. Viboud, C.; Charu, V.; Olson, D.; Ballesteros, S.; Gog, J.; Khan, F.; Grenfell, B.; Simonsen, L. Demonstrating the use of high-volume electronic medical claims data to monitor local and regional influenza activity in the US. PLoS ONE 2014, 9, e102429. [Google Scholar] [CrossRef] [PubMed]
  230. Held, L.; Meyer, S.; Bracher, J. Probabilistic forecasting in infectious disease epidemiology: The 13th Armitage lecture. Stat. Med. 2017, 36, 3443–3460. [Google Scholar] [CrossRef] [PubMed]
  231. Pollett, S.; Boscardin, W.J.; Azziz-Baumgartner, E.; Tinoco, Y.O.; Soto, G.; Romero, C.; Kok, J.; Biggerstaff, M.; Viboud, C.; Rutherford, G.W. Evaluating google flu trends in latin america: Important lessons for the next phase of digital disease detection. Clin. Infect. Dis. 2016, 64, 34–41. [Google Scholar] [CrossRef] [PubMed]
  232. Yang, S.; Kou, S.C.; Lu, F.; Brownstein, J.S.; Brooke, N.; Santillana, M. Advances in using Internet searches to track dengue. PLoS Comput. Biol. 2017, 13, e1005607. [Google Scholar] [CrossRef] [PubMed]
  233. Li, Z.; Liu, T.; Zhu, G.; Lin, H.; Zhang, Y.; He, J.; Deng, A.; Peng, Z.; Xiao, J.; Rutherford, S.; et al. Dengue baidu search index data can improve the prediction of local dengue epidemic: A case study in Guangzhou, China. PLoS Negl. Trop. Dis. 2017, 11, e0005354. [Google Scholar] [CrossRef] [PubMed]
  234. McGough, S.F.; Brownstein, J.S.; Hawkins, J.B.; Santillana, M. Forecasting zika incidence in the 2016 Latin America outbreak combining traditional disease surveillance with search, social media, and news report data. PLoS Negl. Trop. Dis. 2017, 11, e0005295. [Google Scholar] [CrossRef] [PubMed]
  235. Yan, S.; Chughtai, A.; Macintyre, C. Utility and potential of rapid epidemic intelligence from internet-based sources. Int. J. Infect. Dis. 2017, 63, 77–87. [Google Scholar] [CrossRef] [PubMed]
  236. Althouse, B.M.; Scarpino, S.V.; Meyers, L.A.; Ayers, J.W.; Bargsten, M.; Baumbach, J.; Brownstein, J.S.; Castro, L.; Clapham, H.; Cummings, D.A.; et al. Enhancing disease surveillance with novel data streams: Challenges and opportunities. EPJ Data Sci. 2015, 4, 17. [Google Scholar] [CrossRef] [PubMed]
  237. Fung, I.C.; Fu, K.W.; Ying, Y.; Schaible, B.; Hao, Y.; Chan, C.H.; Tse, Z.T. Chinese social media reaction to the MERS-CoV and avian influenza A (H7N9) outbreaks. Infect. Dis. Poverty 2013, 2, 31. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  238. Gesualdo, F.; Stilo, G.; Agricola, E.; Gonfiantini, M.V.; Pandolfi, E.; Velardi, P.; Tozzi, A.E. Influenza-like illness surveillance on twitter through automated learning of naive language. PLoS ONE 2013, 8, e82489. [Google Scholar] [CrossRef] [PubMed]
  239. Harsha, A.K.; Schmitt, J.E.; Stavropoulos, S.W. Know your market: Use of online query tools to quantify trends in patient information-seeking behavior for varicose vein treatment. J. Vasc. Interv. Radiol. 2014, 25, 53–57. [Google Scholar] [CrossRef] [PubMed]
Sustainability 10 03414 g001 550
Figure 1. Data collection and preprocessing Framework.
Figure 1. Data collection and preprocessing Framework.
Sustainability 10 03414 g001
Sustainability 10 03414 g002 550
Figure 2. Annual growth of the publications related to Syndromic Surveillance.
Figure 2. Annual growth of the publications related to Syndromic Surveillance.
Sustainability 10 03414 g002
Sustainability 10 03414 g003 550
Figure 3. Worldwide view of the research output related to syndromic surveillance from 1993 to 2017. The color intensity represents the number of publications. The black circles show the exact number of publications.
Figure 3. Worldwide view of the research output related to syndromic surveillance from 1993 to 2017. The color intensity represents the number of publications. The black circles show the exact number of publications.
Sustainability 10 03414 g003
Sustainability 10 03414 g004 550
Figure 4. Timeline view of the reference co-citation network related to syndromic surveillance from 1993 to 2017.
Figure 4. Timeline view of the reference co-citation network related to syndromic surveillance from 1993 to 2017.
Sustainability 10 03414 g004
Table
Table 1. Ranking of countries and institutions that published articles Related to Syndromic Surveillance from 1993 to 2017.
Table 1. Ranking of countries and institutions that published articles Related to Syndromic Surveillance from 1993 to 2017.
RankCountriesInstitutions
FrequencyCountryFrequencyInstitution
14388USA346The US Centers for Disease Control and Prevention (CDC)
21158England317Harvard University
3723Canada146Johns Hopkins University
4703France127The University of Toronto
5696Australia118The London School of Hygiene and Tropical Medicine
6651China117University of Washington
7460Germany112University of North Carolina
8412Netherlands108Emory University
9349Italy101Colombia University
10319Spain100Public Health England (PHE)
Table
Table 2. The top 10 cited references in syndromic surveillance from 1993 to 2017.
Table 2. The top 10 cited references in syndromic surveillance from 1993 to 2017.
FreqFirst AuthorJournalID
400Freedman [12]New England Journal of Medicine7
314Ginsberg [13]Nature2
165Eysenbach [14]Journal of Medical Internet Research2
163Carneiro [15]Clinical Infectious Diseases2
158Frumkin [16]American Journal of Public Health18
112Heffernan [17]Emerging Infectious Diseases0
110East [18]Gastroenterology Clinics5
108Mandl [19]American Medical Informatics Association0
102Tarpey [20]Nature genetics23
Note: Freq, Frequency; ID, Cluster Identification Number.
Table
Table 3. Summary of the largest three clusters.
Table 3. Summary of the largest three clusters.
IDSizeSilhouetteLabel (TFIDF)Label (LLR)Label (MI)Mean (Cited Year)
0980.827SyndromicOlympic Winter GameDisease/Surveillance2002
Surveillance
1480.902SyndromicESSENCE IIEWMA Control Chart2004
Surveillance
2420.965Social MediaGoogle Flu TrendCrowd-Sourced2011
Note: ID: Cluster Identification Number.
Table
Table 4. The top 25 articles on Syndromic Surveillance with citation burst.
Table 4. The top 25 articles on Syndromic Surveillance with citation burst.
ReferencesYearStrengthBeginEnd
Tsui [31]200319.943420032009
Lazarus [32]200115.852220032007
Lober Wb [33]200215.248520032007
Harrison [34]199813.498919992004
Jernigan [35]200110.747320032007
Wagner [36]20019.368820032009
Grosskurth [37]19959.113519982003
Tsui [31]20018.624120032007
Clericuzio [38]19957.797919962003
Davidson [39]19937.492619932000
Jernigan [35]20016.631820022006
Rasmussen [40]19966.08119982004
Ivanov [41]20025.93820032005
Vergis [42]20005.571620012004
Croen [43]19964.910920002004
Kura [44]19984.593620002003
Wilkinson [45]19984.582619992003
Wong [46]20034.560320032007
Kulldorff [47]19974.154820032005
Harrison [48]20003.767220032004
Espino [49]20013.767220032004
Pollack [50]19983.767220032004
Mayaud [51]19973.672719992004
Rotz [52]20023.560720032005
Garcia [53]19983.426620002001
Table
Table 5. The top 25 articles on Syndromic Surveillance with citation burst.
Table 5. The top 25 articles on Syndromic Surveillance with citation burst.
TermsYearStrengthBeginEnd
Zika Virus199336.73720162018
Pandemic Influenza199323.397220102013
Open Access Article199322.415320152018
Social Media199316.763920132018
Big Data199316.093920152018
Seasonal Influenza199315.697920092012
Spatial Distribution199315.590620142016
Principal Finding199315.354120102014
Google Trends199315.31620162018
General Practitioners199315.114920072011
Dengue Virus199314.888520162018
Phylogenetic Analysis199313.179820162018
Degrees C199313.119920132016
Previous Study199313.06620162018
HIV Infection199312.695520072010
Mean Age199312.442520162018
Syndromic Surveillance199312.347920012006
Influenza B199312.171220152016
H1N1 Pandemic199311.506520122013
Influenza Pandemic199310.343920112014
Disease Outbreaks199310.244520092012
High Prevalence199310.167820122014
Influenza Season19939.800420082012
Neisseria Gonorrhoeae19939.582420132014
Antimicrobial Resistance19939.347720132014
Table
Table 6. Articles of the major clusters and their citing papers.
Table 6. Articles of the major clusters and their citing papers.
ClusterIntellectual BaseResearch Fronts
IDMITotalPapers in the ClusterTotalCiting Papers
10Disease Surveillance98[2,17,19,21,22,23,25,26,27,28,31,32,33,35,36,41,45,46,49,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121].16[24,25,27,28,29,31,63,67,84,97,98,122,123,124,125,126].
1EWMA Control Charts48[41,58,61,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165].25[31,63,90,113,125,141,160,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183].
2Crowd-Sourced42[13,14,15,25,55,56,59,60,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217].34[29,30,62,183,192,197,200,216,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239].
EWMA: Exponentially Weighted Moving Average, MI: Cluster Labels using Mutual Information

Share and Cite

MDPI and ACS Style

Musa, I.; Park, H.W.; Munkhdalai, L.; Ryu, K.H. Global Research on Syndromic Surveillance from 1993 to 2017: Bibliometric Analysis and Visualization. Sustainability 2018, 10, 3414. https://doi.org/10.3390/su10103414

AMA Style

Musa I, Park HW, Munkhdalai L, Ryu KH. Global Research on Syndromic Surveillance from 1993 to 2017: Bibliometric Analysis and Visualization. Sustainability. 2018; 10(10):3414. https://doi.org/10.3390/su10103414

Chicago/Turabian Style

Musa, Ibrahim, Hyun Woo Park, Lkhagvadorj Munkhdalai, and Keun Ho Ryu. 2018. "Global Research on Syndromic Surveillance from 1993 to 2017: Bibliometric Analysis and Visualization" Sustainability 10, no. 10: 3414. https://doi.org/10.3390/su10103414

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop