Machine Learning Approach to Classify Rain Type Based on Thies Disdrometers and Cloud Observations
Abstract
:1. Introduction
- How do classification methods designed for other disdrometer types perform when applied to Thies disdrometer measurements?
- Can we achieve better classification performance by tuning the decision boundary for each method?
- Which rain microstructure parameters are superior as rain type classifiers?
- Do machine-learning techniques support a better classification?
2. Materials and Methods
2.1. Data Sources and Tools
2.2. Cloud Observations
2.3. Thies Disdrometer and Extraction of Rain DSD
2.4. Prior Rain Type Classification
2.5. Rain Microstructure-Based Classification Methods
2.6. Indicators of the Classification Performance
2.7. Advanced Predictive Models
2.7.1. Linear Discriminate Analysis (LDA)
2.7.2. K Nearest Neighbor (KNN)
2.7.3. Naïve Bayes (NB)
2.7.4. Conditional Trees (Ctree)
2.7.5. Random Forests (RF)
2.8. Selecting Rain DSD Parameters
- High computational costs,
- The risk of overfitting the training set,
- Non-informative features that negatively affect some models [60], and
- Constructed models that are difficult to interpret.
- 1-
- The features are clustered into X groups (hierarchal clustering). Each group contains a few correlated features.
- 2-
- One feature is chosen out of each cluster. The chosen feature is the one with the highest AUC value, where AUC is the area under the receiver operating characteristic curve [67]. The AUC value is a measure of the capability of each feature to separate the classes.
3. Results
3.1. Performance of the Classification Methods from the Literature
3.2. Feature Selection
3.3. Performance of the ML Methods
4. Discussion
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Steiner, M.; Smith, J.A. Convective versus stratiform rainfall: An ice-microphysical and kinematic conceptual model. Atmos. Res. 1998, 47–48, 317–326. [Google Scholar] [CrossRef]
- Niu, S.; Jia, X.; Sang, J.; Liu, X.; Lu, C.; Liu, Y. Distributions of raindrop sizes and fall velocities in a semiarid plateau climate: Convective versus stratiform rains. J. Appl. Meteorol. Climatol. 2010, 49, 632–645. [Google Scholar] [CrossRef]
- Islam, T.; Rico-Ramirez, M.A.; Thurai, M.; Han, D. Characteristics of raindrop spectra as normalized gamma distribution from a Joss–Waldvogel disdrometer. Atmos. Res. 2012, 108, 57–73. [Google Scholar] [CrossRef]
- World Meteorological Organization. International Meteorological Vocabulary, 2nd ed.; Secretariat of the World Meteorological Organization: Geneva, Switzerland, 1992; ISBN 9789263021823. [Google Scholar]
- Thompson, E.J.; Rutledge, S.A.; Dolan, B.; Thurai, M. Drop size distributions and radar observations of convective and stratiform rain over the equatorial Indian and West Pacific Oceans. J. Atmos. Sci. 2015, 72, 4091–4125. [Google Scholar] [CrossRef]
- Houze, R.A. Stratiform precipitation in regions of convection: A meteorological paradox? Bull. Am. Meteorol. Soc. 1997, 78, 2179–2196. [Google Scholar] [CrossRef]
- Ferrier, B.S.; Tao, W.-K.; Simpson, J. A double-moment multiple-phase four-class bulk ice scheme. Part II: Simulations of convective storms in different large-scale environments and comparisons with other bulk parameterizations. J. Atmos. Sci. 1995, 52, 1001–1033. [Google Scholar] [CrossRef]
- Ghada, W.; Buras, A.; Lüpke, M.; Schunk, C.; Menzel, A. Rain microstructure parameters vary with large-scale weather conditions in Lausanne, Switzerland. Remote Sens. 2018, 10, 811. [Google Scholar] [CrossRef]
- Bringi, V.N.; Chandrasekar, V.; Hubbert, J.; Gorgucci, E.; Randeu, W.L.; Schoenhuber, M. Raindrop size distribution in different climatic regimes from disdrometer and dual-polarized radar analysis. J. Atmos. Sci. 2003, 60, 354–365. [Google Scholar] [CrossRef]
- Llasat, M.-C. An objective classification of rainfall events on the basis of their convective features: Application to rainfall intensity in the northeast of Spain. Int. J. Climatol. 2001, 21, 1385–1400. [Google Scholar] [CrossRef]
- Bukovčić, P.; Zrnić, D.; Zhang, G. Convective–stratiform separation using video disdrometer observations in central Oklahoma—The Bayesian approach. Atmos. Res. 2015, 155, 176–191. [Google Scholar] [CrossRef]
- Berg, P.; Moseley, C.; Haerter, J.O. Strong increase in convective precipitation in response to higher temperatures. Nat. Geosci. 2013, 6, 181–185. [Google Scholar] [CrossRef]
- Langer, I.; Reimer, E. Separation of convective and stratiform precipitation for a precipitation analysis of the local model of the German weather service. Adv. Geosci. 2007, 10, 159–165. [Google Scholar] [CrossRef]
- Houze, R.A. Cloud Dynamics, 2nd ed.; Elsevier: Amsterdam, NY, USA, 2014; ISBN 978-0-12-374266-7. [Google Scholar]
- Rulfová, Z.; Kyselý, J. Disaggregating convective and stratiform precipitation from station weather data. Atmos. Res. 2013, 134, 100–115. [Google Scholar] [CrossRef]
- Williams, C.R.; Ecklund, W.L.; Gage, K.S. Classification of precipitating clouds in the tropics using 915-MHz wind profilers. J. Atmos. Ocean. Technol. 1995, 12, 996–1012. [Google Scholar] [CrossRef]
- Steiner, M.; Houze, R.A.; Yuter, S.E. Climatological characterization of three-dimensional storm structure from operational radar and rain gauge data. J. Appl. Meteorol. 1995, 34, 1978–2007. [Google Scholar] [CrossRef]
- Churchill, D.D.; Houze, R.A. Development and structure of winter monsoon cloud clusters on 10 December 1978. J. Atmos. Sci. 1984, 41, 933–960. [Google Scholar] [CrossRef]
- Kummerow, C.; Hakkarinen, I.M.; Pierce, H.F.; Weinman, J.A. Determination of precipitation profiles from airborne passive microwave radiometric measurements. J. Atmos. Ocean. Technol. 1991, 8, 148–158. [Google Scholar] [CrossRef]
- Berendes, T.A.; Mecikalski, J.R.; MacKenzie, W.M.; Bedka, K.M.; Nair, U.S. Convective cloud identification and classification in daytime satellite imagery using standard deviation limited adaptive clustering. J. Geophys. Res. 2008, 113, 909. [Google Scholar] [CrossRef]
- Anagnostou, E.N.; Kummerow, C. Stratiform and Convective Classification of rainfall using SSM/I 85-GHz brightness temperature observations. J. Atmos. Ocean. Technol. 1997, 14, 570–575. [Google Scholar] [CrossRef]
- Adler, R.F.; Negri, A.J. A satellite infrared technique to estimate tropical convective and stratiform rainfall. J. Appl. Meteorol. 1988, 27, 30–51. [Google Scholar] [CrossRef]
- Yuter, S.E.; Houze, R.A. Measurements of raindrop size distributions over the pacific warm pool and implications for Z–R relations. J. Appl. Meteorol. 1997, 36, 847–867. [Google Scholar] [CrossRef]
- Thurai, M.; Gatlin, P.N.; Bringi, V.N. Separating stratiform and convective rain types based on the drop size distribution characteristics using 2D video disdrometer data. Atmos. Res. 2016, 169, 416–423. [Google Scholar] [CrossRef]
- Tokay, A.; Short, D.A. Evidence from tropical raindrop spectra of the origin of rain from stratiform versus convective clouds. J. Appl. Meteorol. 1996, 35, 355–371. [Google Scholar] [CrossRef]
- Caracciolo, C.; Prodi, F.; Battaglia, A.; Porcu, F. Analysis of the moments and parameters of a gamma DSD to infer precipitation properties: A convective stratiform discrimination algorithm. Atmos. Res. 2006, 80, 165–186. [Google Scholar] [CrossRef]
- Caracciolo, C.; Porcù, F.; Prodi, F. Precipitation classification at mid-latitudes in terms of drop size distribution parameters. Adv. Geosci. 2008, 16, 11–17. [Google Scholar] [CrossRef] [Green Version]
- Bringi, V.N.; Williams, C.R.; Thurai, M.; May, P.T. Using dual-polarized radar and dual-frequency profiler for DSD characterization: A case study from Darwin, Australia. J. Atmos. Ocean. Technol. 2009, 26, 2107–2122. [Google Scholar] [CrossRef]
- Uijlenhoet, R.; Steiner, M.; Smith, J.A. Variability of raindrop size distributions in a squall line and implications for radar rainfall estimation. J. Hydrometeorol. 2003, 4, 43–61. [Google Scholar] [CrossRef]
- You, C.-H.; Lee, D.-I.; Kang, M.-Y.; Kim, H.-J. Classification of rain types using drop size distributions and polarimetric radar: Case study of a 2014 flooding event in Korea. Atmos. Res. 2016, 181, 211–219. [Google Scholar] [CrossRef]
- Steinert, J.; Tracksdorf, P. On the Verification of DWD’s Polarimetric Hydrometeor Classification and Improved QPE. Available online: https://ams.confex.com/ams/37RADAR/webprogram/Handout/Paper275625/ams2015_poster22_steinert_verification.pdf (accessed on 29 April 2019).
- Adirosi, E.; Roberto, N.; Montopoli, M.; Gorgucci, E.; Baldini, L. Influence of disdrometer type on weather radar algorithms from measured DSD: Application to Italian climatology. Atmosphere 2018, 9, 360. [Google Scholar] [CrossRef]
- Angulo-Martínez, M.; Beguería, S.; Latorre, B.; Fernández-Raga, M. Comparison of precipitation measurements by OTT Parsivel2 and Thies LPM optical disdrometers. Hydrol. Earth Syst. Sci. 2018, 22, 2811–2837. [Google Scholar] [CrossRef]
- Historical Records of Cloud Type Observations in Germany. Available online: ftp://ftp-cdc.dwd.de/pub/CDC/observations_germany/climate/hourly/cloud_type/historical/ (accessed on 6 May 2019).
- R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria, 2018. Available online: https://www.R-project.org/ (accessed on 6 May 2019).
- RStudio Team. RStudio: Integrated Development Environment for R. Boston, MA, USA, 2016. Available online: http://www.rstudio.com/ (accessed on 6 May 2019).
- Cooper, N. Reader: Suite of Functions to Flexibly Read Data from Files. 2017. Available online: https://CRAN.R-project.org/package=reader (accessed on 6 May 2019).
- Wickham, H.; François, R.; Henry, L.; Müller, K. dplyr: A Grammar of Data Manipulation. 2018. Available online: https://CRAN.R-project.org/package=dplyr (accessed on 6 May 2019).
- Wickham, H. Reshaping data with the reshape package. J. Stat. Soft. 2007, 21, 1–20. [Google Scholar] [CrossRef]
- Grolemund, G.; Wickham, H. Dates and times made easy with lubridate. J. Stat. Soft. 2011, 40, 1–25. [Google Scholar] [CrossRef]
- Wickham, H. Stringr: Simple, Consistent Wrappers for Common String Operations. 2018. Available online: https://CRAN.R-project.org/package=stringr (accessed on 6 May 2019).
- Wickham, H. Ggplot2. Elegant Graphics for Data Analysis, 2nd ed.; Springer: Cham, Switzerland, 2016; ISBN 978-3-319-24277-4. [Google Scholar]
- Wilke, C. Cowplot: Streamlined Plot Theme and Plot Annotations for ’ggplot2’. 2018. Available online: https://CRAN.R-project.org/package=cowplot (accessed on 6 May 2019).
- Kuhn, M. Building predictive models in R using the caret package. J. Stat. Soft. 2008, 28. [Google Scholar] [CrossRef]
- Meyer, D.; Dimitriadou, E.; Hornik, K.; Weingessel, A.; Leisch, F. e1071: Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien, 2018. Available online: https://CRAN.R-project.org/package=e1071 (accessed on 6 May 2019).
- Microsoft Corporation; Weston, S. doSNOW: For each Parallel Adaptor for the ’snow’ Package. 2017. Available online: https://CRAN.R-project.org/package=doSNOW (accessed on 6 May 2019).
- Zeileis, A.; Grothendieck, G. Zoo: S3 infrastructure for regular and irregular time series. J. Stat. Soft. 2005, 14, 1–27. [Google Scholar] [CrossRef]
- Venables, W.N.; Ripley, B.D. Modern Applied Statistics with S, 4th ed.; Springer: New York, NY, USA, 2002; ISBN 0-387-95457-0. [Google Scholar]
- Taiyun, W.; Simko, V. R package “corrplot”: Visualization of a Correlation Matrix. 2017. Available online: https://github.com/taiyun/corrplot (accessed on 6 May 2019).
- Robin, X.; Turck, N.; Hainard, A.; Tiberti, N.; Lisacek, F.; Sanchez, J.-C.; Müller, M. pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform. 2011, 12, 77. [Google Scholar] [CrossRef] [PubMed]
- World Meteorological Organization. Principles of Cloud Classification. Available online: https://cloudatlas.wmo.int/principles-of-cloud-classification-genera.html (accessed on 2 May 2019).
- Deutscher Wetterdienst. Datenprüfung/Qualitatskontrolle. Available online: https://cdc.dwd.de/catalogue/Klimadaten_QC.htm (accessed on 2 May 2019).
- Thies Clima. Instructions for use. Laser Precipitation Monitor 5.4110. xx. x00 V2. 4x STD, 2007. Available online: https://www.biral.com/wp-content/uploads/2015/01/5.4110.xx_.xxx_.pdf (accessed on 6 February 2019).
- Friedrich, K.; Kalina, E.A.; Masters, F.J.; Lopez, C.R. Drop-size distributions in thunderstorms measured by optical disdrometers during VORTEX2. Mon. Weather Rev. 2013, 141, 1182–1203. [Google Scholar] [CrossRef]
- Marzuki, M.; Randeu, W.L.; Schönhuber, M.; Bringi, V.N.; Kozu, T.; Shimomai, T. Raindrop size distribution parameters of distrometer data with different bin sizes. IEEE Trans. Geosci. Remote Sens. 2010, 48, 3075–3080. [Google Scholar] [CrossRef]
- Kanofsky, L.; Chilson, P. An analysis of errors in drop size distribution retrievals and rain bulk parameters with a UHF wind profiling radar and a two-dimensional video disdrometer. J. Atmos. Ocean. Technol. 2008, 25, 2282–2292. [Google Scholar] [CrossRef]
- Chen, B.; Wang, J.; Gong, D. Raindrop size distribution in a midlatitude continental squall line measured by Thies optical disdrometers over East China. J. Appl. Meteorol. Climatol. 2016, 55, 621–634. [Google Scholar] [CrossRef]
- Testud, J.; Oury, S.; Black, R.A.; Amayenc, P.; Dou, X. The concept of “normalized” distribution to describe raindrop spectra: A tool for cloud physics and cloud remote sensing. J. Appl. Meteorol. 2001, 40, 1118–1140. [Google Scholar] [CrossRef]
- Ochou, A.D.; Zahiri, E.-P.; Bamba, B.; Koffi, M. Understanding the variability of Z-R relationships caused by natural variations in raindrop size distributions (DSD): Implication of drop size and number. ACS 2011, 1, 147–164. [Google Scholar] [CrossRef]
- Kuhn, M.; Johnson, K. Applied Predictive Modeling; Corrected at 5th printing; Springer: New York, NY, USA, 2013; ISBN 1461468485. [Google Scholar]
- Chinchor, N. MUC-4 evaluation metrics. In Proceedings of the 4th Conference on Message understanding, McLean, Virginia, 16–18 June 1992; p. 22, ISBN 1558602739. [Google Scholar]
- Deepika, J.; Senthil, T.; Rajan, C.; Surendar, A. Machine learning algorithms: A background artifact. IJET 2017, 7, 143. [Google Scholar] [CrossRef]
- Fisher, R.A. The statistical utilization of multiple measurements. Ann. Eugen. 1938, 8, 376–386. [Google Scholar] [CrossRef]
- Press, L.P.a.I.S. 605215, 2009, ISBN 1694-0784. Available online: http://www.ijcsi.org/issues.php (accessed on 6 May 2019).
- Burger, S.V. Introduction to Machine Learning with R. Rigorous Mathematical Analysis, 1st ed.; O’Reilly: Sebastopol, CA, USA, 2018; ISBN 1491976446. [Google Scholar]
- Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef]
- Hanley, J.A.; McNeil, B.J. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143, 29–36. [Google Scholar] [CrossRef]
Cloud Type (Genera) | Abbreviation | Expected Rain Type |
---|---|---|
Cirrus | CI | - |
Cirrocumulus | CC | - |
Cirrostratus | CS | - |
Altocumulus | AC | - |
Altostratus | AS | - |
Nimbostratus | NS | Stratiform |
Stratocumulus | SC | - |
Stratus | ST | Stratiform |
Cumulus | CU | Convective |
Cumulonimbus | CB | Convective |
Abbreviation | Unit | Parameter Name and Relevant Reference 1 | |
---|---|---|---|
R | mm·h−1 | rain intensity [8] | |
Z | dBZ | reflectivity [8] | |
Dm | mm | mass weighted diameter [55] | |
D0 | mm | median volume diameter [56] | |
sd_D | mm | instantaneous (1 min) standard deviation of drop size [2] | |
sd_V | m·h−1 | instantaneous standard deviation of drop velocities [2] | |
Nt | drop·m−3 | total number of drops per cubic meter [57] | |
Nw_Tes | mm−1·m−3 | normalized number of drops [58] | |
Nw_Br | mm−1·m−3 | normalized number of drops [28] | |
logNw | Nw: mm−1·m−3 | logNw = log10(NW_Br) | |
D0_Nt | mm.m3·drop−1 | D0/Nt [59] | |
Lambda_TS | mm−1 | slope of fitted gamma distribution [25] | |
logLambda | Lambda: mm−1 | logLambda = log10(Lambda_TS) [25] | |
mu_TS | - | shape of fitted gamma distribution [25] | |
N0_TS | mm−1−m·m−3 | intercept of fitted gamma distribution [25] | |
logN0 | N0_TS: mm−1−m·m−3 | logN0 = log10(N0_TS) [25] | |
Lambda_Ca06 | mm−1 | slope of fitted gamma distribution [26] | |
mu_Ca06 | - | shape of fitted gamma distribution [26] | |
N0_Ca06 | mm−1−m·m−3 | intercept of fitted gamma distribution [26] | |
Lambda_Ca08 | mm−1 | slope of fitted gamma distribution [27] | |
N0_Ca08 | mm−1−m·m−3 | intercept of fitted gamma distribution [27] | |
Nt_4R | (Drop·m−3)0.25 | 4th root of Nt [11] | |
sd_R_10 | mm·h−1 | | sd_XX_YY: standard deviations of XX over YY minutes [11] |
sd_Dm_10 | mm | ||
sd_D0_10 | mm | ||
sd_Nt_10 | drop | ||
sd_R_30 | mm·h−1 | ||
sd_Dm_30 | mm | ||
sd_D0_30 | mm | ||
sd_Nt_30 | drop | ||
sd_log10(Nt)_30 | - | ||
sd_log10(R)_30 | - | ||
sd_log10(Nt)_10 | - | ||
sd_log10(R)_10 | - |
Method | Reference | Parameters: y ~ x | Decision Boundary |
---|---|---|---|
TS_a | [25] | N0_TS ~ R | convective region above the decision boundary |
TS_b | [25] | Lambda_TS ~ R | convective region above the decision boundary |
Ca_06 | [26] | Lambda_Ca06 ~ mu_Ca06 | convective region below the decision boundary |
Ca_08 | [27] | Lambda_Ca08 ~ N0_Ca08 | convective region below the decision boundary |
Br_09 | [28] | Nw_Br ~ D0 | convective region above the decision boundary |
You_16 | [30] | Nw_Br ~ D0 | > convective region above the decision boundary |
Bu_15 | [11] | Z, Nt_4R, sd_log10(Nt)_30, sd_log10(R)_30 | - |
Prediction of Rain Type | Observed Rain Type * | |
---|---|---|
Convective | Stratiform | |
Convective | True Positive (TP) | False Positive (FP) |
Stratiform | False Negative (FN) | True Negative (TN) |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ghada, W.; Estrella, N.; Menzel, A. Machine Learning Approach to Classify Rain Type Based on Thies Disdrometers and Cloud Observations. Atmosphere 2019, 10, 251. https://doi.org/10.3390/atmos10050251
Ghada W, Estrella N, Menzel A. Machine Learning Approach to Classify Rain Type Based on Thies Disdrometers and Cloud Observations. Atmosphere. 2019; 10(5):251. https://doi.org/10.3390/atmos10050251
Chicago/Turabian StyleGhada, Wael, Nicole Estrella, and Annette Menzel. 2019. "Machine Learning Approach to Classify Rain Type Based on Thies Disdrometers and Cloud Observations" Atmosphere 10, no. 5: 251. https://doi.org/10.3390/atmos10050251