Artificial Intelligence-Based Prediction of Key Textural Properties from LUCAS and ICRAF Spectral Libraries
Abstract
:1. Introduction
2. Materials and Methods
2.1. Description of Soil Datasets
2.2. Pre-Processing of Raw Spectra
2.3. Machine Learning Algorithms for the Prediction of Clay, Silt and Sand Fractions
2.4. Development of Predictive Modeling Process
2.5. Machine Learning Models for Predicting Fine (G1), Medium (G2), and Coarse (G3) Textural Groups
2.6. Data Analysis Platform
2.7. Model Evaluation
3. Results and Discussion
3.1. Performance of Pre-Processing Techniques
3.2. Soil Fractions Prediction and Bias Correction
3.3. Prediction of Textural Groups
4. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Kopittke, P.M.; Menzies, N.W.; Wang, P.; McKenna, B.A.; Lombi, E. Soil and the intensification of agriculture for global food security. Environ. Int. 2019, 132, 105078. [Google Scholar] [CrossRef] [PubMed]
- Saiz-Rubio, V.; Rovira-Más, F. From smart farming towards agriculture 5.0: A review on crop data management. Agronomy 2020, 10, 207. [Google Scholar] [CrossRef] [Green Version]
- Delgado, A.; Gómez, J.A. The Soil. Physical, Chemical and Biological Properties. In Principles of Agronomy for Sustainable Agriculture; Springer: Cham, Germany, 2016; pp. 15–26. [Google Scholar]
- Gee, G.W.; Or, D. 2.4 Particle-Size Analysis. In Methods of Soil Analysis: Part 4 Physical Methods, 5.4; Soil Science Society of America, Inc.: Madison, WI, USA, 2002; pp. 255–293. [Google Scholar]
- Tümsavaş, Z.; Tekin, Y.; Ulusoy, Y.; Mouazen, A.M. Prediction and mapping of soil clay and sand contents using visible and near-infrared spectroscopy. Biosyst. Eng. 2019, 177, 90–100. [Google Scholar] [CrossRef]
- Shahandeh, H.; Wright, A.L.; Hons, F.M. Use of soil nitrogen parameters and texture for spatially-variable nitrogen fertilization. Precis. Agric. 2011, 12, 146–163. [Google Scholar] [CrossRef]
- Bönecke, E.; Meyer, S.; Vogel, S.; Schröter, I.; Gebbers, R.; Kling, C.; Kramer, E.; Lück, K.; Nagel, A.; Philipp, G.; et al. Guidelines for precise lime management based on high-resolution soil pH, texture and SOM maps generated from proximal soil sensing data. Precis. Agric. 2020, 22, 493–523. [Google Scholar] [CrossRef]
- Vories, E.; O’Shaughnessy, S.; Sudduth, K.; Evett, S.; Andrade, M.; Drummond, S. Comparison of precision and conventional irrigation management of cotton and impact of soil texture. Precis. Agric. 2020, 22, 414–431. [Google Scholar] [CrossRef]
- Khiari, L. Échantillonnage Conventionnel des Sols Agricoles au Québec; Centre de Référence en Agriculture et Agroalimentaire du Québec: Québec, QC, Canada, 2014; p. 20. [Google Scholar]
- Chelabi, H.; Khiari, L.; Gallichand, J.; Joseph, C.A. Soil sample preparation techniques on routine analyses in Quebec affect lime and fertilizer recommendations. Can. J. Soil Sci. 2016, 96, 244–255. [Google Scholar] [CrossRef] [Green Version]
- Metwally, M.S.; Shaddad, S.M.; Liu, M.; Yao, R.J.; Abdo, A.I.; Li, P.; Jiao, J.; Chen, X. Soil properties spatial variability and delineation of site-specific management zones based on soil fertility using fuzzy clustering in a hilly field in Jianyang, Sichuan, China. Sustainability 2019, 11, 7084. [Google Scholar] [CrossRef] [Green Version]
- Blake, G.R. Physical and mineralogical properties, including statistics of measurement and sampling. In Methods of Soil Analysis: Part 1 Physical and Mineralogical Properties, Including Statistics of Measurement and Sampling, 9.1; Soil Science Society of America, Inc.: Madison, WI, USA, 1965. [Google Scholar]
- Viscarra Rossel, R.A.; Lark, R.M. Improved analysis and modelling of soil diffuse reflectance spectra using wavelets. Eur. J. Soil Sci. 2009, 60, 453–464. [Google Scholar] [CrossRef]
- Allo, M.; Todoroff, P.; Jameux, M.; Stern, M.; Paulin, L.; Albrecht, A. Prediction of tropical volcanic soil organic carbon stocks by visible-near- and mid-infrared spectroscopy. Catena 2020, 189, 104452. [Google Scholar] [CrossRef]
- Davari, M.; Karimi, S.A.; Bahrami, H.A.; Hossaini, S.M.T.; Fahmideh, S. Simultaneous prediction of several soil properties related to engineering uses based on laboratory Vis-NIR reflectance spectroscopy. Catena 2021, 197, 104987. [Google Scholar] [CrossRef]
- Coblinski, J.A.; Giasson, É.; Demattê, J.A.M.; Dotto, A.C.; Costa, J.J.F.; Vašát, R. Prediction of soil texture classes through different wavelength regions of reflectance spectroscopy at various soil depths. Catena 2020, 189, 104485. [Google Scholar] [CrossRef]
- Katuwal, S.; Knadel, M.; Norgaard, T.; Moldrup, P.; Greve, M.H.; de Jonge, L.W. Predicting the dry bulk density of soils across Denmark: Comparison of single-parameter, multi-parameter, and vis–NIR based models. Geoderma 2020, 361, 114080. [Google Scholar] [CrossRef]
- Stevens, A.; Nocita, M.; Tóth, G.; Montanarella, L.; van Wesemael, B. Prediction of Soil Organic Carbon at the European Scale by Visible and Near InfraRed Reflectance Spectroscopy. PLoS ONE 2013, 8, e66409. [Google Scholar] [CrossRef]
- Tóth, G.; Jones, A.; Montanarella, L. The LUCAS topsoil database and derived information on the regional variability of cropland topsoil properties in the European Union. Environ. Monit. Assess. 2013, 185, 7409–7425. [Google Scholar] [CrossRef]
- Garrity, D.; Bindraban, P. A Globally Distributed Soil Spectral Library Visible Near Infrared Diffuse Reflectance Spectra; ICRAF (International Center for Research in Agroforestry): Nairobi, Kenya; ISRIC (World Soil Information): Wageningen, The Netherlands; Spectral Library: Nairobi, Kenya, 2004. [Google Scholar]
- Reeuwijk, L. Procedures for Soil Analysis; Tech. Pap.; ISRIC: Wageningen, The Netherlands, 2002. [Google Scholar]
- FOSS. NIR Spectroscopy: A Guide to Near-Infrared Spectroscopic Analysis of Industrial Manufacturing Processes; Metrohm: Herisau, Switzerland, 2009. [Google Scholar]
- Stenvens, A.; Ramirez-López, L. Miscellaneous Functions for Processing and Sample Selection of vis-NIR Diffuse Reflectance Data. 2014. CRAN ‘Prospectr’ R; An Introduction to the Prospectr Package. R Package Vignette R Version 0.1. Available online: https://cran.r-project.org/web/packages/prospectr/vignettes/prospectr.html (accessed on 28 July 2021).
- LeDell, E.; Gill, N.; Aiello, S.; Fu, A.; Candel, A.; Click, C.; Kraljevic, T.; Nykodym, T.; Aboyoun, P.; Kurka, M.; et al. R Interface for “H2O”. 2019. Available online: https://cran.r-project.org/web/packages/h2o/index.html (accessed on 28 July 2021).
- R Core Team. R: A Language and Environment for Statistical Computing; R Foundation Statistical Computing: Vienna, Austria, 2019; p. 201. [Google Scholar]
- Rinnan, Å.; van den Berg, F.; Engelsen, S.B. Review of the most common pre-processing techniques for near-infrared spectra. TrAC Trends Anal. Chem. 2009, 28, 1201–1222. [Google Scholar] [CrossRef]
- Duckworth, J. Mathematical Data Preprocessing. In Near-Infrared Spectroscopy in Agriculture; Soil Science Society of America, Inc.: Madison, WI, USA, 2015; Volume 44, pp. 113–132. [Google Scholar]
- Clark, R.N.; Roush, T.L. Reflectance spectroscopy: Quantitative analysis techniques for remote sensing applications. J. Geophys. Res. 1984, 89, 6329–6340. [Google Scholar] [CrossRef]
- Dhanoa, M.S.; Lister, S.J.; Sanderson, R.; Barnes, R.J. The Link between Multiplicative Scatter Correction (MSC) and Standard Normal Variate (SNV) Transformations of NIR Spectra. J. Near Infrared Spectrosc. 1994, 2, 43–47. [Google Scholar] [CrossRef]
- Breiman, L. Random Forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef] [Green Version]
- Breiman, L. Bagging predictors. Mach. Learn. 1996, 24, 123–140. [Google Scholar] [CrossRef] [Green Version]
- Prokhorenkova, L.; Gusev, G.; Vorobev, A.; Dorogush, A.V.; Gulin, A. Catboost: Unbiased boosting with categorical features. In Proceedings of the Advances in Neural Information Processing Systems. arXiv 2017, arXiv:1706.09516. [Google Scholar]
- Chen, T.; Guestrin, C. XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Fr. Calif. 2016, 1, 1–4. [Google Scholar]
- Fan, J.; Wang, X.; Wu, L.; Zhou, H.; Zhang, F.; Yu, X.; Lu, X.; Xiang, Y. Comparison of Support Vector Machine and Extreme Gradient Boosting for predicting daily global solar radiation using temperature and precipitation in humid subtropical climates: A case study in China. Energy Convers. Manag. 2018, 164, 102–111. [Google Scholar] [CrossRef]
- Barbur, V.A.; Montgomery, D.C.; Peck, E.A. Introduction to Linear Regression Analysis. J. R. Stat. Soc. 1994, 43. [Google Scholar] [CrossRef]
- Murtagh, F. Multilayer perceptrons for classification and regression. Neurocomputing 1991, 2, 183–197. [Google Scholar] [CrossRef]
- Rumelhart, D.E.; Hinton, G.E.; Williams, R.J. Learning representations by back-propagating errors. Nature 1986, 323, 533–536. [Google Scholar] [CrossRef]
- Liu, B.; Zhao, Q.; Jin, Y.; Shen, J.; Li, C. Application of combined model of stepwise regression analysis and artificial neural network in data calibration of miniature air quality detector. Sci. Rep. 2021, 11, 1–12. [Google Scholar] [CrossRef]
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
- Song, J. Bias corrections for Random Forest in regression using residual rotation. J. Korean Stat. Soc. 2015, 44, 321–326. [Google Scholar] [CrossRef]
- Ke, G.; Meng, Q.; Finley, T.; Wang, T.; Chen, W.; Ma, W.; Ye, Q.; Liu, T.Y. LightGBM: A highly efficient gradient boosting decision tree. In Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA, 4 December 2017; Volume 30, pp. 3146–3154. [Google Scholar]
- Zhao, Z.; Chow, T.L.; Rees, H.W.; Yang, Q.; Xing, Z.; Meng, F.R. Predict soil texture distributions using an artificial neural network model. Comput. Electron. Agric. 2009, 65, 36–48. [Google Scholar] [CrossRef]
- Zhang, S.-W.; Shen, C.-Y.; Chen, X.-Y.; Ye, H.-C.; Huang, Y.-F.; Lai, S. Spatial Interpolation of Soil Texture Using Compositional Kriging and Regression Kriging with Consideration of the Characteristics of Compositional Data and Environment Variables. J. Integr. Agric. 2013, 12, 1673–1683. [Google Scholar] [CrossRef] [Green Version]
- McKinney, W. Python for Data Analysis: Data Wrangling with Pandas, NumPy, and IPython; O’Reilly Media Inc.: Sebastopol, CA, USA, 2018; Volume 71. [Google Scholar]
- McKinney, W. pandas: A Foundational Python Library for Data Analysis and Statistics. Python High Perform. Sci. Comput. 2011, 14, 1–9. [Google Scholar]
- Stevens, R.J.; Poppe, K.K. Validation of clinical prediction models: What does the “calibration slope” really measure? J. Clin. Epidemiol. 2020, 118, 93–99. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Li, L.; Lu, J.; Wang, S.; Ma, Y.; Wei, Q.; Li, X.; Cong, R.; Ren, T. Methods for estimating leaf nitrogen concentration of winter oilseed rape (Brassica napus L.) using in situ leaf spectroscopy. Ind. Crops Prod. 2016, 91, 194–204. [Google Scholar] [CrossRef]
- Waskom, M. Seaborn: Statistical data visualization. J. Open Source Softw. 2021, 6, 3021. [Google Scholar] [CrossRef]
- Ben-Dor, E.; Inbar, Y.; Chen, Y. The reflectance spectra of organic matter in the visible near-infrared and short wave infrared region (400-2500 nm) during a controlled decomposition process. Remote Sens. Environ. 1997, 61, 1–15. [Google Scholar] [CrossRef]
- Gerighausen, H.; Menz, G.; Kaufmann, H. Spatially explicit estimation of clay and organic carbon content in agricultural soils using multi-annual imaging spectroscopy data. Appl. Environ. Soil Sci. 2012, 2012, 868090. [Google Scholar] [CrossRef]
- Cozzolino, D.; Morón, A. The potential of near-infrared reflectance spectroscopy to analyse soil chemical and physical characteristics. J. Agric. Sci. 2003, 140, 65–71. [Google Scholar] [CrossRef]
- Chang, C.-W.; Laird, D.A.; Mausbach, M.J.; Hurburgh, C.R. Near-Infrared Reflectance Spectroscopy-Principal Components Regression Analyses of Soil Properties. Soil Sci. Soc. Am. J. 2001, 65, 480. [Google Scholar] [CrossRef] [Green Version]
- Ahmadi, A.; Emami, M.; Daccache, A.; He, L. Soil Properties Prediction for Precision Agriculture Using Visible and Near-Infrared Spectroscopy: A Systematic Review and Meta-Analysis. Agronomy 2021, 11, 433. [Google Scholar] [CrossRef]
- Yang, X.; Zhang, Q.; Li, X.; Jia, X.; Wei, X.; Shao, M. Determination of Soil Texture by Laser Diffraction Method. Soil Sci. Soc. Am. J. 2015, 79, 1556–1566. [Google Scholar] [CrossRef] [Green Version]
- UNIL Préparation Pour la Granulométrie Laser. l’Institut des Dynamiques de la Surface Terrestre. 2020. Available online: https://www.unil.ch/idyst/files/live/sites/idyst/files/shared/Analytical%20platform/PDF_protocole/french/sols%20et%20s%C3%A9diments/Preparation%20pour%20la%20Granulometrie%20Laser%202.0.pdf (accessed on 28 July 2021).
- Dharumarajan, S.; Hegde, R. Digital mapping of soil texture classes using Random Forest classification algorithm. Soil Use Manag. 2020. [Google Scholar] [CrossRef]
- Beretta, A.N.; Silbermann, A.V.; Paladino, L.; Torres, D.; Bassahun, D.; Musselli, R.; García-Lamohte, A. Soil texture analyses using a hydrometer: Modification of the Bouyoucos method. Cienc. Investig. Agrar. 2014, 41, 263–271. [Google Scholar] [CrossRef] [Green Version]
- Pirie, A.; Singh, B.; Islam, K. Ultra-violet, visible, near-infrared, and mid-infrared diffuse reflectance spectroscopic techniques to predict several soil properties. Aust. J. Soil Res. 2005, 43, 713–721. [Google Scholar] [CrossRef]
- Ramirez-Lopez, L.; Wadoux, A.M.J.C.; Franceschini, M.H.D.; Terra, F.S.; Marques, K.P.P.; Sayão, V.M.; Demattê, J.A.M. Robust soil mapping at the farm scale with vis–NIR spectroscopy. Eur. J. Soil Sci. 2019, 70, 378–393. [Google Scholar] [CrossRef] [Green Version]
- Pinheiro, É.F.M.; Ceddia, M.B.; Clingensmith, C.M.; Grunwald, S.; Vasques, G.M. Prediction of soil physical and chemical properties by visible and near-infrared diffuse reflectance spectroscopy in the Central Amazon. Remote Sens. 2017, 9, 293. [Google Scholar] [CrossRef] [Green Version]
Soil Property | Pre-Processing | ICRAF | LUCAS | ||||||
---|---|---|---|---|---|---|---|---|---|
Testing Set | Testing Set | ||||||||
R2 * | RMSE * | Intercept | Slope | R2 * | RMSE * | Intercept | Slope | ||
Sand | Spectra | 0.47 | 21.00 | 20.06 | 0.469 | 0.52 | 17.83 | 20.28 | 0.516 |
1st Der | 0.73 | 14.77 | 10.03 | 0.734 | 0.66 | 14.83 | 14.01 | 0.669 | |
2nd Der | 0.71 | 15.38 | 10.79 | 0.717 | 0.67 | 14.61 | 13.56 | 0.676 | |
CR | 0.61 | 17.89 | 14.91 | 0.609 | 0.54 | 17.41 | 19.31 | 0.534 | |
DT | 0.61 | 17.87 | 14.84 | 0.617 | 0.58 | 16.70 | 18.10 | 0.574 | |
SNV | 0.62 | 17.61 | 14.60 | 0.626 | 0.59 | 16.38 | 16.95 | 0.599 | |
Silt | Spectra | 0.47 | 14.49 | 15.51 | 0.468 | 0.47 | 13.13 | 20.39 | 0.475 |
1st Der | 0.71 | 10.62 | 8.37 | 0.712 | 0.60 | 11.44 | 15.61 | 0.599 | |
2nd Der | 0.71 | 10.65 | 8.42 | 0.712 | 0.62 | 11.13 | 14.56 | 0.625 | |
CR | 0.62 | 12.16 | 11.11 | 0.622 | 0.51 | 12.77 | 19.03 | 0.505 | |
DT | 0.62 | 12.10 | 10.79 | 0.633 | 0.51 | 12.69 | 19.02 | 0.512 | |
SNV | 0.58 | 12.78 | 12.24 | 0.587 | 0.51 | 12.67 | 18.50 | 0.518 | |
Clay | Spectra | 0.65 | 13.03 | 11.42 | 0.651 | 0.65 | 7.610 | 6.643 | 0.656 |
1st Der | 0.81 | 9.63 | 6.29 | 0.807 | 0.80 | 5.760 | 3.990 | 0.795 | |
2nd Der | 0.77 | 10.41 | 7.24 | 0.773 | 0.73 | 6.672 | 5.316 | 0.729 | |
CR | 0.71 | 11.83 | 9.57 | 0.711 | 0.57 | 8.444 | 8.320 | 0.571 | |
DT | 0.70 | 12.16 | 10.10 | 0.691 | 0.67 | 7.376 | 6.259 | 0.673 | |
SNV | 0.72 | 11.41 | 8.92 | 0.720 | 0.74 | 6.566 | 5.065 | 0.750 |
Soil Textural Fraction | SSL * | Model | R2 * | RMSE * | Slope (m) | Intercept (a) |
---|---|---|---|---|---|---|
Sand | ICRAF | MLP | 0.78 | 13.55 | 0.99 | 0.29 |
Silt | CatBoost | 0.81 | 8.45 | 0.99 | 0 | |
Clay | MLP | 0.85 | 8.77 | 0.99 | 0.62 | |
Sand | LUCAS | CatBoost | 0.78 | 11.77 | 0.99 | 0.33 |
Silt | CatBoost | 0.76 | 8.68 | 0.98 | 0.57 | |
Clay | CatBoost | 0.85 | 4.99 | 1.01 | −0.3 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gouda, M.Z.; Nagihi, E.M.; Khiari, L.; Gallichand, J.; Ismail, M. Artificial Intelligence-Based Prediction of Key Textural Properties from LUCAS and ICRAF Spectral Libraries. Agronomy 2021, 11, 1550. https://doi.org/10.3390/agronomy11081550
Gouda MZ, Nagihi EM, Khiari L, Gallichand J, Ismail M. Artificial Intelligence-Based Prediction of Key Textural Properties from LUCAS and ICRAF Spectral Libraries. Agronomy. 2021; 11(8):1550. https://doi.org/10.3390/agronomy11081550
Chicago/Turabian StyleGouda, Mohamed Zakaria, El Mehdi Nagihi, Lotfi Khiari, Jacques Gallichand, and Mahmoud Ismail. 2021. "Artificial Intelligence-Based Prediction of Key Textural Properties from LUCAS and ICRAF Spectral Libraries" Agronomy 11, no. 8: 1550. https://doi.org/10.3390/agronomy11081550