Assessment of the Performance of a Low-Cost Air Quality Monitor in an Indoor Environment through Different Calibration Models
Round 1
Reviewer 1 Report
Validation of experimental results need to be added.
All figures (No 4-10) need to be discussed based on the physical phenomena happening during measurement.
In reference No 9 and 38, "et al." needs to be removed by the name of other authors.
Author Response
dear reviewer,
here below the response to your suggestions can be found.
1) Validation of experimental results need to be added.
the experiment exposed in this work was designed in two parts: in the first one, the AQM was calibrated through different calibration model, while the entire second part was conceived as a rigorous validation test to check the experimental results obtained in the first period of the experiment. The quality of the calibration performance was expressed through several indicators (R2, MAE, and RMSE) in table 5. In the same table are reported the values obtained after the validation of the experimental results found in the first part. Moreover, also in table 7 is reported the validation of experimental results obtained in the calibration period (reported in table 6) in terms of R2 and nRMSE. In other words, the second part of the experiment is a rigorous validation of experimental results obtained by calibrating the AQM through MLR, RF, SVM, and ANN models, and the results concerning the validation are exposed in table 5, 7, and in figures 6, 9 ,10. Validation of calibration results is also discussed in the "discussion" section. Thank you for your suggestion.
2) All figures (No 4-10) need to be discussed based on the physical phenomena happening during measurement.
As stressed in the paper, this work is not aimed to assess or identify the physical phenomena leading to the production of CO, NO2, or O3 in homes or apartments, but it is focused on assessing the capabilities of low-cost air quality monitors in measuring the pollutant gases regardless of the physical phenomena leading the production of them. This study is also aimed to investigate if significant differences exist in calibrating AQMs through the MLR, RF, SVM, or ANN models. Having said this necessary consideration, in figure 5 and 6 are reported the sources of the most significant emissions of CO, NO2, or O3, which are: tobacco smoke, food cooking though natural gas burners, candles burning, and laser printer operation. It is well known that all these activities generate CO, NO2 , or O3. Figure 4 means to show the ranges of pollutant gas concentrations and their distributions, and it is useful to assess the level of concentrations to which electrochemical sensors are exposed. This is an useful information, because it is also known that electrochemical sensor performance increases with the target gas concentration level. Figure 7, 8 and 9 show the best performance in terms of coefficient of determination, while figure 10 summarizes the differences in using different sets of predictor variables. Thank you for your suggestion.
3) In reference No 9 and 38, "et al." needs to be removed by the name of other authors.
Done as requested
Author Response File: 
Author Response.docx
Reviewer 2 Report
This paper presents the machine learning-based calibration of CO, NO2, and O3 measured by low-cost air quality monitors in the indoor environment. The approaches have been applied in the low-cost sensors in the previous studies, e.g. Si et al (2020) and Yamamoto et al. (2017). Therefore, the study should present the novelty and uniqueness of this study compared with previous related studies before the publication. Three other comments are also suggested as follows.
1.Thoroughly review the existing studies that machine learning-based calibration methods were applied in the low-cost sensors and add in the introduction section.
- Provide the temperature and relative humidity (RH) range when the observations were performed and discuss how the temperature, RH and interfering gases influenced the targeted pollutants.
- Please provide discussion on why the SVM calibration model is less robust than the other three models.
References:
Si et al. (2020): Evaluation and calibration of a low-cost particle sensor in ambient conditions using machine-learning method, Atmos. Meas. Tech., 13, 1693–1707, https://doi.org/10.5194/amt-13-1693-2020
Yamamoto et al (2017): Machine Learning-Based Calibration of Low-Cost Air Temperature Sensors Using Environmental Data, Sensors, 17, 1290, doi:10.3390/s17061290
Author Response
dear reviewer,
thank you for your suggestions, here below you can find the responses.
- Thoroughly review the existing studies that machine learning-based calibration methods were applied in the low-cost sensors and add in the introduction section.
A review of the studies involving machine learning-based calibration methods for low-cost sensor has been added as well as the reference to the two works proposed as requested by the reviewer.
- Provide the temperature and relative humidity (RH) range when the observations were performed and discuss how the temperature, RH and interfering gases influenced the targeted pollutants.
Temperature and relative humidity range are provided in figure 4b as requested. Moreover, explicative comments have been added in the "Results" section, while the discussion about their influence on the target pollutants has been added in the "Discussion" section.
- Please provide discussion on why the SVM calibration model is less robust than the other three models.
A discussion on the possible causes leading to a less robust performance has been inserted in the "Discussion" section as suggested by the reviewer.
Author Response File: Author Response.docx
Reviewer 3 Report
In this manuscript the authors present the comparison of several calibrations models in order to test low-cost air quality monitors (AQMs) for CO, NO2 and O3 in indoor environment. In the experimental campaign the commercially available Low-Cost gas Sensors (LCSs) were used but some AQM modifications (an electronic board) have been performed in order to better control calibration procedure. Since the LCS calibration process depends of several factors (concentration levels of target gases, the variability range of temperature and humidity, and concentration levels of interfering gases are considered as main factors) it is important to test the AQM in the real indoor environment. Standard procedure involves the comparison of the measurements obtained by low-cost AQM to the reference equipment. In addition, the authors tested several widely used algorithms (Multivariate Linear Regression (MLR), the Support Vector Regression (SVR), the Random Forest Regression (RF), and the Artificial Neural Networks (ANN)) in order to obtained most appropriate one. The obtained results shows that the prediction (measurement) of CO and NO2 are more accurate than the O3 ones.
Generally, the citizen science is definitely of wider interest and development and usage of low-cost sensors are growing. From my point of view the results presented in this manuscript related to the usage of AQMs in an indoor environment are definitely useful. Having in mind limited number of indoor usage and calibration procedures of AQMs the presented results could be interesting for comparison purposes in future studies.
The manuscript is well structured, in the introduction part the goal of the study is clearly defined and most important previous studies and challenges have been analyzed. The experimental part, the setup and measurement techniques are described with reasonably details. The obtained conclusions are reasonably supported by the results obtained. Some of the information could be moved into the Supplementary material.
There are a few shortcomings that should be addressed prior to publication. Please find below several suggestions that can be used for improving the manuscript.
Keyword: Some keywords are redundant (mentioned in the title) and can be omitted (for example Air Quality Monitors, Calibration models, air quality)
In the introduction part, Table 1 could be moved to the supplementary material since it is related to the previous studies in the outdoor environment.
Figure 4. Please expand the figure title and add the reference instruments used for the measurement – it would be useful to avoid potential confusion
Figure 9. “Scatter plots for O32…” Please correct O3
I suggest to move Appendix A into the Supplementary Material
Since the range of the temperature and RH are very important parameters I strongly advice to add some description on the environmental conditions, and to provide table in the supplementary material with relevant parameters during the experimental (calibration and validation) period.
It would be interesting if the calibration procedure and test of the AQMs can be conducted using the same instruments in indoor and outdoor environment. The previous studies related to the outdoor comparisons used different sensors and AQMs, thus providing the experimental results obtained with the same AQMs in both environments would be beneficial.
Author Response
dear reviewer,
please find below the responses to your suggestions.
- Keyword: Some keywords are redundant (mentioned in the title) and can be omitted (for example Air Quality Monitors, Calibration models, air quality)
The keywords indicated by the reviewer have been removed as suggested
- In the introduction part, Table 1 could be moved to the supplementary material since it is related to the previous studies in the outdoor environment.
dear reviewer, previous studies similar to this one, but performed in outdoor environment are exposed in the introduction. Table 1 means to summarize the results of those works, instead of writing down in the main text the results achieved by them. This is important to understand the background of this research (even though the experimental conditions are different). I have done this way because I think that, by doing so, the readablity of the document is much more improved, and also for sake of conciseness. If I move the table (which is an important element for the introduction) in a supplementary document, I force the reader to jump from a point to another of the article, and as a consequence, the readability of the article decreases. I apppreciate a lot your suggestion, but with all the due respect, I think that the table 1 in the introduction makes more comfortable the reading of the present document.
- Figure 4. Please expand the figure title and add the reference instruments used for the measurement – it would be useful to avoid potential confusion
Done as suggested
- Figure 9. “Scatter plots for O32…” Please correct O3
Done as requested.
- I suggest to move Appendix A into the Supplementary Material
Done as suggested
- Since the range of the temperature and RH are very important parameters I strongly advice to add some description on the environmental conditions, and to provide table in the supplementary material with relevant parameters during the experimental (calibration and validation) period.
Temperature and RH measured by the sensors mounted in the AQM are now reported (see figure 4b). Moreover, explicative comments related to those parameters have been inserted in the "Results" and "discussion" section.
- It would be interesting if the calibration procedure and test of the AQMs can be conducted using the same instruments in indoor and outdoor environment. The previous studies related to the outdoor comparisons used different sensors and AQMs, thus providing the experimental results obtained with the same AQMs in both environments would be beneficial.
This is a very intersting topic to explore. However, I think that this is way more suitable to do in a dedicated paper, rather than loading this one with too much elements and data. Surely, it will be considered for future works. Thank you for your suggestion.
Author Response File: Author Response.docx
Round 2
Reviewer 2 Report
Authors have made the revisions accordingly.
Author Response
Thank you very much for your help. i appreciated a lot your suggetsions. You showed relevant skills about this issue.
Kind regards,
Dr. Suriano
