Forecasting PM₁₀ Levels Using Machine Learning Models in the Arctic: A Comparative Study

Round 1

Reviewer 1 Report

This manuscript presents a statistical forecasting framework based on a range of established machine learning algorithms for predicting Particulate Matter (PM) concentrations in the Arctic. The forecasting comparisons among different machine learning algorithms, namely artificial neural networks (ANN), including recurrent neural networks (RNNs), gated recurrent units (GRUs), long short-term memory networks (LSTMs), echo state networks (ESNs), and windowed multi-layer perceptrons (MLPs) are presented. The manuscript is interesting and well written. My specific comments are as under: 

 

1.      At the end of section 1, please provide the novelty of your work. How is your work different from the existing works? What were the shortcomings in the existing works that you addressed in your work? As can be seen from the literature review, the suggested algorithms are extensively used for this task.

2.      It is desirable that the section-wise breakup should be given in one paragraph at the end of section 1.

3.      As written in lines 165-166, “To confirm the stationarity of all measurement series, an Augmented Dickey-Fuller test is conducted”. Is it necessary to conduct? The authors are using nonlinear methods, i.e., nonlinear machine learning techniques.

4.      It is quite surprising that mathematical forms of machine learning techniques are not discussed within the manuscript. In general, the authors provide a complete mathematical structure for these algorithms.

5.      It is not clear how the experiment is conducted. Did the author use an expanding window technique? How are the 365 forecasts obtained?

6.      Why the authors only considered MSE? I think the authors should report other summary statistics. For example, see (Forecasting next-day electricity demand and prices based on functional models).

7.      It is advised to compare the forecasting results with some benchmark models, like ARIMA or the naïve model. This way, the efficacy of the models can be assessed more clearly. See, for example, electricity spot prices forecasting based on ensemble learning.

8.      The last section should be discussion and conclusion.

 

9.      Please add the study limitations and future recommendations to the conclusion section.

Minor corrections are required. 

Author Response

Please find attached the PDF file.

The first part of the document contains comprehensive responses to the questions you kindly posed. In the second part, you will find our revised manuscript, with the sections that have been modified as a result of your inquiries highlighted in blue for easier reference.

Best Regards

Author Response File: Author Response.pdf

Reviewer 2 Report

This study presents a statistical forecasting framework and assesses its efficacy using a range of established machine learning algorithms for predicting Particulate Matter (PM) concentrations in the Arctic, specifically in Pallas (FI), Reykjavik (IS), and Tromso (NO). This reviewer recognizes the importance of such an effort to help address a number of short comings, particularly the dearth of sufficient aerosol and other environmental observations in the Arctic. 

The authors' framework leverages historical ground measures and 24-hour predictions from 9 models provided by the Copernicus Atmosphere Monitoring Service (CAMS) to provide PM predictions for the following 24 hours.  The literature across the globe has demonstrated an improved skill or forecast accuracy of nearly any parameter including Particulate Matter (PM) when adding observations (from actual measuring devices or a climatology).

The framework uses memory cells derived from artificial neural networks(ANN), recurrent neural networks (RNNs), gated recurrent units (GRUs), long short-term memory networks (LSTMs), echo state networks (ESNs), and windowed multi-layer perceptrons (MLPs) to aide their time series forecasting tasks. 

The results demonstrate their proposed framework consistently outperforms the Copernicus Atmosphere Monitoring Service (CAM) models in terms of mean squared error (MSE), with average improvements ranging from 25% to 40%. The results were independent of the memory cell chosen, which implies the memory cell is not adequately configured or its variability is too small or the outliers are masking their effect, perhaps?                              

            The authors did investigate the impact of outliers on the overall model performance.  Little if anything was made of the latter, allowing this reader/reviewer to come away with the conclusion that the outlier replacement influence is masked by the propagated error in the output of the 9 forecast models or not well framed in their model?   What happens if you do not replace the outliers? What do the outliers physically represent or what is their physical basis? Are the outliers from a particular model output used in the author's forecast? This needs to be addressed in the manuscript.

 

 

Not clear what is new in this study.

 

L26-8:  Wrongly stated; suggest a reference or using standard accepted definition. Drops are liquid by the way.  The particles are not necessarily solid and the PM10 is particulate matter with diameters of 10 microns and smaller.  You need a scientific reference to keep the wording you currently have.  No such reference – then remove it!  

 

L365-370: This text block reads, "In the algorithm, the function calculate_optimal_threshold is based on the assumption that the expected data distribution should fit the log-normal. This assumption translates to replace the outliers with the temporally nearest data: the algorithm finds the optimal threshold to correspondingly cut the histogram. The position of this threshold is calculated to minimize the MSE between the histogram of the new data and the corresponding log-normal distribution."                                        Is this assumption made and calculation applied in any of the 9 models? Is it a good assumption and why?

 

    

     Do you have representative measurements of PM in this region?  Have you conducted a scaling analysis to determine the horizontal spacing required to have a representative measurement? Here representative measurement yields the actual value of the parameter measured in space and time.  You have 9 estimated values.  What is the separation or distance between the values derived from the 9 model estimates? 

 

     How does that separation compare to the natural variability in space and time of the PM data at any one of the sites and at the location where you place your estimate?  (what size cut of PM data are used?  Think it is just PM 10. At least use average and range of PM10 values from other studies and determine the values are statistically.  What is the duration of the PM samples as a function of each data source and used in each of the 9 models? Seems like 24 h. So you have a single point or use a single attribute from which you make a forecast. For example, L140-144 implies the chemistry routines are different among the 9 models. Yet there is no chemistry data for the PM data. What about the number of condensation/evaporation cycles did the PM10 samples experience. 

How does the variation of the chemical constituent concentrations etc. as a function of temperature, relative humidity and gas phase concentrations contribute to the influence of each model on the improvement in forecast model performance of each of the 9 models?

The PM10 chemistry data must be considered.  Further the aerosol phase chemistry is not always directly correlated with the gas phase chemistry at a site. 

The natural variation in the conditions that yield a given concentration of PM10 can change very significantly in total or even in contributing attributes associated with the PM10 concentration that could cause memory cell result you reported, for example.  If that is the case, then your forecast model needs to be reconstructed from scratch. 

 

Moderate English language grammar rework needed.

Author Response

Please find attached the PDF file.

The first part of the document contains comprehensive responses to the questions you kindly posed. In the second part, you will find our revised manuscript, with the sections that have been modified as a result of your inquiries highlighted in violet for easier reference.

Best Regards

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

As the authors addressed my concerns, I would recommend the paper for publication in its present form.

good

Reviewer 2 Report

Authors have satisfactorily addressed this reviewer's comments.

No change from my last entry these authors