Round 1

Reviewer 1 Report

The paper was written in good shape, I have some comments, I do suggest authors to apply them in the paper to get more impressive idea.

the topic is highly original and relevant. The intersection of smart city data and healthcare forecasting, especially related to pandemic modeling, is a budding area. The study addresses a gap by showing how non-traditional data sources, like traffic sensors, can be integrated into predictive models for health outcomes, which is not extensively explored in existing literature.

The manuscript stands out by showcasing a novel model that intertwines spatio-temporal patterns from traffic data with COVID-19 incidence. While there are other papers exploring predictive models for COVID-19, the unique integration of traffic sensor data as an indicator of human mobility offers a fresh perspective, and potentially more localized prediction capabilities, especially for urban settings.

The methodology could benefit from a few refinements:

 

The interpretability of the model should be enhanced. Techniques like SHAP or feature importance can be integrated to provide insights on which parts of the traffic data contribute most to predictions.

Potential external factors, like changes in lockdown measures or vaccination rates, should be considered in the model to account for any confounding effects they might have on both traffic patterns and infection rates. Time series cross-validation or other temporal modeling techniques might provide more robust results in this context.

-

Author Response

mathematics-2580332 - Using Traffic Sensors in Smart-cities to Enhance a Spatio-Temporal Deep Learning Model for COVID-19 Forecasting

I want to thank the reviewers for their constructive comments. All of them have been fully and thoroughly taken into account in order to improve the quality of the manuscript. This document captures a point by point description of the implemented changes.

 

Reviewer 1

Comment:

The paper was written in good shape, I have some comments, I do suggest authors to apply them in the paper to get more impressive idea.

the topic is highly original and relevant. The intersection of smart city data and healthcare forecasting, especially related to pandemic modeling, is a budding area. The study addresses a gap by showing how non-traditional data sources, like traffic sensors, can be integrated into predictive models for health outcomes, which is not extensively explored in existing literature.

The manuscript stands out by showcasing a novel model that intertwines spatio-temporal patterns from traffic data with COVID-19 incidence. While there are other papers exploring predictive models for COVID-19, the unique integration of traffic sensor data as an indicator of human mobility offers a fresh perspective, and potentially more localized prediction capabilities, especially for urban settings.

Response:

I want to thank the reviewer for his/her positive comments in the review.

 

Comment:

The methodology could benefit from a few refinements:

The interpretability of the model should be enhanced. Techniques like SHAP or feature importance can be integrated to provide insights on which parts of the traffic data contribute most to predictions.

Response:

A new sub-section 6.3 has been added:

6.3. Model explanation

The estimation of the importance of the input features in order to explain the achieved prediction results is a key part in explainable AI models. Different methods have been proposed for estimating the importance of the input features in explaining the outputs of AI models such as [40-41]. The best preforming machine learning model presented in this paper, as evaluated in section 6.2, is the one combining COVID-19 and traffic information into 2-coloured sequences of images (as presented in figure 3). The Integrated Gradient method [40] will be used in this section in order to assess the importance of the input features since the method scales better for sequences of input images than Shapley values [41].

Integrated Gradients (IG) should be calculated for each input feature. In our case, each point in the sequences of COVID-19 and traffic images is an input feature to the forecasting model. Since far away in space points are expected to play a less significant contribution to the generated results the IG have been computed for a 6 by 6 images containing the information in the surrounding area of the geographical location for which COVID-19 incidence will be estimated. An example for the results achieved in the fifth wave for the geographical location used in section 6.2 is captured in figure 22 (for the COVID-19 images) and figure 23 (for the traffic images). The information to be estimated by the model is the COVID-19 incidence for a time t in the fifth wave (July 2021) for the spatial location with indexes (3,3) in time sequence of input images. The model uses the previous 5 weeks (from t-1 to t-5) to provide an estimation for the COVID-19 incidence at time t and location with indexes (3,3).

Figure 30 shows that the last week COVID-19 data is the most relevant information for the model in order to estimate the incidence values one week later. Figure 22 also captures the surrounding areas which have a higher impact in the estimation of the result. Figure 23 performs a similar analysis for the traffic data. Again, traffic data one week before shows a higher impact in predictions. The traffic in the predicted location shows a significant impact while geographic areas farther in space and time show a smaller contribution to the generated one-week ahead COVID-19 estimations.

[[[ Figure shown in attached file]]]

Figure 22. Integrated gradients for the COVID images in the 5 previous weeks as used by the model in figure 3

[[[ Figure shown in attached file]]]

Figure 23. Integrated gradients for the traffic images in the 5 previous weeks as used by the model in figure 3

 

Comment:

Potential external factors, like changes in lockdown measures or vaccination rates, should be considered in the model to account for any confounding effects they might have on both traffic patterns and infection rates. Time series cross-validation or other temporal modeling techniques might provide more robust results in this context.

Response:

Thanks a lot for this comment. We have added the following text to section 6.2:

Both lockdown policies and vaccination campaigns have paid an impact in the spread of the virus [31-34]. Major lockdowns in Spain were lifted on June 2020 and vaccination campaigns reached the majority of the population by the end of 2021. In this paper, a period of time has been selected after major lockdowns were lifted. In order to avoid the effect of vaccinations on the model performance, the sixth wave has not been used for validation purposes. As a future work, lockdown measures or vaccination rates will be added to the proposed models to improve their prediction performance. 

These variables and an optimized validation method will be done in a future work.

Author Response File: Author Response.docx

Reviewer 2 Report

This paper proposes a novel AI-based model for COVID-19 spreading forecasting. This study is categorized in the class of time series forecasting containing a variety range of ML algorithms. The paper is well-organized and the related studies are comprehensively reviewed. Also, the paper is of interest to a wide range of readers who are working on pattern recognition, real-life data-driven applications, deep learning, etc. There are some serious concerns about the manuscript, especially in the Results section which is presented as follows:

1-      It would be a good suggestion for the Author to add some studies regarding chaotic time series prediction in the related works to enhance the quality of the manuscript.

2-      In Section 3, the author introduces a technique for generating 2D image from the raw input which was interesting for me. (that was not a criticism…)

3-      The model employed in section 4.1 is a common CNN model which has been illustrated in Fig. 4. The quality of the figure is very poor. This figure should be improved and must be supported by a complete description of how to feed the 2D inputs to the model.

4-      According to comment 3, modifications are necessary in Sections 4.2 and 4.3.

5-      Since Figures 4, 6, and 8 and 3,5, and 7 are greatly similar and they occupied a lot of space, a good suggestion would be to eliminate the repetitive figures and suffice to express the differences.

 

6-      One of the challenges of time series forecasting is to deal with the delay issue, which occurs when the predicted values are shifted or lagged behind the actual values. This can happen when the model is unable to capture the dynamics and patterns of the data. In some figures e.g. Fig. 15, 17, 19 , …,and 25 I can see a couple of hints indicating the challenge is occurred. The author must clearly describe this phenomenon and proof the originality of the results.  

Author Response

mathematics-2580332 - Using Traffic Sensors in Smart-cities to Enhance a Spatio-Temporal Deep Learning Model for COVID-19 Forecasting

I want to thank the reviewers for their constructive comments. All of them have been fully and thoroughly taken into account in order to improve the quality of the manuscript. This document captures a point by point description of the implemented changes.

Reviewer 2

Comment:

This paper proposes a novel AI-based model for COVID-19 spreading forecasting. This study is categorized in the class of time series forecasting containing a variety range of ML algorithms. The paper is well-organized and the related studies are comprehensively reviewed. Also, the paper is of interest to a wide range of readers who are working on pattern recognition, real-life data-driven applications, deep learning, etc.

Response:

I want to thank the reviewer for his/her positive comments in the review.

 

Comment:

There are some serious concerns about the manuscript, especially in the Results section which is presented as follows:

1-      It would be a good suggestion for the Author to add some studies regarding chaotic time series prediction in the related works to enhance the quality of the manuscript.

Response:

A new references has been added:

35. Dudukcu, H. V., Taskiran, M., Taskiran, Z. G. C., & Yildirim, T. (2023). Temporal Convolutional Networks with RNN approach for chaotic time series prediction. Applied Soft Computing, 133, 109945.

 

New text has been added to section 2:

Related ideas have been implemented for other chaotic time series prediction (non-linear systems that produce non-periodic output sensitive to initial conditions [35]). The authors in [35] proposed a combination of a former CNN model to extract deep features from chaotic time series and a subsequent RNN to extract time patterns related to the models proposed in this paper.

 

Comment:

2-      In Section 3, the author introduces a technique for generating 2D image from the raw input which was interesting for me. (that was not a criticism…)

Response:

I want to thank the reviewer for his/her comment

 

Comment:

3-      The model employed in section 4.1 is a common CNN model which has been illustrated in Fig. 4. The quality of the figure is very poor. This figure should be improved and must be supported by a complete description of how to feed the 2D inputs to the model.

Response:

For space limitations in the Word file the figure is not properly shown but a high accuracy image will be generated for the on-line version of the paper. As suggested, the description of the inputs to feed the model have been improved and are now better explained in section 4.1 as:

The input layer in figure 4 contains 5 input boxes, each one receives the combined input image for one of the 5 previous weeks. Each image contains two colors, one for traffic data and the other for COVID-19 data. A different CNN model is applied to each image in Figure 4.

 

Comment:

4-      According to comment 3, modifications are necessary in Sections 4.2 and 4.3.

Response:

Similarly, the following text has been added to sections 4.2 and 4.3:

The input layer in figure 6 contains 10 input boxes, the first 5 receive the COVID-19 input image for one of the 5 previous weeks and the last 5 receive the similar traffic images. A different CNN model is applied to each image in Figure 6.

The input layer in figure 8 contains 5 input boxes, each one receives a COVID-19 monochromatic input image for one of the 5 previous weeks. No traffic information is used in this case. A different CNN model is applied to each image in Figure 8.

 

Comment:

5-      Since Figures 4, 6, and 8 and 3,5, and 7 are greatly similar and they occupied a lot of space, a good suggestion would be to eliminate the repetitive figures and suffice to express the differences.

Response:

I thank the reviewer for the comment. Figures 3, 5 and 7 are intended to provide a high-level intuition while figures 4, 6 and 8 provide the implementation details. Both types of figures complement each other (although they describe the same models and are therefore related). In order to save space and reduce the number of figures, some figures have been eliminated in section 6 following the next reviewers’ comment.

Comment:

6-      One of the challenges of time series forecasting is to deal with the delay issue, which occurs when the predicted values are shifted or lagged behind the actual values. This can happen when the model is unable to capture the dynamics and patterns of the data. In some figures e.g. Fig. 15, 17, 19 , …,and 25 I can see a couple of hints indicating the challenge is occurred. The author must clearly describe this phenomenon and proof the originality of the results.  

Response:

I want to thank the reviewer for identifying the delay issue in figures 15, 17, 19 , …,and 25. In faclt, this delay issue is not seen in related figures 14, 16, 18… I have detected and corrected an error when generating the plots in python. Since the cumulative cases are just the addition of the weekly incidence data, and several reviewers have requested to limit the number of figures in the paper, the cumulative figures have been removed since there were errors in the plots and the information is captured in weekly incidence images.

Author Response File: Author Response.docx

Reviewer 3 Report

can be accepted once model and justification is fine the diagrams to be improved what is the significance of using that many graphs .if so use in a proper way of representing all outcomes. minor grammer to be checked

can be accepted once model and justification is fine
the diagrams to be improved
what is the significance of using that many graphs .if so use in a proper way of representing all outcomes.

minor grammer to be checked

Author Response

mathematics-2580332 - Using Traffic Sensors in Smart-cities to Enhance a Spatio-Temporal Deep Learning Model for COVID-19 Forecasting

I want to thank the reviewers for their constructive comments. All of them have been fully and thoroughly taken into account in order to improve the quality of the manuscript. This document captures a point by point description of the implemented changes.

Reviewer 3

Comment:

can be accepted once model and justification is fine the diagrams to be improved what is the significance of using that many graphs .if so use in a proper way of representing all outcomes. minor grammer to be checked

Response:

I want to thank the reviewer. Some redundant figures have been remove following the reviewer suggestion. A new section 6.3 has been added to better explain the results.

Author Response File: Author Response.docx

Reviewer 4 Report

This work focused on COVID-19 forecasting using traffic sensor data. The authors proposed a deep learning model to extract spatio-temporal features from raw sensor data. Although this work provides contributions, there are some issues needed to be answered and improved before publication as follows.

 

 

1) In the abstract, please report the quantitative results.

 

2) In the section 2, the authors should summarize related state-of-the-art works in a table.

 

3) In line 233-236, the font style of variable i should be corrected. The equation (1), the font style of variable i are italic.

 

4) Do ablation study to investigate the performance of the proposed DL network.

 

 

5) Is it possible to adapt the proposed method for forecasting other pandemic?

 

6) What are limitations of this proposed method? Please discuss.

Author Response

mathematics-2580332 - Using Traffic Sensors in Smart-cities to Enhance a Spatio-Temporal Deep Learning Model for COVID-19 Forecasting

I want to thank the reviewers for their constructive comments. All of them have been fully and thoroughly taken into account in order to improve the quality of the manuscript. This document captures a point by point description of the implemented changes.

Reviewer 4

Comment:

This work focused on COVID-19 forecasting using traffic sensor data. The authors proposed a deep learning model to extract spatio-temporal features from raw sensor data.

Response:

I want to thank the reviewer for his/her positive comments.

Comment:

Although this work provides contributions, there are some issues needed to be answered and improved before publication as follows.

1) In the abstract, please report the quantitative results.

Response:

The abstract now reports the major result:

(MSE values are reduced by a 70% factor)

Comment:

2) In the section 2, the authors should summarize related state-of-the-art works in a table.

Response:

The following information has been added at the end of section 2 in order to provide a summary of the state of the art in a table:

A summary of the related studies categorized by the type of ML model used and the variable that is predicted is captured in Table 1.

Table 1. Summary of the related studies by model type and predicted variable.

References	Model types	Predicted variable
6, 12, 13	Regression Trees, Gaussian processes	Traffic volumes
15-18	Shallow ML models	COVID-19 diagnoses
19, 20	Shallow ML models	COVID-19 incidence
21-23	Deep ML models	COVID-19 incidence
11, 25- 27	Space-time models	Mobility enhanced COVID-19 estimations
30-34	Shallow ML models	Mobility estimations caused by COVID-19

 

 

Comment:

3) In line 233-236, the font style of variable i should be corrected. The equation (1), the font style of variable i are italic.

Response:

It has been done as requested

Comment:

4) Do ablation study to investigate the performance of the proposed DL network.

Response:

MSE values:

A new section 6.4 has been added with the results of an ablation study. I want to thank the reviewer for the suggestion which makes the paper more complete. The following content has been added:

 

6.4. Ablation study

In order to investigate the performance of the proposed model in Figure 3 when assessing the importance of each internal block in the model in the contribution to the final result, an ablation study has been performed. The combined images (COVID-19 and traffic) model in Figure 3 is based on 5 major building blocks that extract spatial features independently from an input image in the temporal sequence using a CNN which are then fed into an LSTM layer. An ablation study is presented by removing the CNN processing each sequential image at a time and comparing the MSE values with the overall model.

Table 2 captures the results for the MSE values when removing the different CNN blocks in Figure 4. The optimal value for the MSE is when the entire model is preserved. When removing the processing of an input instant of time in the image sequence processing, the most important blocks are those processing images which are closer in time to the image being forecasted. These results are aligned with the results in figure 22 and figure 23 in which information in closer images in time are better able to explain the results of the model.

 

Table 2. Ablation study removing the CNN processing each instant of time in the sequence of combined images

Block removed	MSE values
None	0.003242
CNN processing image at t-5	0.003245
CNN processing image at t-4	0.004514
CNN processing image at t-3	0.005416
CNN processing image at t-2	0.005744
CNN processing image at t-1	0.006095

 

 

Comment:

5) Is it possible to adapt the proposed method for forecasting other pandemic?

Response:

The following text has been added to the conclusions of the paper:

The proposed model is likely to apply for the forecasting of similar respiratory viruses. As a future study, the model will be applied to influenza and other examples.

Comment:

6) What are limitations of this proposed method? Please discuss.

Response:

The major limitations are now captured in the conclusions. The following text has been added:

The major limitation of the proposed method is the availability of homogeneous data over time and space. Several methods have been used for COVID-19 data collection as described in the paper and traffic sensors are continually been added and removed to monitor the traffic in Madrid. As a future study, a method to homogenize data will be developed which will make the model more generalizable to other zones.

Author Response File: Author Response.docx

Reviewer 5 Report

Attached

Comments for author File: Comments.pdf

NA

Author Response

mathematics-2580332 - Using Traffic Sensors in Smart-cities to Enhance a Spatio-Temporal Deep Learning Model for COVID-19 Forecasting

I want to thank the reviewers for their constructive comments. All of them have been fully and thoroughly taken into account in order to improve the quality of the manuscript. This document captures a point by point description of the implemented changes.

Reviewer 5

Comment:

The paper integrates traffic sensor data with COVID-19 PCR reports from health centers to create predictive models for the virus's spread. Emphasizing a base-line model, it contrasts its findings with prior studies. The models are trained and validated using specific datasets. Notably, the "COVID only" and "traffic only" base-line models have been tested for forecasting new and cumulative cases in specific locations. In essence, the study presents a spatiotemporal model for COVID-19 forecasting, suggesting its potential in guiding public health strategies based on traffic and health data intersections.

Response:

I want to thank the reviewer for his/her positive reviews and comments.

Comment:

This paper is well technically sound, but with the following concerns:

How was data normalization or standardization handled, especially given the diverse nature of traffic and health data?

Response:

The data normalization process is described in section 3. The following text is in the improved version of the paper:

In order to optimize the training of the machine learning models, a normalization process could be applied. Figure 1 shows a COVID incidence image after applying a linear scaling so that the values are normalized between 0 and 1 following the expression in (1).

 

                                                                                  (1)

 

Where i represents the number of positive cases in a particular square of the image, i_min the minimum value in all the incidence images and i_max the maximum value in all the incidence images. The same normalization values are therefore applied to all the images in order to preserve the relative weekly changes in COVID-19 incidence values.

 

Traffic images are computed from traffic intensity data provided by traffic flow sensors. The same region in space will be divided using the same grid of squares as in the COVID-19 incidence image. The value for each square in the image will be calculated as the average value for the traffic intensity measures provided by all traffic sensors located in that square. A similar normalization process could be applied to the generated traffic images in order to speed up the training of the machine learning model. Figure 2 shows an example of a generated traffic image.

 

Comment:

How did the model account for regional differences in COVID-19 testing rates and traffic patterns? E.g. Were certain areas more challenging to predict than others? For instance, did urban centers with more traffic data have more accurate predictions than rural areas?

Response:

The performance of the model is linked to the availability of significant amounts of data. For regions in the outskirts of the city the number of COVID-19 cases as well as the density of the traffic sensors is reduced.as shown in Figures 1 and 2.

Comment:

Were there any discernible patterns or challenges in forecasting during specific times, such as holiday seasons or lockdown periods?

Response:

Lock-down periods and holiday seasons have an impact on predictions. The following text has been added to point out some future research to improve the models including different conditions:

Both lockdown policies and vaccination campaigns have paid an impact in the spread of the virus [31-34]. Major lockdowns in Spain were lifted on June 2020 and vaccination campaigns reached the majority of the population by the end of 2021. In this paper, a period of time has been selected after major lockdowns were lifted. In order to avoid the effect of vaccinations on the model performance, the sixth has not been used for validation purposes. As a future work, lockdown measures or vaccination rates will be added to the proposed models to improve their prediction performance.

Comment:

Which optimization algorithm (e.g., Adam, SGD, RMSprop) was used for training the model, and how were its hyperparameters (like learning rate) determined?

Response:

The following text has been added to the results section:

An Adam optimizer is used for the training of the models using 300 epochs and a learning rate of 0.001. Those parameters have been tuned to achieve stable convergence of the training process in all cases.

Comment:

Were any regularization techniques like L1 or L2 regularization employed to prevent overfitting?

Response:

The following text has been added:

The increase in the validation loss after 10 epochs has been used to stop the training in order to avoid overfitting.

Comment:

Were any methods used to determine the importance or contribution of individual features (e.g., SHAP values or feature importance from tree-based models)?

Response:

A new sub-section 6.3 has been added:

6.3. Model explanation

The estimation of the importance of the input features in order to explain the achieved prediction results is a key part in explainable AI models. Different methods have been proposed for estimating the importance of the input features in explaining the outputs of AI models such as [40-41]. The best preforming machine learning model presented in this paper, as evaluated in section 6.2, is the one combining COVID-19 and traffic information into 2-coloured sequences of images (as presented in figure 3). The Integrated Gradient method [40] will be used in this section in order to assess the importance of the input features since the method scales better for sequences of input images than Shapley values [41].

Integrated Gradients (IG) should be calculated for each input feature. In our case, each point in the sequences of COVID-19 and traffic images is an input feature to the forecasting model. Since far away in space points are expected to play a less significant contribution to the generated results the IG have been computed for a 6 by 6 images containing the information in the surrounding area of the geographical location for which COVID-19 incidence will be estimated. An example for the results achieved in the fifth wave for the geographical location used in section 6.2 is captured in figure 22 (for the COVID-19 images) and figure 23 (for the traffic images). The information to be estimated by the model is the COVID-19 incidence for a time t in the fifth wave (July 2021) for the spatial location with indexes (3,3) in time sequence of input images. The model uses the previous 5 weeks (from t-1 to t-5) to provide an estimation for the COVID-19 incidence at time t and location with indexes (3,3).

Figure 30 shows that the last week COVID-19 data is the most relevant information for the model in order to estimate the incidence values one week later. Figure 22 also captures the surrounding areas which have a higher impact in the estimation of the result. Figure 23 performs a similar analysis for the traffic data. Again, traffic data one week before shows a higher impact in predictions. The traffic in the predicted location shows a significant impact while geographic areas farther in space and time show a smaller contribution to the generated one-week ahead COVID-19 estimations.

[[[Figure shown in attached file]]]

Figure 22. Integrated gradients for the COVID images in the 5 previous weeks as used by the model in figure 3

[[[Figure shown in attached file]]]

Figure 23. Integrated gradients for the traffic images in the 5 previous weeks as used by the model in figure 3

 

Comment:

Was transfer learning considered, especially if there are pre-trained models on similar tasks?

Response:

This idea is very relevant. In the case of the paper each model has been trained from scratch in order to provide a fair comparison among the performance of each different model. When applying the model to a different region it would be important to use transfer learning to avoid a cold start problem for example. This idea is very relevant as a future work.

Author Response File: Author Response.docx

Round 2

Reviewer 4 Report

The authors have addressed all the previous comments.