Predicting the Trend of Stock Market Index Using the Hybrid Neural Network Based on Multiple Time Scale Feature Learning

Round 1

Reviewer 1 Report

This paper uses an approach combining CNN and LSTM to predict the price trend of the stock market index. This seems to be an exciting topic; however, the paper lacks proper presentation. Many parts of the article need to be rewritten for appropriate understanding   There are a few concerns.   Please explain your data, such as columns of data, and their features also cite the source of data.   In related work, instead of just mentioning the citing of the existing work, explain the work specified in that paper and also the limitation of the work which you are going to improve.   The theoretical framework needs to be reinforced, for example, including more references.   Why is only Conv 1D used? why not Conv 2D, 2+1D or 3D? Specify the reason why the only 1D is enough for your proposed approach.?   In Fig2. Multiple fully connected layers are displayed without any change in input to the next layer? The purpose can be achieved with one layer as well.   In section 4.1.4, it is mentioned that Adam optimizer is used for training, which is not mentioned in the architecture of the proposed model.   This work seems to lack novelty, for example, In Fig 3. Model 1 is merely using LSTM.   LSTM can predict medium-term and long-term, and RNN can be used for the short term, then what is the need for CNN? Usually, RNN outperforms CNN in prediction.   Comparison: I think the comparison is limited. How do the models compare to other methodologies, including but not limited to statistical models, RNN, ARIMA?   Authors have used only 1 Evaluation Metric, which is too simple, consider other Evaluation Metrics such as RMSE, MAE as well.   The accuracy of the proposed model is less than 80%, which is not considered as a reasonable accuracy.   Importantly this paper lacks discussion about novelty even though many articles can be found for predictions of the stock market index using NN based models.   There is a need to improve the conclusions explaining how this research helps in expanding previous research and body of knowledge.  

Author Response

Dear Editor and Reviewer,

We are writing in response to your review commentary on our paper, “Predicting the Trend of Stock Market Index Using the Hybrid Neural Network Based on Multiple Time-Scale Feature Learning”.

We are very grateful for the valuable questions and instructive comments that have been raised. The time and effort on reviewing this paper is highly appreciated. We believe that we have been able to address each of the comments. The responses to each of your questions and comments can be found in the attachment file.

At last, thank you again for your suggestions.

Best wishes,

All Authors

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper addresses a challenging problem of stock price prediction, a system dependent on many factors and trading strategies. I consider this a technical contribution, i.e., combination of existing techniques for solving a known problem, nevertheless I believe the contribution is still relevant.

The details of the method should be better described and defined and I would like to see a realistic hard example in price prediction included. What is most worrying is that it seems the experiment has not been correctly set up. Thus parts of the paper will require substantial modifications. I detail my comments below:

1) Section 3.2.3: Please define formally how the features D1-D3 are constructed and what is their dimension. Figure 4 is confusing since it is unclear how the colors and arrows connect to the definitions of the CNN output and LSTM input.

2) Section 3.2.3: Please show clearly in Figure 3 how the outputs of LSTMs are combined. The FCNs are usually defined over a single feature vector -- how is this vector formed? Equation (2) hints at how the outputs might be combined, but since the dimensions of D and W are not defined, one can only guess.

3) Section 3.2.3: The training loss is not defined.

4) Experiments: The training/test data is constructed by randomly shuffling daily prices, but this is a major problem since the day-to-day prices are highly correlated. Thus, from testing in this setup, one cannot reject a very plausible hypothesis that the network is picking up just the correlation and is actually overfitting the data, since parts of the test data are implicitly introduced in the training. This does not properly reflect generalization to future, unseen, data. For proper evaluation, the dataset should be split into training part before time T and testing part after T. For example, take the first 60% of dayly prices in a sequence up to time T for training, the next 20% for validation and the last 20% in the sequence for testing.

4) Experiments: Insights into failure cases should be provided to show the limits of the presented work and chart directions of future works. And with the chosen setup, there is actually a unique and relevant opportunity to do so. An interesting drop occurred in SP5000 between 20.2-20.3. this year. What is performance of the pretrained models on predicting this "rare" event?

5) Please add a baseline simplistic model in the experiments. For example, what is a performance of trend prediction just by fitting a line to the past observations? Is it chance level or lower/higher?

6) Training: Please provide the values of the training parameters (parameters of Adam, training rate, whether it was constant or not, how many epochs, batch size, etc.). Please also specify at which layers the batchnorm were used.

7) Figure 6: It is not clear how the predicted values were obtained. According to equation (2), the network output is only 0/1 (i.e., higher or lower),  not the price value prediction.

Author Response

Dear Editor and Reviewer,

We are writing in response to your review commentary on our paper, “Predicting the Trend of Stock Market Index Using the Hybrid Neural Network Based on Multiple Time-Scale Feature Learning”.

We are very grateful for the valuable questions and instructive comments that have been raised. The time and effort on reviewing this paper is highly appreciated. We believe that we have been able to address each of the comments. The responses to each of your questions and comments can be found in the attachment file.

At last, thank you again for your suggestions.

Best wishes,

All Authors

Author Response File: Author Response.pdf

Reviewer 3 Report

Overall the paper was easy to follow and understand. However, there are some confusions about the proposed architecture. The authors say that " The last part is to combine all the information learned by LSTMs" and it is unclear how such combination is actually happening. Is it concatenation of features coming from three different LSTMs or something else?. Similarly, some information regarding network training is not mentioned like which loss function is used. 

Moreover, since the crux of the proposed architecture is using different time scale features for predicting trends, it would have been better to compare the proposed approach with other 'multi-scale feature extraction' based time series classifiers. For example, variants of DenseNet architecture for time series and I wonder how the proposed scheme will fare against more recent architectures. 

Nonetheless, the underlying idea behind the approach is interesting.

Author Response

Dear Editor and Reviewer,

We are writing in response to your review commentary on our paper, “Predicting the Trend of Stock Market Index Using the Hybrid Neural Network Based on Multiple Time-Scale Feature Learning”.

We are very grateful for the valuable questions and instructive comments that have been raised. The time and effort on reviewing this paper is highly appreciated. We believe that we have been able to address each of the comments. The responses to each of your questions and comments can be found in the attachment file.

At last, thank you again for your suggestions.

Best wishes,

All Authors

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Authors have updated the previous version; still, there are some points to be addressed

• In Section 4.1.1. you must cite the reference of data.
• Rewrite the related work for References 11,12,15,26,27,16,17,18, 19, and 20. You should explain the summary of these works along with their limitations. Also, add a comparison table with previous studies to compare the novelty of your work.
• Equation 6 and 7 need to be revised.
• The alignment of subfigures in Fig 5 needs to be corrected.
• In the comparison section, instead of writing Model 1, Model 2, Write the names of models with whom you have compared your model.?

Author Response

Dear Editor and Reviewer,

We are writing in response to your review commentary on our paper, “Predicting the Trend of Stock Market Index Using the Hybrid Neural Network Based on Multiple Time-Scale Feature Learning”. We are very grateful for the valuable questions and instructive comments that have been raised. The time and effort on reviewing this paper is highly appreciated. We believe that we have been able to address each of the comments. Following are our responses to each of your questions and comments.

Point 1: In Section 4.1.1.you must cite the reference of data.

Response 1: Thanks for your suggestion. We ignored this before. We will add a reference to the data in Section 4.1.1.

Point 2: Rewrite the related work for References 11, 12, 15, 26, 27, 16, 17, 18, 19, and 20. You should explain the summary of these works along with their limitations. Also, add a comparison table with previous studies to compare the novelty of your work.

Response 2: Thank you very much for your suggestion. We will make corresponding changes in Related Work. The limitations of references 11-15 are the same, so are the limitations of references 16-18. So we summarize their limitations at the end of each paragraph. Because some references have the same limitations, we have not used a table to summarize in order to prevent too much duplicate content.

Point 3: Equation 6 and 7 need to be revised. The alignment of subfigures in Fig 5 needs to be corrected.

Response 3: Thank you for reminding. We will correct them.

Point 4: In the comparison section, instead of writing Model 1, Model 2, write the names of models with whom you have compared your model?

Response 4: We will change the names of these models to Model based on F1, Model based on F2 and Model based on F3. Figure 3 shows the structure of these models. These models are described in Section 4.2.1.

At last, thank you again for your suggestions.

Best wishes,

All Authors

Reviewer 2 Report

I have read carefully the revised text and the list of responses. The authors have sufficiently well addressed all my concerns and I recommend the paper to be accepted.

I have detected some minor lapses in the paper formatting:
1) Element-wise product symbol does not show in (6,7).
2) The title Evaluation Metric (line 269) is misplaced.
3) Figure 5: image layout broken into two lines.

I suppose these will be corrected by the journal typesetter.

Author Response

Dear Editor and Reviewer,

We are writing in response to your review commentary on our paper, “Predicting the Trend of Stock Market Index Using the Hybrid Neural Network Based on Multiple Time-Scale Feature Learning”. We are very grateful for the valuable questions and instructive comments that have been raised. The time and effort on reviewing this paper is highly appreciated. We believe that we have been able to address each of the comments. Following are our responses to each of your questions and comments.

Point 1: 1) Element-wise product symbol does not show in (6,7).

              2) The title Evaluation Metric (line 269) is misplaced.

              3) Figure 5: image layout broken into two lines.

Response 1: Thanks for your suggestion. We will correct them.

At last, thank you again for your suggestions.

Best wishes,

All Authors

Reviewer 3 Report

Most of my comments/suggestions are addressed. However, there are many grammatical mistakes and some formatting issues, especially in the new text added after the previous revision.

Author Response

Dear Editor and Reviewer,

We are writing in response to your review commentary on our paper, “Predicting the Trend of Stock Market Index Using the Hybrid Neural Network Based on Multiple Time-Scale Feature Learning”. We are very grateful for the valuable questions and instructive comments that have been raised. The time and effort on reviewing this paper is highly appreciated. We believe that we have been able to address each of the comments. Following are our responses to each of your questions and comments.

Point 1: There are many grammatical mistakes and some formatting issues, especially in the new text added after the previous revision.

Response 1: Thanks for your suggestion. We will correct them.

At last, thank you again for your suggestions.

Best wishes,

All Authors