A Proposal of Transfer Learning for Monthly Macroeconomic Time Series Forecast †
Abstract
:1. Introduction
- ✓
- As far as we know, this is the first study that explores and proposes the generation of pre-trained models that can be used with transfer learning to make predictions in any country using monthly macroeconomic variables.
- ✓
- Some studies compare the performance of several models with macroeconomic variables; however, as far as we know, this is the first that makes a benchmark between deep learning, statistical, and machine learning models.
- ✓
- This study compares the application of deep learning models without transfer learning, as economic researchers tend to do, and the application of deep learning models with transfer learning.
2. Deep Learning Applied to Macroeconomic Variables Forecast
3. Method
3.1. Macroeconomic Time Series
3.2. Datasets
3.3. Models
- ✓
- Three statistical models (St): Auto Arima (arima), ETS (ets), and Theta (theta);
- ✓
- Machine learning models (Ml): Support vector regression (svr), random forest (rf) and XGBoost (xgb);
- ✓
- Deep learning models without transfer learning (Dl/wh): Long short-term memory (lstm), temporal convolutional network (tcn), convolutional neuronal network (cnn);
- ✓
- First proposal of deep learning with transfer Learning (Dl/t_M4). We applied the same deep learning models mentioned previously, but in this case, the models were trained with the 1000 time series concatenated from the M4 subsample dataset. The concatenation was in the input and output. For example, for the prediction of the next three months based on the previous twelve months, the all-time series was transformed into a matrix of k rows and fifteen columns; then, the matrices were concatenated to obtain the final dataset. Finally, when the models were trained, we used them to apply transfer learning for each of the target time series (the 24 time series)
- ✓
- The second proposal of deep learning with transfer learning (Dl/t). The methodology was like that of the previous models, but the models were trained using the macroeconomic dataset.
3.4. Experimental Procedure
4. Results
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Vargas, A.R. Pronóstico del crecimiento trimestral de costa rica mediante modelos de frecuencia mixta. Rev. Cienc. Econ. 2014, 32, 189–226. [Google Scholar] [CrossRef] [Green Version]
- Rodríguez-Vargas, A. Forecasting Costa Rican inflation with machine learning methods. Lat. Am. J. Central Bank. 2020, 1, 100012. [Google Scholar] [CrossRef]
- Aastveit, K.A.; McAlinn, K.; Nakajima, J.; West, M. Multivariate Bayesian Predictive Synthesis in Macroe-conomic Forecasting. J. Am. Stat. Assoc. 2019, 115, 1092–1110. [Google Scholar] [CrossRef] [Green Version]
- Jung, J.-K.; Patnam, M.; Ter-Martirosyan, A. An Algorithmic Crystal Ball: Forecasts-Based on Machine Learning. SSRN Electron. J. 2018, 2018, 1–35. [Google Scholar] [CrossRef]
- Kaushik, M.; Giri, A.K. Forecasting Foreign Exchange Rate: A Multivariate Comparative Analysis between Traditional Econometric, Contemporary Machine Learning & Deep Learning Techniques. arXiv 2020, arXiv:2002.10247. [Google Scholar]
- Mariano-Hernández, D.; Hernández-Callejo, L.; Solís, M.; Zorita-Lamadrid, A.; Duque-Perez, O.; Gonzalez-Morales, L.; Santos-García, F. A Data-Driven Forecasting Strategy to Predict Continuous Hourly Energy Demand in Smart Buildings. Appl. Sci. 2021, 11, 7886. [Google Scholar] [CrossRef]
- Hewage, P.; Behera, A.; Trovati, M.; Pereira, E.; Ghahremani, M.; Palmieri, F.; Liu, Y. Temporal convolutional neural (TCN) network for an effective weather forecasting using time-series data from the local weather station. Soft Comput. 2020, 24, 16453–16482. [Google Scholar] [CrossRef] [Green Version]
- Chen, J.; Zhao, C.; Liu, K.; Liang, J.; Wu, H.; Xu, S. Exchange Rate Forecasting Based on Deep Learning and NSGA-II Models. Comput. Intell. Neurosci. 2021, 2021, 2993870. [Google Scholar] [CrossRef] [PubMed]
- Nguyen, H.T.; Nguyen, D.T. Transfer Learning for Macroeconomic Forecasting. In Proceedings of the 2020 7th NAFOSTED Conference on Information and Computer Science (NICS), Ho Chi Minh City, Vietnam, 26–27 November 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 332–337. [Google Scholar] [CrossRef]
- Pratap, B.; Sengupta, S. Macroeconomic Forecasting in India: Does Machine Learning Hold the Key to Better Forecasts? In RBI Working Paper Series; Reserve Bank of India: Mumbai, India, 2019. [Google Scholar] [CrossRef]
- Kim, S. Macroeconomic and Financial Market Analyses and Predictions through Deep Learning. In Bank of Korea WP 2020-18. Available online: https://ssrn.com/abstract=3684936 (accessed on 25 November 2021).
- Solís, M.; Calvo-Valverde, L.-A. Performance of Deep Learning models with transfer learning for multiple-step-ahead forecasts in monthly time series. Intel. Artif. 2022, 25, 110–125. [Google Scholar] [CrossRef]
- Otović, E.; Njirjak, M.; Jozinović, D.; Mauša, G.; Michelini, A.; Štajduhar, I. Intra-domain and cross-domain transfer learning for time series data—How transferable are the features? Knowl.-Based Syst. 2022, 239, 107976. [Google Scholar] [CrossRef]
- Poghosyan, A.; Harutyunyan, A.; Grigoryan, N.; Pang, C.; Oganesyan, G.; Ghazaryan, S.; Hovhannisyan, N. An Enterprise Time Series Forecasting System for Cloud Applications Using Transfer Learning. Sensors 2021, 21, 1590. [Google Scholar] [CrossRef] [PubMed]
- Mulaudzi, R.; Ajoodha, R. Application of Deep Learning to Forecast the South African Unemployment Rate: A Multivariate Approach. In Proceedings of the 2020 IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE), Gold Coast, Australia, 16–18 December 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1–6. [Google Scholar] [CrossRef]
- Longo, L.; Riccaboni, M.; Rungi, A. A neural network ensemble approach for GDP forecasting. J. Econ. Dyn. Control 2022, 134, 104278. [Google Scholar] [CrossRef]
- Yasir, M.; Afzal, S.; Latif, K.; Chaudhary, G.M.; Malik, N.Y.; Shahzad, F.; Song, O.-Y. An Efficient Deep Learning Based Model to Predict Interest Rate Using Twitter Sentiment. Sustainability 2020, 12, 1660. [Google Scholar] [CrossRef] [Green Version]
- Dodevski, A.; Koceska, N.; Koceski, S. Forecasting exchange rate between macedonian denar and euro using deep learning. J. Appl. Econ. Bus. 2018, 6, 50–61. [Google Scholar]
- Cao, W.; Zhu, W.; Wang, W.; Demazeau, Y.; Zhang, C. A Deep Coupled LSTM Approach for USD/CNY Exchange Rate Forecasting. IEEE Intell. Syst. 2020, 35, 43–53. [Google Scholar] [CrossRef]
- Sun, S.; Wang, S.; Wei, Y. A new ensemble deep learning approach for exchange rates forecasting and trading. Adv. Eng. Informatics 2020, 46, 101160. [Google Scholar] [CrossRef]
- Cook, T.R.; Hall, A.S. Macroeconomic Indicator Forecasting with Deep Neural Networks. In Proceedings of the CARMA 2018—2nd International Conference on Advanced Research Methods and Analytics, Valencia, Spain, 12–13 July 2018. [Google Scholar] [CrossRef] [Green Version]
- M4 Team. M4 Competitor’s Guide: Prizes and Rules. 2018. Available online: https://www.m4.unic.ac.cy/wpcontent/uploads/2018/03/M4-CompetitorsGuide.pdf (accessed on 25 November 2021).
- Hu, M.Y.; Zhang, G.; Jiang, C.X.; Patuwo, B.E. A Cross-Validation Analysis of Neural Network Out-of-Sample Performance in Exchange Rate Forecasting. Decis. Sci. 1999, 30, 197–216. [Google Scholar] [CrossRef]
XGBoost:
| Support Vector Machines:
| Random Forest:
|
CNN:
| TCN:
| LSTM:
|
Time Serie | Type of Model | Best Dl | Best Rest | ||||
---|---|---|---|---|---|---|---|
St | Ml | Dl/wh | Dl/t | Dl/t_M4 | |||
BU_CPI | 0.009 | 0.026 | 0.04 | 0.026 | 0.026 | lstm_M4 = 0.01 | arima = 0.007 |
BU_ER | 0.02 | 0.018 | 0.042 | 0.021 | 0.022 | lstm = 0.017 | arima = 0.017 |
BU_IPI | 0.052 | 0.062 | 0.067 | 0.074 | 0.079 | tcn = 0.069 | ets = 0.05 |
BU_PPI | 0.022 | 0.066 | 0.052 | 0.028 | 0.029 | lstm_M4 = 0.025 | ets = 0.02 |
BU_TVE | 0.074 | 0.149 | 0.104 | 0.077 | 0.076 | tcn_M4 = 0.075 | ets = 0.069 |
CR_CPI | 0.004 | 0.01 | 0.049 | 0.023 | 0.023 | lstm_M4 = 0.004 | ets = 0.004 |
CR_ER | 0.009 | 0.068 | 0.039 | 0.027 | 0.027 | lstm_M4 = 0.011 | theta = 0.008 |
CR_PPI | 0.008 | 0.011 | 0.028 | 0.015 | 0.015 | tcn = 0.011 | ets = 0.007 |
CR_IPI | 0.031 | 0.043 | 0.059 | 0.044 | 0.044 | tcn_M4 = 0.041 | ets = 0.029 |
CR_TVE | 0.073 | 0.163 | 0.129 | 0.078 | 0.078 | tcn_M4 = 0.075 | arima = 0.071 |
KR_CPI | 0.004 | 0.01 | 0.023 | 0.017 | 0.017 | lstm_M4 = 0.005 | theta = 0.003 |
KR_ER | 0.02 | 0.023 | 0.035 | 0.02 | 0.02 | tcn = 0.019 | arima = 0.017 |
KR_PPI | 0.007 | 0.013 | 0.023 | 0.011 | 0.011 | lstm_M4 = 0.008 | arima = 0.007 |
KR_IPI | 0.036 | 0.051 | 0.063 | 0.045 | 0.049 | lstm = 0.041 | arima = 0.036 |
KR_TVE | 0.055 | 0.083 | 0.089 | 0.066 | 0.066 | tcn_M4 = 0.063 | arima = 0.053 |
US_CPI | 0.006 | 0.018 | 0.028 | 0.015 | 0.015 | lstm_M4 = 0.007 | ets = 0.006 |
UK_CPI | 0.004 | 0.013 | 0.023 | 0.014 | 0.015 | lstm = 0.004 | ets = 0.003 |
UK_ER | 0.024 | 0.022 | 0.032 | 0.02 | 0.02 | cnn = 0.019 | XGB = 0.018 |
UK_PPI | 0.007 | 0.01 | 0.025 | 0.013 | 0.012 | tcn = 0.007 | arima = 0.007 |
UK_IPI | 0.015 | 0.019 | 0.028 | 0.024 | 0.027 | tcn = 0.018 | arima = 0.013 |
UE_PPI | 0.007 | 0.019 | 0.031 | 0.014 | 0.015 | lstm_M4 = 0.009 | ets = 0.006 |
UE_IPI | 0.039 | 0.036 | 0.041 | 0.038 | 0.038 | cnn = 0.036 | lstm_wot = 0.033 |
UK_TVE | 0.104 | 0.112 | 0.113 | 0.1 | 0.102 | tcn = 0.097 | RF = 0.099 |
UE_TVE | 0.078 | 0.104 | 0.104 | 0.092 | 0.092 | tcn = 0.09 | theta = 0.076 |
Time Serie | Type of Model | Best Dl | Best Rest | ||||
---|---|---|---|---|---|---|---|
St | Ml | Dl/wh | Dl/t | Dl/t_M4 | |||
BU_CPI | 0.019 | 0.043 | 0.064 | 0.017 | 0.019 | tcn = 0.013 | ets = 0.012 |
BU_ER | 0.047 | 0.052 | 0.087 | 0.045 | 0.046 | cnn = 0.04 | arima = 0.041 |
BU_IPI | 0.036 | 0.053 | 0.063 | 0.042 | 0.042 | cnn_M4 = 0.037 | arima = 0.035 |
BU_PPI | 0.033 | 0.073 | 0.089 | 0.032 | 0.033 | lstm = 0.031 | ets = 0.026 |
BU_TVE | 0.09 | 0.166 | 0.164 | 0.09 | 0.091 | cnn = 0.089 | theta = 0.08 |
CR_CPI | 0.01 | 0.059 | 0.06 | 0.016 | 0.015 | tcn_M4 = 0.01 | arima = 0.009 |
CR_ER | 0.025 | 0.12 | 0.059 | 0.027 | 0.027 | tcn_M4 = 0.024 | theta = 0.023 |
CR_PPI | 0.019 | 0.047 | 0.06 | 0.022 | 0.02 | tcn_M4 = 0.016 | theta = 0.017 |
CR_IPI | 0.027 | 0.116 | 0.064 | 0.033 | 0.034 | cnn_M4 = 0.033 | ets = 0.025 |
CR_TVE | 0.074 | 0.118 | 0.131 | 0.065 | 0.066 | lstm = 0.061 | arima = 0.068 |
KR_CPI | 0.007 | 0.039 | 0.039 | 0.009 | 0.008 | lstm_M4 = 0.005 | theta = 0.004 |
KR_ER | 0.04 | 0.041 | 0.052 | 0.035 | 0.038 | tcn = 0.031 | arima = 0.033 |
KR_PPI | 0.016 | 0.032 | 0.043 | 0.016 | 0.016 | lstm_M4 = 0.015 | ets = 0.013 |
KR_IPI | 0.03 | 0.127 | 0.07 | 0.033 | 0.033 | cnn = 0.032 | theta = 0.025 |
KR_TVE | 0.091 | 0.1 | 0.116 | 0.082 | 0.082 | tcn = 0.079 | RF = 0.079 |
US_CPI | 0.008 | 0.055 | 0.032 | 0.01 | 0.011 | tcn_M4 = 0.009 | arima = 0.008 |
UK_CPI | 0.006 | 0.044 | 0.034 | 0.009 | 0.009 | tcn_M4 = 0.007 | theta = 0.006 |
UK_ER | 0.037 | 0.056 | 0.064 | 0.037 | 0.037 | cnn = 0.035 | arima = 0.035 |
UK_PPI | 0.027 | 0.058 | 0.065 | 0.028 | 0.027 | lstm_M4 = 0.025 | ets = 0.026 |
UK_IPI | 0.022 | 0.024 | 0.036 | 0.027 | 0.028 | lstm = 0.024 | XGB = 0.02 |
UE_PPI | 0.012 | 0.065 | 0.038 | 0.014 | 0.014 | lstm_M4 = 0.011 | theta = 0.01 |
UE_IPI | 0.023 | 0.031 | 0.036 | 0.026 | 0.025 | tcn_M4 = 0.022 | ets = 0.021 |
UK_TVE | 0.101 | 0.116 | 0.122 | 0.087 | 0.092 | tcn = 0.073 | lstm_wot = 0.094 |
UE_TVE | 0.057 | 0.102 | 0.109 | 0.063 | 0.062 | cnn_M4 = 0.061 | theta = 0.049 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Solís, M.; Calvo-Valverde, L.-A. A Proposal of Transfer Learning for Monthly Macroeconomic Time Series Forecast. Eng. Proc. 2023, 39, 58. https://doi.org/10.3390/engproc2023039058
Solís M, Calvo-Valverde L-A. A Proposal of Transfer Learning for Monthly Macroeconomic Time Series Forecast. Engineering Proceedings. 2023; 39(1):58. https://doi.org/10.3390/engproc2023039058
Chicago/Turabian StyleSolís, Martín, and Luis-Alexander Calvo-Valverde. 2023. "A Proposal of Transfer Learning for Monthly Macroeconomic Time Series Forecast" Engineering Proceedings 39, no. 1: 58. https://doi.org/10.3390/engproc2023039058