Application of the Optimised Pulse Width Modulation (PWM) Based Encoding-Decoding Algorithm for Forecasting with Spiking Neural Networks (SNNs) †
Abstract
:1. Introduction
2. Spiking Neural Networks
3. Application of the Optimised PWM Algorithm to SNNs for One-Step Ahead Forecasting
3.1. Optimised PWM Based Encoding-Decoding Algorithm
- Number of carriers (nc), straightly related to the carrier frequency.
- Number of points per carrier (npc), which is related to the resolution of the algorithm (see Figure 2A, where npc is four points represented in orange colour).
- processes point by point the complete time-series (signals formed by values) to encode or decode. However, encodes and decodes each time instant separately by calculating the intersection between the line that joints two values of the time-series and the line that defines the sawtooth (see Figure 2A). This change leads to simplified operations and reductions in memory and computational costs by more than half.
- As shown in Figure 2A, always emits the spike at the npc instant immediately after the intersection, while emits the spike at the closest npc instant. This modification involves a improvement in the accuracy up to double of the encoding and decoding phases.
3.2. Input-Ouput Pairs Formation and SNN Structure
3.3. Surrogate Gradient (SG) Method
4. Experimental Setup
4.1. Datasets
- A sine-wave signal, whose parameters are: frequency Hz, amplitude , sampling frequency and the number of cycles equal to 20.
- The Mackey-Glass Time Series Dataset (MGTSD), which is available in [16].
- The power consumption of the zone 3 in Tetouan. This dataset is available in [17] and the values from 00:00 1 January 2017 to 23:50 31 January 2017 are selected for the validation phase.
- The PM10 hourly monitoring dataset measured by the station of London Bloomsburry in the Greater London Area, which is available in [18]. The period from 10:00 21 February 2017 to 20:00 3 June 2017 is selected for the validation phase.
- The ARTIC dataset that is available in [19] and the voice selected is in the file “arctic_a0001.wav” from the subfolder “US English bdl (made)”. The samples selected for the validation phase are from 11,600 to 14,750.
4.2. Performance Metrics Formulation
4.3. SNNs Selected for the Validation
5. Results
5.1. Accuracy of the Forecasting Results
5.2. Computational and Energy Costs
6. Discussion
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- de Vries, A. The growing energy footprint of artificial intelligence. Joule 2023, 7, 2191–2194. [Google Scholar] [CrossRef]
- Suetake, K.; Ikegawa, S.-i.; Saiin, R.; Sawada, Y. S3NN: Time step reduction of spiking surrogate gradients for training energy efficient single-step spiking neural networks. Neural Netw. 2023, 159, 208–219. [Google Scholar] [CrossRef] [PubMed]
- Deng, L.; Wu, Y.; Hu, X.; Liang, L.; Ding, Y.; Li, G.; Zhao, G.; Li, P.; Xie, Y. Rethinking the performance comparison between SNNS and ANNS. Neural Netw. 2020, 121, 294–307. [Google Scholar] [CrossRef] [PubMed]
- Fang, W.; Chen, Y.; Ding, J.; Yu, Z.; Masquelier, T.; Chen, D.; Huang, L.; Zhou, H.; Li, G.; Tian, Y. SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence. Sci. Adv. 2023, 9, eadi1480. [Google Scholar] [CrossRef] [PubMed]
- Bu, T.; Ding, J.; Yu, Z.; Huang, T. Optimized Potential Initialization for Low-Latency Spiking Neural Networks. Proc. AAAI Conf. Artif. Intell. 2022, 36, 11–20. [Google Scholar] [CrossRef]
- De Abreu, R.S.; Silva, I.; Nunes, Y.T.; Moioli, R.C.; Guedes, L.A. Advancing Fault Prediction: A Comparative Study between LSTM and Spiking Neural Networks. Processes. 2023, 11, 2772. [Google Scholar] [CrossRef]
- Qasim Gilani, S.; Syed, T.; Umair, M.; Marques, O. Skin Cancer Classification Using Deep Spiking Neural Network. J. Digit. Imaging. 2023, 36, 1137–1147. [Google Scholar] [CrossRef] [PubMed]
- Aghabarar, H.; Kiani, K.; Keshavarzi, P. Improvement of pattern recognition in spiking neural networks by modifying threshold parameter and using image inversion. Multimed. Tools Appl. 2023, 83, 19061–19088. [Google Scholar] [CrossRef]
- Arriandiaga, A.; Portillo, E.; Espinosa-Ramos, J.I.; Kasabov, N.K. Pulsewidth Modulation-Based Algorithm for Spike Phase Encoding and Decoding of Time-Dependent Analog Data. IEEE Trans. Neural Netw. Learn. Syst. 2020, 31, 3920–3931. [Google Scholar] [CrossRef] [PubMed]
- Lucas, S.; Portillo, E. Methodology based on spiking neural networks for univariate time-series forecasting. Neural Netw. 2024, 173, 106171. [Google Scholar] [CrossRef] [PubMed]
- Neftci, E.O.; Mostafa, H.; Zenke, F. Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks. IEEE Signal Process. Mag. 2019, 36, 51–63. [Google Scholar] [CrossRef]
- Lucas, S.; Portillo, E.; Guérin, L.; Cabanes, I. Extensión del algoritmo de codificación-decodificación basado en PWM para Redes Neuronales de Impulsos. In Proceedings of the XLIV Jornadas de Automática, Zaragoza. Spain, 6–8 September 2023; pp. 168–173. [Google Scholar] [CrossRef]
- Han, C.S.; Lee, K.M. A Survey on Spiking Neural Networks. Int. J. Fuzzy Log. Intell. Syst. 2021, 21, 317–337. [Google Scholar] [CrossRef]
- Gerstner, W.; Kistler, W.M.; Naud, R.; Paninski, L. Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition; Cambridge University Press: Cambridge, UK, 2014; pp. 1–577. [Google Scholar] [CrossRef]
- Yamazaki, K.; Vo-Ho, V.K.; Bulsara, D.; Le, N. Spiking Neural Networks and Their Applications: A Review. Brain Sci. 2022, 12, 863. [Google Scholar] [CrossRef] [PubMed]
- Waheeb, W. Mackey-Glass Time Series Dataset. 2016. Available online: https://figshare.com/articles/dataset/Mackey-Glass_time_series/4233584 (accessed on 10 January 2023).
- Salam, A.; Hibaoui, A.E. Power Consumption of Tetouan City Data Set. 2021. Available online: https://archive.ics.uci.edu/ml/datasets/Power+consumption+of+Tetouan+city (accessed on 27 February 2023).
- Department for Environment Food & Rural Affairs. UK Air Information Resource. Available online: https://uk-air.defra.gov.uk/data/ (accessed on 15 March 2023).
- Black, A.W. CMU_ARCTIC Speech Synthesis Databases. Available online: http://festvox.org/cmu_arctic/ (accessed on 7 February 2023).
Main Hyperparameters | Time-Series Spliting | LIF Neuron Parameters | ||
---|---|---|---|---|
Dataset | Np | npc | ||
Sine-wave | 5 | 128 | Training: 70% Validation: 20% Test: 10% | Tau = 100 Vthreshold = 1 Vreset = 0 Learning rate = 0.001 |
MGTSD | 1 | 32 | ||
Tetouan energy | 1 | 128 | ||
PM10 concentration | 1 | 64 | ||
ARCTIC | 1 | 64 |
MAE between Decoded Time-Series and Original Time-Series | |||
---|---|---|---|
Vor | Vopt | Difference (%) | |
Sine-wave | 0.0047 | 0.0017 | 63.83% |
MGTSD | 0.0196 | 0.0068 | 65.31% |
Tetouan energy | 0.0048 | 0.002 | 58.33% |
PM10 concentration | 0.0115 | 0.0066 | 42.61% |
ARCTIC | 0.0107 | 0.0057 | 46.73% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lucas, S.; Portillo, E. Application of the Optimised Pulse Width Modulation (PWM) Based Encoding-Decoding Algorithm for Forecasting with Spiking Neural Networks (SNNs). Eng. Proc. 2024, 68, 41. https://doi.org/10.3390/engproc2024068041
Lucas S, Portillo E. Application of the Optimised Pulse Width Modulation (PWM) Based Encoding-Decoding Algorithm for Forecasting with Spiking Neural Networks (SNNs). Engineering Proceedings. 2024; 68(1):41. https://doi.org/10.3390/engproc2024068041
Chicago/Turabian StyleLucas, Sergio, and Eva Portillo. 2024. "Application of the Optimised Pulse Width Modulation (PWM) Based Encoding-Decoding Algorithm for Forecasting with Spiking Neural Networks (SNNs)" Engineering Proceedings 68, no. 1: 41. https://doi.org/10.3390/engproc2024068041