IoT-Oriented Design of an Associative Memory Based on Impulsive Hopfield Neural Network with Rate Coding of LIF Oscillators
Abstract
:1. Introduction
2. Materials and Methods
2.1. Relaxation Oscillators Based on S-Switch Elements
2.2. Feedbacks of LIF Neurons
2.3. Neuron Based on OSC1 (Neuron 1)
2.4. Neuron Based on OSC2 (Neuron 2)
2.5. The Principle of Operation of IHN Based on Rate Coding
3. Results
3.1. IHN with Four LIF Neurons
3.2. IHN with Nine LIF Neurons
4. Discussion
5. Conclusions
Funding
Acknowledgments
Conflicts of Interest
Appendix A
References
- Korzun, D.; Kashevnik, A.; Balandin, S.; Viola, F. Ambient Intelligence Services in IoT Environments: Emerging Research and Opportunities. In Advances in Wireless Technologies and Telecommunication (AWTT) Book Series; IGI Global: Hershey, PA, USA, 2019; p. 199. [Google Scholar] [CrossRef]
- Prutyanov, V.; Melentev, N.; Lopatkin, D.; Menshchikov, A.; Somov, A. Developing IoT Devices Empowered by Artificial Intelligence: Experimental Study. In Proceedings of the 2019 Global IoT Summit (GIoTS), Aarhus, Denmark, 17–21 June 2019; pp. 1–6. [Google Scholar] [CrossRef]
- Cassidy, A.; Georgiou, J.; Andreou, A. Design of silicon brains in the nano-CMOS era: Spiking neurons, learning synapses and neural architecture optimization. Neural Netw. 2013, 45, 4–26. [Google Scholar] [CrossRef]
- Pfeiffer, M.; Pfeil, T. Deep learning with spiking neurons: Opportunities & Challenges. Front. Neurosci. 2018, 12, 774. [Google Scholar] [CrossRef] [Green Version]
- Mausfeld, R. The Biological Function of Sensory Systems. In Neurosciences—From Molecule to Behavior: A University Textbook; Giovanni, G., Pierre-Marie, L., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 239–252. [Google Scholar] [CrossRef] [Green Version]
- Maass, W.; Natschläger, T. Emulation of Hopfield Networks with Spiking Neurons in Temporal Coding. In Computational Neuroscience: Trends in Research; Bower, J., Ed.; Springer: Berlin/Heidelberg, Germany, 1998; pp. 221–226. [Google Scholar]
- Maass, W. Networks of spiking neurons: The third generation of neural network models. Neural Netw. 1997, 10, 1659–1671. [Google Scholar] [CrossRef]
- Paugam-Moisy, H.; Bohte, S. Computing with spiking neuron networks. In Handbook of Natural Computing; Rozenberg, G., Bäck, T., Kok, J.N., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; pp. 335–376. [Google Scholar] [CrossRef] [Green Version]
- Thorpe, S.; Delorme, A.; Van Rullen, R. Spike-based strategies for rapid processing. Neural Netw. 2001, 14, 715–725. [Google Scholar] [CrossRef]
- Velichko, A.; Belyaev, M.; Boriskov, P. A Model of an Oscillatory Neural Network with Multilevel Neurons for Pattern Recognition and Computing. Electronics 2019, 8, 75. [Google Scholar] [CrossRef] [Green Version]
- Brette, R. Philosophy of the Spike: Rate-Based vs. Spike-Based Theories of the Brain. Front. Syst. Neurosci. 2015, 9, 151. [Google Scholar] [CrossRef]
- Koch, C.; Segev, I. Methods in Neuronal Modeling: From Ions to Networks, 2nd ed.; MIT Press: Cambridge, MA, USA, 1998. [Google Scholar]
- Wang, D. Relaxation Oscillators and Networks. In Wiley Encyclopedia of Electrical and Electronics Engineering (18); Webster, J.G., Ed.; Wiley & Sons: New York, NY, USA, 1999; pp. 396–405. [Google Scholar] [CrossRef]
- Borkisov, P.; Velichko, A. Switch Elements with S-Shaped Current-Voltage Characteristic in Models of Neural Oscillators. Electronics 2019, 8, 922. [Google Scholar] [CrossRef] [Green Version]
- Del Valle, J.; Ramírez, J.G.; Rozenberg, M.J.; Schuller, I.K. Challenges in materials and devices for resistive-switching-based neuromorphic computing. J. Appl. Phys. 2018, 124, 211101. [Google Scholar] [CrossRef]
- Xia, Q.; Yang, J.J. Memristive crossbar arrays for brain-inspired computing. Nat. Mater. 2019, 18, 309–323. [Google Scholar] [CrossRef] [PubMed]
- Pergament, A.; Velichko, A.; Belyaev, M.; Putrolaynen, V. Electrical switching and oscillations in vanadium dioxide. Phys. B Condens. Matter 2018, 536, 239–248. [Google Scholar] [CrossRef] [Green Version]
- Velichko, A.; Boriskov, P. Concept of LIF Neuron Circuit for Rate Coding in Spike Neural Networks. IEEE Trans. Circuits Syst. II Express Briefs 2020. [Google Scholar] [CrossRef]
- Hopfield, J.J. Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci. USA 1982, 79, 2554–2558. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hopfield, J.J. Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Nat. Acad. Sci. USA 1984, 81, 3088–3092. [Google Scholar] [CrossRef] [Green Version]
- Yu, Z.; Abdulghani, A.M.; Zahid, A.; Heidari, H.; Imran, M.A.; Abbasi, Q.H. An Overview of Neuromorphic Computing for Artificial Intelligence Enabled Hardware-based Hopfield Neural Network. IEEE Access 2020, 8, 67085–67099. [Google Scholar] [CrossRef]
- Tankimanova, A.; James, A.P. Neural network-based analog-to-digital converters. In Memristor and Memristive Neural Networks; James, A.P., Ed.; Nazarbayev University: Astana, Kazakhstan, 2018; pp. 297–314. [Google Scholar] [CrossRef] [Green Version]
- Marblestone, A.H.; Wayne, G.; Kording, K.P. Toward an integration of deep learning and neuroscience. Front. Syst. Neurosci. 2016, 10, 94. [Google Scholar] [CrossRef] [PubMed]
- Keeler, J.D. Basins of Attraction of Neural Network Model. AIP Conf. Proc. 1986, 151, 259–264. [Google Scholar] [CrossRef]
- Yang, X.; Zhao, L.; Megson, G.M.; Evans, D.J. A system-level fault diagnosis algorithm based on preprocessing and parallel Hopfield neural network. In Proceedings of the 4th IEEE Workshop RTL High Level Test, Xi’an, China, 20–21 November 2003; pp. 189–196. [Google Scholar]
- Yang, H.; Wang, B.; Yao, Q.; Yu, A.; Zhang, J. Eficient hybrid multifaults location based on hopfield neural network in 5G coexisting radioand optical wireless networks. IEEE Trans. Cogn. Commun. Netw. 2019, 5, 1218–1228. [Google Scholar] [CrossRef]
- Wang, B.; Yang, H.; Yao, Q.; Yu, A.; Hong, T.; Zhang, J.; Kadoch, M.; Cheriet, M. Hopfield neural network-based fault location in wireless and optical networks for smart city IoT. In Proceedings of the 15th International Wireless Communications & Mobile Computing Conference (IWCMC), Tangier, Morocco, 24–28 June 2019; pp. 1696–1701. [Google Scholar]
- Levine, W.S. The Control Handbook, 1st ed.; CRC Press: New York, NY, USA, 1996. [Google Scholar]
- Darabi, H. Radio Frequency Integrated Circuits and Systems, 2nd ed.; University of California: Los Angeles, CA, USA, 2020. [Google Scholar]
- Hebb, D.O. The organization of behavior. In Neurocomputing: Foundations of Research; Anderson, J.A., Rosenfeld, E., Eds.; MIT Press: Cambridge, MA, USA, 1988. [Google Scholar]
- Nikonov, D.E.; Csab, G.; Porod, W.; Shibata, T.; Voils, D.; Hammerstrom, D.; Young, I.A.; Bourianoff, G. Coupled-oscillator associative memory array operation for pattern recognition. IEEE J. Explor. Solid State Comput. Devices Circuits 2015, 1, 85–93. [Google Scholar] [CrossRef]
- López, A.; Suarez, J.; Varela, D. Execution and analysis of classic neural network algorithms when they are implemented in embedded systems. In Proceedings of the 23rd International Conference on Circuits, Systems, Communications and Computers (CSCC 2019), Marathon Beach, Athens, 14–17 July 2019; EDP Sciences: Paris, France, 2019; Volume 292, pp. 1–8. [Google Scholar] [CrossRef]
- Korzun, D.; Voronin, A.; Shegelman, I. Semantic Data Mining Based on Ranking in Internet-Enabled Information Systems. In Frontiers in Artificial Intelligence and Applications, Book Series; IOS Press: Amsterdam, The Netherlands, 2020; Volume 320, pp. 237–242. [Google Scholar] [CrossRef]
- Kosko, B. Bi-directional associative memories. IEEE Trans. Syst. Man Cybern. 1987, 18, 49–60. [Google Scholar] [CrossRef] [Green Version]
- Jordan, M.I. Serial Order: A Parallel Distributed Processing Approach. Adv. Psychol. 1997, 121, 471–495. [Google Scholar] [CrossRef]
- Jaeger, H.; Haas, H. Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication. Science 2004, 304, 78–80. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Maass, W.; Natschläger, T.; Markram, H. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Comput. 2002, 14, 2531–2560. [Google Scholar] [CrossRef] [PubMed]
- Zhang, X.; Li, C.; Huang, T. Hybrid impulsive and switching Hopfield neural networks with state-dependent impulses. Neural Netw. 2017, 93, 176–184. [Google Scholar] [CrossRef] [PubMed]
- Nagamani, G.; Ramasamy, S.; Meyer-Baese, A. Robust dissipativity and passivity based state estimation for discrete-time stochastic Markov jump neural networks with discrete and distributed time-varying delays. Neural Comput. Appl. 2015, 28, 717–735. [Google Scholar] [CrossRef]
Switch | OSC1 | OSC2 | ||||||
---|---|---|---|---|---|---|---|---|
Uth, V | Ith, mA | Uh, V | Ih, mA | Ron, Ω | Roff, kΩ | C0, nF | C1, nF | C2, µF |
4 | 0.1 | 2 | 10 | 200 | 40 | 5 | 5 | 1 |
Neuron 1 | Neuron 2 | ||||||||
---|---|---|---|---|---|---|---|---|---|
Imax, mA | Umax, V | Imin, mA | Umin, V | Rmax, MΩ | Umax, V | Rmin, Ω | Umin, V | Ro, Ω | U0, V |
1.6 | 0.5 | 0.4 | 0.1 | 1 | 0.69 | 0 | ~0 | 200 | 0.15 |
IHN1 (Template A) | IHN2 (Template A) | ||||||
---|---|---|---|---|---|---|---|
Io1, mA | Io2, mA | Io3, mA | Io4,mA | Ro1, Ω | Ro2, Ω | Ro3, Ω | Ro4, Ω |
1.18 | 1.0 | 1.31 | 1.0 | 300 | 200 | 450 | 200 |
Figure 10 | Figure 9 | ||||||
τo, s | KI, mA·V−1 | τo, s | KR, kΩ·V−1 | ||||
(a) | 0.1 | 3 | (a) | 0.25 | 103 | ||
(b) | 0.1 | 103 | |||||
(b) | 0.1 | 5 | (c) | 0.1 | 10 | ||
(d) | 0.07 | 1 |
IHN1 (Figure 12a) | IHN2 (Figure 12b) | ||||||||
---|---|---|---|---|---|---|---|---|---|
Template C, Figure 11 | Neuron out voltage (j = 9) | τo, s | KI, mA·V−1 | Template C, Figure 11 | Neuron out voltage (j = 9) | τo, s | KR, kΩ·V−1 | ||
Neuron, number | Ioi, mA | Neuron, number | Roi, Ω | ||||||
1 | 1.55 | U9(1) | 0.1 | 3 | 1 | 1000 | U9(1) | 0.1 | 105 |
2 | 0.45 | 2 | 50 | ||||||
3 | 1.4 | 3 | 500 | ||||||
4 | 0.4 | U9(2) | 0.1 | 5 | 4 | 30 | U9(2) | 0.1 | 103 |
5 | 1.5 | 5 | 800 | ||||||
6 | 1 | 6 | 200 | ||||||
7 | 1.2 | U9(3) | 0.1 | 9 | 7 | 350 | U9(3) | 0.1 | 1 |
8 | 0.48 | 8 | 80 | ||||||
9 | 0.9 | 9 | 150 |
© 2020 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Boriskov, P. IoT-Oriented Design of an Associative Memory Based on Impulsive Hopfield Neural Network with Rate Coding of LIF Oscillators. Electronics 2020, 9, 1468. https://doi.org/10.3390/electronics9091468
Boriskov P. IoT-Oriented Design of an Associative Memory Based on Impulsive Hopfield Neural Network with Rate Coding of LIF Oscillators. Electronics. 2020; 9(9):1468. https://doi.org/10.3390/electronics9091468
Chicago/Turabian StyleBoriskov, Petr. 2020. "IoT-Oriented Design of an Associative Memory Based on Impulsive Hopfield Neural Network with Rate Coding of LIF Oscillators" Electronics 9, no. 9: 1468. https://doi.org/10.3390/electronics9091468