Progress in Developing an Emulation of a Neuromorphic Device That Is Predicted to Enhance Existing Cortical Prosthetic Vision Technology by Engaging Desired Visual Geometries
Abstract
:1. Introduction
1.1. CPV Systems Provide a Distribution of Phosphenes in Visual Space
1.2. Models of Phosphenes and the Visual Geometry That They Occupy
1.3. A Hypothesis Regarding the Nature of Relationships between Objective and Subjective Aspects of CPV
1.4. How Desired Visual Geometries Might Be Engaged Using Available CPV Technology
1.5. Aims of the Present Research
2. Methods
2.1. Overall Design of Computer Simulations and Phosphene Number Classifications
2.2. Details of Computer Simulations
2.3. Details of the Multinomial Logistic Regression Classification of Numbers of Spikes Produced by Excitatory and Inhibitory Neurons in 49 Columns
3. Results
3.1. Replication and Increase in Duration of Neural Network Simulations Spanning 25 Cortical Columns Using NEST
3.2. Simulations of a Neural Network Spanning 49 Cortical Columns Using NEST
3.3. Using Distributions of Numbers of Spikes Produced by Neural Network Neurons to Detect the Number of Phosphenes
3.4. Simulating Essential Features of a Neuromorphic Device That Is Predicted to Engage Desired Visual Geometries
- (1)
- Intermittently recording activity from and stimulating each population for which an electrode is available;
- (2)
- Generating a neuromorphic spike if each recording of population activity is greater than a threshold;
- (3)
- Delivering neuromorphic spikes from all populations to a neuromorphic neuron that does not produce spikes for each population via neuromorphic G system conductance-based synapses; and
- (4)
- Using the membrane potential of each model neuron to determine the contribution of the recorded activity to modulation of stimulation amplitude for each population.
4. Conclusions
5. Future Work
6. Patents
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Correction Statement
References
- Button, J.C. Electronics brings light to the blind. Radio Electron. 1958, 29, 53–55. [Google Scholar]
- Bourne, R.R.A.; Flaxman, S.R.; Braithwaite, T.; Cicinelli, M.V.; Das, A.; Jonas, J.B.; Keeffe, J.; Kempen, J.H.; Leasher, J.; Limburg, H.; et al. Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: A systematic review and meta-analysis. Lancet Glob. Health 2017, 5, e888–e897. [Google Scholar] [CrossRef]
- Varma, R.; Vajaranant, T.S.; Burkemper, B.; Wu, S.; Torres, M.; Hsu, C.; Choudhury, F.; McKean-Cowdin, R. Visual impair-ment and blindness in adults in the United States: Demographic and geographic variations from 2015–2050. JAMA Ophthal-Mol. 2016, 134, 802–809. [Google Scholar] [CrossRef]
- Lewis, P.M.; Rosenfeld, J.V. Electrical stimulation of the brain and the development of cortical visual prostheses: An historical perspective. Brain Res. 2016, 1630, 208–224. [Google Scholar] [CrossRef] [PubMed]
- Niketeghad, S.; Pouratian, N. Brain Machine Interfaces for Vision Restoration: The Current State of Cortical Visual Prosthetics. Neurotherapeutics 2018, 16, 134–143. [Google Scholar] [CrossRef]
- Roe, A.W.; Chen, G.; Xu, A.G.; Hu, J. A roadmap to a columnar visual cortical prosthetic. Curr. Opin. Physiol. 2020, 16, 68–78. [Google Scholar] [CrossRef]
- de Vries, S.E.J.; Lecoq, J.A.; Buice, M.A.; Groblewski, P.A.; Ocker, G.K.; Oliver, M.; Feng, D.; Cain, N.; Ledochowitsch, P.; Millman, D.; et al. A large-scale standardized physiological survey reveals functional organization of the mouse visual cortex. Nat. Neurosci. 2019, 23, 138–151. [Google Scholar] [CrossRef] [PubMed]
- Olshausen, B.A.; Field, D.G. How close are we to understanding V1? Neural Comput. 2005, 17, 1665–1699. [Google Scholar] [CrossRef]
- Chalmers, D.J. Facing up to the problem of consciousness. J. Conscious. Stud. 1995, 2, 200–219. [Google Scholar]
- Chalmers, D.J. The Conscious Mind: In Search of a Fundamental Theory; Oxford University Press: New York, NY, USA, 1996. [Google Scholar]
- Seth, A.K.; Bayne, T. Theories of consciousness. Nat. Rev. Neurosci. 2022, 23, 439–452. [Google Scholar] [CrossRef]
- Gutierrez, G. Orion Shines a Light in the Dark for the Blind, Baylor College of Medicine Blog. July 2019. Available online: https://blogs.bcm.edu/2019/07/11/from-the-labs-orion-turns-on-a-light-in-the-dark-for-the-blind/ (accessed on 30 September 2022).
- Fernández, E.; Normann, R.A. Cortivis approach for an intracortical visual prosthesis. In Artificial Vision: A Clinical Guide; Gabel, V.P., Ed.; Springer: Munich, Germany, 2017; pp. 191–201. [Google Scholar]
- Lowery, A.J.; Rosenfeld, J.V.; Rosa, M.G.P.; Brunton, E.; Rajan, R.; Mann, C.; Armstrong, M.; Mohan, A.; Josh, H.; Kleeman, L.; et al. Monash Vision Group’s Gennaris Cortical Implant for Vision Restoration. In Artificial Vision: A Clinical Guide; Gabel, V.P., Ed.; Springer: Munich, Germany, 2017; pp. 215–225. [Google Scholar]
- Troyk, P.R. The intracortical visual prosthesis project. In Artificial Vision: A Clinical Guide; Gabel, V.P., Ed.; Springer: Munich, Germany, 2017; pp. 203–214. [Google Scholar]
- Schiller, P.H.; Tehovnik, E.J. Visual Prosthesis. Perception 2008, 37, 1529–1559. [Google Scholar] [CrossRef] [PubMed]
- Komarov, M.; Malerba, P.; Golden, R.; Nunez, P.; Halgren, E.; Bazhenov, M. Selective recruitment of cortical neurons by electrical stimulation. PLoS Comput. Biol. 2019, 15, e1007277. [Google Scholar] [CrossRef] [PubMed]
- Rattay, F. The basic mechanism for the electrical stimulation of the nervous system. Neuroscience 1999, 89, 335–346. [Google Scholar] [CrossRef]
- Tootell, R.B.H.; Silverman, M.S.; Switkes, E.; De Valois, R.L. Deoxyglucose Analysis of Retinotopic Organization in Primate Striate Cortex. Science 1982, 218, 902–904. [Google Scholar] [CrossRef] [PubMed]
- Bosking, W.H.; Beauchamp, M.S.; Yoshor, D. Electrical Stimulation of Visual Cortex: Relevance for the Development of Visual Cortical Prosthetics. Annu. Rev. Vis. Sci. 2017, 3, 141–166. [Google Scholar] [CrossRef] [PubMed]
- Bak, M.; Girvin, J.P.; Hambrecht, F.T.; Kufta, C.V.; Loeb, G.E.; Schmidt, E.M. Visual sensations produced by intracortical microstimulation of the human occipital cortex. Med. Biol. Eng. Comput. 1990, 28, 257–259. [Google Scholar] [CrossRef]
- Beauchamp, M.S.; Oswalt, D.; Sun, P.; Foster, B.L.; Magnotti, J.F.; Niketeghad, S.; Pouratian, N.; Bosking, W.H.; Yoshor, D. Dynamic Stimulation of Visual Cortex Produces Form Vision in Sighted and Blind Humans. Cell 2020, 181, 774–783.e5. [Google Scholar] [CrossRef]
- Brindley, G.S.; Lewin, W.S. The sensations produced by electrical stimulation of the visual cortex. J. Physiol. 1968, 196, 479–493. [Google Scholar] [CrossRef]
- Doebelle, W.H. Artificial vision for the blind by connecting a television camera to the visual cortex. Am. Soc. Artif. Intern. Organs J. 2000, 46, 3–9. [Google Scholar] [CrossRef]
- Dobelle, W.H.; Quest, D.O.; Antunes, J.L.; Roberts, T.S.; Girvin, J.P. Artificial Vision for the Blind by Electrical Stimulation of the Visual Cortex. Neurosurgery 1979, 5, 521–527. [Google Scholar] [CrossRef]
- Schmidt, E.M.; Bak, M.J.; Hambrecht, F.T.; Kufta, C.V.; O’Rourke, D.K.; Vallabhanath, P. Feasibility of a visual prosthesis for the blind based on intracortical micro stimulation of the visual cortex. Brain 1996, 119, 507–522. [Google Scholar] [CrossRef] [PubMed]
- Rushton, D.N.; Brindley, G.S. Short- and long-term stability of cortical electrical phosphenes. In Physiological Aspects of Clinical Neurology; Rose, R.C., Ed.; Blackwell Scientific Publishing: London, UK, 1997; pp. 123–153. [Google Scholar]
- Sahel, J.-A.; Boulanger-Scemama, E.; Pagot, C.; Arleo, A.; Galluppi, F.; Martel, J.N.; Degli Esposti, S.; Delaux, A.; de Saint Aubert, J.-B.; de Montleau, C.; et al. Partial recovery of visual function in a blind patient after optogenetic therapy. Nat. Med. 2021, 27, 1223–1229. [Google Scholar] [CrossRef] [PubMed]
- Dobelle, W.H.; Mladejovsky, M.G. Phosphenes produced by electrical stimulation of human occipital cortex, and their application to the development of a prosthesis for the blind. J. Physiol. 1974, 243, 553–576. [Google Scholar] [CrossRef]
- Caspi, A.; Barry, M.P.; Patel, U.K.; Salas, M.A.; Dorn, J.D.; Roy, A.; Niketeghad, S.; Greenberg, R.J.; Pouratian, N. Eye movements and the perceived location of phosphenes generated by intracranial primary visual cortex stimulation in the blind. Brain Stimul. 2021, 14, 851–860. [Google Scholar] [CrossRef] [PubMed]
- Caspi, A.; Roy, A.; Wuyyuru, V.; Rosendall, P.E.; Harper, J.W.; Katyal, K.D.; Barry, M.P.; Dagnelie, G.; Greenberg, R.J. Eye Movement Control in the Argus II Retinal-Prosthesis Enables Reduced Head Movement and Better Localization Precision. Investig. Ophthalmol. Vis. Sci. 2018, 59, 792–802. [Google Scholar] [CrossRef]
- Bosking, W.H.; Sun, P.; Ozker, M.; Pei, X.; Foster, B.L.; Beauchamp, M.S.; Yoshor, D. Saturation in Phosphene Size with Increasing Current Levels Delivered to Human Visual Cortex. J. Neurosci. 2017, 37, 7188–7197. [Google Scholar] [CrossRef] [PubMed]
- Tehovnik, E.J.; Slocum, W.M. Phosphene induction by microstimulation of macaque V1. Brain Res. Rev. 2007, 53, 337–343. [Google Scholar] [CrossRef]
- Dayan, P.; Abbott, L.F. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems; The MIT Press: Cambridge, MA, USA, 2001. [Google Scholar]
- Pavloski, R. Sense Element Engagement Theory Explains How Neural Networks Produce Cortical Prosthetic Vision. U.S. Provisional Patent Application No. 63/287,286, 8 December 2021. [Google Scholar]
- Pavloski, R. Sense Element Engagement Theory Explains How Neural Networks Produce Cortical Prosthetic Vision. IEEE Trans. Neural Netw. Learn. Syst. 2021; Submitted. Available online: https://www.techrxiv.org/articles/preprint/Sense_Element_Engagement_Theory_Explains_How_Neural_Networks_Produce_Cortical_Prosthetic_Vision/17161187/1(accessed on 30 September 2022).
- Schöner, G.; Kelso, J.A.S. Dynamic Pattern Generation in Behavioral and Neural Systems. Science 1988, 239, 1513–1520. [Google Scholar] [CrossRef]
- Kelso, J.A.S. Dynamic Patterns: The Self-Organization of Brain and Behavior; The MIT Press: Cambridge, MA, USA, 1995. [Google Scholar]
- Pehle, C.; Billaudelle, S.; Cramer, B.; Kaiser, J.; Schreiber, K.; Stradmann, Y.; Weis, J.; Leibfried, A.; Müller, E.; Schemmel, J. The BrainScaleS-2 Accelerated Neuromorphic System With Hybrid Plasticity. Front. Neurosci. 2022, 16, 795876. [Google Scholar] [CrossRef]
- Eppler, J.M.; Helias, M.; Muller, E.; Diesmann, M.; Gewaltig, M.-O. PyNEST: A convenient interface to the NEST simulator. Front. Neuroinf. 2008, 2, 12. [Google Scholar] [CrossRef] [PubMed]
- Fardet, T.; Vennemo, S.B.; Mitchell, J.; Mørk, H.; Graber, S.; Hahne, J.; Spreizer, S.; Deepu, R.; Trensch, G.; Weidel, P.; et al. NEST 2.20.0 (2.20.0). Zenodo. Available online: https://doi.org/10.5281/zenodo.3605514 (accessed on 30 September 2022).
- Xu, Z.; Tao, D.; Zhang, Y.; Wu, J.; Tsoi, A.C. Architectural Style Classification Using Multinomial Latent Logistic Regression. In LNCS 8689—Architectural Style Classification Using Multinomial Latent Logistic Regression; Fleet, D., Ed.; ECCV: 2014; Springer International Publishing: Cham, Switzerland, 2014; Available online: https://springer.com (accessed on 30 September 2022).
- Saputro, D.R.S.; Widyaningsih, P. Limited memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) method for the parameter estimation on geographically weighted ordinal logistic regression model (GWOLR). In AIP Conference Proceedings; AIP Publishing LLC: Melville, NY, USA, 2017; Volume 1868, p. 040009. [Google Scholar] [CrossRef]
- Pavloski, R. Progress in Developing a Neuromorphic Device That Is Predicted to Enhance Cortical Prosthetic Vision by Enabling the Formation of Multiple Visual Geometries. U.S. Provisional Patent Application No. 63/354,959, 23 June 2022. [Google Scholar]
Region | Nearest Neighbors | Region | Nearest Neighbors | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 26 | 28 | 11 | 9 | 27 | |||
2 | 9 | 8 | 10 | 3 | 1 | 5 | 11 | 27 | 26 | 9 | 10 | 29 | |||
3 | 8 | 12 | 13 | 2 | 4 | 1 | 14 | 28 | 30 | 25 | 11 | 26 | |||
4 | 3 | 13 | 15 | 1 | 16 | 17 | 6 | 29 | 27 | 10 | 18 | 39 | |||
5 | 10 | 2 | 18 | 1 | 7 | 19 | 9 | 30 | 49 | 25 | 28 | ||||
6 | 1 | 4 | 16 | 7 | 20 | 21 | 22 | 31 | 34 | 48 | 47 | 14 | 24 | 32 | |
7 | 5 | 1 | 6 | 19 | 21 | 23 | 2 | 32 | 14 | 35 | 34 | 31 | 12 | 33 | 13 |
8 | 11 | 24 | 9 | 12 | 3 | 2 | 25 | 33 | 32 | 35 | 13 | 14 | 17 | 15 | |
9 | 26 | 11 | 8 | 27 | 2 | 10 | 28 | 34 | 31 | 32 | 48 | 14 | |||
10 | 27 | 9 | 29 | 2 | 5 | 18 | 26 | 35 | 32 | 34 | 14 | 33 | |||
11 | 28 | 25 | 26 | 24 | 8 | 9 | 30 | 36 | 45 | 22 | 17 | 15 | 37 | 16 | |
12 | 14 | 24 | 31 | 32 | 8 | 13 | 3 | 37 | 33 | 17 | 45 | 15 | |||
13 | 14 | 12 | 33 | 32 | 3 | 15 | 17 | 38 | 22 | 16 | 36 | 20 | 45 | ||
14 | 32 | 12 | 31 | 34 | 35 | 13 | 33 | 38 | 29 | 18 | 40 | ||||
15 | 17 | 13 | 33 | 4 | 22 | 36 | 37 | 40 | 39 | 18 | 19 | 41 | 29 | ||
16 | 22 | 4 | 15 | 17 | 36 | 6 | 38 | 41 | 40 | 19 | 23 | 46 | 18 | ||
17 | 15 | 33 | 13 | 37 | 36 | 22 | 4 | 42 | 20 | 38 | 43 | ||||
18 | 29 | 10 | 5 | 39 | 19 | 40 | 27 | 43 | 21 | 20 | 44 | 42 | |||
19 | 18 | 5 | 7 | 40 | 23 | 41 | 10 | 44 | 23 | 41 | 46 | 43 | 7 | ||
20 | 6 | 16 | 38 | 21 | 22 | 42 | 43 | 45 | 36 | 37 | 17 | 15 | 22 | ||
21 | 7 | 6 | 23 | 20 | 43 | 44 | 1 | 46 | 41 | 23 | 44 | 19 | |||
22 | 16 | 36 | 15 | 17 | 38 | 4 | 45 | 47 | 48 | 49 | 25 | 31 | |||
23 | 19 | 7 | 21 | 42 | 44 | 46 | 5 | 48 | 47 | 31 | 49 | ||||
24 | 25 | 47 | 31 | 11 | 12 | 8 | 48 | 49 | 30 | 47 | 25 | ||||
25 | 30 | 49 | 47 | 28 | 24 | 11 | 48 |
Expected Number of Phosphenes | Regions Stimulated | |
---|---|---|
Non-Altered Visual Geometry | Altered Visual Geometry | |
0 | 0 | None |
1 | 1 | 1 |
1 | 1 | 4 |
1 | 1 | 5 |
1 | 1 | 12 |
1 | 1 | 1, 4 |
1 | 1 | 1, 5 |
1 | 1 | 1, 4, 5 |
2 | 1 | 4, 5 |
2 | 2 | 1, 12 |
2 | 2 | 4, 12 |
2 | 2 | 5, 12 |
3 | 2 | 4, 5, 12 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Pavloski, R. Progress in Developing an Emulation of a Neuromorphic Device That Is Predicted to Enhance Existing Cortical Prosthetic Vision Technology by Engaging Desired Visual Geometries. Prosthesis 2022, 4, 600-623. https://doi.org/10.3390/prosthesis4040049
Pavloski R. Progress in Developing an Emulation of a Neuromorphic Device That Is Predicted to Enhance Existing Cortical Prosthetic Vision Technology by Engaging Desired Visual Geometries. Prosthesis. 2022; 4(4):600-623. https://doi.org/10.3390/prosthesis4040049
Chicago/Turabian StylePavloski, Raymond. 2022. "Progress in Developing an Emulation of a Neuromorphic Device That Is Predicted to Enhance Existing Cortical Prosthetic Vision Technology by Engaging Desired Visual Geometries" Prosthesis 4, no. 4: 600-623. https://doi.org/10.3390/prosthesis4040049