*2.1. VO<sup>2</sup> Neuron Model*

The VO2-neuron model is created on the basis of the LIF neuron model, which is widely used due to the simplicity of implementation and the possibility of generating biosimilar spikes [39]. Its main element is a bi-stable two-electrode VO<sup>2</sup> switch [25–27]. The operation principle of the switch is based on the metal-insulator phase transition in VO<sup>2</sup> films, which happens near the transition temperature *T*th ~ 340 K. The critical temperature *T*th in the film is achieved due to the Joule heating effect when passing a current, which leads to a sharp abrupt change in the resistance [27]. In addition to the thermal effect, when modeling electric switching, the effect of the electric field on the concentration of charge carriers is taken into account [40,41]. The model I – V characteristic of the VO<sup>2</sup> switch corresponds to the experimental I – V characteristic (Figure 1), measured in our previous work [25], on a planar switch with a channel size of 2.5–3 µm, and a VO<sup>2</sup> film thickness of ~ 250 nm, with a current limiting resistor of 250 Ω connected in series.

at the output.

**Figure 1.** Experimental I–V characteristic of a planar VO<sup>2</sup> switch.

**Figure 1.** Experimental I–V characteristic of a planar VO2 switch.

Figure 1 demonstrates the dependence of the switch current *I*sw on the voltage *V*sw supplied to the switch. Reaching the threshold switching voltages *V*th = 5.6 V and holding voltage *V*h = 2.2 V, the switch passes from a high-resistance state to a low-resistance state and vice versa. The high-resistance and low-resistance branches of the I – V characteristic are approximated by linear dependencies on Figure 1 demonstrates the dependence of the switch current *I*sw on the voltage *V*sw supplied to the switch. Reaching the threshold switching voltages *V*th = 5.6 V and holding voltage *V*<sup>h</sup> = 2.2 V, the switch passes from a high-resistance state to a low-resistance state and vice versa. The high-resistance and low-resistance branches of the I – V characteristic are approximated by linear dependencies on the voltage *V*sw with resistance values *R*off ~ 14 kΩ and *R*on ~ 300 Ω, respectively.

the voltage *V*sw with resistance values *R*off ~ 14 kΩ and *R*on ~ 300 Ω, respectively. To conduct SPICE simulations of the VO2 neuron, a standard voltage-controlled switch was used To conduct SPICE simulations of the VO<sup>2</sup> neuron, a standard voltage-controlled switch was used with parameters corresponding to the experimental I – V characteristics (*R*off, *R*on, *V*th*,* and *V*h).

with parameters corresponding to the experimental I – V characteristics (*R*off, *R*on, *V*th*,* and *V*h). The electrical circuit of the VO2 neuron is shown in Figure 2. The neuron model has n inputs, and one output *V*out. Resistances 1 n w w *R* ...*R* play the role of a synaptic weights between neurons. The smaller the resistance, the more the signal from the *i*-th input affects the neuron. The spikes coming from the inputs through the resistances are accumulated on the *C*sum capacitance, by charging it with the cumulative charge. The charge from the *C*sum capacitor gradually flows through the resistance *R*in. *C*sum capacitance voltage is an effective input signal that affects the current state of a neuron. The supply voltage *V*dd is selected so that the VO2 switch stays in the off state in the absence of input signals. The most clear way to achieve this condition is to set the voltage *V*dd less than the switching voltage *V*th. In this model, the inactive state of the neuron corresponds to the switched off VO2 switch when it is in the subthreshold mode (*V*sw <*V*th). To activate the neuron, the switch should be turned on by setting the voltage on the switch to *V*sw≥*V*th. To achieve this, the supply voltage *V*dd must have The electrical circuit of the VO<sup>2</sup> neuron is shown in Figure 2. The neuron model has n inputs, and one output *V*out. Resistances *R* 1 <sup>w</sup> . . . *R* n <sup>w</sup> play the role of a synaptic weights between neurons. The smaller the resistance, the more the signal from the *i*-th input affects the neuron. The spikes coming from the inputs through the resistances are accumulated on the *C*sum capacitance, by charging it with the cumulative charge. The charge from the *C*sum capacitor gradually flows through the resistance *R*in. *C*sum capacitance voltage is an effective input signal that affects the current state of a neuron. The supply voltage *V*dd is selected so that the VO<sup>2</sup> switch stays in the off state in the absence of input signals. The most clear way to achieve this condition is to set the voltage *V*dd less than the switching voltage *V*th. In this model, the inactive state of the neuron corresponds to the switched off VO<sup>2</sup> switch when it is in the subthreshold mode (*V*sw <*V*th). To activate the neuron, the switch should be turned on by setting the voltage on the switch to *V*sw≥*V*th. To achieve this, the supply voltage *V*dd must have negative values, and the spikes supplied to the input must have a positive polarity.

negative values, and the spikes supplied to the input must have a positive polarity. To activate a neuron, the voltage across the capacitance *C*sum should increase to a threshold value *V*c\_th, which depends on *V*dd, the resistance of the switch in the off state *R*off, and the values of the resistors *R*s and *R*in. After the switch is turned on, its resistance decreases to *R*on, which leads to the discharge of the capacitance *C*c through the resistances *R*in and *R*out. The capacitance *C*c serves as a reservoir of charge, which is necessary for generating a spike when a neuron is activated. When *C*c is discharged, a spike of positive polarity is generated at the output *V*out of the neuron. By connecting the outputs of some neurons with the inputs of other neurons, SNNs with excitatory coupling can be obtained. The resistance *R*s is load resistance and sets the operating current through the switch. *R*s is selected in a way that the VO2 switch turns off after discharging the capacitance *C*c. In fact, the neuron circuit is tuned to generate a single current spike through the VO2 switch, i.e., generate a single spike To activate a neuron, the voltage across the capacitance *C*sum should increase to a threshold value *V*c\_th, which depends on *V*dd, the resistance of the switch in the off state *R*off, and the values of the resistors *R*<sup>s</sup> and *R*in. After the switch is turned on, its resistance decreases to *R*on, which leads to the discharge of the capacitance *C*<sup>c</sup> through the resistances *R*in and *R*out. The capacitance *C*<sup>c</sup> serves as a reservoir of charge, which is necessary for generating a spike when a neuron is activated. When *C*<sup>c</sup> is discharged, a spike of positive polarity is generated at the output *V*out of the neuron. By connecting the outputs of some neurons with the inputs of other neurons, SNNs with excitatory coupling can be obtained. The resistance *R*<sup>s</sup> is load resistance and sets the operating current through the switch. *R*<sup>s</sup> is selected in a way that the VO<sup>2</sup> switch turns off after discharging the capacitance *C*c. In fact, the neuron circuit is tuned to generate a single current spike through the VO<sup>2</sup> switch, i.e., generate a single spike at the output.

**Figure 2.** Electrical circuit of a VO<sup>2</sup> neuron. **Figure 2.** Electrical circuit of a VO2 neuron.

**Figure 2.** Electrical circuit of a VO2 neuron. Figure 3a presents the electrical circuit of a VO2 neuron. The pulses from the voltage generator are supplied to the input of the neuron, and the output is connected to the input stage of the subsequent neuron (to simulate the output load of the neuron). The circuit modeling was performed in the LTspice XVII simulation software. Resistance and capacitance values: *R*w\_1 = 500 Ω, *R*w\_2 = 1 kΩ, *R*s = 700 Ω, *R*in = 1 kΩ, *R*out = 10 kΩ, *C*sum = 1 nF, and *C*c = 10 nF. Supply voltage *V*dd = -5.75 V. A pulse of positive polarity with an amplitude of 2 V and a duration of 0.3 μs is supplied from the generator. Figure 3b depicts the oscillograms of the input *V*in and output *V*out voltages, as well as the voltages *V*<sup>c</sup> Figure 3a presents the electrical circuit of a VO<sup>2</sup> neuron. The pulses from the voltage generator are supplied to the input of the neuron, and the output is connected to the input stage of the subsequent neuron (to simulate the output load of the neuron). The circuit modeling was performed in the LTspice XVII simulation software. Resistance and capacitance values: *R*w\_1 = 500 Ω, *R*w\_2 = 1 kΩ, *R*<sup>s</sup> = 700 Ω, *R*in = 1 kΩ, *R*out = 10 kΩ, *C*sum = 1 nF, and *C*<sup>c</sup> = 10 nF. Supply voltage *V*dd = −5.75 V. A pulse of positive polarity with an amplitude of 2 V and a duration of 0.3 µs is supplied from the generator. Figure 3b depicts the oscillograms of the input *V*in and output *V*out voltages, as well as the voltages *V*<sup>c</sup> at the capacitance *C*sum, which demonstrates the spikes' dynamics. Figure 3a presents the electrical circuit of a VO2 neuron. The pulses from the voltage generator are supplied to the input of the neuron, and the output is connected to the input stage of the subsequent neuron (to simulate the output load of the neuron). The circuit modeling was performed in the LTspice XVII simulation software. Resistance and capacitance values: *R*w\_1 = 500 Ω, *R*w\_2 = 1 kΩ, *R*s = 700 Ω, *R*in = 1 kΩ, *R*out = 10 kΩ, *C*sum = 1 nF, and *C*c = 10 nF. Supply voltage *V*dd = -5.75 V. A pulse of positive polarity with an amplitude of 2 V and a duration of 0.3 μs is supplied from the generator. Figure 3b depicts the oscillograms of the input *V*in and output *V*out voltages, as well as the voltages *V*<sup>c</sup> at the capacitance *C*sum, which demonstrates the spikes' dynamics.

(**a**) **Figure 3.** *Cont*.

(**b**)

**Figure 3.** (**a**) An example of an electrical circuit of a VO2 neuron activated by a voltage generator, and (**b**) oscillograms of voltages *V*in, *V*out, and *V*c illustrating the spikes' dynamics. **Figure 3.** (**a**) An example of an electrical circuit of a VO<sup>2</sup> neuron activated by a voltage generator, and (**b**) oscillograms of voltages *V*in, *V*out, and *V*<sup>c</sup> illustrating the spikes' dynamics.

The threshold voltage at the *C*sum capacitance, required to initiate the output spike, is *V*c\_th ~ 0.33 V (dashed line in Figure 3b). After turning on the VO2 switch, the capacitance *C*c starts to discharge, and it leads to the appearance of a leading edge and a spike with a voltage amplitude of ~ 3.2 V. After turning on the switch, a decrease in the voltage *V*c to negative values is associated with active recharging of the *C*sum capacitor through an open switch due to the negative voltage on *C*c capacity and *V*dd power supply. The spike duration is ~ 170 ns, which is determined by the discharge time of the capacitance *C*c until the moment, when the voltage at the switch *V*sw is not less than *V*h. The trailing edge of the pulse appears when the switch goes off. The duration of the output spike can be significantly longer than the duration of the initiating pulse. The threshold voltage at the *C*sum capacitance, required to initiate the output spike, is *V*c\_th ~ 0.33 V (dashed line in Figure 3b). After turning on the VO<sup>2</sup> switch, the capacitance *C*<sup>c</sup> starts to discharge, and it leads to the appearance of a leading edge and a spike with a voltage amplitude of ~ 3.2 V. After turning on the switch, a decrease in the voltage *V*<sup>c</sup> to negative values is associated with active recharging of the *C*sum capacitor through an open switch due to the negative voltage on *C*<sup>c</sup> capacity and *V*dd power supply. The spike duration is ~ 170 ns, which is determined by the discharge time of the capacitance *C*<sup>c</sup> until the moment, when the voltage at the switch *V*sw is not less than *V*h. The trailing edge of the pulse appears when the switch goes off. The duration of the output spike can be significantly longer than the duration of the initiating pulse.

The VO2-neuron model is able to demonstrate various properties of real neurons, such as spike latency, subthreshold oscillations, refractory period, threshold behavior, and spike frequency adaptation [28,39]. The VO2-neuron model is able to demonstrate various properties of real neurons, such as spike latency, subthreshold oscillations, refractory period, threshold behavior, and spike frequency adaptation [28,39].

For example, Figure 4a demonstrates that the higher the amplitude of the input pulse exists, the smaller the time delay between the leading edges of the input and output pulses remains, called spike latency. With a pulse amplitude of 2 V, the latency between the input and output signals is 140 ns, and with a pulse amplitude of 1 V, the latency reaches 440 ns. Therefore, the amplitude and duration of the input pulse, required to initiate the spike, can lie in a wide range. However, when the amplitude of the input pulse is less than *V*c\_th, the initiation of the output pulse does not occur. For example, Figure 4a demonstrates that the higher the amplitude of the input pulse exists, the smaller the time delay between the leading edges of the input and output pulses remains, called spike latency. With a pulse amplitude of 2 V, the latency between the input and output signals is 140 ns, and with a pulse amplitude of 1 V, the latency reaches 440 ns. Therefore, the amplitude and duration of the input pulse, required to initiate the spike, can lie in a wide range. However, when the amplitude of the input pulse is less than *V*c\_th, the initiation of the output pulse does not occur.

If the input pulse is sufficiently long, several spikes can be obtained at the output of the circuit. Figure 4b demonstrates the response of a VO2 neuron to a pulse with an amplitude of 1 V and a duration of 3.6 μs, which forms five spikes at the output. The latter mode resembles the occurrence of oscillations when an excitation signal is applied (subthreshold oscillations). The delay between the spikes *T*r, called the refractory period, is approximately 630 ns and is determined by the charging time of the capacitor *C*sum to voltage *V*c\_th. The refractory period depends on the amplitude of the pulse. For example, at a pulse amplitude of 2 V, the period *T*r is 300 ns. In addition, the refractory period *T*r is slightly increasing (see the values indicated in Figure 4b), because of the small increase in *V*c\_th from spike to spike, since the capacitance *C*c does not have time to charge to its original values. This increase in the time period between the spikes under constant exposure is similar to biological neurons (spike frequency adaptation) [28,39]. If the input pulse is sufficiently long, several spikes can be obtained at the output of the circuit. Figure 4b demonstrates the response of a VO<sup>2</sup> neuron to a pulse with an amplitude of 1 V and a duration of 3.6 µs, which forms five spikes at the output. The latter mode resembles the occurrence of oscillations when an excitation signal is applied (subthreshold oscillations). The delay between the spikes *T*r, called the refractory period, is approximately 630 ns and is determined by the charging time of the capacitor *C*sum to voltage *V*c\_th. The refractory period depends on the amplitude of the pulse. For example, at a pulse amplitude of 2 V, the period *T*<sup>r</sup> is 300 ns. In addition, the refractory period *T*<sup>r</sup> is slightly increasing (see the values indicated in Figure 4b), because of the small increase in *V*c\_th from spike to spike, since the capacitance *C*<sup>c</sup> does not have time to charge to its original values. This increase in the time period between the spikes under constant exposure is similar to biological neurons (spike frequency adaptation) [28,39].

**Figure 4.** (**a**) Oscillograms of *V*in, *V*c, and *V*out, applying to the VO2-neuron input pulses of different duration and amplitude, (**b**) one long pulse, and (**c**) two pulses with a small delay between them. **Figure 4.** (**a**) Oscillograms of *V*in, *V*c, and *V*out, applying to the VO<sup>2</sup> -neuron input pulses of different duration and amplitude, (**b**) one long pulse, and (**c**) two pulses with a small delay between them.

If the delay between the spikes is less than the refractory period, the neuron generates only one spike. Figure 4c demonstrates two input pulses with an amplitude of 1 V and a delay of 300 ns, and the neuron generates a spike only for the first input pulse. If the delay between the spikes is less than the refractory period, the neuron generates only one spike. Figure 4c demonstrates two input pulses with an amplitude of 1 V and a delay of 300 ns, and the neuron generates a spike only for the first input pulse.

To implement the wide functionality of neural networks, in addition to excitation connections, the possibility to add inhibitory connections is required. Inhibitory connections are widely used in the SNN output layer to implement the WTA rule. Such connections allow the first spike-generated neuron to deactivate all other related neurons using the inhibitory connections. As a result, only one neuron, which is associated with a recognized class, is activated. Figure 5a demonstrates a diagram of two neurons interconnected via capacitances *C*inh = 10 nF, which act as inhibitory connections. The capacitance and resistance values correspond to the single neuron circuit shown in Figure 3a, with the exception of *R*in = 200 Ω and *R*out = 200 Ω. Due to the presence of *C*inh capacities, upon activation of one of the neurons and the discharge of its capacitance *C*c, the voltage on the capacitance *C*c of an inactive neuron decreases. In this case, the first (in time) activated neuron will suppress all other neurons connected to it by inhibitory connections. Namely, in such a group of neurons, the WTA rule is implemented. In order to trace the activation of neurons in this circuit, it is convenient to monitor the current *I*sw and voltage *V*sw on two switches (Figure 5b). The delay between the supplied pulses *V*in\_1 and *V*in\_2 is 2 μs. When the first pulse *V*in\_1 arrives at the first switch (Figure 5b), the switch turns on, the current *I*sw\_1 increases sharply, and the on mode lasts for ~ 4.2 μs. Switching on occurs because the voltage *V*sw\_1 reaches the threshold value *V*th (Figure 5b). After turning on the first switch, the To implement the wide functionality of neural networks, in addition to excitation connections, the possibility to add inhibitory connections is required. Inhibitory connections are widely used in the SNN output layer to implement the WTA rule. Such connections allow the first spike-generated neuron to deactivate all other related neurons using the inhibitory connections. As a result, only one neuron, which is associated with a recognized class, is activated. Figure 5a demonstrates a diagram of two neurons interconnected via capacitances *C*inh = 10 nF, which act as inhibitory connections. The capacitance and resistance values correspond to the single neuron circuit shown in Figure 3a, with the exception of *R*in = 200 Ω and *R*out = 200 Ω. Due to the presence of *C*inh capacities, upon activation of one of the neurons and the discharge of its capacitance *C*c, the voltage on the capacitance *C*<sup>c</sup> of an inactive neuron decreases. In this case, the first (in time) activated neuron will suppress all other neurons connected to it by inhibitory connections. Namely, in such a group of neurons, the WTA rule is implemented. In order to trace the activation of neurons in this circuit, it is convenient to monitor the current *I*sw and voltage *V*sw on two switches (Figure 5b). The delay between the supplied pulses *V*in\_1 and *V*in\_2 is 2 µs. When the first pulse *V*in\_1 arrives at the first switch (Figure 5b), the switch turns on, the current *I*sw\_1 increases sharply, and the on mode lasts for ~ 4.2 µs. Switching on occurs because the voltage *V*sw\_1 reaches the threshold value *V*th (Figure 5b).

After turning on the first switch, the voltage *V*sw\_1 drops sharply and it leads to a decrease in voltage *V*sw\_2 on the second switch, as the signal is transmitted through the capacitors *C*inh. The second pulse arriving at the input of the second neuron (*V*in\_2) does not activate it, because the voltage *V*sw\_2 does not reach the threshold value (*V*sw\_2 <*V*th). The activation of the first neuron inhibits the activation of the second neuron. If an excitation pulse is applied to the second neuron after deactivation of the first neuron, then the second neuron will go into an active mode. *Electronics* **2019**, *8*, x FOR PEER REVIEW 7 of 19 signal is transmitted through the capacitors *C*inh. The second pulse arriving at the input of the second neuron (*V*in\_2) does not activate it, because the voltage *V*sw\_2 does not reach the threshold value (*V*sw\_2 <*V*th). The activation of the first neuron inhibits the activation of the second neuron. If an excitation pulse is applied to the second neuron after deactivation of the first neuron, then the second neuron will go into an active mode.

**Figure 5.** (**a**) Connection diagram of two oscillators with inhibitory connections. (**b**) Oscillograms of the input signals *V*in\_1, *V*in\_2, voltages *V*sw\_1, *V*sw\_2 and currents *I*sw\_1, *I*sw\_2 on the switches, and (**c**) oscillograms of the input signals *V*in\_1, *V*in\_2 and voltages on the integrating capacitors *V*c\_1, *V*c\_2, when applying two voltage pulses with a delay of 2 μs. **Figure 5.** (**a**) Connection diagram of two oscillators with inhibitory connections. (**b**) Oscillograms of the input signals *V*in\_1, *V*in\_2, voltages *V*sw\_1, *V*sw\_2 and currents *I*sw\_1, *I*sw\_2 on the switches, and (**c**) oscillograms of the input signals *V*in\_1, *V*in\_2 and voltages on the integrating capacitors *V*c\_1, *V*c\_2, when applying two voltage pulses with a delay of 2 µs.

To determine the activity of neurons in the output layer, the most appropriate solution would be to convert the *I*sw current pulses into output voltage pulses. However, this solution requires additional external circuits. As neurons are connected by *C*inh capacities, voltages, taken from *R*out resistors, are correlated, and it is not advisable to use them. Schematically, as activity markers, the voltages *V*c\_1 and *V*c\_2 can be used, and their dynamics are shown in Figure 5c. When the first neuron is activated, the voltage *V*c\_1 drops sharply due to the recharging of the capacitor *C*sum, which forms a strong pulse of negative polarity, and the positive pulse *V*c\_2 is weakly expressed on the inactive neuron. To determine the activity of neurons in the output layer, the most appropriate solution would be to convert the *I*sw current pulses into output voltage pulses. However, this solution requires additional external circuits. As neurons are connected by *C*inh capacities, voltages, taken from *R*out resistors, are correlated, and it is not advisable to use them. Schematically, as activity markers, the voltages *V*c\_1 and *V*c\_2 can be used, and their dynamics are shown in Figure 5c. When the first neuron is activated, the voltage *V*c\_1 drops sharply due to the recharging of the capacitor *C*sum, which forms a strong pulse of negative polarity, and the positive pulse *V*c\_2 is weakly expressed on the inactive neuron.

### *2.2. SNN Architecture 2.2. SNN Architecture*

For pattern recognition problems, various SNN architectures are used, which differ in the number of layers and in the way neurons are connected [14,16]. One of the simplest SNN architectures is a two-layer network (Figure 6a), where image information is supplied to the input (first layer), and one of the neurons associated with a certain class of images is activated at the output (second layer) [42–44]. Each of the first layer neurons is connected to each neuron of the second layer through excitatory connections. The connection strength between each pair of neurons is specified through synaptic weights, which can vary among themselves. All neurons of the output layer are interconnected by inhibitory connections. When applying signals to the first layer neurons, they are activated and transmit an excitation effect to the second layer. The neuron of the output layer, which is activated first, sends an inhibitory signal on all other output neurons. This prevents their activation. In this way, the WTA rule is implemented, when data is classified by defining the only active neuron in the output layer. The activation speed of the output layer neurons depends on the input signals and synaptic weights between the particular output neuron and each neuron from the input layer. For the correct pattern recognition, during the network training, it is necessary to correctly set the synaptic weights for each group of the output neuron on the input neurons. For pattern recognition problems, various SNN architectures are used, which differ in the number of layers and in the way neurons are connected [14,16]. One of the simplest SNN architectures is a two-layer network (Figure 6a), where image information is supplied to the input (first layer), and one of the neurons associated with a certain class of images is activated at the output (second layer) [42–44]. Each of the first layer neurons is connected to each neuron of the second layer through excitatory connections. The connection strength between each pair of neurons is specified through synaptic weights, which can vary among themselves. All neurons of the output layer are interconnected by inhibitory connections. When applying signals to the first layer neurons, they are activated and transmit an excitation effect to the second layer. The neuron of the output layer, which is activated first, sends an inhibitory signal on all other output neurons. This prevents their activation. In this way, the WTA rule is implemented, when data is classified by defining the only active neuron in the output layer. The activation speed of the output layer neurons depends on the input signals and synaptic weights between the particular output neuron and each neuron from the input layer. For the correct pattern recognition, during the network training, it is necessary to correctly set the synaptic weights for each group of the output neuron on the input neurons.

**Figure 6.** *Cont*.

*Electronics* **2019**, *8*, x FOR PEER REVIEW 9 of 19

**Figure 6.** (**a**) Architecture of a two-layer neural network for pattern recognition and (**b**) circuit implementation of neurons in the input and output layers. **Figure 6.** (**a**) Architecture of a two-layer neural network for pattern recognition and (**b**) circuit implementation of neurons in the input and output layers.

Figure 6b presents a coupling diagram of neurons of the input and output layers of a pulsed neural network for image classification. Each element of the input layer corresponds to one pixel of the image. Therefore, to solve the problem of classifying images with a size of 3 × 3 pixels, nine neurons in the input layer are required. The number of output neurons depends on the number of patterns that the network is supposed to recognize. In this study, we will demonstrate the classification of images using three patterns, so the number of neurons in the output layer will be three. The input and output layers of the SNN are connected using synaptic weights, implemented through the resistances *R*w\_i, j, where *i* is the number of the input neuron and *j* is the number of the output neuron. The resistance values *R*w\_i, j will change during training. Memristors [16], where resistance can be adjusted, are often used as resistances in the circuits. In this study, we do not consider a circuit implementation that allows the change of resistances *R*w\_i, j during the training process. Instead, we assume to have control over the elements' resistances. The range of resistance Figure 6b presents a coupling diagram of neurons of the input and output layers of a pulsed neural network for image classification. Each element of the input layer corresponds to one pixel of the image. Therefore, to solve the problem of classifying images with a size of 3 × 3 pixels, nine neurons in the input layer are required. The number of output neurons depends on the number of patterns that the network is supposed to recognize. In this study, we will demonstrate the classification of images using three patterns, so the number of neurons in the output layer will be three. The input and output layers of the SNN are connected using synaptic weights, implemented through the resistances *R*w\_i, j, where *i* is the number of the input neuron and *j* is the number of the output neuron. The resistance values *R*w\_i, j will change during training. Memristors [16], where resistance can be adjusted, are often used as resistances in the circuits. In this study, we do not consider a circuit implementation that allows the change of resistances *R*w\_i, j during the training process. Instead, we assume to have control over the elements' resistances. The range of resistance values *R*w\_i, j varies from 1.5 kΩ to 2.5 kΩ.

values *R*w\_i, j varies from 1.5 kΩ to 2.5 kΩ. A signal from the generator, which encodes information about the color of the pixel, is supplied to the input of each input layer neuron. In this study, we use eight-bit grayscale images, so the information encoded by the generator reflects a gray scale, where the black pixel corresponds to the number 0 and the white pixel corresponds to the number 255. The generator is connected to the input layer neuron using an excitation connection. Then, all nine neurons of the input layer are connected A signal from the generator, which encodes information about the color of the pixel, is supplied to the input of each input layer neuron. In this study, we use eight-bit grayscale images, so the information encoded by the generator reflects a gray scale, where the black pixel corresponds to the number 0 and the white pixel corresponds to the number 255. The generator is connected to the input layer neuron using an excitation connection. Then, all nine neurons of the input layer are connected by excitation connections to the three neurons of the output layer, which forms 9 × 3 = 27 connections.

by excitation connections to the three neurons of the output layer, which forms 9 × 3 = 27 connections. All output neurons are interconnected by inhibitory connections, which are implemented by All output neurons are interconnected by inhibitory connections, which are implemented by connecting to the inhibitory bus using the capacitance *C*inh = 10 nF.

connecting to the inhibitory bus using the capacitance *C*inh = 10 nF. The remaining elements, depicted in Figure 6b, have the following ratings: *R*w = 500 Ω, *R*s = 700 Ω, *R*in\_i = 1 kΩ, *R*in\_j = 200 Ω, *R*out\_i = 10 kΩ, *R*out\_j = 200 Ω, *C*sum = 1 nF, and *C*c = 10 nF. The supply voltage The remaining elements, depicted in Figure 6b, have the following ratings: *R*<sup>w</sup> = 500 Ω, *R*<sup>s</sup> = 700 Ω, *R*in\_i = 1 kΩ, *R*in\_j = 200 Ω, *R*out\_i = 10 kΩ, *R*out\_j = 200 Ω, *C*sum = 1 nF, and *C*<sup>c</sup> = 10 nF. The supply voltage of all neurons is *V*dd = −5.75 V.

### of all neurons is *V*dd = -5.75 V. *2.3. SNN Training*

*2.3. SNN Training*  Before considering the network training algorithm, it is necessary to determine the method of information coding. A large number of information coding methods for SNN has been defined: rate coding, rank coding, time to first spike, latency coding, phase coding, population coding, and others [37]. Typically, two-layer neural networks, used to classify images, apply the rate coding method [43,45,46]. However, in the current study, we use the time to the first spike method [37]. This coding method requires fewer spikes for a single recognition act, and, as a result, less energy is spent on the Before considering the network training algorithm, it is necessary to determine the method of information coding. A large number of information coding methods for SNN has been defined: rate coding, rank coding, time to first spike, latency coding, phase coding, population coding, and others [37]. Typically, two-layer neural networks, used to classify images, apply the rate coding method [43,45,46]. However, in the current study, we use the time to the first spike method [37]. This coding method requires fewer spikes for a single recognition act, and, as a result, less energy is spent on the circuit operation, as most of the energy is spent on generating spikes.

circuit operation, as most of the energy is spent on generating spikes. The information coding is performed as follows. The signals from the generators arrive on the first layer of the neural network with a delay Δ*t* relative to the start time of the circuit *t* = 0, and the The information coding is performed as follows. The signals from the generators arrive on the first layer of the neural network with a delay ∆*t* relative to the start time of the circuit *t* = 0, and the delay ∆*t* determines the brightness of the image pixel. The value ∆*t* = 0 corresponds to brightness

delay Δ*t* determines the brightness of the image pixel. The value Δ*t* = 0 corresponds to brightness 0

0 (black color), and the maximum delay ∆*t*max = 2 µs corresponds to brightness 255 (white color). The signal from the generator is a rectangular pulse with an amplitude of 2 V and a duration of 0.3 µs. The delay time is counted relative to the leading edge of the pulse. signal from the generator is a rectangular pulse with an amplitude of 2 V and a duration of 0.3 μs. The delay time is counted relative to the leading edge of the pulse. The network training process is based on the standard STDP mechanism [14,16,19–21,38]. This

*Electronics* **2019**, *8*, x FOR PEER REVIEW 10 of 19

The network training process is based on the standard STDP mechanism [14,16,19–21,38]. This mechanism is an implementation of the Hebbian learning rule and causes a change in synaptic weight depending on the delay ∆*t*in-out between pre-synaptic and post-synaptic spikes [45]. The traditional rule is an exponential function [45], which depends on ∆*t*in-out. However, various studies use the simplified versions [42,47], which significantly facilitate the calculations, while maintain the main ideas of the SPDT method. In this study, we use the function presented in Figure 7, where the form is given in the SNN training papers [42,45,47]. Since an increase in synaptic weight corresponds to a decrease in resistance *R*w\_i, j, the function is inverted in relation to the axes of an ordinate. Resistance decreases, if the post-synaptic spike (from the neuron in the output layer) arrives with a delay in the range of 0 to 0.5 µs after the pre-synaptic spike (from the neuron in the input layer). In other cases, the resistance *R*w\_i, j increases. mechanism is an implementation of the Hebbian learning rule and causes a change in synaptic weight depending on the delay Δ*t*in-out between pre-synaptic and post-synaptic spikes [45]. The traditional rule is an exponential function [45], which depends on Δ*t*in-out. However, various studies use the simplified versions [42,47], which significantly facilitate the calculations, while maintain the main ideas of the SPDT method. In this study, we use the function presented in Figure 7, where the form is given in the SNN training papers [42,45,47]. Since an increase in synaptic weight corresponds to a decrease in resistance *R*w\_i, j, the function is inverted in relation to the axes of an ordinate. Resistance decreases, if the post-synaptic spike (from the neuron in the output layer) arrives with a delay in the range of 0 to 0.5 μs after the pre-synaptic spike (from the neuron in the input layer). In other cases, the resistance *R*w\_i, j increases.

**Figure 7.** The function of the resistance change the between the input and output neuron Δ*R*w\_i, j **Figure 7.** The function of the resistance change the between the input and output neuron ∆*R*w\_i, j depending on the delay between the pre-synaptic and post-synaptic spikes ∆*t*in-out.

depending on the delay between the pre-synaptic and post-synaptic spikes Δ*t*in-out. Typically, the STDP-based training procedure is used in SNN with unsupervised learning [19– 21,42,44]. In this case, when training input data is supplied, the output neurons are randomly associated with input data patterns. Nevertheless, there are studies on SNN training mechanisms that implement supervised learning [48–50]. In these studies, the input pattern is forcibly assigned to a specific output neuron using back error propagation algorithms. In our study, we tried to implement a simplified approach that allows us to implement supervised learning. During the training, the supply voltage *V*dd was set to be non-zero only at one of the three output neurons (see Table 1). The remaining neurons are forcibly electrically deactivated. They do not emit spikes, and it causes all the associated weights *R*w\_i, j to increase. During network training (Table 1), power at the output neuron No. 1 is present (*V*dd ≠ 0) only when "Pattern 1" images are inputted. When "Pattern 2" and "Pattern Typically, the STDP-based training procedure is used in SNN with unsupervised learning [19–21, 42,44]. In this case, when training input data is supplied, the output neurons are randomly associated with input data patterns. Nevertheless, there are studies on SNN training mechanisms that implement supervised learning [48–50]. In these studies, the input pattern is forcibly assigned to a specific output neuron using back error propagation algorithms. In our study, we tried to implement a simplified approach that allows us to implement supervised learning. During the training, the supply voltage *V*dd was set to be non-zero only at one of the three output neurons (see Table 1). The remaining neurons are forcibly electrically deactivated. They do not emit spikes, and it causes all the associated weights *R*w\_i, j to increase. During network training (Table 1), power at the output neuron No. 1 is present (*V*dd , 0) only when "Pattern 1" images are inputted. When "Pattern 2" and "Pattern 3" images are supplied, the voltage Vdd is zero.

3" images are supplied, the voltage Vdd is zero. **Table 1.** An example of the supply voltage setting *V*dd of the output layer, using the supervised **Table 1.** An example of the supply voltage setting *V*dd of the output layer, using the supervised learning method in SNN training.


Pattern 2 0 -5.75 0

The SNN training algorithm consists of the steps listed in Figure 8. First, arbitrary resistances *R*w\_i, j are set in the range from 1.5 kΩ to 2.5 kΩ. Second, the iterative process of changing the resistances *R*w\_i, j begins. Initially, one of the patterns, that the network should be trained to express, is arbitrarily selected (the number of patterns should be equal to the number of the output layer neurons). In accordance with the pattern and the information coding scheme, the pulse delays are set to be supplied from the generators to the input layer neurons. Then, in accordance with Table 1, the *V*dd values of the output neurons are set. Next, the circuit modelling starts in the SPICE simulator. Based on the simulation results, delays between pre-synaptic and post-synaptic spikes ∆*t*in-out are calculated, and ∆*R*w\_i, j are calculated using the resistance change function (Figure 7). After that, the new values of *R*w\_i, j are set, and, if the values are outside the range of 1.5 kΩ – 2.5 kΩ, *R*w\_i, j is set equal to the nearest border value. The training cycle is repeated, until all the *R*w\_i, j values stop changing. The SNN training algorithm consists of the steps listed in Figure 8. First, arbitrary resistances *R*w\_i, j are set in the range from 1.5 kΩ to 2.5 kΩ. Second, the iterative process of changing the resistances *R*w\_i, j begins. Initially, one of the patterns, that the network should be trained to express, is arbitrarily selected (the number of patterns should be equal to the number of the output layer neurons). In accordance with the pattern and the information coding scheme, the pulse delays are set to be supplied from the generators to the input layer neurons. Then, in accordance with Table 1, the *V*dd values of the output neurons are set. Next, the circuit modelling starts in the SPICE simulator. Based on the simulation results, delays between pre-synaptic and post-synaptic spikes Δ*t*in-out are calculated, and Δ*R*w\_i, j are calculated using the resistance change function (Figure 7). After that, the new values of *R*w\_i, j are set, and, if the values are outside the range of 1.5 kΩ – 2.5 kΩ, *R*w\_i, j is set equal to the nearest border value. The training cycle is repeated, until all the *R*w\_i, j values stop changing. *Electronics* **2019**, *8*, x FOR PEER REVIEW 11 of 19 The SNN training algorithm consists of the steps listed in Figure 8. First, arbitrary resistances *R*w\_i, j are set in the range from 1.5 kΩ to 2.5 kΩ. Second, the iterative process of changing the resistances *R*w\_i, j begins. Initially, one of the patterns, that the network should be trained to express, is arbitrarily selected (the number of patterns should be equal to the number of the output layer neurons). In accordance with the pattern and the information coding scheme, the pulse delays are set to be supplied from the generators to the input layer neurons. Then, in accordance with Table 1, the *V*dd values of the output neurons are set. Next, the circuit modelling starts in the SPICE simulator. Based on the simulation results, delays between pre-synaptic and post-synaptic spikes Δ*t*in-out are

calculated, and Δ*R*w\_i, j are calculated using the resistance change function (Figure 7). After that, the

*Electronics* **2019**, *8*, x FOR PEER REVIEW 11 of 19

**Figure 8.** SNN training algorithm. **Figure 8.** SNN training algorithm. **Figure 8.** SNN training algorithm.

### **3. Results 3. Results**

**3. Results**  Three patterns for training with a dimension of 3 × 3 pixels are presented in Figure 9. The patterns have the same number of black and white pixels. If the number of black pixels is different, Three patterns for training with a dimension of 3 × 3 pixels are presented in Figure 9. The patterns have the same number of black and white pixels. If the number of black pixels is different, then normalization by color intensity can be applied to obtain more accurate results [51]. Three patterns for training with a dimension of 3 × 3 pixels are presented in Figure 9. The patterns have the same number of black and white pixels. If the number of black pixels is different, then normalization by color intensity can be applied to obtain more accurate results [51].

then normalization by color intensity can be applied to obtain more accurate results [51].

**Figure 9.** Set of patterns used for SNN training. **Figure 9.** Set of patterns used for SNN training.

"Pattern 2."

Figure 10 illustrates the resistance values *R*w\_i, 1, *R*w\_i, 2, and *R*w\_i, 3 between all input neurons and three output neurons before and after network training. Resistance values are grouped by nine pieces according to the number of output neurons. Figure 10 illustrates the resistance values *R*w\_i, 1, *R*w\_i, 2, and *R*w\_i, 3 between all input neurons and three output neurons before and after network training. Resistance values are grouped by nine pieces according to the number of output neurons.

*Electronics* **2019**, *8*, x FOR PEER REVIEW 12 of 19

**Figure 10.** Distribution of resistances *R*w\_i, 1, *R*w\_i, 2, and *R*w\_i, 3 before and after training. **Figure 10.** Distribution of resistances *R*w\_i, 1, *R*w\_i, 2, and *R*w\_i, 3 before and after training.

Before training, the resistances *R*w\_i, j were randomly generated in the range from 1.5 kΩ to 2.5 kΩ. Then, the SNN training procedure was performed for ~100 cycles, described in detail in Chapter 2.3, using the input patterns in Figure 9. As a result, the distribution of resistances for each output neuron began to correspond to the pattern assigned to each output neuron. This distribution is an expected result for two-layer networks operating, according to the WTA mechanism [44]. If the training patterns were a set of images distributed by classes (for example, numbers written in different handwriting), then the distribution of weights would be averaging all patterns belonging to Before training, the resistances *R*w\_i, j were randomly generated in the range from 1.5 kΩ to 2.5 kΩ. Then, the SNN training procedure was performed for ~100 cycles, described in detail in Chapter 2.3, using the input patterns in Figure 9. As a result, the distribution of resistances for each output neuron began to correspond to the pattern assigned to each output neuron. This distribution is an expected result for two-layer networks operating, according to the WTA mechanism [44]. If the training patterns were a set of images distributed by classes (for example, numbers written in different handwriting), then the distribution of weights would be averaging all patterns belonging to the same class. Such a training outcome is observed in the studies using the MNIST database [42].

the same class. Such a training outcome is observed in the studies using the MNIST database [42]. Analysis of the trained network reveals the following results. If patterns from the training set were input, then, as expected, the neurons corresponding to the associated pattern were activated at Analysis of the trained network reveals the following results. If patterns from the training set were input, then, as expected, the neurons corresponding to the associated pattern were activated at the SNN output.

the SNN output. Furthermore, images corresponding to distorted patterns in which pixel color intensities were randomly changed were inputted. The results of the SNN operation with the number of the activated output neuron are presented in Figure 11. The first three images corresponding to the distorted patterns from the training set were correctly classified by SNN. The first image corresponds to "Pattern 3," the second image corresponds to "Pattern 1," and the third image corresponds to Furthermore, images corresponding to distorted patterns in which pixel color intensities were randomly changed were inputted. The results of the SNN operation with the number of the activated output neuron are presented in Figure 11. The first three images corresponding to the distorted patterns from the training set were correctly classified by SNN. The first image corresponds to "Pattern 3," the second image corresponds to "Pattern 1," and the third image corresponds to "Pattern 2."

**Figure 11.** Examples of image classification of distorted patterns indicating activated output **Figure 11.** Examples of image classification of distorted patterns indicating activated output neurons.

neurons. When inputting the fourth image, which is a highly distorted template from the training set ("Pattern 3"), none of the output neurons were activated, and it could be interpreted as an undefined When inputting the fourth image, which is a highly distorted template from the training set ("Pattern 3"), none of the output neurons were activated, and it could be interpreted as an undefined result. Inactivity of the output neurons reflects that the voltage on the capacitors *C*sum of the output neurons never reaches the threshold values *V*<sup>c</sup> < *V*c\_th.

result. Inactivity of the output neurons reflects that the voltage on the capacitors *C*sum of the output neurons never reaches the threshold values *V*c < *V*c\_th. This uncertainty could be avoided by reducing the time range for spikes coding Δ*t*max (see Section 2.3). For example, if Δ*t*max = 2 μs is reduced by half to Δ*t*max = 1 μs, then the fourth image in Figure 11 This uncertainty could be avoided by reducing the time range for spikes coding ∆*t*max (see Section 2.3). For example, if ∆*t*max = 2 µs is reduced by half to ∆*t*max = 1 µs, then the fourth image in Figure 11 is correctly identified as "Pattern 3." The pulses have shorter time intervals, and the voltage on the integrating capacitance *C*sum is more likely to increase to the threshold value *V*c\_th.

is correctly identified as "Pattern 3." The pulses have shorter time intervals, and the voltage on the

#### integrating capacitance *C*sum is more likely to increase to the threshold value *V*c\_th. **4. Discussion**

**4. Discussion**  When modelling neurons, on the one hand, we could strive for greater bio-similarity of a neuron, as implemented in some models (FitzHugh-Nagumo, Izhikevich, Hodgkin-Huxley) [35,39], or, on the other hand, we could try to minimize the number of electrical components in the circuit to contribute to its miniaturization in practical implementation. The integrate-and-fire neuron model, which we propose, permits, on the one hand, to obtain a number of properties observed in biological systems (Section 2.1), and, on the other hand, it contains only one switching element and a power source, in contrast to more complex models [28]. In a number of studies of neurons based on VO2, silicon semiconductor devices (field effect transistor, diode) are used as additional circuit elements [22,29]. This imposes certain restrictions on the compatibility of technological processes for manufacturing the silicon and non-silicon parts of the circuit. This drawback is avoided in the presented neuron When modelling neurons, on the one hand, we could strive for greater bio-similarity of a neuron, as implemented in some models (FitzHugh-Nagumo, Izhikevich, Hodgkin-Huxley) [35,39], or, on the other hand, we could try to minimize the number of electrical components in the circuit to contribute to its miniaturization in practical implementation. The integrate-and-fire neuron model, which we propose, permits, on the one hand, to obtain a number of properties observed in biological systems (Section 2.1), and, on the other hand, it contains only one switching element and a power source, in contrast to more complex models [28]. In a number of studies of neurons based on VO2, silicon semiconductor devices (field effect transistor, diode) are used as additional circuit elements [22,29]. This imposes certain restrictions on the compatibility of technological processes for manufacturing the silicon and non-silicon parts of the circuit. This drawback is avoided in the presented neuron model, where all circuit elements (switch, resistors, and capacitors) are manufactured using vanadium oxides of various stoichiometry (VO, VO2, V2O3, and V2O5).

model, where all circuit elements (switch, resistors, and capacitors) are manufactured using vanadium oxides of various stoichiometry (VO, VO2, V2O3, and V2O5). All the results presented in the current study were obtained by modeling the VO2 neuron using numerical methods in the LTspice simulator. However, the I–V characteristic of the VO2 switch corresponding to the experimental data was used in the model (see Section 2.1). Therefore, a All the results presented in the current study were obtained by modeling the VO<sup>2</sup> neuron using numerical methods in the LTspice simulator. However, the I–V characteristic of the VO<sup>2</sup> switch corresponding to the experimental data was used in the model (see Section 2.1). Therefore, a discussion of the physical mechanisms that affect the I–V characteristics is of great importance for predicting the areas of practical application and comparing the characteristics of neuron models.

discussion of the physical mechanisms that affect the I–V characteristics is of great importance for predicting the areas of practical application and comparing the characteristics of neuron models. An increase in the ambient temperature *T*0 leads to a decrease in the threshold voltage *V*th, and at *T*0~ *T*th, the effect of electrical switching will be suppressed, because the VO2 channel will achieve a highly conductive state [41,52,53]. It imposes a limitation on the use of a VO2 neuron, where the operability will be limited at *T*0 > *T*th. The value of *T*th is not high, and reaches *T*th ~ 340 K (67 °C) and creates the question - how to increase *T*th? A good overview of the *T*th modulation methods by doping with various elements is presented in Reference [54]. For example, Cr doping can increase *T*th by 10 °C [55]. An alternative way to increase *T*th is to use other materials with an S-type I–V characteristic. NbO2–based structures, having *T*th ~ 1070 K, demonstrate electrical switching up to temperatures of *T*0 ~ 300 °C [56], and can be used in the presented neuron model. The model is invariant to the use of other materials with the effect of electrical switching, and the main requirement is the presence of an S-type I–V characteristic. VO2-based structures are a good model object and are often used in neural An increase in the ambient temperature *T*<sup>0</sup> leads to a decrease in the threshold voltage *V*th, and at *T*0~ *T*th, the effect of electrical switching will be suppressed, because the VO<sup>2</sup> channel will achieve a highly conductive state [41,52,53]. It imposes a limitation on the use of a VO<sup>2</sup> neuron, where the operability will be limited at *T*<sup>0</sup> > *T*th. The value of *T*th is not high, and reaches *T*th ~ 340 K (67 ◦C) and creates the question - how to increase *T*th? A good overview of the *T*th modulation methods by doping with various elements is presented in Reference [54]. For example, Cr doping can increase *T*th by 10 ◦C [55]. An alternative way to increase *T*th is to use other materials with an S-type I–V characteristic. NbO2–based structures, having *T*th ~ 1070 K, demonstrate electrical switching up to temperatures of *T*<sup>0</sup> ~ 300 ◦C [56], and can be used in the presented neuron model. The model is invariant to the use of other materials with the effect of electrical switching, and the main requirement is the presence of an S-type I–V characteristic. VO2-based structures are a good model object and are often used in neural circuits. Nevertheless, the task of finding switching structures with a wide temperature range is a promising endeavour.

Another problem is the variation of *V*th with the temperature. Therefore, if it is necessary to stabilize the operation of a neuron, it is necessary to come up with additional thermal compensation schemes.

To optimize the circuit presented in Figure 2, we propose to exclude the capacitor *C*sum from the circuit. When switching a VO<sup>2</sup> switch by rectangular pulses, there is an effect of a time delay of switching on the switch. An inverse dependence of the time delay on the pulse amplitude is associated with the thermal heating of the switching channel to the phase transition temperature of the metal insulator. The physics of the electrical switching process is described in detail in Reference [27]. If pulse durations and coding time intervals ∆*t*max are used within the delay times of the switching, the pulse integration effect can be implemented without *C*sum capacity, and will be caused by the heat accumulation in the region of the switching channel. A similar idea to use heat storage in the switch region to accumulate action in a neuron was proposed in Reference [31]. The role of capacitors in the oscillator circuit is discussed in a number of sources [25,57,58], and the oscillations can be obtained without an integrating capacitor, while being only due to the effects of heat storage. The study of the effect of temperature integration of input pulses could be the subject of future research.

The coupling between the switches in the network can be implemented not only by electrical coupling through resistors *R*w\_i, j, but, as described in our previous studies, can be organized through the thermal coupling of the switches [59,60]. The development of spike neural networks with thermal coupling could be the subject of further research.

The reduction of SNN classification uncertainty, when applying a highly distorted template (the right image in Figure 11), by reducing the time interval for coding spikes ∆*t*max, has its own limitations. The current SNN model does not take into account the effect of the turn-on and turn-off delay of the switch described above. For example, the turn-on time of the VO<sup>2</sup> switch, using our input signal amplitudes, does not exceed 10 ns [25], while the turn-off time can be much longer (hundreds of nanoseconds). In the time scale of the current SNN model, operating in microsecond intervals, by taking into account the effect of the turn-on delay, does not affect the results of the SNN operation. However, the turn-on and turn-off times can vary significantly, when using other switches, resistors, and resistances. It should be taken into account when designing SNN.

An important characteristic of the network is the pattern recognition time [61]. The SNN architecture, which we propose, provides recognition time of the coding interval order, corresponding to 2–3 µs. After the recognition is completed, the system requires ~ 7–8 µs to reach the initial state, caused by the recharging of the capacities. Therefore, the current SNN is able to recognize up to 10<sup>5</sup> images per second, and its performance can be increased by reducing the capacitance rating and scaling the VO<sup>2</sup> switches [25]. The implemented method of information coding allows the use of single spikes. It does not only reduce the power consumption compared to the networks using rate coding, but minimizes the time to perform one image recognition operation.

At the end of this section, we present a comparison table of neurons with other proposed neuron devices. The neurons in Table 2 are divided into two groups: neurons based on silicon (CMOS) technology and neurons based on VO<sup>2</sup> switches. The main parameters of the neurons are the size of the active element and the energy consumption. Although the silicon neurons have an advantage in these parameters, in our study, the spikes duration is of the least importance. Another advantage of the VO<sup>2</sup> neuron compared to the CMOS neuron is, apparently, the high noise level of the current channel that leads to the stochastic behavior of the neuron described in References [33,34]. This property allows the network to escape local minima and reach the global minimum of the error surface.


**Table 2.** Comparison of neurons with other proposed neuron devices.

The spike amplitude, power, and energy consumption of the VO<sup>2</sup> neuron depend on the threshold switching voltage *V*th, if only the energy release on the VO<sup>2</sup> switch is taken into account. The main technological parameters affecting the *V*th value are the resistivity in the insulator phase ρoff and the contact geometry [25,32]. In Reference [25], we obtained an equation for approximating *V*th.

$$V\_{\rm th} = \frac{\sqrt{\lambda \cdot \rho\_{off} \cdot (T\_{\rm th} - T\_0)}}{\sqrt{d}} \cdot a^{\beta} \tag{1}$$

where *d* is the thickness of the VO<sup>2</sup> film, *a* is the inter-electrode distance, λ is the heat-transfer coefficient, and β is the exponential coefficient that determines the effective area of the heated zone in the inter-electrode gap (β < 1).

By decreasing the value of *a*, the structures with reduced *V*th can be obtained. Using Equation (1) and the current value when the structure is turned on, estimated as *I*on = *V*th/*R*on (where *R*on = ρon/*d*), we can propose an equation for the maximum power per spike (<sup>λ</sup> <sup>=</sup> 35 W/<sup>m</sup> · K, <sup>ρ</sup>off <sup>=</sup> <sup>4</sup> · <sup>10</sup>−<sup>2</sup> <sup>Ω</sup> · m, <sup>ρ</sup>on <sup>=</sup> <sup>4</sup> · <sup>10</sup>−<sup>4</sup> <sup>Ω</sup> · m, <sup>β</sup> <sup>=</sup> 0.56, *<sup>T</sup>*th <sup>=</sup> 340 K, *<sup>T</sup>*<sup>0</sup> <sup>=</sup> 300 K [25]):

$$P\_{\text{max}} = V\_{\text{th}} \cdot I\_{\text{on}} = \lambda \cdot \frac{\rho\_{\text{off}}}{\rho\_{\text{on}}} \cdot (T\_{\text{th}} - T\_0) \cdot a^{2\beta} \tag{2}$$

By reducing the size of the inter-electrode distance *a* and the ratio (ρoff/ρon), we would significantly reduce the value of *P*max. For example, at *a* = 1 µm, the maximum power *P*max is ~ 26 mW, while at *a* = 100 nm, the maximum power *P*max drops to 2 mW.

*E*spike can be estimated by multiplying the spike duration by the maximum power, *E*spike~ *P*max·∆*t*spike. In Reference [25], we modelled the switch on and switch off durations of switches, which determine the minimum ∆*t*spike values. We demonstrated that the durations decrease with decreasing *a*. Therefore, it is possible to predict a significant decrease in *E*spike with a decrease in the size of switching elements. The estimates are the following: *E*spike ~ 6.4 nJ at *a* = 1 µm (∆*t*spike = 240 ns), and *E*spike ~ 105 pJ at *a* = 100 nm (∆*t*spike = 52 ns).

The last column in Table 2 demonstrates that the majority of the previous studies gives only the model of VO<sup>2</sup> neuron itself. In the current study, we present the simple neural network that is capable of pattern recognition, using the timing method of information coding, which has a clear energy advantage over the firing rates coding method [33].
