See discussions, stats, and author profiles for this publication at: [Link]

net/publication/348361235

RRAM-Based Neuromorphic Computing Systems

Chapter · January 2021

DOI: 10.1007/978-981-15-6912-8_12

CITATIONS READS

3 364

3 authors:

Andhita Dananjaya Roshan Gopalakrishnan

Nanyang Technological University Agency for Science, Technology and Research (A*STAR)
31 PUBLICATIONS 606 CITATIONS 12 PUBLICATIONS 106 CITATIONS

SEE PROFILE SEE PROFILE

Wen Siang Lew

Nanyang Technological University
300 PUBLICATIONS 4,283 CITATIONS

SEE PROFILE

All content following this page was uploaded by Roshan Gopalakrishnan on 25 April 2023.

The user has requested enhancement of the downloaded file.

 RRAM based neuromorphic algorithms
Roshan Gopalakrishnan
Institute for Infocomm Research (I2R), ASTAR, Singapore

1 INTRODUCTION
Human brain performs massively parallel and low power operations. It can outperform present age
microprocessors on many tasks involving pattern recognition and input classification. The underlying
neurons are heavily inter-connected; on average each neuron is connected to 10,000 (or up to 100,000)
other neurons [1, 2, 3]. Despite the complexity of the human brain, research to understand the human brain
is on-going, with the hope of emulating it in terms of its functionalities.
arXiv:1903.02519v1 [[Link]] 12 Jan 2019

In recent years, RRAM devices have emerged as a major memory component in mimicking the
functionality of synapses in the human brain[4]. This is mainly because RRAM can be used both as
a memory element and computation unit. As mentioned in [28], there are two ways of looking at RRAM
based neuromorphic algorithms. From the deep learning perspective, one is to design algorithms for
inference only, i.e to map the pre-trained deep learning models which fulfil certain hardware constraints
onto the RRAM based neuromorphic hardware without any further training. While another way is to
perform on-chip training on the RRAM based neuromorphic hardware, which will require additional
interface circuitry for specific algorithms. Inference alone requires the conversion of existing pre-trained
deep learning algorithms in high precision digital domain to the binary event-based (or spiking) domain so
as to be able to be mapped onto RRAM based neuromorphic hardware. Whereas, on-chip training may be
implemented at the RRAM synapse in the neuromorphic hardware by emulating local spike timing based
algorithms such as spike timing dependent plasticity or its variants. These two methods belong to a new
computational paradigm known as spiking deep neural network (SDNN).
Other than the aforementioned learning algorithms that can be implemented on RRAM based
neuromorphic hardware, low precision convolutional neural networks (CNN) such as the binarized neural
network [5], binaryNet [6], XNOR-NET [7] and DoReFa-NET [8] can be mapped onto a chip containing
RRAM based synaptic crossbar array [9]. In such an approach, the computations performed in the CNN
maybe converted to bitwise operations, such as bitwise convolution, batch normalization and pooling etc.,
as shown in figure 1 of [9]. Contrary to other paradigms, mapping is much easier with such an approach as
it does not involve spiking neurons. Irrespective of the mapping algorithms implemented on the RRAM
based neuromorphic hardware, we should expect a drop in accuracy due to hardware noise, especially the
noise inherent in RRAM synapses (Set or reset variability [10], Random Telegraph Noise (RTN) [11] etc.).
One plausible approach to mitigate the drop in accuracy is to account for the noise itself during training,
which may help to alleviate the accuracy loss to some extent.
1.1 RRAM Synapse
RRAM is a two terminal non-volatile device with a conducting dielectric layer sandwiched between two
electrodes as shown in fig. 1. Electrically induced resistive switching effects shown in metal-insulator-
metal systems are the basis for RRAM [12]. Manipulation of oxygen vacancies in the conductance layer
using positive and negative voltages helps in controlling current flow in RRAM. The state of RRAM
reflects the current passed through it in the history, making it useful for modelling the synaptic weights
of neurological synapses and implementing neural network architectures. In the case of spiking neural
network (SNN), the tunable resistive state of RRAM synapses is analogous to the synaptic plasticity in

1
 RRAM Weight
(Synapse)

Axon Neuron
(Pre-synaptic neuron) (Post-synaptic neuron)

Figure 1. RRAM synapse between an axon and a neuron.

brain. The electrical connection between a presynaptic neuron and a postsynaptic neuron (as shown in fig.
1) changes, strengthening or weakening the synaptic impulses thus making it a case for brain-like pattern
recognition.
1.2 Crossbar array of RRAM Synapses

Word Line (WL)

Top
Electrode
Bottom
Electrode
Input Axons

RRAM Synapse

Output Neurons

Figure 2. Crossbar array of synapses in a neuromorphic core.

A crossbar architecture is shown in Fig. 4. Input axons are the input connections from the output neurons
in the previous convolution layer which was mapped onto another neuromorphic core. Output neurons
are spiking neurons. Spiking neurons recieve input current from many other spiking neurons (input axons
as per the figure) and fire a spike when the integrated current input reaches the neuron threshold. These
building blocks like axons, neurons and synapses together can perform mathematical operations. Matrix
dot vector multiplcations can be performed efficiently with these crossbar structure [13]. Each column in a
crossbar produces the sum of product of input from axons and the weights stored at each RRAM synapses.
1.3 Computation in a crossbar array of RRAM synapses
The convolution operation in a convolutional neural network is implemented with the help of crossbar
array of synapses in a neuromorphic core. Suppose a 3x3 convolution filter kernel after training a
 W1 W1 0
W2 0 W2
W1 W2 W3 W3 W3 0
W4 0 W4
Weight, W = W4 W5 W6 W5 W5 & 0
W6 0 W6
W7 W8 W9 W7 W7 0
W8 0 W8
Here, W2, W4, W6 and W8 are W9 W9 0
negative, rest weights are positive

X1 X1 -X1
X2 X2 -X2
X1 X2 X3 X3 X3 -X3
X4 X4 -X4
Input, X = X4 X5 X6 X5 X5 & -X5
X6 X6 -X6
X7 X8 X9 X7 X7 -X7
X8 X8 -X8
X9 X9 -X9

Figure 3. The weights and input activations used in the crossbar architecture of a neuromorphic core.
Weights and inputs marked in different color is corresponding to its sign as marked with the similar color
in fig. 4.

convolutional neural network is saved as weights, W as shown in fig. 3. Out of these weights W2,
W4, W6 and W8 are negative weights and rest of the weights are positive. Thus for the implementation
of convolution operation this weight kernel W has to be separated into positive weights (marked in blue)
and negative weights (marked in pink). Similarly for the input, X, it is divided into two matrices one is
positive (marked in green) and another is negated input, -X (marked in orange). Once the weight matrix
and input matrix is ready, the weight matrix can be written into the RRAM synapses – positive weights
occupy the top of the crossbar column while negative weights occupy the bottom part, while the inputs
are also respectively fed into axons – positive inputs are given to the top part of the axons and negative
inputs to the bottom part of the axons. The convolution operation in a crossbar array between the inputs
and the weights kernel is explicitly illustrated in fig. 4 as mentioned in [14]. One of the disadvantage
of such implementation is the utilization of double the amount of input axons (2 x filter size) needed as
well as double the number of RRAM synapses. Half of the RRAM synapses has to be written with low
conductance state. This work is also extended to make the architecture extremely parallel by stretching the
separated weight matrices as in the toeplitz matrix [15]. But, the same disadvantages of poor utilization of
axons and synapses as mentioned above will remain. A slightly different approach of implementation is
utilized in IBM’s truenorth chip [16]. They have only ternary weights (-1, 0, +1) and uses two crossbar
synapses in a column as a single synapse to implement ternary weights. This will also end up using double
the amount of physical synapses on neuromorphic chip compared to actual number of synapses in a weight
kernel. Hence, truenorth also has a disadvantage of poor utilization of axons and synapses. Truenorth’s
actual physical core size, meaning number of axons X number of neurons, is 256 X 256, but literally their
core size is only 128 X 256 to implement ternary weights.
 Convolution filter with size 3x3

-ve weights
+ve weights

+ve Input pixels

18 input neurons ( 2xfilter size)

-ve Input pixels

Input image (Eg. MNIST)

Figure 4. Illustration of computation in a crossbar array of synapses in a neuromorphic core.

1.4 Deep Neural Network (DNN) architecture

Deep learning has made much progress in recent years so much so that it has even outperformed humans
in certain tasks, for instance, beating the current GO world champion [17]. DNN or deep CNN has achieved
state-of-the-art accuracy in many image classification or patten recognition tasks such as handwritten
digit recognition [18], and several other datasets such as CIFAR [19], and ImageNet [20]. However, these
networks typically need large amount of labeled training data; ImageNet has over 1 million labeled images
for training.
A conventional CNN is shown in figure 5. It comprises of mainly three blocks: the first block is made
up of convolution layers, the second of fully connected layers and the third is the softmax layer. The
convolution block contains convolution layers that perform the convolution operation on intermediate
output activations. The convolution block also contains other layers that perform batch normalization or
pooling. The fully connected block contains several layers of fully connected neural network. These two
blocks are mainly for feature extraction. The final layer is a fully connected classifier which gives an output
 Input

Feature Extractor
Convolution block

Fully-connected block
Classifier

Softmax

Output

Figure 5. Block diagram of a conventional deep neural network architecture (the convolutional
neural network). It comprises of mainly three blocks: the convolutional block, the fully connected block
and the softmax layer.

based on the softmax function. A typical learning algorithm used in a CNN is backpropagation of errors
with stochastic gradient descent. The network parameters such as weights and biases are adjusted during
training so as to predict the object label of an input image during testing.
1.5 Spiking Neural Network (SNN) architecture

Spiking neural network (SNN) is considered as the third generation of neural networks [21]. SNN is
inspired by biological neural networks while the DNN less so; hence the DNN is also commonly referred
to as artificial neural networks (ANN). DNN does not have any biological roots apart from the hierarchical
structure it possess [22]. SNN is event based: neural activations are communicated through spikes. Spiking
neurons integrate incoming input spikes and emit a spike which is a threshold crossing event, as and when
new information needs to be processed or communicated. These spikes are communicated through synapses
which are associated with a weight quantity.
A neuron in a SNN and its hardware implementation is shown in figure 6. The above figure 6 (a) shows
a single neuron (as part of a SNN) with its input and output mechanisms. The synapse is the connection
between the axon of a pre-synaptic neuron and the dendrite of a post-synaptic neuron. A neuron integrates
the incoming spikes received through its dendrites and may then emit a spike in the event of threshold
crossing through its axon to its post-synaptic neurons. Figure 6 (b) shows a block diagram representation
of the biological model as in figure 6 (a). The synapse is a storage element with input spikes and output
current. Neuron computation is done using an integrator and a comparator. The integrator accumulates
 Input Axon Dendrite Neuron / Cell Axon Output
Spikes body Spikes

Synapse

(a)

Weight
Input update Output
Spikes Integrator Comparator Spikes
and
storage
Reset

(b)

Figure 6. Block diagram of a neuron in a SNN: implemented in blocks as shown, namely, the synapse
and the neuron.

the input currents in terms of potential difference, which emulates the membrane potential in biological
neurons. The comparator then checks if the membrane potential crosses the voltage threshold; a spike is
emitted if crossed and the membrane potential is then reset to its baseline value.

1.6 Conversion of DNN to the spike-based domain: Spiking Deep Neural Network
(SDNN)

In a conventional CPU or GPU, it requires more time and energy to run a SDNN, whereas the power
consumption and computational latency in neuromorphic analog or digital dedicated hardwares [24, 25, 26]
are orders of magnitude less. The substantial computational cost incurred during training and inference in a
deep network for real world practical applications has created [23] :

• a need for specialized hardware acceleration.

• a new computational paradigm.

One emerging approach is to convert the pre-trained DNN into SNN (while retaining its parameters) so
that it can be mapped directly onto a neuromorphic hardware with little performance loss.

The spike based computation in the SNN consumes much less power compared to the high precision
digital computation in the DNN. DNN has better classification accuracy compared to SNN. Hence, mapping
a deep CNN to a SDNN potentially allows us to achieve better accuracy with high energy efficiency.

Spiking Neural Network => Low Energy

Deep Neural Network => Better Accuracy (1)
SDNN => Better Accuracy Low Energy (BALE) Neural Network

While it is difficult to achieve in a mapped SDNN the same level of accuracy as the DNN, research is
ongoing to develop better mapping techniques.
1.6.1 DNN to SNN conversion: SDNN background
DNN to SNN conversion techniques were developed in the ongoing research to map a trained neural
network in conventional frame-based vision system representation to an event-based one [27]. Neurons
in the frame-based CNN were converted to event-based neurons with leak, membrane potential reset and
refractory periods.
One of the first research paper on CNN to SNN conversion is [28]. The conventional CNN is first
converted into a tailored CNN which fulfils the requirements of the SNN. This tailored CNN is then trained.
Finally, this tailored CNN is converted into a spiking CNN, while retaining the trained weights. The
requirements imposed by the SNN on the tailored CNN are

• using RELU [29] as activation functions,

• removing biases from convolution and fully connected layers and
• using spatial linear subsampling in place of maxpooling.

[23] extended the work of [28] by adding weight normalization techniques to improve the conversion
accuracy. The approximation errors in SNNs due to either excessive or too little spikes are avoided by
rescaling of weights. Model based and data based weight normalization techniques were proposed; data
based normalization gives no loss in conversion accuracy for classification of MNIST dataset.
The integrate and fire (IF) neuron model was extensively used in SDNN until [30] demonstrated that a
CNN can also be mapped onto a SDNN made up of leaky integrate and fire (LIF) neurons which are more
biological plausible. This is achieved by using a modified LIF neuron known as the softened LIF neuron
and by training the network with noise so as to improve network robustness against the variability inherent
in spikes.
The hardware constrained neuromorphic algorithm is implemented in [16] on the IBM Truenorth
neuromorphic chip. The hardware constraints are namely, low precision weights and restricted connectivity
among spiking neurons.
Adapting SNN is introduced in [31], which is based on adaptive spiking neurons. Asynchronous pulsed
sigma-delta coding scheme is used by these spiking neurons to efficiently encode information in spike
trains, while homeostatically optimizing the firing rate. This method uses an order of magnitude less spikes
compared to other SDNN approaches; the RELU neurons in an ANN could be directly mapped to adaptive
spiking neurons during conversion.
1.6.2 General steps for conversion
The conversion of a pre-trained DNN to the event-based domain is for inference purposes. The principle
of the conversion technique as mentioned in [28] is that the time averaged firing rate of a spiking neuron
must be correlated with the activation value of the corresponding neuron in the ANN. The generic steps
involved for network conversion is as mentioned below:
• Choose a CNN to train.
• Use ReLU for activation functions in the CNN.
• Fix the bias to zero throughout training using stochastic gradient descent.
• Save all the weights after training.
• Replace neurons in the CNN with integrate and fire neurons without refractory period.
• Map the saved weights to the SNN.
• Convert the input image to poisson spike trains with firing rates proportional to each pixel intensity
value.
1.6.3 Factors affecting conversion accuracy
The issues affecting conversion accuracy as mentioned in [28] are: in the CNN the weights and biases
can be negative. Since input integration is a weighted sum of inputs and the bias, the output can be negative.
If the sigmoid function is used for activation it may also be negative. It is difficult to represent negative
activations in the CNN on a SNN. It is also difficult to represent biases in the SNN. Two layer neural
network is needed to implement spatial maxpooling in the SNN.
CNN to SNN mapping requires the input image to be converted to poisson spike trains with firing rates
proportional to the pixel intensity value. As a result, the loss of accuracy during conversion can happen due
to the factors [23]: Input spikes are not enough to result in threshold crossing, hence no output spike is
emitted when activation values in the CNN are below threshold. If the spiking neuron receives too many
input spikes in a single timestep or if some of its synaptic weights are higher than threshold, then the
spiking neuron should emit more than one spike per timestep, which it cannot, and hence introducing error
in the process. Due to the non-uniformity of the spike trains or the stochastic nature of the spiking input, a
specific feature set could be over- or under- activated by incoming spikes.
An analysis of conversion and its theory is proposed in [32]. One on one mapping of the spiking neuron
and the activation function of the CNN, reveals that during threshold crossing, the membrane potential
reached maybe of any value above threshold. This error would accumulate over time.
1.6.4 Solution to the issues affecting conversion accuracy
The solution to the above-mentioned issues are the following (listed as above): 1. as mentioned in [28],
are to remove biases from convolution layers, use ReLU as activation function and use spatial linear
subsampling instead of maxpooling. 2. as mentioned in [23] use weight normalization. 3. as mentioned in
[32] use reset by subtraction instead of reset to zero for spiking neurons. Instead of removing biases from
convolutional layers, a constant input current can be applied to emulate the biases. Also apply normalization
techniques. 4. as mentioned in [31], to reduce the variability of input spikes, the multi-bit values of the
input maybe fed directly into the first hidden layer and spikes are then output henceforth. 5. as mentioned
in [33], pooling layers can be avoided in deep neural network. Hence, even though there are techniques to
convert pooling layers in SDNN, we can remove these layers from the DNN for simplicity sake.
1.7 Spike based algorithms
In the past decade, spike timing dependent plasticity (STDP) has been a popular unsupervised learning
method due to its biological plausibility [34, 35, 36]. STDP mechanism depends on the timing difference
between the pre-synaptic and post-synaptic spikes to adjust the synaptic weight. In the simple, doublet STDP
[37, 38, 39], when a post-synaptic spike happens after a pre-synaptic spike has arrived (pre-post event),
then the weight of the synapse increases i.e. synaptic potentiation takes place; whereas, if a post-synaptic
spike happens before a pre-synaptic spike (post-pre event), then the weight of the synapse decreases, i.e.
depotentiation takes place. Similar to the doublet STDP, there is another variant of STDP called the triplet
STDP [40, 41, 42, 43], whereby, three spike events are considered (pre-post-pre, post-pre-post etc.).
There are CMOS devices such as the floating gate MOSFET or nano-technology devices such as the
memristors, Resistive Random Access Memories (RRAM), Phase Change Memories (PCM) and Spin-
Transfer Torque Magnetic Random Access Memories (STT-MRAMs) used for the implementation of
artificial synapses. One of the challenge is to integrate these nano-technology devices with CMOS. The
characteristics of high synaptic density on neuromorphic hardware has to be compromised. Understanding
the device physics becomes the key for the implementation of artificial synapses, especially while using
any of the technologies such as floating gate MOSFET, memristors or the more recent spin devices to
implement plasticity rules such as STDP.

1.8 Conclusion

For future work, it maybe worth investigating conversion of DNN using different encoding schemes such
as temporal coding or latency coding instead of just rate coding. This would reduce the number of spikes
required to represent an input and result in more efficient computing. In the long run however, hardware
compatible SNN algorithms should be developed that enable on-chip learning and inference for various
applications. This will eliminate the need for conversion of DNN to SNN; the challenge would be how one
may improve the accuracy of such SNN algorithms.
Here, we have given an overview of the current state-of-art neuromorphic algorithms on RRAM based
neuromorphic devices. While research is on-going to develop SNN for on-chip learning, the current reliable
approach for a real-world application is to do inferencing on-chip based on a converted DNN that is
pre-trained off-chip. During the mapping of a DNN to neuromorphic hardware, hardware constraints such
as number of neurons and synapses, core size, fan in-fan out degrees, routing, spike traffic congestion etc.
have to be taken into consideration. Should any of the above constraints not be met, the DNN architecture
will have to be modified accordingly, so as to fit into a specific neuromorphic hardware. Given its small
form factor and energy efficiency, neuromorphic hardware is well suited for edge computing applications
in the fields of robotics, surveillance, unmanned aerial vehicles etc.

FUNDING

This research is supported by Programmatic grant no. A1687b0033 from the Singapore governments
Research, Innovation and Enterprise 2020 plan (Advanced Manufacturing and Engineering domain).

REFERENCES
[1]Koch C. Biophysics of Computation: Information Processing in Single Neurons. Oxford Univ. Press;
New York: 1999
[2]Geoffrey W. Burr, Robert M. Shelby, Abu Sebastian, Sangbum Kim, Seyoung Kim, Severin Sidler,
Kumar Virwani, Masatoshi Ishii, Pritish Narayanan, Alessandro Fumarola, Lucas L. Sanches, Irem
Boybat, Manuel Le Gallo, Kibong Moon, Jiyoo Woo, Hyunsang Hwang and Yusuf Leblebici
 (2017) Neuromorphic computing using non-volatile memory, Advances in Physics: X, 2:1, 89-124,
DOI:10.1080/23746149.2016.1259585
[3]Laughlin SB, Sejnowski TJ. Communication in Neuronal Networks. Science (New York, NY).
2003;301(5641):1870-1874. DOI:10.1126/science.1089662.
[4]G. Indiveri, E. Linn, and S. Ambrogio, “ ReRAM-based neuromorphic computing,” in Resistive
Switching: From Fundamentals of Nanoionic Redox Processes to Memristive Device Applications.
Weinheim, Germany : Wiley-VCH Verlag GmbH & Co. KGaA, 2016, pp. 715-735.
[5]Itay Hubara and Matthieu Courbariaux and Daniel Soudry and Ran El-Yaniv and Yoshua Bengio,
Binarized Neural Networks, NIPS, 2016.
[6]Matthieu Courbariaux and Yoshua Bengio, BinaryNet: Training Deep Neural Networks with Weights
and Activations Constrained to +1 or -1, CoRR, abs/1602.02830, 2016.
[7]M Rastegari, V Ordonez, J Redmon, A Farhadi, Xnor-net: Imagenet classification using binary
convolutional neural networks, European Conference on Computer Vision, 525-542, 2016.
[8]S Zhou, Y Wu, Z Ni, X Zhou, H Wen, Y Zou, DoReFa-Net: Training Low Bitwidth Convolutional
Neural Networks with Low Bitwidth Gradients, arXiv preprint arXiv:1606.06160, 2016.
[9]Leibin Ni, Zichuan Liu, Hao Yu and Rajiv V. Joshi, An Energy-Efficient Digital ReRAM-Crossbar-
Based CNN With Bitwise Parallelism, IEEE Journal on Exploratory Solid-State Computational
Devices and Circuits, 2017.
[10]Stefano Ambrogio, Simone Balatti, Antonio Cubeta, Alessandro Calderoni, Nirmal Ramaswamy and
Daniele Ielmini, Statistical Fluctuations in HfOx Resistive-Switching Memory: Part I - Set/Reset
Variability, IEEE Transactions on Electron Devices, vol. 61, no. 8, Aug 2014.
[11]Stefano Ambrogio, Simone Balatti, Antonio Cubeta, Alessandro Calderoni, Nirmal Ramaswamy and
Daniele Ielmini, Statistical Fluctuations in HfOx Resistive-Switching Memory: Part II - Random
Telegraph Noise, IEEE Transactions on Electron Devices, vol. 61, no. 8, Aug 2014.
[12]R. Waser and M. Aono, “Nanoionics-based resistive switching memories,” Nature Materials, vol. 6,
no. 11, pp. 833-840, 2007.
[13]M. Hu et al., “Dot-product engine for neuromorphic computing: Programming 1T1M crossbar to
accelerate matrix-vector multiplication,” 53nd ACM/EDAC/IEEE Design Automation Conference
(DAC), pp. 1-6, Austin, TX, 2016.
[14]Chris Yakopcic, Md Zahangir Alom and Tarek M. Taha, “Memristor Crossbar Deep Network
Implementation Based on a Convolutional Neural Network,” International Joint Conference on Neural
Networks (IJCNN), 2016, pp. 963-970.
[15]Chris Yakopcic, Md Zahangir Alom and Tarek M Taha, “Extremely parallel memristor crossbar
architecture for convolutional neural network implementation,” International Joint Conference on
Neural Networks (IJCNN), 2017, pp. 1696-1703.
[16]Steven K. Esser, Paul A. Merolla, John V. Arthur, Andrew S. Cassidy, Rathinakumar Appuswamy,
Alexander Andreopoulos, David J. Berg, Jeffrey L. McKinstry, Timothy Melano, Davis R. Barch,
Carmelo di Nolfo, Pallab Datta, Arnon Amir, Brian Taba, Myron D. Flickner and Dharmendra S.
Modha, “Convolutional networks for fast, energy-efficient neuromorphic computing,” Proceedings of
the National Academy of Sciences, Oct 2016, 113 (41) 11441-11446.
[17][Link]
[18]LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document
recognition. Proceedings of the IEEE, 86, 2278-2324.
[19]Alex Krizhevsky, Learning Multiple Layers of Features from Tiny Images, 2009.
[20]J. Deng and W. Dong and R. Socher and L. Li and Kai Li and Li Fei-Fei, ImageNet: A large-scale
hierarchical image database, IEEE Conference on Computer Vision and Pattern Recognition, 248-255,
10.1109/CVPR.2009.5206848, June, 2009.
[21]Maass, Wolfgang (1997). ”Networks of spiking neurons: The third generation of neural network
models”. Neural Networks. 10 (9): 1659-1671. doi:10.1016/S0893-6080(97)00011-7. ISSN 0893-
6080.
[22]K. Fukushima, “ Neocognitron: A self-organizing neural network model for a mechanism of pattern
recognition unaffected by shift in position, ” Biological Cybernetics, vol. 36, no. 4, pp. 193-202, 1980.
[23]P. U. Diehl, D. Neil, J. Binas, M. Cook, S. Liu and M. Pfeiffer, ”Fast-classifying, high-accuracy
spiking deep networks through weight and threshold balancing,” 2015 International Joint Conference
on Neural Networks (IJCNN), Killarney, 2015, pp. 1-8.
[24]B. V. Benjamin et al., ”Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural
Simulations,” in Proceedings of the IEEE, vol. 102, no. 5, pp. 699-716, May 2014.
[25]Plana, L. A., Clark, D., Davidson, S., Furber, S., Garside, J., Painkras, E., Pepper, J., Temple, S., and
Bainbridge, J. 2011. SpiNNaker: Design and implementation of a GALS multicore system-on-chip.
ACM J. Emerg. Technol. Comput. Syst. 7, 4, Article 17 (December 2011), 18 pages.
[26]F. Akopyan et al., ”TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable
Neurosynaptic Chip,” in IEEE Transactions on Computer-Aided Design of Integrated Circuits and
Systems, vol. 34, no. 10, pp. 1537-1557, Oct. 2015.
[27]J. A. Pérez-Carrasco and B. Zhao and C. Serrano and B. Acha and T. Serrano-Gotarredona and S.
Chen and B. Linares-Barranco, IEEE Transactions on Pattern Analysis and Machine Intelligence,
Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding
and Coincidence Processing–Application to Feedforward ConvNets. Nov 2013, 35(11), 2706-2719.
[28]Cao, Y. and Chen, Y. and Khosla, D. Spiking Deep Convolutional Neural Networks for
Energy-Efficient Object Recognition, International Journal of Computer Vision (2015) 113: 54.
[Link]
[29]Krizhevsky, A., Sutskever, I, and Hinton, G., ImageNet classification with deep convolutional neural
networks. In: Advances in Neural Information Processing Systems 25 (pp. 1106-1114), 2012.
[30]Hunsberger, E. and Eliasmith, C., Spiking Deep Networks with LIF Neurons. arXiv:1510.08829 [cs],
pages 1-9. 2015
[31]Zambrano, D. and Bohte, S. M., Fast and efficient asynchronous neural computation with adapting
spiking neural networks. arXiv preprint arXiv:1609.02053, 2016.
[32]Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, and Michael Pfeiffer, Theory and Tools for
the Conversion of Analog to Spiking Convolutional Neural Networks. Workshop “Computing with
Spikes”, 29th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain.
[33]Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox and Martin Riedmiller, “Striving for
simplicity: The all convolutional net,” accepted as a workshop contribution at ICLR 2015.
[34]G.-Q. Bi and M.-M. Poo, Synaptic modifications in cultured hippocampal neurons: Dependence
on spike timing, synaptic strength, and postsynaptic cell type, J. Neurosci., vol. 18, no. 24, pp.
10464-10472, Dec. 1998.
[35]L. I. Zhang, H. W. Tao, C. E. Holt, W. A. Harris, and M.-M. Poo, “A critical window for cooperation
and competition among developing retinotectal synapses,” Nature Neurosci., vol. 395, pp. 37-44, Sep.
1998.
[36]L. F. Abbott and S. B. Nelson, Synaptic plasticity: Taming the beast, Nature Neurosci., vol. 3, pp.
1178-1183, Nov. 2000.
[37]S. Ramakrishnan, P. Hasler, and C. Gordon. Floating gate synapses with spike-time-dependent
plasticity. IEEE Transactions on Biomedical Circuits and Systems, 5(3):244-252, Jun. 2011.
[38]Roshan Gopalakrishnan and Arindam Basu, Robust Doublet STDP in a Floating-Gate Synapse,
IJCNN, July 6-11, 2014, Beijing, China.
[39]Roshan Gopalakrishnan and Arindam Basu, On the Non-STDP Behavior and Its Remedy in a Floating-
Gate Synapse, in IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 10, pp.
2596-2601, Oct. 2015.
[40]J.-P. Pfister and W. Gerstner, Triplets of spikes in a model of spike timing-dependent plasticity, J.
Neurosci., vol. 26, no. 38, pp. 9673-9682, Sep. 2006.
[41]M. R. Azghadi, S. Al-Sarawi, D. Abbott, and N. Iannella. A neuromorphic vlsi design for spike timing
and rate based synaptic plasticity. Neural Networks, 45:70-82, 2013.
[42]Roshan Gopalakrishnan and Arindam Basu, Triplet Spike Time-Dependent Plasticity in a Floating-
Gate Synapse, IEEE TNNLS, vol. 28, no. 4, April 2017.
[43]Roshan Gopalakrishnan and Arindam Basu, Triplet spike time dependent plasticity in a floating-gate
synapse, IEEE International Symposium on Circuits and Systems (ISCAS), pp. 710-713, 2015.

View publication stats