FeNN: A RISC-V vector processor for Spiking

Neural Network acceleration

Zainab Aizaz∗ § , James C. Knight∗ § , and Thomas Nowotny∗
∗ School of Engineering and Informatics, University of Sussex,Brighton, BN1 9QJ, UK §
Equal contribution
Email: {[Link], [Link], [Link]}@[Link]

Abstract—Spiking Neural Networks (SNNs) have the potential much lower arithmetic intensity. Bautembach et al. [3], Knight
to drastically reduce the energy requirements of AI systems. and Nowotny [4] previously showed that SNNs can be simu-
However, mainstream accelerators like GPUs and TPUs are lated effectively on GPUs. However, the low arithmetic inten-
arXiv:2506.11760v1 [[Link]] 13 Jun 2025

designed for the high arithmetic intensity of standard ANNs

so are not well-suited to SNN simulation. FPGAs are well- sity of SNNs means that SNN simulations can easily saturate
suited to applications with low arithmetic intensity as they a GPU’s memory bandwidth but leave its compute resources
have high off-chip memory bandwidth and large amounts of under-utilised. Furthermore, while low-precision weights have
on-chip memory. Here, we present a novel RISC-V-based soft been widely adopted in GPU designs to reduce memory band-
vector processor (FeNN), tailored to simulating SNNs on FPGAs. width requirements, they are typically handled by the systolic
Unlike most dedicated neuromorphic hardware, FeNN is fully
programmable and designed to be integrated with applications MAC arrays which do not support the operations required
running on standard computers from the edge to the cloud. We for simulating SNNs at low precision [5, 6]. Finally, because
demonstrate that, by using stochastic rounding and saturation, simulating SNNs is an iterative process and GPUs have limited
FeNN can achieve high numerical precision with low hardware capacity for global synchronisation, several kernels typically
utilisation and that a single FeNN core can simulate an SNN are launched every simulation timestep causing a significant
classifier faster than both an embedded GPU and the Loihi
neuromorphic system. overhead (order of 10 µs).
Index Terms—SNN, FPGA, RISC-V Digital neuromorphic systems [7, 8] are tailored to the low
arithmetic intensity of SNN simulations, typically only having
I. I NTRODUCTION modest computational resources but large amounts of fast local
memory. These systems are very efficient but typically not
Artificial Neural Networks (ANNs) have demonstrated flexible enough for developing new algorithms. Furthermore,
super-human performance in areas ranging from image classi- neuromorphic systems are not available off the shelf and unlike
fication to playing go [1]. However, training and using ANNs GPUs cannot easily be integrated with applications running on
is computationally very expensive and only possible due to standard computers as they are typically standalone systems.
modern hardware and software. The primary computational Field-Programmable Gate Arrays (FPGAs) represent an inter-
primitive ANNs require is matrix multiplication, which has esting middle ground as they can be integrated seamlessly with
a very high arithmetic intensity. Numerous accelerators in- standard computers and not only offer high external memory
cluding GPUs and TPUs have been designed to accelerate bandwidth similar to GPUs but also large amounts of on-chip
matrix multiplication, typically by streaming data from high memory that can be accessed in a single cycle (around 40 MiB
bandwidth external memory into systolic arrays of multiply- on a high-end Alveo U55C). Although FPGAs also have DSP
accumulate units. However, high arithmetic intensity leads to cores to enable single-cycle integer multiplication, they run
high energy requirements, representing a growing problem at relatively low clock speeds (typically a few 100 MHz)
for the AI industry. At the same time, the biological neural so cannot compete with GPUs for arithmetically-intensive
network of the human brain can still outperform ANNs in tasks. This has meant that they have thus far only been
many tasks while only consuming 20 W. used for inference with standard ANNs [9]. However, FPGAs
While ANNs are inspired by the brain, they differ in several are well-suited to SNN simulations because of their lower
key areas, one of which is that biological neurons communi- arithmetic intensity. Over the years, there has been a plethora
cate with sparse binary events called spikes. This has been of fixed-function FPGA SNN accelerators which demonstrate
incorporated in artificial Spiking Neural Networks (SNNs) this [10, 11]. However, such systems have a high entry barrier
which have already been successful in spatio-temporal pro- as they require re-synthesis using FPGA tools and potentially
cessing applications such as audio denoising and autonomous even writing HDL if the emulated model changes. Naylor
driving [2]. As spikes carry no ‘value’ to multiply, simulating et al. [12] demonstrated that a fully programmable vector
SNNs does not require matrix multiplication and thus has a soft-core processor running on an Altera Stratix IV FPGA
could simulate SNNs faster than the then state-of-the-art, and
This work was funded by EPSRC grants EP/V052241/1 and EP/S030964/1;
and the EU’s Horizon 2020 research and innovation programme under Grant more recent programmable systems [13, 14] have shown that
Agreement 945539. Hardware was provided by the Xilinx University Program. this approach continues to be competitive on newer FPGA
 TABLE I
RISCV LOAD STORE BRAM H ARDWARE UTILISATION . B RACKETED VALUES ARE PER LANE /PE.

RISCV RISCV RISCV RISCV Design Lanes Frequency Utilisation

FETCH DECODE EXECUTE WRITEBACK [MHz] LUT FF
SNAVA [14] 100 125 148 774 (1488) 97 824 (978)
Issue Commit Result GaBAN [13] 8 286 28 759 (3595) 9605 (1201)
FeNN 32 166 32 915 (1028) 32 547 (1017)
BRAM FENN FENN FENN
(IM) DECODE EXECUTE WRITEBACK
to make the CV32E40X more suitable for FPGA synthesis,
FENN LOAD STORE URAM
we have simplified its fetch and load-store stages to work
with synchronous Block RAMS – removing the need for any
Fig. 1. Block diagram of FeNN, tightly coupled to CV32E40X RISC-V core. instruction prefetch or alignment logic.
Black arrows indicate pipeline stages and key signals. Blue arrows highlight
the ‘Issue’, ‘Commit’ and ‘Result’ signals defined by the CV32E40X’s XIF
extension interface.
B. 3-stage pipelined vector co-processor
While it is common for inference accelerators to use 8 bit
or even 4 bit weights, we want to offer more flexibility and,
architectures like AMD’s UltraScale+. in future, also accelerate SNN training workloads. Hence, our
Here, we present our FPGA-Enhanced Neural Net- accelerator operates on 16 bit types throughout. Specifically,
work (FeNN) accelerator: a vector soft-processor, tailored to FeNN operates on vectors of 32 16 bit values using a SIMD
the needs of SNN simulation and the hardware resources avail- architecture and a standard register-register instruction set,
able on modern FPGAs. Uniquely, it is designed to operate as backed by a three-port vector register file with 32 512 bit
an in-memory accelerator, capable of scaling from embedded registers, implemented using distributed memory. The majority
systems based on Zynq MPSoCs to workstations or cloud of on-chip memory on UltraScale+ FPGAs is provided through
nodes equipped with large accelerators such as the Xilinx ‘wide’ (72 bit) ‘UltraRAM’ memory which we use to imple-
Alveo U55C. We show that, by using stochastic rounding and ment local memories, large enough to store all model state
saturation, complex spiking neuron models can be simulated in our current prototype. These memories are created using 8
accurately using 16 bit fixed-point arithmetic and demonstrate parallel sets of chained UltraRAM blocks, enabling FeNN to
a recurrent SNN that classifies spoken digits [15] on FeNN. load and store entire vectors in a single cycle.
This classifier runs significantly faster than a similar model In order to provide scope for further expansion, FeNN
running on Loihi and twice as fast as the same model running is implemented within a full 30 bit RISC-V instruction set
on an embedded GPU while using half the energy. quadrant (binary prefix 10). Within this quadrant, our initial
FeNN design implements a minimal instruction set supporting:
II. M ETHODS
• Addition, subtraction and multiplication of vectors.
A. Scalar core • Loading and storing of vectors in UltraRAM.
Soft vector processors can either be tightly coupled, where • Moving data between scalar and vector registers.
individual instructions are offloaded from the scalar processor • Generating random numbers.
to the vector processor or decoupled, where ‘kernels’ of • Computing masks by comparing vectors.
vector instructions are assembled separately and copied to • Selecting between elements of two vectors using masks.
the accelerator. While tightly coupled architectures allow fine- Multiplication is implemented using DSP blocks and, as these
grained communication between scalar and vector cores, their can operate at several times our operating frequency, this
performance can be restricted by the throughput of the scalar leaves time in the execute cycle to apply a 16 bit barrel shift
processor and each application must be compiled for the novel to the result in order to implement fixed-point multiplication.
architecture [12]. However, decoupled vector processors have Because FeNN has 32 vector lanes, the masks used for non-
to implement the entire control flow for the desired algorithms uniform control flow conveniently fit in a standard 32-bit
which would introduce significant complexity. RISC-V register. Thus, in our tightly-coupled architecture,
We have developed a best-of-both-worlds architecture where masks can be processed using vector and scalar instructions,
our FeNN vector processor works as a decoupled, in-memory removing hardware requirements for separate mask registers.
accelerator for a standard host CPU (a quad-core ARM Cortex- 1) Pipeline: Figure 1 illustrates how FeNN is pipelined
A53 on the Kria KV260) running Linux but is tightly coupled alongside the CV32E40X with synchronisation occurring
to its own scalar core. The scalar core is a CV32E40X through the XIF extension interface. All instructions can be
developed by the Open Hardware Group and is a small and executed within a single execute clock cycle and we include
efficient, 32-bit, in-order RISC-V core with a 4-stage pipeline. additional logic between the writeback and execute stages to
Data and control hazards notwithstanding, this core can issue enable bypassing of RAW hazards between vector registers.
1 instruction per cycle and provides a convenient extension UltraRAM resources are synchronous with a 1-cycle latency
interface upon which to implement our co-processor. In order so addresses are calculated in the execute cycle and values
 A B
1.0
0.5
0.15
0.0
1.0 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00
P(x=k)

0.10 0.5
0.0
0.05 1.0 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00
0.5
0.00 0.0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00
k Bit error
Fig. 2. (A) Histogram of 3200 Poisson variates sampled from distribution with λ = 5, generated on FeNN. Horizontal lines indicate Probability Mass
Function of distribution. (B) Histograms showing the bit error distribution of 21 760 random S0.15 multiplication operations with (top) round-to-zero (middle)
round-to-nearest and (bottom) stochastic rounding (after Hopkins et al. [6]).

read from memory are written back to the register file in the and instruction BRAMs connected to the scalar RISC-V core
writeback cycle. are exposed to the ARM core via standard AXI BRAM
2) Random number generation: Efficient pseudorandom controllers. Similarly, control and status registers are exposed
number generation is important in many SNN models as via an AXI GPIO controller. These resources are then memory
well as for stochastic rounding [6] so an efficient Random mapped by user-space applications running on the ARM
Number Generator (RNG) is a key requirement for FeNN. core. Applications use a C++-based Just-in-Time assembler
Because FeNN operates on 16 bit numbers, we require an to generate FeNN instructions and copy them, alongside any
RNG with a small state size that can generate 16 bit of initial state, to the memory-mapped BRAMs. FeNN is then
randomness per cycle in each vector lane. Inspired by its use reset and simulates multiple SNN timesteps while the host
in the Propeller 2 microcontroller, we chose a Xoroshiro32++ polls, waiting for FeNN to finish.
generator [16] which is relatively high quality and extremely
D. Adaptive Leaky Integrate-and-Fire neuron model
hardware friendly – requiring only a handful of addition,
XOR, rotation and shift operations. However, if it were to Standard Recurrent SNNs (RSNNs) using Leaky Integrate-
be implemented as a standard RISC-V instruction – which and-Fire (LIF) neurons have inferior short-term memory ca-
reads at most two operands and writes back another – the pacities compared to recurrent ANN architectures such as
instruction to generate a random number would need to read LSTMs. Adaptive LIF (ALIF) neurons – which augment a
32 bit and write 48 bit of state per cycle requiring extra standard LIF neuron with a slowly adapting threshold – are
register file ports. Furthermore, incorporating the RNG into one solution to this problem [18]. The dynamics of an ALIF
a stochastic multiplication instruction would require reading neuron’s membrane voltage V can be calculated using an
two more operands and writing back yet another. Instead, we Exponential Euler scheme,
add two special two-port registers dedicated to holding the V [t + 1] =αV [t] + I syn [t + 1] − S[t]Vth (1)
RNG state. These can be populated from vector memory using
−1
special variants of the vector load instruction. where α = e controls the neuron’s leak (τm = 20), I syn is
τm

3) Stochastic rounding: A standard fixed point multipli- the input current and Vth = 0.6 is the baseline firing threshold
cation is performed by calculating (A * B) >> N where A which is subtracted from V when a spike occurs. Spikes are
and B are two fixed-point encoded operands and N is the triggered when the membrane voltage crosses the adaptive
number of fractional bits in the representation. The right shift threshold:
by N truncates the result leading to round-to-zero behaviour.
However, Mikaitis [17] demonstrated that rounding can be S[t] =H (V [t] − (Vth + βA[t])) (2)
implemented simply by turning the multiplication into a MAC where β = 0.0174 is the adaptation strength and H the
( (( A * B) + R) >> N) which is supported in hardware by Heaviside function. The adaptation A evolves as
the DSP blocks. Round-to-nearest can be implemented simply
by setting R to 0.5 (in fixed-point with N fractional bits) and A[t + 1] =ρA[t] + S[t] (3)
stochastic rounding by adding N bits of randomness (the mean −1

of which will be 0.5) from the RNG. where ρ = e τa controls the rate of adaptation (τa = 2000).

C. System-on-Chip design
In our current prototype we have instantiated a single FeNN
core on an AMD Kria KV260 development board. The data
 $ &
 
 
9

$
   
           
% '
   
 
9

$
 
 
  

9

$
   
           
7LPH>PV@ 9 $ 7LPH>PV@

Fig. 3. ALIF neuron simulations (A) ALIF stimulated with two periods of Poisson spiking input, separated by a period of low background rate, simulated
using 64 bit floating point (B) Detail from the shaded area in A) comparing 16 bit fixed point (solid lines) against 64 bit floating point (dashed lines) using
standard round-to-zero (top) and stochastic rounding (bottom). (C) ALIF stimulated by Poisson spiking input with increasing rates, simulated using 64 bit
floating point (D) Detail from C) comparing 16 bit fixed point (solid lines) against 64 bit floating point (dashed lines) using stochastic rounding (top) and
stochastic rounding with saturation (bottom). Simulations with stochastic rounding were run 32 times and the standard deviations are shown as shaded areas.

III. R ESULTS value < 1 so no overflow will occur) using FeNN. Figure 2B
All code used to generate the results in this section and shows that the bit error distributions obtained by comparing
a bitstream for programming a Kria KV260 with FeNN are the results to 64 bit floating point multiplication match those
available at [Link] ise. obtained by Hopkins et al..
Using ALIF neurons has been shown to significantly im-
A. Synthesis prove the performance of SNN models [18]. However, ac-
We synthesised our design using Xilinx Vivado 2023.2. curately simulating their dynamics can prove challenging on
Table I compares hardware utilisation and clock speed with digital hardware as fixed-point numbers implemented with
recent programmable FPGA-based SNN accelerators [13, 14]. standard round-to-zero integer operations prematurely under-
Due to our focus on 16 bit fixed point, a FeNN core requires flow to zero. Figure 3 shows simulations of ALIF neurons
significantly fewer LUTs per-lane than other designs. On the driven by challenging input spike trains. Figure 3B highlights
other hand, FeNN’s deeper pipeline means that it requires a period where the 16 bit fixed point simulation with round-
more FlipFlops than the other designs, but these are not to-zero diverges from the 64 bit floating point reference sim-
the scarcest resource and its deeper pipeline means FeNN ulation while the version with stochastic rounding follows
can process neurons in fewer clock cycles (approximately 20 closely. Overall, stochastic rounding significantly reduces the
compared to 85 for GaBAN). Normalised Root-Mean Square Error (NRMSE) compared to
the floating point reference (NRMSE of V reduces from 0.13
B. Random number generation to 0.045 and A from 0.019 to 0.0066).
In order to test the random number generator described in When training SNN models or performing inference on live
Section II-B2, we implemented a Poisson generator on FeNN data, stochastic rounding alone is not sufficient for numerically
using the direct method first described by Knuth [19] and safe simulations. As figure 3D shows, if a neuron’s input
sampled 3200 variates with λ = 5. As Poisson processes increases temporarily above the calibrated range of the fixed-
are often used to encode stimuli in SNN models [20], this point formats, state variables overflow and wrap. This results
is a representative test of how random number generation in catastrophic failure (NRMSE of V of 0.29 and A of 0.39)
is likely to be used on FeNN. Figure 2A shows that the that can, however, be mitigated by saturating additions and
sampled variates follow the desired distribution and our timing subtractions (NRMSE of V of 0.098 and A of 0.019).
measurements show that, on average, the Poisson generator
produces 32 variates every 81 clock cycles (corresponding to D. Classification
around 64 × 106 s−1 ). To demonstrate the performance of our FeNN prototype
on a real machine learning task, we trained an ALIF-based
C. Rounding and saturation Recurrent SNN (RSNN) classifier with a single 256 neu-
In order to test the correctness of the rounding mechanisms ron hidden layer on the Spiking Heidelberg Digits (SHD)
described in Section II-B3, we repeated the experiment de- dataset [15], quantized the trained weights and deployed it onto
scribed by Hopkins et al. [6] and performed 21 760 multi- FeNN. Running on FeNN, this classifier obtains an accuracy
plication operations with round-to-zero, round-to-nearest and of (79.50±0.44) % on the test set – an insignificant difference
stochastic rounding on random 16 bit operands in S0.15 fixed- in performance compared to the 32 bit floating point GeNN
point format (S0.15 can only represent numbers with absolute implementation which obtains 79.6 %. Figure 4A shows the
 A B C
2.0

Number of instructions [billion]

Hidden neuron Input neuron

500 40

Time per timestep [µs]

1.5
30
0
20 1.0
0 250 500 750 1000
200
10 0.5
100
0 0 0.0
0 250 500 750 1000 Jetson Jetson Loihi FeNN Input Hidden Hidden Output
Time [ms] CPU GPU spikes spikes neurons neurons
ALU Vec. ALU Other
Mem. Vec. Mem.

Fig. 4. SHD inference running on FeNN (A) Raster plot of input and hidden neuron activity during one example. (B) Comparison of average time (over
the whole test set) taken to simulate a single SNN timestep on FeNN with Jetson Orin Nano using GeNN (batch size=1) [21] and similar RSNN running on
Loihi [22]. GPU times were measured using CUDA timing events and CPU and FeNN times using std :: chrono :: high resolution clock (C) Distribution of
different types of instructions in different stages of the simulation.

TABLE II significant amount of time is spent streaming SHD digits to

SHD CLASSIFICATION ENERGY. ‘T OTAL INFERENCE ENERGY ’ IS ALL the accelerators which is not currently well-optimised on either
ENERGY SPENT DURING INFERENCE . I DLE POWER AND TIME SPENT
COPYING DATA ARE REMOVED TO CALCULATE ‘S IMULATION ENERGY ’ system. As neither of these factors are of direct interest to this
AND THIS IS DIVIDED BY THE TOTAL NUMBER OF SYNAPTIC EVENTS TO work, we also show simulation energy with these removed and
GET ‘E NERGY PER SYNAPTIC EVENT ’. energy per-synaptic event in Table II. Again, these illustrate
that FeNN takes approximately half the energy of the Jetson
Device Total inference Simulation Energy per synaptic
energy [J] energy [J] event [nJ] for the same task.
FeNN 347 44 8 IV. C ONCLUSIONS AND F UTURE W ORK
Jetson GPU 645 97 18
Jetson CPU 956 113 20 Here we have presented a first prototype of our FeNN
accelerator which is already capable of efficiently simulating a
moderately-sized RSNN classifier, faster than both embedded
input spikes from an example digit and the hidden layer GPU and neuromorphic systems. Furthermore, FeNN performs
activity within the network. In Figure 4B, we compare the these simulations using around half of the energy of an
average time (over the whole SHD test set) taken to simulate embedded GPU.
a single SNN timestep using FeNN and a Jetson Orin Nano Figure 4C shows the distribution of different instruction
(as this has a similar form factor and power usage to a Kria types executed by FeNN while evaluating the SHD test.
KV 260) using GeNN [21]. Additionally, we include the time The ratio of ‘Vector Memory’ and ‘Vector ALU’ instruc-
reported by Rao et al. [22] for a similar ALIF-based RSNN tions clearly illustrates the low arithmetic intensity of SNN
classifier on the Loihi [7] neuromorphic system. simulation, especially in the spike-processing code where
this is 3:1. When processing spikes, a significant number of
E. Energy usage scalar ALU instructions are spent iterating over spikes and
Neither the Jetson Orin Nano or Kria KV260 provide a calculating the addresses to load weights from. To reduce this
sensor for measuring the power used by the GPU or FPGA overhead and also allow FeNN to simulate larger models with
accelerator. Therefore, we recorded the power usage during weights stored in external memory, we plan to develop Direct
SHD inference using a consumer power measurement device Memory Access (DMA) hardware which which will calculate
at the mains socket, captured the output with a webcam addresses from spikes and asynchronously copy weights from
and performed optical character recognition using the ssocr external memory to UltraRAM. This DMA hardware will
Ubuntu package. We then integrated the power time series also enable the transfer of data between FeNN cores using
using the trapezoidal rule to calculate the energy spent during an AXI crossbar. As well as these system-on-chip features,
the whole simulation (we dis-regarded initialisation as it was we also plan on adding lane-local memories – for efficient
implemented very differently in the two simulations). As Ta- processing of sparse connectivity [12] – and instructions to
ble II shows, FeNN takes approximately half the energy of the perform reductions across vector lanes to each core. Finally,
Jetson GPU and a third of the Jetson CPU to evaluate the SHD the SNNs simulated here have all been hand-coded in RISC-V
test set. However, both systems were running desktop Linux assembly language. However, we are developing a backend for
operating systems so the majority of this energy goes to driving our GeNN library [21] to convert high-level model descriptions
the display, performing background tasks etc. Furthermore, a into FeNN instructions.
 With these developments in place, we intend to scale up our [6] Michael Hopkins, Mantas Mikaitis, Dave R Lester, and
prototype to a 32 core version instantiated on an Alveo U55C Steve Furber. Stochastic rounding and reduced-precision
accelerator which will be capable of batched Eventprop [23] fixed-point arithmetic for solving neural ordinary dif-
training in addition to inference. Because Eventprop training ferential equations. Philosophical Transactions of the
has a similar computational complexity as inference, we would Royal Society A: Mathematical, Physical and Engineer-
expect this system to be able to train models at a similar ing Sciences, 378(2166):20190052, March 2020. ISSN
speed and energy cost to the inference task we perform here. 1364-503X. doi: 10.1098/rsta.2019.0052. URL https:
For inference, the 8 nJ per-SOP achieved by FeNN is not //[Link]/doi/10.1098/rsta.2019.0052.
competitive with the latest neuromorphic systems (at least [7] Loihi : a Neuromorphic Manycore Processor with On-
partly due to the 10× overhead of FPGA vs ASIC logic [24]). Chip Learning. IEEE Micro, 30(1):82–99, 2018. doi:
However, due to its greater flexibility, including the ability to 10.1109/MM.2018.112130359.
support inference and Eventprop training in future, FeNN is [8] Hector A. Gonzalez, Jiaxin Huang, Florian Kel-
a valuable intermediate step towards the wider adoption of ber, Khaleelulla Khan Nazeer, Tim Langer, Chen
neuromorphic technologies. Liu, Matthias Lohrmann, Amirhossein Rostami, Mark
Schöne, Bernhard Vogginger, Timo C. Wunderlich, Yexin
R EFERENCES
Yan, Mahmoud Akl, and Christian Mayr. SpiNNaker2: A
[1] David Silver, Aja Huang, Chris J. Maddison, Arthur Large-Scale Neuromorphic System for Event-Based and
Guez, Laurent Sifre, George van den Driessche, Julian Asynchronous Machine Learning, January 2024. URL
Schrittwieser, Ioannis Antonoglou, Veda Panneershel- [Link] arXiv:2401.04491 [cs].
vam, Marc Lanctot, Sander Dieleman, Dominik Grewe, [9] Anouar Nechi, Lukas Groth, Saleh Mulhem, Farhad
John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Merchant, Rainer Buchty, and Mladen Berekovic. FPGA-
Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore based Deep Learning Inference Accelerators: Where Are
Graepel, and Demis Hassabis. Mastering the game of Go We Standing? ACM Transactions on Reconfigurable
with deep neural networks and tree search. Nature, 529 Technology and Systems, 16(4):1–32, December 2023.
(7587):484–489, 2016. ISSN 0028-0836. doi: 10.1038/ ISSN 1936-7406, 1936-7414. doi: 10.1145/3613963.
nature16961. URL [Link] URL [Link]
1038/nature16961. [10] Simon W. Moore, Paul J. Fox, Steven J.T. Marsh,
[2] Sumit Bam Shrestha, Jonathan Timcheck, Paxon Frady, a. Theodore Markettos, and Alan Mujumdar. Blue-
Leobardo Campos-Macias, and Mike Davies. Effi- hive - A Field-Programable Custom Computing Ma-
cient Video and Audio Processing with Loihi 2. In chine for Extreme-Scale Real-Time Neural Network
ICASSP 2024 - 2024 IEEE International Conference Simulation. In 2012 IEEE 20th International
on Acoustics, Speech and Signal Processing (ICASSP), Symposium on Field-Programmable Custom Comput-
pages 13481–13485, Seoul, Korea, Republic of, April ing Machines, pages 133–140, Toronto, April 2012.
2024. IEEE. ISBN 9798350344851. doi: 10.1109/ IEEE. ISBN 978-1-4673-1605-7. doi: 10.1109/FCCM.
ICASSP48485.2024.10448003. URL [Link] 2012.32. URL [Link]
[Link]/document/10448003/. [Link]?arnumber=6239804.
[3] Dennis Bautembach, Iason Oikonomidis, and Antonis [11] Kevin Kauth, Tim Stadtmann, Vida Sobhani, and To-
Argyros. Even Faster SNN Simulation with Lazy+Event- bias Gemmeke. neuroAIx-Framework: design of future
driven Plasticity and Shared Atomics. In 2021 neuroscience simulation systems exhibiting execution of
IEEE High Performance Extreme Computing Conference the cortical microcircuit model 20× faster than biological
(HPEC), pages 1–8, September 2021. doi: 10.1109/ real-time. Frontiers in Computational Neuroscience, 17:
HPEC49654.2021.9622805. URL [Link] 1144143, April 2023. ISSN 1662-5188. doi: 10.3389/
[Link]/document/9622805/?arnumber=9622805. ISSN: fncom.2023.1144143. URL [Link]
2643-1971. articles/10.3389/fncom.2023.1144143/full.
[4] James C Knight and Thomas Nowotny. Efficient GPU [12] Matthew Naylor, Paul J. Fox, A. Theodore Markettos,
training of LSNNs using eProp. In Neuro-Inspired and Simon W. Moore. Managing the FPGA memory
Computational Elements Conference, pages 8–10, New wall: Custom computing or vector processing? In 2013
York, NY, USA, March 2022. ACM. ISBN 978-1- 23rd International Conference on Field programmable
4503-9559-5. doi: 10.1145/3517343.3517346. URL Logic and Applications, pages 1–6, Porto, September
[Link] 2013. IEEE. ISBN 978-1-4799-0004-6. doi: 10.
[5] T. Fernandez-Hart, James C. Knight, and T. Kalganova. 1109/FPL.2013.6645538. URL [Link]
Posit and floating-point based Izhikevich neuron: A Com- document/6645538/.
parison of arithmetic. Neurocomputing, page 127903, [13] Jiajie Chen, Le Yang, and Youhui Zhang. GaBAN:
May 2024. ISSN 09252312. doi: 10.1016/[Link].2024. a generic and flexibly programmable vector neuro-
127903. URL [Link] processor on FPGA. In Proceedings of the 59th
S092523122400674X. ACM/IEEE Design Automation Conference, pages 931–
 936, San Francisco California, July 2022. ACM. ISBN 2018.
978-1-4503-9142-9. URL [Link] [19] Donald E Knuth. The Art of Computer Programming:
3489517.3530561. Seminumerical Algorithms, Volume 2. Addison-Wesley
[14] Athul Sripad, Giovanny Sanchez, Mireya Zapata, Professional, 2014.
Vito Pirrone, Taho Dorta, Salvatore Cambria, Albert [20] Michael Pfeiffer and Thomas Pfeil. Deep Learning With
Marti, Karthikeyan Krishnamourthy, and Jordi Madrenas. Spiking Neurons: Opportunities and Challenges. Fron-
SNAVA—A real-time multi-FPGA multi-model spiking tiers in Neuroscience, 12:774, October 2018. ISSN 1662-
neural network simulation architecture. Neural Net- 453X. doi: 10.3389/fnins.2018.00774. URL [Link]
works, 97:28–45, January 2018. ISSN 08936080. doi: [Link]/article/10.3389/fnins.2018.00774/full.
10.1016/[Link].2017.09.011. URL [Link] [21] James C. Knight and Thomas Nowotny. GPUs Out-
[Link]/retrieve/pii/S0893608017302150. perform Current HPC and Neuromorphic Solutions in
[15] Benjamin Cramer, Yannik Stradmann, Johannes Schem- Terms of Speed and Energy When Simulating a Highly-
mel, and Friedemann Zenke. The Heidelberg Spiking Connected Cortical Model. Frontiers in Neuroscience,
Data Sets for the Systematic Evaluation of Spiking 12(December):1–19, 2018. ISSN 1662-453X. doi:
Neural Networks. IEEE Transactions on Neural Net- 10.3389/fnins.2018.00941. URL [Link]
works and Learning Systems, pages 1–14, 2022. ISSN org/article/10.3389/fnins.2018.00941/full.
2162-237X. doi: 10.1109/TNNLS.2020.3044364. URL [22] Arjun Rao, Philipp Plank, Andreas Wild, and Wolfgang
[Link] arXiv: Maass. A Long Short-Term Memory for AI Applica-
1910.07407. tions in Spike-based Neuromorphic Hardware. Nature
[16] David Blackman and Sebastiano Vigna. Scrambled Lin- Machine Intelligence, 4(5):467–479, May 2022. ISSN
ear Pseudorandom Number Generators. ACM Transac- 2522-5839. doi: 10.1038/s42256-022-00480-w. URL
tions on Mathematical Software, 47(4):1–32, December [Link]
2021. ISSN 0098-3500, 1557-7295. doi: 10.1145/ [23] Timo C Wunderlich and Christian Pehle. Event-
3460772. URL [Link] based backpropagation can compute exact gradients for
[17] Mantas Mikaitis. Stochastic Rounding: Algorithms spiking neural networks. Scientific Reports, 11(1):
and Hardware Accelerator. In 2021 International 12829, December 2021. ISSN 2045-2322. doi: 10.
Joint Conference on Neural Networks (IJCNN), pages 1038/s41598-021-91786-z. URL [Link]
1–6, July 2021. doi: 10.1109/IJCNN52387.2021. s41598-021-91786-z.
9533756. URL [Link] [24] Ian Kuon and Jonathan Rose. Measuring the gap be-
9533756/?arnumber=9533756. ISSN: 2161-4407. tween FPGAs and ASICs. Proceedings of the inter-
[18] Guillaume Bellec, Darjan Salaj, Anand Subramoney, nation symposium on Field programmable gate arrays
Robert Legenstein, and Wolfgang Maass. Long short- - FPGA’06, page 21, 2006. doi: 10.1145/1117201.
term memory and learning-to-learn in networks of spik- 1117205. URL [Link]
ing neurons. In Advances in Neural Information Pro- 1117201.1117205. Publisher: ACM Press Place: New
cessing Systems, volume 2018-Decem, pages 787–797, York, New York, USA ISBN: 1595932925.