2412 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 69, NO.

6, JUNE 2022

SRAM-Based In-Memory Computing Macro

Featuring Voltage-Mode Accumulator and
Row-by-Row ADC for Processing
Neural Networks
Junjie Mu , Graduate Student Member, IEEE, Hyunjoon Kim , Graduate Student Member, IEEE,
and Bongjin Kim , Senior Member, IEEE

Abstract— This paper presents a mixed-signal SRAM-based

in-memory computing (IMC) macro for processing binarized
neural networks. The IMC macro consists of 128 × 128 (16K)
SRAM-based bitcells. Each bitcell consists of a standard 6T
SRAM bitcell, an XNOR-based binary multiplier, and a pseudo-
differential voltage-mode driver (i.e., an accumulator unit).
Multiply-and-accumulate (MAC) operations between 64 pairs of
inputs and weights (stored in the first 64 SRAM bitcells) are
performed in 128 rows of the macro, all in parallel. A weight-
stationary architecture, which minimizes off-chip memory
accesses, effectively reduces energy-hungry data communications.
A row-by-row analog-to-digital converter (ADC) based on
32 replica bitcells and a sense amplifier reduces the ADC Fig. 1. Energy consumption from memory access and computation [5].
area overhead and compensates for nonlinearity and variation.
The ADC converts the MAC result from each row to an storage. Platforms with limited energy and compute resources,
N-bit digital output taking 2N -1 cycles per conversion by
sweeping the reference level of 32 replica bitcells. The remaining such as edge devices and battery-operated mobile devices,
32 replica bitcells in the row are utilized for offset calibration. have restricted the use of cutting-edge DNN models [1].
In addition, this paper presents a pseudo-differential voltage- Researchers have proposed various solutions to address this
mode accumulator to address issues in the current-mode or issue. Among them, binarized neural networks (BNNs) stand
single-ended voltage-mode accumulator. A test chip including out as promising models for integrating them into resource-
a 16Kbit SRAM IMC bitcell array is fabricated using a
65nm CMOS technology. The measured energy- and area- constrained devices. BNNs can effectively reduce memory
efficiency is 741-87TOPS/W with 1-5bit ADC at 0.5V supply and and energy requirements minimizing accuracy degradation
3.97TOPS/mm2 , respectively. compared to baseline models with full precision [1]–[4].
Index Terms— Mixed-signal, in-memory computing, binarized The conventional von Neumann architecture faces inevitable
neural network, multiply-and-accumulate, voltage-mode, SRAM. challenges due to excessive energy consumption from
billions of arithmetic operations and the associated memory
read/write operations for processing BNNs with millions of
I. I NTRODUCTION binary weights. Fig. 1 compares energy consumption from
memory access and arithmetic logic operations with different
R ECENTLY, deep neural networks (DNNs) have proved
excellent capabilities for image classification and speech
recognition tasks. As the DNN models become more and more
precisions [5]. A large amount of energy is consumed by
moving data back and forth between memory and computing
sophisticated, they require higher computing power and larger units. In particular, external DRAM access requires 2nJ,
equivalent to the energy needed for the 2,000 times repeated
Manuscript received November 18, 2021; revised January 29, 2022; operations of 32bit addition.
accepted February 14, 2022. Date of publication March 3, 2022; date of In conventional von Neumann architecture, computations
current version May 27, 2022. This article was recommended by Associate
Editor T. Hanyu. (Corresponding author: Bongjin Kim.) are performed in the processing unit while storing both
Junjie Mu and Hyunjoon Kim are with the School of Electrical and instruction and data in memory, as shown in Fig. 2(a).
Electronic Engineering, Nanyang Technological University, Singapore 639798 Therefore, we need increasingly high bandwidth for the
(e-mail: junjie003@[Link]; kimh0003@[Link]).
Bongjin Kim is with the Department of Electrical and Computer communication between processor and memory, which
Engineering, University of California at Santa Barbara, Santa Barbara, becomes a critical performance bottleneck in the computer
CA 93106 USA (e-mail: bongjin@[Link]). systems, as there is a clear limit that we can achieve in terms
Color versions of one or more figures in this article are available at
[Link] of the physical system bandwidth [6]. Both near-memory and
Digital Object Identifier 10.1109/TCSI.2022.3152653 in-memory computing architectures are attractive candidates
1549-8328 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See [Link] for more information.

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.
MU et al.: SRAM-BASED IMC MACRO FEATURING VOLTAGE-MODE ACCUMULATOR AND ROW-BY-ROW ADC 2413

Fig. 3. Design challenge of the current-mode CIM using a 6T SRAM bitcell.

Fig. 2. (a) Traditional von neumann architecture and (b) in-memory
computing (IMC) architecture.

for reducing the required communication bandwidth. The idea

of processing near memory tracks back to the 1960s [7]
and the near-memory computing (NMC) system was first
appeared in the early 1990s [8], [9]. Recently, the NMC
has regained attention as it can minimize memory access
while computing DNNs using local arithmetic units. While it
lowers the energy for memory access by reducing the distance
between computing units and memory, the latency has not been
much improved as it still accesses memory row-by-row as we
did in the traditional compute architecture. Unlike NMC, IMC
brings the processing task into the memory array to boost the Fig. 4. Analog CIM implementations: (a) voltage-domain; (b) charge-domain.
overall processing speed and minimize energy by eliminating
required memory access. Fig. 2(b) shows that data are now
computed in the memory unit rather than moving them from/to a compact bitcell size, its dynamic range is limited to prevent
the processor [11]. Thus, we can maximize energy efficiency read disturbance and variation. The 8T SRAM with two extra
and throughput by removing massive data communications read transistors [14] has decoupled SRAM read and write.
between computing units and memory. However, the limited dynamic range has yet to be resolved.
Numerous DNN hardware accelerators based on analog Biswas et al. [13] presented a 10T SRAM-based bitcell
and digital custom ASIC implementations have been pro- with two decoupled read-ports to resolve the read disturbance
posed [12]–[23], [36]–[38]. Advantages of digital accelerators issue. Moreover, the rail-to-rail dynamic range is achieved
include the robustness to process variation, noise tolerance, by voltage-mode operation (charge sharing), as shown in
high computing precision, and reconfigurability. However, Fig. 4(b). However, it suffers from a large ADC/DAC
the achievable energy and area efficiency is much lower overhead. A ternary bit-scalable CNN accelerator based on
than the analog counterpart. Therefore, analog accelerators 17T SRAM for XNOR neural network has been proposed
have gained significant attention despite the analog-specific in [35]. The 17T SRAM consisting of two 6T SRAM and five
non-idealities. NMOS performs ternary 1/4-bit XNOR-based multiplication
Fig. 3 shows a typical current-mode 6T SRAM-based IMC and current-mode accumulation. However, multibit (4-bit)
design with minimum size transistors. In this work, short inputs are applied serially to bitcells, resulting in long
pulses (inputs=’1’) are applied to wordlines (WLs) to activate computing latency. And additional circuits are demanded to
multiple bitcells for massively parallel multiplications in an merge partial sums, thereby increasing the area and power
SRAM array. The multiplication result is represented by a consumption. Yin et al. [16] proposed a voltage-mode IMC to
unit discharge current (Iunit ), and accumulation is realized by perform a ternary XNOR-and-accumulate (XAC) computation,
the sum of discharge current followed by the bitline (BL) where a voltage driver is formed by PU and PD drivers in
voltage drop. The compact 6T SRAM-based IMC has two the same column. The accumulation is proportional to the
major challenges: read disturbance and PVT variation. The ratio between the number of PU and PD drivers, where the
accessed bitcells risk being overwritten in computing mode imbalance between these drivers introduces nonlinearity and
when the BL is discharged to a low voltage. On the other increases the design complexity.
hand, the wider distribution of Iunit (σ ≈ 23%μ) significantly This work proposes an SRAM-based IMC macro with two
affects linearity and accuracy because the BL voltage drop is key features: 1) pseudo-differential voltage-mode accumulator;
proportional to Iunit . Kim et al. [15] introduced a voltage-mode 2) 1-5bit reconfigurable row-by-row ADC. The proposed
IMC macro based on 6T SRAM bitcell with split wordlines, work with pseudo-differential accumulation provides better
where the accumulation is performed by programming the linearity, reduced variation, and a wider (2x) output range. The
number of pull-up (PU) and pull-down (PD) paths. While it has embedded ADCs with replica bitcells maximize throughput.

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.
2414 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 69, NO. 6, JUNE 2022

Fig. 5. The proposed bitcell schematic and layout.

This article is organized as follows. Section II introduces the

proposed SRAM bitcell and its MAC operations. Section III
presents the overall architecture, proposed embedded ADC,
and offset calibration design. The key contributions of this
paper are described in section IV. Experimental results
and discussions are given in Sections V and VI. Finally,
Section VII concludes this paper.

II. P ROPOSED I N -M EMORY C OMPUTING M ACRO

A. Proposed SRAM-Based Bitcell Design
Fig. 5 shows the bitcell schematic and layout of the
proposed IMC macro. It comprises a standard 6T SRAM
weight storage cell, an XNOR-gate binary multiplier, and
Fig. 6. Proposed MAC operations: (a) XNOR-gate binary multiplication;
a pseudo-differential voltage-mode accumulation unit (i.e., (b) voltage-mode accumulation; (c) a table summarizing binary input and
a pair of P and N inverter drivers). Traditional SRAM bitlines weight multiplication result.
(BL and BLB) are used for driving inputs (X and /X) in
a vertical direction, and the voltage outputs (VP and VN )
same binary value (+1 or −1), the output Y is connected to
are in a horizontal direction. The inverter pair (P/N) are
VDD (pull-up). When they have the opposite value, the output
two tri-state inverters, where a pair of PMOS and NMOS is
Y is connected to VSS (pull-down). Fig. 6(c) shows a table
connected in series with two parallel inverters. The proposed
that summarizes binary input and weight multiplication results.
65nm bitcell is designed using only minimum-sized transistors,
and it occupies 1.8 × 4μm2 , as shown in Fig. 5, right. The
proposed bitcells were designed following the logic rule. C. In-Memory Voltage-Mode Accumulation
The accumulator occupies the largest area (i.e., 44.5%) while
the 6T SRAM bitcell is 1.8 × 1.015 μm2, which occupies The binary multiplication results from XNOR gates are then
25.4% of the IMC bitcell area. accumulated using pseudo-differential voltage-mode drivers
in each bitcells. Fig. 6(b) describes the operation of the
proposed voltage-mode accumulation. A pair of inverters in
B. In-Memory Binary Multiplication each bitcell drive the shared positive (V P ) and negative (VN )
Fig. 6 describes the binary MAC operations using output voltage nodes, and the 128 bitcells in the same macro
an XNOR-gate binary multiplier and a pseudo-differential row build pull-up (PU) and pull-down (PD) paths controlled by
voltage-mode accumulator in the proposed CIM bitcell. In the the XNOR binary multiplier outputs. Thus, a voltage divider
normal SRAM mode, two NMOS access transistors in the is formed between VDD and VSS, and the resulting voltage
standard 6T SRAM cell are turned on, and the weight is difference between V P and VN (V P -VN ) is now proportional
written into the SRAM cell via BL drivers. The weight is to the number of PU and PD drivers connected to V P and VN .
stored as a binary value (i.e., +1 when W is ‘high’ or Detailed analysis of the proposed voltage accumulation is as
VDD and −1 when W is ‘low’ or VSS) in the SRAM cell. follows. Suppose each row has a total of N bitcells, of which
In the IMC mode, the weights (W and /W) from the SRAM U is the number of PU bitcells, and D is the number of PD
internal nodes are directly connected to the XNOR gate and bitcells. Note that the number of PU (or PD) bitcells represents
multiplied with the external inputs via the conventional SRAM the sum of +1 (or −1) XNOR outputs. Let the resistance of
bitlines (BL and BLB). Fig. 6(a) presents four different binary PU and PD bitcells be RU and R D , respectively. The total PU
multiplication operations based on the combinations of binary resistance is RU /U , and the total PD resistance is R D /D. V P
inputs and weights. When the input and the weight share the and VN can be formulated as (1) and (2) based on the voltage

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.
MU et al.: SRAM-BASED IMC MACRO FEATURING VOLTAGE-MODE ACCUMULATOR AND ROW-BY-ROW ADC 2415

to X63 ) are multiplied to the same number of weights stored

in the bitcells via XNOR gates and accumulated into a voltage
difference (VP -VN ) using a pair of inverters (P/N), working
as a voltage-mode accumulator unit. The accumulated voltage
difference is converted to a digital code using an embedded
ADC, comprising a sense amplifier and 32 replica bitcells used
for generating ADC references. The other 32 bitcells located
on the far right are used to calibrate offsets. Note that the
proposed macro could achieve the highest level of parallelism
thanks to the row-by-row ADCs embedded in each macro row.

A. Power-Gating Design
The voltage-mode drivers consume significant power since
a DC path is formed while accumulating based on voltage-
mode operation, and hence a power gating is necessary to save
Fig. 7. Overall architecture of the proposed mixed-signal IMC macro. power. The power gating circuit comprises a sense amplifier
(used for ADC), a decision detector (DET), and a pair of
division. PMOS/NMOS in the voltage-mode driver unit of each bitcell,
RD
as shown in Fig. 9(a). A pair of P/N inverters of the voltage-
RDU mode driver unit is controlled by a pair of enable signals
VP = D
= (1)
RD
+ Ru R D U + Ru D (EN and ENB) generated by the decision detector. After the
D U
RD sense amplifier detects the accumulated voltage, EN goes to
U RD D
VN = = (2) low (and ENB goes to high), and hence the voltage-mode
RD
+ Ru R D D + Ru U drivers in the bitcells are turned off to save power. A timing
U D
The voltage difference between VP and VN is given in (3). diagram of the proposed power-gating scheme is described
in Fig. 9(b). On the rising edge of the clock, the sense
RDU RD D amplifier outputs (OP and ON ) are pre-charged to high. After
V P − VN = − (3)
R D U + Ru D R D D + Ru U pre-charging, the DET circuit enables PMOS/NMOS of the
Exchanging the number of PD and PU units (D and U), the voltage-mode circuits, and the voltage difference (VP -VN ) is
voltage difference becomes settled based on the voltage-mode accumulate operation. Now,
the DET circuit detects the difference and disables the power-
RD D RDU
V P − VN = − (4) gating PMOS/NMOS transistors of the bitcells. Meanwhile,
R D D + Ru U R D U + Ru D the sensed data is stored in the register. Note that the power-
Equations (3) and (4) indicate that the proposed accumulator gating immediately shuts off the current flow of the drivers
shows symmetric and monotonic transfer characteristics. The and hence maximizes the power saving.
residual nonlinearity due to the mismatches between PU and
PD resistances has been compensated using the proposed B. Row-by-Row ADC
ADC, discussed in Sections III and IV.
A compact row-by-row ADC scheme is proposed to convert
analog dot-product results from each row of the CIM macro
III. OVERALL A RCHITECTURE while the macro operates in a massively parallel manner. The
Fig. 7 shows the overall architecture of the proposed mixed- row-by-row ADC includes a sense amplifier and 32 replica
signal IMC macro. A 128 × 128 SRAM bitcell array is bitcells in each macro row. The replica bitcells are used
integrated for 128x massively parallel dot-product operations. for generating a 32-level ADC reference by sweeping the
Each macro row consists of 128 bitcells, where each bitcell inputs while the SRAM bitcells store 0, and a StrongARM
comprises three circuit blocks: a standard 6T SRAM cell, sense amplifier [33], [34] is used as a 1bit quantizer. The
an XNOR gate, and a voltage mode accumulator unit. Besides sense amplifier amplifies the input difference using two cross-
a 16K core bitcell array, the I/O periphery includes a wordline coupled inverters and produces a rail-to-rail output in response
decoder, bitline drivers, and output generators (comparators to the polarity of the input difference. The outputs are
and registers). discharged to VSS when the clock signal (CK) is disabled.
Fig. 8 shows the block diagram of the proposed mixed- When CK goes high, the cross-couple pair will pull up the
signal artificial neuron. A neuron with 64 pairs of inputs output to VDD once the input difference is positive. While
and weights (Fig. 8, left) in a neural network layer is the ADC reference is swept sequentially, the VP and VN
implemented using 128 bitcells and an output generator, nodes are further pulled up or down by the replica bitcells
including 64 bitcells for a dot-product and the other 64 bitcells co-integrated with other bitcells in the same macro row. The
for ADC and offset calibration (Fig. 8, right). The proposed analog accumulated voltage is converted to a thermometer
CIM macro can perform vector-matrix multiplication with code bit-by-bit, which is eventually converted to a binary code
64 inputs and 64×128 weights in a cycle. Sixty-four inputs (X0 using a thermometer-to-binary decoder.

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.
2416 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 69, NO. 6, JUNE 2022

Fig. 8. Block diagram of the proposed mixed-signal artificial neuron with 128 SRAM-based bitcells and an output generator.

results (positive outputs ‘1’ and negative outputs ‘0’). The 7bit
thermometer code is generated at the moment that the output
of the comparator flips to 0 from 1. Eventually, the 7bit
thermometer code is converted to a 3bit binary digital code.

C. Binary-Searching Based Offset Calibration

The proposed ADC compensates for the mismatches
between PU/PD resistances by using 32x replica bitcells to
generate a reference. Meanwhile, we assigned the additional
32x replica bitcells dedicated to calibrating the comparator
offsets based on the binary-searching offset calibration
scheme [15]. The offset is calibrated while the bitcells for
the dot-product and ADC operations are balanced, as shown
in Fig. 11. Half of the bitcells for offset calibration are initially
set to +1, while the other half is set to −1. After observing
the sense amplifier output, the bitcells are updated based on
the observed output binary value. If it is 1 (or 0), a quarter
of the bitcells are now set to +1 (or −1), while the other
three quarters are set to −1 (or +1). Note that the number
of bitcells to update is reduced by half for each iteration (i.e.,
Fig. 9. The circuit design and operation of the proposed power-gating. binary-searching). The operation continues until the number of
updated bitcell is 1. For 32 replica bitcells, it takes five cycles,
and the calibrated replica bitcells will minimize the variation
A dot-product with 64 pairs of inputs and weights requires
due to offsets (Fig. 16, right).
a full dynamic range of −64 to +64. However, the proposed
ADC reduces its input range to half of the full range (i.e.,
IV. K EY C ONTRIBUTIONS
−32 to +32), considering the distribution of the accumulated
results of neural networks with margin. For instance, the A. Pseudo-Differential Voltage-Mode Accumulator
accumulator results are distributed between −26 and +28 from Yin et al. have presented a single-ended voltage-mode
the second hidden layer of an example multi-layer perceptron accumulator [16], where all XNOR results in the same column
(MLP). Note that the 512 input activations are divided into drive a shared RBL. However, owing to the inconsistent
eight sets of 64 inputs to fit in the proposed macro size. The driving strength, the analog accumulation result is highly
ADC resolution is reconfigured by the step size of the replica asymmetric and nonlinear. In this work, a pseudo-differential
bitcell sweeping, illustrated in Fig. 10. The ADC conversion voltage-mode accumulator is proposed to compensate for
rate is determined by the number of ADC reference sweep non-ideal properties originating from the single-ended driver
cycles (i.e., 2N -1 cycles at N-bit mode). Fig. 10 illustrates strength imbalance. The simulated accumulator transfer
the operating sequence of the proposed ADC at 3bit mode. characteristics are shown in Fig. 12. Single-ended and
After the accumulator results are fixed (i.e., −26 to +28 in pseudo-differential accumulators are composed of 64 units
this example), ADC reference bitcells are swept from the (an inverter as a unit for the single-ended and a pair of
highest (+24) to the lowest (−24) with the step size of positive/negative inverters as a unit for the pseudo-differential
eight since the ADC resolution is 3bit. In each conversion accumulator). The accumulator input range is from −64 to
cycle, the ADC reference is compared with the accumulation +64. As shown in Fig. 12(left), the baseline single-ended
results, and the comparator outputs the decision according accumulator shows severe nonlinearity, asymmetry, and wide
to the sign of the sum of ADC reference and accumulation variation in its transfer curve. In contrast, a symmetrical

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.
MU et al.: SRAM-BASED IMC MACRO FEATURING VOLTAGE-MODE ACCUMULATOR AND ROW-BY-ROW ADC 2417

Fig. 10. Operating sequence of the proposed row-by-row ADC at 3bit mode.

analog multiplexer to minimize the area overhead of the IMC

macro [16]. Due to a multiplexed architecture, the overall
macro throughput has been reduced by 64-times than the
ideal case where 64 columns are operating parallel. A charge-
sharing-based ADC is used to convert the accumulated analog
voltage to a 7bit digital code, where 16 such ADCs are shared
by 256 rows [15]. Four single-sloped ADCs are embedded
in the macro with 512 × 256 bitcells [25]. Besides the 64:1
multiplexing, the high-resolution (8bit) single-sloped ADC
substantially lowers the macro throughput. A macro with a
64×64 bitcell array and sixteen Flash ADCs is presented [26],
where the multiplexing ratio (i.e., 4:1) is lower than other
macros [15], [16], [25]. In addition to the low throughput due
Fig. 11. Offset-calibration based on binary searching. Eight offset columns to the multiplexing architecture, the fixed ADC resolution of
are shown as an example.
prior works limits the broad utilization of the IMC macros.
Okumura [35] et al.. proposed one 8-bit ADC comprising
128 reference cells and a 1-bit comparator in a single column
(148 rows x 4 banks). The reference cells are slightly modified
from the bitcells and connected to PBL and NBL. The voltages
of PBL and NBL are iteratively compared until the current
flowing PBL and NBL are the same by gradually changing
the number of accessed reference cells. Although this design
can offer a high throughput (i.e., single column embeds one
ADC), it still achieves fixed ADC resolution.
The embedded row-by-row ADC is proposed in this work
to improve parallelism with a minimal hardware overhead.
Fig. 13 compares the architecture of the conventional and
Fig. 12. Transfer characteristics of voltage-mode accumulators. the proposed ADC. In this work, the accumulated voltage is
converted to the digital code using a comparator and replica
bitcells (working as a reference generator). The area overhead
transfer characteristics curve with better linearity and reduced of the replica bitcells is negligible compared to conventional
variation is shown in the transfer curve of the proposed bulky ADCs with many comparators and passive devices
accumulator. The accumulator output in the proposed work is (resistors and capacitors). The proposed ADC enables high
defined by a voltage difference between the positive and the throughput by embedding ADC in each macro row (i.e., row-
negative single-ended accumulator. In addition to the enhanced by-row ADC). Besides, the proposed ADC resolves residual
transfer characteristic, the proposed accumulator also doubles nonlinearity and variation issues. Fig. 14 shows the linearity
the dynamic range. of conventional ADC with fixed references. The ADC directly
quantizes the accumulator output, and hence both nonlinearity
B. Row-by-Row ADC and variation remain as it is.
The ADC is a critical building block to realize high In contrast, the proposed replica-based ADC eliminates the
precision output for the IMC macro design. A Flash ADC residual nonlinearity and process variation originating from the
with 11 output levels is shared within 64 columns via a 64:1 accumulator’s non-ideal transfer characteristics. Fig. 15 shows

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.
2418 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 69, NO. 6, JUNE 2022

TABLE I
ADC C OMPARISON TABLE

Fig. 13. The comparison of conventional ADC and proposed row-by-row

ADC with the proposed accumulator.

Fig. 16. Measured linearity before and after offset calibration.

Fig. 14. Quantization of pseudo-differential voltage-mode accumulator using

the conventional ADC (simulated: a 4bit conventional ADC).

Fig. 17. Measured DNL&INL of the proposed ADC at 5bit mode.

V. E XPERIMENT R ESULTS
Fig. 15. Quantization of pseudo-differential voltage-mode accumulator using A. Linearity and ADC Measurement
the proposed row-by-row ADC (simulated: a 4bit row-by-row ADC).
The measured output linearity of 5bit ADCs from 128 rows
is shown in Fig. 16. While it offers good linearity, the
simulated results, and we can observe that the nonlinearity and measured results without calibration show a significant
the variation have been removed from the output code. Besides variation due to offsets between rows. The variation is reduced
the improved linearity and process insensitivity, the proposed after row-by-row calibrations using replica bitcells, as shown
ADC offers design simplicity since an extra circuit to generate in Fig. 16 (right). The measured DNL and INL of the ADCs
analog references is not required. However, the area overhead from 128 rows are +0.16/−0.12LSB and +0.02/−0.41LSB,
(57.6%) of ADCs is still high compared to the MAC bitcell respectively, after offset calibration, as shown in Fig. 17.
array (64 × 128 bitcells) since 128x ADCs are embedded
in the CIM macro. Some prior works implement limited
conventional ADCs; therefore, we propose the metric of B. Power Consumption
normalized area (per ADC area/MAC array area) to compare Fig. 18 shows the simulated power consumptions with swept
ADCs areas fairly. Table I compares the proposed and the dot-product results from −128 to 128 when the supply voltage
other ADCs from the recently published in-memory computing is 0.5-to-1V. The results show symmetric power consumptions
macros [13], [16], [25], [26]. The proposed ADC has the for the dot-product results with the same absolute values. The
smallest normalized area (0.0045) in comparison to other peak power is 272.5μW at 1V when the dot-product result
works [16], [26]. is 0 (i.e., when the number of pull-up and pull-down drivers

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.
MU et al.: SRAM-BASED IMC MACRO FEATURING VOLTAGE-MODE ACCUMULATOR AND ROW-BY-ROW ADC 2419

Fig. 21. Mapping input feature maps and filter weights into a 128 × 64 array
in (a) fully connected layers and (b) convolution layers.
Fig. 18. Simulated power consumption versus dot-product result
(−128 to 128). TABLE II
MLP M ODEL FOR MNIST D ATASET

TABLE III
VGG-L IKE M ODEL FOR C IFAR -10 D ATASET

Fig. 19. Simulated power consumptions without and with power-gating.

Fig. 20. Measured energy per operation (OP) and energy efficiency. D. Accuracy
The proposed macro is used for processing binarized neural
enabled for the accumulation are the same). Fig. 19 compares networks. Fig. 21 shows a mapping example of the first hidden
the simulated power consumptions without and with a power- layers of the MLP and VGG-like model in Tables II and III.
gating operation at 1V (left) and 0.5V (right). It shows The input layers are not processed using the macro as the
17.4-19.8x (or 7.1-18.5x) power saving at 1V (or 0.5V) with precision of the input dataset (MNIST or CIFAR-10) is not
power-gating. binarized.
The IMC macro with a two-dimensional bitcell array is an
excellent fit for processing fully connected layers. For instance,
the proposed macro with 128 × 64 bitcells directly maps a
C. Energy Efficiency
fully connected layer with 64 inputs and 128 outputs. The
Fig. 20 shows the measured energy per operation and weights of the layer (i.e., 64 × 128 binary weights) are stored
energy efficiency while sweeping supply voltage (0.5V-1V) in the SRAM bitcells and used for massive parallel multiply-
and ADC resolution (1-5bit). As the resolution increases and-accumulate operations. For processing a convolutional
from 1bit to 5bit, the energy consumption increases from layer, filter weights stored in a three-dimensional kernel (e.g.,
16fJ to 135fJ (8.4-times) at 1V. The measured energy is 3 × 3 × 128 weights in 128 input channels with 3 × 3 filter
1.3fJ (or 11.5fJ) at 1bit (or 5bit) when the supply voltage size) are unrolled and stored in the SRAM bitcells in the same
reduces to 0.5V. The measured energy efficiency with 1bit (or macro row for accumulation.
5bit) is 741TOPS/W (or 87TOPS/W) at 0.5V and 63TOPS/W When the kernel size exceeds the macro size, the large
(or 7.4TOPS/W) at 1V. kernel is split into many smaller sub-matrices, and then the

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.
2420 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 69, NO. 6, JUNE 2022

Fig. 22. MNIST & CIFAR-10 classification accuracy versus standard

deviation.

partial sum of the sub-matrices is post-accumulated to obtain

output feature maps, as shown in Fig. 21(a). In this example, Fig. 23. Implementation of multi-bit weight precision using multiple drivers.
a 512 × 512 kernel is split into four eight-channel 128 × 64
sub-matrices (i.e., macro size). Here, the ADC resolution is
set to 5bit to retain accuracy when post-accumulating partial
sums from sub-matrices.
For the convolutional layers with a large kernel size, we also
split the kernel into sub-matrices. For instance, a 3×3 × 128 ×
128 kernel (i.e., 3 × 3 filters with 128 input and 128 output
channels) is split into two nine-channel 128 × 64 sub-matrices
(i.e., macro size), as shown in Fig. 21(b). Partial sums from
each macro are post-accumulated to generate output feature
Fig. 24. Die micrograph.
maps. The weights stored in the sub-matrices are reused
900-times to complete a 128-channel 30 × 30 output feature
map.
We evaluated the impact of process variation in the proposed
macro on classification accuracy based on an MLP for
MNIST [27] and a VGG-like CNN model for CIFAR-10 [28].
Training and inferences are performed on binarized neural
networks, and the detailed model parameters are summarized
in Tables II and III. The image classification accuracy
gradually decreases with the increase of the standard deviation
(std-dev). The accuracy shows a steep decline when the std-
dev/full-range exceeds 4.5%. The classification accuracy of the
Fig. 25. Area breakdown: a macro (left) and a row (right).
MLP-based model degrades faster than the VGG-like CNN
model. The worst-case variation is 29.01mV based on 1K
runs Monte-Carlo simulation, and the std-dev/full-range is and the bottom bitcell stores the magnitude. The driver is a
1.8% (when the full range is 1.6V). In the MLP model, the tri-state inverter, where two extra pairs of NMOS and PMOS
simulated classification accuracy is 96.62%, which is 0.51% switches are connected in series for pull-down and pull-up
lower than the software baseline. In the VGG-like model, paths. The inverter is turned off when the magnitude is 0
the classification accuracy is 86.51% (1.04% lower than the (i.e., W1 = 0 in Fig. 23). A table summarizing combinations
benchmark), as shown in Fig. 22. of a 7-level weight and a 2-level binary input is shown
in Fig. 23 (right).
VI. D ISCUSSION This work somewhat relaxes the data conversion overhead
The weight can be reconfigured to the higher bit precision of conventional analog CIM macros by embedding a replica-
by integrating multiple voltage mode drivers in the bitcell. based ADC in each artificial neuron. The variation issue
Fig. 23 describes the block diagram of a bitcell comprising also has been addressed with a pseudo-differential voltage-
four SRAM bitcells, two XNOR gates, and three voltage-mode mode accumulator. Nevertheless, we still have a residual
drivers for realizing a 7-level weight. A bitcell consists of two nonlinearity and variation in the voltage-mode accumulation
parts: LSB (two 6T SRAM bitcells, an XNOR gate, and a due to mismatches between drivers. The residual non-idealities
driver) and MSB (two 6T SRAM bitcells, an XNOR gate, and are accumulated in cascaded feed-forward neural network
two drivers). Two SRAM bitcells are included in each part, layers and eventually degrade the overall accuracy. We can
where the top SRAM bitcell stores the sign of the weight, alleviate such issues by introducing the mismatches in the

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.
MU et al.: SRAM-BASED IMC MACRO FEATURING VOLTAGE-MODE ACCUMULATOR AND ROW-BY-ROW ADC 2421

TABLE IV
P ERFORMANCE C OMPARISON W ITH S TATE - OF - THE - ART IN -M EMORY C OMPUTING M ACROS

forward propagation while training [24]. In the training phase, using MNIST dataset is 96.62%, and the accuracy in the
we can add a random noise with normal distribution following VGG-like model using the CIFAR-10 dataset is 86.51%.
the standard deviation of the mismatches. After this process,
the trained weights can minimize the impact of the residual R EFERENCES
non-idealities on the accuracy.
A die micrograph of the fabricated 65nm test chip with [1] M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, “XNOR-net:
Imagenet classification using binary convolutional neural networks,”
16Kb SRAM occupying 0.118mm2 is shown in Fig. 24. 2016, arXiv:1603.05279.
Fig. 25 shows area breakdowns of the fabricated macro and its [2] M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, and Y. Bengio,
row. The macro includes a 16Kb bitcell array (51.2%), output “Binarized neural networks: Training deep neural networks with weights
and activations constrained to +1 or -1,” 2016, arXiv:1602.02830.
generators (2.7%), WL/BL decoders (5.1%), input/output
[3] I. Hubara, M. Courbariaux, D. Soudry, E. El-Yaniv, and Y. Bengio,
buffer (41%), and others (2%). Each row includes 128x bitcells “Binarized neural networks,” in Proc. Adv. NIPS, Barcelona, Spain,
and an output generator, which occupies 95.3% and 4.7% of 2016, pp. 4114–4122.
the row area. Each bitcell consists of three building blocks, [4] J. Choi, S. Venkataramani, V. Srinivasan, K. Gopalakrishnan, Z. Wang,
and P. Chuang, “Accurate and efficient 2-bit quantized neural networks,”
a multiplier, an accumulator, and an SRAM bitcell. Each in Proc. Conf. Syst. Mach. Learn. (SysML), 2019, pp. 348–359.
occupies 26.5%, 41.1%, and 27.7% of the row area. Table IV [5] M. Horowitz, “1.1 computing’s energy problem (and what we can do
compares the proposed work with the state-of-the-art IMC about it),” in IEEE ISSCC Dig. Tech. Papers, Feb. 2014, pp. 10–14.
[6] J. Wang et al., “A 28-nm compute SRAM with bit-serial logic/arithmetic
macros. operations for programmable in-memory vector computing,” IEEE
J. Solid-State Circuits, vol. 55, no. 1, pp. 76–86, Jan. 2020.
[7] H. S. Stone, “A logic-in-memory computer,” IEEE Trans. Comput.,
VII. C ONCLUSION vol. C-19, no. 1, pp. 73–78, Jan. 1970.
[8] D. G. Elliott, W. M. Snelgrove, and M. Stumm, “Computational RAM:
A memory-SIMD hybrid and its application to DSP,” in Proc. IEEE
This paper presents a mixed-signal SRAM-based IMC Custom Integr. Circuits Conf., May 1992, pp. 30–36.
macro for processing binarized neural networks. The proposed [9] M. Gokhale, B. Holmes, and K. Iobst, “Processing in memory: The
macro enables high throughput and energy efficiency using terasys massively parallel PIM array,” Computer, vol. 28, no. 4,
pp. 23–31, Apr. 1995.
massively parallel mixed-signal in-memory arithmetic with [10] M. Huang, M. Mehalel, R. Arvapalli, and S. He, “An energy efficient
compact and reconfigurable ADC. The macro comprises 32-nm 20-mb shared on-die L3 cache for Intel Xeon processor E5
128 rows of bitcells, where a single row consists of 64 family,” IEEE J. Solid-State Circuits, vol. 48, no. 8, pp. 1954–1962,
Aug. 2013.
bitcells for dot-product, 32 to generate a reference for ADC, [11] N. Verma et al., “In-memory computing: Advances and prospects,” IEEE
and 32 for offset calibration. A pseudo-differential voltage- Solid State Circuits Mag., vol. 11, no. 3, pp. 43–55, Aug. 2019.
mode accumulator compensates for the PMOS and NMOS [12] W. S. Khwa et al., “A 65 nm 4 Kb algorithm-dependent computing-in-
memory SRAM unit-macro with 2.3ns and 55.8TOPS/W fully parallel
driver strength imbalance. The residual variation between product-sum operation for binary DNN edge processors,” in IEEE ISSCC
ADCs from different rows has been reduced by calibrating Dig. Tech. Papers, Feb. 2018, pp. 496–497.
offsets. The proposed N-bit ADC requires 2N -1 cycles to [13] A. Biswas and A. P. Chandrakasan, “CONV-SRAM: An energy-
complete each conversion. The measured DNL and INL are efficient SRAM with in-memory dot-product computation for low-power
convolutional neural networks,” IEEE J. Solid-State Circuits, vol. 54,
+0.16/−0.12LSB and +0.02/−0.41LSB after calibration. The no. 1, pp. 217–230, Jan. 2019.
measured energy efficiency ranges from 741TOPS/W at 0.5V [14] C. Yu, T. Yoo, T. Kim, K. Chai, and B. Kim, “A 16 K current-based
and 1bit (the highest) to 7.4TOPS/W at 1V and 5bit (the 8T SRAM compute-in-memory macro with decoupled read/write and 1-
5bit column ADC,” in Proc. IEEE Custom Integr. Circuits Conf. (CICC),
lowest). The classification accuracy based on a four-layer MLP Mar. 2020, pp. 1–4.

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.
2422 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 69, NO. 6, JUNE 2022

[15] J. Kim et al., “Area-efficient and variation-tolerant in-memory BNN [37] E. Lee et al., “A charge-domain scalable-weight in-memory computing
computing using 6T SRAM array,” in Proc. Symp. VLSI Circuits, macro with dual-SRAM architecture for precision-scalable DNN
Jun. 2019, pp. C118–C119. accelerators,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 68, no. 8,
[16] S. Yin, Z. Jiang, J.-S. Seo, and M. Seok, “XNOR-SRAM: In-memory pp. 3305–3316, Aug. 2021.
computing SRAM macro for binary/ternary deep neural networks,” IEEE [38] J. Song et al., “TD-SRAM: Time-domain-based in-memory computing
J. Solid-State Circuits, vol. 55, no. 6, pp. 1733–1743, Jun. 2020. macro for binary neural networks,” IEEE Trans. Circuits Syst. I, Reg.
[17] J. Zhang, Z. Wang, and N. Verma, “A machine-learning classifier Papers, vol. 68, no. 8, pp. 3377–3387, Aug. 2021.
implemented in a standard 6T SRAM array,” IEEE J. Solid-State
Circuits, vol. 52, no. 4, pp. 915–924, Apr. 2017.
[18] D. Shin, J. Lee, J. Lee, and H.-J. Yoo, “DNPU: An 8.1 TOPS/W
reconfigurable CNN-RNN processor for general-purpose deep neural
networks,” in IEEE ISSCC Dig. Tech. Papers, Feb. 2017, pp. 240–241. Junjie Mu (Graduate Student Member, IEEE)
[19] J. Lee, C. Kim, S. Kang, D. Shin, S. Kim, and H.-J. Yoo, “UNPU: received the B.E. degree from the College of
An energy-efficient deep neural network accelerator with fully variable Information Science and Engineering, Northeastern
weight bit precision,” IEEE J. Solid-State Circuits, vol. 54, no. 1, University, Shenyang, China, in 2017, and the M.S.
pp. 173–185, Jan. 2019. degree in electronics from Nanyang Technological
[20] S. Yin et al., “An energy-efficient reconfigurable processor for binary- University, Singapore, in 2018, where she is
and ternary-weight neural networks with flexible data bit width,” IEEE currently pursuing the Ph.D. degree with the School
J. Solid-State Circuits, vol. 54, no. 4, pp. 1120–1136, Apr. 2019. of Electrical and Electronic Engineering.
[21] H. Kim, Q. Chen, T. Yoo, T. T.-H. Kim, and B. Kim, “A 1-16 b precision Her research interests include hardware accel-
reconfigurable digital in-memory computing macro featuring column- erators for solving mathematical problems and
MAC architecture and bit-serial computation,” in Proc. IEEE 45th Eur. memory-centric circuits.
Solid State Circuits Conf. (ESSCIRC), Sep. 2019, pp. 345–348.
[22] H. Kim, Q. Chen, and B. Kim, “A 16K SRAM-based mixed-signal in-
memory computing macro featuring voltage-mode accumulator and row-
by-row ADC,” in Proc. IEEE Asian Solid-State Circuits Conf. (A-SSCC), Hyunjoon Kim (Graduate Student Member, IEEE)
Nov. 2019, pp. 35–36. received the B.A. degree in physics from the
[23] D. Bankman, L. Yang, B. Moons, M. Verhelst, and B. Murmann, “An Oberlin College, Oberlin, OH, USA, in 2008, and
always-on 3.8 μ J/86% CIFAR-10 mixed-signal binary CNN processor the M.S. degree in electrical engineering from the
with all memory on chip in 28-nm CMOS,” IEEE J. Solid-State Circuits, University of Minnesota, Minneapolis, MN, USA,
vol. 54, no. 1, pp. 158–172, Jan. 2019. in 2012. He is currently pursuing the Ph.D. degree in
[24] D. Miyashita, S. Kousai, T. Suzuki, and J. Deguchi, “A neuromorphic digital computing-in-memory (CIM) circuit design
chip optimized for deep learning and CMOS technology with time- and architecture for machine learning applications
domain analog and digital mixed-signal processing,” IEEE J. Solid-State with Nanyang Technological University, Singapore.
Circuits, vol. 52, no. 10, pp. 2679–2689, Oct. 2017. From 2013 to 2018, he was with Gainspan
[25] M. Kang, S. K. Gonugondla, A. Patil, and N. R. Shanbhag, Corporation, San Jose, CA, USA, where he worked
“A multi-functional in-memory inference processor using a standard 6T as an RF Test Engineer for IEEE 802.11 standards. In 2018, he joined
SRAM array,” IEEE J. Solid-State Circuits, vol. 53, no. 2, pp. 642–655, Nanyang Technological University. His research interests include memory-
Feb. 2018. centric systems, neural network accelerators, and its design methodologies.
[26] M. E. Sinangil et al., “A 7-nm compute-in-memory SRAM macro
supporting multi-bit input, weight and output and achieving 351
TOPS/W and 372.4 GOPS,” IEEE J. Solid-State Circuits, vol. 56, no. 1,
pp. 188–198, Jan. 2021.
[27] Y. LeCun, C. Cortes, and C. Burge. (1998). The MNIST Bongjin Kim (Senior Member, IEEE) received the
Database of Handwritten Digits. [Online]. Available: B.S. and M.S. degrees from POSTECH, Pohang,
[Link] South Korea, in 2004 and 2006, respectively, and
[28] A. Karpathy. (2011). Lessons Learned from Manually Classi- the Ph.D. degree from the University of Minnesota,
fying CIFAR-10. [Online]. Available: [Link] Minneapolis, MN, USA, in 2014.
04/27/manually-classifying-CIFAR10 He was with Rambus, Sunnyvale, CA, USA,
[29] J. Yang et al., “24.4 sandwich-RAM: An energy-efficient in-memory where he was a Senior Staff Member and worked
BWN architecture with pulse-width modulation,” in IEEE ISSCC Dig. on the research of high-speed serial link circuits
Tech. Papers, Feb. 2019, pp. 394–395. and microarchitectures. He was a Post-Doctoral
[30] X. Si et al., “A twin-8T SRAM computation-in-memory unit-macro for Research Fellow with Stanford University, Stanford,
multibit CNN-based AI edge processors,” IEEE J. Solid-State Circuits, CA, USA. From 2006 to 2010, he was with
vol. 55, no. 1, pp. 189–202, Jan. 2020. Samsung Electronics, Yongin, South Korea, where he performed research
[31] S. K. Gonugondla, M. Kang, and N. Shanbhag, “A 42 pJ/decision on clock generators for high-speed serial links and clock generators.
3.12 TOPS/W robust in-memory machine learning classifier with From 2012 to 2014, he worked as a Research Intern with Texas Instruments,
on-chip training,” in IEEE ISSCC Dig. Tech. Papers, Feb. 2018, Dallas, TX, USA; IBM TJ Watson Research, Yorktown Heights, NY, USA;
pp. 490–491. and Rambus. He was an Assistant Professor with Nanyang Technological
[32] K. Ando et al., “BRein memory: A single-chip binary/ternary University, Singapore, from 2017 to 2020. He is currently an Assistant
reconfigurable in-memory deep neural network accelerator achieving Professor with the Department of Electrical and Computer Engineering
1.4 TOPS at 0.6 W,” IEEE J. Solid-State Circuits, vol. 53, no. 4, (ECE), University of California at Santa Barbara, CA, USA. His current
pp. 983–994, Apr. 2018. research interests include quantum-inspired and brain-inspired computing,
[33] B. Razavi, “The StrongARM latch [A circuit for all seasons],” IEEE memory-centric computing devices, circuits, and architectures, hardware
Solid-State Circuits Mag., vol. 7, no. 2, pp. 12–17, Spring 2015. accelerators, alternative computing, and mixed-signal circuit design techniques
[34] T. Kobayashi, K. Nogami, T. Shirotori, Y. Fujimoto, and O. Watanabe, and methodologies. He was a recipient of the Prestigious Doctoral Dissertation
“A current-mode latch sense amplifier and a static power saving input Fellowship Award based on his Ph.D. research, the ISLPED International
buffer for low-power architecture,” in Proc. Symp. VLSI Circuits Dig. Low Power Design Contest Award, and Intel/IBM/Catalyst Foundation Award
Tech. Papers, 1992, pp. 28–29. from CICC. He has been serving as a Technical Program and Review
[35] S. Okumura, M. Yabuuchi, K. Hijioka, and K. Nose, “A ternary based Committee Member for DAC, CICC, and AICAS, and a Guest Editor for the
bit scalable, 8.80 TOPS/W CNN accelerator with many-core processing- IEEE J OURNAL ON E MERGING AND S ELECTED T OPICS IN C IRCUITS AND
in-memory architecture with 896K synapses/mm2,” in Proc. Symp. VLSI S YSTEMS . His research works appeared at top peer-reviewed integrated circuit
Technol., Jun. 2019, pp. C248–C249. design and automation conference proceedings and journals, including ISSCC,
[36] C. Yu, T. Yoo, H. Kim, T. Kim, K. Chai, and B. Kim, “A logic- VLSI Symposium, IEEE J OURNAL OF S OLID -S TATE C IRCUITS , IEEE
compatible eDRAM compute-in-memory with embedded ADCs for T RANSACTIONS ON C IRCUITS AND S YSTEMS I: R EGULAR PAPERS , CICC,
processing neural networks,” IEEE Trans. Circuits Syst. I, Reg. Papers, ESSCIRC, ASSCC, ISLPED, DATE, ICCAD, and IEEE T RANSACTIONS ON
vol. 68, no. 2, pp. 667–679, Feb. 2021. V ERY L ARGE S CALE I NTEGRATION (VLSI) S YSTEMS .

Authorized licensed use limited to: Marvell International Ltd. Downloaded on July 28,2024 at 23:46:59 UTC from IEEE Xplore. Restrictions apply.