MDCIM: MRAM-Based Digital Computing-in-Memory Macro for Floating-Point Computation with High Energy Efficiency and Low Area Overhead

Abstract

Computing-in-Memory (CIM) is a novel computing architecture that enormously improves energy efficiency and reduces computing latency by avoiding frequent data movement between the computation and memory units. Currently, digital CIM is regarded as more suitable for high-precision operations represented in floating-point arithmetic, as it is not limited by the bit width of ADC/DAC in analog CIM. However, the development of DCIM still faces two problems: On the one hand, mainstream SRAM-based DCIM memory cells introduce large area overheads, which contain at least six transistors per cell. On the other hand, existing DCIM solutions can only support the computing precision up to FP32, failing to meet the demands of high-accuracy application scenarios. To overcome these problems, this work designs a novel SOT-MRAM-based digital CIM macro (MDCIM) with higher area/energy efficiency and achieves double-precision floating-point (FP64) computation with a modified fused multiply–accumulate (FMA) module. The proposed design is synthesized with a 55 nm CMOS technology node, achieving 0.62 mW power consumption, 26.9 GOPS/W, and 0.332 GOPS/mm² energy efficiency at 150 MHz with 1.08 V supply. Circuit level simulation results show that the MDCIM can achieve higher area utilization compared to previous SRAM-based CIM designs.

1. Introduction

For decades, von Neumann architecture has been the foundation of modern computing systems. However, with the explosive growth in data scale, the physical separation between the computation and memory units leads to serious energy and latency issues due to the massive data movement between them [1]. The memory wall problem has become a major constraint for developing high-performance processors, especially for data-intensive applications. Recently, computing-in-memory (CIM) technology has garnered interest as a promising method to break the memory bottleneck existing in the traditional computing architecture, which integrates the computing logics into the memory array and enables in situ data processing without extra data movements [2]. According to the previous works, CIM architecture has demonstrated great advantages in improving energy efficiency, reducing latency, and boosting parallelism [3,4].

Distinguished from the computing paradigm, there are mainly two categories of CIM: analog CIM (ACIM) and digital CIM (DCIM) [5,6,7]. ACIM performs computation by applying voltage or current signals directly to the bit cells, possessing high on-chip bandwidth and computing parallelism. However, ACIM suffers from limited computing accuracy caused by transistor variation and the analog-to-digital converter (ADC), which consumes a large area and energy overhead as well [5]. In contrast, a DCIM encodes data as discrete states and executes arithmetic through a digital logic unit inside the memory array, enabling high computing precision and versatile application scenarios [6,7]. To date, various DCIM implementations have been explored based on different memories. Static random-access memory (SRAM) is one of the most prevalent candidates for DCIM, which can achieve superior speed and flexibility owing to its mature fabrication techniques with advanced CMOS technology [8]. Nonetheless, a volatile SRAM bit cell suffers from unavoidable static power dissipation, which is caused by the leaking current. Furthermore, a basic SRAM cell contains at least six transistors (6T), incurring considerable area overhead as the array size is scaled up. In contrast, nonvolatile memories like resistive random-access memory (ReRAM), phase-change memory (PCM), and magnetoresistive random-access memory (MRAM), allow for the CIM arrays to preserve data without static power dissipation. In addition, among these emerging memories, MRAM exhibits excellent performance in terms of fast speed, high endurance, CMOS compatibility, etc., making it an ideal candidate for DCIM.

MRAM based on new mechanisms has made rapid progress in CIM design. On one side, spin–transfer–torque MRAM (STT-MRAM) exhibits significant scalability with the bit cell structure of one transistor and one MTJ (1T-1MTJ) and holds vast potential for a wide range of CIM applications. For example, Wang et al. [9] proposed a tandem STT-MRAM-based CIM scheme to perform analog multiply and accumulate (MAC) operations through resistance summation of serial MTJs, acquiring a 2 Kb array size and achieving an energy efficiency of 63.7 TOPS/W for 8-bit precision. On the other side, spin–orbit–torque MRAM (SOT-MRAM), characterized by a three-terminal device (2T-1MTJ), decouples the read-and-write path with heavy metal and offers better stability, shorter switching latency, and lower write current. Doevenspeck et al. [10] constructed an optimized SOT-MRAM for weight memory and demonstrated the feasibility of implementing DNN inference with an ACIM scheme. Nevertheless, most prior research on MRAM-based CIM primarily focuses on the analog computing paradigm, while investigations into MRAM-based DCIM are still in the exploratory stages. In order to support high-precision computations (as illustrated in the above example, with precision not exceeding 8-bit), especially for floating-point (FP) computation, along with the progressively rising demand for computation accuracy in AI applications, the research into MRAM-based DCIM circuits has become more and more critical.

In this work, we proposed a digital SOT-MRAM-based CIM macro (MDCIM), integrating it with modified digital multiply and accumulate (MAC) units that are finely combined with SOT-MRAM arrays. The primary contributions of this work are summarized as follows:

(1)

Design a digital computing circuit based on SOT-MRAM to reduce the area overhead of MAC units.

(2)

Implement FP64 MAC operations based on FMA instructions to reduce the latency of FP operations.

(3)

Achieve higher calculation frequency based on digital CIM to reduce the energy consumption of the macro.

This paper is organized as follows: In Section 2, we briefly introduce the architecture of digital CIM circuits and floating-point MAC operations. Then, a pipeline MAC array is proposed for computation in Section 3, and this model is validated using EDA tools in Section 4. Finally, this work is summarized in Section 5.

2. Background

2.1. Working Modes of DCIM

As shown in Figure 1, the DCIM macro is primarily constituted by a memory array, a parallel adder tree, and a partial sum accumulator [11]. The memory array is chiefly composed of memory cells and logic operation gates. The memory cells are used to store reusable weights, which perform parallel multi-bit multiplication with input data in the logical operation units. The parallel adder tree is a full adder array that achieves carry addition by expanding bit width. The partial sum accumulator is made up of shifters and registers. Based on the DCIM macro, matrix vector multiplication (MVM) can be implemented. First, the weight parameters are loaded into the memory cells from the input buffer and are then fed into the memory array in sequence. Subsequently, the memory array performs parallel multiplication of 1-bit inputs and multi-bit weights in one computational cycle and transfers the product to a multilevel full adder to obtain a partial sum. Finally, the shift accumulator accomplishes the addition of products of the low-bit input and high-bit input.

The basic memory cell in a DCIM memory array is principally implemented with SRAM technology, whose elemental structure is depicted in Figure 2a. The 6T-SRAM cell consists of two cross-coupled inverters formed by transistors M1–M4, which store a single bit of data, and two additional access transistors, M5 and M6, controlling the access to the cell during read and write operations. This fundamental structure induces low storage density, incurring substantial area overhead in large-scale integrated arrays. Concurrently, the volatility of SRAM also leads to high static power dissipation. On the contrary, a SOT-MRAM cell merely necessitates two transistors, significantly diminishing the integrated area cost, as shown in Figure 2b. The SOT-MRAM cell contains an MTJ for data storage, consisting of two ferromagnetic layers separated by a thin insulator. The value of ‘0/1’ is represented by the low/high resistance state of the MTJ. Transistors T1 and T2 are utilized to regulate the reading and writing of the stored data, respectively. Moreover, SOT-MRAM also exhibits excellent characteristics including nonvolatility, high endurance, and rapid switching, rendering it an ideal candidate for DCIM [12,13,14,15,16,17].

2.2. Principle of FP64 Computation

According to the definition of the IEEE binary floating-point arithmetic standard (IEEE 754), the FP64 format utilizes 8 bytes for data encoding and storage, partitioned into three segments. The highest bit that expresses sign is abbreviated to S. The middle 11 bits, which express an exponent, are abbreviated as E, while the remaining 52 bits express the fraction of a number, approximating the significant digits of the number, denoted as M [18,19]. The actual value of a normalized FP64 number N can be expressed as follows:

$$N={\left(-1\right)}^S\times \left(1.M\right)\times 2^{E-1023}$$

(1)

Based on this storing format, the execution process of FP64 multiplication can be split up into five steps, as shown in Figure 3, described as follows:

①

Unpack the operands into the FP multiplier. The sign, exponent, and mantissa of each operand are separated. Then, the mantissa is converted to the significand by adding hidden bit 1 at the top digit.

②

XOR the sign bits of two operands to obtain the sign bits of the product.

③

Multiply the significands of two operands to obtain the product, which is the key step that restricts the overall computational speed.

④

Add the exponents of two operands. The sum is shifted with the normalization of the product.

⑤

Pack parts as products. Remove the hidden bit, and reassemble the sign, exponent, and mantissa of the results.

2.3. Process of MAC Operation

The MAC operation is the most common operation in neural network computing, accounting for over 90% of the computational workload [20,21]. One form of this operation is a ‘multiplier + adder’. However, the complex data format of floating-point numbers results in high area overhead and latency. Therefore, we constructed a fused multiply–accumulate (FMA) instruction to achieve the MAC operation:

$$a\times x+b=d$$

(2)

As shown in Figure 4, we first use the exponential difference ‘$e_a+e_x-e_b$’ to align $s_b$. Then, we add $s_b$ to the partial product of $s_a$ and $s_x$. Finally, perform normalization and rounding operations. Unlike the ‘multiplier + adder’, the FMA instruction only performs one rounding operation on the sum of full precision products ($s_a\times s_x$) and the addend ($s_b$), allowing for better results in high-precision calculations. At the same time, the instruction integrates a portion of multipliers and adders, reducing area overhead. It also combines multiplication and addition into a one-step operation, reducing latency.

3. Our MDCIM Work

3.1. Algorithm and Architecture of FP64 Computation

In the process of FP64 MAC operations, the power consumption and latency of multipliers are much higher than that of adders. In traditional FP64 multiplication, the 52-bit mantissa multiplier includes 52 $\times$ 52 AND gates, 52 registers of 52 bits each to store column results, and 51 full adders for shift accumulation, with their bit width increasing from 54 bits to 104 bits. Since the number of AND gates in the multiplier cannot be reduced, and the full adder has the highest bit width and is the part with the largest area overhead, we consider reducing the bit width of the shift adder to decrease the area overhead of the entire multiplier.

Segmenting the two high bit-width multiplicands is an effective way to reduce the bit width of the multiplier, and we can decrease the area overhead required for a single MAC unit by increasing the number of computations. As shown in Figure 5, the 52-bit mantissa is divided into 4 sets of 13b inputs, e.g., $a_0~a_3$ and $x_0~x_3$, and then they multiply each other to obtain 16 sets of 26-bit partial products: $a_0{x}_0~a_3{x}_3$. In this process, the number of AND gates and registers remains unchanged (13 $\times$ 13 $\times$ 4 $\times$ 4 = 52 $\times$ 52), but due to the reduction in input bit width, the bit width of the full adder is greatly reduced, with the largest full adder bit width being 26 bits. Then, we accumulate these partial products. It can be seen from Figure 5 that there are partial products with the same shift, and we first accumulate them. For instance, $p_3$ is the sum of $a_3{x}_0$, $a_2{x}_1$, $a_1{x}_2$, and $a_0{x}_3$. Since it is same-position accumulation, we only need thirteen 26-bit full adders, then use eight sets of 26-bit registers to store the results. Ultimately, we obtain seven sets of 26-bit partial sums: $p_0~p_6$ and a set of 15-bit partial sum with carry: $p_7$. These partial sums can be rearranged into two groups of high bit-width numbers based on the parity of the base, and then, through a 106-bit full adder, the 52-bit mantissa product result is obtained. This method of mantissa segmentation can greatly reduce the area overhead of high bit-width multipliers. Under the same process node, the transistor count of the segmented mantissa multiplier is 0.74× that of the traditional multiplier, and it can also achieve a reduction in power consumption.

We proposed the MDCIM macro by mapping mantissa segment multiplication to digital circuits. Since the algorithm involves a large number of independent MAC operations, we constructed a pipeline multiplication array as the overall architecture. As shown in Figure 6, the macro consists of 16 sets of MAC units, 3 sets of adders, and multilayer latches. A MAC unit group including a 13-bit MRAM cell, a 13-bit multiplier, a carry–save adder (CSA), and a full adder (FA).

As shown in Figure 6, the MAC unit performs preliminary MAC operation between the segmented $m_a$ and $m_{x }$. The MRAM cell is used to save the segmented mantissa $m_x$, i.e., $x_0$, $x_1$, $x_2$, and $x_3$. The multiplier multiplies two segmented mantissas, e.g., $a_0$ and $x_0$. The CSA is a three-input and two-output adder. The first input to CSA is the 26-bit partial product, e.g., $a_0{x}_0$, from the multiplier while the second input is the carry from the low-order MAC unit, and the final input is the sum of the same position of MAC units. Taking MAC unit ③ as an example, inputs of its CSA are $a_2{x}_1$, carry from unit ②, and sum from unit ①. The CSA will output the $su{m}_{mid}$ and $carr{y}_{mid}$ of 26 bits, which have overlapping bits. Therefore, the two are fed into the full adder again to obtain a 26-bit sum and a 1-bit carry that are sent to other MAC units. We constructed a 4 × 4 pipeline array multiplier with 16 sets of MAC units and 3 sets of adders to obtain 8 sets of partial sums: $p_0~p_7$. In addition, because each column undergoes different computational cycles, adding multilevel latches to the array can ensure time synchronization.

However, in the multiplier, the mantissa only participates in a portion of the operation. We still need to consider the contribution of hidden bits. As shown in Figure 5, after adding hidden bits, the product of two significands ($s_a\times s_x$) has two additional sources, including the shifted $s_a$ and $m_{x }$. In order to reduce the complexity of digital circuits, we placed them in the adder tree. As shown in Figure 7, the sum of the shifted $s_a$, $m_{x }$ and partial sums ($p_0~p_7$) represents the product of two significands ($s_a\times s_x$), while the fifth set of data is the exponentially aligned addend $s_b$. After calculation in the multilevel adder tree, the significand of the result $d$ can be obtained.

3.2. Array of SOT-MRAM

The memory array we constructed based on SOT-MRAM is a part of the CIM macro. Compared to SRAM and STT-MRAM, the advantages of SOT-MRAM are primarily reflected in its higher write speeds and storage density. To evaluate their performance, we used CMOS 55 nm technology to separately synthesize three sets of 16 $\times$ 13 bit memory arrays based on SRAM, STT-MRAM, and SOT-MRAM. We added peripheral read–write circuits to them and measured the relevant parameters. The models of STT-MRAM and SOT-MRAM referred to the designs in References [22,23]. We evaluated the write speed of the model by measuring the write latency of circuits and assessed the storage density of the model by measuring the layout area of arrays.

As shown in Figure 8, the memory density of SOT-MRAM arrays is not significantly different from that of STT-MRAM arrays, but both are much higher than SRAM arrays. Using SOT-MRAM arrays in CIM macros can reduce the area overhead of the chip. Meanwhile, SOT-MRAM resolves the slow writing issue of STT-MRAM. It can be observed from the figure that the write time of SOT-MRAM arrays has now surpassed that of SRAM arrays, which can further lower the latency in the calculation process.

4. Simulation and Discussion

The proposed MDCIM macro model is described using Verilog-A language while large amount of test data was used to simulate and validate the functions designed in the model. FP64 can be converted to the scientific notation representation of decimal numbers, with defined ranges for its exponent and mantissa. Therefore, during the process of generating random inputs, we constrain the range of the data’s exponent to cover the decimal number exponent range that FP64 can represent, while the mantissa is generated using the rand function defined in the stdlib.h header file of C99. In this manner, we tested two million sets of data covering all ranges in FP64. Due to the pseudorandom distribution nature of the rand function, the random test inputs used in this study follow a normal distribution.

Table 1. Some data examples used to test the function of digital circuits.

Input			Output
a	x	b	d
0x40454d4a9a95352a (42.60)	0x40425f44be897d13 (36.74)	0x408073cc6798cf32 (526.47)	0x40a057d8496a0647 (2091.92)
0x402b9cf739ee73dd (13.81)	0x4045124e249c4939 (42.14)	0x4081a05840b08161 (564.04)	0x4091e7931bfbd1a3 (1145.89)
0x4019d4b3a96752cf (6.46)	0x404265e8cbd197a3 (36.80)	0x407edc24b8497093 (493.76)	0x4086db06849fa665 (731.38)
0x405676f4ede9dbd4 (89.86)	0x40340aa015402a80 (20.04)	0x407726f04de09bc1 (370.43)	0x40a0f6acacac57ff (2171.34)
0x40580ec81d903b20 (96.23)	0x40585af4b5e96bd3 (97.42)	0x4077c61f8c3f187e (380.38)	0x40c30da89eda44c6 (9755.32)

shows some data examples used to test the function of digital circuits. We use hexadecimal numbers starting with 0x to represent 64b operands. At the set of normalized numbers, a is 0x40454d4a9a95352a, which is approximately 42.60383922849208 in decimal representation; x is 0x40425f44be897d13, which is approximately 36.74428540910062 in decimal representation; b is 0x408073cc6798cf32, which is approximately 526.4748069704276 in decimal representation; and d is 0x40a057d8496a0647, which is approximately 2091.9224351055777 in decimal representation. Tests have shown that the output of the DMCIM macro is consistent with the standard reference, with code coverage reaching 100%.

In order to evaluate the model, we synthesized the MDCIM macro using CMOS 55 nm technology and measured the relevant parameters. At a working frequency of 150 MHz and a power supply voltage of 1.08 V, the macro’s power consumption is 0.62 mW. The power efficiency of MDCIM can reach 26.9 GOPS/W. We also drew and measured the layout of the memory array and peripheral circuits, with an area of 51,814 μm², achieving a power density of 0.322 GOPS/mm². As shown in

Table 2. Comparison of the proposed MDCIM with previous works.

	ISSCC’19 [26]	JSSC’20 [27]	VLSI’21 [25]	ESSCIRC’22 [24]	This Work
Memory array	28 nm SRAM	28 nm SRAM	28 nm SRAM	28 nm SRAM	SOT-MRAM
Supply voltage (V)	0.6–1.1	0.6–1.1	0.76–1.1	0.5–0.9	0.9–1.32
Supported floating-point format	FP8	FP8	BF16	FP32	FP64
Macro size (mm2)	2.55 (Chip)	2.7 (Die)	5.832 (Die)	0.0439	0.051814
Frequency (MHz)	475	114	250	400	150
GOPS/W/FP64	8.59	4.86	47.5	24.75	26.9
GOPS/mm2/FP64	0.354	0.426	1.28	1.21	0.322

, we compared MDCIM with other works supporting floating-point calculation. Ref. [24] is quite representative among them, as it designs an architecture capable of executing FP32 MAC operations. Ref. [24] also adopts the FMA computational philosophy. Initially, it performs the multiplication of the mantissas, then identifies the maximum exponent, completes the shifting for both the product and the addend, and finally carries out the accumulation uniformly. Since the mantissa of FP32 only has 23 bits, the author did not perform segmentation operations on the multiplication of mantissas, but instead allocated sufficient space for both the multiplier and adder. In order to fairly compare the energy efficiency and energy density of different works, we normalized the metrics, as shown in Table 2, to make them equivalent to the scenario of processing FP64. Upon comparison, the energy efficiency of our proposed MDCIM macro: 26.9 GOPS/W/FP64, has already surpassed that of a majority of studies of the same kind, including Ref. [24], which is based on the 28 nm process for FP32 MAC operations. The advantage primarily comes from the segmentation algorithm, as well as the low write time and low leakage current of SOT-MRAM. During the comparison process, we also discovered some works with better performance, such as Ref. [25]. The differences in performance metrics arise from multiple factors: Firstly, due to the constraints of MRAM manufacturing technology, our architecture’s performance still lags behind some leading-edge SRAM-based floating-point computing architectures. However, as the backend integration progress of MTJ and CMOS technologies advances, the performance gap is expected to narrow. Meanwhile, we noticed that the data format handled by Ref. [25] is BF16, which is different from FP16. The mantissa of BF16 has only 7 bits, while that of FP16 has 10 bits. This implies that the bit width of the multiplier, which is the major bottleneck restricting the area and speed of the compute macro, has been reduced. Therefore, when performing BF16 MAC operations, the memory requirement of the compute macro is halved, and the throughput is doubled. This provides a significant insight for us in designing floating-point computing circuits. By lowering the precision requirements of the data, more optimization space can be provided to the hardware. In fact, approximate computing also accelerates computation based on this method. Finally, to our knowledge, this is the first work exploring an MRAM-based DCIM tailored for high-precision FP64 operations, and we believe this is a noteworthy step forward.

5. Conclusions

DCIM has natural advantages in high-precision computing, but there are still challenges in terms of power consumption and area overhead. This paper proposed an all-digital CIM Macro based on SOT-MRAM (MDCIM), which can perform FP64 FMA instructions, with CSA and a pipeline MAC array to optimize digital computing circuits for reducing power consumption and latency. This work firstly proposed the application of digital CIM with SOT-MRAM in FP64 computing. However, the high bit width and large computation volume brought by floating-point calculations still pose a challenge for on-chip deployment. Currently, introducing approximate computing into floating point could significantly enhance energy efficiency [28,29]. Approximate computing can substantially accelerate the speed of operations and reduce the complexity of circuit. At the same time, for binary matrices where elements are only zero and one, the precision loss brought about by approximate matrices is within an acceptable degree. In MAC operations, we can choose suitable approximate computing methods based on the sparsity of the matrices involved in the operations. This could potentially reduce the circuit area used for computations in digital memory computation macros. Therefore, approximate floating-point computation will become a very important research approach in the near future.