A Study of Advancing Ultralow-Power 3D Integrated Circuits with TEI-LP Technology and AI-Enhanced PID Autotuning

Abstract

The 3D integrated circuit (3D-IC) is garnering significant attention from academia and industry as a key technology leading the post-Moore era, offering new levels of efficiency, power, performance, and form-factor advantages to the semiconductor industry. However, thermal management in 3D-ICs presents a critical challenge that must be overcome to ensure prosperity for this technology. Unlike traditional thermal management solutions that perceive heat generation in 3D-ICs negatively and aim to eliminate it, this paper proposes, for the first time, a thermal management method that positively utilizes heat to achieve low-power operation in 3D-ICs. This approach is based on a novel discovery that circuits can reduce power consumption at higher temperatures by leveraging the temperature effect inversion (TEI) phenomenon in ultralow-voltage (ULV) operating circuits, a characteristic of low-power techniques (TEI-LP techniques). Along with a detailed explanation of this discovery, this paper introduces new thermal management technologies for practical application in 3D-ICs. Furthermore, to achieve optimal energy efficiency with the proposed technology, we develop a temperature controller essential for this purpose. The developed controller is a deep learning-based PID autotuner. This paper proves the theoretical validity of the AI control algorithm designed for this purpose and demonstrates the functional correctness and power-saving effectiveness of the developed controller through intensively conducted simulations.

1. Introduction

Moore’s Law has been a driving force in the downsizing of semiconductors over recent decades, facilitating the integration of more transistors onto a chip. However, device scaling has now reached the quantum mechanical limits, especially in sub-nanometer processes, leading to significantly lower yields and astronomically high chip development costs. Chiplet technology, which divides a chip into multiple smaller chiplets instead of creating a monolithic die, and the associated 3D integrated circuit (3D-IC) technology that allows vertical stacking of these chiplets not only overcome the limitations of device scaling but also offer benefits such as reduced wire delay, high interconnect bandwidth, and improved energy efficiency. These technologies are thus considered pivotal in leading the post-Moore era [1,2,3]. Despite the many advantages of 3D-ICs, the increased packaging density achieved by stacking multiple dies in a 3D structure presents a critical challenge—heat management [4,5]. The core of 3D-ICs relies almost exclusively on thermal conduction through-silicon vias (TSVs), making thermal management even more challenging. To address this, various cooling techniques [6,7] and thermal-aware floor-planning methods [8,9] have been explored.

This paper begins by questioning whether heat should only be seen as a challenge to be overcome in 3D-ICs. This inquiry is motivated by our recent discovery that heat is no longer a negative aspect in ultralow-voltage (ULV) operating chips. ULV circuit technology has been a fundamental approach to achieving low-power chips since the early 2000s. It has been extensively researched to overcome the major hurdle of ULV circuits, namely their vulnerability to process, voltage, and temperature (PVT) variations [10,11,12]. As a result of these efforts, ULV-operating chips have been realized. Unfortunately, the inevitable performance sacrifice for low power in these ULV-operating chips has historically limited the range of applications that could operate effectively on these chips, thus significantly constraining the practicality and application of ULV circuit technology. However, recent developments have signaled a shift to this trend. With the advent of the artificial intelligence (AI) era, and the shift in hardware computing power from relying on a high-performance single processing unit to multiple low-performance processing units inherently capable of the parallel processing of AI applications [13,14], the value of ULV circuit technology has significantly increased. As a consequence, recently, various ULV chips have been actively developed and have emerged [15,16,17].

Over the past decade, we have conducted in-depth research on ULV circuits, focusing on the unique phenomenon where circuit speed increases with rising temperature [18], termed the temperature effect inversion (TEI) phenomenon [19]. We have dedicated our efforts to developing TEI-aware low-power (TEI-LP) techniques, which are now in a mature state [20,21,22,23,24,25,26,27]. Furthermore, we have recently developed a processor chip using the 28nm FDSOI process, incorporating the TEI-LP technology. During our experiments with this chip, we obtained a fascinating result: as the temperature rises, the power consumption decreases. This finding is contrary to the conventional wisdom that higher temperatures lead to increased power consumption in semiconductor chips. Our discovery highlights the remarkable impact of TEI-LP technology in producing such an unexpected outcome. In this paper, we present this discovery for the first time and provide a detailed analysis of its implications and underlying mechanisms.

Our new finding suggests that heat may no longer need to be viewed negatively in 3D-ICs. Instead of unconditionally reducing chip temperature through traditional thermal management techniques, maintaining heat at a certain level could improve the chip’s energy efficiency. However, existing control systems developed for all previous 3D-IC thermal management technologies are not suitable for our proposed new thermal management approach, necessitating the development of a dedicated controller, which we have addressed in this paper.

To maximize the power reduction effect in 3D-ICs utilizing our proposed TEI-LP technology, we developed a cooling system controller that precisely maintains the optimal temperature without overshooting. The design of this controller encountered challenges due to the heterogeneous configuration of the 3D-IC dies and the variety of interface materials used in packaging [4]. To overcome these challenges, we engineered a deep learning-based proportional–integral–derivative (PID) autotuning controller, capable of adaptively adjusting gain values for variable thermal models. Specifically, we enhanced a conventional autotuning PID controller [28] with a neural network, integrating a reinforcement learning-based secondary autotuning process. This neural network, designed with an error-driven algorithm, enabled the controller to overcome inaccuracies in 3D-IC thermal models. Notably, the network was trained to minimize overshooting during the control process, thereby maximizing the power-saving benefits of the TEI-LP technology for enhanced energy efficiency.

To validate the efficacy of our developed controller, we first implemented an autotuning PID controller using the first-order plus dead-time (FOPDT) control model. Then, we derived a simplified 3D thermal model to design the neural network and conducted a hyperparameter optimization. Subsequently, we tested the performance of our proposed controller not only on the derived thermal model but also on a more realistic 3D-IC structure using HotSpot [29] simulations. Through these simulations, we confirmed that our proposed controller adaptively learned across various thermal models and robustly performed PID control with minimal overshooting. As a result, we demonstrated that the utilization of our TEI-LP technology can achieve up to 22.1% energy savings in 3D-ICs.

The remainder of this paper is organized as follows. Section 2 introduces the TEI-LP technology and our pioneering discovery of the positive relationship between temperature and power consumption. It also explains the impact of controller overshooting on energy efficiency in relation to voltage scaling performance in the 3D-IC cooling system. In Section 3, we introduce a deep learning-based autotuning PID controller for applying the TEI phenomenon in 3D-ICs and evaluate the performance of the proposed controller under a simplified thermal model. Section 4 presents the results of a control simulation of the proposed controller using a more realistic 3D-IC model. Finally, Section 5 offers conclusions.

2. Application of TEI-Aware Low-Power Techniques in 3D-IC: Embracing Higher Temperatures for Efficiency

2.1. TEI-Aware Voltage Scaling Technique

The occurrence of increased speed with rising temperature in ULV circuits contrasts with the traditional temperature–speed relationship seen in standard circuits, marking a significant shift in understanding. This characteristic can be analyzed by examining the variations in the on current strength $I_{on}$ of semiconductor transistors in relation to the operating temperature T. In more detail, $I_{on}$ is a function of T, and its impact varies depending on the circuit’s operating regime. In ULV circuits where the gate–source voltage $V_{gs}$ (or simply considered as the supply voltage $V_{dd}$) is close to or nearly equal to the threshold voltage $V_{th}$, the effect of T on $I_{on}$ is different compared to superthreshold voltage (STV) operating circuits, or nominal supply voltage operations, where $V_{gs}\le V_{th}$. Mathematically, this can be expressed as [18]:

$$I_{on}\propto \left\{\begin{array}{cc}\mu \left(T\right)·e^{{(V_{gs}-V_{th}\left(T\right))}^{\gamma }}\hfill & \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{S}\mathrm{T}\mathrm{V} \mathrm{r}\mathrm{e}\mathrm{g}\mathrm{i}\mathrm{m}\mathrm{e}\\ \mu \left(T\right)·e^{\frac{V_{gs}-V_{th}\left(T\right)}{S\left(T\right)}}\hfill & \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{U}\mathrm{L}\mathrm{V} \mathrm{r}\mathrm{e}\mathrm{g}\mathrm{i}\mathrm{m}\mathrm{e}\end{array}\right.,$$

(1)

where $\gamma$ is the velocity saturation effect factor, $\mu$ is the carrier mobility, and S is the subthreshold swing. From the equation, for an STV circuit, as T increases, $\mu \left(T\right)$ decreases significantly compared to the effect of $V_{th}\left(T\right)$ on $I_{on}$, resulting in a decrease in $I_{on}$. On the other hand, in the NTV circuit operating with ULV, as T increases, it has an exponential effect on $I_{on}$ on $V_{th}\left(T\right)$ and $S\left(T\right)$, so even if $\mu \left(T\right)$ decreases, $I_{on}$ increases.

Subsequently, the delay of the transistor ${\tau }_D$ can be formulated as a function of $I_{on}$ in the following manner [12]:

$${\tau }_D=\frac{C}{I_{on}}\frac{V_{dd}}{2},$$

(2)

where C is the transistor capacitance. From (1) and (2), we can deduce that as the temperature rises, in the STV regime, the circuit delay increases (i.e., the speed of the circuit decreases), while in the ULV regime, the circuit delay decreases (i.e., the speed of the circuit increases). Furthermore, when actual semiconductor transistor characteristic parameters are applied, the extent of the speed/delay variation with temperature in the STV and ULV regimes is markedly different, with the difference in the ULV circuit being significantly greater than that in the STV. We previously named this phenomenon temperature effect inversion (TEI) in our prior research [19].

The TEI phenomenon has catalyzed a paradigm shift in the low-power design field, as it implies that increasing temperatures can generate a delay margin, which can be advantageous for low-power technologies. More specifically, circuits have a defined operational temperature range ($T_{min}\le T\le T_{max}$) to ensure normal functioning, and the circuit’s operational speed is determined by the slowest speed within this temperature range. In other words, the circuit’s clock speed (or target clock frequency, $f_{target}$) is decided based on the worst-case corner delay within the temperature range (i.e., $f_{target}=\frac{1}{{\tau }_D}$).

Thus, in ULV circuits, $f_{target}$ is determined by the delay ${\tau }_{target}$ at $T_{min}$. As T increases, the difference in delay, resulting from ${\tau }_{target}$, becomes more significant, and this delay margin, as shown in Figure 1, is defined as the TEI benefit. We have confirmed through previous studies that this TEI benefit is evident across the entire operating temperature range of the chip (typically from −25 °C to 125 °C) in various semiconductor technology processes [19,27].

TEI-LP techniques exploit the TEI benefit through various approaches, such as TEI-aware voltage scaling (VS) [22,24,27,30], frequency scaling (FS [21], body biasing (BB) [26], or dynamic power management (DPM) [25]. Among these, the best-known technique is TEI-aware VS (TEI-VS), which converts the TEI benefit into power savings through voltage downscaling. This is because $V_{dd}$ is closely related to the circuit delay ${\tau }_D$ and power consumption P. First, the relationship between $V_{dd}$ and ${\tau }_D$ can be derived as follows from (1) and (2):

$${\tau }_D=\frac{C}{2\beta }\frac{V_{dd}}{e^{V_{dd}/n{V}_{th}}},$$

(3)

where $\beta$ and n are the strength of transistor and the subthreshold factor, respectively. Next, P is the sum of dynamic power $P_{dynamic}$ and static power $P_{static}$, each expressible as a function of $V_{dd}$:

$$P_{dynamic}=\alpha ·C·V_{dd}^2·f,P_{Static}=V_{dd}·I_{off},$$

(4)

where $\alpha$ and $I_{off}$ are the activity factor and off current, respectively. Therefore, by (3), we can reduce $V_{dd}$ by the amount of TEI benefit, which translates into power savings as per (4).

2.2. TEI-VS-Based Thermal Management for 3D-IC: Utilizing the New Finding of a Positive Relationship between Temperature and Power Savings

In our recent research, we designed an ultralow-power (ULP) system-on-chip (SoC) based on ULV circuitry and fabricated it using a 28 nm FDSOI process [27]. We demonstrated the significant effectiveness of TEI-VS with this chip, and Figure 2 reports the results of applying TEI-VS at 10 °C intervals. Our developed chip supported an $f_{target}$ of 50 MHz and 100 MHz, with the worst-case corner minimum $V_{dd}$ for these speeds being 0.54 V and 0.61 V, respectively, at the lowest operating temperature of −40 °C. For reference, the nominal $V_{dd}$ for the 28 nm FDSOI process we used is 0.7 V. As shown in the figure, the value of $V_{TEI‐VS}$, which is the result of applying TEI-VS, decreases with increasing temperature. This clearly confirms the following equation:

$$V_{TEI-VS}\left(T_1\right)\ge V_{TEI-VS}\left(T_2\right)\mathrm{f}\mathrm{o}\mathrm{r}{T}_{min}\le T_1\le T_2.$$

(5)

Applying TEI-VS results in reduced power consumption. The power consumption corresponding to the TEI-VS results in Figure 2 is presented in Figure 3, with results for an $f_{target}$ of (a) 50 MHz and (b) 100 MHz. In both figures, the black line represents the power consumption without applying TEI-VS, at $V_{dd}=$ (a) 0.54 V, (b) 0.61 V. Here, as per common sense, power consumption increases with temperature. In contrast, the blue line shows the results with TEI-VS applied, where, surprisingly, power consumption decreases with increasing temperature. This is because the reduction in power consumption through voltage scaling down with TEI-VS is greater than the increase induced by rising temperatures. In addition, this phenomenon is observed across the entire operating temperature range, with the change being more pronounced at lower temperatures. More specifically, as the temperature increases, as shown in the figure, the rate of decrease tends to diminish, becoming negligible around 80 °C. Consequently, by applying TEI-VS in the range from −40 °C to 80 °C, temperature is transformed from a negative to a positive factor in terms of power consumption.

This discovery has the potential to intelligently liberate 3D-ICs, which traditionally suffer from heat treatment issues. In other words, the heat that is generated can be utilized to reduce the power consumption of dies operating at ULV by applying TEI-VS.

However, to actualize this idea, we need to devise several practical solutions. Firstly, a specific plan for the resolution of voltage scaling control must be established. This is due to the limited resolution of the DC-DC converter responsible for voltage scaling within the circuit, as typically, DC-DC converters can adjust voltage levels discretely in tens to hundreds of millivolts. In other words, temperature-based voltage scaling like that shown in Figure 2 is challenging to implement in reality, and instead, temperatures must be set according to the voltage levels provided by the DC-DC converter. To elaborate on our proposed method, if we define the set of effective output voltage levels of the DC-DC converter as $V_s=V_{s,i}|1\le i\le M$, we can derive $T_{th,i}$ corresponding to each $V_{s,i}$ through the following equation:

$$V_{s,i}=V_{TEI‐VS}\left(T_{th,i}\right).$$

(6)

Thus, we can establish a set of controllable points, S, composed of feasible temperature values and corresponding voltage levels for TEI-VS application, as follows:

$$S=\left\{(T_{th,i},V_{s,i})\right|T_{min}\le i\le T_{max}\}$$

(7)

Finally, for a given die temperature $T_{die}$, $V_{TEI‐VS}$ is determined by finding $min\left(k\right)$ in S satisfying $T_{s,k}-T_{die}\ge 0$, and the paired $V_{s,k}$ becomes the target $V_{TEI‐VS}\left(T_{die}\right)$.

Next, for the practical application of TEI-VS in 3D-ICs, designing a cooling controller with minimal overshooting is crucial. As previously discussed, when finding k and setting the die temperature to $T_{s,k}$ for a given $T_{die}$ using a cooling controller, any overshooting $T_{overshoot}$ during control means the die’s $V_{dd}$ must be $V_{TEI‐VS}(T_{die}-T_{overshoot})$ instead of $V_{s,k}$. This is because maintaining $V_{dd}=V_{s,k}$ would prevent the circuit from meeting the target frequency, leading to potentially critical chip errors. Furthermore, it is crucial to note that since we cannot predict $T_{overshoot}$ at runtime, we typically cannot perform voltage scaling until the system has stably converged to the set temperature (i.e., $T_{s,k}$). As a result, excessive $T_{overshoot}$ in the cooling controller makes voltage downscaling infeasible until stabilization, significantly diminishing the benefits of power reduction and energy savings. Therefore, to fully exploit the power/energy-saving potential of TEI-VS in 3D-ICs, it is necessary to develop a cooling controller that quickly converges to the set temperature while causing minimal overshooting.

3. Deep Learning-Based Autotuning PID Controller for the 3D-IC Cooling System

PID control is a method that maintains the output of a controlled object at a desired target value using three operations: proportional, integral, and differential. PID control has the advantages of being simple and easy to understand, having excellent stability and robustness, and being applicable to a wide range of systems [31,32,33]. However, inappropriate setting of PID gains may lead to excessive overshooting [34]. Overshooting, a phenomenon where the output exceeds the target value, occurs when the proportional gain $K_p$ is too high or the integration time $T_i$ is too short. As discussed earlier, excessive overshooting in the cooling system controller of 3D-ICs necessitates providing a larger voltage margin. This requirement means that to meet the target speed of the chip, a higher voltage than the lowest level resulting from TEI-VS must be supplied. Given the difficulty in predicting the extent of overshooting at runtime, the most stable operation choice inevitably becomes using the nominal voltage instead of the result from TEI-VS. Consequently, this leads to an unavoidable decrease in energy efficiency. To tackle this issue, the crux of our controller design is to set appropriate proportional, integral, derivative, and antiwindup gains, $K_p$, $K_i$, $K_d$, and $K_{aw}$, respectively, so as to minimize the overshooting.

In our situation, characterized by the heterogeneous configuration of 3D-IC dies and the diverse interface materials used in packaging, obtaining comprehensive information about the controlled object is challenging. This complexity makes it difficult to determine the ideal values of the four key parameters: $K_p$, $K_i$, $K_d$, and $K_{aw}$. While various autotuning methods, such as the Ziegler–Nichols method [35], damped oscillation method [36], and Cohen–Coon method [37], exist for automatically calculating PID parameters, their effectiveness can significantly diminish in scenarios where the plant characteristics frequently change or external noise is present. The controller proposed in this paper leverages deep learning to surmount the limitations of these existing autotuning techniques, thereby enabling robust control across various plants in 3D-IC.

3.1. Control Model

This section is dedicated to explaining traditional PID control through the adoption of the FOPDT model [28,38]. The FOPDT model, commonly used in cooling and heating systems, is represented by its transfer function, which can be expressed as follows:

$$G\left(s\right)=\frac{K_p}{\tau s+1}{e}^{-\theta s},$$

(8)

where $\tau$ and $\theta$ are the time constant and delay, respectively. Then, the standard form of PID control can be expressed as follows:

$$u=K_{p}e\left(t\right)+\frac{K_p}{T_i}{\int }_0^{t}e\left(t\right)dt+K_p{T}_d\frac{de\left(t\right)}{dt},$$

(9)

where u is the controller input and $T_i$, $T_d$, and $e\left(t\right)$ are the integration time, differentiation time, and difference between the temperature set point and the actual temperature in heating and cooling systems (error), respectively. Using $T_i$ and $T_d$, we can derive $k_i=\frac{k_p}{T_i}$ and $k_d=k_p\times T_d$.

Applying antiwindup to reduce overshooting in the standard form of the PID control in (9) allows the formula to be expressed as follows:

$$\begin{array}{c}\hfill u=K_{p}e\left(t\right)+{\int }_0^t\frac{K_p}{T_i}e\left(\tau \right)+K_{aw}(u_{sat}\left(\tau \right)-u\left(\tau \right))d\tau +k_p{T}_d\frac{de\left(t\right)}{dt}\end{array},$$

(10)

where $u_{sat}$ can be expressed as:

$$u_{sat}=\left\{\begin{array}{ccc}\hfill u& \mathrm{f}\mathrm{o}\mathrm{r}& u_{min}<u<u_{max}\hfill \\ \hfill u_{max}& \mathrm{f}\mathrm{o}\mathrm{r}& u>u_{max}\hfill \\ \hfill u_{min}& \mathrm{f}\mathrm{o}\mathrm{r}& u<u_{min}\hfill \end{array}\right.$$

(11)

In (11), $u_{max}$ and $u_{min}$ are the maximum and minimum values of the control input, respectively.

Meanwhile, in our PID control, we employed a digital first-order low-pass filter (LPF) to filter out high-frequency noise and to reduce the impact of unnecessary sudden changes in the signal. Accordingly, the final control model is as follows:

$$\begin{array}{c}\hfill u=K_{p}e\left(t\right)+{\int }_0^t\frac{K_p}{T_i}e\left(\tau \right)+K_{aw}(u_{sat}\left(\tau \right)-u\left(\tau \right))d\tau +k_p{T}_d\frac{d{e}_f\left(t\right)}{dt}\end{array},$$

(12)

where $e_f\left(t\right)$ is $e\left(t\right)$ filtered.

3.2. Proposed Deep Learning-Based Autotuning Method

We propose a deep learning-based autotuning process for precise temperature control while minimizing overshooting. Our designed process is composed of a two-phase approach: the first phase involves a coarse-grain autotuning using conventional methods, and the second phase is a fine-grain autotuning process that sets the PID gain based on deep learning.

In the first coarse-grain autotuning phase, as shown in Figure 4a, the user sets the set point, which is the desired temperature and the maximum and minimum margins around this set temperature. Then, as depicted in Figure 4b, the output of the cooler (or heater) is set to on (100%) or off (0%), and the response of the target die within the 3D-IC is observed. Based on the results of this observation, the PID gain determined by the initial autotuning can be set as follows:

$$\begin{array}{c}\hfill K_p=0.35\frac{4}{\pi y_{diff}},T_i=0.5T_{osc},T_d=0.125T_{osc},\begin{array}{c}\hfill K_{aw}=\frac{1}{\sqrt{T_i·T_d}}\end{array},\end{array}$$

(13)

where $y_{diff}$ and $T_{osc}$ means $y_{max}-y_{min}$ and $\frac{(T_{osc1}+T_{osc2})}{2}$, respectively; $y_{max}$ and $y_{min}$ are the maximum and minimum values of the control response, respectively; $T_{osc}$ represents the period of oscillation, as also illustrated in Figure 4a; and $K_{aw}$ can be derived from [39].

Following the first autotuning phase, the second step of fine-grain autotuning is executed based on the PID gains that were set. This second tuning phase employs deep learning. For the learning process, PID control is carried out with variations in the set point (i.e., the target temperature), relying on the PID gains established by the first autotuning and the output from the neural network. The employed learning algorithm is a reinforcement learning model, based on an error-driven approach. This involves using the temperature difference between the plant’s response under PID control and the set point temperature (i.e., the error) as the input for the neural network. As previously mentioned, minimizing overshooting when the target temperature is lower than the current temperature is critical. Hence, the set point is systematically lowered from a higher temperature at regular intervals, enabling the neural network to adequately learn about and respond to overshooting scenarios. The learning process and its components are depicted in a block diagram in Figure 5.

The detailed structure of the neural network is depicted in the block diagram in Figure 6. The neural network consists of fully connected layers with a configuration of $8\times 200\times 400\times 400\times 3$, and its outputs are $T_d$, $T_i$, and $K_{aw}$. Since the neural network’s output is directly utilized as the input for the PID controller, a negative parameter value could lead to divergence in the controller’s output. It is crucial to recognize that the neural network includes a switching layer, which ensures that the outputs, $T_d$, $T_i$, and $K_{aw}$, do not become negative. The switching layer is a recursive layer where the output is fed back as input. If the output of the $n^{th}$ input in the switching layer is negative, then the output of the ${(n-1)}^{th}$ input is used as the $n^{th}$ output again. The output at the switching layer is determined independently for each parameter.

To facilitate the training of the neural network in a direction that reduces overshooting in the PID controller, we incorporated an overshooting term into the loss function for weight updates. We defined the loss function of the neural network as follows:

$$\begin{array}{c}\hfill L=\frac{1}{2}{(y_{sp}-y_{output})}^2+\left(overshooting\right),\end{array}$$

(14)

$$overshooting=\left\{\begin{array}{ccc}\hfill 0& \mathrm{f}\mathrm{o}\mathrm{r}& y_{output}>y_{sp}\hfill \\ \hfill e^{(y_{output}-y_{sp})}& \mathrm{f}\mathrm{o}\mathrm{r}& y_{output}<y_{sp}\hfill \end{array}\right.,$$

(15)

where $y_{sp}$ and $y_{output}$ are the set point (i.e., target temperature) and the actual output inside the plant through the plant’s control input, respectively.

In addition, we adopted the Adam (adaptive moment estimation) algorithm [40] as the optimizer for its widely recognized performance benefits. Adam combines the strengths of momentum and RMSprop optimization methods and is known for its computational efficiency and robustness, becoming a popular choice for deep learning applications. To minimize time overhead in the second-phase tuning process, we set the learning rate to 1, leveraging Adam’s ability to adaptively adjust learning rates based on the magnitude of gradients.

3.3. Validation of the Proposed Method with a Basic 3D-IC Thermal Modeling

We planned a two-step experimental process—simulations in a basic thermal model and simulations in a complex 3D-IC model—to verify the proper functioning of the proposed autotuning controller and assess its improvement over traditional methods. First, this section describes the initial simulations in the basic thermal model. In the basic model, the heat source is located at the center of the unit volume modeling the 3D-IC, with the surrounding dies acting as the receivers of heat. Additionally, we omit the interface material between each die, assuming a uniform thermal conductivity for all dies. As illustrated in Figure 7, the heat energy flowing into the system per unit time, ${\dot{E}}_{in}$, and the heat energy released from the system per unit time, ${\dot{E}}_{out}$, are each defined within a three-dimensional Cartesian coordinate system as follows:

$${\dot{E}}_{in}=q_x+q_y+q_z,$$

(16)

$${\dot{E}}_{out}=q_{x+dx}+q_{y+dy}+q_{z+dz},$$

(17)

where $q_{x+dx}$, $q_{y+dy}$, and $q_{z+dz}$ are defined, respectively, as follows:

$$q_{x+dx}=q_x+\frac{\partial q_x}{\partial x},q_{x+dy}=q_y+\frac{\partial q_y}{\partial x},q_{x+dz}=q_z+\frac{\partial q_z}{\partial z}.$$

(18)

Using the definition provided in (17) and (18) can be expressed as follows:

$${\dot{E}}_{out}=(q_x+\frac{\partial q_x}{\partial x}dx)+(q_y+\frac{\partial q_y}{\partial y}dy)+(q_z+\frac{\partial q_z}{\partial z}dz).$$

(19)

Regarding the energy generated in unit volume per unit time, ${\dot{E}}_g$, it can be expressed as follows:

$${\dot{E}}_g=\dot{q}dxdydz.$$

(20)

Moreover, in the time-dependent system, the energy difference within the volume, ${\dot{E}}_{st}$, can be expressed as:

$${\dot{E}}_{st}=\rho c_p\frac{\partial T}{\partial t}dxdydz,$$

(21)

where $\rho$ and $c_p$ are the density and specific heats at constant pressure, respectively.

Applying (16)–(21) to the law of conservation of energy, it can be expressed as follows:

$${\dot{E}}_{in}-{\dot{E}}_{out}+{\dot{E}}_g={\dot{E}}_{st},$$

(22)

$$(-\frac{\partial q_x}{\partial x}dx)+(-\frac{\partial q_y}{\partial y}dy)+(-\frac{\partial q_z}{\partial z}dz)+\dot{q}dxdydz=\rho c_p\frac{\partial T}{\partial t}dxdydz$$

(23)

Meanwhile, by applying Fourier’s law of heat conduction, the terms $q_x$, $q_y$, and $q_z$ can be articulated as follows:

$$q_x=-k\left(dydz\right)\frac{dT}{dx},q_y=-k\left(dzdx\right)\frac{dT}{dy},q_z=-k\left(dxdy\right)\frac{dT}{dz},$$

(24)

where k is the thermal conductivity. Substituting (24) into (23), we can derive the following result:

$$\frac{\partial }{\partial x}(k\frac{\partial T}{\partial x})+\frac{\partial }{\partial x}(k\frac{\partial T}{\partial x})+\frac{\partial }{\partial x}(k\frac{\partial T}{\partial x})+\dot{q}=\rho c_p\frac{\partial T}{\partial t}.$$

(25)

Assuming that the thermal conductivity of each die in the 3D-IC is the same, and since the heat conduction equation is being obtained at the specific coordinates of the die that does not generate heat, (25) can be reformulated as follows, by disregarding the boundary condition:

$$k{\nabla }^{2}T=\rho c_p\frac{\partial T}{\partial t}.$$

(26)

Differentiating both sides of (26) with time and assuming that this is the heat conduction equation at specific coordinates, the equation can be expressed as follows:

$$\alpha \frac{dT(x_0,y_0,z_0,t)}{dt}=\frac{d^{2}T(x_0,y_0,z_0,t)}{d{t}^2},$$

(27)

where $\alpha$ ($\alpha =\frac{k}{\rho c_p}$) is the heat capacity. Based on differential Equation (27), if there is no output from the cooler and only heat generation by the core is assumed, the temperature at a specific coordinate of the die will exhibit an exponential change over time.

We validated the functionality of the controller using the derived thermal model. In the FOPDT model transfer function (8), the values of each parameter used in the simulation were set as $k_p=-10$, $\tau =0.5$, and $\theta =0.1$. The simulation scenario was as follows: the core temperature was initially set to 50 °C, and the temperature at a specific coordinate of the die was also assumed to be in a steady state converging to 50 °C. For the second autotuning, the set point was reduced by 10 °C. It was assumed that after the second-phase (deep learning-based) autotuning was completed and the PID gain was set, the core thermally ran away, causing both the core and die temperatures to increase to 130 °C. Subsequently, we simulated a scenario where the set point was lowered to 80 °C using the set PID gain.

The simulation results are shown in Figure 8: Figure 8a displays the outcome of implementing only the coarse-grain autotuning, while Figure 8b illustrates the results after applying our proposed two-phase autotuning, which includes both coarse-grain autotuning followed by deep learning-based fine-grain autotuning. The control parameters for each case are reported in

Table 1. Control parameters for the conventional autotuning and the proposed autotuning used in Figure 8.

	${\mathit{K}}_{\mathit{p}}$	${\mathit{T}}_{\mathit{i}}$	${\mathit{T}}_{\mathit{d}}$	${\mathit{K}}_{\mathit{aw}}$
Conv.	−0.0181	0.3244	1.2975	1.5414
Proposed	−0.0181	0.0451	0.0098	0.0261

. As depicted, in both cases, the temperature converges to the set point of 80 °C under the control of the proposed autotuning controller. However, in Figure 8a, an overshooting by up to 11 °C occurs relative to the set point, whereas in Figure 8b where the second tuning is applied, the overshooting is significantly reduced to within 1 °C. Furthermore, the time taken to stabilize at the set 80 °C is substantially shorter in the latter case.

Meanwhile, to demonstrate that PID autotuning shows superior performance compared to traditional static PID control, we designed a static PID controller and conducted experiments in the same environment as the previous ones. The results are presented in Figure 9.

Since the static PID controller does not have autotuning, users must manually input the control parameters. Therefore, we arbitrarily set the parameters and performed simulations as shown in Figure 9a,b. While the static PID controller does not require the setup time demanded by autotuning, and appropriate user-selected parameters can yield good results as shown in Figure 9a, inputting suboptimal parameters can lead to a significant overshooting and increased stabilization time, as illustrated in Figure 9b.

Finally, based on the results of the simulation, it is evident that our proposed method is the most energy-efficient, maximizing the TEI-VS effect. In the case of applying TEI-VS, for the former scenario, TEI-VS cannot be applied for a significant duration (about 415 s) until the plant’s temperature stabilizes at 80 °C, thus requiring the supply of the nominal supply voltage without any voltage scaling down. In contrast, for the latter scenario, as the system stabilizes quickly within 140 s, TEI-VS can be immediately applied thereafter to reduce power consumption. Based on the fact established in Section II that ULV circuits with TEI-VS exhibit more energy-efficient characteristics at higher temperatures, these results validate our proposed two-phase autotuning solution as a technology capable of maximizing energy efficiency.

4. Experimental Result

In this section, we report the temperature control performance of the proposed controller on a realistic 3D-IC and the energy efficiency improvements achieved through the application of the TEI-VS technique. The simulations conducted for evaluation were a cosimulation of the thermal simulation of the 3D-IC and the proposed controller simulation. We modeled a 3D-IC and extracted temperature data according to the RPM values of the fan using the HotSpot tool. Subsequently, heat control simulations were performed using the obtained heat trace data.

Figure 10a displays the 3D-IC model used in the simulations. It includes crucial components like a heatsink and heat spreader, along with detailed information on the TSVs and thermal interface material (TIM). In this structure, based on our discovery that the core die operating under ULV conditions becomes more energy-efficient at higher temperatures due to the effects of TEI-VS, we positioned the target die in the center of the 3D structure where it experienced relatively higher temperatures. The target die comprised the Alpha 21264/EV6 microprocessor core [41] provided by the HotSpot simulator [29], and the thermal map of this die obtained from the simulation is depicted in Figure 10b.

For the verification of the functional validity and effectiveness of the proposed AI-based PID autotuning controller, we performed a cosimulation combining the thermal simulation of the 3D-IC and the proposed controller. Initially, we conducted a steady-state simulation of the 3D-IC using key parameters outlined in

Table 2. List of parameters applied to 3D-IC’s thermal simulation.

Parameter	Value (C)	Parameter	Value $\left(\mathit{m}\right)$
Initial temp.	80.0	Fan radius	0.02
Ambient temp.	40.0	Motor radius	0.01
Thermal threshold	120.0

. The heat capacity and conductance of various materials, including silicon, and the specifications for heatsinks and heat spreaders, were adopted from the default values provided by HotSpot. In the context of TEI-VS, due to the power/area overhead of the multiple on-chip DC-DC converters, in general, one DC-DC converter is equipped per die, so that voltage scaling is only feasible at the level of each die. Therefore, the reference temperature of $T_{die}$ should be the lowest temperature for each die, as TEI-VS performs a lower voltage scaling at higher temperatures. Accordingly, we performed a thermal transient simulation for 100 s based on the lowest temperature block within the die, as shown in Figure 10b. The transient simulation results for each RPM value of the fan are displayed in Figure 11. At 100 s, the temperature of the target block reached 92.9 °C and 74.1 °C at fan speeds of 1 RPM and 1000 RPM, respectively. Based on these results, we mapped the minimum and maximum outputs of the proposed controller to the effective RPM range of the fan.

Subsequently, we performed a control simulation using the heat trace data of the target block. The control simulation using the new thermal model was conducted in the same manner as in Section 3.3 to ensure a fair evaluation. Additionally, all target temperatures used in both the learning phase of the neural network and the control phase were set identically.

Figure 12 displays the results of the control simulation using a realistic 3D-IC thermal model: Figure 12a shows the results for the conventional coarse autotuning, while Figure 12b presents the outcomes of the proposed two-phase autotuning, which includes coarse-grain autotuning followed by deep learning-based fine-grain autotuning. The control parameters for each are reported in

Table 3. Control parameters for the conventional autotuning and the proposed autotuning used in Figure 12.

	${\mathit{K}}_{\mathit{p}}$	${\mathit{T}}_{\mathit{i}}$	${\mathit{T}}_{\mathit{d}}$	${\mathit{K}}_{\mathit{aw}}$
Conv.	−0.0743	43.8125	10.9525	0.0457
Proposed	−0.0743	0.0288	0.0063	0.0217

. Compared to the results in Figure 8, the tendency for overshooting was different with conventional autotuning. Although it appeared as a slight improvement, the control simulation was conducted fairly with only the thermal model differing, suggesting a high dependence of the conventional controller on the thermal model, indicating a limitation. In contrast, the proposed controller adaptively learned from different thermal models via the neural network, showing simulation results with minimal overshooting, similar to previous simulations. These results validate the robust control capability of the proposed controller in varying thermal models of 3D-ICs.

Finally, we discuss the energy savings achievable by applying TEI-VS to 3D-ICs using the proposed autotuning technique instead of conventional methods. In the simulation, power values were derived from measurements obtained from our chip developed using a 28 nm process (refer to Figure 3), and the maximum operating temperature was set to 80 °C. As depicted in Figure 12, the time taken to stabilize at the set temperature point of 80 °C, $t_{settling}$, was 556.86 s for conventional autotuning and 190.36 s for our proposed method. Considering that TEI-VS could not be applied during $t_{settling}$, the faster stabilization of the proposed method led to more energy-efficient results. In other words, the conventional method could not utilize TEI-VS during the $t_{settling}$ of 556.86 s, requiring a 0.54 V supply at the 50 MHz operating frequency and a 0.61 V supply at the 100 MHz operating frequency, whereas the proposed method can operate at 80 °C TEI-VS results of 0.44 V supply at 50 MHz operating frequency and 0.49 V supply at 100 MHz operating frequency after 190.36 s.

Table 4. Energy saving results based on the settling time of the conventional method, when the target temperature was set to 80 C.

	Clock Frequency: 50 MHz	Clock Frequency: 100 MHz
$E_{Conv.}$ (J)	1.430	3.151
$E_{Proposed}$ (J)	1.134	2.455
Energy saving (%)	20.70	22.09

reports the energy consumption results based on these differences. The proposed controller achieved 20.7% and 22.1% energy savings at 50 MHz and 100 MHz, respectively, compared to the conventional controller.

Additionally, to demonstrate that our proposed controller still performs well and achieves excellent power consumption reduction at different set points, we also conducted simulations with the set temperature point at 60 °C. The results are depicted in Figure 13, with each figure representing (a) the conventional method, and (b) our proposed method. The control parameter values for the conventional method were the same as in Table 3, while for our proposed method, $K_p=-0.0743$, $T_i=0.0302$, $T_d=0.0002$, and $K_{aw}=0.0108$. Compared to the results in Figure 12, while the $t_{settling}$ increased to achieve a lower set temperature, our proposed method’s $t_{settling}$ of 255 s was still significantly faster than the 638 s of the conventional method. The energy savings from this experiment are reported in

Table 5. Energy saving results based on the settling time of the conventional method, when the target temperature is set to 60 °C.

	Clock Frequency: 50 MHz	Clock Frequency: 100 MHz
$E_{Conv.}$ (J)	1.571	3.491
$E_{Proposed}$ (J)	1.334	2.921
Energy saving (%)	17.77	16.33

. The proposed controller showed remarkable savings, achieving 17.8% and 16.3% reductions in energy consumption at 50 MHz and 100 MHz, respectively, when compared to the conventional controller.

5. Conclusions

The 3D-IC is garnering significant attention from academia and industry as a key technology leading the post-Moore era, offering new levels of efficiency, power, performance, and form-factor advantages to the semiconductor industry. We have successfully challenged the traditional approach to thermal management in 3D-ICs, which typically views heat generation negatively and focuses on its elimination. Instead, our novel approach utilizes this heat positively to achieve low-power operation in 3D-ICs. Central to our method is the innovative use of the TEI phenomenon in ULV operating circuits, a key feature of the TEI-VS technique. This approach allows for a reduction in power consumption as temperatures rise, overturning conventional understandings of thermal effects in 3D-ICs that includes plants with ULV operation. We not only introduced new thermal management technologies suitable for practical implementation in 3D-ICs but also developed a critical component for achieving optimal energy efficiency: a deep learning-based PID autotuning temperature controller. The effectiveness and theoretical soundness of this AI control algorithm were thoroughly validated through extensive simulations, demonstrating both its functional accuracy and its ability to enhance power efficiency.

Our work represents a significant stride towards more efficient and sustainable semiconductor technologies. By embracing and utilizing the inherent thermal properties of 3D-ICs, we pave the way for more innovative and energy-efficient solutions in semiconductor design and operation. Moreover, our solution concretely demonstrates that optimization can be achieved through AI-based algorithms and control, clearly showing that semiconductor design and operation can be a prime application area for the improvement of AI algorithms and systems.