Comparative Study of Dynamic Programming and Pontryagin’s Minimum Principle on Energy Management for a Parallel Hybrid Electric Vehicle

Abstract

This paper compares two optimal energy management methods for parallel hybrid electric vehicles using an Automatic Manual Transmission (AMT). A control-oriented model of the powertrain and vehicle dynamics is built first. The energy management is formulated as a typical optimal control problem to trade off the fuel consumption and gear shifting frequency under admissible constraints. The Dynamic Programming (DP) and Pontryagin’s Minimum Principle (PMP) are applied to obtain the optimal solutions. Tuning with the appropriate co-states, the PMP solution is found to be very close to that from DP. The solution for the gear shifting in PMP has an algebraic expression associated with the vehicular velocity and can be implemented more efficiently in the control algorithm. The computation time of PMP is significantly less than DP.

1. Introduction

Due to the co-operation flexibility introduced by the multiple energy storages or power sources, hybrid electric vehicles (HEVs) have the potential to reduce fuel consumption and emissions in comparison to the conventional vehicles. To ensure the vehicle remains drivable, the total power from the battery and engine should meet the driver’s power requirement at each instant. Beyond this point-wise constraint, there is still plenty of flexibility in manipulating the engine and battery power for various optimization purposes. The power distribution solutions obtained from the energy management strategy are typically solved by numerical or analytical optimization techniques.

The energy management strategy in HEVs consists of the decision for the power distribution among multiple power sources and the regulation of the power transmission, which results in the balancing among the different performance, including fuel economy, emissions and other possible cost [1,2]. Various hard and soft constraints, if applicable, have to be respected when the energy management strategy is designed. For instance, the state of charge (SOC) should be maintained within a certain range to mitigate battery degradation. The torque capacity of the engine and electric motor vary with their rotational speed or environmental temperature. And the engine and motor must be limited to the specific speeds to ensure safety and reliability. The energy management algorithm can be solved using the optimal control techniques since its objective is minimizing a performance index defined over a specific problem horizon. Technically, the desired output trajectory is known a priori when the vehicle follows a certain drive cycle. Dynamic programming (DP) [3,4,5] and the analytical optimal control techniques [6,7,8] can then be used to obtain the theoretical optimal results. The results obtained through DP are unbeatable but, unfortunately, is an optimal input but not a control algorithm, and thus is not suitable for real-time implementation. Consequently, a post-processing step, namely the rule extraction, is required, e.g., through Neural Networks, which approximates the results of the optimal control pattern. But it is impractical to obtain the control strategy to mime the optimal behaviors under the all driving conditions. Hence, the controller based on DP is effective only for the specific driving cycle that is used for the rule extraction. To remedy this problem, the stochastic dynamic programming (SDP) method [9,10,11] and the driving pattern detection within multiple driving cycles [12] had been suggested as the possible solutions. As a general case of the Euler-Lagrange equation, Pontryagin’s minimum principle (PMP) was also introduced to obtain optimal control solution for hybrid vehicles [13,14,15,16], where the Hamiltonian is considered as an analytical function. Based on the theoretical backgrounds of PMP, the equivalent consumption minimization strategy (ECMS) [17,18,19] was proposed to associate the electrical energy usage with future fuel consumption, and the equivalent fuel consumption is minimized at each time instant. Comparative studies of DP and PMP in the energy management of a series and power-split hybrid vehicle are presented in [20] and [21] respectively, in which normally one state variable, the battery’s SOC, and one control variable, the power distribution of the power sources, are involved. However, for both series and power split hybrid vehicles, there is no discrete transmission shift involved.

On the other hand, Automatic Manual Transmission (AMT) has been developed as a potential replacement of traditional automatic transmission with a torque converter, especially, on the parallel HEVs. In this paper, DP and PMP are applied to design the optimal control for a parallel hybrid vehicle with AMT. The powertrain is taken as the dynamic system with the two state variables, the battery’s SOC and the gear position of AMT, and two independent control variables, the engine throttle signal and gear shifting action. The gear is defined as a state in this optimal control problem because it relates directly to the torque of the engine and the shifting action should not be skipped in the control problem. The results of PMP are found to be near-optimal, close to those of DP and the co-state corresponding to the gear position state has an algebraic expression associated with the vehicular speed, which results in the global near-optimal behavior of PMP and gives the major contribution in this paper.

The remainder of this paper is organized as follows: in Section 2, the hybrid powertrain is modeled and the optimal control problem is formulated. DP and PMP-based methods are applied and results are analyzed in Section 3. The solutions from DP and PMP are compared in section 4 and the conclusions are provided in Section 5.

2. Hybrid Powertrain Modeling

The schematic diagram of the parallel hybrid powertrain and its power flows are shown in Figure 1, where P_req is the power request, P_b is the electric power from the battery pack and P_e is the mechanical power output from the diesel engine, respectively, ${\dot{m}}_f$ is fuel consumption rate, normally decided by the brake specific fuel consumption (BFSC) map derived through the bench test. The arrows of the lines in Figure 1 indicate the directions of the power flows. The torques of the engine and the electric motor are combined before the AMT. The 7.0 L diesel engine is adopted, giving 155 kW maximum power at the speed of 2000 rpm and 900 Nm maximum output torque within the speed range from 1300 rpm to 1600 rpm. The electric motor can output 90 kW maximum power, 600 Nm maximum torque and has 2400 rpm maximum speed. The 60 Ah lithium-ion battery pack gives 312.5 V rated voltage. The AMT is configured with nine ratios: 12.11, 8.08, 5.93, 4.42, 3.36, 2.41, 1.76, 1.3 and 1. The curb weight of the vehicle is 16,000 kg, the tire radius is 0.508 m, the final ratio is 4.769 and the fronted area is 6.2 m².

Given a driving cycle defined by the vehicle velocity history v(t), $t\in [t_0,t_f]$. The power request P_req(t) is calculated as in Equation (1):

(1)

Where m is the vehicle mass; δ is the mass factor, combining the moving and rotation inertial together; f is the coefficient of the rolling resistance; C_D is the aerodynamic coefficient; A is the fronted area; g is the acceleration of the gravity and α(t) is the gradeability.

At the same time, the power balancing should be maintained, as shown in Equation (2):

P_req(t) = (P_e(t) + P_b(t)η_m)η_T

(2)

Where η_m is the efficiency of the electric motor, decided by the motor speed and torque, normally derived from the test bench; and η_T is the efficiency of the transmission and axle, taken as the constant 0.9. Clearly the engine power P_e(t) can be regulated by the engine’s throttle signal th(t). It means that P_batt could be decided by th(t) at any time when P_req(t) is known in advance. Thus the engine throttle signal th(t) can be chosen as a control variable, ranging from 0 to 1.

Figure 1. The pre-transmission parallel hybrid electric powertrain.

Figure 1. The pre-transmission parallel hybrid electric powertrain.

The state charge of the battery, soc, relates to the battery current by:

$$s\dot{o}c=-\frac{1}{C_{batt}}I\left(t\right)$$

(3)

where C_batt is the battery capacity; I(t) is the battery current. The variation of soc is calculated by Equation (4):

(4)

where V_oc(soc) is the open circuit voltage; R_int(soc) is the internal resistance of the battery; R_t is the terminal resistance. It should be noted the explicit dependence of soc through open-circuit voltage V_oc and the internal resistance R_int. However, the assumption that the variation of soc depends only on P_batt is made reasonably when soc changes within a narrow range for the hybrid electric vehicles. V_oc(soc) and R_int(soc) can be obtain through the bench test, as shown in Figure 2, where soc stays in the primary usage range and thus V_oc and R_int can be assumed to be independent of soc.

Figure 2. V_oc and R_int varying as soc changes.

Figure 2. V_oc and R_int varying as soc changes.

The current gear position is defined as a state because the shifting strategy plays a key role for this vehicle and the skip of it is not expected. The variation of current gear position i_g(t), $i_{\mathrm{g}}\left(t\right)\in \{1,2,\dots ,9\}$ is associated with gear shifting action sh(t) and calculated by Equation (5):

$${\dot{i}}_g\left(t\right)=sh\left(t\right)$$

(5)

where sh(t) can be −1, 0, or 1, representing downshift, hold and upshift, respectively.

Over the entire optimization horizon [t₀, t_f], the system state $x\left(t\right)=\left[soc\right(t),i_{\mathrm{g}}(t\left)\right]\prime$ evolves according to Equation (6):

(6)

where $u\left(t\right)=\left[th\right(t),sh(t\left)\right]\prime$, f represents Equations (1–5). The inequality constraints for the state and control variables include:

P_b,min ≤ P_b(t) ≤ P_b,max

(7)

ω_e,min ≤ ω_e(t) ≤ ω_e,max

(8)

0 ≤ P_e(t) ≤ P_e,max

(9)

soc_min ≤ soc(t) ≤ soc_max

(10)

where P_b,min and P_b,max are the battery recharge and discharge power limitation; ω_e,min(t), ω_e(t) and ω_e,max(t) is the idling speed, normal rotational speed and maximum speed of the engine; P_e,max is the power limitation of the engine; The variable soc(t) is constrained within the permitted minimum value soc_min and the maximum value soc_max.

The cost function to be minimized is normally the compromise of the economy and other performance, here the accumulated fuel consumption and the gear shifting events are included, as shown in Equation (11):

(11)

where J is the cost metrics. The portion β|sh(t)| is introduced to avoid the excessive shifting and β is a positive weighting factor, here tuned to 0.01 to reach the equilibrium between the shift frequency and fuel consumption [22].

The state terminal conditions are shown in Equation (12):

$$x\left(t_0\right)=x\left(t_f\right)=[0.6,1]\prime$$

(12)

The terminal constraint for x(1) are imposed to ensure battery energy is sustained. The heavy-duty vehicle natural driving cycle used for the simulation is shown in Figure 3, which is obtained from a heavy garbage collecting vehicle under the normal operation full of the frequent stops and brakes.

Figure 3. Heavy-duty vehicle natural driving cycle.

Figure 3. Heavy-duty vehicle natural driving cycle.

3. Application of DP and PMP

3.1. The DP-Based Numerical Optimization

The DP technique solves the multi-step horizon optimization problem based on the Bellman’s principle of optimality and it guarantees the global optimality through exhaustive search of all control and state grids. The principle of optimality dictates that for the discrete system, if u(k) (k = 0, 1, 2, …, N−1) is the optimal control over the whole problem horizon, then the truncated sequence u(k) (k = s, s+1 ,…, N−1, 0 < s < N) is the optimal control for , which results in:

(13)

When x(k) and u(k) are discretized into the finite states and Equation (13) is solved backwards, the optimal control and corresponding cost are stored and then the optimal solution with the specific initial states is retrieved forwardly by applying the optimal controls through the horizon. However, the backward procedure means that the optimal solution can be obtained mostly offline because the driving cycle should be known in advance. The casual control algorithm approximating DP behavior should be pursued in the following practical controller design, which is not covered in the paper. The resolution of soc is 0.01 and the discrete time step is 1 second. The DP algorithm was implemented in MATLAB and applied to obtain the optimal control solution [22].

3.2. The PMP-Based Optimization

PMP [23] states that if the control law u^*(t) is optimal for Equation (11), the following conditions should be satisfied as a necessary but not sufficient condition.

• The u^*(t) minimizes the Hamiltonian H(x(t), u(t), t, p(t)) for all $t\in [t_0,t_f]$:

  (14)

  where the Hamiltonian is defined as:

  (15)

  With p(t) is a vector of the auxiliary variables called co-states and the dimension of p(t) is the same as x(t).
• The co-state p(t) satisfies the following equation:

  (16)
• The terminal condition is similar to Equation (12).

The conditions given by PMP are only necessary for optimality, but not sufficient. The solution that satisfies the necessary conditions is called the extremal solution. In general, the following two approaches are effective to check whether an extremal solution is sufficient for optimality: (1) The optimal trajectory obtained from PMP is the unique trajectory that satisfies the necessary and boundary conditions; (2) Some geometrical properties of the optimal field provide the possibility of optimality verification, such as the optimal field is convex [21]. In the practical applications, PMP can be used to find the candidate solution through computing and minimizing the Hamiltonian for all $t\in [t_0,t_f]$ and obtain the extremal controls.

The optimal control variables sh(t) and th(t) are obtained at each instant to minimize the Hamiltonian, expressed by Equation (17):

(17)

When Equation (6), Equation (15) and Equation (16) are associated, p₁(t) and p₂(t) are solved through Equation (18):

(18)

As the variation of soc depends only on P_b, then the assumption that the co-state p₁(t) corresponding to soc is a constant can be presented [21], which simplifies the computation tremendously. The co-state p₂(t) corresponding to the gear shifting is decided by the vehicle velocity and acceleration as in reality. The co-state p(t) can be solved when Equations (5), (6), (12) and (18) are associated, while enforcing the condition u^*(t) = arg min H to identify a solution candidate. Despite of being completely defined, the two-point boundary values problem can be solved numerically only using an iterative procedure because one boundary condition is defined at the terminal time. The procedure is known as the shooting method and consists of replacing the two-point boundary value problem with a conventional initial-condition problem [24]. An iterative procedure, as shown in Figure 4, has been used to obtain the final values of co-states, making the PMP significantly faster than dynamic programming. It should be noted that similar resolution for soc and the discrete time step as DP are used.

The value of co-state p₁(t) and algebraic expression of p₂(t) associated with vehicle velocity are shown in Equation (19):

(19)

Where γ is determined as 0.011 based on the multiple rounds of the explorations; v(k) is the vehicle speed at k step, when the whole problem is discretized. It should be noted that Equation (19) is dependent on the particular cycles. The value of p₁ and γ can be the different value when the cycle changes.

The existence and uniqueness of the solutions cannot be proved formally in the general case, but it is reasonable to assume that at least one optimal solution exists for the energy management problem in the sense that there is at least one sequence of controls giving the lowest possible fuel consumption. If the minimum principle generates only one extremal solution, it is the optimal solution. If there is more than one extremal solution, they are all compared and the one yields the lowest total cost is chosen.

Figure 4. The iterative procedure of PMP.

Figure 4. The iterative procedure of PMP.

4. Comparative Analysis for the Results from PMP and DP

The SOC curves from DP and PMP respectively are shown in Figure 5. The similar tendency of both curves is found, decreasing first, and then increasing steadily when the engine provides more power. During t = 1500~2000 s, the vehicle operates at the higher velocity, the engine and battery provide the power together and the value of SOC reaches the minimum valley, no matter in DP or PMP case. However, the solutions of DP and PMP have the minor differences.

The gear distribution in DP and PMP with the different power requirement and velocity range is shown in Figure 6. Due to the time-varying co-state p₂, which controls the gear shifting in PMP, the result in PMP is basically similar to that in DP. Additionally, the minor difference of the gear shifting is found in Figure 6, as highlighted in the black rectangle area. It can be expressed further in Figure 7. As shown in the green ellipse of Figure 7, the gear in PMP is slightly lower than that in DP, which causes that the operating points of the engine in PMP are mildly lower than those in DP, as noted by the black circles in Figure 8. The total fuel consumption in PMP is slightly more than that in DP can be explained by the operating points of engine and fuel consumption curves in Figure 9. Moreover, the difference of the SOC curves in Figure 5 before the minimum valley can be explained by that the power provided by engine in PMP is lower than that in the DP, as shown in Figure 8.

Figure 5. SOC curves of DP and PMP.

Figure 5. SOC curves of DP and PMP.

Figure 6. Gear distribution of DP and PMP. (a) Gear distribution of DP; (b) Gear distribution of PMP.

Figure 6. Gear distribution of DP and PMP. (a) Gear distribution of DP; (b) Gear distribution of PMP.

Figure 7. Gear shifting of DP and PMP.

Figure 7. Gear shifting of DP and PMP.

Figure 8. Work area of engine in DP and PMP. (a) Working zone of the engine in DP; (b) Working zone of the engine in PMP.

Figure 8. Work area of engine in DP and PMP. (a) Working zone of the engine in DP; (b) Working zone of the engine in PMP.

Consequently, it is significant to observe that when tuned with an appropriate co-state value, PMP can give a solution close to DP although it has been implemented with an instantaneous minimization process. The co-states p₁ and p₂ are compared with the analogue quantities extracted from DP in Figure 9 and Figure 10 respectively, according to Equation (20) [23]:

$$J_x^{*}\left(x^{*}\right(t),t)=p^{*}\left(t\right)$$

(20)

where J_x^*(x^*(t),t) is the partial derivative of the cost function J with the respect to x when the optimal trajectory x*(t) is adopted, p*(t) is the optimal co-state trajectory. It is found that co-states generated by the iterative procedure as shown in Figure 4 and Equation (19) approximate the optimal solution from DP effectively. It also explains the similar behavior of DP and PMP as shown in Figure 6, Figure 7, Figure 8. The fuel consumption in the PMP case is 5.37L, 0.4% more than that of 5.35L in DP case.

Figure 9. ${J^{*}}_{x_1}$ and p₁.

Figure 9. ${J^{*}}_{x_1}$ and p₁.

Figure 10. ${J^{*}}_{x_2}$ and p₂.

Figure 10. ${J^{*}}_{x_2}$ and p₂.

5. Conclusions

The DP and PMP techniques are used to obtain the energy management algorithm for a parallel HEV with AMT. The PMP was demonstrated to be effective in generating near-optimal results, close to those of DP, even though it has been implemented with an instantaneous minimization process. The constant co-state assumption associated with SOC doesn’t influence the gear position state in the results considerably. The co-state p₂(t) corresponding to the gear position state i_g(t) has an algebraic expression associated with the velocity, which results in the global optimal behavior of PMP and is the major contribution in this paper. Additionally, the total fuel consumption in the PMP case is close to that in DP. On the other hand, PMP can save the approximately 77% time of DP, including the iteration to approximate the co-states. Future research efforts will find another quadratic function of the co-state p₂(t) associated with the velocity and acceleration, and estimating the co-state p₂(t) online to adapt the gear position state to the driver’s style and actual drive cycle.