Approximate Global Energy Management Based on Macro–Micro Mixed Traffic Model for Plug-in Hybrid Electric Vehicles

Abstract

A multi-scale physical process management system is presented in this paper, taking the plug-in hybrid electric vehicle system as the physical interface connecting the macro traffic system to the micro energy conversion process, with the ultimate goal of global energy management in the full temporal–spatial domain for autonomous plug-in hybrid electric vehicles. This novel method adopts a macro traffic flow model at a large time scale, in which only the initial conditions and the traffic information of key road sections are required, and a car following model at the micro scale. Furthermore, local replanning of energy management is carried out by adjusting the power threshold and the efficiency weight through the type of reinforcement learning that is closest to human learning, once a short term speed disturbance is induced by unknown disturbances in the macro traffic flow. Due to the nonlinear relationship between speed fluctuation and power fluctuation, it is necessary to map the vehicle speed and acceleration characteristics to the power characteristics, instead of directly utilizing the traffic model characterized by the speed and acceleration characteristics. The results show that novel multi-scale physical management can achieve a smaller deviation from the global optimal solution and enhanced robustness of global energy management. Additionally, close coupling between the dynamic characteristics of vehicle components and speed fluctuation ensures correct tracking of the optimized target value.

1. Introduction

The cyber hierarchy and interactional network (CHAIN) concept-based full-lifespan management method provides a new idea for carbon neutralization and carbon peaking, which enables integrated solutions at both the macro application scenario and micro mechanism levels [1]. At the micro level corresponding to the vehicle itself, electric propulsion systems as secondary energy sources are added to conventional internal combustion engine (ICE)-based powertrain systems to form hybrid electric vehicles (HEVs) or plug-in hybrid electric vehicles (PHEVs) [2,3]. In these hybrid powertrains, the secondary energy sources can assist ICEs to always run in the high efficiency area and can improve the dynamic performance of the vehicles [4,5,6,7,8]. Consequently, benefiting from such powertrain configurations, dependence on fossil fuels is reduced and thus less exhaust gas will be emitted, which will lead to a cleaner transportation system.

However, how to achieve an approximate global solution, as well as to provide sufficient energy according to changeable external power requirements, remains a great challenge for HEVs/PHEVs [9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]. One of the most important reasons for this is the uncertainty of macroscopic physical boundary conditions, i.e., future driving behavior according to traffic environment judgments [27,28,29,30,31,32,33,34].

Based on the CHAIN concept, solutions can be sought from a new perspective at the macro level, i.e., using the macro continuous traffic flow method, which takes the transportation system as a whole. Macro flow models use the average value (e.g., the spatial average speed of continuous vehicles) in sampling, in which the most suitable q–k or u–k relationship is established by statistical regression. Thus, it is more suitable for large-scale driving behavior prediction than microscopic car following models, which have too many unknown factors and require a large amount of calculation. The results of macro flow models are certainly more stable, since the average parameter value is estimated by the average sample, which is not only easier for the collection of data, but also more consistent with the basic statistical assumptions in model calibration. Spiliopoulou et al. calibrate two well-known macroscopic traffic flow models, i.e., the first-order model CTM (Cell Transmission Model) and the second-order model METANET (Modéle d’Écoulement de Trafic sur Autoroute NETworks), based on various real traffic data, by three optimization algorithms: the deterministic Nelder–Mead algorithm, a stochastic genetic algorithm and the stochastic cross-entropy method, respectively. These resulting models are validated and able to converge in robust model parameter sets, albeit achieving different levels of accuracy [35]. Poole et al. calibrate a macroscopic traffic flow model by determining the density and speed equation parameters with fundamental diagram parameter variation. Through the model calibration, using data from three different days for a motorway section, it is shown that properly selected data are capable of capturing the underlying dynamics of the traffic flow process [36]. In the future, automatic vehicles can be controlled by a servo-mechanism, and driving behavior can be intervened upon and optimized, which can reduce or correct unreasonable driving behavior. At the same time, this can improve the predictability, orderliness and regularity of the system, and reduce random, uncertain driving behavior, which further improves the practical value of the macro traffic model.

In reality, driving behavior is sometimes in an unsteady state (i.e., acceleration and deceleration) and sometimes in a steady state (i.e., constant speed). The former state changes greatly, and it is difficult to accurately describe unsteady behavior in car following, even with a high-order macro model. Instead, the car following theory at the micro level is suitable for the description of car following behavior in the unsteady state, in which rules concerning how a driver follows his immediate leading vehicle are established. The car following theory attempts to describe driving behavior in detail at the micro level and is directly related to the driver’s safety, providing a solution to overcome the defects of the macro traffic flow [37,38,39,40,41]. Many classic car following models have been proposed, including the intelligent driver model (IDM), the Gazis–Herman–Rothery (GHR) model, the full velocity difference (FVD) model, etc., which have been widely explored and have made great contributions to traffic theory.

Although macroscopic models (i.e., continuum methods) excel at large-scale traffic simulation with aggregated behavior, agent-based traffic simulations (i.e., microscopic models) excel at the modeling of individual vehicles with inhomogeneous governing behaviors, so Sewall et al. [42] combined these two types of approaches and proposed a hybrid method to take advantage of these complementary features. Their approach simulates traffic in the areas of interest using an agent-based model, while the rest areas use a continuum model. On adjoining regions of a road network, vehicles passing from one regime to another must be converted to the representation used in the destination regime. They introduce flux capacitors to convert continuum flow to discrete agents, and use car averaging to handle discrete vehicles that flow into continuum regions. Hybrid models can combine mesoscopic models with microscopic or macroscopic models, as well as mixed-class and multi-class models. More valuable results can be obtained by combining ideas from different branches, as well as combining the advantages of different types of model [43].

Micro traffic models represented by car following models can be applied to the development of a vehicle’s autonomous intelligent cruise control system (AICCS) [44] and autonomous cruise control system (ACCS) [45,46], etc., to ensure more efficient and stable movement of connected automatic control vehicles.

However, to the best of our knowledge, few studies have considered the collaborative optimization control of the physical process, from the macro large-scale traffic system to the vehicle level of the micro subsystem. The purpose of this paper is to propose a CHAIN concept energy management strategy for hybrid electric vehicles under large scale macroscopic boundary conditions, which enables seamless and smooth combination from macro to micro.

A global energy planning method from the perspective of multi-scale mixed traffic management is proposed in this paper, where a macro traffic flow model is adopted at a large time scale and a corresponding car following model is used at a micro scale. In addition, energy trajectory replanning is carried out, when the expected driving behavior details at the micro scale change due to increase of uncertainty or information entropy. This novel method is suitable for system management from the macro traffic scenario to micro mechanism levels, taking the intelligent transportation system as the object, e.g., global energy management in the full temporal–spatial domain for plug-in hybrid electric vehicles.

2. Macro–Micro Physical Processes of Traffic Management

2.1. Multi-Scale Mixed Traffic Modeling

As a physical interface between the macro physical environment and the micro physical process, the vehicle provides a way for macro physical boundaries to control micro physical processes cooperatively. In this paper, a multi-scale mixed traffic-based model for predictive control of plug-in hybrid electric vehicles is developed for the collaborative optimization control from the macro physical boundary to the micro physical boundary, during the process of state change of the physical interface described by PHEV/HEV vehicles. In this method, a macro traffic flow model is adopted at the large time scale and a corresponding car following model is used at the micro scale.

Figure 1 and Figure 2 show the schematic of the multi-scale physical process management system. According to the future driving behavior predicted by the physical model from macro traffic flow to micro car following behavior, the energy distribution of the motor and engine belonging to the micro physical process can be optimized for plug-in hybrid electric vehicles. Moreover, when the actual speed is greatly different from the predicted speed in advance (e.g., due to a change in the traffic environment), energy distribution is locally replanned.

The specific treatment process for the multi-scale physical processes management system is as follows:

Step 1: Establishing the multi-scale traffic model

A multi-scale traffic model is used to predict the future driving behavior for optimizing energy distribution of plug-in hybrid electric vehicles. In this method, a macro traffic flow model is adopted to predict the future driving behavior at a long time scale, and a corresponding car following model is used at a short time scale.

(a).

The macro traffic flow model derived from a micro car following model

In order to ensure the smooth transition between the macro traffic flow model and the micro car following model in time scale, the selected car following model needs to have a one-to-one correspondence in a steady state with the macro flow model. In order to achieve this goal, the macro model was derived from the micro car following model.

Future driving behavior at the short time scale can be calculated by a micro car following model of full velocity difference form, as follows [47]:

$$\frac{\mathrm{d}{v}_n}{dt}=k(V(\Delta x_n)-v_n)+\lambda \Delta v_n$$

(1)

This model considers the effects both of the distance and the relative speed of two successive vehicles, and many classical car-following models (e.g., the OVM) are special cases of this model.

In order to deduce the macroscopic traffic model, the microscopic variables of the car-following model can be transferred to the macroscopic variables as follows:

$$\begin{array}{l}{v}_{n+1}\left(\mathrm{t}\right)=u(x,t),\\ v_n\left(\mathrm{t}\right)=u(x+\Delta x,t),\\ V(\Delta x)=u_e(k(x,t)),\\ k=1/T,\\ \lambda =1/\tau \end{array}$$

(2)

where T is the relaxation time and $\tau$ is the time needed for the backward propagated disturbance to travel a distance of $\Delta x$.

Unlike other researchers, we added the physical driving resistance term $F_{\mathrm{physical}}$, including rolling resistance, slope resistance, air resistance and acceleration resistance, into the car-following model (Equation (1)) to consider its impact on driving behavior, which can be rewritten as:

$$\begin{array}{ll}\frac{\mathrm{d}u(x,t)}{\mathrm{d}t}& =\frac{u_e(k)-u(x,t)}{T}+\frac{u(x+\Delta x,t)-u(x,t)}{\tau }+F_{\mathrm{physical}}\\ & =\frac{u_e(k)-u(x,t)}{T}+\frac{\Delta x\frac{\partial u(x,t)}{\partial x}+o(\Delta x)}{\tau }+F_{\mathrm{physical}}\\ & \approx \frac{u_e(k)-u(x,t)}{T}+\frac{\Delta x}{\tau }\frac{\partial u(x,t)}{\partial x}+F_{\mathrm{physical}}\end{array}$$

(3)

The motion equation for the high-order macroscopic traffic flow can be written as

$$\frac{\mathrm{d}u(x,t)}{\mathrm{d}t}=\frac{\partial u(x,t)}{\partial t}+u\frac{\partial u(x,t)}{\partial x}=\frac{u_e(k)-u(x,t)}{T}+\frac{\Delta x}{\tau }\frac{\partial u(x,t)}{\partial x}+F_{\mathrm{physical}}$$

(4)

$$u_e=V\left(\frac{1}{k}\right)=6.75+7.91tanh\left(0.13/k-1.57\right)$$

(5)

where $v_n$, $\Delta x_n=x_{n-1}-x_n$ are the nth vehicle’s velocity and headway at time t ($x_n$ is the nth vehicle’s position), k is the reaction time, and V (·) is the optimal velocity function, respectively.

The conservation equation for the macro continuum traffic flow models is in the following form:

$$\frac{\partial k}{\partial t}+\frac{\partial \left(ku\right)}{\partial x}=q$$

(6)

where q is the generation rate and is set to 0 for simplification.

The macroscopic traffic flow model consists of Equations (4) and (6).

(b).

The multi-scale mixed traffic model

In the multi-scale mixed traffic model proposed in this paper, the macro traffic flow model (Equations (4) and (6)) is used to calculate driving behavior for the long time scale, while the micro car following model (Equation (1)) is used for the short time scale. Therefore, the multi-scale mixed traffic flow model is in the following form:

Micro model of short time scale

$$\frac{\mathrm{d}{v}_n}{dt}=k(V(\Delta x_n)-v_n)+\lambda \Delta v_n+F_{\mathrm{physical}}$$

(7)

Macro model of long time scale

$$\begin{array}{l}\frac{\partial k}{\partial t}+\frac{\partial \left(ku\right)}{\partial x}=q\\ \frac{\partial u(x,t)}{\partial t}+u\frac{\partial u(x,t)}{\partial x}=\frac{V(1/k)-u(x,t)}{T}+\frac{\Delta x}{\tau }\frac{\partial u(x,t)}{\partial x}+F_{\mathrm{physical}}\end{array}$$

(8)

Step 2: Establishing the vehicle dynamics model with the acceleration and velocity obtained in step 1 as the boundary conditions. With this model, the feasible working point set of the engine, motor and transmission system, which determine microphysical processes, can be obtained corresponding to the driving behavior in the prediction horizon of Step 1.

The structure of the PHEV vehicle used in this paper is shown in Figure 3. The vehicle in this project has a single axle hybrid system, and thus the engine and motor have the same speed. The main parameters of the vehicle are shown in

Table 1. Main parameters of PHEVs.

Engine	Maximum torque 100 Nm
Motor/generator	Maximum torque ± 100 Nm
Battery Transmission Final drive	Ternary Li, capacity: 60 A h 138 V CVT, 0.5–2.5, Torque converter Ratio 8

.

With the acceleration and velocity obtained in Step 1, the required torque T_CVT and velocity ratio i_CVT of CVT transmission can be obtained according to the following formula:

$${\eta }_T\cdot i_{CVT}\cdot i_d\cdot T_{CVT}+T_b=\frac{d{v}_n}{dt}+\left[mg{f}_r\cos \theta +\frac{1}{2}{C}_D\cdot \rho \cdot A\cdot v_{{}_n}^2+mg\sin \theta \right]\cdot \frac{1}{\delta \cdot m\cdot r}$$

(9)

where ${\eta }_T$ is the transmission efficiency; $i_{CVT},i_d$ are the numerical ratios of the CVT and the final drive, respectively; $T_{CVT},T_b$ are the torque of CVT and brake system; m is the vehicle weight; $f_r$ is the rolling resistance coefficient; $\theta$ is the road slope angle; $\delta$ is the coefficient of rotational mass; $\rho$ is the air density; $C_D$ is the aerodynamic drag coefficient; $A$ is the vehicle’s frontal area; $r$ is the wheel radius; and g is the gravitational acceleration.

The pump velocity n_HTC and pump torque T_HTC of a hydraulic torque converter can be obtained by K-factor and C-factor [48]:

$$\begin{array}{l}{K}_{factor}=f_K\left(\frac{n_{CVT}}{n_{HTC}},\frac{T_{CVT}}{T_{HTC}}\right)\\ C_{factor}=f_C\left(\frac{n_{CVT}}{n_{HTC}},\frac{T_{CVT}}{T_{HTC}}\right)\end{array}$$

(10)

The torque of the hydraulic torque converter is the coupling torque of the engine and the motor:

$$T_{HTC}=T_{\mathrm{e}}{\eta }_e+T_m{\eta }_m$$

(11)

The engine efficiency, motor/generator efficiency and fuel consumption rate are obtained through the bench test:

$$\begin{gathered} {\eta }_e=f_{e,\eta }(T_e,{\omega }_e) \\ {\eta }_{\mathrm{m}}=f_{m,\eta }(T_m,{\omega }_m) \\ {b}_e=f_{\mathrm{b}}(T_e,{\omega }_e) \end{gathered}$$

(12)

where $T_e,{\omega }_e$ are the engine’s torque and velocity; and $T_m,{\omega }_m$ are the motor’s torque and velocity.

The fuel consumption can be calculated by the following formula based on the bench test data:

$$\dot{m}=T_e\cdot {\omega }_e\cdot b_e$$

(13)

The motor/generator power and the SOC changes are obtained as follows:

$$P_{EM}=\{\begin{array}{l}\frac{T_m{\omega }_m}{{\eta }_{mot}},T_m>0\\ T_m{\omega }_m{\eta }_{gen},T_m\le 0\end{array}$$

(14)

$$V_{oc}(t)=d_{0}SOC(t)+d_1$$

(15)

$$\stackrel{•}{SOC}=\frac{dSOC}{dt}=-\frac{V_{oc}-\sqrt{V_{oc}^2-4R_{bat}\frac{P_{EM}}{{\mathrm{n}}_{bat}}}}{2R_{bat}{Q}_{bat}}\in [I_{\min },I_{\max }]$$

(16)

where $V_{oc},R_{bat},Q_{bat},I$ are the battery pack’s open circuit voltage, internal resistance, capacity and electric current respectively; ${\eta }_{mot},{\eta }_{gen}$ are the motor’s driving efficiency and generation efficiency; and $d_0,d_1$ are constants.

Step 3: Rolling optimization of the feasible working point set of engine, motor and transmission system obtained in Step 2 for every time step, which dominate the micro process, including the chemical reaction of the engine and the electrochemical reaction of the battery. The state quantity x and control quantity u are defined as:

$$x=\left[\begin{array}{l}SOC\\ v_{\mathrm{n}}\end{array}\right]$$

(17)

$$u=\left[\begin{array}{l}{T}_e\\ T_m\\ T_b\\ i_{CVT}\end{array}\right]$$

(18)

Considering the practicability of the energy management strategy, the constraints of state quantity and control quantity are as follows:

$$\begin{array}{l}{T}_{e,\min }\left({\omega }_e\right)\le T_e\left({\omega }_e\right)\le T_{e,\max }\left({\omega }_e\right)\\ T_{m,\min }\left({\omega }_m\right)\le T_m\left({\omega }_m\right)\le T_{m,}{}_{\max }\left({\omega }_m\right)\\ g_{CVT}\left(k-1\right)-2\le g_{CVT}\left(k\right)\le g_{CVT}\left(k-1\right)+1\\ \Delta T_{e,}{}_{\min }\left({\omega }_e\right)\le \Delta T_e\left({\omega }_e\right)\le \Delta T_{e,}{}_{\max }\left({\omega }_e\right)\\ {\dot{T}}_{e,\min }\le {\dot{T}}_e\le {\dot{T}}_{e,\max }\end{array}$$

(19)

where g_CVT is gear position and $\Delta T_e$ is the engine torque fluctuation.

The control variables are optimized within the above constraints, and the cost function of the multi-scale physical processes management is defined as:

$$J={\displaystyle {\int }_{t_0}^{t_f}\left\{L_e\left(\frac{1}{{\eta }_e}\right)+L_m\left[SOC\left(\tau \right)-SO{C}_r\left(\tau \right)\right]+L_u\left({\dot{T}}_e\right)\right\}}d\tau$$

(20)

where $[t_0,t_f]$ is the time domain for prediction, and $L_e,L_m,L_u$ are the weight coefficients for engine efficiency, the target SOC_r tracking and engine torque fluctuation.

The reference SOC_r is calculated according to the vehicle power:

$$SO{C}_r=SO{C}_{ini}-\left(K\cdot P_v\right)\cdot \left[SO{C}_{ini}-SO{C}_L\right]$$

(21)

where SOC_ini, SOC_L are the initial SOC and the minimum SOC, K is the proportionality factor, and $P_v$ is the vehicle’s demand power.

In order to extend the high-efficiency operating range of the engine, the engine efficiency weight L_e was improved through reinforcement learning (RL), which has the ability to learn anthropomorphic behavior:

$$\begin{array}{c}{L}_e={\lambda }_{L_e}\cdot L_e\\ P_e^{\prime }={\lambda }_{P_e^{\prime }}\cdot P_e^{\prime }\\ Q(\left[\frac{P_v}{P_{hi}},\frac{P_v}{P_{lo}}\right],\left[{\lambda }_{L_e},{\lambda }_{P_e^{\prime }}\right])=AC(\left[\frac{P_v}{P_{hi}},\frac{P_v}{P_{lo}}\right],\left[{\lambda }_{L_e},{\lambda }_{P_e^{\prime }}\right])+\\ {\lambda }_{RL}\cdot \underset{{\left[{\lambda }_{L_e},{\lambda }_{P_e^{\prime }}\right]}_{\mathrm{next}}}{\max }\left\{Q({\left[\frac{P_v}{P_{hi}},\frac{P_v}{P_{lo}}\right]}_{\mathrm{next}},{\left[{\lambda }_{L_e},{\lambda }_{P_e^{\prime }}\right]}_{\mathrm{next}})\right\}\end{array}$$

(22)

where $P_{\mathrm{hi}}$ and $P_{lo}$ are the high and low threshold of power, respectively, and $P_e^{\prime }$ is the reduced engine power when it is too low. ${\lambda }_{L_e},{\lambda }_{P_e^{\prime }}$ represent the corrections to $L_e$ and $P_e^{\prime }$ respectively. For simplification, it is assumed that $P_{\mathrm{hi}}$ = $P_{lo}$. Q() represents the learned knowledge and experience, and $AC\left(\right)$ represents the accumulated consumption of fuel and electricity. The subscript “next” represents the value of the next state. ${\lambda }_{RL}$ represents the learning rate of the reinforcement learning.

Step 4: Return the real-time state of the vehicle to the multi-scale traffic model for the next time step.

2.2. Energy Distribution Trajectory Replanning

Since the macro traffic model cannot describe the driving behavior in sufficient detail, especially when the traffic environment changes, the actual driving behavior may be inconsistent with the planned driving behavior in advance. Thus, it requires local replanning for integrated driving behavior for macro traffic planning in real time.

When the driving behavior from the micro model at the short time scale conflicts with that from the macro model at the long time scale, the power threshold $P_{\mathrm{hi}}$, $P_{lo}$ and the weight $L_e$ are adjusted accordingly. Specifically, the power threshold $P_{\mathrm{hi}}$, $P_{lo}$ and $L_e$ are adjusted via the RL machine learning method as follows:

$$\begin{array}{l}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{L}_e={\lambda }_{plan}\cdot {\lambda }_{L_e}\cdot L_e\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{P}_{hi}={\lambda }_{hi}\cdot P_{hi}\\ Q(\frac{P_v}{P_{plan}},\left[{\lambda }_{plan},{\lambda }_{hi}\right])=AC(\frac{P_v}{P_{plan}},\left[{\lambda }_{plan},{\lambda }_{hi}\right])+\\ {\lambda }_{RL}\cdot \underset{{\left[{\lambda }_{plan},{\lambda }_{hi}\right]}_{\mathrm{next}}}{\max }\left\{Q({\left[\frac{P_v}{P_{plan}}\right]}_{\mathrm{next}},{\left[{\lambda }_{plan},{\lambda }_{hi}\right]}_{\mathrm{next}})\right\}\end{array}$$

(23)

where $P_{plan}$ indicates the required vehicle power during planning, and ${\lambda }_{plan},{\lambda }_{hi}$ indicate the corrections to $L_e$ and $P_{\mathrm{hi}}$, respectively.

In this paper, it is assumed that two typical types of short term velocity disturbance exist at a certain time. Two typical velocity disturbances are studied, respectively, i.e., the acceleration disturbance (Mode 1) and the deceleration disturbance (Mode 2) in Figure 4. The velocity disturbance is assumed to change sinusoidally with time in the early stage [49] but exponentially in the later period.

In the closed-loop tracking strategy, the motor current is controlled to track the vehicle speed, and the engine throttle to track the torque. In energy management replanning, only the speed (not the torque) of the motor/engine is adjusted, which can weaken the coupling effect between torque and speed control.

According to the method proposed in this article, the cross-sectional flow and density information can be obtained through V2X measurement. From this, macroscopic spatio–temporal traffic flow information corresponding to the microscopic vehicle operation can be obtained for global speed planning. Then, each vehicle automatically travels and obeys the global speed planning mentioned above. When there is a conflict between global planning and real-time traffic scenarios, short-term local replanning is carried out. This can achieve a vehicle’s autonomous cruise control based on the micro model, and meet the overall global planning of the macro traffic model.

3. Numerical and Experimental Results

To verify the effectiveness of the global energy management for the plug-in hybrid vehicles based on a multi-scale hybrid traffic model, we tested the PHEV powertrain on a vehicle in the loop (VIL) testing system, as shown in Figure 5. A rapid prototype of a vehicle controller has been established on the dSPACE real-time simulation platform, which is used to send engine and motor load signals through the CAN bus. The engine and motor systems can be dynamically controlled independently or jointly, with great flexibility and reliability. All vehicle subsystems are connected to the CAN network. A high-performance parallel server computer was used to obtain target control sequences for engines, motors, and gearboxes under the multi-scale framework. The priority of motor speed is higher than that of other control variables, including engine torque, CVT speed, and motor torque, etc.

Numerical and experimental studies have been carried out to analysis the multi-scale energy management of PHEVs during the starting and braking processes in this paper, which are widely existing traffic situations. In all cases, the initial SOC is set to 60%, and the other initial conditions are as follows:

$$k(x,0)=\{\begin{array}{l}0.13,x\le 10000 \mathrm{m}\\ 0.04,x>10000 \mathrm{m}\end{array}$$

(24)

$$u(x,0)=V(k(x,0))$$

(25)

The boundary conditions are as follows:

$$k(0,t)=0.13$$

(26)

$$\frac{\partial k(x_{end},t)}{\partial t}=0,t\le 1000 \mathrm{s}$$

(27)

$$k(x_{end},t)=0.04,t>1000 \mathrm{s}$$

(28)

$$u(x_0,t)=0$$

(29)

$$u(x_{end},t)=V(k(x_{end},t))$$

(30)

where $x_0=0 \mathrm{m}$ and $x_{end}=80000 \mathrm{m}$ are the starting and ending coordinates of the road section, respectively. The first two cars in the lane queue are studied in this paper, i.e., the leading car and the following car, since the behavior of the leading car determines that of all subsequent vehicles. The driving behavior of the following vehicle is used to analyze novel global energy management based on multi-scale traffic model, while the driving behavior of the leading car is obtained according to the macroscopic traffic flow model alone.

According to the macroscopic traffic flow model, the distribution of speed and density in the spatiotemporal domain can be obtained, as shown in Figure 6. It can be seen that, from a global perspective, the speed variation and the density distribution in the whole road section can be obtained by the macro traffic flow model. For this method, only the initial conditions and the traffic flow information of key road sections are needed, and thus it is easier to collect data and more consistent with the basic statistical assumptions in model calibration. The vehicle speed for any time and space can be obtained through this method, which makes it global energy planning possible. This method, with faster runtime, is quite suitable for the large transportation domain where a large number of data sets and calculation resources are needed. In contrast, using the car following model to calculate the driving behavior of each vehicle in the whole space–time domain requires too many computing resources and too much time.

The traffic system can be design by an understanding of a servomechanism when the vehicle of the future is to be automatic as well as automobile. Automated vehicles can achieve servo-control to intervene and optimize driving behavior through AICCS and ACCS, thereby reducing improper driving behavior or correcting unreasonable driving behavior. Simultaneously, unlike human driving behavior, it has less randomness and can improve the predictability, orderliness and regularity of the system. In addition, with the support of vehicle-to-everything (V2X) communication [50], it is possible to predict macro traffic flow and induce vehicle driving in practical application scenarios.

The prediction of driving behavior in the future can be used for energy management from the global perspective. Longer time scale prediction of driving behavior has the advantage of achieving longer time scale energy optimization. The method in this paper makes full use of the macro scale advantages of the macro traffic flow model and provides the possibility of long-distance and large-scale traffic energy management specific to each (hybrid) vehicle. Whether it is near or far away from the charging station, one can make a reasonable overall plan in advance. This can improve the overall efficiency of the engine while ensuring reasonable charging and discharging (e.g., SOC not too low) of the battery. Furthermore, the overall optimization control, from the overall system point of view, can ensure the smooth overall density of macro traffic flow and reduce the fluctuation and corresponding dissipation of the traffic system.

The disordered physical processes at the micro scale will present a certain order at the macro scale. A large number of driving behaviors in the whole transportation system are obtained by the car following model at the micro scale level, which show disorder characteristics and thus make it difficult to achieve global optimization. However, the order of the macro scale level obtained by the macro traffic flow model enables the overall optimization of the traffic system. Furthermore, by extending the prediction time of the micro car following behavior, it can become closer to the global prediction. The longer the prediction time domain, the closer the global optimal control achieved. In this paper, by adopting a unified model both at the macro level and the micro level, the micro driving behavior follows the macro traffic flow state smoothly, which avoids disturbance dissipation in the transmission process from macro variable to micro variable.

For multi-scale physical processes management in this paper, the macro traffic flow model is used to realize large-scale planning, and the car following model at the vehicle level is adopted to describe specific driving behavior in detail. In this hierarchical control method, the micro car following behavior obeys the evolution process of the macro state of the traffic system. Figure 7 shows the car following behavior in macro traffic scenarios, where the driving behavior of the leading car is determined by the macro traffic flow model, but the rear car follows the leading car according to the micro car following model. The results show that the following car closely follows the leading car, indicating that the time scale of driving behavior obtained from the car following model is much smaller than that from the macro traffic flow model. This means that the time scale division of the multi scale model in Figure 1 is reasonable. In the initial process of smooth acceleration, the micro car following model can track the macro planning objectives well. However, in the later deceleration process, the difference between the micro car following and the macro traffic behavior is relatively significant. This is because the acceleration in the decelerating process is larger than that of the accelerating process, as well as the effect of faster speed variations, which is consistent with the actual vehicle characteristics and the physiological response characteristics of the driver. Improving the ordering of the macro system can reduce the disorder of its micro subsystems. For example, according to the overall fluctuation trend of macro traffic flow, the time series of micro driving behavior can be improved appropriately to reduce the fluctuations of instantaneous driving behavior.

The macroscopic traffic flow is suitable for long time scale prediction, while the microscopic car following model is suitable for short time scale prediction. Multi-scale physical processes management in this paper fully utilizes the advantages of the micro and macro methods. This can describe not only short-term car following behavior, but also long-term integrated driving behavior.

The macro flow models use the average value (e.g., the spatial average speed of continuous vehicles) in sampling, in which the most suitable q-k or u-k relationship is established by statistical regression. The result is certainly more stable, since the average parameter value is estimated by the average sample. However, since the macro traffic flow does not focus on specific individual vehicles, but on regional average vehicles, the actual driving behavior of each vehicle will be different from that of the regional average statistics. In addition, some unknown factors, or the imprecision of the models, will lead to differences between real driving behavior and predicted driving behavior for the macro traffic flow model. Therefore, when there is a short term speed disturbance in the whole macro traffic flow, replanning of local driving behavior and the corresponding energy management is carried out. Figure 8, Figure 9 and Figure 10 show the velocity evolution, velocity deviation component, and power evolution of the following car, when the car following behavior at the short time scale deviates from the driving behavior at the long time scale obtained by the macro traffic flow model, corresponding to Mode1 and the Mode2, respectively. The whole process is divided into three stages, i.e., the initial acceleration stage, the middle stable cruising stage and the final braking stage. In order to study the influence of speed fluctuation on the multiscale energy distribution, speed disturbance is assumed in the middle cruising stage to avoid the complex effects of the starting and braking process. Figure 10 shows that the velocity disturbance causes obvious power fluctuation. It is assumed that the vehicles are in information interconnection via V2X communication, and thus the short term velocity fluctuation can be accurately predicted in time through the car following model. Therefore, the energy distribution can be replanned immediately when the disturbance occurs. Since the relationship between speed fluctuation and the power fluctuation is nonlinear, it is unreasonable to plan energy utilization based only on speed characteristics. In other words, the traffic model characterized by speed and acceleration cannot be directly applied simply to energy management planning. Reasonable energy planning needs to map the vehicle speed and acceleration characteristics to power characteristics, which directly determine energy efficiency and are suitable for optimal planning of energy utilization.

The novel model considers the impact of unknown disturbances on the planning, in order to reduce the deviation in the global optimal solution caused by the error for large macro planning, and thus the model has good robustness even when there are inevitable unknowns, such as unmodeled factors, or chaotic systems with sensitive initial values. Figure 11 and Figure 12 show the SOC trajectories planned by the original strategy and the replanning strategy for Mode 1 and Mode2, respectively, which determine the distribution of the battery energy and fuel energy.

In order to make a reasonable comparison, the initial and final SOC of the two disturbance modes are the same (see Figure 11 and Figure 12). Figure 13, Figure 14, Figure 15 and Figure 16 show the numerical and experimental results (i.e., the torque and speed of the engine and motor) of both the original and the replanning strategies for Mode 1 and Mode 2, respectively. The results show that the experimental results are in good agreement with the optimized target values.

In traffic planning, the traffic efficiency is closely related to the average speed of vehicles, which mainly involves a steady state or an equilibrium state. However, whether from the perspective of physical mechanism or data-based principal component analysis, the final energy consumption and emissions depend on the actual vehicle power, which mainly depends on the transient process away from equilibrium, so the replanning of instantaneous energy distribution needs to take the power factors into consideration [51].

Figure 17 and Figure 18 show the experimental results of the power distribution of the engine and motor under the original strategy and the replanning strategy for Mode 1. For convenience in distinguishing the original and the RL based replanning strategies, Figure 19 shows the difference between them for Mode 1. Figure 20 and Figure 21 show the experimental results of the power distribution of the engine and motor under the original strategy and the replanning strategy for Mode 2, and Figure 22 shows the difference between these two strategies for Mode 2.

Global energy optimization can be achieved via the short-to-long time scale prediction of the driving behavior. The original strategy for global optimization is to distribute the HEV energy according to the demand power of the vehicle on the road. When the vehicle’s demand power is relatively higher (Mode 1) than the original plan, the engine power should be appropriately increased in order to fully exploit the energy saving potential; while the vehicle’s demand power is relatively lower (Mode 2), the engine power should be appropriately reduced in order to reduce the operating range of high fuel consumption. Furthermore, in order to extend the high-efficiency operating range of the engine, the engine efficiency weight L_e is increased when the vehicle’s demand power is very high. When the vehicle’s demand power increases, the engine power increases compared with the original plan; while the vehicle’s demand power decreases, the engine power decreases compared with its original plan.

For two typical types of speed deviation (i.e., Mode 1 and Mode 2), corresponding replanning is carried out, respectively. When significant deviation of speed occurs, the power threshold $P_{\mathrm{hi}}$, $P_{lo}$ and the weight $L_e$ are adjusted through reinforcement learning methods. For Mode 1, the power threshold $P_{\mathrm{hi}}$, $P_{lo}$ and the weight $L_e$ are increased when the actual power is higher than the predicted power of the original plan; but for Mode 2 the power threshold $P_{\mathrm{hi}}$, $P_{lo}$ and the weight $L_e$ are reduced when the actual power is lower than the predicted power of the original plan. This method further exploits the fuel saving potential of the engine at high power and reduces its inefficient operation at low power when the actual vehicle speed significantly deviates from the predicted speed of the original plan, not significantly affecting the original global optimization of the SOC trajectory.

Vehicle power has a nonlinear function with its speed, and the frequency and amplitude of the speed fluctuation directly affect that of power fluctuation. The fluctuation frequency is related to the response time of the engine and motor, and the fluctuation amplitude is related to the fuel consumption gradient and motor efficiency. When replanning, it is necessary to deeply couple the dynamic characteristics (e.g., time constant) of the engine and motor with the speed fluctuation, in order to ensure that the replanning strategy can be executed by the subassemblies.

For a clearer description of the results of reinforcement learning, the blue shaded area in Figure 13, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18 and Figure 20 and Figure 21 presents the replanning of the original strategy when the actual speed at the micro level deviates from that planned by the macro model, and the grey shaded areas A and B in Figure 19 and Figure 22 present the detailed difference for the engine operating area between the original strategy and the replanning strategy. The principle of re-optimization is to move the engine operation from the low efficiency area to the high efficiency area, that is, to replace area A with area B. The purpose of the replanning strategy is to convert the energy utilization of area A into that of area B, since the engine’s fuel consumption rate for area B is lower than that for area A. The fuel consumption for the original strategy and the replanning strategy for Mode 1 and Mode 2 are shown in

Table 2. Energy consumption of the sum of areas A and B for the original strategy and the replanning strategy.

Strategy	Accumulated Energy Consumption of Engine (KJ)	Tested Fuel Consumption (g)	Tested Average Brake Special Fuel Consumption (g/kwh)	Fuel Saving Percentage (%)
Original Mode 1	152	10.7	253	-
Modified Mode 1	152	9.6	228	10.0
Original Mode 2	142	10.3	260	-
Modified Mode 2	142	9.3	236	9.2

. Table 2 shows that the replanning strategy can reduce the fuel consumption by 9–10% compared with the original strategy, indicating that the replanning strategy can make further utilization of the engine’s fuel saving potential by improving the efficient operation of the engine at high power and reducing its inefficient operation at low power. The replanning strategy has strong adaptability and can make energy management close to global optimization, despite the deviation in actual driving behavior from the initial predicted value, which may be caused by inevitable errors of the macro traffic flow model and traffic environment changes. The novel PHEV self-driving, from the perspective of the full space–time chain, achieved the integrated management of transportation planning and energy utilization.

4. Conclusions

In this paper, multi-scale physical processes management is proposed to investigate the collaborative physical effect between the micro vehicle system and the macro road transportation system, which can contribute to constraints in planning methods for the global optimization of energy consumption for intelligent transportation system. This novel model adopts a macro traffic flow model at the large time scale and a car following model at the micro scale, which combines the advantages of both the micro and macro models. Furthermore, energy trajectory replanning is carried out through human-like reinforcement learning once the expected driving behavior at the micro scale conflicts with the traffic behavior from the macro traffic model due to increase in uncertainty. This method, taking the intelligent transportation system as the object, is suitable for system management from the macro traffic scenario to micro mechanism levels, e.g., global energy management in the full temporal–spatial domain for plug-in hybrid electric vehicles.

The experimental and numerical results show that:

(1)

Based on the initial conditions and the traffic information for key road sections, the speed and density distribution in the whole road section can be determined according to the macro traffic flow model, especially when vehicles of the future are to be automatic. Furthermore, the macro driving behavior in the whole road section provides the possibility for large-scale traffic energy management specific to each (hybrid) vehicle, and improvement in driving behavior to reduce the speed fluctuation and dissipation of the traffic system.

(2)

The novel model takes into account errors in large macro planning induced by unknown disturbances, which is achieved by RL based replanning of energy management when there is a short term speed disturbance in the macro traffic flow. Thus, it can reduce the deviation in the global optimal solution and has good robustness when there are inevitable unknowns.

(3)

Since the relationship between speed fluctuation and power fluctuation is nonlinear, the frequency and amplitude of the speed fluctuation directly affect that of power fluctuation. Thus, it is necessary to map the vehicle speed and acceleration characteristics power characteristics, which directly determine energy efficiency and are suitable for energy utilization planning, instead of directly utilizing the traffic model characterized by speed and acceleration characteristics. Additionally, the dynamic characteristics of engine and motor should be coupled deeply with speed fluctuation, since the fluctuation frequency of speed fluctuation is related to the response time of engine and motor, which guarantees that the target value can be tracked correctly by vehicle components, and its fluctuation amplitude to the fuel consumption gradient and motor efficiency.

(4)

By analyzing the results of the RL-based replanning scheme, the following conclusions can be drawn. When significant speed deviation occurs and the vehicle’s demand power is relatively higher (Mode 1) than the original plan, the engine power should be appropriately increased by adjusting the power threshold $P_{\mathrm{hi}}$, $P_{lo}$ and the weight $L_e$; while the vehicle’s demand power is relatively low (Mode 2), the engine power should be appropriately reduced in order to reduce the operating range for high fuel consumption. The replanning strategy can further exploit the fuel saving potential of the engine at high power and reduces its inefficient operation at low power when the actual vehicle speed significantly deviates from the predicted speed of the original plan, which can reduce at least 9–10% of fuel consumption compared with the original strategy.