3.1. Lagrangian Solution
For any
and
, define the Lagrangian function of Problem 2
where
are called the Lagrangian multipliers or dual variables.
Rearranging some terms, it can be rewritten as
where
and
Based on the Lagrangian function, the KKT condition can be summarized as
Primal feasibility condition:
Complementary slackness condition:
Dual feasibility condition:
Stationary condition
:
where
.
Using the KKT condition, we establish a modified Riccati equation for solving the multi-objective problem in the following.
Proposition 1. Suppose that is fixed and arbitrary. Consider the Riccati equationfor all with , and define withwhere the superscript λ is included to designate the dependence on λ. Then, is a primal feasible point of Problem 2 uniquely satisfying the primal feasibility condition (3) and uniquely satisfies the stationary condition (7)–(10). Proof. Using Assumption 1,
implies that
is nonsingular, and consequently, (
9) implies
. Similarly,
with (
10) implies
for all
. On the other hand, (
7) and (
8) with the assumption that
for any
in Assumption 1 ensure
is nonsingular for all
. Therefore, the feedback gains are uniquely determined by
for all
. Plugging this expression into (
7) and (
8) leads to the construction in (
12) with the Riccati Equation (
11). Note that under Assumption 1 and fixed
, the KKT point is uniquely determined. □
If the inequality constraint is removed and
, then the Riccati equation in (
11) is reduced to the standard Riccati equation. In this case, it is clear that the solution that satisfies the KKT condition is unique. Therefore, the solution obtained from the Riccati equation is the unique optimal solution, which is a well-known fact.
Proposition 2. Consider Problem 2 without the inequality constraint. Then, is the unique optimal solution of Problem 2.
Proof. It is clear that the tuples uniquely satisfy the KKT condition. Since the KKT condition is a necessary condition for optimality, it is a unique optimal solution of Problem 2 without the inequality constraint. □
Proposition 1 tells us that the Riccati equation can be induced from the Lagrangian function and KKT condition in optimization theory instead of the classical argument from the value function and HJB equation. Moreover, we can see that the solution of the multi-objective LQG defined in Problem 1 is nothing but the solution of a standard LQG problem with modified weight and an appropriately chosen . Let us now focus on how to determine the Lagrange multiplier satisfying the KKT condition. We need to consider the following three scenarios:
If the strict inequality is already satisfied with obtained using the standard Riccati equation, then solves the complementary slackness condition. We do not need to do anything in this case.
Moreover, if the equality is satisfied with obtained using the standard Riccati equation, then any solves the complementary slackness condition. However, when , the corresponding may be different from . Therefore, to use the variables obtained in Proposition 1 as a solution to the KKT condition, we need to set .
Lastly, assume that holds with obtained using the standard Riccati equation. Then, some solves the complementary slackness condition if . Suppose that is such a number. Then, the corresponding tuple satisfies the KKT condition.
For simplicity of the presentation, we only focus on the last case because the other cases are trivial, and we formalize it in the following assumption.
Assumption 3 (Nontrivial scenario). Throughout the paper, we assume that holds with obtained using the standard Riccati equation.
To proceed further, we need to establish some properties of the function
defined as
which evaluates the error in the inequality constraint. In the following, we study various properties of
f, which play important roles throughout this paper.
Proposition 3 (Properties of
f)
. Define the function asThen, the following statements hold:- 1.
f is continuous over ;
- 2.
holds for any ;
- 3.
If holds for some , then ;
- 4.
;
- 5.
There exists a such that ;
- 6.
Define the set-valued mapping . Then, is a closed line segment.
Proof. From the definition, is linear in , is rational, whose entries are finite for a finite because the inverse matrix in is finite for all . Therefore, from the definition, is also rational and finite over , which implies that is continuous in . This completes the proof.
We only need to prove the inequality for
. By contradiction, suppose that
holds. For a fixed
, we see from the KKT condition that the problem is nothing but the optimization
with an augmented objective. Since
is the optimal solution corresponding to
, it follows that
where
is the optimal solution corresponding to
. On the other hand, we have
which leads to
Combining (
16) with (
14) yields
which contradicts with our hypothesis. This completes the proof.
Assume
holds for some
. Then, (
14) leads to
, while (
15) yields
. Combining the two inequalities leads to the desired conclusion.
The fourth statement is true due to Assumption 3.
Note that the objective in (
13) can be replaced with
without changing the optimal solutions. As
, the objective converges to
, which implies
as
due to the strict feasibility assumption in Assumption 2. Since
f is continuous over
from the first statement, there should exists
such that
.
Define and . From the continuity of f, the supremum and infimum are attained; otherwise, f should be discontinuous. Therefore, we can define and . From the second statement, we see that for all . It completes the proof.
□
Proposition 3 suggests that f is monotonically decreasing over the non-negative real numbers. Moreover, we can choose a such that . Let . Then, is a monotonically decreasing over , which connects and . The graph of f is illustrated in the following example.
Example 2. Let us consider Example 1 again. With and , the function value is plotted in Figure 1, demonstrating the monotonically non-increasing property and the zero crossing property in Proposition 3. Based on Proposition 3, we can characterize the set of the KKT points. In particular, it turns out that the KKT point corresponding to should satisfy the constraint ·
Proposition 4. The set of variables satisfying the KKT condition is all tuples such that and .
Proof. If
, the KKT condition is obviously not satisfied because the complementary slackness condition (
5) is not satisfied with
. If
, then the complementary slackness condition is not satisfied with
. Only the case that KKT is satisfied with
is the case that
holds. This completes the proof. □
Proposition 4 gives us a clue on how to decide the KKT point. However, since the KKT condition is a necessary condition for the optimality, there is no guarantee that a KKT point found is actually an optimal solution. Fortunately, we can prove that all the KKT points characterized in Proposition 4 constitute the optimal solutions.
Proposition 5. Consider any tuples such that and . The set of such tuples is formally defined asThen, the corresponding with is an optimal solution of the constrained LQG problem in Problem 2. Proof. From Proposition 4, we conclude that is the set of all KKT points. Therefore, there exists at least one such that the corresponding is an optimal solution of the constrained LQG problem in Problem 2. From statement 3) of Proposition 3, other elements in have the same objective function value . Therefore, for all , the corresponding is an optimal solution of the constrained LQG problem in Problem 2. This completes the proof. □
Proposition 5 tells us that if we can find a root satisfying , then we can find an optimal solution of Problem 2. Therefore, the problem is reduced to finding a root of . Our next goal is to develop a simple algorithm to solve the multi-objective LQG problem.
3.3. Suboptimality
Once an approximate is found from Algorithm 1, the corresponding solution can be easily found. The solution obtained by Algorithm 1 is -accurate in terms of , while it does not guarantee the -accuracy in terms of the objective or other variables induced from the -accuracy of , which depend on their sensitivities in . From the structures of or , we can conclude that if is -accurate, then is -accurate, i.e., for some function such that as . The function depends on the system parameters such as , , and N.
Due to the finite precision error in the bisection search, it is hard to satisfy the equality
or
exactly. Assume that
is
-accurate, i.e.,
. This implies
Then, such
is the dual variable such that
is satisfied, and the corresponding tuple
satisfies the KKT condition with
. We can conclude that
is an optimal solution of Problem 2 with
replaced with
.
Proposition 6. Suppose that given , is ρ-accurate, i.e., , and defineThen, for the corresponding tuple , is an optimal solution of Problem 2 with γ replaced with . Proof. The corresponding tuple satisfies the KKT condition with the complement slackness condition replaced with . The proof is concluded using Proposition 5. □
Proposition 6 suggests that the solution obtained by using Algorithm 1 is a suboptimal solution of Problem 2 with replaced with .
3.4. Computational Efficiency
The number of variables in the problem is upper bounded by
. If we use an SDP to solve the multi-objective problem using interior point algorithms, the time complexity is known to be upper bounded by
to obtain an
-accurate solution [
33]. Therefore, the computational time may explode cubically as
. On the other hand, the bisection line search is known to find an
-accurate solution within the number of iterations bounded by
, where
is the initial bracket size. The time complexity of the proposed algorithm per iteration is
. Therefore, the overall time complexity is bounded by
, which is linear in
N. Assuming that both notions of the
-accuracy is reasonably compatible for fair comparisons, the proposed bisection algorithm may perform much faster than the interior-point algorithms, especially when
N is large, which is the case in most applications. In particular, when the model predictive control is applied, where Problem 2 is solved at every iteration, the proposed scheme could play an important role. The compatibility of the
-accuracy of both approaches is hard to address within the scope of this paper. We will provide a numerical comparative analysis at the end of this paper to demonstrate the efficiency of the algorithm.
3.5. Deterministic Cases
The previous results assume that the noises are stochastic and the covariance matrices V and W are positive definite. However, it does not cover important applications where some variables are deterministic or fixed. To cover more practical cases, we will extend the results to the generic case that some elements of the noise vectors are deterministic. In this case, should be relaxed to . Note that if , then the KKT point in Proposition 1 uniquely satisfies the KKT condition for any fixed . For the deterministic case or , it still satisfies the KKT condition, while it may not be a unique solution. Therefore, we cannot preserve the optimality arguments in Proposition 5. Fortunately, the previous results hold in this case under mild assumptions.
Proposition 7. Suppose that the second condition, , in Assumption 1 is relaxed to . Assume that f is a bijection for the given . Consider any tuples such that and . Then, the corresponding is an optimal solution of the constrained LQG problem in Problem 2.
Proof. We first prove the continuity of
as a function of
V and
W, and denote them by
. First of all, suppose that
is fixed. Then,
and
do not depend on
V and
W, and hence are continuous as functions of
. Moreover,
depends on
linearly, and thus, is continuous in
. Similarly, so are
and
as functions of
. Now,
is also continuous as a function of
and
. Consider the set-valued mapping
. If
f is bijective for
and
, then the output of
T is singleton, and
is the point on the graph of
, which crosses zero because the Lagrange multiplier
, which solves the KKT condition as a root of
. Therefore, from the continuity of
on
and
, we can prove that
is also continuous as follows. First of all, note that by the continuity of
on
,
is a bijection for all
around
. In the sequel, assume that
always lies inside such a set. Note also that
and
is continuous in
y by the continuity of
in
. We will show that for any
, there exists
such that
implies
. To proceed, let us define
and
. By the continuity of
in
V and
W, for any
, there exists
such that
implies
. Moreover, by the continuity of
in
y, for any
, there exists
such that
implies
. With
, we have
Therefore, this proves the continuity of
in
V and
W. Hence,
is continuous as a function of
V and
W. Note that
is also a function of
. As the next step, consider a sequence
such that
and
so that
as
and
for all
. Then, the corresponding
is the unique optimal solution corresponding to
and
, where
is the Lagrange multiplier corresponding to
and
. From the continuity of the KKT point in
, we can arrive at the desired conclusion. □
3.6. Example
In this section, we will provide a simple example to illustrate the validity and efficiency of the proposed approach. The numerical example was treated with the help of MATLAB R2020a. The SDPs were solved with SeDuMi [
34] and Yalmip [
35]. Let us consider Example 1 again with the following setting:
Running the proposed algorithm with the initial search interval
and the accuracy
leads to
with the elapsed time
seconds. With the obtained control policy, the evolution of the indoor temperature
(
C), reference temperature
(
C), input
(
C), and the histograms of the objective function and constraint cost are depicted in
Figure 2. The histogram has been obtained over 3000 samples, and the empirical average of the constraint cost is 25,129, which meets the inequality constraint approximately.
Running the proposed algorithm with the same setting except for
= 10,000 leads to
. The corresponding simulation results are given in
Figure 3. The empirical average of the constraint cost from samples in histogram is 9956, which meets the inequality constraint approximately.
A semidefinite programming problem (SDP) for solving the same problem, Problem 2 is given by
with
, which can be readily obtained by modifying the results in [
29]. The histograms of the elapsed times of the proposed algorithm and the above SDP problem to solve the building problem are shown in
Figure 4 over 30 samples.
The average elapsed times are s and s for the proposed method and the SDP problem, respectively. The result demonstrates that the proposed algorithm is computationally more efficient than the SDP approach.