Next Article in Journal
Complexity in Economic and Social Systems
Next Article in Special Issue
On-Road Detection of Driver Fatigue and Drowsiness during Medium-Distance Journeys
Previous Article in Journal
Non-Normalizable Quasi-Equilibrium Solution of the Fokker–Planck Equation for Nonconfining Fields
Previous Article in Special Issue
The Application of Fractal Transform and Entropy for Improving Fault Tolerance and Load Balancing in Grid Computing Environments
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Partial Exactness for the Penalty Function of Biconvex Programming

1
School of Management, Zhejiang University of Technology, Hangzhou 310023, China
2
School of Economics, Zhejiang University of Technology, Hangzhou 310023, China
*
Author to whom correspondence should be addressed.
Entropy 2021, 23(2), 132; https://doi.org/10.3390/e23020132
Submission received: 17 December 2020 / Revised: 16 January 2021 / Accepted: 16 January 2021 / Published: 21 January 2021
(This article belongs to the Special Issue Entropy and Its Applications across Disciplines II)

Abstract

:
Biconvex programming (or inequality constrained biconvex optimization) is an important model in solving many engineering optimization problems in areas like machine learning and signal and information processing. In this paper, the partial exactness of the partial optimum for the penalty function of biconvex programming is studied. The penalty function is partially exact if the partial Karush–Kuhn–Tucker (KKT) condition is true. The sufficient and necessary partially local stability condition used to determine whether the penalty function is partially exact for a partial optimum solution is also proven. Based on the penalty function, an algorithm is presented for finding a partial optimum solution to an inequality constrained biconvex optimization, and its convergence is proven under some conditions.

1. Introduction

Multi-convex programming is a non-convex optimization problem [1,2], where biconvex programming is a special case. It is ubiquitous nowadays in fields such as control [3,4,5], machine learning [6,7], signal and information processing [8,9], communication [10,11], and also NP-hard problems [12]. The existing research on multi-convex programming mainly solves some very special models [12,13,14,15,16,17,18,19]. These studies all give specific methods for each special model. When problems change, new methods need to be studied. Biconvex programming is a simple case of multi-convex programming, and it is also studied in research [14] as a special case. Therefore, it is of great significance to study the algorithm of biconvex programming for solving practical engineering optimization problems.
Bilinear programming is the simplest case of biconvex programming and is the earliest and most studied as per [14]. There are two main ways to solve bilinear programming. One is the simplex algorithm based on sub-problems, and the other is the alternating direction method. For example, Liang and Bai [15] and Hajinezhad and Shi [16] both proposed the alternating direction method of multipliers (ADMM) algorithm for two special bilinear programming problems, where the extended Lagrangian penalty function uses a square penalty. Furthermore, Charkhgard et al. [17] presented a multi-linear programming algorithm using the linear programming algorithm.
In 2007, Gorski et al. [14] reviewed the development of the theory and algorithms of biconvex optimization. The biconvex functions have some good properties that are similar to convex functions, such as the biconvex separation theorem and an equivalence between local optimal solutions and stationary points. The biconvex programming algorithm is based on the idea that the solution to alternating sub-problems can converge to the stationary point of the original problem. For example, in 2015, Li et al. [18] studied an alternating convex search method to solve a stationary point problem of biconvex programming. In 2016, Shah et al. [19] presented an alternating search method with a square penalty function to solve biconvex programming, which had great effect in image recognition. Al-Khayyaltt and Falk [20] discussed an algorithm of biconvex programming using the idea of branch-and-bound.
In short, based on the above, the mainstream method used to solve biconvex programming is an alternative to subproblem solving, because the bi-convexity of biconvex programming guarantees the convergence of alternative search methods and effectually cuts down the scale of problem solving via decomposition calculation. Hence, in view of the large scale of biconvex programming(BCP), alternative subproblem solving will be the main method adopted in further research regarding biconvex programming.
In this paper we consider biconvex programming with inequality constraints as follows:
( BCP ) min h ( u 1 , u 2 ) s . t . g i ( u 1 , u 2 ) 0 , i = 1 , 2 , , m ,
where h , g i : R n 1 × R n 2 R ( i = 1 , 2 , , m ) are biconvex if h ( u 1 , u 2 ) and g i ( u 1 , u 2 ) are convex in u 1 R n 1 for every fixed u 2 and in u 2 R n 2 for every fixed u 1 . Let u = ( u 1 , u 2 ) and I = { 1 , 2 , , m } . The feasible set of (BCP) is denoted by:
U = { ( u 1 , u 2 ) R n 1 × R n 2 g i ( u 1 , u 2 ) 0 , i I } .
Let
U ( u 2 ) = { u 1 R n 1 g i ( u 1 , u 2 ) 0 , i I } .
When u 2 is fixed, define the suboptimal problem:
( BCP ) ( u 2 ) min h ( u 1 , u 2 ) s . t . u 1 U ( u 2 ) .
Let
U ( u 1 ) = { u 2 R n 2 g i ( u 1 , u 2 ) 0 , i I } .
When u 1 is fixed, define the suboptimal problem:
( BCP ) ( u 1 ) min h ( u 1 , u 2 ) s . t . u 2 U ( u 1 ) .
Many practical engineering optimization problems can be transformed into biconvex programming, for example the two-cardinality sparse convex optimization problem and the nonconvex quadratic programming problem.
It is well known that an effective method to solve constrained optimization is by penalty function [21]. Its main idea is to transform a constrained optimization problem into a sequence of unconstrained optimization subproblems that are easier to solve. When the penalty function is not exact, many calculations are needed to solve unconstrained optimization subproblems so as to obtain an approximate solution to inequality constrained optimization. For example, it is proven that the exact penalty function method for solving constrained optimization is very efficient, as first proposed by Zangwill (1967) in [22]. In theory, if the penalty function is exact, to obtain an optimal solution to a constrained optimization problem only one unconstrained optimization subproblem is solved. Hence, the exact penalty function algorithm takes less time than the inexact penalty function algorithm. Additionally, this approach has recently received great attention from both theoretical and practical arenas. Many studies were later presented based on the exact penalty function algorithm, such as Rosenberg (1986) [23] and Di Pillo (1986) [24]. In addition, it is essential to determine the exactness of a penalty function under the stability condition [25,26]. Hence, this paper mainly focuses on the relationship between the partial exactness of the penalty function and the partial stability of biconvex programming. On the other hand, there exist other approaches to reduce constrained optimization problems to unconstrained ones; for example, the index method presented in the monograph [27] or the method for computable boundaries presented in [28]. This shows the significance of the study of the partial exact penalty function.
In order to ensure exactness, we propose the following penalty function:
H p ( u 1 , u 2 , ρ ) = h ( u 1 , u 2 ) + ρ i I max { g i ( u 1 , u 2 ) , 0 } p ,
where penalty parameter ρ > 0 and 0 < p 1 . By the definition of exactness in [29], a penalty function is exact at p = 0.5 , but is not exact at p = 1 , such that min x R 1 f ( x ) = x s . t . x 2 0 . Hence, the exactness of an optimization problem depends on the structure of the problem. In this paper, we will study the exactness of a more extensive penalty function for biconvex programming than the one presented in [30].
The remainder of the paper is organized as follows. In Section 2, for a partial optimum solution, the partial exactness of the penalty function is proven under the partial Karush–Kuhn–Tucker (KKT) condition or the partial stableness condition. An algorithm is presented to find out a partial optimum solution to (BCP) with convergence.

2. Partial Exactness and a Penalty Function for (BCP)

According to Gorski et al. [14], defining the partial optimum of (BCP) is very meaningful. The concept of the partial optimum of (BCP) is given as follows:
Definition 1.
Let ( u 1 * , u 2 * ) U . If:
h ( u 1 * , u 2 * ) h ( u 1 , u 2 * ) , u 1 U ( u 2 * ) ,
h ( u 1 * , u 2 * ) h ( u 1 * , u 2 ) , u 2 U ( u 1 * ) ,
then ( u 1 * , u 2 * ) is called a partial optimum of (BCP). A partial optimum ( u 1 * , u 2 * ) of (BCP) means that u 1 * is an optimal solution to (BCP)( u 2 * ) and u 2 * is an optimal solution to (BCP)( u 1 * ).
Next, let us give the equivalence of a partial optimum of (BCP) to a partial KKT point under some conditions.
Let h , g i : R n 1 × R n 2 R , i I = { 1 , 2 , , m } be biconvex and differentiable. Let ( u 1 * , u 2 * ) U . If there are α i ( i = 1 , 2 , , m ) such that:
h ( u 1 * , u 2 * ) + i = 1 m α i g i ( u 1 * , u 2 * ) = 0 ,
α i g i ( u 1 * , u 2 * ) = 0 , α i 0 , i = 1 , 2 , , m ,
then ( u 1 * , u 2 * ) is a KKT point of (BCP).
Let ( u 1 * , u 2 * ) U . If there are α i ( 1 ) , α i ( 2 ) ( i = 1 , 2 , , m ) such that:
u 1 h ( u 1 * , u 2 * ) + i = 1 m α i ( 1 ) u 1 g i ( u 1 * , u 2 * ) = 0 ,
u 2 h ( u 1 * , u 2 * ) + i = 1 m α i ( 2 ) u 2 g i ( u 1 * , u 2 * ) = 0 ,
α i ( 1 ) g i ( u 1 * , u 2 * ) = 0 , α i ( 2 ) g i ( u 1 * , u 2 * ) = 0 , α i 1 0 , α i 2 0 , i = 1 , 2 , , m ,
then ( u 1 * , u 2 * ) is a partial KKT point of (BCP).
Let ( u 1 * , u 2 * ) U . The constraint of (BCP) is called a partial Slater constraint qualification at ( u 1 * , u 2 * ) , if there is ( u ¯ 1 , u ¯ 2 ) R n 1 × R n 2 such that:
g i ( u 1 * , u ¯ 2 ) < 0 , g i ( u ¯ 1 , u 2 * ) < 0 , i = 1 , 2 , , m .
In fact, if the Slater constraint qualification is satisfied for a convex programming, an optimal solution of the convex programming is equal to a KKT condition. For biconvex programming, we have the results in [28] as follows.
Theorem 1.
Let ( u 1 * , u 2 * ) u . If (BCP) is satisfied with partial Slater constraint qualification at ( u 1 * , u 2 * ) , then ( u 1 * , u 2 * ) is a partial optimum of (BCP) if and only if ( u 1 * , u 2 * ) is a partial KKT point of (BCP).
Corollary 1.
Let ( u 1 * , u 2 * ) u be a partial optimum of (BCP). If (BCP) is satisfied with partial Slater constraint qualification at ( u 1 * , u 2 * ) , then ( u 1 * , u 2 * ) is a KKT point of (BCP) if and only if (5), (6), and (7) hold with α i 1 = α i 2 , i = 1 , 2 , , m .
Example 1.
Let the biconvex programming:
( B C P 1 ) min h ( x , y ) = x y s . t . g 1 ( x , y ) = 1 x y 0 ,
where h , g 1 : R × R R . For θ ( 0 , 1 ) , it is clear that ( x , y ) = ( θ , 1 θ ) is the partial KKT point and the partial optimum of (BCP1). If y = 1 x , then h ( x , y ) = x x 2 . We have h ( x , y ) as x . Hence, a local optimal solution to (BCP1) cannot be solved, such as in ( x , y ) = ( 1 2 , 1 2 ) .
Example 1 means that if there is no optimal solution to biconvex programming, there may exist a partial KKT point or a partial optimum. It is obvious that an optimal solution to biconvex programming is the partial optimum. For biconvex programming, there is a partial optimum even if an optimal solution to biconvex programming is not obtained. Example 1 further indicates that the partial optimum is very important to biconvex programming.
Let ρ > 0 be given. Consider the following optimization problem:
( BCP ( u 2 ; ρ ) ) min H p ( u 1 , u 2 , ρ ) , s . t . u 1 R n 1 .
where u 1 is a decision variable when u 2 is fixed, and
( BCP ( u 1 ; ρ ) ) min H p ( u 1 , u 2 , ρ ) , s . t . u 2 R n 2 .
where u 2 is a decision variable when u 1 is fixed.
Definition 2.
Let ( u ¯ 1 * , u ¯ 2 * ) R n 1 × R n 2 . If:
H p ( u ¯ 1 * , u ¯ 2 * , ρ ) H p ( u 1 , u ¯ 2 * , ρ ) , u 1 R n 1 ,
H p ( u ¯ 1 * , u ¯ 2 * , ρ ) H p ( u ¯ 1 * , u 2 , ρ ) , u 2 R n 2 ,
then ( u ¯ 1 * , u ¯ 2 * ) is called a partial optimum of H p ( u 1 , u 2 , ρ ) .
Next, we define the concept of a partially exact penalty function for biconvex programming as follows.
Definition 3.
(1) Let ( u 1 * , u 2 * ) be a partial optimum of (BCP). If there is a ρ such that ( u 1 * , u 2 * ) is a partial optimum of H p ( u 1 , u 2 , ρ ) for ρ > ρ , then H p ( u 1 , u 2 , ρ ) is called a partially exact penalty function.
(2) Let ( u 1 * , u 2 * ) be a partial optimum of H p ( u 1 , u 2 , ρ ) . If ( u 1 * , u 2 * ) of H p ( u 1 , u 2 , ρ ) is a partial optimum of (BCP), then ρ is called a partially exact value of the penalty parameter.
Example 2 below shows the partial exactness of a penalty function under different p values.
Example 2.
Let the biconvex programming:
( B C P 2 ) min h ( x , y ) = x y + x + y s . t . x 2 + y 2 0 .
It is clear that ( 0 , 0 ) is a partial optimum of (BCP2). The penalty function for (BCP2) is defined by:
H p ( x , y ; ρ ) = x y + x + y + ρ max { x 2 + y 2 , 0 } p .
When p = 1 , a partial optimum of min x , y R H 1 ( x , y ; ρ ) is ( x ( ρ ) , y ( ρ ) ) = ( 1 2 ρ + 1 , 1 2 ρ + 1 ) . Letting ρ + , we have ( x ( ρ ) , y ( ρ ) ) ( 0 , 0 ) . Hence, H 1 ( x , y , ρ ) is not a partially exact penalty function. When p = 0.5 , we have:
H 0.5 ( x , y ; ρ ) = x y + x + y + ρ x 2 + y 2 .
A partial optimum of min x , y R H 0.5 ( x , y ; ρ ) is ( 0 , 0 ) for ρ 0 . Hence, H 0.5 ( x , y , ρ ) is a partially exact penalty function for ρ 1 . Example 2 means that the partial exactness of penalty function depends on the parameter p.
Example 3.
Let the biconvex programming:
( B C P 3 ) min h ( x , y ) = x y s . t . x y 1 .
It is easy to verify that ( x * , y * ) = ( θ , 1 θ ) T is an optimal solution to (BCP3) for any θ 0 . Thus, ( x * , y * ) = ( θ , 1 θ ) T is a partial optimum of (BCP3) too. The example illustrates that all partial optimums of (BCP) may be optimal solutions.
( x * , y * ) = ( 1 , 1 ) T is a partial optimum of (BCP3). The square penalty function for (BCP3) is defined by:
H 2 ( x , y ; ρ ) = x y + ρ max { 0 , 1 x y } 2 .
It is easy to check that ( x * , y * ) = ( 1 δ ( 1 1 2 ρ ) , δ ) is a partial optimum of H 2 ( x , y , ρ ) for δ 0 and ρ 1 . It is easy to check that H 2 ( x , y ; ρ ) is not partially exact.
For 0 < p 1 , a penalty function for (BCP3) is defined by:
H p ( x , y ; ρ ) = x y + ρ max { 0 , 1 x y } p .
We have that ( x * , y * ) = ( 1 , 1 ) is a partial optimum of H p ( x , y ; ρ ) for ρ > 1 . It is easy to check that H p ( x , y ; ρ ) is partially exact for ρ > 1 . We easily check that ( 1 , 1 ) is a KKT point.
Example 3 illustrates that the partially exact penalty function for a partial optimum can be as good as traditional exact penalty functions.
We prove the similarity of the partially exact penalty result to [28].
Theorem 2.
Let h ( u 1 , u 2 ) 0 , ( u 1 , u 2 ) R n 1 × R n 2 . Let ( u 1 * , u 2 * ) U be a partial optimum of (BCP). If ( u 1 * , u 2 * ) is a KKT partial point of (BCP), i.e., there are α i ( 1 ) , α i ( 2 ) ( i = 1 , 2 , , m ) such that (5), (6), and (7) are true, then H p ( u 1 , u 2 , ρ ) is a partially exact penalty function for ρ > ρ * , where:
ρ * = max { h ( u 1 * , u 2 * ) , α i ( 1 ) , α i ( 2 ) | i = 1 , 2 , , m } .
Proof. 
If ( u 1 * , u 2 * ) is a partial KKT point of (BCP), there are α i ( 1 ) and α i ( 2 ) ( i = 1 , 2 , , m ) such that (5), (6) and (7) are true. Because h ( u 1 , u 2 ) , g i ( u 1 , u 2 ) ( i I ) are biconvex functions for u 1 R n 1 and u 2 R n 2 , we have:
h ( u 1 , u 2 * ) h ( u 1 * , u 2 * ) + u 1 h ( u 1 * , u 2 * ) T ( u 1 u 1 * ) ,
g i ( u 1 , u 2 * ) g i ( u 1 * , u 2 * ) + u 1 g i ( u 1 * , u 2 * ) T ( u 1 u 1 * ) , i I ,
h ( u 1 * , u 2 ) h ( u 1 * , u 2 * ) + u 2 h ( u 1 * , u 2 * ) T ( u 2 u 2 * ) ,
g i ( u 1 * , u 2 ) g i ( u 1 * , u 2 * ) + u 2 g i ( u 1 * , u 2 * ) T ( u 2 u 2 * ) , i I .
By (5), (7), (10), and (11), we have:
h ( u 1 , u 2 * ) h ( u 1 * , u 2 * ) i = 1 m α i ( 1 ) u 1 g i ( u 1 * , u 2 * ) T ( u 1 u 1 * ) h ( u 1 * , u 2 * ) + i = 1 m α i ( 1 ) ( g i ( u 1 * , u 2 * ) g i ( u 1 , u 2 * ) ) = h ( u 1 * , u 2 * ) i = 1 m α i ( 1 ) g i ( u 1 , u 2 * ) .
Let P ( t ) = max { t , 0 } p , t R , and 0 < p 1 . Let us take ρ > ρ * with ρ * = max { h ( u 1 * , u 2 * ) , α i ( 1 ) , α i ( 2 ) | i = 1 , 2 , , m } . If there is a g i ( u 1 , u 2 * ) > 1 , then P ( g i ( u 1 , u 2 * ) ) > 1 . We have:
h ( u 1 , u 2 * ) + ρ i = 1 m P ( g i ( u 1 , u 2 * ) ) ρ * h ( u 1 * , u 2 * ) ,
i.e.,
H p ( u 1 , u 2 * , ρ ) H p ( u 1 * , u 2 * , ρ ) .
Otherwise, if g i ( u 1 , u 2 * ) 1 for i I , from (14) we have:
h ( u 1 , u 2 * ) + ρ i = 1 m P ( g i ( u 1 , u 2 * ) ) h ( u 1 * , u 2 * ) + i = 1 m ( ρ max { g i ( u 1 , u 2 * ) , 0 } p α i ( 1 ) g i ( u 1 , u 2 * ) ) .
Hence, for ρ > ρ * , we have:
H p ( u 1 * , u 2 * , ρ ) H p ( u 1 , u 2 * , ρ ) .
Similarly, for ρ > ρ * , we have:
H p ( u 1 * , u 2 * , ρ ) H p ( u 1 * , u 2 , ρ ) .
By Definition 2, we have that H p ( u 1 , u 2 , ρ ) is a partially exact penalty function for ρ > ρ * .
Note. If there is a number A such that h ( u 1 , u 2 ) A for any ( u 1 , u 2 ) R n 1 × R n 2 , then the conclusion of Theorem 2 holds too. If we let h 1 ( u 1 , u 2 ) = h ( u 1 , u 2 ) A , we have h 1 ( u 1 , u 2 ) 0 on R n 1 × R n 2 . It is clear that the problem
( BCP ) 1 min h 1 ( u 1 , u 2 ) s . t . g i ( u 1 , u 2 ) 0 , i = 1 , 2 , , m ,
is equal to the problem (BCP). Particularly, when p = 1 , we have the following conclusion.
Theorem 3.
Let ( u 1 * , u 2 * ) U be a partial optimum of (BCP). If ( u 1 * , u 2 * ) is a KKT partial point of (BCP), i.e., there are α i ( 1 ) and α i ( 2 ) ( i = 1 , 2 , , m ) such that (5), (6), and (7) are true, then H 1 ( u 1 , u 2 , ρ ) is a partially exact penalty function with p = 1 for ρ > ρ * , where:
ρ * = max { α i ( 1 ) , α i ( 2 ) | i = 1 , 2 , , m } .
Theorem 3 is consistent with Theorem 2 in [28].
Similar to that for a constrained penalty function presented in [25,26], the concept of stability for a penalty function of (BCP) is defined. Let s 1 = ( s 1 1 , s 2 1 , , s m 1 ) and:
U ( u 2 , s 1 ) = { u 1 R n 1 g i ( u 1 , u 2 ) s i 1 , i I } .
When u 2 is fixed, define a perturbed problem:
( BCP ) ( u 2 , s 1 ) min h ( u 1 , u 2 ) s . t . u 1 U ( u 2 , s 1 ) .
Let s 2 = ( s 1 2 , s 2 2 , , s m 2 ) and:
U ( u 1 , s 2 ) = { u 2 R n 2 g i ( u 1 , u 2 ) s i 2 , i I } .
When u 1 is fixed, define a perturbed problem:
( BCP ) ( u 1 , s 2 ) min h ( u 1 , u 2 ) s . t . u 2 U ( u 1 , s 2 ) .
Definition 4.
Let ( u 1 * , u 2 * ) be a partial optimum of (BCP), and u 1 s 1 * and u 2 s 2 * be optimal solutions to (BCP)( u 2 * , s 1 ) and (BCP)( u 1 * , s 2 ), respectively, for any s 1 , s 2 R m . If there is a ρ > 0 such that:
h ( u 1 * , u 2 * ) h ( u 1 s 1 * , u 2 * ) ρ | s 1 | , ρ > ρ ,
h ( u 1 * , u 2 * ) h ( u 1 * , u 2 s 2 * ) ρ | s 2 | , ρ > ρ ,
where | s 1 | = i = 1 m P ( s i 1 ) and | s 2 | = i = 1 m P ( s i 2 ) , then H p ( u 1 , u 2 , ρ ) is partially stable at ( u 1 * , u 2 * ) . Furthermore, if there are a γ > 0 and a ρ > 0 such that (16) and (17) hold for | s 1 | γ and | s 2 | γ , then H p ( u 1 , u 2 , ρ ) is partially locally stable at ( u 1 * , u 2 * ) .
Theorem 4.
Let ( u 1 * , u 2 * ) be a partial optimum of (BCP). If H p ( u 1 , u 2 , ρ ) is partially stable, then H p ( u 1 , u 2 , ρ ) is a partially exact penalty function at ( u 1 * , u 2 * ) .
Proof. 
Let us prove that H p ( u 1 , u 2 , ρ ) is a partially exact penalty function when H p ( u 1 , u 2 , ρ ) is partially stable at ( u 1 * , u 2 * ) . Suppose that H p ( u 1 , u 2 , ρ ) is not a partially exact penalty function. According to the definition of partial stability, for any s 1 , s 2 , we obtain that there is a ρ satisfying that:
h ( u 1 * , u 2 * ) h ( u 1 s 1 * , u 2 * ) ρ | s 1 | , ρ > ρ ,
h ( u 1 * , u 2 * ) h ( u 2 * , u 2 s 2 * ) ρ | s 2 | , ρ > ρ .
Then, there always exists some ρ > ρ such that ( u 1 * , u 2 * ) is not a partial optimum of H p ( u 1 , u 2 , ρ ) , i.e., there is some ( u 1 , u 2 ) such that:
H p ( u 1 , u 2 * , ρ ) < H p ( u 1 * , u 2 * , ρ ) = h ( u 1 * , u 2 * ) ,
H p ( u 1 * , u 2 , ρ ) < H p ( u 1 * , u 2 * , ρ ) = h ( u 1 * , u 2 * ) .
Thus,
h ( u 1 , u 2 * ) + ρ i I P ( g i ( u 1 , u 2 * ) ) < h ( u 1 * , u 2 * ) ,
h ( u 1 * , u 2 ) + ρ i I P ( g i ( u 1 * , u 2 ) ) < h ( u 1 * , u 2 * ) .
Suppose that u 1 U ( u 2 * ) , u 2 U ( u 1 * ) . We have:
h ( u 1 , u 2 * ) < h ( u 1 * , u 2 * ) ,
h ( u 1 * , u 2 ) < h ( u 1 * , u 2 * ) .
This implies that h ( u 1 , u 2 * ) < h ( u 1 * , u 2 * ) < h ( u 1 , u 2 * ) and h ( u 1 * , u 2 ) < h ( u 1 * , u 2 * ) < h ( u 1 * , u 2 ) , which shows that ( u 1 * , u 2 * ) is not a partial optimum of (BCP). A contradiction occurs. Hence, u 1 U ( u 2 * ) and u 2 U ( u 1 * ) do not hold, and i I P ( g i ( u 1 , u 2 * ) ) > 0 or i I P ( g i ( u 1 * , u 2 ) ) > 0 .
Let s 1 = ( s 1 1 , s 2 1 , , s m 1 ) with s i 1 = g i ( u 1 , u 2 * ) and s 2 = ( s 1 2 , s 2 2 , , s m 2 ) with s i 2 = g i ( u 1 * , u 2 ) , i = 1 , 2 , , m , and u 1 s 1 * and u 2 s 2 * be optimal solutions to (BCP)( u 2 * , s 1 ) and (BCP)( u 1 * , s 2 ), respectively. Then, h ( u 1 s 1 * , u 2 * ) h ( u 1 , u 2 * ) and h ( u 1 * , u 2 s 2 * ) h ( u 1 * , u 2 ) . Thus,
h ( u 1 s 1 * , u 2 * ) h ( u 1 , u 2 * ) ,
h ( u 1 * , u 2 s 2 * ) h ( u 1 * , x 2 ) .
Therefore,
h ( u 1 s 1 * , u 2 * ) + ρ i I P ( s i 1 ) h ( u 1 , u 2 * ) + ρ i I P ( s i 1 ) = H p ( u 1 , u 2 * , ρ ) < h ( u 1 * , u 2 * ) ,
and:
h ( u 1 * , u 2 s 2 * ) + ρ i I P ( s i 2 ) h ( u 1 * , u 2 ) + ρ i I P ( s i 2 ) = H p ( u 1 * , u 2 , ρ ) < h ( u 1 * , u 2 * ) ,
which shows that:
h ( u 1 * , u 2 * ) h ( u 1 * , u 2 s 2 * ) > ρ | s 1 | ,
h ( u 1 * , u 2 * ) h ( u 1 s 1 * , u 2 * ) > ρ | s 2 | ,
where | s 1 | = i I P ( s i 1 ) and | s 2 | = i I P ( s i 2 ) . These inequalities contradict (18) and (19). Hence, that H p ( x , y , ρ ) is not partially stable yields a contradiction with the assumption, which proves that H p ( x , y , ρ ) is a partially exact penalty function. □
Theorem 5.
Let ( u 1 * , u 2 * ) be a partial optimum of (BCP). If H p ( u 1 , u 2 , ρ ) is a partially exact penalty function at ( u 1 * , u 2 * ) , then H p ( u 1 , u 2 , ρ ) is partially locally stable. In particular, for p = 1 , H 1 ( u 1 , u 2 , ρ ) is partially stable.
Proof. 
Let us prove that H p ( x , y , ρ ) is partially locally stable when H p ( x , y , ρ ) is a partially exact penalty function. According to the definition of a partially exact penalty function, if ( u 1 * , u 2 * ) is a partial optimum of (BCP), there always exist some ρ > ρ such that:
H p ( u 1 * , u 2 * , ρ ) H p ( u 1 , u 2 * , ρ ) , u 1 R n 1 ,
H p ( u 1 * , u 2 * , ρ ) H p ( u 1 * , u 2 , ρ ) , u 2 R n 2 .
Let u 1 s 1 * and u 2 s 2 * be optimal solutions to (BCP)( u 2 * , s 1 ) and (BCP)( u 1 * , s 2 ), respectively, for any s 1 , s 2 R m . By (20), and (21), we have:
h ( u 1 * , u 2 * ) = H p ( u 1 * , u 2 * , ρ ) H p ( u 1 s 1 * , u 2 * , ρ ) ,
h ( u 1 * , u 2 * ) = H p ( u 1 * , u 2 * , ρ ) H p ( u 1 * , u 2 s 2 * , ρ ) .
It is clear that for s i 1 1 , s i 2 1 ( i I ) , we have:
P ( g i ( u 1 s 1 * , u 2 * ) ) P ( s i 1 ) , P ( g i ( u 1 * , u 2 s 2 * ) ) P ( s i 2 ) , i I .
From (22), (23), and (24), for | s 1 | m and | s 2 | m :
h ( u 1 * , u 2 * ) h ( u 1 s 1 * , u 2 * ) ρ | s 1 | , ρ > ρ ,
h ( u 1 * , u 2 * ) h ( u 1 * , u 2 s 2 * ) ρ | s 2 | , ρ > ρ .
It follows from the definition that H p ( x , y , ρ ) is partially locally stable. □
Theorem 4 and Theorem 5 mean that the stability condition is sufficient for the partial exact penalty function, but the necessary condition of the partial exact penalty function is partially locally stable, as shown in the following example.
Example 4.
Let the perturbed problem (BCP3), and its (BCP3( x , s 1 )) and (BCP3( y , s 2 )):
( B C P 3 ) ( x , s 1 ) min y h ( x , y ) = x y s . t . 1 x y s 1 , ( B C P 3 ) ( y , s 2 ) min x h ( x , y ) = x y s . t . 1 x y s 2 .
It is clear that ( x * , y * ) = ( 1 , 1 ) is a partial optimum of (MCP3) and its objective function value is 1. x s 1 * = 1 s 1 and y s 2 * = 1 s 2 are optimal solutions to (BCP3)( 1 , s 1 ) and (BCP3)( 1 , s 2 ), respectively, for any s 1 , s 2 R . Let P ( t ) = max { t , 0 } p and s 1 , s 2 1 with 0 < p < 1 . For ρ > 1 ,
h ( x * , y * ) h ( x * , y s 1 * ) = 1 ( 1 s 1 ) ρ P ( s 1 ) ,
h ( x * , y * ) h ( x s 1 * , y * ) = 1 ( 1 s 2 ) ρ P ( s 2 ) ,
then H p ( x , y , ρ ) is partially locally stable. When p = 1 , H 1 ( x , y , ρ ) is partially stable.

3. Partial Optimum Penalty Function Algorithm for (BCP)

Now, we present an algorithm to solve a partial optimum of (BCP) by solving the penalty function problem BCP( ρ ) as follows.
B C P ( ρ ) min H p ( u 1 , u 2 , ρ ) s . t . ( u 1 , u 2 ) R n 1 × R n 2 .
Based on the above results, the Partial Optimum Penalty Function Algorithm (Algorithm 1) was designed to compute a partial optimum of (BCP).
We prove the convergence of the Algorithm 1 in Theorem 6. Let:
S ( L , h ) = { ( u 1 , u 2 ) L h ( u 1 , u 2 ) , }
be a level set. If for any given L > 0 , S ( L , h ) is bounded, then S ( L , h ) is called bounded.
Algorithm 1: POPFA Algorithm
Step 1: Choose ρ 1 > 0 , N > 1 , 0 < p 1 and k = 1 .
Step 2: Solve ( u 1 k , u 2 k ) to be a partial optimum of
min H p ( u 1 , u 2 , ρ k ) s . t . ( u 1 , u 2 ) R n 1 × R n 2 .
Step 3: If ( u 1 k , u 2 k ) is feasible to (BCP), stop and ( u 1 k , u 2 k ) is a partial optimum of (BCP).
Otherwise, ρ k + 1 = N ρ k , k = : k + 1 and go to Step 2.
Theorem 6.
Let { ( u 1 k , u 2 k ) } be the sequence generated by the Algorithm 1 and h ( · ) be continuous.
(i) If { ( u 1 k , u 2 k ) } ( k = 1 , 2 , , k ¯ ) is a finite sequence (i.e., the Algorithm 1 stops at the k ¯ -th iteration), then ( u 1 k ¯ , u 2 k ¯ ) is a partial optimum of (BCP).
(ii) Let { ( u 1 k , u 2 k ) } be an infinite sequence, sequence { H p ( u 1 k , u 2 k , ρ k ) } be bounded and the level set S ( L , h ) be bounded. Then { ( u 1 k , u 2 k ) } is bounded and any limit point ( u 1 * , u 2 * ) of it is a partial optimum of (BCP).
Proof. 
(i) The conclusion is clear.
(ii) By the Algorithm 1, since { H p ( u 1 k , u 2 k , ρ k ) } is bounded as k + , there must be some L > 0 such that:
L > H p ( u 1 k , u 2 k , ρ k ) h ( u 1 k , u 2 k ) + ρ k i = 1 m P ( g i ( u 1 k , u 2 k ) ) h ( u 1 k , u 2 k ) .
Since h is continuous and the level set S ( L , h ) is bounded and closed, { ( u 1 k , u 2 k ) } and { h ( u 1 k , u 2 k ) } are bounded. Without loss of generality, suppose ( u 1 k , u 2 k ) ( u 1 * , u 2 * ) . Hence, there is an A > 0 such that h ( u 1 k , u 2 k ) > A . From (25), we have:
i = 1 m P ( g i ( u 1 k , u 2 k ) ) 1 ρ k ( L h ( u 1 k , u 2 k ) ) < L + A ρ k .
We have i = 1 m P ( g i ( u 1 k , u 2 k ) ) 0 as ρ k + . Hence, ( u 1 * , u 2 * ) is a feasible solution to (BCP).
Let any u ¯ 1 U ( u 2 * ) and u ¯ 2 U ( u 1 * ) . Since ( u 1 k , u 2 k ) is a partial optimum of
min H p ( u 1 , u 2 , ρ k ) s . t . ( u 1 , u 2 ) R n 1 × R n 2 ,
we have:
h ( u 1 k , u 2 k ) H p ( u 1 k , u 2 k , ρ k ) H p ( u ¯ 1 , u 2 k , ρ k ) ,
h ( u 1 k , u 2 k ) H p ( u 1 k , u 2 k , ρ k ) H p ( u 1 k , u ¯ 2 , ρ k ) .
Let k + , and the above inequations are:
h ( u 1 * , u 2 * ) h ( u ¯ 1 , u 2 * ) ,
h ( u 1 * , u 2 * ) h ( u 1 * , u ¯ 2 ) .
Hence, ( u 1 * , u 2 * ) is a partial optimum of (BCP). □
Theorem 6 means that the Algorithm 1 has good convergence in theory. From Theorem 4, when the penalty function is partially stable, the Algorithm 1 solves a single unconstrained optimization problem for the smaller penalty parameter ρ . Since the penalty function H p ( u 1 , u 2 , ρ ) is nonsmooth for 0 < p < 1 , it is necessary to smooth the constrained nonsmooth term to design an effective algorithm. Therefore, the smoothing algorithm of a partial exact penalty function is worthy of further study. In fact, for p = 1 , we have published a paper regarding a smoothing partially exact penalty function algorithm [30], where two numerical examples show the proposed algorithm is effective for biconvex programming.

4. Conclusions

In this paper, we studied the partial optimum solution to biconvex programming using the penalty function, which is partially exact. The form of this penalty function is more general than that in [28]. We proved that the partial exactness of the penalty function for biconvex programming is equivalent to the partial KKT condition, and we proved that the partial exactness of the penalty function for biconvex programming is equivalent to partially local stability. Based on the penalty function, the Algorithm 1 was theoretically presented to solve a partial optimum solution to biconvex programming. The convergence of the algorithm was also proven. The Algorithm 1 may solve a partial optimum solution to biconvex programming under the smaller penalty parameter. In the future, we may study the smoothing problem of the partial exact penalty function and its algorithm.

Author Contributions

Formal analysis and methodology, M.J.; investigation and methodology, Z.M.; methodology, R.S. All authors have read and agreed to the published version of the manuscript.

Funding

This work is supported by the National Natural Science Foundation of China (grant No.11871434) and the Natural Science Foundation of Zhejiang Province (grant No.LY18A010031).

Acknowledgments

This work is supported by the National Natural Science Foundation of China (grant No.11871434) and the Natural Science Foundation of Zhejiang Province (grant No.LY18A010031). The authors would like to express their gratitudes to anonymous referees’ detailed comments and remarks that help us improve our presentation of this paper considerably.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Shen, X.; Diamond, S.; Udell, M.; Gu, Y. Disciplined multi-convex programming. In Proceedings of the Control And Decision Conference (CCDC), Chongqing, China, 28–30 May 2017. [Google Scholar]
  2. Grant, M.; Boyd, S.; Ye, Y. Disciplined convex programming. In Global Optimization: From Theory to Implementation, Nonconvex Optimization and its Applications; Liberti, L., Maculan, N., Eds.; Springer: New York, NY, USA, 2006; pp. 155–210. [Google Scholar]
  3. Chiu, W.Y. Method of reduction of variables for bilinear matrix inequality problems in system and control designs. IEEE Trans. Syst. Man Cybern. Syst. 2017, 47, 1241–1256. [Google Scholar] [CrossRef] [Green Version]
  4. Ichihara, H.; Nobuyama, E. Difference of multiconvex relaxation of parameterized LMIs: Control applications. In Proceedings of the SICE 2003 Annual Conference, Fukui, Japan, 4–6 August 2003. [Google Scholar]
  5. Hours, J.; Jones, C. A parametric multiconvex splitting technique with application to real-time NMPC. In Proceedings of the 53rd IEEE Conference on Decision and Control, Los Angeles, CA, USA, 15–17 December 2014. [Google Scholar]
  6. Suh, S.; Shin, S.; Lee, J.; Reddy, C.K.; Choo, J. Localized user-driven topic discovery via boosted ensemble of nonnegative matrix factorization. Knowl. Inf. Syst. 2018, 56, 503–531. [Google Scholar] [CrossRef]
  7. Udell, M.; Horn, C.; Zadeh, R.; Boyd, S. Generalized low rank models. Found. Trends Mach. Learn. 2016, 9, 1–118. [Google Scholar] [CrossRef]
  8. Fu, X.; Huang, K.; Sidiropoulos, N.D. On identifiability of nonnegative matrix factorization. IEEE Signal Process. Lett. 2018, 25, 328–332. [Google Scholar] [CrossRef]
  9. Hong, M.; Luo, Z. On the linear convergence of the alternating direction method of multipliers. Math. Program. 2017, 162, 165–199. [Google Scholar] [CrossRef] [Green Version]
  10. Serbetli, S.; Yener, A. Transceiver optimization for multiuser MIMO systems. IEEE Trans. Signal Process. 2004, 52, 214–226. [Google Scholar] [CrossRef]
  11. Al-Shatri, H.; Li, X.; Ganesa, R.S.S. Maximizing the Sum Rate in Cellular Networks Using Multiconvex Optimization. IEEE Trans. Wirel. Commun. 2016, 15, 3199–3211. [Google Scholar] [CrossRef] [Green Version]
  12. Wen, Z.; Yin, W.; Zhang, Y. Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm. Math. Program. Comput. 2012, 4, 333–361. [Google Scholar] [CrossRef]
  13. Xu, Y.; Yin, W. A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. Siam J. Imaging Sci. 2013, 6, 1758–1789. [Google Scholar] [CrossRef]
  14. Gorski, J.; Pfeuffer, F.; Klamroth, K. Biconvex sets and optimization with biconvex functions: A survey and extensions. Math. Methods Oper. Res. 2007, 66, 373–407. [Google Scholar] [CrossRef]
  15. Liang, X.; Bai, J. Preconditioned ADMM for a class of bilinear programming problems. Math. Probl. Eng. 2018, 2018, 1–9. [Google Scholar] [CrossRef] [Green Version]
  16. Hajinezhad, D.; Shi, Q.J. Alternating direction method of multipliers for a class of nonconvex bilinear optimization: Convergence analysis and applications. J. Glob. Optim. 2018, 70, 261–288. [Google Scholar] [CrossRef]
  17. Charkhgard, H.; Savelsbergh, M.; Talebian, M. A linear programming based algorithm to solve a class of optimization problems with a multi-linear objective function and affine constraints. Comput. Oper. Res. 2018, 89, 17–30. [Google Scholar] [CrossRef]
  18. Li, G.; Wen, C.; Zheng, W.X.; Zhao, G. Iterative identification of block-oriented nonlinear systems based on biconvex optimization. Syst. Control. Lett. 2015, 79, 68–75. [Google Scholar] [CrossRef]
  19. Shah, S.; Yadav, A.K.; Castillo, C.D.; Jacobs, D.W.; Studer, C.; Goldstein, T. Biconvex Relaxation for Semidefinite Programming in Computer Vision. In Proceedings of the European Conference on Computer Vision: Computer Vision, Amsterdam, The Netherlands, 8–16 October 2016. [Google Scholar]
  20. Al-Khayyaltt, F.A.; Falk, J.E. Jointly constrained biconvex programming. Math. Oper. Res. 1983, 8, 273–286. [Google Scholar] [CrossRef]
  21. Pardalos, P.M.; Resende, M.G.C. Handbook of Applied optimization; Oxford University Press: Oxford, UK, 2002. [Google Scholar]
  22. Zangwill, W.I. Nonlinear programming via penalty function. Manag. Sci. 1967, 13, 334–358. [Google Scholar] [CrossRef]
  23. Rosenberg, E. Globally convergent algorithms for convex programming. Math. Oper. Res. 1981, 6, 437–443. [Google Scholar] [CrossRef]
  24. Di Pillo, G.; Grippo, L. An exact penalty function method with global conergence properties for nonlinear programming problems. Math. Program. 1981, 36, 1–18. [Google Scholar] [CrossRef]
  25. Clarke, F.H. Optimization and Nonsmooth Analysis; Wiley: New York, NY, USA, 1983. [Google Scholar]
  26. Burke, J.V. Calmness and exact penalization. Siam J. Control. Optim. 1991, 29, 493–497. [Google Scholar] [CrossRef]
  27. Strongin, R.G.; Sergeyev, Y.D. Global Optimization with Non-Convex Constraints; Science & Business: Dordrecht, The Netherlands, 2000. [Google Scholar]
  28. Grishagin, V.; Israfilov, R. Multidimensional Constrained Global Optimization in Domains with Computable Boundaries. Available online: https://elar.urfu.ru/bitstream/10995/36002/1/ural_pdc-2015-09.pdf (accessed on 16 January 2021).
  29. Yang, X.Q.; Meng, Z.Q. Lagrange Multipliers and Calmness Conditions of Order p. Math. Oper. Res. 2007, 32, 95–101. [Google Scholar] [CrossRef]
  30. Shen, R.; Meng, Z.Q.; Jiang, M. Smoothing Partially Exact Penalty Function of Biconvex Programming. Asia-Pac. J. Oper. Res. 2020, 37, 27. [Google Scholar] [CrossRef]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Jiang, M.; Meng, Z.; Shen, R. Partial Exactness for the Penalty Function of Biconvex Programming. Entropy 2021, 23, 132. https://doi.org/10.3390/e23020132

AMA Style

Jiang M, Meng Z, Shen R. Partial Exactness for the Penalty Function of Biconvex Programming. Entropy. 2021; 23(2):132. https://doi.org/10.3390/e23020132

Chicago/Turabian Style

Jiang, Min, Zhiqing Meng, and Rui Shen. 2021. "Partial Exactness for the Penalty Function of Biconvex Programming" Entropy 23, no. 2: 132. https://doi.org/10.3390/e23020132

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop