2.1. Deducing Monotonicity
The monotonicity significantly helps in global optimization. If a function
is monotonically nondecreasing on a segment
, then
,
and the segment
can be eliminated from further consideration after updating the record (best known solution so far). A similar statement is valid for a nonincreasing function. This techniques is known as the monotonicity test [
26,
41,
42]. Moreover, as it is shown below, the monotonicity is crucial for evaluating the convexity/concavity of a composite function.
The usual way to ensure the monotonicity of a differentiable univariate function on an interval is to compute an interval extension for its derivative . If , then the function is nondecreasing monotonic on . Similarly, if , then the function is nonincreasing monotonic on .
If a function is not differentiable, its monotonicity can still be evaluated using the rules described below. The Proposition 1 lists rules for evaluating an expression’s monotonicity composed with the simple arithmetic operations.
Proposition 1. The following rules hold:
if on then on ;
if on then on ;
if and on then on ;
if , and , on then on ;
if and on then on ;
if and on then on .
The proof of Proposition 1 is obvious. The rules for evaluating the monotonicity of the composition of functions are summarized in Proposition 2. The proof is intuitive and not presented here.
Proposition 2. Let be a composition of univariate functions and : . Then, the following four statements hold.
If on , on and then on .
If on , on and then on .
If on , on , then on .
If on , on , then on .
The monotonicity of elementary univariate functions on a given interval can easily be established as these functions’ behavior is well-known (
Table 2).
The monotonicity of a composite function defined by an arbitrary complex algebraic expression can be evaluated automatically using Propositions 1 and 2 and the data from the
Table 2. Let us consider an example.
Example 1. Evaluate the monotonicity of the function , where . This function is nonsmooth: it can be easily shown that does not have derivatives in two points on . Apply rules from Propositions 1 and 2: Thus, is nonincreasing monotonic on . In the same way, it can be established that is nonincreasing monotonic on . From the Proposition 1, it follows that is also nonincreasing monotonic on .
It is worth noting that the rules outlined above help to prove the monotonicity of nondifferentiable functions. However, for differentiable functions, the analysis of the the range of the first derivative is a better way to establish monotonicity. For example, a function is monotonic on an interval . Indeed, the range of its first derivative computed by the natural interval expansion is non-negative. However, its monotonicty cannot be established by the outlined rules since is not monotonic on . The general recommendation is to compute the first derivative’s range when the function is smooth and use Propositions 1 and 2 otherwise.
Monotonicity itself plays a vital role in optimization. The following obviously valid Proposition shows how the interval bounds can be computed for a monotonic function.
Proposition 3. Let be a monotonic function on an interval . Then 2.2. Deducing Convexity
First, we recall some well-known mathematical notions used in the rest of the paper. A function
is
convex on an interval if
for any
,
and any
,
. A function
is called concave on the interval
if
is convex on
.
Convexity plays an important role in optimization due to the following two observations. If a function is convex on some interval, then a minimum point of
can be efficiently found by well elaborated local search techniques [
43,
44]. If a function
is concave on
, then
.
If the function is two times differentiable, the convexity can be deduced from the second derivative. If one can prove that on a segment , then is convex (concave) on this segment. However, if the function is nonsmooth, the convexity property should be computed in some other way. Even if is smooth, the accurate bounding of its second derivative can be a complicated task, and the convexity test becomes difficult.
The conical combination and the maximum of two functions are known to preserve convexity. The proof can be found in seminal books on convex analysis, e.g., [
43]. For the sake of completeness, we reproduce these rules in the following Proposition 4.
Proposition 4. Let and be convex functions on an interval . Then, the following statements hold:
is convex on ,
is convex on if ,
is convex on .
The product of two convex functions is not always a convex function. For example,
is not convex while both
and
are convex functions on
. In [
45], it is proved that if
f and
g are two positive convex functions defined on an interval
, then their product is convex provided that they are synchronous in the sense that
for all
. However checking this general property automatically is difficult. Instead, we propose the following sufficient condition that can be effectively evaluated.
Proposition 5. Let and be convex positive functions on an interval such that and are both nonincreasing or both nondecreasing. Then, the function is convex on .
Proof. Consider
,
and
. Since
and
are convex, we get
Since
and
, we get
where
is a quadratic function. Since
and
are both nonincreasing or both nondecreasing, we have that
. Therefore
is convex. Note that
,
. From convexity of
, we obtain the following inequality:
This completes the proof. □
Propositions 4 and 5 can be readily reformulated for concave functions. The following Proposition gives rules for evaluating the convexity of a composite function.
Proposition 6. Let and there be intervals , such that . Then, the following holds:
g is convex and nondecreasing on , h is convex on , then f is convex on ,
g is convex and nonincreasing on , h is concave on , then f is convex on ,
g is concave and nondecreasing on , h is concave on , then f is concave on ,
g is concave and nonincreasing on , h is convex on , then f is concave on .
The proof of the Proposition 6 can be found in numerous books for convex analysis, e.g., [
43].
Many elementary functions are convex/concave on a whole domain of the definition, e.g.,
,
,
for even natural
n. For other functions, the intervals of concavity/convexity can be efficiently established as these function’s behavior is well-known (
Table 3).
Propositions 4–6 enable an automated convexity deduction for composite functions, as the following examples show.
Example 2. Consider the function on the interval . The function is convex on and nondecreasing. The function is convex on . According to the Proposition 6 function, is convex. Since is also convex, we conclude (Proposition 4) that is convex.
It is worth noting that the convexity can be proved by computing the interval bounds for the second derivative in the considered example. Indeed, is obviously positive on . Since there are plenty of tools for automatic differentiation and interval computations, the convexity can be proved automatically.
However, a convex function does not necessarily have derivatives in all points. Moreover, even if it is piecewise differentiable, locating the points where the function is not continuously differentiable can be difficult. Fortunately, the theory outlined above efficiently copes with such situations.
Example 3. Consider the following functionon an interval . Since is concave on , we conclude that is convex on . The convexity of follows from the convexity of the linear function and the Proposition 6. From the convexity of x, , and Proposition 4 we derive that is convex. Notice that automatic symbolic differentiation techniques cannot compute the derivative of because it involves computing the intersection points of x and functions, which is a rather complex problem.