Brute Force Computations and Reference Solutions

Abstract

In this paper, we consider the application of brute force computational techniques (BFCTs) for solving computational problems in mathematical analysis and matrix algebra in a floating-point computing environment. These techniques include, among others, simple matrix computations and the analysis of graphs of functions. Since BFCTs are based on matrix calculations, the program system MATLAB^® is suitable for their computer realization. The computations in this paper are completed in double precision floating-point arithmetic, obeying the 2019 IEEE Standard for binary floating-point calculations. One of the aims of this paper is to analyze cases where popular algorithms and software fail to produce correct answers, failing to alert the user. In real-time control applications, this may have catastrophic consequences with heavy material damage and human casualties. It is known, or suspected, that a number of man-made catastrophes such as the Dharhan accident (1991), Ariane 5 launch failure (1996), Boeing 737 Max tragedies (2018, 2019) and others are due to errors in the computer software and hardware. Another application of BFCTs is finding good initial guesses for known computational algorithms. Sometimes, simple and relatively fast BFCTs are useful tools in solving computational problems correctly and in real time. Among particular problems considered are the genuine addition of machine numbers, numerically stable computations, finding minimums of arrays, the minimization of functions, solving finite equations, integration and differentiation, computing condensed and canonical forms of matrices and clarifying the concepts of the least squares method in the light of the conflict remainders vs. errors. Usually, BFCTs are applied under the user’s supervision, which is not possible in the automatic implementation of computational methods. To implement BFCTs automatically is a challenging problem in the area of artificial intelligence and of mathematical artificial intelligence in particular. BFCTs allow to reveal the underlying arithmetic in the performance of computational algorithms. Last but not least, this paper has tutorial value, as computational algorithms and mathematical software are often taught without considering the properties of computational algorithms and machine arithmetic.

1. Introduction and Notation

1.1. Preliminaries

In the last 300 years, sophisticated computational algorithms have been developed to solve a wide variety of problems in the mathematical and engineering sciences [1,2,3,4,5]. With the advent of digital computers in the mid-20th century, it became evident that some historical algorithms are unsuitable for direct computer implementation without significant modifications. Meanwhile, other algorithms, despite their cleverness, became obsolete. Consequently, paradigms in scientific computation have evolved, resulting in the creation and deployment of novel computational algorithms as computer codes.

When implemented as computer codes in floating-point computer environment, such as the one realized in the program systems MATLAB^® [6,7] and Octave [8], some of these computational algorithms may produce wrong results without warning. In such cases, BFCTs may give acceptable results. BFCTs may also reveal flaws in symbolic codes such as sym(x,′e′) from [6].

Other computer systems, such as Maple [9] and Mathematica [10,11], use variable precision arithmetic to potentially avoid these pitfalls. However, this may lead to an increase in the time required for algorithm performance. In such cases, BFCTs and a detailed analysis of computational problems with reference solution may be useful.

To evaluate and improve BFCTs’ scalability, optimization or parallelization strategies can be employed on larger datasets and higher-dimensional problems. Techniques such as parallel computing, CPU acceleration [12,13], and cloud computing [14] are particularly advantageous. This seems to be a new and challenging research area.

BFCTs may also be used to obtain initial guesses for computational algorithms intended to solve finite equations and minimize functions of one or several scalar arguments. We stress finally that Monte Carlo algorithms [15] are also a form of BFCT. As a perspective, a combination of artificial intelligence [16,17] with BFCTs is yet to be developed.

The computations in this paper are performed using double precision binary floating-point arithmetic obeying the IEEE Standard [18]. Some of the matrix notations used are inspired by the language of MATLAB^®. Some of the codes for numerical experiments are given so the results can be independently checked. We finally stress that sometimes, BFCTs reveal the underlying machine arithmetic of the used computing environment.

1.2. General Notations

Next, we use the following general notations.

• $\mathbb{Z}$ —set of integers;
• $\mathbb{Z}[m,n]$—set of integers $m,m+1,\dots ,n$, where $m\le n$;
• ${\mathbb{Z}}_m\subset \mathbb{Z}$—set of integers $\ge m$;
• $\mathbb{R}\left(\mathbb{C}\right)$—set of real (complex) numbers;
• $\mathrm{i}\in \mathbb{C}$—imaginary unit, where ${\mathrm{i}}^2=-1$;
• $\mathrm{sum}(a_k,n)$—sum of the scalar quantities $a_1,a_2,\dots ,a_n$;
• $\mathrm{int}(f,a,b)$—definite integral of the function $f:\mathbb{R}\to \mathbb{R}$ on the interval $[a,b]$;
• ≺—lexicographical order on the set of pairs $(a,b)\in {\mathbb{R}}^2$ defined as $(a_1,b_1)\prec (a_2,b_2)$ if $a_1<a_2$, or $a_1=a_2$ and $b_1<b_2$; we write $(a_1,b_1)⪯(a_2,b_2)$ if $(a_1,b_1)\prec (a_2,b_2)$, or $(a_1,b_1)=(a_2,b_2)$;
• $\mathbb{K}(m,n)$—set of $m\times n$ matrices A with elements $A(p,q)\in \mathbb{K}$, where $\mathbb{K}$ is one of the sets $\mathbb{Z}$, $\mathbb{R}$ or $\mathbb{C}$;
• $\mathbb{K}\left(m\right)=\mathbb{K}(m,m)$;
• $A(p,:)\in \mathbb{K}(1,n)$—pth row of $A\in \mathbb{K}(m,n)$;
• $A(:,q)\in \mathbb{K}(m,1)$—qth column of $A\in \mathbb{K}(m,n)$;
• $\parallel ·\parallel$—spectral norm in $\mathbb{K}(m,n)$;
• $\mathrm{Ones}(m,n)\in \mathbb{R}(m,n)$—matrix with unit elements;
• $\mathrm{rank}\left(A\right)\in {\mathbb{Z}}_0$—rank of the matrix $A\in \mathbb{K}(m,n)$;
• $\mathrm{tr}\left(A\right)\in \mathbb{K}$—trace of the matrix $A\in \mathbb{K}(m,n)$;
• $det\left(A\right)\in \mathbb{K}$—determinant of the matrix $A\in \mathbb{K}\left(m\right)$;
• $I_n\in \mathbb{R}\left(n\right)$—identity matrix with $I_n(p,p)=1$ and $I_n(p,q)=0$ for $p\ne q$;
• $E_{i,j}$—matrix with unique nonzero element, equal to 1, in position $(i,j)$;
• $N_n\in \mathbb{R}\left(n\right)$—nilpotent matrix with $N_n(p,p+1)=1$ and $N_n(p,q)=0$ for $q\ne p+1$;
• ${\lambda }_k={\lambda }_k\left(A\right)$—eigenvalues of the matrix $A\in \mathbb{K}\left(n\right)$, where $k=1,2,\dots ,n$; we usually assume that ${\lambda }_1⪯{\lambda }_2⪯\dots ⪯{\lambda }_n$;
• ${\sigma }_k={\sigma }_k\left(A\right)$—singular values of the matrix $A\in \mathbb{K}(m,n)$, where $k=1,2,\dots ,p$, $p=\min \{m,n\}$; the singular values are ordered as ${\sigma }_1\ge {\sigma }_2\ge \dots \ge {\sigma }_p\ge 0$;
• $\parallel A\parallel ={\sigma }_1$—spectral norm of the matrix $A\in \mathbb{K}(m,n)$;
• $\mathrm{cond}\left(A\right)=\parallel A\parallel \parallel A^{-1}\parallel ={\sigma }_1/{\sigma }_n\ge 1$—condition number of the non-singular matrix $A\in \mathbb{K}\left(n\right)$.

1.3. Machine Arithmetic Notations

We use the following notations from binary floating-point arithmetic, where codes and results in the MATLAB^® environment are written in typewriter font.

• ${\mathbb{M}}_{+}$$\left({\mathbb{M}}_{-}\right)$—set of finite positive (negative) machine numbers;
• $\mathbb{M}=M_{-}\cup \left\{0\right\}\cup {\mathbb{M}}_{+}$—set of finite machine numbers;
• $\mathrm{fl}\left(x\right)\in \mathbb{M}$—rounded value of $x\in \mathbb{R}$; $\mathrm{fl}\left(x\right)\in \mathbb{M}$ is computed as $m/2^n$ by the command sym(x,’f’), where $m\in \mathbb{Z}$, $n\in {\mathbb{Z}}_0$ and m is odd if $n\in {\mathbb{Z}}_1$;
• ${\mathbb{M}}_{+}$$\left({\mathbb{M}}_{-}\right)$—set of positive (negative) machine numbers; we have $\mathbb{M}={\mathbb{M}}_{-}\cup \left\{0\right\}\cup {\mathbb{M}}_{+}$;
• $X^{+}>X$ ($X^{-}<X$)—machine number right (left) nearest to $X\in \mathbb{M}$; $X^{+}$ is obtained as X + eps(X);
• $\mathrm{eps}=2^{-52}$—machine epsilon; we have $1^{+}=1+\mathrm{eps}$, $1^{-}=1-\mathrm{eps}/2$; $\mathrm{eps}$ is obtained as eps(1), or simply as eps;
• $\rho =\mathrm{eps}/2=2^{-53}\simeq 1.11\times {10}^{-16}$—rounding unit;
• $R=2^{53}=9007199254740992\simeq 9.01\times {10}^{15}$—maximal integer such that positive integers $\le R$ are machine numbers; we have $R^{+}=R+2$, $R^{-}=R-1$;
• $E=2^{1023}$—maximal machine integer degree of 2; we have $2E\notin \mathbb{M}$;
• $X_{\max }=(2-\mathrm{eps})\mathrm{E}\simeq 1.80\times {10}^{308}$—maximal machine number; it is also denoted as realmax and we have eps(realmax) = 2^971;
• $X_{\min }=2^{-1022}\simeq 2.23\times {10}^{-308}$—minimal positive normal machine number; it is also denoted as realmin;
• $R_{\min }=2\mathrm{eps}/\mathrm{E}=2^{-1074}\simeq 4.94\times {10}^{-324}$—minimal positive subnormal machine number; it is found as eps(0);
• $\mathrm{fl}\left(x\right)\in \mathbb{M}$—rounded value of $x\in \mathbb{R}$;
• $\mathcal{X}=[X_{\min },X_{max}]\subset \mathbb{R}$—normal range; numbers $x\in \mathcal{X}$ are rounded with relative error less than $\rho$;
• Inf (-Inf)—positive (negative) machine infinity;
• NaN—result of mathematically undefined operation, representing Not a Number;
• ${\mathbb{M}}_{\mathrm{ext}}=\mathbb{M}\cup \{\pm \mathrm{Inf}\}\cup \left\{\mathrm{NaN}\right\}$—extended machine set.

1.4. Software and Hardware

The computations were conducted with the program system MATLAB^®, Version 9.9 (R2020b), on a Lenovo X1 Carbon 7th Generation Ultrabook, Memory: 16 GB DDR4 2133 Mhz (2 × 8 GB DIMMs), Video: Intel HD 620, Wholesale China Products, Lenovo Laptop Manifacturer.

1.5. Problems with Reference Solution

An effective strategy for employing BFCTs involves solving CPRSs. CPRSs are specially designed computational problems with a priori known, or  reference, solutions. It is supposed that the numerical algorithm which is to be checked does not “know” this fact and solves the computational problem in the standard way. Consider, for example, the finite equation $f\left(x\right)=0$, where $f\left(x\right)$ and x are scalars or vectors. Then, a CPRS is obtained choosing $f\left(x\right)=g\left(x\right)-g\left(x_{\mathrm{ref}}\right)$, where $x_{\mathrm{ref}}$ is the reference solution. We usually choose $x_{\mathrm{ref}}$ to be a vector with unit elements, e.g., x_ref = ones(n,1) and $\parallel x_{\mathrm{ref}}\parallel =\sqrt{n}$.

Let $x_{\mathrm{ref}}\ne 0$ be a solution of the equation $f\left(x\right)=0$ and $x_{\mathrm{comp}}$ be an approximate solution computed in FPA. The relative error $\parallel x_{\mathrm{ref}}-x_{\mathrm{comp}}\parallel /\parallel x_{\mathrm{ref}}\parallel$ of the computed solution depends on the rounding of initial data and on errors made during the computational procedure [19,20]. To eliminate the effects of rounding initial data, we may consider CPRSs with relatively small integer data, or more generally, with data consisting of machine numbers.

Example 1.

Computational algorithms and software for solving linear algebraic equations $Ax=b$, where the matrix $A\in \mathbb{C}\left(n\right)$ is invertible, may be checked choosing $A\in \mathbb{Z}\left(n\right)$ and $b=A{x}_{\mathrm{ref}}$, where x_ref = ones(n,1). The computed solution is x_comp = A\b and we may compare the relative error

$${\displaystyle \frac{\parallel x_{\mathrm{ref}}-x_{\mathrm{comp}}\parallel }{\sqrt{n}}}$$

with the a priori error estimate eps∗cond(A).

Example 2.

Let $F:[a,b]\to \mathbb{R}$ be a differentiable highly oscillating function with derivative $F^{\prime }:[a,b]\to \mathbb{R}$. The numerical integration of $f=F^{\prime }$ may be a problem. For such integrands, we may formulate a numerically difficult computational problem $J=\mathrm{int}(f,a,b)$ with reference solution $J_{\mathrm{ref}}=1$. Systems with mathematical intelligence such as MATLAB^® [6], Maple [9] and Mathematica [10,11] may find the integral with high accuracy by the NLF $J=F\left(b\right)-F\left(a\right)$, thus avoiding numerical integration.

Example 3.

To check the codes for eigenstructure analysis of a matrix $A\in \mathbb{C}\left(n\right)$, one can select $A=T(I_n+N_n)T^{-1}$, where $T,T^{-1}$ are integer matrices with $det\left(T\right)=1$. The eigenvalues of A are equal to 1 (the reference solution). Computational algorithms such as the QR algorithm and software for spectral matrix analysis may produce wrong results even for matrices $A\in \mathbb{Z}\left(2\right)$ of moderate norm.

Example 4.

When solving the differential equation $y^{\prime }\left(t\right)=f(t,y\left(t\right))$ with initial condition $y\left(t_0\right)=y_0$, we may choose a reference solution $y_{\mathrm{ref}}$ as follows. The function $y_{\mathrm{ref}}$ satisfies the differential equation $y^{\prime }\left(t\right)=\Phi (t,y\left(t\right))$, where

$$\Phi (t,y\left(t\right))=f(t,y\left(t\right))+y_{\mathrm{ref}}^{\prime }\left(t\right)-f(t,y_{\mathrm{ref}}\left(t\right))$$

with initial condition $y_{\mathrm{ref}}\left(t_0\right)=y_0$.

An important observation is that for some computational problems such as integration and finding the extrema of functions, BFCTs may produce better results than certain sophisticated algorithms used in modern computer systems for calculating mathematics; see, e.g., Section 7.

2. Machine Arithmetic

2.1. Preliminaries

In this section, we consider computations in FPA obeying the IEEE Standard [18]. Numerical computations in MATLAB^® are calculated according to this standard. The FPA consists of a set of machine numbers $\mathbb{M}$ and rules for performing operations, e.g.,  addition/subtraction, multiplication/division, exponentiation, etc. Rounding and arithmetic operations in FPA are performed with an RRE of order $\rho$. The sum $A+B+C$ and the product $ABC$ are computed as $(A+B)+C$ and $\left(AB\right)C$.

A major source of errors in computations is the (catastrophic) cancellation when relatively exact close numbers are subtracted so that the information coded in their left-most digits is lost [21]. Thus, catastrophic cancellation is the phenomenon when subtracting good approximations to close numbers may cause a bad approximation to their difference. Less known is that genuine addition in FPA may also cause large and even unlimited errors.

A tutorial example of cancellation is solving quadratic equations $a{x}^2+bx+c=0$, $a\ne 0$. During the last 4000 years, students are taught to use the formula $x_{1,2}=(-b\pm \sqrt{d})/\left(2a\right)$, $d=b^2-4ac$, without warning that this may be a bad way to solve numerically quadratic equations. Similar considerations are valid for the use of explicit expressions for solving cubic and quartic algebraic equations. Genuine subtraction is numerically dangerous, and genuine addition in FPA is not less dangerous, although this fact is not very well known.

2.2. Violation of Arithmetic Laws

In FPA, the commutative law for addition and multiplication is valid, but the associative law for these operations is violated, i.e., it may happen that

$$(A+B)+C\ne A+(B+C),\left(AB\right)C\ne A\left(BC\right)$$

The distributive law $A(B+C)=AB+AC$ is also violated. This may lead to unexpectedly large (in fact, arbitrarily large) errors.

For example, we have an incorrect result $1+R-R=(1+R)-R=R-R=0$. The reason for this machine calculation is that $1+R$ is not a machine number (this is the smallest positive integer with this property). It lies in the middle of the successive machine numbers R and $2+R$ and is rounded to R. At the same time, we have $1+(R-R)=1+0=1$, which is the correct answer. Thus, we obtained the famous wrong equality $\begin{array}{|c|}\hline 0=1\\ \hline\end{array}$ arising in mathematical jokes.

This is not the end of the story. Consider the expression $S=R+1+1+\cdots +1-R$ consisting of $2+R$ members, where there are R members equal to 1. We have $S=R$. Computing S from left to right, we obtain the wrong answer $S=(R+1+1+\cdots +1)-R=R-R=0$. Computing S starting with the ones, i.e.,  $1+1+\cdots +1+R-R=R+R-R$ we obtain the correct answer $S=R$. Now, we have obtained the more impressive result $\begin{array}{|c|}\hline 0=9007199254740992\\ \hline\end{array}$. Summing (at least theoretically) a sufficient number of 1s, we compute the sum as Inf and hence $\begin{array}{|c|}\hline 0=\mathrm{Inf}\\ \hline\end{array}$ (note that the summation of arbitrarily large number of collectibles is not a problem for Turing-Post machines).

The associative law for multiplication is also violated in FPA. For example, the value of the expression $P=2^{512}\times 2^{512}\times 2^{-512}$ is $2^{512}$ and it is computed correctly as

$$P=2^{512}\times \left(2^{512}\times 2^{-512}\right)=2^{512}\times 1=2^{512}$$

At the same time, the machine computation of P from left to right gives

$$(2^{512}\times 2^{512})\times 2^{-512}=\mathrm{fl}\left(2^{1024}\right)\times 2^{-512}=\mathrm{Inf}\times 2^{-512}=\mathrm{Inf}$$

where $\mathrm{Inf}$ is the symbol for infinity in FPA. Thus, we obtained $\begin{array}{|c|}\hline 2^{512}=\mathrm{Inf}\\ \hline\end{array}$.

Similar false equalities are obtained by the addition and subtraction of a small number of machine numbers. Set $E=2^{1023}$. In standard arithmetic, we have $E+E-E-E=0$. In FPA, it is fulfilled $\mathrm{fl}(E+E-E-E)=\mathrm{fl}(E+E)-E-E=\mathrm{Inf}-E-E=\mathrm{Inf}$. This corresponds to the wrong equality $\begin{array}{|c|}\hline 0=\mathrm{Inf}\\ \hline\end{array}$ although E is a machine number at a good distance $E-2^{971}$ to $X_{max}$.

We also have $\mathrm{fl}(E+E-E-E-E-E)=\mathrm{fl}(E+E)-E-E-E-E=\mathrm{Inf}-E-E-E-E=\mathrm{Inf}$, although the exact result is $-2E$ and should be rounded to -Inf. Thus, we obtained $\begin{array}{|c|}\hline -\mathrm{Inf}=\mathrm{Inf}\\ \hline\end{array}$. The reason is that the maximum positive machine number in FPA is $X_{max}=(2-\mathrm{eps})\mathrm{E}$ and the number $E+E=2^{1024}$ is set to $\mathrm{Inf}$. Moreover, the maximum positive integer that is still rounded to $X_{\max }$ is $X_{max}+2^{969}(2-\mathrm{eps})$ while the number $X_{\max }+2^{970}$ is set to Inf.

These entertaining exercises may produce undetermined results as well. For example, we have $U=2^{1024}/2^{1024}-1=0$ but the computed value is U = Inf/Inf − 1 = NaN. This may be interpreted as the exotic equality $\begin{array}{|c|}\hline 0=\mathrm{NaN}\\ \hline\end{array}$.

Working with positive numbers $x\le X_{\min }$ also leads to violation of the distributive law $(x+y)z=xz+yz$. In particular, we usually think that for $x\in \mathbb{R}$, it is fulfilled $(x+x)/2=x/2+x/2=x$. But setting $x=X_{\min }$, we obtain (x + x)/2 = 2∗x/2 = x and x/2 + x/2 = 0 + 0 = 0. We stress that the minimal positive quantity that is still rounded to $X_{\min }$ instead to 0 is $X_{\min }/(2-\mathrm{eps})$.

The distributive law in the form (1/m + 1/n)/p = 1/m/p + 1/n/p, where at least one of the operands $m,n,p\in \mathbb{Z}$ is not an integer degree of 2, is also violated. In this case, the rounded values of both sides of the equality are usually neighboring machine numbers.

Example 5.

Let $m=2$, $n=3$, $p=5$. Then, $\mathrm{fl}(1/2/5+1/3/5)=\mathrm{fl}(1/6)+2^{-55}$ and $\mathrm{fl}(1/2/5-1/3/5)=\mathrm{fl}(1/30)+2^{-57}$.

Example 5 shows that the machine subtraction of close numbers $a,b$ is usually performed with a small relative error. Moreover, if the operands are machine numbers and $a/2\le b\le 2a$, then the machine subtraction is exact [21].

The rounded value $\mathrm{fl}\left(X\right)\in \mathbb{M}$ of $X\in \mathbb{R}$ is computed by the command sym(X,’f’) in the form $\pm m/2^n$, where $m,n\in {\mathbb{Z}}_0$ and m is odd when $n\ge 1$. In particular, the command sym(realmax,’f’) will give 309 decimal digits of the integer $X_{max}$ = realmax.

2.3. Numerical Convergence of Series

We shall conclude our brief excursion in the machine summation of numbers with the consideration of numerical series with positive elements $a_k$. For $n\in {\mathbb{Z}}_1$, set $\mathrm{sum}(a_k,n)=a_1+a_2+\cdots +a_n$.

Definition 1.

The symbol $\mathrm{sum}(a_k,\infty )$ is called a numerical series. The quantity $A_n=\mathrm{sum}(a_k,n)$ is the n-th partial sum of this series.

The partial sums are computed in FPA as $A_1=a_1$, $A_{m+1}=\mathrm{fl}(a_m+A_m)\in \mathbb{M}$, $m\in {\mathbb{Z}}_1$. Theoretically, the series $\mathrm{sum}(a_k,\infty )$ is divergent when $a_k\ge X_{\min }$. In FPA, there are three mutually disjoint possibilities according to the next definition.

Definition 2.

The series $\mathrm{sum}(a_k,\infty )$ adheres to the following:

1. 

Numerically convergent if there is a positive $S\in \mathbb{M}$ and $m\ge 1$ such that $A_n=S$ for $n\ge m$; the number S is called the numerical sum of the series $\mathrm{sum}(a_k,\infty )$;

2. 

Numerically divergent if there is $m>1$ such that $A_n=\mathrm{Inf}$ for $n\ge m$; in such cases, $A_n=\mathrm{Inf}$ for $n\ge m$;

3. 

Numerically undefined if there is $m>1$ such that $A_n\ne \mathrm{NaN}$ for $n\le m$ and $A_{n+1}=\mathrm{NaN}$.

Divergent numerical series may be numerically convergent. For example, the divergent harmonic series with $a_k=1/k$ is numerically convergent in FPA to a sum $S\simeq 36$. The divergent series with $a_k=1$ is numerically convergent to $S=R$. Also, it is easy to prove the following assertion.

Proposition 1.

For any $S\in {\mathbb{M}}_{+}$ there is a series $\mathrm{sum}(a_k,\infty )$ which is numerically convergent to S.

Proposition 1 for FPA is an analogue to the Riemann series theorem [22], which states that the members of a convergent real series which is not absolutely convergent may be reordered so that the sum of the new series is equal to any prescribed real number.

2.4. Average Rounding Errors

We recall that the normal range of FPA is the interval $\mathcal{X}=[R_{\min },X_{max}]\subset \mathbb{R}$ such that numbers $x\in \mathcal{X}$ are rounded with an RRE less than $\rho$.

Let $x\in \mathcal{X}$ and $\mathrm{fl}\left(x\right)\in \mathbb{M}$ be the rounded value of x. It is well known [18] that the RRE satisfies the estimate

$$\mathrm{rre}\left(x\right)={\displaystyle \frac{|x-\mathrm{fl}(x\left)\right|}{\left|x\right|}}\le {\displaystyle \frac{\rho }{1+\rho }}<\rho$$

(1)

The code sym(x,’f’) computes $\mathrm{fl}\left(x\right)$ in the form $m/2^n$, where $m\in \mathbb{Z}$ is odd if $n\ge 1$. Let $n\in \mathbb{Z}[0,2^{52}-1]$. The expression $\mathrm{rre}\left(x\right)$ between two successive machine numbers

$$X_n=1+2n\rho ,X_{n+1}=1+2(n+1)\rho$$

in the interval $[1,2)$ is as follows. Let

$$x_n={\displaystyle \frac{1}{2}}(X_n+X_{n+1})=1+(2n+1)\rho \notin \mathbb{M}$$

Then,

$$\mathrm{rre}\left(x\right)=\left\{\begin{array}{cc}\hfill 1-X_n/x& X_n\le x\le x_n\\ \hfill X_{n+1}/x-1& x_n<x\le X_{n+1}\end{array}\right.$$

The maximum value of $\mathrm{rre}\left(x\right)$ in the interval $[X_n,X_{n+1}]$ is achieved for $x=x_n$ and is equal to

$$e_n=\mathrm{rre}\left(x_n\right)={\displaystyle \frac{X_{n+1}-X_n}{X_n+X_{n+1}}}={\displaystyle \frac{\rho }{1+(2n+1)\rho }}$$

A part of the graph of the function $\mathrm{rre}:[1,2)\to \mathbb{R}$ is shown in Figure 1.

For $x>X_n$ near to $X_n$, the maximum RRE is about $\rho$, while for $x<X_{n+1}$ near to $X_{n+1}$, the maximum RRE is close to $\rho /2$. In particular, we have

$$\begin{array}{ccc}\hfill \mathrm{rre}(1+\rho )& =& {\displaystyle \frac{\rho }{1+\rho }}=\rho -{\rho }^2+\mathrm{O}\left({\rho }^3\right)\hfill \\ \hfill \mathrm{rre}(2-2\rho )& =& {\displaystyle \frac{\rho }{2(1-\rho )}}={\displaystyle \frac{1}{2}}\rho +{\displaystyle \frac{1}{2}}{\rho }^2+\mathrm{O}\left({\rho }^3\right)\hfill \end{array}$$

(2)

That actual behavior of RRE is not governed by (1) but is considerably smaller and has been known to specialists in computer calculations for a long time. To clarify this problem, we performed intensive numerical experiments.

BFCTs showed that the actual RRE is considerably lower than $\rho$ and this led to the definition and use of average RREs, which are more realistic in comparison to the relative error $\rho$. Based on this observation [23], we have defined and calculated an integral ARRE as the definite integral $\mathrm{int}(\mathrm{rre},1,2)$. This integral is of order

$$K\rho +\mathrm{O}\left({\rho }^2\right),K={\displaystyle \frac{1}{2}}log\left(2\right)\simeq 0.3466$$

(3)

At the same time, the average of the RRE maximums is

$$2K\rho =log\left(2\right)\rho \simeq 0.6931\rho .$$

Intensive numerical BFCT experiments [23] with the rounding of large sequences of numbers $x\notin \mathbb{M}$ had shown that the actual ARRE is about $0.40\rho$. This is 2.5 times less than the widely used bound $\rho$ in the literature [18]. In these experiments, numbers $x\in \mathbb{M}$ had been excluded, because $\mathrm{rre}\left(x\right)=0$. This explains why the multiplier of $\rho$ was computed as 0.40 instead of 0.35.

In another experiment, we have computed and averaged the RRE for fractions $x=1/k$, $x=1+1/k$ and $x=2-1/k$ for $k=1:m$ and m up to 1000, and we have computed the ratios $\mathrm{rre}\left(x\right)/\rho$. The results are shown in

Table 1. Ratios $\mathrm{rre}\left(x\right)/\rho$ for certain fractions.

m	$1/\mathit{m}$	$1+1/\mathit{m}$	$2-1/\mathit{m}$
10	$0.3000$	$0.2423$	$0.1950$
100	$0.3417$	$0.2443$	$0.2413$
1000	$0.3323$	$0.2649$	$0.2316$

. The actual RRE in rounding of unit fraction $1/m$ (second column) is very close to the integral ARRE (3).

For fractions $1+1/m$ close to 1, the computed ARRE is larger than for fractions $2-1/m$ close to 2. This confirms the observation that the rounding of $x\in [2^k,2^{k+1}]$, $k\in \mathbb{Z}$, is made with a lower (although not twice as low) RRE for x close to the left limit $2^k$ compared with close to the right limit $2^{k+1}$ of the interval; see (2). For example, the result in cell (4, 2) of Table 1 is obtained by the code

for m = 1:1000

 R(m) = double(abs((sym(1/m,’e’)−1/m))∗m∗2/eps);

end

mean(R)

We stress that in such experiments, we cannot use pseudo-random number generators such as rand, randn and randi from MATLAB^® since they produce machine numbers for which the rounding errors are zero.

We summarize the above considerations in the next important proposition.

Proposition 2.

The known rounding error bounds in all computational algorithms realized in FPA may be reduced three times (!) if the integral ARREs are used instead of the maximum RRE.

It must be pointed out that some widely used computer codes for evaluation of the rounding process have serious flaws. The MATLAB^® code sym(x,’e’) should compute the rounded value $\mathrm{fl}\left(x\right)\in \mathbb{M}$ of $x\in \mathbb{R}$ in the form $x+E\left(x\right)$ with signed absolute rounding error $E\left(x\right)$. However, for $x=2^m/1161$ and $m=-6:5$, the value of $\mathrm{fl}\left(x\right)$ is computed with a relative error of almost 100%.

3. Numerically Stable Computations

3.1. Lipschitz Continuous Problems

The numerical stability of computational procedures is a major issue in numerical analysis [19]. A large variety of computational problems (actually, almost all computational problems) may be formulated as follows; see [19,20]. Let $\mathcal{D}\subset \mathbb{C}(m,1)$ and $\mathcal{R}\subset \mathbb{C}(n,1)$ be given sets and let $f:\mathcal{D}\to \mathcal{R}$ be a continuous function. We shall use the following informal definition.

Definition 3.

The computational problem is described by the function f, while the particular way of computing the result $r=f\left(d\right)$ for a given data $d\in \mathcal{D}$ is the computational algorithm.

Thus, the computational problem is identified with the pair $(f,d)$, or with the equality $r=f\left(d\right)$. For many computational problems, the function f is Lipschitz continuous.

Definition 4.

The computational problem $r=f\left(d\right)$ is said to be (locally) Lipschitz continuous if there exist constants $L>0$ and $b>0$ such that

$$\parallel f\left(d_1\right)-f\left(d_2\right)\parallel \le L\parallel d_1-d_2\parallel$$

whenever $\parallel d_1\parallel ,\parallel d_2\parallel \le b$. The problem is globally Lipschitz continuous if $b=\infty$.

If the function f has a locally bounded derivative, it is (locally) Lipschitz continuous. The concept of Lipschitz continuity is illustrated by the next example.

Example 6.

Let $f:\mathcal{D}\to \mathbb{R}$, $\mathcal{D}\subset \mathbb{R}$, and $a>0$ be a given constant. Then, the following assertions take place.

1. 

The function $f\left(d\right)=1/d$ is Lipschitz continuous if $\mathcal{D}=(-\infty ,a]\cup [a,\infty )$ and is not Lipschitz continuous if $\mathcal{D}=\mathbb{R}\setminus \left\{0\right\}$.

2. 

The function $f\left(d\right)=d^{1/3}$ is Lipschitz continuous if $\mathcal{D}=(-\infty ,a]\cup [a,\infty )$ and is not Lipschitz continuous if $\mathcal{D}=\mathbb{R}\setminus \left\{0\right\}$.

3. 

The function $f\left(d\right)=d^m$, $m\in {\mathbb{Z}}_2$, is Lipschitz continuous if $\mathcal{D}=[-a,a]$ and is not Lipschitz continuous if $\mathcal{D}=\mathbb{R}$.

4. 

The function $f\left(d\right)=d^{4/3}sin(1/d)$ for $d\ne 0$ and $f\left(0\right)=0$, is differentiable on $\mathbb{R}$ but is not Lipschitz continuous.

Suppose now that the algorithm is realized in FPA, e.g., in the MATLAB^® computing environment.

Definition 5.

The computational algorithm $r=f\left(d\right)$ is said to be numerically stable if the computed result $r_{\mathrm{comp}}$ is close to the result $f\left(d^{\ast }\right)$ of a close problem with data $d^{\ast }$.

Definition 5 includes the popular concepts of forward and backwards numerical stability. To make this definition formal, suppose that there exist non-negative constants $A,B$ such that

$$\parallel r_{\mathrm{comp}}-f\left(d^{\ast }\right)\parallel \le \rho A\parallel r\parallel ,\parallel d-d^{\ast }\parallel \le \rho B\parallel d\parallel$$

and $d\ne 0$, $r\ne 0$. Further on, we have

$$\begin{array}{ccc}\hfill \parallel r_{\mathrm{comp}}-r\parallel & =& \parallel r_{\mathrm{comp}}-f\left(d^{\ast }\right)+f\left(d^{\ast }\right)-f\left(d\right)\parallel \hfill \\ & \le & \parallel r_{\mathrm{comp}}-f\left(d^{\ast }\right)\parallel +\parallel f\left(d^{\ast }\right)-f\left(d\right)\parallel \hfill \\ & \le & \rho A\parallel r\parallel +L\parallel d^{\ast }-d\parallel \le \rho A\parallel r\parallel +\rho BL\parallel d\parallel \hfill \end{array}$$

Dividing the last inequality by $\parallel r\parallel$, we obtain the following estimate for the relative error in the computed solution

$${\displaystyle \frac{\parallel r_{\mathrm{comp}}-r\parallel }{\parallel r\parallel }}\le \rho C,C=A+{\displaystyle \frac{BL\parallel d\parallel }{\parallel f\left(d\right)\parallel }}$$

(4)

Definition 6.

The quantity $C=C(A,B,d)$ is said to be the relative condition number of the computational problem $r=f\left(d\right)$.

The remarkable Formula (4) reveals the three main factors that determine the precision of the calculated solution [20,23].

Proposition 3.

The precision of the computed solution $r_{\mathrm{comp}}$ depends on the following factors.

1. 

The sensitivity of the computational problem to perturbations in the data d measured by the Lipschitz constant L of the function f.

2. 

The stability of the computational algorithm expressed by the constants $A,B$.

3. 

The FPA characterized by the rounding unit ρ and the requirement that the intermediate computed results belong to the normal range $\mathcal{X}$.

The error estimate (4) is used in practice as follows. For  a number of problems, the Lipschitz constant L may be calculated or estimated as in the solution of linear algebraic equations and the computation of the eigenstructure of a matrix with simple eigenvalues. Finally, the value of $\rho$ is known exactly for FPA as well as for other machine environments.

Estimation of the constants $A,B$ may be a problem. Often, the heuristic assumption $A=0$ and $B=1$ is made giving

$$C=C(f,d)={\displaystyle \frac{L\parallel d\parallel }{\parallel f\left(d\right)\parallel }}.$$

(5)

It follows from (5) that C will be large if L and/or $\parallel d\parallel$ are large and/or $\parallel f\left(d\right)\parallel$ is small. The constant L is usually outside the user’s control. The quantities $\parallel d\parallel$ and $\parallel f\left(d\right)\parallel$ may be changed by scaling of the computational problem.

Definition 7.

The computational problem is said to adhere to the following:

1. 

Well conditioned if $\rho C\ll 1$; in this case, we may expect about $-{log}_{10}\left(\rho C\right)$ true decimal digits in the computed solution;

2. 

Poorly conditioned if $\rho C\simeq 1$; in this case, there may be no true digits in the computed solution.

More precise classification of the conditioning (well, medium and poor) of computational problems is also possible. Computational problems that are not Lipschitz but Hölder continuous may also be analyzed: see, e.g., [23].

The above considerations confirm a fundamental rule in numerical computations, namely that if the data d and/or some of the intermediate results in the computation of $f\left(d\right)$ are large and/or the result $r=f\left(d\right)$ is small, then large relative errors in the computed solution $r_{\mathrm{comp}}$ can be expected; see Section 3.3. For some computational problems $r=f\left(d\right)$, there are a priory error estimates for the relative error $\parallel r_{\mathrm{comp}}-r\parallel /\parallel r\parallel$ in the computed solution $r_{\mathrm{comp}}$ in the form of explicit expressions in $\rho$ and d. The  importance of such estimates is that they may be obtained a priori before solving the computational problem.

Most a priory error estimates are heuristic. Among computational problems with such estimates is the solution of linear algebraic equations $Ax=b$ and the computation of the eigenvalues of a matrix A with simple spectrum. To check the accuracy and practical usefulness of such error estimates by BFCTs, a set of CPRSs is designed, and the error estimates are compared with the observed errors.

3.2. Hölder Continuous Problems

Definition 8.

The computational problem $r=f\left(d\right)$ is said to be Hölder continuous with exponent $h>0$ if there exist constants $L>0$ and $b>0$ such that

$$\parallel f\left(d_1\right)-f\left(d_2\right)\parallel \le L\parallel d_1-d_2{\parallel }^h$$

whenever $\parallel d_1\parallel ,\parallel d_2\parallel \le b$.

Hölder continuity implies uniform continuity, but the converse is not true. It is supposed that $h<1$. Indeed, if $h=1$, the problem is Lipschitz continuous, and if $h>1$, the function f is constant.

The machine computation of $r=f\left(d\right)$ may be performed with large errors of order ${\mathrm{eps}}^h$ when the exponent $h>0$ is small. Such cases arise in the calculation of a multiple zero $d_0$ of a smooth function $f:\mathbb{R}\to \mathbb{R}$, where $f^{\left(k\right)}\left(d_0\right)=0$, $k\in \mathbb{Z}[0,m-1]$ and $f^{\left(m\right)}\left(r_0\right)\ne 0$. In this case, $h=1/m$.

Let $r\ne 0$ and $r_{\mathrm{comp}}$ be the computed value of r. Then, a heuristic accuracy estimate for the solution of a Hölder problem is

$${\displaystyle \frac{\parallel r_{\mathrm{comp}}-r\parallel }{\parallel r\parallel }}\le H{\rho }^h,H=L{\displaystyle \frac{{\parallel d\parallel }^h}{\parallel r\parallel }}.$$

Experimental results confirming this estimate are presented later on. A typical example of a Hölder problem is the computation of the roots of an algebraic equation of n degree with multiple roots, where $n\ge 3$ (the case $n=2$ is treated by special algorithms). There are three codes in MATLAB^® that can be used for solving algebraic equations which may be characterized as follows.

• The code roots is fast and works on middle power computer platforms with equations of degree n up to several thousands but gives large errors in case of multiple roots. This code solves only algebraic equations.
• The code vpasolve works with VPA corresponding to 32 true decimal digits with equations of degree n up to several hundreds but may be slow in some cases. This code works with general equations as well finding one root at a time.
• The code fzero is fast but finds one root at a time. It may not work properly in case of multiple roots of even multiplicity. This code works with general finite equations as well.

3.3. Golden Rule of Computations

The results presented in this and previous sections confirm the golden rule of computing both exact and machine arithmetic, which may be formulated as follows.

Proposition 4 (Golden Rule of Computations).

If in a chain of numerical computations the final result is small relative to the initial data and/or to the intermediate results, then large relative errors in the computed solution are to be expected.

Note that the already mentioned catastrophic cancellation in subtraction of close numbers is a particular case of Proposition 4.

An important class of computations in FPA are the so-called computations with maximum accuracy. Let the vector r with elements $r_k$ be the exact solution of the vector computational problem $r=f\left(d\right)$ and let the vector $r_{\mathrm{comp}}$ with elements $r_k^{\ast }\in \mathbb{M}$ be the computed solution.

Definition 9.

The vector $r_{\mathrm{comp}}$ is computed with maximum accuracy if its elements $r_k^{\ast }$ satisfy $r_k^{\ast }\in \{r_k^{-},r_k^{+}\}$.

Solutions with maximum accuracy are the best that we may expect from a computational algorithm implemented in FPA.

4. Extremal Elements of Arrays

Finding minimal and maximal elements of real and complex arrays is a part of many computational algorithms. This problem may look simple, but it has pitfalls in some cases.

4.1. Vectors

Let $w=[w_1,w_2,\cdots ,w_n]$ be a given real n-vector. The problem of determining an extremal (minimal or maximal) element $w_k$ of w together with its index k arises as part of many computational problems. The  solution of this problem is obtained in MATLAB^® by the codes [U,K] = min(w) and [V,L] = max(w), where $U=w_K$ is the minimal element of the vector w and K is its index. Similarly, $V=w_L$ is the maximal element of the vector w and L is its index. These codes work for complex vectors as well if the complex numbers are ordered lexicographically by the relation ⪯; see [6].

An important rule here is that if there are several elements $w_k$ of w equal to its minimal element U, then the computed index K is supposed to be the minimal one. Similarly, if there are several elements $w_l$ of w equal to its maximal element V, then the computed index L is again the minimal one. The delicate point here is the calculation of the indexes K and L. For example, for $w=[c,c,c]$, where $c\in \mathbb{R}$ or $c\in \mathbb{C}$, the codes min and max produce $U=c$, $K=1$ and $V=c$, $L=1$ as expected.

Due to rounding, the above codes may produce unexpected results. Indeed, the performance of the codes depends on the form in which the elements of the vector w are written.

Example 7.

The quantities $A=1/3$, $B=4/3-1$ and $C=1/2-1/6$ are equal but their rounded values are different. The rounded value $\mathrm{fl}\left(X\right)\in \mathbb{M}$ of $X\in \mathbb{R}$ is computed by the code sym(X,’f’) . It produces the answer in the form $m{2}^{-n}$, where $m,n\in {\mathbb{Z}}_0$. We have $\mathrm{fl}\left(A\right)=6004799503160661/2^{54}$ and $\mathrm{fl}\left(B\right)=\mathrm{fl}\left(A\right)-2^{-54}$, $\mathrm{fl}\left(C\right)=\mathrm{fl}\left(A\right)+2^{-54}$. Hence, $\mathrm{fl}\left(B\right)<\mathrm{fl}\left(A\right)<\mathrm{fl}\left(C\right)$.

Example 8.

Let w be a 3-vector with elements $A,B,C$ at any positions, where $A=B=C$ are defined in Example 7. Then, the code [U,K] = min(w) will give $U=\mathrm{fl}\left(B\right)$ and K will be the index of B as an element of w. The code [V,L] = max(w) will give $V=\mathrm{fl}\left(C\right)$ and L will be the index of C as an element of w. At the same time, both codes are supposed to give $K=L=1$ and $U=V$.

These details of the performance of the codes min and max may not be very popular even among experienced users of sophisticated computer systems for mathematical calculations such as MATLAB^®.

4.2. Matrices

Similar problems as in Section 4.1 arise in finding extremal (minimal or maximal) elements of multidimensional arrays and in particular of real and complex matrices A.

Consider a matrix $A\in \mathbb{C}(m,n)$ with elements $A(p,q)$. Let the minimal element of A relative to the lexicographical order be $A_{\min }=A(i_1,i_2)$. Then, the code

[a,b] = min(A); [A_min,N] = min(a); A_min, ind = [b(N),N]

finds the minimal element $A_{\min }$ of A and the pair $(i_1,i_2)$ of its indexes as

A_min, ind = i_1  i_2

where i_1 = ind(1), i_2 = ind(2). Here, a and b are n-vectors and ind is a 2-vector.

If there is more than one minimal element, then the computed pair $(i_1,i_2)$ of its indexes is minimal relative to the lexicographical order. This result, however, may depend on the way the elements of A are specified.

Finding minimal elements of matrices may be a part of improved algorithms to compute the extrema of functions of two variables; see Section 7.2. A scheme for finding minimal elements of $nD$-arrays, $n>2$, may also be derived and applied to the minimization of functions of n variables.

4.3. Application to Voting Theory

The above details in using the codes min and max from MATLAB^® [6], although usually neglected, may be important. For example, in certain automatic voting computations (in which one of the authors of this paper had participated back in 2013) with the Hamilton method for the bi-proportional voting system used in Bulgaria, these phenomena actually occurred and caused problems. To resolve quickly the problem, we applied BFCTs using hand calculations (!) being in an emergency situation.

The computational algorithms for realization of the Hamilton method must be improved, replacing the fractional remainders by integer remainders [23] as follows. Let n parties with votes $v_1,v_2,\cdots ,v_n$ take part in the distribution of $S>n$ parliamentary seats by the Hamilton method [24]. Set $V=v_1+v_2+\cdots +v_n$ and $m_k=S{v}_k/V$.

Next, the rationals $m_k$ are represented as $\mathrm{fix}\left(m_k\right)+{\mu }_k/V$, where $\mathrm{fix}\left(x\right)\in {\mathbb{Z}}_0$ is the integer part of $m_k$, ${\mu }_k/V$ is the fractional remainder and ${\mu }_k\in \mathbb{Z}[1,V-1]$ is the integer remainder of $m_k$ modulo V. Initially, the kth party obtains $\mathrm{fix}\left(m_k\right)$ seats. If the sum $S_0$ of initial seats is less than S, then the first $S-S_0$ parties with largest integer remainders ${\mu }_k$ obtain one seat more. Thus, to avoid small errors in the computation of fractional remainders that may cheat the code max, we must work with exactly computed integer remainders ${\mu }_k$.

5. Problems with Reference Solutions

5.1. Evaluation of Functions

Evaluation of functions f defined by an explicit expression

$$y=f\left(x\right),0\ne x\in \mathbb{R},0\ne y\in \mathbb{R}$$

is a CPRS, since $f\left(x\right)$ may be found with high precision in a suitable computing environment. However, the relative error in computing y may be large even for well-behaved functions f and arguments x far from the limits of the normal range [23]. The reason is that instead with the argument $x\in \mathbb{R}$, the corresponding computational algorithm works with the rounded value $x^{\ast }=\mathrm{fl}\left(x\right)\in \mathbb{M}$. At the same time, for $x\in \mathbb{M}$, the computed quantity $f\left(x^{\ast }\right)$ is usually exact to working precision.

Example 9.

We have

[cos(realmax);sin(realmax)] =  −0.999987689426560

                  0.004961954789184

The computed result is correct to full precision, although the argument realmax is written with 309 decimal digits, which may be displayed by the command sym(realmax,’f’).

Let the function f be locally Lipschitz continuous with constant $L\left(x\right)$ in a neighborhood of x and let the result computed in FPA be $y^{\ast }=f\left(x^{\ast }\right)$. Then, the relative error of $y^{\ast }$ is estimated as

$${\displaystyle \frac{|y^{\ast }-y|}{\left|y\right|}}\le \rho C,C=C(f,x)={\displaystyle \frac{L\left(x\right)\left|x\right|}{\left|f\right(x\left)\right|}}$$

where $C(f,x)$ is the relative condition number of the computational problem $y=f\left(x\right)$.

Example 10.

Consider the vector $y=[cos(2k\pi );sin(2k\pi \left)\right]$, where $k\in \mathbb{Z}$. We should have $y=[1;0]$ but for large k, this may not be so. Let the vector $X\in \mathbb{R}(1,8)$ have elements $X\left(p\right)=\pi 2^{46+p}$, i.e., 

X = pi∗[2^47,2^48,2^49,2^50,2^51,2^52,2^53,2^54]

The command Y = [cos(X);sin(X)] gives the matrix

Y = 0.9999  0.9994  0.9976  0.9905  0.9622  0.8517  0.4509 −0.5934

   0.0172  −0.0345 −0.0689 −0.1374 −0.2723  −0.5240 −0.8926 −0.8049

While the first column Y(:,1) of Y still resembles the exact vector [1;0], the next columns are heavily wrong with Y(:,8) being a complete catastrophe with relative error 170%. The commands Y1 = [C1;S1] and Y2 = [C2;S2], where C1 = vpa(cos(X)), S1 = vpa(sin(X)), C2 = cos(vpa(X)) and S2 = sin(vpa(X)) produce the same wrong result Y1 = Y2 = Y.

The reader is advised to plot the graph of the eight functions $f\left(x\right)=sin(x+X)$, $x\in [-2\pi ,2\pi ]$, by the command syms x; fplot(sin(x+X)), for the vector X from Example 10. A way out from such wrong calculations is to evaluate standard $2\pi$-periodic trigonometric functions for arguments modulo $2\pi$.

Example 11.

The command

YY = [cos(mod(X,2∗pi));sin(mod(X,2∗pi))]

produces the matrix

YY =  1     1     1     1     1     1     1     1

     0     0     0     0     0     0     0     0

which is exact to full precision.

We recall that for $\left|x\right|\ge R$, the rounded value $\mathrm{fl}\left(x\right)$ of x is an even integer. For example, the quantity $a=23\ast {\pi }^{32}/20$ is rounded to $\mathrm{fl}\left(a\right)=9321670908397306$, which is demonstrated by the command sym(23∗pi^32/20,’f’). Another instructive example is rounding integer decimal degrees ${10}^n$.

Example 12.

We have $\mathrm{fl}\left({10}^n\right)={10}^n$ for $n\in \mathbb{Z}[16,22]$ but

sym([10^23;10^24;10^25],’f’) =  99999999999999991611392

                  999999999999999983222784

                 10000000000000000905969664

If $x\in \mathbb{M}$, then for most elementary and special functions f, the quantity $y=f\left(x\right)$ is computed with maximum accuracy.

If the function f is locally Lipschitz with constant $L\left(x\right)$, then large relative computational errors in the evaluation of $y=f\left(x\right)\ne 0$ may occur when $C(f,x)$ is large, i.e., when $L\left(x\right)$ and/or $\left|x\right|$ is large and/or $\left|y\right|$ is small. For the trigonometric functions above the quantities, $L\left(x\right)$ and $\left|y\right|$ are equal to 1, but the argument $x\notin \mathbb{M}$ is large.

A standard approach to improve the performance of computational algorithms is to scale the initial and/or intermediate data to avoid large errors. In  computing the matrix exponential $exp\left(A\right)$, $A\in \mathbb{C}\left(n\right)$, the scaling $A\to A{2}^{-m}$ is accomplished by a scaling factor $2^{-m}$, $m\in {\mathbb{Z}}_1$, which is a binary degree in order to reduce rounding errors [19].

5.2. Linear Algebraic Equations

An instructive example, which illustrates BFCTs to the solution of vector algebraic equations $Ax=b$ with integer data, is designed as follows. Let $A\in \mathbb{Z}\left(n\right)$ be invertible matrix and $b\in \mathbb{Z}(n,1)$. The command A = fix(100∗randn(n)) generates a matrix $A\in \mathbb{Z}\left(n\right)$ that is normally distributed with integer elements of moderate magnitude and zero mean (similar effect is achieved by the command randi). Choosing the reference solution as X_0 = ones(n,1), we obtain b = A∗X_0. The computed solution is X_comp = A\b with relative error E_n = norm(X_comp − X_0)/sqrt(n).

The a priori relative error estimate is est = eps∗cond(A). Usually, $E_n$ is a small multiple of $\mathrm{est}$, and this may be checked by BFCTs. We consider three types of computed solutions as follows.

• Standard solution X_1 = A\b obtained by Gaussian elimination with partial pivoting based on LR decomposition $PA=LR$ of A, where L is a lower triangular matrix with unit diagonal elements, R is an upper triangular matrix with nonzero diagonal elements, and P is a permutation matrix.
• Solution X_2 = R\(Q’∗b) obtained by QR decomposition [Q,R] = qr(A) of A, where $A=QR$, the matrix Q is orthogonal and the matrix R is upper triangular.
• Solution X_3 = inv(A)∗b based on finding the inverse $\mathrm{inv}\left(A\right)=A^{-1}$ of A.

The solution $X_1$ is computed as a default in MATLAB^® and is accurate and fast. The solution $X_2$ is very accurate although more expensive and is also used. The solution $X_3$ is not recommended and is included here for tutorial purposes. Indeed, computing inv(A) and multiplying the vector b by this matrix are unnecessary operations which are expensive and reduce accuracy.

The above computational techniques are illustrated by the commands

n = 1000; A = fix(100∗randn(n)); X_0 = ones(n); b = A∗X_0;…

X_1 = A\b; [Q,R] = qr(A); X_2 = R\(Q’∗b); X_3 = inv(A)∗b;…

E_1 = norm(X_0-X_1)/sqrt(n);E_2 = norm(X_0-X_2)/sqrt(n);…

E_3 = norm(X_0-X_3)/sqrt(n);est = eps∗cond(A);…

e_1 = E_1/est; e_2 = E_2/est; e_3 = E_3/est; [e_1;e_2;e_3]

Each execution of these commands produces different results because of the code randn. The computed quantity e_1 for systems with up to $n=1000$ unknowns is usually less than 20, the  quantity e_2 is about 40 and the quantity e_3 is less than 4. More precisely, intensive computations with $n=1000$ showed that $\mathrm{mean}\left(e_1\right)=18$, $\mathrm{mean}\left(e_2\right)=3$ and $\mathrm{mean}\left(e_3\right)=40$.

Tests with $n=5000$ had also been conducted, but they require more computational time. For such values of n, we have observed average values $\mathrm{mean}\left(e_1\right)=30$, $\mathrm{mean}\left(e_2\right)=3$, and $\mathrm{mean}\left(e_3\right)=150$.

The comparison of $e_1$ and $e_2$ with $e_3$ confirms, among others, the tutorial conclusion that solving linear vector algebraic equations by matrix inversion must definitely be avoided. Rather, the opposite is true. If the inverse matrix $B=A^{-1}$ is needed for some reason, it is found solving n equations $A{b}_k=i_k$ for the columns $b_k$ of B, where $i_k$ is the kth column of $I_n$.

Our main observation, based on BFCTs, featuring a comparison of the methods of Gaussian elimination and QR decomposition for solving linear algebraic equations, is formulated in the next proposition.

Proposition 5.

For matrices of order up to 1000, the QR decomposition method gives a relative error that is between 5 and 10 times less than the relative error of the Gaussian elimination method.

In addition, the solution of the equation $Ax=b$ via QR decomposition is backward stable, which is not always the case with Gaussian elimination with partial pivoting. Thus, Proposition 5 is a message to the developers of mathematical software such as MATLAB^®.

The Gaussian elimination method based on LR decomposition is preferred to the QR decomposition method because it requires half the floating-point operations. Since fast computational platforms are now widely available, this consideration is no longer valid.

Although the implementation of the code x = A\b gives good results for matrices $A\in \mathbb{R}\left(n\right)$ with n of order up to 1000, large errors in x may occur even for $n=2$ in some cases.

Example 13.

Consider the algebraic equation $A\left(m\right)x=b\left(m\right)$, where the matrices $A\left(m\right)\in \mathbb{R}\left(2\right)$ are defined by the relation (7) below. For $x_{\mathrm{ref}}=[1;1]$, we have $b\left(m\right)=[1-{10}^m;-1-{10}^m]$. The computed results for $m=2:5$ are presented in

Table 2. Errors and estimates for low order systems.

m	err	est	err/est
2	$3.1601\times {10}^{-9}$	$9.0612\times {10}^{-8}$	0.0349
3	$1.6691\times {10}^{-5}$	$8.9027\times {10}^{-4}$	0.0187
4	$2.4926\times {10}^{-1}$	9.1475	0.0272
5	1.0002	4.6283	0.2161

. In cases $m=4,5$, there is a warning that the matrix $A\left(m\right)$ is close to singular or badly scaled, since $\mathrm{cond}\left(A\right(m\left)\right)$ is larger than ${10}^{16}$. As shown in Table 2, for these two cases, the relative error is indeed 25% and 100%, respectively.

5.3. Eigenvalues of 2 by 2 Integer Matrices

The integer $2\times 2$ matrix

$$A=\left[\begin{array}{cc}\hfill -100009999& \hfill 100000000\\ \hfill -100020001& \hfill 100010001\end{array}\right]$$

(6)

with condition number $4.1\times {10}^{16}$ has double eigenvalue $\lambda =1$ and Jordan form $J_2=I_2+N_2$. The elements of A are integers of magnitude ${10}^8$. This is far from the quantity $R=2^{53}$ such that integers $n>R$ may not be machine numbers.

Example 14.

The MATLAB^® code e = eig(A) computes the column 2-vector e of eigenvalues of A as $e_{\mathrm{comp}}=[-0.3955;2.3955]$ instead of $e=[1;1]$. The relative error is 140%, and there is no true digit in the computed vector $e_{\mathrm{comp}}$ (even the sign of the first computed eigenvalue is wrong). The trace of A is fortunately computed as trace(A) = 2 and is the sum of the wrong eigenvalues. Next, the determinant of A (which is equal to 1) is computed wrongly in MATLAB^® as det(A) =  0.2214.

Surprisingly, the vector p of the coefficients of the characteristic polynomial of A, computed as p = poly(A), is [1,−2,−0.9475]. The last element $p\left(3\right)=-0.9475$ of the vector p has nothing in common with the computed determinant 0.2214, which may further confuse the user. Next, the Schur form $S=U^{\mathrm{H}}AU$ of A is computed by the MATLAB^® command [U,S] = schur(A) as

$$S=\left[\begin{array}{cc}\hfill -0.3955& \hfill 200020001.0000\\ \hfill 0.0000& \hfill 2.3955\end{array}\right]$$

and we have $S(1,1)S(2,2)=-0.9475$. This explains the computed value of $p\left(3\right)$.

Finally, we may try to compute $det\left(A\right)$ as $\pm {\sigma }_1{\sigma }_2$, where ${\sigma }_k$ represents the singular values of A. Based on the MATLAB^® code svd, the computed result is $det\left(A\right)=0.9711$, which is the best approximation to the exact value 1 of the determinant up to now. The same result for $det\left(A\right)$ is obtained by the QR decomposition of A by the MATLAB^® command [Q,R] = qr(A). The above results confirm the rule that SVD and QR decomposition are the best numerical tools to compute the determinant of a general matrix (if this determinant is ever needed). For general $2\times 2$ matrices, however, the most reliable way to compute the spectrum is a variant of the BFCT described below. Here, we recall the modern paradigm in numerical spectral analysis and root finding of polynomials: the eigenvalues of a matrix are not computed as roots of its characteristic polynomial but rather the contrary: the roots of a polynomial are computed as the eigenvalues of its companion matrix.

The Maple [9] code jordan(A), incorporated in MATLAB^®, computes correctly the Jordan form $I_2+N_2$ of A. The vector of eigenvalue condition numbers, computed by the MATLAB^® command condeig(A), is $[c,c]$, where $c\simeq 7.2\times {10}^7$ is relatively large. This indicates that something is wrong. In reality, things are even worse, since the 2-vector of eigenvalue condition numbers of the matrix A does not exist because A has only one linearly independent eigenvector. At this moment, the user may or may not be aware that something is wrong (with exception of the code jordan which is not very popular and is of restricted use). Now comes the time of BFCTs to find the spectrum and the Jordan form of A as follows.

The sum $e_1+e_2$ of the eigenvalues of A is equal to the trace $A(1,1)+A(2,2)=2$ which is computed exactly. If the eigenvalues are close or equal, because the eigenvalue condition numbers are large, then we should have $e_1+e_2\simeq 2e_1=2$ and $e_1=e_2=1$. The  Jordan form of A is then either $J_2$ or $I_2$. But the only matrix with Jordan form $I_2$ is $I_2$ itself, and hence the Jordan form of A must be $J_2$. We have obtained the correct result using BFCTs and simple logical reasoning.

The matrix (6) may be written as

$$A\left(m\right)=\left[\begin{array}{cc}1-a-a^2& a^2\\ -1-2a-a^2& 1+a+a^2\end{array}\right],a={10}^m$$

(7)

for $m=4$. Thus, we have a family of matrices $\left\{A\right(m\left)\right\}$ parametrized by the integer m. The trace of $A\left(m\right)$ is equal to 2, the determinant of $A\left(m\right)$ is equal to 1, the eigenvalues of $A\left(m\right)$ are $e_1=e_2=1$ and the Jordan form of $A\left(m\right)$ is $I_2+N_2$.

Below, we give the values of the coefficients $c_2=\mathrm{tr}\left(A\right)=2$ and $c_3=det\left(A\right)=1$ of the characteristic polynomial ${\lambda }^2-c_2\lambda +c_3$ of A as found by the code poly(A), and of the eigenvalues $e_1=e_2=1$ of A, as computed by the code eig(A) in MATLAB^®.

• For $m=2$, we have $c_2=2.0000$, $c_3=1.0000$ and $e_{1,2}=1.0000\pm 0.0001\mathrm{i}$, where the quantity 1.0000 has at least 5 true decimal digits and $0.0001\mathrm{i}$ is a small imaginary tail which may be neglected. This result seems acceptable.
• For $m=3$, we have $c_2=2.0000$, $c_3=1.0000$ and $e_1=0.9937$, $e_2=1.0063$. With certain optimism, these results may also be declared as acceptable.
• For $m=4$, a computational catastrophe occurs with $c_2=2.0000$ (true), $c_3=-0.9475$ (completely wrong) and $e_1=-0.3955$, $e_2=2.3955$ (also completely wrong). Here, surprisingly, $det\left(A\right)$ is computed as $0.2214$, which differs from 1 (which is to be expected) but differs also from $c_3$. It is not clear what and why had happened. Using the computed coefficients $c_2,c_3$, the roots of the characteristic equation ${\lambda }^2-c_2\lambda +c_3=0$ now are ${\lambda }_1=0.7709$, ${\lambda }_2=1.2291$ instead of the computed $e_1$ and $e_2$. This is a strange wrong result.
• We give also the results for $m=5$ which are full trash and are served without any warning to the user. We have $c_2=2.0000$, $c_3=5591.5$ and $e_{1,2}=1.0000\pm 74.770\mathrm{i}$. We also obtain $det\left(A\right)=-4472.2$ for completeness.

In all cases, the sum $c_2=2.0000$ of the wrongly computed eigenvalues is almost exact, which is a well known fact in the numerical spectral analysis of matrices [19]. The conclusion from the above considerations is that BFCTs in this case, in contrast to the standard software, always give $e_1\simeq e_2=c_2/2=1.0000$ with five true decimal digits. This is a good approximation to the eigenvalues $e_1=e_2=1$ of the matrices $A\left(m\right)$ for $m=3,4,5$ when the codes eig and poly fail.

The main reason for the large errors considered in this section is the bad conditioning of the integer $2\times 2$ matrices. Nevertheless, these errors are served without warning to the user. This should stimulate teaching computer methods and algorithms with an emphasis on cases where these tools produce wrong (sometimes completely wrong) results. This may also be a stimulus for producers of scientific software to warn users of the possible unreliable behavior of computer codes. Such behavior may be due to the particularities of FPA obeying the 2019 IEEE Standard but also to inappropriate algorithms and programming flaws such as in the code sym(1/1161,’e’).

6. Zeros of Functions

Computing zeros of functions and of polynomials in particular is a classical problem in numerical analysis. Although powerful computational algorithms for this purpose have been developed, the use of BFCTs in this area is still useful.

Consider first the scalar case. Calculating roots of equations $f\left(x\right)=0$, where $f:\mathbb{R}\to \mathbb{R}$ is a continuous function, in a given interval $T=[a,b]$, may be a difficult problem. We distinguish three main tasks.

• Find a single particular solution $r_1$ in the interval T.
• Find the general solution $f^{-1}\left(0\right)\cap T$ in the interval T.
• Find all roots $[r_1;r_2;\dots ;r_n]$ of a given n degree polynomial $f\left(x\right)=P\left(x\right)$.

The condition $f\left(a\right)f\left(b\right)<0$ guarantees that there is at least one solution in the interval $(a,b)$. Whether f has zeros at the points $a,b$ is checked by direct computation.

Solving real and complex polynomial equations $P\left(x\right)=0$ of n degree

$$P\left(x\right)=p_1{x}^n+p_2{x}^{n-1}+\dots +p_{n}x+p_{n+1}=0,c_1\ne 0,$$

is achieved by the code vpasolve(P(x) == 0) from MATLAB^®. It finds all roots of $P\left(x\right)$ with quadrupole precision of 32 true decimal digits. For $n=100$, the speed of the corresponding software on a medium computer platform is still acceptable. For $n=1000$ and higher, the performance of the code may be slow even on fast platforms, and hence it may not be applicable to RTA (if such computations are ever needed in RTA). Note that we write this at the beginning of year 2025.

A faster solution may be obtained by the code roots(p), where p is the vector of coefficients $p_1,p_2,\cdots ,p_{n+1}$ of $P\left(x\right)$. This code computes the column vector of roots $r=[r_1;r_2;\cdots ;r_n]\in \mathbb{C}(n,1)$ of $P\left(x\right)$ as the vector of eigenvalues of the companion matrix $\mathrm{C}\left(A\right)$ of $P\left(x\right)$ using the QR algorithm, i.e., r = eig(compan(p)). The code roots(p) works relatively fast for n up to several thousands but may produce wrong results even for small values of n when the polynomial has multiple roots and the computational problem is poorly conditioned. BFCTs allow to reveal the behavior of this solver.

It is known that errors in computing multiple roots of equations $f\left(x\right)=0$ in FPA, where f has a sufficient number of bounded derivatives around a root $x=r$, are of order $\gamma {\mathrm{eps}}^{1/n}$, where n is the multiplicity of r and $\gamma >0$ is a constant. To estimate $\gamma$, consider the equation $P\left(x\right)=0$ when the n degree polynomial $P\left(x\right)$ has n-tuple root $r_1=1$. Let

$$E_n=\parallel r_{\mathrm{comp}}-\mathrm{ones}(n,1)\parallel /\sqrt{n}$$

be the relative error in the vector $r_{\mathrm{comp}}$ computed by the code roots. The ratio ${\gamma }_n=E_n/{\left(\mathrm{eps}\right)}^{1/n}$ is shown in

Table 3. Constants in error estimates for code roots.

n	3	4	5	6	7	8	9	10	11
${\gamma }_n$	1.73	1.78	1.53	1.95	1.62	1.60	1.84	1.72	2.03
n	12	13	14	15	16	17	18	19	20
${\gamma }_n$	1.76	2.04	1.94	2.10	1.92	2.06	2.01	1.97	1.91

for the extended form $x^n-n{x}^{n-1}+\cdots +{(-1)}^n$ of the polynomial $P\left(x\right)={(x-1)}^n$ and values of n from 3 to 20. For  $n=2$, the computed roots are exact and ${\gamma }_2=0$.

The relative error in the computed solution for $n\le 20$ satisfies the heuristic estimate $E_n\le {\gamma }_n{\mathrm{eps}}^{1/n}$, where the average of ${\gamma }_n$ is $1.87$ or approximately $1.9$. The expanded form of the polynomial $P\left(x\right)={(x-1)}^n$ may be found by the command expand or asking the intelligent dialogue system WolframAlpha [11] to find the coefficients of this polynomial. We stress that for all n values, the code vpasolve(P(x) == 0) gives the exact solution vector $r=[1;1;\cdots ;1]$.

Usually, the computed roots are unacceptable if their relative error is greater than 1%. For example, it follows from Table 3 that the root $r=1$ of multiplicity 20 is computed by the code roots with a relative error of 32%, which is unacceptable. It must be stressed that this is not due to program disadvantages but rather to the high sensitivity of the eigenvalues of the companion matrix $\mathrm{C}\left(A\right)$. If the polynomial equation $P\left(x\right)=0$ has a root of multiplicity n, the relative error is estimated as $1.9{\mathrm{eps}}^{1/n}$. Solving the inequality $1.9{\mathrm{eps}}^{1/n}\le 0.01$, we obtain $n\ge 7$. We summarize these slightly unexpected observations as follows.

Proposition 6.

The errors in computing roots of n-degree polynomials by the code roots may be unacceptable when $n\ge 7$.

Consider finally the problem of finding all roots of the equation $f\left(x\right)=0$ in the interval $T=[a,b]$, i.e., finding the set $f^{-1}\left(0\right)\cap T$, where, e.g., $f\left(x\right)=xsin(cos(x\left)\right)-1$, $T=[2,12]$. Here, the use of BFCTs seems necessary. A direct approach is to plot the graph of the function $x\mapsto \left|f\right(x\left)\right|$, $x\in T$. Let N be a sufficiently large integer. We may use the commands X = linspace(a,b,N); Y = abs(f(X)); plot(X,Y) to plot the graph of the function $x\mapsto \left|f\right(x\left)\right|$, $x\in T$. The zeros of f are now marked as inverted peaks at the roots $r_1<r_2<r_3<\cdots$. In the latter case, we have $r_1\simeq 5$, $r_2\simeq 8$ and $r_3\simeq 11$. Next, the codes vpasolve(f,r_k) or fzero(f,r_k) may be used around each of the points $r_k$ to specify the roots with full precision. This approach is well known in the computational practice and is recommended in some MATLAB^® guides.

A difficult problem is the solution of nonlinear vector equations $f\left(x\right)=0$ and of matrix equations $F\left(X\right)=0$, where x and $f\left(x\right)$ are n-vectors, and X and $F\left(X\right)$ are $n\times n$ matrices. The MATLAB^® code fsolve is intended to solve nonlinear vector equations, while the codes are, axxbc, care, dare, dlyap, dric, lyap, ric, sylv and others solve linear and nonlinear matrix equations. The performance of all these codes may be checked by BFCTs and CPRSs. This a challenging problem which will be considered elsewhere.

7. Minimization of Functions

7.1. Functions of Scalar Argument

The minimization of the expression $y=f\left(x\right)$, $x\in T=[a,b]$, where $f:T\to \mathbb{R}$ is a continuous function, is accomplished in MATLAB^® by a variant of the method of the golden ratio. The corresponding code [x_0,y_0] = fminbnd(f,a,b) finds a local minimum in the interval T, where $y_0=f\left(x_0\right)$. To find the global minimum $[x_{\min },y_{\min }]$ of f in the interval T, we may use the graph of the function f plotted by the commands x = linspace(a,b,n); y = f(x); plot(x,y). Then, the approximate global minimum $[c,f(c\left)\right]$ of f is determined visually, considering also the pairs $[a,f(a\left)\right]$ and $[b,f(b\left)\right]$ at the end points of T. The code

[x_min,y_min] = fminbnd(f,c-h,c+h)

is then used in a neighborhood of the approximate minimum $[c,f(c\left)\right]$. This approach requires human interference in the process and may be avoided as shown later.

The problem with the code fminbnd is not that it may find local minimums. The real problem with this and similar codes is that they can miss the global minimum even when the function f is unimodal, i.e., when it has a unique minimum. BFCTs for such functions are a must. The problem is that usually, we do not know whether the function is unimodal or not and whether the computed minimum is minimum at all: the more so when performing complicated algorithms in RTA.

Example 15.

Consider the function

$$f\left(x\right)=2-x^{2}exp(1-x^2),x\in [0,\infty )$$

This function is unimodal with minimum $[x_{\min };y_{\min }]=[1;1]$. In some cases, this minimum is found as [x_min,y_min] = fminbnd(f,0,b) for some $b>0$. But in other cases, it is not found correctly, and here, we should use BFCTs. The code works well for $1\le b\le 16.85$ but gives a completely wrong result for $b\ge 16.86$ without warning. Indeed, we have the good result

[x_min;y_min] = fminbnd(f,0,16.85) =  0.999992123678799

                      1.000000000124073

and the bad result

[x_min;y_min] = fminbnd(f,0,16.86) = 16.859937887512295

                     2

The second result has a 1586% relative error for $x_{\min }$ and 100% relative error for $y_{\min }$.

The reason for the disturbing events in Example 15 is complicated. We may expect that $f\left(x\right)=2$ for values of x such that the expression $\phi \left(x\right)=x^{2}exp(1-x^2)$ is less than $\rho$ since then $f\left(x\right)=2-\phi \left(x\right)$ is rounded to 2. The root $r_0>0$ of the equation $\phi \left(x\right)=\rho$ is $r_0=6.38$, and we indeed have $\phi \left(r_0\right)=2$. But $r_0$ is much less then 16.86, and hence the code fminbnd works with a certain sophisticated version of VPA instead of with FPA. In particular, the quantity $\alpha =\phi \left(16.86\right)$ is much less than $\rho$ with $\alpha /\rho \simeq 2.46\times {10}^{-105}\ll 1$. We shall summarize the above considerations in the next statement.

Proposition 7.

The software for the minimization of functions of one variable such as fminbnd(f,a,b) from MATLAB^® may fail to produce correct results even for unimodular functions f.

In contrast, BFCTs can always be used to find the global minimum automatically. Let the function $f:T\to \mathbb{R}$ be unimodal. Consider the vectors X = linspace(a,b,n+1) with elements $X\left(k\right)=a+(k-1)(b-a)/n$ and $Y=f\left(X\right)$ with elements $Y\left(k\right)=f\left(X\right(k\left)\right)$. The MATLAB^® command [y,m] = min(Y) computes an approximation $\left[X\right(m),y]$ to the minimum $[x_{\min },y_{\min }]$. Finally, the (almost) exact minimum is computed between the neighboring elements of $X\left(m\right)$ as [x_min,y_min] = fminbnd(f,X(m−1),X(m+1)). This approach may be used to develop improved versions of the code fminbnd.

7.2. Functions of Vector Argument

For $n\ge 2$, the minimization of continuous functions $f:\mathbb{R}(n,1)\to \mathbb{R}$ of vector argument meets difficulties similar to those described in Section 7.1 and other specific difficulties as well. Unconstraint minimization $f\left(x\right)\to \min$ is accomplished in MATLAB^® by the Nelder–Mead algorithm [25,26] and realized by the code fminsearch [6]. We stress that the code finds a local minimizer $[x_{\min },y_{\min }]$ of the function f, where $y_{\min }=f\left(x_{\min }\right)$.

The function f may not be differentiable, which is an advantage of this algorithm. The algorithm compares the function values $f\left(x\right)$ at the vertexes of an $(n+1)$-simplex and then replaces the vertex with the highest value of f by another vertex. The simplex usually (but not always, as shown below) contracts on a minimum of f which may be local or global. The corresponding command is [x_min,y_min] = fminsearch(f,x_0), where $x_0$ is the initial guess. The computed result depends on the next five factors, which include the following:

• The computational problem;
• The computational algorithm;
• The FPA;
• The computer platform;
• The starting point $x_0$.

Factors 1–3 are usually out of the control of the user. Factor 4 is partially under control, e.g., the user may use another platform. Only factor 5, which is very important, is entirely under control. As mentioned above, the computed value $x_{\min }$ depends on the starting point $x_0$. This is denoted as $x_{\min }=\Psi \left(x_0\right)$.

Definition 10.

The point $x_0$ is said to be a numerical fixed point (NFP) of the computational procedure if $x_0=\Psi \left(x_0\right)$.

The computed result $x_{\min }$ is the NFP of the computational procedure. The NFP depends not only on the computational problem and the computational algorithm but also on the FPA and on the particular computer platform where the algorithm is realized. Unfortunately, computed values for $x_{\min }$ which are far from any actual minimum are often NFPs of the computational procedures, and they are served without warning to the user.

For a class of optimization problems, the function f has the form

$$f\left(x\right)=c+p\left(x\right)e\left(x\right),e\left(x\right)=exp\left(-a_1{x}_1^2-a_2{x}_2^2-\cdots -a_n{x}_n^2\right)$$

(8)

where $c>0$, $p\left(x\right)$ is a polynomial in x and $a_k$ are positive constants. Since $f\left(x\right)\to c$, when $\parallel x\parallel \to \infty$, the function (8) has (at least one) global minimum $[x_{\min },y_{\min }]$.

Numerical experiments with functions f of type (8) show that not only the computed valus for $x_{\min }$ and $y_{\min }$ depend on the initial guess $x_0$ but that in many cases, the code fminsearch produces a wrong solution for $x_{\min }$ which is far from any minimizer of the function f. The reason is that the value of $p\left(x\right)e\left(x\right)$ may become very small relative to c. Then, $f\left(x\right)$ is computed as c, and the algorithm stops at a point which is far from any minimizer of f. Usually, no warning is issued for the user in such cases. This may be a problem when the minimization code is a part of an automatic computational procedure which is not under human supervision. In such cases, application of the techniques of AI and MAI may be useful.

Example 16.

Let $n=2$ and consider the function of type (8)

$$f\left(x\right)=2-x_1{x}_{2}exp\left(1-{\displaystyle \frac{1}{2}}(x_1^2+x_2^2)\right)$$

i.e., $c=2$, $p\left(x\right)=-exp\left(1\right)x_1{x}_2$ and $a_1=a_2=1/2$. The function f is coded as

f = @(x)2-x(1)∗x(2)∗exp(1-x(1)^2/2-x(2)^2/2)

The function $y=f\left(x\right)$, $x\in \mathbb{R}\times \mathbb{R}$, has two global minimizers $x_{\min ,1}=[1,1]$ and $x_{\min ,2}=[-1,-1]$ for which $y_{\min ,1}=y_{\min ,2}=1.$ It also has two global maximizers $x_{max,1}=[1,-1]$ and $x_{max,2}=[-1,1]$ for which $y_{max,1}=y_{max,2}=3.$

Let the initial guess for the code [x_min,y_min] = fminsearch(f,x_0) be $x_0=[c,c]$. For $c=6.4308$, we obtain the relatively good result

x_min = −1.000041440968428   −0.999998218805417

y_min =  1.000000001720503

For the slightly different value $c=6.4309$, the computed result $x_{\min }=[c,c]$ is a numerical fixed point which is not a minimizer. Indeed, we have

x_min = 6.430900000000000   6.430900000000000

y_min = 2.000000000000000

The second computed result in Example 16 is a numerical catastrophe with 543% relative error in $x_{\min }$ and 100% relative error in $y_{\min }$. The reason is that $c^{2}exp(1-c^2)\le \rho$ for $c\ge 6.4391$ and $f\left(c\right)$ is rounded to 2. The conclusion is that the algorithm of the moving simplex uses FPA rather than VPA (compare with Example 15).

The minimization problem in Example 16 is not unimodal, since it has two solutions. Solving unimodal problems by the code fminsearch may also lead to large errors, as shown in the next example.

Example 17.

Let $n=2$ and consider the function of type (8)

$$f\left(x\right)=2-x_{1}exp\left({\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{2}}{x}_1^2-{(x_2-1)}^2)\right)$$

The function f is coded as

f = @(x)2−x(1)∗exp(1/2−x(1)^2/2−(x(2)−1)^2)

The minimization problem has unique solution $x_{\min }=[1,1]$, $y_{\min }=1$. For $c=5.81$, the code

[x_min,y_min] = fminsearch(f,[c,c])|

gives a quite acceptable result, namely

x_min =  1.000039724696251   1.000017661431510

y_min =  1.000000001889957

For a slightly different value of c, the computed result $x_{\min }=[c,c]$ is a numerical fixed point which is not a minimizer. Indeed, for $c=5.82$, the computed result is wrong, namely

x_min =  5.820000000000000   5.820000000000000

y_min =  2

There is a 482% relative error in the computed argument $x_{\min }$ and 100% relative error in the computed minimum $y_{\min }$.

The output of such situations is, at least for small n, to use BFCTs as follows. Let $n=2$ and let $f:T=T_1\times T_2\to \mathbb{R}$ be the function which has to be minimized, where $T_k=[a_k,b_k]\subset \mathbb{R}\times \mathbb{R}$. We choose positive integers $n_1,n_2$ and compute the grids

X_k = linspace(a_k,b_k,n_k)

and the quantities $A(i,j)=f(X_1\left(i\right),X_2\left(j\right)).$ Thus, we construct a matrix $F\in \mathbb{R}(n_1,n_2)$ which is a discrete analogue of the surface $z=f(x,y)$, $(x,y)\in T$.

Next, the minimal element $F(k_1,k_2)$ and its indexes $(k_1,k_2)$ are found by the BFCTs described in Section 4.2. Now, the approximate global minimum of f is $[X_{\min },Y_{\min }]$, where $X_{\min }\in \mathbb{R}(2,1)$ may be used as a starting point $x_0$ for the code fminsearch(f,x_0). Extensive numerical experiments show that the code now works properly.

The simplex method [25], utilized in fminsearch, lacks gradient use and may not ensure convergence to a local minimum as Example 17 demonstrates. To enhance robustness, alternative optimization techniques such as Differential Evolution, Simulated Annealing, and Trust-Region Methods may be considered for their potential to improve convergence and scalability. These techniques are promising due to their ability to navigate complex search spaces and avoid local and/or false minima, making them particularly beneficial in optimization problems with nonlinear constraints or multiple variables. To develop reliable codes for multivariate static optimization is still an actual task in modern computer methods. These problems shall be addressed in more detail elsewhere in connection with the use of BFCTs.

8. Canonical Forms of Matrices

8.1. Preliminaries

An interesting observation is that important properties of linear operators in n-dimensional vector spaces can be revealed for dimensions as low as $n=2$; see [27]. In this section, we consider some modified definitions of canonical forms of matrices and give instructive examples for the sensitivity of $2\times 2$ Jordan forms using BFCTs.

8.2. Jordan and Generalized Jordan Forms

In this section, we use some not very popular definitions of canonical forms of matrices, e.g., a definition of a Jordan form of a matrix, without using eigenvalues, eigenvectors and generalized eigenvectors.

Definition 11.

Let $n\in {\mathbb{Z}}_2$. The upper triangular bi-diagonal matrix $J\in \mathbb{C}\left(n\right)$ is said to be a Jordan matrix if

1. 

$J(k,k+1)\in \{0,1\}$

2. 

$J(k,k+1)=1$ implies $J(k,k)=J(k+1,k+1)$

3. 

$J(k,k)\ne J(k+1,k+1)$ implies $J(k,k+1)=0$

for $k=1,2,\dots ,n-1$. The set of Jordan matrices is denoted as $\mathbb{J}\left(n\right)$.

Example 18.

The set $\mathbb{J}\left(2\right)$ consists of matrices $\mathrm{diag}(\lambda ,\mu )$ and $\lambda I_2+E_{1,2}$, where $\lambda ,\mu \in \mathbb{C}$.

Definition 12.

The Jordan matrix J is spectrally ordered if $J(k,k)⪯J(k+1,k+1)$ for $k=1,2,\dots ,n-1$. The set of spectrally ordered Jordan matrices is denoted as $\tilde{\mathbb{J}}\left(n\right)$.

Definition 13.

The Jordan problem for a nonzero matrix $A\in \mathbb{C}\left(n\right)$ is to find a pair $(X_A,J_A)\in GL\left(n\right)\times \mathbb{J}\left(n\right)$ such that $X_A{J}_A=A{X}_A$. Here, $X_A$ is the transformation matrix and $J_A$ is the Jordan form of A.

We do not use the spectrum of A or the concept of Jordan blocks. Indeed, given a general matrix A (even with integer elements and of very low size as in (6)), then a priori, we know neither its spectrum nor its Jordan structure. Note that the Jordan form $J_A$ is not unique and is defined by the stacking of 1s on its super-diagonal unless $A=\lambda I_n$ when $J_A=A$ is unique and $X_A$ is any invertible matrix. The transformation matrix $X_A$ is always non-unique, since the matrix $\gamma X_A$, $0\ne \gamma \in \mathbb{C}$, is also a transformation matrix. The Jordan form may be defined uniquely to become a canonical Jordan form in terms of group theory [28] and integer partition theory [29], although this is rarely seen in the literature.

Example 19.

If the matrix $A\in \mathbb{C}\left(n\right)$ has a single eigenvalue, then there are $2^{n-1}$ orderings of the 1s on the $n-1$ positions $(k,k+1)$ of the super-diagonal of A.

Definition 14.

Let $J\in \mathbb{J}\left(n\right)$. The matrix $X\in GL\left(n\right)$ is said to be a stabilizer of J if $X^{-1}JX\in \mathbb{J}\left(n\right)$. The set of stabilizers of A is denoted as $\mathrm{Stb}\left(A\right)\subset GL\left(n\right)$. The matrix $I_n$ is called the center of $\mathrm{Stb}\left(A\right)$.

Note that $\mathrm{Stb}\left(A\right)$ is not necessarily a group unless $A=\lambda I_n$, and  $\left\{I_n\right\}$ is not a center in the group-theoretical sense.

Example 20.

Let $n=2$ and $A\in \mathbb{J}\left(2\right)$. Then, the following three cases are possible.

1. 

If $A=\lambda I_2$, then $\mathrm{Stb}\left(A\right)$ is the group $GL\left(2\right)$.

2. 

If $A=\mathrm{diag}({\lambda }_1,{\lambda }_2)$, ${\lambda }_1\ne {\lambda }_2$, then $\mathrm{Stb}\left(A\right)$ is the set of matrices $\mathrm{diag}(a_1,a_2)$ and $a_1{E}_{1,2}+a_2{E}_{2,1}$, where $a_1{a}_2\ne 0$.

3. 

If $A=\lambda I_2+E_{1,2}$, then $\mathrm{Stb}\left(A\right)$ is the set of matrices $a(E_{1,2}+E_{2,1})$, $a\ne 0$.

Definition 15.

The upper triangular bi-diagonal matrix $G\in \mathbb{C}\left(n\right)$ is said to be a generalized Jordan matrix if

1. 

$G(k,k+1)\ne 0$ implies $G(k,k)=G(k+1,k+1)$;

2. 

$G(k,k)\ne G(k+1,k+1)$ implies $G(k,k+1)=0$.

The set of generalized Jordan matrices is denoted as $\mathbb{G}\left(n\right)$.

Generalized Jordan matrices G have the structure of Jordan matrices, where the nonzero elements $G(k,k+1)$ (if any) are not necessarily equal to 1. Hence, these matrices are less sensitive to perturbations in the matrix A.

Example 21.

The elements of $\mathbb{G}\left(2\right)$ are $\mathrm{diag}(\lambda ,\mu )$ and $\lambda I_2+a{E}_{1,2}$, where $\lambda ,\mu ,a\in \mathbb{C}$ and $a\ne 0$.

Definition 16.

The generalized Jordan problem for a nonzero matrix $A\in \mathbb{C}\left(n\right)$ is to find a pair $(Y_A,G_A)\in GL\left(n\right)\times \mathbb{G}\left(n\right)$ such that $Y_A{G}_A=A{Y}_A$. Here, $Y_A$ is the transformation matrix and $G_A$ is the generalized Jordan form of A.

Consider now perturbations of Jordan forms. Let $A\left(\epsilon \right)=A_0+\epsilon A_1$, where $A_0,A_1\in \mathbb{C}\left(n\right)$, $\epsilon \in (-{\epsilon }_0,{\epsilon }_0)$ and ${\epsilon }_0>0$ is a small parameter. If the matrix $A_0$ has multiple eigenvalues, then the transformation matrix $X\left(\epsilon \right)=X_{A\left(\epsilon \right)}$ and the Jordan form $J\left(\epsilon \right)=J_{A\left(\epsilon \right)}$ of $A\left(\epsilon \right)$ may be discontinuous at the point $\epsilon =0$. If the matrix $A_0$ has simple eigenvalues, then the matrices $X\left(\epsilon \right)$ and $J\left(\epsilon \right)$ depend continuously on $\epsilon$. To illustrate these facts by BFCTs, consider matrices $A\in \mathbb{C}\left(2\right)$. For such matrices A with rational elements, the matrices $X_A$ and $J_A$ are computed by the MATLAB^® command [X_A,J_A] = jordan(A) exactly.

Example 22.

Let $A_0=I_2$, $A_1=E_{2,1}$ and $\epsilon =2^{-m},m\in {\mathbb{Z}}_1$. Then, $X\left(\epsilon \right)=E_{2,1}+E_{1,2}/\epsilon$ and $J\left(\epsilon \right)=I_2+E_{1,2}$. The command [X,J] = jordan(A) computes the matrices $X\left(\epsilon \right)$, $J\left(\epsilon \right)$ exactly for $m\le 1074$. For $m\ge 1075$, the matrices $X_A$ and $J_A$ are computed as $I_2$, since ε is rounded to zero.

Example 23.

Consider the matrix $A=I_2+E_{1,2}$ and let $A\left(\epsilon \right)=I_2+\epsilon E_{1,1},\epsilon =2^{-m},m\in {\mathbb{Z}}_1$. Then, the Jordan forms of $A\left(\epsilon \right)$ seem to be $J_1\left(e\right)=\mathrm{diag}(1+\epsilon ,1)$ and $J_2\left(\epsilon \right)=\mathrm{diag}(1,1+\epsilon ).$ The form $J_1\left(\epsilon \right)$ is unachievable. For the form $J_2\left(\epsilon \right)$, we have infinitely many transformation matrices $X_2\left(\epsilon \right)$, e.g., $X_2\left(\epsilon \right)=a{E}_{1,2}-E_{2,1}-{\epsilon }^{-1}{E}_{1,1}$, where $0\ne a\in \mathbb{C}$ is arbitrary. The code [X,J] = jordan(A) computes exactly the matrices $X_2\left(\epsilon \right)$ (with $a=1$) and $J_2\left(\epsilon \right)$ for $\epsilon =2^{-m}$ and $m\le 44$. For $m=45$, these matrices are computed wrongly as $X_2\left(\epsilon \right)=I_2$ and $J_2\left(\epsilon \right)=I_2$.

Proposition 8.

A Jordan form J of a matrix A with multiple eigenvalues may be sensitive to perturbations in A due to three main restrictions:

1. 

Because $J(i,j)=0$ for $i\ge j+1$;

2. 

Because $J(i,j)=0$ for $j\ge i+2$ (if $n\ge 3$);

3. 

Because the nonzero elements $J(i,i+1)$ (if any) are equal to 1.

A generalized Jordan form of A is usually less sensitive to perturbations in A, since restriction 3 is absent. A Schur form is even less sensitive, since both restrictions 2 and 3 are absent. A generalized Schur form is least sensitive since, in addition, any of its diagonal blocks is either upper triangular or lower triangular [30]. All these observations are confirmed in practice by BFCTs.

8.3. Condensed Schur Forms

Schur forms $S_A$ of matrices A satisfy only the condition $S_A(i,j)=0$ for $i\ge j+1$ and are thus less sensitive to perturbations in A. Nevertheless, they may be discontinuous in a neighborhood of a matrix with multiple eigenvalues. The case of simple eigenvalues is studied in more detail; see, e.g., [30].

Definition 17.

The matrix $S\in \mathbb{C}\left(n\right)$ is said to be a Schur matrix if it is upper triangular. The set of Schur matrices is denoted as $\mathbb{S}\left(n\right)$.

We recall that we study only cases $n\ge 2$. Now, it is easy to see that $\mathbb{S}\left(n\right)=\mathbb{G}\left(n\right)$ if and only if $n=2$.

Each matrix $A\in \mathbb{C}$ is unitarily similar to a Schur matrix $S_A\in \mathbb{S}\left(n\right)$ such that $U_A{S}_A=A{U}_A^{\mathrm{H}}$ for some matrix $U_A\in U\left(n\right)$. The matrix $U_A$ is the transformation matrix and the matrix $S_A\in U\left(n\right)$ is the condensed Schur form of A. The Schur problem for A is to describe the set of pairs $(U_A,S_A)$ and to study their sensitivity relative to perturbations in A.

Although less sensitive to perturbations in A, the solution $(U_A,S_A)$ of the Schur problem may also be sensitive and even discontinuous as a function of A at points $A_0$, where the matrix $A_0$ has multiple eigenvalues.

Let $A=\lambda I_2$ and $\epsilon E$ be a perturbation in A. The Schur form of A is A itself, and we may consider the trivial (or central) solution $(I_2,A)$ of the Schur problem for A. For arbitrary small $\epsilon \ne 0$, the Schur forms of $A+\epsilon E_{2,1}$ are $S=I_2\pm \epsilon E_{1,2}$ with transformation matrices $V=\pm E_{1,2}\pm E_{2,1}$. Thus, the transformation matrix jumped from $U=I_2$ to $U=V$ for $\epsilon \ne 0$.

Example 24.

Let $m=50$, $a=2^{-m}$ and $A=[1,0;a,1]$. The command [X,J] = schur(A) gives the correct answer

X =  0   −1       J = 1.000000000000000  −0.000000000000001

    1     0         0           1.000000000000000

For $m=51$, we obtain the wrong answer $X=J=I_2$ due to rounding errors. Obviously, the code schur works with FPA in contrast to jordan, which uses VPA.

9. Least Squares Revisited

9.1. Preliminaries

In least squares methods (LSMs), the aim is to minimize the sum of squared residuals. A not very popular fact is that smaller residuals may correspond to larger errors [31,32]. Moreover, this phenomenon may be observed on an infinite set of residuals and errors. This may have far-reaching consequences, so the LSM has to be revisited, taking into account such cases. Note that a least squares problem (LSP) may arise naturally in connection with a given approximation scheme or as an equivalent formulation of a zero finding method.

9.2. Nonlinear Problems

The solution of the (overdetermined) equation $f\left(x\right)=0$, where x and $f\left(x\right)$ are vectors and f is a continuous function, may be reduced to minimization of the quantity ${\parallel f\left(x\right)\parallel }^2$. Such minimization problems arise also in a genuinely optimization statement in, e.g., approximation of data.

Let $\phi :\mathbb{C}(n,1)\to {\mathbb{R}}_{+}$ be a continuous function and let the problem $\phi \left(x\right)\to \min$ be unimodal, i.e., there is a unique (global) minimum $[x_0;y_0]\in \mathbb{C}(n,1)\times {\mathbb{R}}_{+}$, where $y_0=\phi \left(x_0\right)$. Denote by $e_k=\parallel x_k-x_0\parallel$ the distance between the vectors $x_k$ and $x_0$ or the absolute error of the vector $x_k$ when $x_k$ is interpreted as the computed solution.

Let $n=1$ and $\phi :\mathbb{R}\to {\mathbb{R}}_{+}$ and let $\left(x_k\right)$ be a sequence of computed solutions. Then, it is possible that $r_k=\phi \left(x_k\right)$ tends to 0 and $e_k$ tends to ∞ when $k\to \infty$, as the next example shows.

Example 25.

Let

$$\phi \left(x\right)=1+{\displaystyle \frac{{(x-1)}^2}{1+{(x-1)}^4}},x\in \mathbb{R}$$

The function φ is unimodal with minimum $[x_0,y_0]=[1,1]$. Let $x_k=k$. Then, $r_k=\phi \left(x_k\right)\to 0$ and $e_k=|k-1|\to \infty$ as $k\to \infty$.

This result is easily extended to the case $n\ge 2$.

Example 26.

Let

$$\phi \left(x\right)=1+{\displaystyle \frac{{(\parallel x\parallel -1)}^2}{1+{(\parallel x\parallel -1)}^4}},x\in \mathbb{R}(n,1)$$

The function φ has infinitely many minimums $[x_0,1]$ on the unit sphere $\parallel x_0\parallel =1$. Let $x_k=k[1;0;\cdots ;0]$ for $k\in {\mathbb{Z}}_1$, i.e., $\parallel x_k\parallel =k$. Then, $r_k\to 0$ and $e_k\to \infty$ as $k\to \infty$.

Thus, the residual $r_k$ and the error $e_k$ may have opposite behavior. This phenomenon, namely larger the remainder $r_k$, the smaller the error $e_k$, is known as Remainders vs. Errors; see [32]. It may concern the stopping rule in the implementation of LSM.

9.3. Linear Problems

Consider the matrix $A\in \mathbb{R}(m,n)$ of full column rank $n<m$ and let $b\in \mathbb{R}(m,1)$ be a given vector which is not in the range of A. The linear least squares problem (LLSP) is to find the vector $x\in \mathbb{R}(n,1)$ such that $r\left(x\right)=\parallel Ax-b\parallel \to \min$. This problem is unimodal and its theoretical solution is $A^{†}b$, where $A^{†}={\left(A^{\top }A\right)}^{-1}{A}^{\top }\in \mathbb{R}\left(n\right)$ is the pseudo-inverse of the matrix A. This representation of the solution is not used in computational practice, since it is ineffective and may be connected with a substantial loss of accuracy. Instead, the solution is found by, e.g., QR decomposition $A=QR$ of the matrix A. Let $R=[S;T]$ and $Q^{\top }b=[c;d]$, where the matrix $S\in \mathbb{R}\left(n\right)$ is upper triangular and invertible, and  $c\in \mathbb{R}(n,1)$. The solution $x^{\ast }$ is then obtained by the code x = A\b, which in fact computes x = S\c.

Intensive experiments based on BFCTs with random data show that the error of the “bad” solution $A^{†}b$ is between 5 and 10 times larger than the error of the “good” solution $x^{\ast }=A\setminus b$. In addition, finding $x_0$ requires many more computations.

Let $x_1,x_2\in \mathbb{R}(n,1)$ be two approximate solutions of the above LLSP which are different from the exact solution $x_0$. Set $r_k=\parallel A{x}_k-b\parallel$ and $e_k=\parallel x_k-x_0\parallel$, $k=1,2$. We have ${\sigma }_n\le r_k/e_k\le {\sigma }_1$, where ${\sigma }_1=\parallel A\parallel$ and ${\sigma }_n=1/{\parallel A^{-1}\parallel }^{-1}$ are the maximal and minimal singular values of A. Next, it is possible [32] to choose $x_k$ so that $r_1/e_1={\sigma }_1$ and $r_2/s_2={\sigma }_n$. It is fulfilled

$${\displaystyle \frac{r_1}{r_2}}=\mathrm{cond}\left(A\right){\displaystyle \frac{e_1}{e_2}},\mathrm{cond}\left(A\right)={\displaystyle \frac{{\sigma }_1}{{\sigma }_n}}.$$

(9)

when the matrix A is poorly conditioned, i.e.,  $\mathrm{cond}\left(A\right)\gg 1$, the behavior of remainders and errors is opposite: the larger the remainder, the smaller the error. This fact deserves a separate formulation.

Proposition 9.

When $\mathrm{cond}\left(A\right)$ is large, larger remainders may correspond to smaller errors. In this case, checking the accuracy of the computed solutions by the remainders is misleading.

The situation with remainders and errors is ironic. If the matrix A is well conditioned with $\mathrm{cond}\left(A\right)$ close to 1, the computed approximations are most probably good, and there is no need to check their accuracy by remainders. But if the matrix A is poorly conditioned and the accuracy test seems necessary, the accuracy check by remainders may lead to wrong conclusions.

Example 27.

Consider the simplest case $n=2$ and $A=[B;0]\in \mathbb{R}(m,2)$, $b=0$, where $B=[{\mu }^3,-1;{\mu }^3,1]\in \mathbb{R}\left(2\right)$ and $\mu >0$ is a small parameter. The only solution of the LSP is $x_0=0$. Let $x_1=[0;\mu ]$ and $x_2=[1/\mu ;0]$ be two approximate solutions (the vector $x_2$ with error $e_2=1/\mu \gg 1$ can hardly be recognized as an approximate solution, but we shall ignore this fact). We have $r_1=\mu \sqrt{2}$, $e_1=\mu$, $r_2={\mu }^2\sqrt{2}$, $e_2=1/\mu$ and

$${\displaystyle \frac{r_1}{r_2}}={\displaystyle \frac{1}{\mu }}\gg 1,{\displaystyle \frac{e_1}{e_2}}={\mu }^2\ll 1$$

This is possible, since the matrix B is very badly conditioned with $\mathrm{cond}\left(B\right)={\mu }^{-3}\gg 1$ and the estimate (9) is achieved.

Denote $T=(0,1)\subset {\mathbb{R}}_{+}$ and let $\left\{x\right(t):t\in T\}\subset \mathbb{R}(n,1)$ be a family of approximate solutions of the equation $Ax=b$, where the function $x:T\to \mathbb{R}(n,1)$ is differentiable. Let $x\left(t\right)$ tends to $x_0$ as $t\to \infty$ and denote by $e\left(t\right)=\parallel x\left(t\right)-x_0\parallel$ and $r\left(t\right)=\parallel Ax\left(t\right)-b\parallel$ the error and the remainder of the approximate solution $x\left(t\right)$. These relations define the remainder r as a parametric function of e and vice versa.

Proposition 10.

There exists a smooth function $x:T\to \mathbb{R}(n,1)$ such that the following assertions hold true.

1. 

The function $e:T\to {\mathbb{R}}_{+}$ is smooth and decreasing.

2. 

The function $r:T\to {\mathbb{R}}_{+}$ is smooth and non-monotone in each interval $(0,\beta )$, where $0<\beta <1$.

3. 

The function $e\mapsto r\left(e\right)$ (whenever defined) is smooth and non-monotonic in each arbitrarily small subinterval $(0,\theta )$ of the interval $(0,\epsilon )$ where $\epsilon =sup\left\{e\right(t):t\in T\}$.

Example 28.

Let $n=2$,

$$A=\left[\begin{array}{cc}a& 0\\ 0& 1\end{array}\right],x\left(t\right)=\left[\begin{array}{c}{\xi }_1\left(t\right)\\ {\xi }_2\left(t\right)\end{array}\right],b=\left[\begin{array}{c}0\\ 0\end{array}\right]$$

and $a\notin \{-1,0,1\}$. Then, $x_0=0$ and

$$r\left(t\right)=\sqrt{a^2{\xi }_1^2\left(t\right)+{\xi }_2^2\left(t\right)},e\left(t\right)=\sqrt{{\xi }_1^2\left(t\right)+{\xi }_2^2\left(t\right)}$$

Let $t\in T$ and ${\xi }_1\left(t\right)=t^{3}cos(1/t)$, ${\xi }_2\left(t\right)=t^{3}sin(1/t)$. We have $e\left(t\right)=t^3$ and

$$r\left(t\right)=t^3\sqrt{1+(a^2-1){cos}^2(1/t)}$$

Therefore,

$$r\left(e\right)=e\sqrt{1+(a^2-1){cos}^2\left(e^{-1/3}\right)}$$

The differentiable function $e\mapsto r\left(e\right)$ is not monotone at each interval $(0,\theta )$

This non-monotonicity means that for any $\theta >0$, there exist infinitely many points $e_1,e_2\in (0,\theta )$ with $e_1<e_2$ and $r\left(e_1\right)>r\left(e_2\right)$. Thus, the LSM may be misleading not only for linear algebraic equations but also for LLSPs for which the method had been specially designed.

10. Integrals and Derivatives

10.1. Integrals

The calculation of the definite integral $I=\mathrm{int}(f,a,b)$ is one of the oldest computational problems in numerical mathematical analysis. According to [33], a variant of the trapezoid rule was used in Babylon 50 years BCE for integrating the velocity of Jupiter along the ecliptic.

Usually, the integral is solved for $a,b\in \mathbb{R}$, $a<b$, but the case when one or both of the limits a, b are infinite and the integral is improper is also considered. There are many sophisticated algorithms and computer codes for solving such integrals; see [1]. In MATLAB^®, definite integrals of functions of one variable $f:[a,b]\to \mathbb{R}$ are computed by the command integral(f,a,b) or by a variant of the symbolic command int.

In this section, we apply BFCTs to solving integrals with highly oscillating integrands and with power integrands by the standard trapezoidal rule [34]. Let $a=x_1<x_2<\cdots <x_n=b$ be a partition of $[a,b]$. Sophisticated integration schemes for computing I using values $y_k=f\left(x_k\right)$ had been developed in the pre-computer ages (before 1950) when the computation of $y_k$ for n of order ${10}^3$ was a problem. Now, BFCTs allow to obtain results for large n up to ${10}^8$, using simple quadratures such as the formula of trapezoids with equal spacing [35] described below as the MATLAB^® code

n = 10^m; X = linspace(a,b,n+1); Y = f(X); h = (b−a)/n; …

T_n = h∗(sum(Y(2:n)) + (Y(1) + Y(n+1))/2)

If the function f is twice continuously differentiable, then the absolute error of the trapezoidal method with equal spacing is estimated as

$$|T_n-I|\le E_n(f,a,b)={\displaystyle \frac{\gamma (f,a,b)}{n^2}}$$

(10)

where

$$\gamma (f,a,b)={\displaystyle \frac{{(b-a)}^3}{12}}{\mu }_2\left(f\right),{\mu }_2\left(f\right)=max\left\{\right|f^{\prime \prime }\left(x\right)|:x\in [a,b]\}$$

Example 29.

Consider the integral $I=\mathrm{int}(f,a,b)=F\left(b\right)-F\left(a\right)$, where $f=F^{\prime }$, $a>0$ and

$$F\left(x\right)=cx+x^{2}sin\left(x^{-1}\right),x\in [a,b]$$

Then,

$$f\left(x\right)=c+2xsin\left(x^{-1}\right)-cos\left(x^{-1}\right),f^{\prime \prime }\left(x\right)=x^{-4}cos\left(x^{-1}\right)$$

For

$$a={\displaystyle \frac{1}{m\pi }},b={\displaystyle \frac{1}{\pi }},c={\displaystyle \frac{m\pi }{m-1}},m\in {\mathbb{Z}}_2$$

we have

$$sin\left(a^{-1}\right)=0,cos\left(a^{-1}\right)=-1,F\left(a\right)=ac,sin\left(b^{-1}\right)=0,F\left(b\right)=bc$$

and $I=c(b-a)=1$.

For $m=11$, the number $a\simeq 0.0289$ is relatively small, and the function f is highly oscillating close to a; see Figure 2. Next, we have

$${\mu }_2\left(f\right)=\left|f^{″}\left(a\right)\right|={\left(11\pi \right)}^4,\gamma (f,a,b)=2750\pi /3\simeq 2880$$

and $E_n(f,a,b)=2880n^{-2}$.

The error $|T_n-1|$ and the ratio $r_n(f,a,b)=|T_n-1|/E_n(f,a,b)$ for $n={10}^m$ and m up to 8 are shown in

Table 4. Errors and estimates for the trapezoidal rule.

n	$|{\mathit{T}}_{\mathit{n}}-1|$	${\mathit{r}}_{\mathit{n}}(\mathit{f},\mathit{a},\mathit{b})$
5	$2.7522\times {10}^{-2}$	$2.3724\times {10}^{-4}$
10	$9.5472\times {10}^{-4}$	$3.2918\times {10}^{-5}$
100	$2.0633\times {10}^{-5}$	$7.1142\times {10}^{-5}$
1000	$4.3650\times {10}^{-7}$	$1.5051\times {10}^{-4}$
10,000	$4.3842\times {10}^{-9}$	$1.5117\times {10}^{-4}$
100,000	$4.3844\times {10}^{-11}$	$1.5117\times {10}^{-4}$
1,000,000	$4.4420\times {10}^{-13}$	$1.5316\times {10}^{-4}$
1,500,000	$1.9340\times {10}^{-13}$	$1.5004\times {10}^{-4}$
2,000,000	$1.0059\times {10}^{-13}$	$1.3873\times {10}^{-4}$
5,000,000	$3.4084\times {10}^{-14}$	$1.5189\times {10}^{-4}$
10,000,000	$1.9762\times {10}^{-14}$	$6.8139\times {10}^{-4}$
50,000,000	$2.1316\times {10}^{-14}$	$1.8375\times {10}^{-2}$
100,000,000	$5.0848\times {10}^{-14}$	$1.7533\times {10}^{-1}$

.

For n from 1000 to 1,000,000, the error (which is both absolute and relative since $I=1$) decreases as $0.4n^{-2}$, while the ratio of the error and the estimate $E_n(f,a,b)$ is approximately constant about $1.5\times {10}^{-4}$. The behavior of the error $|T_n-1|$ is correctly evaluated, but the bound ${\mu }_2\left(f\right)=a^{-4}$ for $|f^{″}\left(x\right)|$ considerably overestimates the real behavior of $|f^{″}\left(x\right)|$, although this bound is achieved for $x=a$ and hence is not improvable.

For n between 1,500,000 and 100,000,000, the accuracy of the computed solution does not increase with n and shows chaotic behavior due to the accumulation of rounding errors. The result $T_n$ for $n=1000$ and even for $n=100$ is satisfactory from a practical viewpoint.

For f and $a,b$ as in Example 29, the numerical MATLAB^® code integral(f,a,b) produces the answer with error $7.1720\times {10}^{-14}$. At the same time, the NLF $F\left(b\right)-F\left(a\right)$ gives result $1-\rho$ with error $\rho \simeq 1.1011\times {10}^{-16}$.

The next example shows that a sophisticated quadrature may give completely wrong results for a polynomial integrand, while a BFCT approach still gives satisfactory results.

Example 30.

Let $f_N\left(x\right)=(N+1)x^N$, where $N\in {\mathbb{Z}}_1$. Consider the integral $I_N=\mathrm{int}(f_N,0,1)=1$. For $N\le N_0=51,221,433$, the MATLAB^® code integral(f_N,0,1) works well with errors of order ${10}^{-9}$. The BFCT result $T_N$ is also acceptable, although it is slightly worse. However, for $N=N_0+1$, the code integral(f_N,0,1) gives a completely wrong result $\simeq {10}^{-10}$ with 100% error, while the BFCT formula $T_N$ still works relatively well.

Of course, in cases as in Example 30 with known primitives, we may use the MATLAB^® symbolic code int based on NLF, e.g., syms x; I_N = int((N+1)∗x^N,0,1). This gives the exact answer $I_N=1$ for large $N\le X_{max}$.

This leads to the following proposition.

Proposition 11.

To check the performance of sophisticated quadrature formulas by BFCT such as the trapezoidal rule with a large number of knots is a good numerical strategy.

10.2. Derivatives

The numerical differentiation of a scalar function f of scalar or vector argument x is used in many cases, i.e., when analytical expression $f\left(x\right)$ is not available or is too complicated, when numerical algorithms are applied that use approximate derivatives f, etc. In the simplest yet most spread case, the derivative $f^{\prime }\left(x\right)$ is approximated numerically by the first-order finite difference ${\Delta }_h(f,x)=(f(x+h)-f\left(x\right))/h$, where $\left|h\right|$ is small but not too small. The behavior of ${\Delta }_h(f,x)$ as a function of h resembles a saw; see Example 31.

Example 31.

Consider the first difference ${\Delta }_h(x,1)$ for the simplest non-constant function $f\left(x\right)=x$ at the point $x=1$. We have ${\Delta }_h(x,1)=(1+h-1)h^{-1}$. The graph of the computed function $h\mapsto (1+h-1)h^{-1}$ for $h\in [-3\mathrm{eps},3\mathrm{eps}]\setminus \left\{0\right\}$ is shown in Figure 3 in blue. The graph of the exact function $h\mapsto 1$ is the horizontal red line.

Let $f^{\prime }\left(x\right)\ne 0$ and $x\ne 0$. When $h/x$ is too small, e.g., $h/x\in [-\rho /2,\rho ]$, the computed value of $f(x+h)$ is $f\left(x\right)$, hence ${\Delta }_h(f,x)=0$ and the derivative $f^{\prime }\left(x\right)$ is computed as 0 with 100% relative error. This numerical catastrophe may be avoided by BFCTs revealing the real behavior of ${\Delta }_h(f,x)$ for small values of h. The general principle here is that h must be chosen as $C{\left(\mathrm{eps}\right)}^{1/2}\simeq \mathrm{C}\times {10}^{-8}$, and the problem is how to estimate the constant C especially in RTA. Note that when no information for a constant $C>0$ is available, the heuristic rule is to set $C=1$.

11. VPA Versus FPA

There are many types of variable precision arithmetics (VPAs) and computer languages in which these VPAs are realized. Here, we include systems using symbolic objects like $\pi$, $\sqrt{2}$, etc. VPAs may work with very high precision (actually, with infinite precision), but the performance of numerical algorithms with VPAs may be slow and not suitable for RTA. In contrast, binary double-precision floating point arithmetic (FPA) works with a finite set of machine numbers, has finite precision and works relatively fast. Thus, FPA is suitable for RTA. Algorithms combining FPA and VPA based on MAI are also used recently. Here, BFCTs may be very useful as well.

12. Conclusions

Our main results are summarized in Propositions 1–6, 11 and 20. Most of the results presented are valid for using BFCTs in FPA. Using extended precision as in VPA, some of the observed effects may no longer occur. This may be accomplished, although it increases the computation time, potentially limiting RTA.

The combination of BFCTs and CPRS provides a powerful tool for solving problems and checking computed solutions and a priori accuracy estimates for many computational problems. According to the authors, BFCTs may become a useful complement in the implementation of techniques of artificial intelligence to the systems for calculating mathematics such as MATLAB^® [6,7] and Maple [9].