A Modification of the Fast Inverse Square Root Algorithm

Abstract

We present a new algorithm for the approximate evaluation of the inverse square root for single-precision floating-point numbers. This is a modification of the famous fast inverse square root code. We use the same “magic constant” to compute the seed solution, but then, we apply Newton–Raphson corrections with modified coefficients. As compared to the original fast inverse square root code, the new algorithm is two-times more accurate in the case of one Newton–Raphson correction and almost seven-times more accurate in the case of two corrections. We discuss relative errors within our analytical approach and perform numerical tests of our algorithm for all numbers of the type float.

1. Introduction

Floating-point arithmetic has became widely used in many applications such as 3D graphics, scientific computing and signal processing [1,2,3,4,5], implemented both in hardware and software [6,7,8,9,10]. Many algorithms can be used to approximate elementary functions [1,2,10,11,12,13,14,15,16,17,18]. The inverse square root function ($x\to 1/\sqrt{x}$) is of particular importance because it is widely used in 3D computer graphics, especially in lightning reflections [19,20,21], and has many other applications; see [22,23,24,25,26,27,28,29,30,31,32,33,34,35,36]. All of these algorithms require an initial seed to start the approximation. The more accurate is the initial seed, the fewer iterations are needed. Usually, the initial seed is obtained from a look-up table (LUT), which is memory consuming.

In this paper, we consider an algorithm for computing the inverse square root using the so-called magic constant instead of an LUT [37,38,39,40]. The zeroth approximation (initial seed) for the inverse square root of a given floating-point number is obtained by a logical right shift by one bit and subtracting this result from an specially-chosen integer (“magic constant”). Both operations are performed on bits of the floating-point number interpreted as an integer. Then, a more accurate value is produced by a certain number (usually one or two) of standard Newton–Raphson iterations. The following code realizes the fast inverse square root algorithm in the case of single-precision IEEE Standard 754 floating-point numbers (type float).

The code InvSqrt (see Algorithm 1) consists of two main parts. Lines 4 and 5 produce in a very inexpensive way a quite good zeroth approximation of the inverse square root of a given positive floating-point number x. Lines 6 and 7 apply the Newton–Raphson corrections twice (often, a version with just one iteration is used, as well). Originally, R was proposed as $0x5F3759DF$; see [37,38]. More details, together with a derivation of a better magic constant, are given in Section 2.

Algorithm 1:InvSqrt.

1. float InvSqrt(float x){

2.   float halfnumber = 0.5f*x;

3.   int i = *(int*) &x;

4.   i = R - ($\mathrm{i}>>1$);

5.   y = *(float*) &i;

6.   y = y*(1.5f - halfnumber*y*y);

7.   y = y*(1.5f - halfnumber*y*y);

8.   return y;

9. }

InvSqrt is characterized by a high speed, more that three-times higher than computing the inverse square root using library functions. This property was discussed in detail in [41]. The errors of the fast inverse square root algorithm depend on the choice of the “magic constant” R. In several theoretical papers [38,41,42,43,44] (see also Eberly’s monograph [19]), attempts were made to determine analytically the optimal value of the magic constant (i.e., to minimize errors). In general, this optimal value can depend on the number of iterations, which is a general phenomenon [45]. The derivation and comprehensive mathematical description of all the steps of the fast inverse square root algorithm were given in our recent paper [46]. We found the optimum value of the magic constant by minimizing the final maximum relative error.

In the present paper, we develop our analytical approach to construct an improved algorithm (InvSqrt1) for fast computing of the inverse square root; see Algorithm 2 in Section 4. The proposed modification does not increase the speed of data processing, but increases, in a significant way, the accuracy of the output. In both codes, InvSqrt and InvSqrt1, magic constants serve as a low-cost way of generating a reasonably accurate first approximation of the inverse square root. These magic constants turn out to be the same. The main novelty of the new algorithm is in the second part of the code, which is changed significantly. In fact, we propose a modification of the Newton–Raphson formulae, which has a similar computational cost, but improve the accuracy by several fold.

2. Analytical Approach to the Algorithm InvSqrt

In this paper, we confine ourselves to positive single-precision floating-point numbers (type float). Normal floating-point numbers can be represented as:

$$x=(1+m_x)2^{e_x}$$

(1)

where $m_x\in [0,1)$ and $e_x$ is an integer (note that this formula does not hold for subnormal numbers). In the case of the IEEE-754 standard, a floating-point number is encoded by 32 bits. The first bit corresponds to a sign (in our case, this bit is simply equal to zero); the next eight bits correspond to an exponent $e_x$; and the last 23 bits encode a mantissa $m_x$. The integer encoded by these 32 bits, denoted by $I_x$, is given by:

$$I_x=N_m(B+e_x+m_x)$$

(2)

where $N_m=2^{23}$ and $B=127$ (thus $B+e_x=1,2,\dots ,254$). Lines 3 and 5 of the InvSqrt code interpret a number as an integer (2) or float (1), respectively. Lines 4, 6, and 7 of the code can be written as:

$$\begin{array}{c}{I}_{y_0}=R-\lfloor I_x/2\rfloor ,y_1=\frac{1}{2}{y}_0(3-y_0^{2}x),y_2=\frac{1}{2}{y}_1(3-y_1^{2}x).\hfill \end{array}$$

(3)

The first equation produces, in a surprisingly simple way, a good zeroth approximation $y_0$ of the inverse square root $y=1/\sqrt{x}$. Of course, this needs a very special form of R. In particular, in the single precision case, we have $e_R=63$; see [46]. The next equations can be easily recognized as the Newton–Raphson corrections. We point out that the code InvSqrt is invariant with respect to the scaling:

$$x\to \tilde{x}=2^{-2n}x,y_k\to {\tilde{y}}_k=2^n{y}_k(k=0,1,2),$$

(4)

like the equality $y=1/\sqrt{x}$ itself. Therefore, without loss of the generality, we can confine our analysis to the interval:

$$\tilde{A}:=[1,4).$$

(5)

The tilde will denote quantities defined on this interval. In [46], we showed that the function ${\tilde{y}}_0$ defined by the first equation of (3) can be approximated with a very good accuracy by the piece-wise linear function ${\tilde{y}}_{00}$ given by:

$${\tilde{y}}_{00}(\tilde{x},t)=\left\{\begin{array}{cc}{\displaystyle -\frac{1}{4}\tilde{x}+\frac{3}{4}+\frac{1}{8}t}\hfill & \mathrm{for}\tilde{x}\in [1,2)\hfill \\ {\displaystyle -\frac{1}{8}\tilde{x}+\frac{1}{2}+\frac{1}{8}t}\hfill & \mathrm{for}\tilde{x}\in [2,t)\hfill \\ {\displaystyle -\frac{1}{16}\tilde{x}+\frac{1}{2}+\frac{1}{16}t}\hfill & \mathrm{for}\tilde{x}\in [t,4)\hfill \end{array}\right.$$

(6)

where:

$$t=2+4m_R+2N_m^{-1},$$

(7)

and $m_R:=N_m^{-1}R-\lfloor N_m^{-1}R\rfloor$ ($m_R$ is the mantissa of the floating-point number corresponding to R). Note that the parameter t, defined by (7), is uniquely determined by R.

The only difference between $y_0$ produced by the code InvSqrt and $y_{00}$ given by (6) is the definition of t, because t related to the code depends (although in a negligible way) on x. Namely,

$$|{\tilde{y}}_{00}-{\tilde{y}}_0|⩽\frac{1}{4}{N}_m^{-1}=2^{-25}\approx 2.98\times {10}^{-8}.$$

(8)

Taking into account the invariance (4), we obtain:

$$\left|\frac{y_{00}-y_0}{y_0}\right|⩽2^{-24}\approx 5.96\times {10}^{-8}.$$

(9)

These estimates do not depend on t (in other words, they do not depend on R). The relative error of the zeroth approximation (6) is given by:

$${\tilde{\delta }}_0(\tilde{x},t)=\sqrt{\tilde{x}}{\tilde{y}}_{00}(\tilde{x},t)-1$$

(10)

This is a continuous function with local maxima at:

$${\tilde{x}}_0^I=(6+t)/6,{\tilde{x}}_0^{II}=(4+t)/3,{\tilde{x}}_0^{III}=(8+t)/3,$$

(11)

given respectively by:

$$\begin{array}{c}{\displaystyle {\tilde{\delta }}_0({\tilde{x}}_0^I,t)=-1+\frac{1}{2}{\left(1+\frac{t}{6}\right)}^{3/2},}\hfill \\ {\displaystyle {\tilde{\delta }}_0({\tilde{x}}_0^{II},t)=-1+2{\left(\frac{1}{3}\left(1+\frac{t}{4}\right)\right)}^{3/2},}\hfill \\ {\displaystyle {\tilde{\delta }}_0({\tilde{x}}_0^{III},t)=-1+{\left(\frac{2}{3}\left(1+\frac{t}{8}\right)\right)}^{3/2}.}\hfill \end{array}$$

(12)

In order to study the global extrema of ${\tilde{\delta }}_0(\tilde{x},t)$, we need also boundary values:

$$\begin{array}{c}{\displaystyle {\tilde{\delta }}_0(1,t)={\tilde{\delta }}_0(4,t)=\frac{1}{8}\left(t-4\right),{\tilde{\delta }}_0(2,t)=\frac{\sqrt{2}}{4}\left(1+\frac{t}{2}\right)-1,{\tilde{\delta }}_0(t,t)=\frac{\sqrt{t}}{2}-1,}\hfill \end{array}$$

(13)

which are, in fact, local minima. Taking into account:

$$\begin{array}{c}{\displaystyle {\tilde{\delta }}_0(1,t)-{\tilde{\delta }}_0(t,t)=\frac{1}{8}{\left(\sqrt{t}-2\right)}^2⩾0,{\tilde{\delta }}_0(2,t)-{\tilde{\delta }}_0(t,t)=\frac{\sqrt{2}}{8}{\left(\sqrt{t}-\sqrt{2}\right)}^2⩾0,}\hfill \end{array}$$

(14)

we conclude that:

$$\underset{\tilde{x}\in \tilde{A}}{\min }{\tilde{\delta }}_0(\tilde{x},t)={\tilde{\delta }}_0(t,t)<0.$$

(15)

Because ${\tilde{\delta }}_0({\tilde{x}}_0^{III},t)<0$ for $t\in (2,4)$, the global maximum is one of the remaining local maxima:

$$\underset{\tilde{x}\in \tilde{A}}{\max }{\tilde{\delta }}_0(\tilde{x},t)=\max \{{\tilde{\delta }}_0({\tilde{x}}_0^I,t),{\tilde{\delta }}_0({\tilde{x}}_0^{II},t)\}.$$

(16)

Therefore,

$$\underset{x\in \tilde{A}}{\max }|{\tilde{\delta }}_0(\tilde{x},t)|=\max \{|{\tilde{\delta }}_0(t,t)|,{\tilde{\delta }}_0({\tilde{x}}_0^I,t),{\tilde{\delta }}_0({\tilde{x}}_0^{II},t)\}.$$

(17)

In order to minimize this value with respect to t, i.e., to find $t_0^r$ such that:

$$\underset{x\in \tilde{A}}{\max }|{\tilde{\delta }}_0(\tilde{x},t_0^r)|<\underset{x\in \tilde{A}}{\max }|{\tilde{\delta }}_0(\tilde{x},t)|\mathrm{for}t\ne t_0^r,$$

(18)

we observe that $|{\tilde{\delta }}_0(t,t)|$ is a decreasing function of t, while both maxima (${\tilde{\delta }}_0({\tilde{x}}_0^I,t)$ and ${\tilde{\delta }}_0({\tilde{x}}_0^{II},t)$) are increasing functions. Therefore, it is sufficient to find $t=t_0^I$ and $t=t_0^{II}$ such that:

$$|{\tilde{\delta }}_0(t_0^I,t_0^I)|={\tilde{\delta }}_0({\tilde{x}}_0^I,t_0^I),|{\tilde{\delta }}_0(t_0^{II},t_0^{II})|={\tilde{\delta }}_0({\tilde{x}}_0^{II},t_0^{II}),$$

(19)

and to choose the greater of these two values. In [46], we showed that:

$$|{\tilde{\delta }}_0(t_0^I,t_0^I)|<|{\tilde{\delta }}_0(t_0^{II},t_0^{II})|.$$

(20)

Therefore, $t_0^r=t_0^{II}$, and:

$${\tilde{\delta }}_{0\max }:=\underset{t\in (2,4)}{\min }\left(\underset{x\in \tilde{A}}{\max }|{\tilde{\delta }}_0(\tilde{x},t)|\right)=|{\tilde{\delta }}_0(t_0^r,t_0^r)|.$$

(21)

The following numerical values result from these calculations [46]:

$$t_0^r\approx 3.7309796,R_0=0x5F37642F,{\tilde{\delta }}_{0\max }\approx 0.03421281.$$

(22)

Newton–Raphson corrections for the zeroth approximation (${\tilde{y}}_{00}$) will be denoted by ${\tilde{y}}_{0k}$ ($k=1,2,\dots$). In particular, we have:

$$\begin{array}{c}{\tilde{y}}_{01}(\tilde{x},t)=\frac{1}{2}{\tilde{y}}_{00}(\tilde{x},t)(3-{\tilde{y}}_{00}^2(\tilde{x},t)\tilde{x}),\hfill \\ {\tilde{y}}_{02}(\tilde{x},t)=\frac{1}{2}{\tilde{y}}_{01}(\tilde{x},t)(3-{\tilde{y}}_{01}^2(\tilde{x},t)\tilde{x}).\hfill \end{array}$$

(23)

and the corresponding relative error functions will be denoted by ${\tilde{\delta }}_k(\tilde{x},t)$:

$${\tilde{\delta }}_k(\tilde{x},t):=\frac{{\tilde{y}}_{0k}(\tilde{x},t)-\tilde{y}}{\tilde{y}}=\sqrt{\tilde{x}}{\tilde{y}}_{0k}(\tilde{x},t)-1,(k=0,1,2,\dots ),$$

(24)

where we included also the case $k=0$; see (10). The obtained approximations of the inverse square root depend on the parameter t directly related to the magic constant R. The value of this parameter can be estimated by analyzing the relative error of ${\tilde{y}}_{0k}(\tilde{x},t)$ with respect to $1/\sqrt{\tilde{x}}$. As the best estimation, we consider $t=t_k^{\left(r\right)}$ minimizing the relative error ${\tilde{\delta }}_k(\tilde{x},t)$:

$${\forall }_{t\ne t_k^{\left(r\right)}}\left({\tilde{\delta }}_{k\max }\equiv \underset{\tilde{x}\in \tilde{A}}{\max }|{\tilde{\delta }}_k(\tilde{x},t_k^{\left(r\right)})|<\underset{\tilde{x}\in \tilde{A}}{\max }|{\tilde{\delta }}_k(\tilde{x},t)|\right).$$

(25)

We point out that in general, the optimum value of the magic constant can depend on the number of Newton–Raphson corrections. Calculations carried out in [46] gave the following results:

$$\begin{array}{c}{t}_1^r=t_2^r=3.7298003,R_1^r=R_2^r=0x5F375A86,\hfill \\ {\tilde{\delta }}_{1\max }\approx 1.75118\times {10}^{-3},{\tilde{\delta }}_{2\max }\approx 4.60\times {10}^{-6}.\hfill \end{array}$$

(26)

We omit the details of the computations except one important point. Using (24) to express ${\tilde{y}}_{0k}$ by ${\tilde{\delta }}_k$ and $\sqrt{\tilde{x}}$, we can rewrite (23) as:

$${\tilde{\delta }}_k(\tilde{x},t)=-\frac{1}{2}{\tilde{\delta }}_{k-1}^2(\tilde{x},t)(3+{\tilde{\delta }}_{k-1}(\tilde{x},t)),(k=1,2,\dots ).$$

(27)

The quadratic dependence on ${\tilde{\delta }}_{k-1}$ means that every Newton–Raphson correction improves the accuracy by several orders of magnitude (until the machine precision is reached); compare (26).

The Formula (27) suggests another way of improving the accuracy because the functions ${\tilde{\delta }}_k$ are always non-positive for any $k⩾1$. Roughly speaking, we are going to shift the graph of ${\tilde{\delta }}_k$ upwards by an appropriate modification of the Newton–Raphson formula. In the next section, we describe the general idea of this modification.

3. Modified Newton–Raphson Formulas

The Formula (27) shows that errors introduced by Newton–Raphson corrections are nonpositive, i.e., they take values in intervals $[-{\tilde{\delta }}_{k\max },0]$, where $k=1,2,\dots$. Therefore, it is natural to introduce a correction term into the Newton–Raphson formulas (23). We expect that the corrections will be roughly half of the maximal relative error. Instead of the maximal error, we introduce two parameters, $d_1$ and $d_2$. Thus, we get modified Newton–Raphson formulas:

$$\begin{array}{c}{\displaystyle {\tilde{y}}_{11}(\tilde{x},t,d_1)=2^{-1}{\tilde{y}}_{00}(\tilde{x},t)(3-{\tilde{y}}_{00}^2(\tilde{x},t)\tilde{x})+\frac{d_1}{2\sqrt{\tilde{x}}},}\hfill \\ {\displaystyle {\tilde{y}}_{12}(\tilde{x},t,d_1,d_2)=2^{-1}{\tilde{y}}_{11}(\tilde{x},t,d_1)(3-{\tilde{y}}_{11}^2(\tilde{x},t,d_1)\tilde{x})+\frac{d_2}{2\sqrt{\tilde{x}}},}\hfill \end{array}$$

(28)

where zeroth approximation is assumed in the form (6). In the following section, the term $1/\sqrt{\tilde{x}}$ will be replaced by some approximations of $\tilde{y}$, transforming (28) into a computer code. In order to estimate a possible gain in accuracy, in this section, we temporarily assume that $\tilde{y}$ is the exact value of the inverse square root. The corresponding error functions,

$${\tilde{\delta }}_k^{{}^{″}}(\tilde{x},t,d_1,\dots ,d_k)=\sqrt{\tilde{x}}{\tilde{y}}_{1k}(\tilde{x},t,d_1,\dots ,d_k)-1,k\in \{0,1,2,\dots \},$$

(29)

(where ${\tilde{y}}_{10}(\tilde{x},t):={\tilde{y}}_{00}(\tilde{x},t)$), satisfy:

$${\tilde{\delta }}_k^{{}^{″}}=-\frac{1}{2}{\tilde{\delta }}_{k-1}^{{}^{″}2}(3+{\tilde{\delta }}_{k-1}^{{}^{″}})+\frac{d_k}{2},$$

(30)

where: ${\tilde{\delta }}_0^{{}^{″}}(\tilde{x},t)={\tilde{\delta }}_0(\tilde{x},t)$. Note that:

$${\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t,d_1)={\tilde{\delta }}_1(\tilde{x},t)+\frac{1}{2}{d}_1.$$

(31)

In order to simplify notation, we usually will suppress the explicit dependence on $d_j$. We will write, for instance, ${\tilde{\delta }}_2^{{}^{″}}(\tilde{x},t)$ instead of ${\tilde{\delta }}_2^{{}^{″}}(\tilde{x},t,d_1,d_2)$.

The corrections of the form (28) will decrease relative errors in comparison with the results of earlier papers [38,46]. We have three free parameters ($d_1,d_2$, and t) to be determined by minimizing the maximal error (in principle, the new parameterization can give a new estimation of the parameter t). By analogy to (25), we are going to find $t=t^{\left(0\right)}$ minimizing the error of the first correction (25):

$${\forall }_{t\ne t^{\left(0\right)}}\underset{\tilde{x}\in \tilde{A}}{\max }|{\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t^{\left(0\right)})|<\underset{\tilde{x}\in \tilde{A}}{\max }|{\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t)|,$$

(32)

where, as usual, $\tilde{A}=[1,4]$.

The first of Equation (30) implies that for any t, the maximal value of ${\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t)$ equals $\frac{1}{2}{d}_1$ and is attained at zeros of ${\tilde{\delta }}_0^{{}^{″}}(\tilde{x},t)$. Using the results of Section 2, including (15), (16), (20), and (21), we conclude that the minimum value of ${\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t)$ is attained either for $\tilde{x}=t$ or for $\tilde{x}=x_0^{II}$ (where there is the second maximum of ${\tilde{\delta }}_0^{{}^{″}}(\tilde{x},t)$), i.e.,

$$\underset{\tilde{x}\in \tilde{A}}{\min }{\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t)=\min \left\{{\tilde{\delta }}_1^{{}^{″}}(t,t),{\tilde{\delta }}_1^{{}^{″}}(x_0^{II},t)\right\}$$

(33)

Minimization of $|{\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t)|$ can be done with respect to t and with respect to $d_1$ (these operations obviously commute), which corresponds to:

$$\underset{{\tilde{\delta }}_{1\max }^{{}^{″}}}{\underbrace{\underset{\tilde{x}\in \tilde{A}}{\max }{\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t^{\left(0\right)})}}=-\underset{\tilde{x}\in \tilde{A}}{\min }{\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t^{\left(0\right)}).$$

(34)

Taking into account:

$$\underset{\tilde{x}\in \tilde{A}}{\max }{\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t^{\left(0\right)})=\frac{d_1}{2},\underset{\tilde{x}\in \tilde{A}}{\min }{\tilde{\delta }}_1^{{}^{″}}(\tilde{x},t^{\left(0\right)})={\tilde{\delta }}_1^{{}^{″}}(t^{\left(0\right)},t^{\left(0\right)})=-{\tilde{\delta }}_{1\max }+\frac{d_1}{2},$$

(35)

we get from (34):

$${\tilde{\delta }}_{1\max }^{{}^{″}}=\frac{1}{2}{d}_1=\frac{1}{2}{\tilde{\delta }}_{1\max }\simeq 8.7559\times {10}^{-4},$$

(36)

where:

$${\tilde{\delta }}_{1\max }:=\underset{t\in (2,4)}{\min }\left(\underset{x\in \tilde{A}}{\max }\left|{\tilde{\delta }}_1(\tilde{x},t)\right|\right).$$

(37)

and the numerical value of ${\tilde{\delta }}_{1\max }$ is given by (26). These conditions are satisfied for:

$$t^{\left(0\right)}=t_1^{\left(r\right)}\simeq 3.7298003.$$

(38)

In order to minimize the relative error of the second correction, we use equation analogous to (34):

$$\underset{{\tilde{\delta }}_{2\max }^{{}^{″}}}{\underbrace{\underset{\tilde{x}\in \tilde{A}}{\max }{\tilde{\delta }}_2^{{}^{″}}(\tilde{x},t^{\left(0\right)})}}=-\underset{\tilde{x}\in \tilde{A}}{\min }{\tilde{\delta }}_2^{{}^{″}}(\tilde{x},t^{\left(0\right)}),$$

(39)

where from (30), we have:

$$\underset{\tilde{x}\in \tilde{A}}{\max }{\tilde{\delta }}_2^{{}^{″}}(\tilde{x},t^{\left(0\right)})=\frac{d_2}{2},\underset{\tilde{x}\in \tilde{A}}{\min }{\tilde{\delta }}_2^{{}^{″}}(\tilde{x},t^{\left(0\right)})=-\frac{1}{2}{\tilde{\delta }}_{1\max }^{{}^{″}2}\left(3+{\tilde{\delta }}_{1\max }^{{}^{″}}\right)+\frac{d_2}{2}.$$

(40)

Hence:

$${\tilde{\delta }}_{2\max }^{{}^{″}}=\frac{1}{4}{\tilde{\delta }}_{1\max }^{{}^{″}2}\left(3+{\tilde{\delta }}_{1\max }^{{}^{″}}\right).$$

(41)

Expressing this result in terms of formerly computed ${\tilde{\delta }}_{1\max }$ and ${\tilde{\delta }}_{2\max }$, we obtain:

$${\tilde{\delta }}_{2\max }^{{}^{″}}=\frac{1}{8}{\tilde{\delta }}_{2\max }+\frac{3}{32}{\tilde{\delta }}_{1\max }^3\simeq 5.75164\times {10}^{-7}\simeq \frac{{\tilde{\delta }}_{2\max }}{7.99},$$

(42)

where:

$${\tilde{\delta }}_{2\max }=\frac{1}{2}{\tilde{\delta }}_{1\max }^2(3-{\tilde{\delta }}_{1\max }).$$

Therefore, the above modification of Newton–Raphson formulas decreases the relative error two times after one iteration and almost eight times after two iterations as compared to the standard InvSqrt algorithm.

In order to implement this idea in the form of a computer code, we have to replace the unknown $1/\sqrt{\tilde{x}}$ (i.e., $\tilde{y}$) on the right-hand sides of (28) by some numerical approximations.

4. New Algorithm with Higher Accuracy

Approximating $1/\sqrt{\tilde{x}}$ in Formulas (28) by values on the left-hand sides, we transform (28) into:

$$\begin{array}{c}{\displaystyle {\tilde{y}}_{21}=\frac{1}{2}{\tilde{y}}_{20}(3-{\tilde{y}}_{20}^2\tilde{x})+\frac{1}{2}{d}_1{\tilde{y}}_{21},}\hfill \\ {\displaystyle {\tilde{y}}_{22}=\frac{1}{2}{\tilde{y}}_{21}(3-{\tilde{y}}_{21}^2\tilde{x})+\frac{1}{2}{d}_2{\tilde{y}}_{22},}\hfill \end{array}$$

(43)

where ${\tilde{y}}_{2k}$ ($k=1,2,\dots$) depend on $\tilde{x},t$ and $d_j$ (for $1⩽j⩽k$). We assume ${\tilde{y}}_{20}\equiv {\tilde{y}}_{00}$, i.e., the zeroth approximation is still given by (6). We can see that ${\tilde{y}}_{21}$ and ${\tilde{y}}_{22}$ can be explicitly expressed by ${\tilde{y}}_{20}$ and ${\tilde{y}}_{21}$, respectively.

Parameters $d_1$ and $d_2$ have to be determined by minimization of the maximum error. We define error functions in the usual way:

$${\Delta }_k^{\left(1\right)}=\frac{{\tilde{y}}_{2k}-\tilde{y}}{\tilde{y}}=\sqrt{\tilde{x}}{\tilde{y}}_{2k}-1.$$

(44)

Substituting (44) into (43), we get:

$${\Delta }_1^{\left(1\right)}(\tilde{x},t,d_1)=\frac{d_1}{2-d_1}-\frac{1}{2-d_1}{\tilde{\delta }}_0^2(\tilde{x},t)(3+{\tilde{\delta }}_0(\tilde{x},t))=\frac{d_1+2{\tilde{\delta }}_1(\tilde{x},t)}{2-d_1},$$

(45)

$${\Delta }_2^{\left(1\right)}(\tilde{x},t,d_{,}{d}_2)=\frac{d_2}{2-d_2}-\frac{1}{2-d_2}{\left({\Delta }_1^{\left(1\right)}(\tilde{x},t,d_1)\right)}^2\left(3+{\Delta }_1^{\left(1\right)}(\tilde{x},t,d_1)\right).$$

(46)

The equation (45) expresses ${\Delta }_1^{\left(1\right)}(\tilde{x},t,d_1)$ as a linear function of the nonpositive function ${\tilde{\delta }}_1(\tilde{x},t)$ with coefficients depending on the parameter $d_1$. The optimum parameters t and $d_1$ will be estimated by the procedure described in Section 3. First, we minimize the amplitude of the relative error function, i.e., we find $t^{\left(1\right)}$ such that:

$$\underset{\tilde{x}\in \tilde{A}}{\max }{\Delta }_1^{\left(1\right)}(\tilde{x},t^{\left(1\right)})-\underset{\tilde{x}\in \tilde{A}}{\min }{\Delta }_1^{\left(1\right)}(\tilde{x},t^{\left(1\right)})⩽\underset{\tilde{x}\in \tilde{A}}{\max }{\Delta }_1^{\left(1\right)}(\tilde{x},t)-\underset{\tilde{x}\in \tilde{A}}{\min }{\Delta }_1^{\left(1\right)}(\tilde{x},t)$$

(47)

for all $t\ne t^{\left(1\right)}$. Second, we determine $d_1^{\left(1\right)}$ such that:

$$\underset{\tilde{x}\in \tilde{A}}{\max }{\Delta }_1^{\left(1\right)}(\tilde{x},t^{\left(1\right)},d_1^{\left(1\right)})=-\underset{\tilde{x}\in \tilde{A}}{\min }{\Delta }_1^{\left(1\right)}(\tilde{x},t^{\left(1\right)},d_1^{\left(1\right)}).$$

(48)

Thus, we have:

$$\underset{\tilde{x}\in \tilde{A}}{\max }|{\Delta }_1^{\left(1\right)}(\tilde{x},t^{\left(1\right)},d_1^{\left(1\right)})|⩽\underset{\tilde{x}\in \tilde{A}}{\max }|{\Delta }_1^{\left(1\right)}(\tilde{x},t,d_1)|$$

(49)

for all real $d_1$ and $t\in (2,4)$. ${\Delta }_1^{\left(1\right)}(\tilde{x},t)$ is an increasing function of ${\tilde{\delta }}_1(\tilde{x},t)$; hence:

$$-\frac{d_1^{\left(1\right)}-2{\max }_{\tilde{x}\in \tilde{A}}|{\tilde{\delta }}_1(\tilde{x},t_1^{\left(1\right)})|}{2-d_1^{\left(1\right)}}=\frac{d_1^{\left(1\right)}}{2-d_1^{\left(1\right)}},$$

(50)

which is satisfied for:

$$d_1^{\left(1\right)}=\underset{\tilde{x}\in \tilde{A}}{\max }|{\tilde{\delta }}_1(\tilde{x},t_1^{\left(1\right)})|={\tilde{\delta }}_{1\max }.$$

(51)

Thus, we can find the maximum error of the first correction ${\Delta }_1^{\left(1\right)}(\tilde{x},t_1^{\left(1\right)})$ (presented in Figure 1):

$$\underset{\tilde{x}\in \tilde{A}}{\max }|{\Delta }_1^{\left(1\right)}(\tilde{x},t^{\left(1\right)})|=\frac{{\max }_{\tilde{x}\in \tilde{A}}\left|{\tilde{\delta }}_1(\tilde{x},t^{\left(1\right)})\right|}{2-{\max }_{\tilde{x}\in \tilde{A}}\left|{\tilde{\delta }}_1(\tilde{x},t^{\left(1\right)})\right|},$$

(52)

which assumes the minimum value for $t^{\left(1\right)}=t_1^{\left(r\right)}$:

$${\Delta }_{1\max }^{\left(1\right)}=\frac{{\max }_{\tilde{x}\in \tilde{A}}|{\tilde{\delta }}_1(\tilde{x},t_1^{\left(r\right)})|}{2-{\max }_{\tilde{x}\in \tilde{A}}|{\tilde{\delta }}_1(\tilde{x},t_1^{\left(r\right)})|}=\frac{{\tilde{\delta }}_{1\max }}{2-{\tilde{\delta }}_{1\max }}\simeq 8.7636\times {10}^{-4}\simeq \frac{{\tilde{\delta }}_{1\max }}{2.00}.$$

(53)

This result practically coincides with ${\tilde{\delta }}_{1\max }^{{}^{″}}$ given by (36).

Analogously, we can determine the value of $d_2^{\left(1\right)}$ (assuming that $t=t^{\left(1\right)}$ is fixed):

$$-\frac{d_2^{\left(1\right)}-{\max }_{\tilde{x}\in \tilde{A}}|{\Delta }_1^{\left(1\right)2}(\tilde{x},t^{\left(1\right)})(3+{\Delta }_1^{\left(1\right)}(\tilde{x},t^{\left(1\right)}))|}{2-d_2^{\left(1\right)}}=\frac{d_2^{\left(1\right)}}{2-d_2^{\left(1\right)}}.$$

(54)

Now, the deepest minimum comes from the global maximum:

$$\underset{\tilde{x}\in \tilde{A}}{\max }|{\Delta }_1^{\left(1\right)2}(\tilde{x},t^{\left(1\right)})(3+{\Delta }_1^{\left(1\right)}(\tilde{x},t^{\left(1\right)}))|=\frac{2{\tilde{\delta }}_{1\max }^2(3-{\tilde{\delta }}_{1\max })}{{(2-{\tilde{\delta }}_{1\max })}^3}.$$

(55)

Therefore, we get:

$$d_2^{\left(1\right)}=\frac{{\tilde{\delta }}_{1\max }^2(3-{\tilde{\delta }}_{1\max })}{{(2-{\tilde{\delta }}_{1\max })}^3}\simeq 1.15234\times {10}^{-6},$$

(56)

and the maximum error of the second correction is given by:

$${\Delta }_{2\max }^{\left(1\right)}=\frac{d_2^{\left(1\right)}}{2-d_2^{\left(1\right)}}\simeq 5.76173\times {10}^{-7}\simeq \frac{{\tilde{\delta }}_{2\max }}{7.98},$$

(57)

which is very close to the value of ${\tilde{\delta }}_{2\max }^{{}^{″}}$ given by (42).

Thus, we have obtained the algorithm InvSqrt1, see Algorithm 2, which looks like InvSqrt with modified values of the numerical coefficients.

Algorithm 2:InvSqrt1.

1. float InvSqrt1(float x){

2.   float simhalfnumber = 0.500438180f*x;

3.   int i = *(int*) &x;

4.   i = 0x5F375A86 - ($\mathrm{i}>>1$);

5.   y = *(float*) &i;

6.   y = y*(1.50131454f - simhalfnumber*y*y);

7.   y = y*(1.50000086f - 0.999124984f*simhalfnumber*y*y);

8.   return y;

9. }

Comparing $InvSqrt1$ with $InvSqrt$, we easily see that the number of algebraic operations in $InvSqrt1$ is greater by one (an additional multiplication in Line 7, corresponding to the second iteration of the modified Newton–Raphson procedure). We point out that the magic constants for InvSqrt and InvSqrt1 are the same.

5. Numerical Experiments

The new algorithm $InvSqrt1$ was tested on the processor Intel Core i5-3470 using the compiler TDM-GCC 4.9.2 32-bit. Using the same hardware for testing the code $InvSqrt$, we obtained practically the same values of errors as those obtained by Lomont [38]. The same results were obtained also on Intel i7-5700. In this section, we analyze the rounding errors for the code $\mathit{InvSqrt}\mathit{1}$.

Applying algorithm $InvSqrt1$, we obtain relative errors $\mathrm{InvSqrt}1\left(\tilde{x}\right)$ characterized by “oscillations” with a center slightly shifted with respect to the analytical approximate solution ${\tilde{y}}_{22}(\tilde{x},t^{\left(1\right)})$; see Figure 2. The observed blur can be expressed by a relative deviation of the numerical result from ${\tilde{y}}_{22}\left(\tilde{x}\right)$:

$${\epsilon }^{\left(1\right)}\left(\tilde{x}\right)=\frac{\mathrm{InvSqrt}1\left(\tilde{\mathrm{x}}\right)-{\tilde{y}}_{22}(\tilde{x},t^{\left(1\right)})}{{\tilde{y}}_{22}(\tilde{x},t^{\left(1\right)})}.$$

(58)

The values of this error are distributed symmetrically around the mean value $\langle {\epsilon }^{\left(1\right)}\rangle$:

$$\langle {\epsilon }^{\left(1\right)}\rangle =2^{-1}{N}_m^{-1}\sum _{x\in [1,4)}{\epsilon }^{\left(1\right)}\left(\tilde{x}\right)=-1.398\times {10}^{-8}$$

(59)

enclosing the range:

$${\epsilon }^{\left(1\right)}\left(\tilde{x}\right)\in [-9.676\times {10}^{-8},6.805\times {10}^{-8}],$$

(60)

see Figure 3. The blur parameters of the function ${\epsilon }^{\left(1\right)}(\tilde{x},t)$ show that the main source of the difference between analytical and numerical results is the use of precision float and, in particular, rounding of constant parameters of the function $\mathrm{InvSqrt}1$. We point out that in this case, the amplitude of the error oscillations is about $40\%$ greater than the amplitude of oscillations of $({\tilde{y}}_{00}-{\tilde{y}}_0)/{\tilde{y}}_0$ (i.e., in the case of $InvSqrt$); see the right part of Figure 2 in [46]. It is worth noting that for the first Newton–Raphson correction, the blur is of the same order, but due to a much higher error value in this case, its effect is negligible (i.e., Figure 1 would be practically the same with or without the blur). The maximum numerical errors practically coincide with the analytical result (53), i.e.,

$${\Delta }_{1,N\min }^{\left(1\right)}\approx -8.76\times {10}^{-4},{\Delta }_{1,N\max }^{\left(1\right)}\approx 8.76\times {10}^{-4}.$$

(61)

In the case of the second Newton–Raphson correction, we compared results produced by $InvSqrt1$ with exact values of the inverse square root for all numbers x of the type float such that $e_x\in [-126,128)$. The range of errors was the same for all these intervals (except $e_x=-126$):

$${\Delta }_{2;N}^{\left(1\right)}\left(x\right)=\mathrm{sqrt}\left(x\right)\ast \mathrm{InvSqrt}1\left(x\right)-1.\in (-6.62\times {10}^{-7},6.35\times {10}^{-7}).$$

(62)

For $e_x=-126$, the interval of errors was slightly wider: $(-6.72\times {10}^{-7},6.49\times {10}^{-7})$, which can be explained by the fact that the analysis presented in this paper is not applicable to subnormal numbers; see (1). Therefore, our tests showed that relative errors for all numbers of the type float belong to the interval $({\Delta }_{2,N\min }^{\left(1\right)},{\Delta }_{2,N\max }^{\left(1\right)})$, where:

$${\Delta }_{2,N\min }^{\left(1\right)}\approx -6.72\times {10}^{-7},{\Delta }_{2,N\max }^{\left(1\right)}\approx 6.49\times {10}^{-7}.$$

(63)

These values are significantly higher than the analytical result ($5.76\times {10}^{-7}$) (see (57)), but are still much lower than the analogous error for the algorithm $InvSqrt$ ($4.60\times {10}^{-6}$; see [46]).

6. Conclusions

In this paper, we presented a modification of the famous code InvSqrt for fast computation of the inverse square root of single-precision floating-point numbers. The new code had the same magic constant, but the second part (which consisted of Newton–Raphson iterations) was modified. In the case of one Newton–Raphson iteration, the new code InvSqrt1 had the same computational cost as InvSqrt and was two-times more accurate. In the case of two iterations, the computational cost of the new code was only slightly higher, but its accuracy was higher by almost seven times.

The main idea of our work consisted of modifying coefficients in the Newton–Raphson method and demanding that the maximal error be as small as possible. Such modifications can be constructed if the distribution of errors for Newton–Raphson corrections is not symmetric (like in the case of the inverse square root, when they are non-positive functions).