Multithreaded Reproducible Banded Matrix-Vector Multiplication

Abstract

Reproducibility refers to getting bitwise identical floating point results from multiple runs of the same program, is an important basis for debugging or correctness checking in many codes. However the round-off error and non-associativity of floating point makes attaining reproducibility a challenge in large-scale, long-term parallel computing or solving ill conditioned problems. The $\mathrm{dgbmv}$ performs general banded matrix-vector multiplication for double precision, is the most basic Level-2 operation in BLAS. First, we designed a reproducible algorithm for banded matrix-vector multiplication $\mathrm{repro}\_\mathrm{dgbmv}$ based on the technique of error-free transformation. Then the error of the algorithm is analyzed. Second, the algorithm is parallelized into $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ on ARM and x86 platforms. The numerical test results verify that $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ is reproducible and has higher accuracy than ordinary $\mathrm{dgbmv}$. In numerical experiments on ARM platform, as the number of threads increases from 1 to 8, the run time of this algorithm is reduced by 5.2–7 times, while the run time of multithreaded $\mathrm{dgbmv}$ is only reduced by 2.2–3.8 times. In numerical experiments on x86 platform, as the number of threads increases from 1 to 15, the run time of this algorithm is reduced by 7.7–10.6 times, while the run time of multithreaded $\mathrm{dgbmv}$ is only reduced by 4.2–6.8 times.

1. Introduction

The development in the field of high-performance computing (HPC) have been remarkable, with the computational scale of scientific and engineering calculations continuously increasing. The limited word length of computers leads to unavoidable rounding errors in floating-point operations [1]. The combination of dynamic scheduling of parallel computing resources, and floating point nonassociativity results in irreproducibility of floating-point calculation results. For instance, the continued summation or product of multiple floating-point numbers depends on the order of calculations.

Reproducibility is the ability to obtain a bit-wise identical and accurate result for multiple executions on the same data in various parallel environments [2]. We use a floating-point arithmetic that consists in approximating real numbers by a finite, fixed-precision representation number adhering to the IEEE 754 standard, which requires correctly rounded results for the basic arithmetic operations. The correct rounding criterion guarantees a unique, well-defined answer. The main idea is to keep track of both the result and the error during the course of computations.

In the 1960s and 1970s, Knuth, Kahan, and Dekker [3] proposed the idea of error-free transformations. In 2005, Japanese scholar Ogita, Oishi, and German scholar Rump [4,5,6] systematically summarized compensation algorithms and formally introduced the concept of error-free transformations(EFTs). One approach uses EFTs to compute both the result and the rounding error and stores them in a floating-point expansion (FPE), whose components are ordered in magnitude with minimal overlap to cover the whole range of exponents [7]. Typically, FPE relies upon the use of the traditional EFT for addition. Another approach projects the finite range of exponents of floating-point numbers into a long (fixed-point) accumulator and stores every bit there. For instance, Kulisch proposed to use a 4288-bit long accumulator for the exact dot product of two vectors composed of binary64 numbers with hardware supports [8].

Emerging attention to reproducibility strives to draw more careful attention to the problem by the computer arithmetic community. Static data scheduling and deterministic reduction ensure the numerical reproducibility of the Intel Math Kernel Library [9]. Nevertheless the number of threads has to be set for all runs. Starting from version 11.0, Intel’s MKL library introduced the Conditioned Numerical Reproducibility (CNR) mode [10]. This mode provides the capability to obtain reproducible floating-point results when calling library functions from the application, under the condition of a limited number of threads. Unfortunately this decreases significantly the performance especially on recent architectures, and requires the number of threads to remain the same from run to run to ensure reproducible results. Reproblas [11,12], developed by the University of California, Berkeley, utilizes pre-rounding and 1-Reduction techniques, and is designed for CPU and distributed parallel environments. However, the current version of this library only has the most basic functions and does not support thread-level parallelism. Exblas [13,14], developed by the University of Paris-Sud, combines together long accumulator and floating-point expansion into algorithmic solutions as well as efficiently tunes and implements them on various architectures, including conventional CPUs, Nvidia and AMD GPUs, and Intel Xeon Phi co-processors. Ozblas [15,16], developed by Tokyo Woman’s University in Japan, achieves reproducibility using the Ozaki scheme. This software supports adjustable precision on both CPUs and GPUs. However, these libraries are mainly developed to demonstrate the methods proposed in their respective research papers and are not yet widely used in practical applications [17].

The latest research is a general framework for deriving reproducible and accurate variants of a Krylov subspace algorithm, which proposed by Iakymchuk [18]. The framework is illustrated on the preconditioned BiCGStab method for the solution of non-symmetric linear systems with message-passing. The algorithmic solutions are build around the ExBLAS project.

Modern supercomputer architectures consist of multiple levels of parallelism. As the level of parallelism increases, the uncertainty of computations also increases [19,20,21]. The purpose of this paper is to analyze the blocking implementation approaches of the $\mathrm{dgbmv}$ function in the BLAS library, We designed and optimized a multi-threaded reproducible algorithm for banded matrix-vector multiplication.

The paper is organized as follows. Section 2 introduces the relevant knowledge of reproducible techniques. Section 3 describes how to achieve reproducibility in the banded matrix-vector multiplication algorithm and analyses the error bound. Then, the multi-level parallel optimization design we have performed on this algorithm is explained. Section 4 presents numerical experiments on the reproducibility, accuracy, and performance of the algorithm. The experimental results demonstrate that the algorithm is reproducible, efficient, and reliable.

2. Background

In scientific computing, the most basic operation is the addition of two floating-point numbers. The use of error-free transformations can greatly control the accumulation of rounding errors.

The algorithm $\mathrm{FastTwoSum}$ [3] is used to add two floating-point numbers a and b. The core idea is to transform the floating-point pair $(a,b)$ into the floating-point pair $(x,y)$ and use compensation to improve the calculation results. The calculation result is corrected by combining a high-order term x with a low-order term y.

The algorithm $\mathrm{ExtractVector}$ is mainly used for operations on a vector of floating-point numbers, where it transforms the elements $v_i$ of the vector v into the sum of a high-order part $q_i$ and a trailing part  $r_i$ [22]. T is the exact sum of $q_i$, and r is a vector composed of $r_i$. Our paper uses the reproducible k-fold accurate summation algorithm $\mathrm{repro}\_\mathrm{dsum}\_\mathrm{k}$ in to-nearest rounding mode (Algorithm 7, [22]). This algorithm requires $4kn+\mathcal{O}\left(1\right)$ FLOPs, where k is the number of fold, n is the length of vector v.

3. Parallel Reproducible Banded Matrix-Vector Multiplication

In this chapter, we will explain how we implemented the reproducible banded matrix-vector multiplication algorithm for double-precision in to-nearest rounding mode.

3.1. Basic Architecture

$\mathrm{dgbmv}$ is a special matrix function in BLAS Level 2. Both the memory access and computational complexity reach $\mathrm{O}\left(n^2\right)$. After special memory access processing, $\mathrm{dgbmv}$ converts to $\mathrm{dot}$ or $\mathrm{axpy}$ operations. Therefore, the performance heavily relies on the implementation of the level 1 algorithms.

The function $\mathrm{dgbmv}$ is defined as: $y=\alpha Ax+\beta y$, where A is an n-row by m-column banded matrix, with $a_{i,j}$ being the element at the i-th row and j-th column of A. $\alpha$ and $\beta$ are scalars, x is a vector of size m, with $x_i$ being the i-th element in x. y is a vector of size n, with $y_i$ being the i-th element in y. Let $ku$ and $kl$ be the number of superdiagonals and subdiagonals, respectively.

Taking $m=n$, $kl=ku=1$ as an example, A can be represented as follows:

$$\left(\begin{array}{cccccc}{a}_{1,1}& a_{1,2}& & & & \\ a_{2,1}& a_{2,2}& a_{2,3}& & & \\ & a_{3,2}& a_{3,3}& a_{3,4}& & \\ & & \dots & \dots & \dots & \\ & & & a_{n-1,m-2}& a_{n-1,m-1}& a_{n-1,m}\\ & & & & a_{n,m-1}& a_{n,m}\end{array}\right)$$

Compressed banded matrix exhibits special matrix properties. During storage, compressed banded matrices only store the effective computational elements (referring to the elements on the subdiagonal, main diagonal, and superdiagonal). During computation, dgbmv requires a one-to-one correspondence between matrix elements and vector elements in terms of computation order. The actual memory required for storage is smaller than the size of the uncompressed original matrix.

This paper mainly considers the row-major storage format. In memory, the banded matrix ignores the ineffective computational elements (referring to the subdiagonal and superdiagonal elements that do not participate in the computation) and rearranges them according to certain rules.

Each row has an offset added to the starting address. The offset for the first row is $ku$, and for each subsequent row, the offset is reduced by 1 until the offset becomes 0, after which each row is not offset.

Additionally, the effective computational elements of each row need to be recorded to ensure accurate and complete storage. From this, we can conclude that the memory access requires knowledge of the offset and the number of effective elements in each row. Therefore, the storage of matrix A is as follows:

$$\left(\begin{array}{ccc}& a_{1,1}& a_{1,2}\\ a_{2,1}& a_{2,2}& a_{2,3}\\ a_{3,2}& a_{3,3}& a_{3,4}\\ ⋮& ⋮& ⋮\\ a_{n-1,m-2}& a_{n-1,m-1}& a_{n-1,m}\\ a_{n,m-1}& a_{n,m}& \end{array}\right)$$

In actual calculation, the compressed banded matrix takes out each row based on the offset and the number of effective elements, and also takes out the corresponding vector portion that participates in the computation. It then uses $\mathrm{dot}$ to perform the entire matrix-vector multiplication. The pseudo-code of the specific algorithm implementation is shown in Algorithm 1.

Algorithm 1 Reproducible Compressed Banded Matrix-Vector Multiplication: $y=\mathrm{repro}\_\mathrm{dgbmv}(\alpha ,CA,x,\beta ,y)$

Require:  Given that matrix A is a banded matrix of size $n\times m$, with $ku$ subdiagonals and $kl$ superdiagonals, stored in the compressed matrix $CA$ of size $n\times (ku+kl+1)$. x is a vector of length m, $\alpha$ and $\beta$ are coefficients, and y is a vector of length n.   1: $offset\_u=ku+1$   2: $offset\_l=ku+m$   3: for $i=1$ to $min(n,m+ku)$ do   4:     $jstart=max(offset\_u,1)$   5:     $jend=min(offset\_l,ku+kl+1)$   6:     $length=jend-jstart+1$   7:     for $j=jstart$ to $jend$ do   8:        $tmp\left(j\right)=\alpha \ast A(i,j)\ast x\left(j-offset\_u\right)$   9:     end for 10:    $tmp(length+1)=\beta \ast y\left(i\right)$ 11:    $y\left(i\right)=\mathrm{repro}\_\mathrm{dsum}\_\mathrm{k}\left(tmp\right)$ 12:    $offset\_u=offset\_u-1$ 13:    $offset\_l=offset\_l-1$ 14: end for Ensure:  $y=\alpha Ax+\beta y$.

Since the multiplication of two floating-point numbers is reproducible, we calculate the product at each corresponding position and store it in an array $tmp(1:length)$. We also store the initial value of y multiplied by $\beta$ in $tmp(length+1)$. Then, we use the algorithm $\mathrm{repro}\_\mathrm{dsum}\_\mathrm{k}$ to perform the reproducible summation for the array $tmp$, with the number of folds k set to 2.

3.2. Error Analysis

Let $\epsilon$ be the machine precision, which is the distance between 1 and the closest floating-point number. $\epsilon =2^{-p}$, where p is the number of significant digits of a floating-point number. In double precision, $p=53$.

Theorem 1. 

In to-nearest rounding mode, for a vector v of length h and using the k-fold reproducible summation algorithm $T=\mathrm{repro}\_\mathrm{dsum}\_\mathrm{k}\left(v\right)$, when $h·\epsilon \ll 1$, the absolute error bound $E_{sum.k}$ between the numerical result and the exact result is:

$$\begin{array}{c}{E}_{sum.k}\le \left[k+(1+k\epsilon )2{\epsilon }^{k-1}{\theta }^k{h}^k\right]h\epsilon \underset{1\le j\le h}{max}\left(\left|v_j\right|\right),\end{array}$$

(1)

where $\theta \in [2,4]$.

Proof of Theorem 1. 

Let $v_j$ be the j-th element of v, and $S_0$ be the exact sum of all elements in v. In the i-th step of the error-free vector transformation, where $1\le i\le k$, we introduce the following notations: $M_i$ is the corresponding boundary; $q_{i,j}$ is the high-order part extracted in the i-th error-free vector transformation for each element; $T_i$ is the numerical result of the summation of $q_{i,j}$, which can be computed exactly; $r_{i,j}$ is the low-order part remaining after the i-th extraction; $S_i$ is the exact sum of $r_{i,j}$, which is used for proof purposes. On one hand, for EFTs,

$$\begin{gathered} T_1+\sum _{j=1}^h{r}_{1,j}-\sum _{j=1}^h{v}_j=\sum _{j=1}^h\left(v_j-\left(q_{1,j}+r_{1,j}\right)\right)=0, \\ {T}_i+\sum _{j=1}^h{r}_{i,j}-\sum _{j=1}^h{r}_{i-1,j}=\sum _{j=1}^h\left(r_{i-1,j}-\left(q_{i,j}+r_{i,j}\right)\right)=0, \end{gathered}$$

where $2\le i<k$. Combining the above two cases, we can get:

$$\begin{array}{c}\sum _{i=1}^k\left(T_i+S_i-S_{i-1}\right)=0.\end{array}$$

On the other hand,

$$\begin{array}{c}\left|S_{i-1}-T_i\right|=\left|S_i\right|\le 2·\epsilon ·h·M_i,\end{array}$$

Therefore,

$$\begin{array}{cc}\hfill \left|\sum _{i=1}^k{T}_i-S_0\right|& = \left|\sum _{i=1}^k\left(T_i+S_i-S_{i-1}\right)\right|+\left|T_k-S_{k-1}\right|\hfill \\ \hfill & \le 2·\epsilon ·h·M_k.\hfill \end{array}$$

By applying Lemma 5 from reference [22],

$$\begin{array}{c}{M}_i=c·{\theta }^i·h^i·{\epsilon }^{i-1}·\underset{1\le j\le h}{max}\left(\left|v_j\right|\right),\end{array}$$

we can deduce that

$$\begin{gathered} \left|\sum _{i=1}^k{T}_i-S_0\right|\le 2·{\epsilon }^k·h^{k+1}·{\theta }^k·\underset{j}{max}\left(\left|v_j\right|\right) \\ \hspace{1em} =:\varphi (k,\underset{j}{max}\left(\left|v_j\right|\right)), \end{gathered}$$

(2)

where $\varphi$ will be used in proof of Theorem 2. Therefore,

$$\begin{array}{cc}\hfill E_{sum.k}& = \left|T-\sum _{j=1}^h{v}_j\right|\le \left|T-\sum _{i=1}^k{T}_i\right|+\left|\sum _{i=1}^k{T}_i-S_0\right|\hfill \\ \hfill & \le k·\epsilon ·\left|\sum _{i=1}^k{T}_i\right|+\left|\sum _{i=1}^k{T}_i-S_0\right|\hfill \\ \hfill & \le k·\epsilon ·\left(\left|S_0\right|+\left|\sum _{i=1}^k{T}_i-S_0\right|\right)+\left|\sum _{i=1}^k{T}_i-S_0\right|\hfill \\ \hfill & \le k·\epsilon ·\left|S_0\right|+(1+k·\epsilon )·2·{\epsilon }^k·h^{k+1}·{\theta }^k·\underset{j}{max}\left(\left|v_j\right|\right).\hfill \end{array}$$

(3)

Furthermore, since

$$\begin{array}{c}\left|S_0\right|\le h·\underset{j}{max}\left(\left|v_j\right|\right),\end{array}$$

we can get (1) through simple derivation. □

Based on Theorem 1, we can derive Theorem 2. For convenience, we set $h=ku+kl+1$ is the number of elements in a row. In the banded matrix A, the ith row is denoted as $A_i$.

Theorem 2. 

In simplified algorithm $\mathrm{repro}\_\mathrm{dgbmv}$ to compute $Ax=y$, $A\in {\mathbb{R}}^{m\times n}$, $x\in {\mathbb{R}}^n$, $y\in {\mathbb{R}}^m$. $ku$ and $kl$ are the number of superdiagonals and subdiagonals, respectively. The error bound $E_{rep}$ for each element $y_i$ in the numerical solution y is:

$$\begin{array}{c}\hfill E_{rep\_dgbmv}\le \left[(1+2k)\epsilon +(1+(1+k)\epsilon +k{\epsilon }^2)\frac{{max}_i{E}_i}{1-{max}_i{E}_i}\right]·h·M,\end{array}$$

where

$$\begin{array}{c}{E}_i=2·{\theta }^k·{\epsilon }^k·h^{k+1}·M_i,\end{array}$$

and M is the vector of $M_i$, $i=1,\cdots ,m$, and $m_i$ is the maximum value of

$$\begin{array}{c}\left\{a_{i,j\mathit{start}}·x_{\mathit{jstart}},\cdots ,a_{i,jend}·x_{\mathit{jend}},\right.\left.\beta ·y_i,\right\}.\end{array}$$

Proof of Theorem 2. 

To perform a dot product operation between each row of A and x, it is necessary to first calculate the product result of each $a_{ij}$ and $x_i$, store it in a vector, and then perform a reproducible sum. We call this process as $\mathrm{repro}\_\mathrm{dot}$. By using $E_i=\varphi (k,M_i)$ in (2) and the same method in (3), the detailed calculation process for $A_i$ and x is as follows:

$$\begin{array}{cc}\hfill \mathrm{repro}\_\mathrm{dot}(A_i,x)\le (1+k·\epsilon )·(1+\delta ) ·& [a_{i1}{x}_i{(1+E_i/h)}^h+a_{i2}{x}_2{(1+E_i/h)}^{h-1}\hfill \\ \hfill & +\cdots +a_{in}{x}_n(1+E_i/h)]+k·\epsilon ·h·M_i,\hfill \end{array}$$

(4)

where $\delta <\epsilon$. From ${(1+E_i/h)}^h<eps\left(E_i\right)$, We can derive that

$$\begin{array}{cc}\hfill {(1+E_i/h)}^h-1& <E_i+\frac{E_i^2}{2!}+\frac{E_i^3}{3!}+\cdots \hfill \\ \hfill & <E_i\left[1+\frac{E_i}{2}+{\left(\frac{E_i}{2}\right)}^2+\cdots \right]\hfill \\ \hfill & =E_i\left[1/\left(1-\frac{E_i}{2}\right)\right]\hfill \\ \hfill & <\frac{E_i}{1-E_i}.\hfill \end{array}$$

(5)

By combining equality (4) and inequality (5), we can get

$$\mathrm{repro}\_\mathrm{dot}(A_i,x)<(1+k·\epsilon )·(1+\epsilon )·\left[1+\frac{E_i}{1-E_i}\right]·h·M_i+k·\epsilon ·h·M_i.$$

Then the error bound is

$$E_{rep\_dgbmv}\le (1+k)\epsilon ·h·M+(1+(1+k)\epsilon +k{\epsilon }^2)\frac{{max}_i{E}_i}{1-{max}_i{E}_i}·h·M+k·\epsilon ·h·M$$

where

$$\begin{array}{c}{E}_i=2·{\theta }^k·{\epsilon }^k·h^{k+1}·M_i,\end{array}$$

which is easy to deduce the result. □

Note that the error bound of $\mathrm{dgbmv}$ [1] is

$$\begin{array}{c}{E}_{dgbmv}\le \frac{h\epsilon }{1-h\epsilon }·h·M.\end{array}$$

Therefore if

$$h^k·{\epsilon }^{k-1}<<1,$$

$\mathrm{repro}\_\mathrm{dgbmv}$ has a smaller error bound than $\mathrm{dgbmv}$. As for $\alpha Ax+\beta y=y$, it’s easy to derive in a similar way.

3.3. Blocking and Parallel Optimization

We separated the compressed banded matrix A into blocks, and used multi-level parallel reproducible summation algorithms. The number of blocks is determined by the number of threads, $nthreads$. We used $\mathrm{blas}\_\mathrm{quickdivide}$ to allocate tasks to each block. The segmentation process is as follows: first, we calculated the number of rows that each block needs to allocate based on the number of threads and the number of matrix rows. Then, we used $\mathrm{exec}\_\mathrm{blas}$ to transfer the calculated control information to each thread. For example, when the number of threads is 4, the blocking is as shown below:

$$\left(\begin{array}{ccc}& a_{1,1}& a_{1,2}\\ a_{2,1}& a_{2,2}& a_{2,3}\\ ⋮& ⋮& ⋮\\ ⋮& ⋮& ⋮\\ ⋮& ⋮& ⋮\\ ⋮& ⋮& ⋮\\ ⋮& ⋮& ⋮\\ ⋮& ⋮& ⋮\\ a_{n-1,m-2}& a_{n-1,m-1}& a_{n-1,m}\\ a_{n,m-1}& a_{n,m}& \end{array}\right)$$

Finally, according to the received control information in different threads, we accessd A, x, and y, and used the $\mathrm{repro}\_\mathrm{dgbmv}$ algorithm to calculate. The calculated results are assigned to the corresponding positions of y.

We refered to this optimized multi-threaded compressed banded matrix-vector multiplication as $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$.

4. Numerical Experiments

In this chapter, we conducted numerical experiments to verify the reproducibility, accuracy, and performance of the $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ algorithm. In the reproducible summation used, we set the number of folds k to 2.

In experiments to verify reproducibility and accuracy, we generated a set of data using sine function to make the data more ill-conditioned, and store data in a banded matrix A of size $5000\times 5000$. The number of subdiagonals and superdiagonals is set to 500. We also generate floating-point vectors x and y of size 5000 using the sine function.

The experiments were conducted on two different testing platforms: an ARM processor (National University of Defense Technology, Changsha, Hunan) and a x86 processor Intel(R) Xeon(R) Platinum 8180M (Xeon Platinum CPU), with the parameters shown in the

Table 1. Hardware parameters of two platforms.

processor		ARM	x86
CPU clock speed		2.2 GHz	2.50 GHz
number of core		16	8
cache	L1I	48 KB	256 KB
	L1D	32 KB	256 KB
	L2	32 MB	8 MB
	L3	–	77 MB
memory size		16 × 8 GB	8 × 8 GB

.

4.1. Reproducibility Verification

We performed the ordinary banded matrix-vector multiplication algorithm $\mathrm{dgbmv}$ and the $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ algorithm on two different platforms with a single thread to verify the reproducibility.

We defined the irreproducibility rate as: the proportion of elements in the numerical results that have the same index but values differ between two results, when using the same program and the same set of input data.

To compare the numerical results, we calculated the absolute errors between different numerical results, as shown in Figure 1. In the figure, the result calculated on the ARM platform is denoted as $y_1$, and the result calculated on the x86 platform is denoted as $y_2$. The absolute error is calculated as:

$$\begin{array}{c}\mathrm{absolute}\_\mathrm{error}=y_1-y_2.\end{array}$$

To verify the reproducibility of the algorithm, we also calculated the maximum absolute error between the two results $y_1$ and $y_2$,

$$max\left(\right|y_1-y_2\left|\right)$$

as well as the irreproducibility rate, which is the ratio of the number of irreproducible elements to the total number of elements

$$=\mathrm{nonReproNum}/5000.$$

We showed the rate in

Table 2. Comparison of results executed by same algorithm on ARM and x86.

	Maximum Absolute Error	Irreproducibility Rate
$\mathrm{dgbmv}$	3.996802 $\times {10}^{-15}$	$83.38\%$
$\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$	0	$0\%$

.

From Figure 1 and Table 2, we can see that the $\mathrm{dgbmv}$ algorithm produces different results on the two platforms, with a maximum absolute error greater than 0. In the entire result vector, 83.38% of the elements are different. In contrast, the $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ algorithm produces results with a maximum absolute error of 0 on both platforms, and the irreproducibility rate is 0, which means all corresponding elements of the result vectors are exactly the same. This indicates that $\mathrm{dgbmv}$ is irreproducible, while $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ is reproducible.

4.2. Accuracy Verification

On the ARM platform, we denoted these two results of $\mathrm{dgbmv}$ and $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ as y and $ry$, and then calculate the accurate result $ey$ using MPFR [23]. We compare y and $ry$ with $ey$, and the error results are shown in Figure 2. The maximum absolute error between exact solution and numerical results is computed by $max\left(\right|y-ey\left|\right)=3.996802$$\times {10}^{-15}$, $max\left(\right|ry-ey\left|\right)=4.440892$$\times {10}^{-16}$.

It can be seen that the $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ has higher accuracy than the $\mathrm{dgbmv}$.

4.3. Performance Verification

We tested multiple sets of subdiagonals and superdiagonals for the matrix A of size $5000\times 5000$.

First, on the ARM platform, we used $\mathrm{dgbmv}$ and $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ to perform different sizes of matrices with single thread and 8 threads. We recorded the running time, as shown in Figure 3, and calculated speedup, as shown in Figure 4.

Furthermore, it could be calculated that with single thread, the running time of $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ is 4 to 4.2 times that of the $\mathrm{dgbmv}$. With 8 threads, the running time of $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ is 1.7 to 2.8 times that of the $\mathrm{dgbmv}$. Compared to $\mathrm{dgbmv}$, $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ has higher parallel efficiency.

Second, on the x86 platform, we used single thread, 15 threads, and 30 threads to execute $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$. The running time are shown in Figure 5, and the speedup between single thread and 15 threads is shown in Figure 6.

It could be calculated that the running time of $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$ with single thread is 7.7 to 10.6 times that of 15 threads, and the running time with 15 threads is 1.1 to 1.5 times that of 30 threads. However, the running time of $\mathrm{dgbmv}$ with single thread is 4.2 to 6.8 times that of 15 threads, and the running time with 15 threads is 0.8 to 1.2 times that of 30 threads.

Increasing the number of threads further does not improve the parallel efficiency or reduce the running time, because the data size processed by each individual thread is not large enough, leading to increased the proportion of communication time and decreased parallel efficiency.

5. Conclusions

In this paper, we optimized the banded matrix-vector multiplication algorithm based on the error-free vector transformation. We implemented the reproducible extension and high-precision improvement of the algorithm. In addition, We conducted error analysis for the algorithm. The program implementation of multi-threaded parallel reproducible banded matrix-vector multiplication algorithm was on the ARM and x86 platforms. The numerical experimental results verified the reproducibility and accuracy improvement of the $\mathrm{repro}\_\mathrm{dgbmv}\_\mathrm{thread}$, as well as the performance improvement and better parallel efficiency compared to $\mathrm{dgbmv}$.