Singular Value Decomposition in Embedded Systems Based on ARM Cortex-M Architecture

Abstract

Singular value decomposition (SVD) is a central mathematical tool for several emerging applications in embedded systems, such as multiple-input multiple-output (MIMO) systems, data analytics, sparse representation of signals. Since SVD algorithms reduce to solve an eigenvalue problem, that is computationally expensive, both specific hardware solutions and parallel implementations have been proposed to overcome this bottleneck. However, as those solutions require additional hardware resources that are not in general available in embedded systems, optimized algorithms are demanded in this context. The aim of this paper is to present an efficient implementation of the SVD algorithm on ARM Cortex-M. To this end, we proceed to (i) present a comprehensive treatment of the most common algorithms for SVD, providing a fairly complete and deep overview of these algorithms, with a common notation, (ii) implement them on an ARM Cortex-M4F microcontroller, in order to develop a library suitable for embedded systems without an operating system, (iii) find, through a comparative study of the proposed SVD algorithms, the best implementation suitable for a low-resource bare-metal embedded system, (iv) show a practical application to Kalman filtering of an inertial measurement unit (IMU), as an example of how SVD can improve the accuracy of existing algorithms and of its usefulness on a such low-resources system. All these contributions can be used as guidelines for embedded system designers. Regarding the second point, the chosen algorithms have been implemented on ARM Cortex-M4F microcontrollers with very limited hardware resources with respect to more advanced CPUs. Several experiments have been conducted to select which algorithms guarantee the best performance in terms of speed, accuracy and energy consumption.

1. Introduction

Singular value decomposition (SVD) is one of the basic and most important mathematical tools in modern signal processing. The method was firstly established by Beltrami and Jordan [1] and successively generalized by Autonne [2], Eckart and Young [3]. Since then SVD has successfully been applied on a huge number of different application fields, such as biomedical signal processing [4,5,6,7,8,9,10], image processing [11,12,13,14,15,16,17,18,19], Kalman filtering [20,21,22], array signal processing [23], dynamic networks [24], speech processing [25], simultaneous localization and mapping (SLAM) systems [26], and variable digital filter design [27], to cite just a few.

Various algorithms have been developed during the last decades for solving SVD [28], among these Golub–Reisch [29], Demmel–Kahan [30], Jacobi rotation [31], one-sided Jacobi rotation [32], and divide and conquer [33] algorithms are largely used since they guarantee good performance in common applications. An overview of SVD as well as the methods for its computation can be found in [34,35].

In recent years, there has been intense interest for emerging applications in embedded systems, such as multiple-input multiple-output (MIMO) systems [36,37], data analytics [38,39,40,41], sparse representation of signals [42,43,44,45,46,47], that require efficient SVD algorithms. Since SVD algorithms reduce to solve an eigenvalue problem, that is computationally expensive, both specific hardware solutions [48,49,50,51,52,53,54,55,56] and parallel implementations [57,58] have been proposed to overcome this bottleneck.

For example, optimized libraries such as Linear Algebra PACKage (LAPACK) and Basic Linear Algebra Subprograms (BLAS) (in their several implementations) that represent the state-of-the-art solution on computer-class CPUs, use several optimizations to efficiently take advantage of the hardware resources. An important optimization is reorganizing the operation flow and the data storage to make a better use of the processor’s cache memory, in order to speed-up the access to the large amount of data from and to the slower RAM. Another improvement comes from parallelism, that is splitting the problem in several smaller subsets and exploiting the parallel execution of modern multi-core CPUs. A further form of parallelism is given by single instruction multiple data (SIMD) technology, consisting of single instructions that perform the same operation in parallel on multiple data elements of the same type and size. For example, a CPU that normally adds two 32-bit values, performs instead four parallel additions of 8-bit values in the same amount of time. Therefore, SIMD makes better use of the available resources in a microprocessor observing that when performing operations on large amount of data on a microprocessor, parts of the computation units are unused, while continuing to consume power.

It is worth to notice that all the SVD algorithms exhibit a large quantity of data-level parallelism, making them well suited to be implemented with SIMD acceleration and multi-core parallelism.

However, none of the cited hardware resources (cache memory, parallel execution units) exist in microcontrollers, so all the relevant optimization techniques are not useful on this class of embedded systems. About SIMD, actually the Cortex-M4 and newer embedded architectures offer a primitive form of SIMD instructions that are limited to operate on 8 or 16-bit integer values, so they are not suitable for complex computations requiring higher precision floating-point values for their stability.

Regarding more complex architectures, there is a rich literature on how to implement mathematical algorithms on Field Programmable Gate Arrays (FPGAs) and other programmable hardware [53,54,55,56]. Those are generally sophisticated and expensive systems, used for high-end applications and exploiting a completely different computation model, based on massive parallelism. Conversely the scope of this paper is on affordable, low-power and low-resource embedded systems, generally based on microcontrollers, as universally used in distributed scenarios like for example the Internet of Things concept.

Many embedded systems have a very limited amount of available memory, especially data memory, and may not employ an underlying operating system to further reduce resource usage. Because of this, the existing optimized libraries intended for computer-class CPUs are not readily usable in embedded systems, nor easily portable. For this reason, the demand of specific accelerating techniques that are able to implement SVD algorithms in embedded processors, has seen a growing interest in the past few years [59].

Regarding the SVD implementation a large variety of algorithms exist, thus from the embedded system designer point of view, comparing the performance achieved with different algorithms implemented in a high-performance embedded processor is of great interest.

The aim of this paper is to present an efficient implementation of the SVD algorithm on ARM Cortex-M (Arm Ltd., Cambridge, UK). More specifically, as the SVD algorithm is a central mathematical tool for several emerging applications in embedded systems and, to our knowledge, no previous implementations of this algorithm exists in literature for bare-metal microcontrollers, i.e., without an operating system, it is of paramount importance to develop a library suitable for these devices, specifically for the ARM Cortex-M4F architecture [60]. To achieve this goal, we proceeded:

(i)

to provide a comprehensive treatment of the most common SVD algorithms, writing the existing formulations found in literature in a more compact and uniform manner, suitable to be used as a comprehensive reference;

(ii)

to implement them on an ARM Cortex-M4F (Arm Ltd., Cambridge, UK). microcontroller, in order to develop a library that is suitable for embedded systems that work without an operating system. As already mentioned, LAPACK and BLAS represent the state-of-the-art solution on computer-class CPUs, but these optimized libraries are not readily available and easily portable for embedded systems, because of the very limited available memory and the lack of an underlying operating system of these devices. To this end, we decided to implement five common SVD algorithms (Golub–Reinsch, Demmel–Kahan, Jacobi rotation, one-sided Jacobi rotation and divide and conquer) in order to adapt them to operate in limited resource embedded systems and without an operating system.

(iii)

to find the best implementation that fits the low-resources of bare-metal embedded systems. For the purpose of finding the algorithms with the best performance for an embedded system, this paper reports a comparative study of their performance in terms of several implementation metrics: speed, accuracy and energy consumption.

(iv)

to show a practical real-time application of the SVD in an embedded system, by using the proposed optimized library for ARM Cortex-M4F in the Kalman filtering of an inertial measurement unit (IMU). This example also proofs the advantage of implementing the SVD on this class of systems, showing the improvement of the numerical accuracy of Kalman filtering by applying the SVD algorithm with respect to the conventional Kalman approach.

Thus, the paper is a good starting point for researchers and practitioners interested in developing new algorithms to be implemented in low-resource microcontrollers.

This paper is organized as follows. Section 2 is mainly focused on the five most representative SVD algorithms and gives a comprehensive treatment of them, with Appendix A summarizing the basic concepts of matrix algebra related to SVD transformation. Section 3 reports a comparative study of the five algorithms performance. Section 4 presents an application of SVD to Kalman filtering of data from an IMU, showing important improvements with respect to traditional algorithm implementation. Finally, some conclusions end this work.

2. Algorithms for the Singular Value Decomposition

Let A be a real $(m\times n)$ matrix with $m\ge n$. It is known that the decomposition

$$A=U\Sigma V^T$$

(1)

where

$$U^{T}U=V^{T}V=V{V}^T=I,\Sigma =\mathrm{diag}({\sigma }_1,\dots ,{\sigma }_n)$$

(2)

exists [35]. The matrix U consists of n orthonormal eigenvectors corresponding to the n largest eigenvalues of $A{A}^T$, and the matrix V consists of the orthonormal eigenvectors of $A^{T}A$. The diagonal elements of $\Sigma$ are the non-negative square roots of the eigenvalues of $A{A}^T$, called singular values. Assuming

$${\sigma }_1\ge {\sigma }_2\ge \dots \ge {\sigma }_n\ge 0$$

(3)

thus if $rank\left(A\right)=M$, it results ${\sigma }_{M+1}={\sigma }_{M+2}=\dots ={\sigma }_n=0$. The decomposition (1) is called the singular value decomposition (SVD) of matrix A.

2.1. Golub–Reinsch Algorithm

This method, developed by G. H. Golub and C. Reinsch [29], acts directly on the matrix A thus avoiding unnecessary numerical inaccuracy due to the computation of $A^{T}A$. The algorithm can be divided into these consecutive steps:

(i)

Householder’s bidiagonalization (see Appendix A.2 for details);

(ii)

implicit QR method with shift (see Appendix A.6 for details);

The pseudo-code of the algorithm is reported in Algorithm 1.

Algorithm 1 Golub–Reinsch

Require: $A\in {\mathbb{R}}^{m\times n}(m\ge n)\in$ a small multiple of the unit round-off      Use Algorithm A1 to compute bidiagonalization.        $\left[\begin{array}{c}B\\ 0\end{array}\right]\leftarrow {(U_1\dots U_n)}^{T}A(V_1\dots V_{n-2})$        Repeat      for $i=1,\dots ,n-1$ do          •  Set $b_{i,i+1}$ to zero if $|b_{i,i+1}|\le ϵ(|b_{ii}|+|b_{i+1,i+1}\left|\right)$          •  Find the largest q and the smallest p such that if          $B=\begin{array}{l}\left[\begin{array}{ccccc}{B}_{11}& & 0& & 0\\ 0& & B_{22}& & 0\\ 0& & 0& & B_{33}\end{array}\right]\begin{array}{l}p\\ n-p-q\\ q\end{array}\\ \begin{array}{cccccc}& p& n-p-q& q& & \end{array}\end{array}$          than $B_{33}$ is diagonal and $B_{22}$ has a nonzero superdiagonal.          if $q=n$ then               STOP          end if          if any diagonal entry in $B_{22}$ is zero then               zero the superdiagonal entry in same row          else               apply the Algorithm A3          end if      end for

2.2. Demmel–Kahan Algorithm

The structure of the algorithm is based on the Golub–Reinsch algorithm previously described. Nevertheless the Demmel–Kahan algorithm [30] can achieve a better accuracy for small singular values. This algorithm consists of the following main consecutive steps:

(i)

Householder’s bidiagonalization (see Appendix A.2 for details);

(ii)

QR iteration with zero-shift (see Appendix A.7 for details);

The description of this algorithm is shown in Algorithm 2.

Algorithm 2 Demmel–Kahan

Require: $A\in {\mathbb{R}}^{m\times n}(m\ge n)\in$ a small multiple of the unit round-off      Use Algorithm A1 to compute bidiagonalization.        $\left[\begin{array}{c}B\\ 0\end{array}\right]\leftarrow {(U_1\dots U_n)}^{T}A(V_1\dots V_{n-2})$        Repeat      for $i=1:n-1$ do          •  Set $b_{i,i+1}$ to zero if a relative convergence criterion is met          •  Find the largest q and the smallest p such that if          $B=\begin{array}{l}\left[\begin{array}{ccc}{B}_{11}& 0& 0\\ 0& B_{22}& 0\\ 0& 0& B_{33}\end{array}\right]\begin{array}{l}\mathrm{p}\\ \mathrm{n-p-q}\\ q\end{array}\\ \begin{array}{cccccc}& \mathrm{p}& \mathrm{n-p-q}& \mathrm{q}& & \end{array}\end{array}$          than $B_{33}$ is diagonal and $B_{22}$ has a nonzero superdiagonal.          if $q=n$ then               STOP          end if          if any diagonal entry in $B_{22}$ is zero then               zero the superdiagonal entry in same row          else               apply the implicit zero-shift QR algorithm          end if      end for

2.3. Jacobi Rotation Algorithm

In this case given the real and symmetric matrix $A\in {\mathbb{R}}^{n\times n}$ the algorithm [31] aims to obtain a diagonal matrix $B\in {\mathbb{R}}^{n\times n}$ through the transformation

$$B=J^{T}AJ$$

(4)

where J represents a sequence of rotation matrices. In particular for the k-th rotation or sweep can be rewritten as

$$A_{k+1}=J_k^T{A}_k{J}_k,A_0=A$$

(5)

where $J_k=J(p,q,\theta )$ is the Jacobi rotation matrix that rotates rows and columns p and q of $A_k$ through the angle $\theta$ so that the $(p,q)$ and $(q,p)$ entries are zeroes. The p and q values are chosen properly at each iteration step. With reference to the sub matrices corresponding to the $p,q$ columns we have

$$\left[\begin{array}{cc}{b}_{pp}& b_{pq}\\ b_{qp}& b_{qq}\end{array}\right]={\left[\begin{array}{cc}c& s\\ -s& c\end{array}\right]}^T\left[\begin{array}{cc}{a}_{pp}& a_{pq}\\ a_{qp}& a_{qq}\end{array}\right]\left[\begin{array}{cc}c& s\\ -s& c\end{array}\right]$$

(6)

The key point of the algorithm is to determine the rotation coefficients c and s in such a way the off-diagonal terms $b_{pq}$ and $b_{qp}$ are zeroed.

The algorithm is reported in Algorithm 3.

Algorithm 3 Jacobi rotation

Require: $A\in {\mathbb{R}}^{n\times n}$ symmetric      $B\leftarrow A\in {\mathbb{R}}^{n\times n}$      Repeat      for $i=1:n-1$ do         for $j=i+1:n$ do            Compute the rotations coefficients $s,c$ such that $${\displaystyle \left[\begin{array}{cc}{b}_{ii}& 0\\ 0& b_{jj}\end{array}\right]}={\left[\begin{array}{cc}c& s\\ -s& c\end{array}\right]}^T\left[\begin{array}{cc}{a}_{ii}& a_{ij}\\ a_{ji}& a_{jj}\end{array}\right]\left[\begin{array}{cc}c& s\\ -s& c\end{array}\right]$$ (7)            if $\mathrm{off}\left(A\right)<ϵ$, where $\mathrm{off}\left(A\right)=\sqrt{{\sum }_{i\ne j}{a}_{ij}^2}$ and $ϵ$ is a small multiple of the unit            round-off then               STOP            end if         end for      end for

2.4. One-Sided Jacobi Rotation Algorithm

The main idea of this algorithm [32] is to rotate columns i and j of A through the angle $\theta$ so that they become orthogonal to each other. In such a way the $(i,j)$ element of $A^{T}A$ is implicitly zeroed resulting in the scalar product of the $i,j$ columns.

Let $J(i,j,\theta )$ be the Givens matrix that when applied to the matrix A yields

$$B=(b_{:1}\cdots b_{:i}\cdots b_{:n})=AJ=(a_{:1}\cdots a_{:i}\cdots a_{:n})\left[\begin{array}{ccccc}1& \cdots & 0& \cdots & 0\\ \ddots & & c& s& \\ & & -s& c& 0\\ 0& 0& \cdots & 0& 1\end{array}\right]$$

(8)

where the $i,j$ columns of B are given by

$$\begin{array}{c}\hfill b_{:i}=c{a}_{:i}-s{a}_{:j}\\ \hfill b_{:j}=s{a}_{:i}+c{a}_{:j}\end{array}$$

(9)

The key point of the algorithm is to determine the rotation coefficients c and s in such a way the elements ${\left(B^{T}B\right)}_{ij}$ and ${\left(B^{T}B\right)}_{ji}$ of the product $B=AJ(i,j,\theta )$ are zeroed.

The pseudo-code of the algorithm is reported in Algorithm 4.

Algorithm 4 One-sided Jacobi rotation

Require: $A\in {\mathbb{R}}^{n\times n}$ symmetric      $B\leftarrow A\in {\mathbb{R}}^{n\times n}$      Repeat      for $i=1:n-1$ do         for $j=i+1:n$ do            Compute the rotations coefficients $s,c$ such that $${\displaystyle \left[\begin{array}{cc}{b}_{ii}& 0\\ 0& b_{jj}\end{array}\right]}=\left[\begin{array}{cc}{a}_{ii}& a_{ij}\\ a_{ji}& a_{jj}\end{array}\right]\left[\begin{array}{cc}c& s\\ -s& c\end{array}\right]$$ (10)            if $\mathrm{off}\left(A\right)<ϵ$, where $\mathrm{off}\left(A\right)=\sqrt{{\sum }_{i\ne j}{a}_{ij}^2}$ and $ϵ$ is a small multiple of the unit            round-off then               STOP            end if         end for      end for

2.5. Divide and Conquer Algorithm

For a square symmetric matrix A a relationship between singular values and eingenvalues, as well as between singular vectors and eigenvectors exists. Indeed, as A can be diagonalized we can write

$$A=Q\Lambda Q^T=A\mathrm{sign}(\Lambda )|\Lambda |Q^T$$

(11)

where $\Lambda$ is the eigenvalue diagonal matrix and Q is a unitary matrix (orthogonal in real case). Since we can always assume that the elements of $|\Lambda |$ are in decreasing order, then to the diagonalization (11) corresponds an SVD such that $U=Q\mathrm{sign}(\Lambda ),\Sigma =|\Lambda |$ and $V^T=Q^T$. In words the singular values are the absolute values of eigenvalues and the singular vectors are the eigenvectors (same norm and direction, but not necessary the same versus). For a non symmetric matrix the singular values and the singular vectors are not directly related to the eigenvalues and eigenvectors, instead there is a strict relationship with eigenvalues and eigenvectors of the symmetric matrices $A^{T}A\in {\mathbb{R}}^{M\times M}$ and $A{A}^T\in {\mathbb{R}}^{N\times N}$. In fact it can be easily shown that

$$\begin{array}{c}\hfill A^{T}A=V{\Sigma }^2{V}^T\\ \hfill A{A}^T=U{\Sigma }^2{U}^T\end{array}$$

(12)

where A is decomposed as $A=U\Sigma V^T$. From (12) it results that the singular values of A are eigenvalues of the matrices $A^{T}A$ and $A{A}^T$ (except those equal to zero for the latter). Additionally the right and left singular vectors are the eigenvectors of $A^{T}A$ and $A{A}^T$ respectively, which can differ for a sign (note that the matrix A can be correctly reconstructed provided the correct sign is known). Therefore, on the basis of previous considerations, the singular value decomposition reduces to the eigenvalue problem of a symmetric matrix. In general, the methods for solving such a problem are iterative methods that include two stages:

(i)

in the first stage the matrix A is transformed to a matrix B whose structure makes the computation of the eigenvalues and eigenvectors easier. A typical choice, that is assumed here, is a tridiagonal form.

(ii)

in the second stage an iterative method is applied to determine the eigenvalues and eigenvectors.

With reference to the second stage the divide and conquer algorithm aims at reducing a complex problem to a singular one [33]. The algorithm is intended to be applied to a tridiagonal and symmetric matrix T of dimension $N\times N$.

The pseudo-code of the algorithm is reported in Algorithm 5.

Algorithm 5 Divide and conquer

Require:  $T\in {\mathbb{R}}^{n\times n}$ tridiagonal symmetric        Divide T as $T=\left[\begin{array}{cc}{T}_1& 0\\ 0& T_2\end{array}\right]+\rho u{u}^T$        for $i=1,2$ do         Compute ${\Lambda }_i,Q_i$ eigenvalues/vectors of $T_i$ as follows:               if $T_i$ suitably small then            Compute eigenvalues/vectors directly         else            Apply recursively divide and conquer algorithm to $T_i$         end if      end for      Use factorized $T_1$, $T_2$ to compute $D+\rho v{v}^T$, where $$D=\left[\begin{array}{cc}{\Lambda }_1& 0\\ 0& {\Lambda }_2\end{array}\right],v=\left[\begin{array}{cc}{Q}_1^T& 0\\ 0& Q_2^T\end{array}\right]u$$ (13)      Compute eigenvalues $\Lambda$ by solving the secular equation $D+\rho v{v}^T=Q\Lambda Q^T$ through      Li’s algorithm        Compute eigenvectors as $\left[\begin{array}{cc}{Q}_1& 0\\ 0& Q_2\end{array}\right]Q$

3. Experimental Results

The five algorithms described in the previous section have been implemented both in MATLAB on a desktop computer and on a Cortex-M4F microcontroller.

The algorithms have been tested with several matrices $A\in {\mathbb{R}}^{m\times n}$. To ensure that the algorithms worked in the most general cases, several sets of matrices of increasing size have been chosen, each set with a different row/column ratio. Considering the limited memory of the microcontroller, three sets of matrices have been tested, with a ratio of $m/n=1$, $\frac{4}{3}$ and 2, respectively. Further increasing the ratio would have caused the matrices to not fit into the limited memory of the microcontroller without significantly reducing their rank (memory occupation must take into account not only the input and output matrices, but also the intermediate matrices and vectors needed by the various algorithms). Indeed, preference has been given to maintaining comparable ranks between the set of matrices. The low amount of RAM memory also limits the absolute size of matrices, so smaller matrices have been used for the experiments on the microcontroller, with respect to the tests in MATLAB.

The biggest matrix A has been randomly generated, and all the other matrices used are the top-left portion of the full matrix A.

For algorithms requiring a symmetric matrix, the input matrix is converted to a symmetric form as follows. First it is converted to bidiagonal form by applying Algorithm A1, then, being B the upper triangular portion of the bidiagonal matrix, the SVD algorithm is applied to $B{B}^T$. In such a way the singular values of the original matrix are the square roots of eigenvalues of the latter tridiagonal matrix.

3.1. MATLAB Implementation

The algorithms have initially been implemented in MATLAB to test their correctness, on 6-core, 12-thread Intel i7 CPU with 32 GB RAM (Intel, Santa Clara, CA, USA).

Figure 1 shows, as a matter of comparison, the timings of the various algorithms as implemented in MATLAB, as functions of the number of columns n for a set of matrices having $m/n=\frac{4}{3}$.

Table 1. Average errors of SVD algorithms implemented in MATLAB vs. MATLAB built-in.

Size	Golub-Reinsch	Demmel-Kahan	Jacobi Rot.	One-Sided J.	Divide-Conquer
32 × 24	0	2.0 × 10${}^{-8}$	0	0	0
48 × 36	0	2.7 × 10${}^{-8}$	0	0	0
96 × 72	0	5.4 × 10${}^{-8}$	0	0	0
128 × 96	0	7.9 × 10${}^{-8}$	0	0	0
160 × 120	0	8.6 × 10${}^{-8}$	0	0	0
200 × 150	0	1.9 × 10${}^{-7}$	0	0	0

reports the accuracy of a sample of the same tests, with the algorithms implemented in MATLAB with respect to the MATLAB built-in SVD function. The errors reported are computed as the average of relative errors between the singular value matching pairs, as computed by MATLAB built-in function and the given routines.

As you can see, all the algorithms, with the exception of Demmel–Kahan, ensure the same accuracy of the MATLAB built-in SVD function. This could be expected, as the Demmel–Kahan algorithm was specifically designed to deal well with very small singular values, while the random-generated matrices we employed in our experiments have singular values close to unity.

3.2. Cortex-M4F Implementation

Cortex-M4F implementations of the different algorithms were tested on a STM32F429ZI microcontroller from STMicroelectronics, Geneva, Switzerland, (https://www.st.com/en/microcontrollers-microprocessors/stm32f429zi.html), mounted on an 32F429IDISCOVERY evaluation board (https://www.st.com/en/evaluation-tools/32f429idiscovery.html).

The STM32F429ZI microcontroller is based on an ARM 32-bit Cortex-M4F [60] CPU clocked at 180 MHz, with 2 MB of flash memory for code and read-only data, and 256 KB of RAM. In addition it has several hardware peripherals that are not relevant to this work.

A main feature of the Cortex-M4F core is the presence of a 32-bit hardware floating-point unit (FPU), as implied by the additional “F” in its name. An FPU is essential for any kind of heavy computational work in the floating-point domain, as is the case for the experiments on SVD performed in this article. The Cortex-M4F FPU is limited to 32 bits (https://developer.arm.com/docs/ddi0439/latest/floating-point-unit/about-the-fpu), so the algorithms have been implemented using single-precision (32 bits) values. Implementing this kind of algorithms on a CPU with no FPU, or with larger precision than that managed by the hardware, would require the use of software floating-point mathematical libraries, that would be prohibitive in an already resource-constrained system.

The algorithms were implemented in C language. No particular development environment was used, the code was compiled with the GCC software suite for ARM on a GNU-Linux machine, using a custom makefile and with the aid of the STM32F4xx Standard Peripherals Drivers (STMicroelectronics, Geneva, Switzerland), a set of libraries provided by ST for their specific microcontrollers and encompassing all aspects of hardware management, from low-level initialization to use of hardware peripherals. The firmware is of the “bare-metal” kind, so no real-time operating system (RTOS) or other middlewares have been added.

The hardware system requested no particular design besides what was already provided by the 32F429IDISCOVERY board (STMicroelectronics, Geneva, Switzerland). The device has been clocked at its maximum speed of 180 MHz. The board also integrates all the hardware needed for programming and debugging the microcontroller, namely the ST-LINK/V2 interface (STMicroelectronics, Geneva, Switzerland), offering USB connectivity with the computer. On the computer side, communication with such interface has been established by using OpenOCD (http://openocd.org), a free software for debugging and programming of ARM and other systems. OpenOCD finally acts as a server for GDB, the standard debugger from the GCC suite, used when needed to transfer the code to the device and examine its memory for the results of the tests.

Regarding input and output, read-only data for the program, like the bigger matrix from which smaller ones are generated, or the reference vectors of singular values to compute the accuracy, are stored in the program memory (flash memory) that is more abundant than RAM. Once the program is run for a series of tests, the numerical outputs can be examined through the debugger, by interrupting the program in convenient points. The timing of the single routines is computed by the software itself, using the SysTick timer built in the Cortex-M core.

Besides the optimizations performed by the compiler, special care has been exercised in trying to optimize the critical mathematical routines, while keeping a low usage of program memory and needed RAM, in order to speed up the computation as much as possible while fitting in the constrained resources of the system.

A first optimization comes from choosing the best arrangement of data in memory. Bidimensional matrices can be stored in RAM (which is a linear array of bytes) in two different ways: row-major or column-major order, that is storing data sequentially by row or column respectively (see Figure 2). The former one is the most common way in the C language or anywhere the mathematical convention of having the row as the first index is followed. This has a much more crucial impact in CPUs with cache memory, where cache is filled with sequential data from RAM, and so it gives a huge speed boost in accessing sequential data with respect to sparse ones. As said, a microcontroller has no cache memory, so this is not directly the case; nevertheless a non-negligible advantage exists in accessing data sequentially, due to the load/store assembly instructions with auto-increment, that is instructions that read or write data from memory and increment the address register in a single execution cycle.

As a quantitative example of the effect of row-major or column-major data storing, we can consider the one-sided Jacobi rotation algorithm, that differs from the other ones for accessing the input matrix exclusively by column.

Table 2. Timing differences for one-sided Jacobi rotation with respect to data arrangement in memory.

Matrix Size	Row-Major	Column-Major	Speed Increase	Transposition
$48\times 24$	19.6 ms	19.1 ms	2.6%	0.07 ms
$72\times 36$	73.1 ms	71.9 ms	1.7%	0.15 ms
$96\times 48$	192 ms	189.7 ms	1.2%	0.25 ms
$120\times 60$	370.6 ms	366.8 ms	1%	0.39 ms
$144\times 72$	634 ms	628.6 ms	0.9%	0.55 ms

shows the different timings of the algorithm for the biggest set of matrices, both in absolute times and in percentage of speed increase. As said, the increase in speed is minimal, even if appreciable. The last column shows the time needed to transpose the matrix, in the case it is stored in the opposite way, which is approximately one order of magnitude less than the time improvement obtained. Moreover, often the input matrix to SVD is generated from previous computations, and so it can be generated already in the more convenient order.

Besides accessing data in the convenient order, that results in a modest speed increment, and lacking hardware resources to be further exploited, other optimizations must be necessarily obtained by reducing the unneeded computations as much as possible. A significant example is matrix multiplication, one of the most computationally expensive operations in matrix algebra. Generally speaking, if $C=A·B$, the generic element of C is given by

$$c_{i,j}=\sum _{k=1}^n{a}_{i,k}{b}_{k,j}$$

(14)

where n is the number of columns of A. Computing all the elements of C in this way requires a triple nested loop which is very computationally expensive, especially for large matrices.

The number of operations performed for a matrix multiplication can be reduced by observing the properties of the specific matrices. For example a recurrent operation in the exposed algorithms requires the multiplication of a square matrix by its transpose, like $C=A·A^{\prime }$. In this case (14) becomes

$$c_{i,j}=\sum _{k=1}^n{a}_{i,k}{a}_{j,k}.$$

(15)

First we can notice that in the inner loop A is always traversed by row, so we can have the advantage of reading data always in the most convenient order if the matrix is stored in row-major order. Most importantly it can easily be seen that $A·A^{\prime }$ is a symmetric matrix, so we can actually compute approximately half of its elements, sensibly reducing the number of operations (Figure 3).

A further reduction of the number of operations is possible in the case of $C=A·A^{\prime }$ where A is an upper-diagonal matrix, another common case in the given algorithms. Given that $a_{i,j}\ne 0$ only for $j=i$ or $j=i+1$, from (15) follows that $c_{i,j}\ne 0$ only for $j=i-1$, i or $i+1$ (indeed the resulting matrix is tridiagonal) and that the only non-zero terms in the sum are those for which $a_{i,k}$ and $a_{j,k}$ are both non-zero. The resulting formula is:

$$c_{i,j}=\left\{\begin{array}{cc}{a}_{i,i}{a}_{j,i}+a_{i,i+1}{a}_{j,i+1},\hfill & j=i\hfill \\ a_{i,i+1}{a}_{j,i+1},\hfill & j=i+1\hfill \\ a_{i,i}{a}_{j,i},\hfill & j=i-1\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(16)

where the ${(i+1)}^{th}$ element may not exist. Being also symmetric, the reduction of the previous case also applies (Figure 4).

Another special example is the multiplication of a matrix by a Givens matrix, in the form of (A14), to perform a Givens rotation. Let us call G the Givens matrix to avoid confusion with indices, and let’s limit for simplicity to the case of left multiplication, as in $C=G·A$. If we initially set $C=A$, it is clear from the definition of G that only rows p and q of C are affected by the multiplication. Moreover, only elements p and q of a given column of A are used in the computation (Figure 5). So the only elements of C that need to be updated are those at rows p and q, and their values are:

$$c_{p,j}=g_{p,p}{a}_{p,j}+g_{p,q}{a}_{q,j},c_{q,j}=g_{q,p}{a}_{p,j}+g_{q,q}{a}_{q,j}$$

(17)

for every column j. A similar formula holds for right-side multiplication.

The complexity of the previous computations, corresponding to matrix products for different special cases of input matrices, can be compared in terms of number of scalar multiplications with respect to the size of input matrices. Results are shown in

Table 3. Number of scalar multiplications in matrix products for different types of $n\times n$ input matrices.

Matrix Product	Scalar Products	Code Size (Bytes)
$C=A·B$, generic	$n^3$	148
$C=A·A^{\prime }$	$n^3/2+n^2/2$	210
$C=A·A^{\prime }$, A	$3n-1$	304
upper-diagonal
$C=G·A$, Givens rotation	$4n$	108

, together with the code size of the specific software routines.

The impact of such optimizations on computation speed can be measured, in particular those from (15) and (16) and from the property of symmetry (Givens rotation is never actually implemented as a matrix multiplication, since its matrix is expressly constructed to only modify a few elements in the result).

Table 4. Speed increase from optimized matrix multiplication algorithms.

Matrix Size	Golub-Reinsch	Demmel-Kahan
$48\times 24$	21%	16%
$72\times 36$	18%	16%
$96\times 48$	36%	23%
$120\times 60$	44%	16%
$144\times 72$	18%	16%

shows the speed increase for the set of biggest matrices when using two algorithms where these optimizations are mostly relevant.

Besides trying to optimize mathematical routines, it is important to avoid the problems arising from the limited power and precision of the microcontroller’s FPU. In case of operations that can be carried out in different ways, choosing the right procedure can make a substantial difference. For example the assembly multiplication instruction (VMUL.F32) takes 1 clock cycle to execute, while the division (VDIV.F32) takes 14 cycles (https://developer.arm.com/docs/ddi0439/b/floating-point-unit/fpu-functional-description/fpu-instruction-set). In some cases the compiler can automatically substitute an expensive operation with a cheaper one during the optimization phase of compilation, for example when dividing by a constant.

Another problem arising from the limits of the FPU is the loss of precision in certain operations in the 32-bit domain. For example a recurring problem in the given algorithms is computing $1/\sqrt{1+x^2}$. This kind of operation causes the loss of many significant bits in the original value of x when $x\ll 1$. It must be verified experimentally when this is tolerable and when not. In some cases the loss of precision causes a worsening of the final accuracy by an order of magnitude. A possible solution is switching temporarily to 64-bit precision, then converting back to single precision when the sensitive computation is done. Of course this sensibly increases the execution time, using software libraries instead of the hardware FPU. A better solution is applying the logarithm to the value to be computed, performing intermediate computations in the logarithm domain and finally applying exponentiation. In this case $log(1/\sqrt{1+x^2})=-0.5log(1+x^2)$, which can take advantage of a special function in the C mathematical library, called “log1pf”, that is optimized to compute $log(1+x)$ with high accuracy even if the value of x is near zero. Tests show that using logarithm and exponentiation software libraries while remaining in the 32-bit domain is faster and gives similar or better results than computing the root square in software in double precision.

Figure 6 shows the timings of the full SVD algorithms implemented on the microcontroller for the three sets of matrices of different row/column ratio, with respect to the total matrix size ($m\times n$). The same results are also reported in

Table 5. Timings of SVD algorithms on Cortex-M4F.

Size	Golub-Reinsch	Demmel-Kahan	Jacobi Rot.	One-Sided J.	Divide-Conquer
24 × 24	17 ms	22 ms	28 ms	13 ms	18 ms
36 × 36	78 ms	133 ms	112 ms	45 ms	76 ms
48 × 48	229 ms	441 ms	310 ms	114 ms	223 ms
60 × 60	681 ms	1100 ms	693 ms	218 ms	523 ms
72 × 72	1092 ms	1232 ms	1396 ms	408 ms	1061 ms
32 × 24	28 ms	34 ms	39 ms	16 ms	29 ms
48 × 36	130 ms	186 ms	164 ms	57 ms	127 ms
64 × 48	388 ms	463 ms	482 ms	131 ms	381 ms
80 × 60	939 ms	1272 ms	1100 ms	252 ms	906 ms
96 × 72	2963 ms	2878 ms	2182 ms	430 ms	1847 ms
48 × 24	58 ms	64 ms	69 ms	19 ms	58 ms
72 × 36	283 ms	299 ms	316 ms	72 ms	279 ms
96 × 48	1115 ms	1434 ms	970 ms	190 ms	870 ms
120 × 60	2429 ms	2280 ms	2299 ms	367 ms	2103 ms
144 × 72	4427 ms	4669 ms	4672 ms	628 ms	4336 ms

, but with the three matrix sets listed separately.

From the experimental results it can be seen that the time of the algorithms is roughly dependent on the total matrix size, but with irregularities suggesting that other factors, like the matrix rank, affect the total time.

On a comparative basis, as you can see the one-sided Jacobi rotation algorithm gives the lowest execution time, compared to the others.

Table 6. Average errors of SVD algorithms on Cortex-M4F vs. MATLAB built-in.

Size	Golub-Reinsch	Demmel-Kahan	Jacobi Rot.	One-Sided J.	Divide-Conquer
24 × 24	3.8 × 10${}^{-7}$	7.8 × 10${}^{-7}$	1.0 × 10${}^{-6}$	1.9 × 10${}^{-7}$	2.2 × 10${}^{-6}$
36 × 36	4.8 × 10${}^{-7}$	2.3 × 10${}^{-6}$	1.7 × 10${}^{-6}$	3.5 × 10${}^{-7}$	7.0 × 10${}^{-6}$
48 × 48	4.7 × 10${}^{-7}$	7.1 × 10${}^{-6}$	3.0 × 10${}^{-6}$	2.4 × 10${}^{-7}$	6.6 × 10${}^{-6}$
60 × 60	5.3 × 10${}^{-7}$	4.8 × 10${}^{-6}$	7.3 × 10${}^{-7}$	3.0 × 10${}^{-7}$	4.3 × 10${}^{-6}$
72 × 72	4.6 × 10${}^{-7}$	2.6 × 10${}^{-6}$	1.7 × 10${}^{-6}$	3.4 × 10${}^{-7}$	4.7 × 10${}^{-6}$
32 × 24	2.7 × 10${}^{-7}$	5.7 × 10${}^{-7}$	2.2 × 10${}^{-7}$	1.7 × 10${}^{-7}$	2.9 × 10${}^{-7}$
48 × 36	3.6 × 10${}^{-7}$	2.4 × 10${}^{-6}$	4.0 × 10${}^{-7}$	1.7 × 10${}^{-7}$	2.6 × 10${}^{-7}$
64 × 48	2.9 × 10${}^{-7}$	2.7 × 10${}^{-6}$	2.9 × 10${}^{-7}$	1.7 × 10${}^{-7}$	1.9 × 10${}^{-7}$
80 × 60	4.1 × 10${}^{-7}$	5.9 × 10${}^{-6}$	5.4 × 10${}^{-7}$	2.4 × 10${}^{-7}$	4.0 × 10${}^{-7}$
96 × 72	1.6 × 10${}^{-6}$	6.5 × 10${}^{-6}$	6.0 × 10${}^{-7}$	2.7 × 10${}^{-7}$	3.5 × 10${}^{-7}$
48 × 24	3.0 × 10${}^{-7}$	1.7 × 10${}^{-6}$	1.6 × 10${}^{-7}$	1.7 × 10${}^{-7}$	9.5 × 10${}^{-8}$
72 × 36	2.7 × 10${}^{-7}$	6.5 × 10${}^{-7}$	3.7 × 10${}^{-7}$	1.5 × 10${}^{-7}$	1.4 × 10${}^{-7}$
96 × 48	1.0 × 10${}^{-6}$	8.7 × 10${}^{-6}$	4.1 × 10${}^{-7}$	1.8 × 10${}^{-7}$	1.5 × 10${}^{-7}$
120 × 60	5.9 × 10${}^{-7}$	3.9 × 10${}^{-6}$	4.8 × 10${}^{-7}$	2.0 × 10${}^{-7}$	1.5 × 10${}^{-7}$
144 × 72	4.8 × 10${}^{-7}$	5.3 × 10${}^{-6}$	5.8 × 10${}^{-7}$	3.1 × 10${}^{-7}$	1.3 × 10${}^{-7}$

reports the accuracy of the specific tests, implemented on the microcontroller, with respect to the MATLAB built-in SVD function. As in Section 3.1, the errors reported are computed as the average of relative errors of matching pairs of singular values, as computed by MATLAB built-in function and the given routines. As you can see, the accuracy of Cortex-M4F implementation is significantly lower than the equivalent MATLAB code; this is due to the lower precision (32 bits) of the Cortex-M4F hardware floating-point unit. In this case, both the one-sided Jacobi rotation and divide and conquer algorithms achieve a better accuracy than the others.

Finally,

Table 7. Energy consumption of SVD algorithms on Cortex-M4F.

Size	Golub-Reinsch	Demmel-Kahan	Jacobi Rot.	One-Sided J.	Divide-Conquer
32 × 24	5.7 mJ	7.1 mJ	10.9 mJ	1.2 mJ	5.8 mJ
48 × 36	26.7 mJ	30.6 mJ	38.3 mJ	9.0 mJ	26.1 mJ
64 × 48	79.7 mJ	87.8 mJ	113.6 mJ	26.5 mJ	78.8 mJ
80 × 60	193.5 mJ	217.5 mJ	250.7 mJ	50.9 mJ	187.2 mJ
96 × 72	594.6 mJ	1150.9 mJ	487.8 mJ	86.7 mJ	382.1 mJ

reports the energy consumption of the tests relative to one matrix set, measured by sampling the voltage and current with an INA226 from Texas Instruments (https://www.ti.com/product/INA226). The INA226 is a current and voltage monitor with I²C interface, with a maximum gain error of 0.1%, maximum input offset of 10 µV and 16-bit resolution. A 0.1 $\Omega$ shunt resistor has been used in series with the microcontroller power supply, and the data have been acquired through an external single-board computer.

As you can see, the results are coherent with execution times. As a matter of comparison, Figure 7 shows side-by-side the current consumption of the five algorithms for one of the matrices. The one-sided Jacobi rotation algorithm, besides being faster, clearly has the lowest average current consumption. It is worth noting that the other algorithms exhibit the same pattern at the beginning, corresponding to the Householder’s bidiagonalization step. This step has therefore a significant relevance, both in time and energy, for the algorithms that need it.

4. Application: Kalman Filtering of Inertial System Data

As a practical application to an embedded system, and as a proof of the advantage of implementing the SVD on this class of systems, we will show how SVD can be used to improve the numerical accuracy of Kalman filtering applied to an inertial measurement unit (IMU). Sensors included in an IMU are typically accelerometers and gyroscopes. Data from such sensors, with proper filtering and data-fusion techniques, allow the computation of the spatial orientation of a moving body (its attitude), namely the relative angles of the body with respect to the spatial axes.

Accelerometers have the problem of high-frequency noise added to the signal, together with additional acceleration terms when a force is applied to the body. Gyroscopes measure the angular rate of change around each axis, that must be integrated to get angular positions; this leads to a sensible drift in the resulting data, because gyroscopes are affected by a bias in the measurement that is non-constant and unpredictable, resulting in a drift increasing with time.

Kalman filter [61] is a common technique used to estimate the state of such a system (namely the current angles along the chosen axes) given a set of discrete measurements over time that are subject to statistical noise. To that end, the system dynamics must be written in form of space-state equations:

$$\begin{array}{c}\hfill x_t=A{x}_{t-1}+B{u}_{t-1}+w_{t-1}\end{array}$$

(18)

$$\begin{array}{c}\hfill y_t=C{x}_t+v_t\end{array}$$

(19)

where $x_t$ is the unknown system state vector to be estimated, vector $u_t$ is the control input to the system and $y_t$ is the measurement vector. w and v are additive noise on process and measurement data respectively, both assumed to be zero-mean Gaussian processes: $w_t\sim \mathcal{N}(0,Q_t)$, $v_t\sim \mathcal{N}(0,R_t)$. Initial state condition $x_0$ is also a random variable: $x_0\sim \mathcal{N}(0,{\Pi }_0)$.

It is shown in [22] that traditional Kalman filter implementations, like square-root algorithms based on the Cholesky decomposition, are effective with well-conditioned problems but may fail if equations are ill-conditioned, due to roundoff errors, as those algorithms rely on matrix inversion in several places. On the other hand [22] shows how an SVD-based Kalman filter variant works much better in ill-conditioned cases.

In the IMU example, a case of ill-conditioned problem is when multiple sets of the same measurements are available. To better show a practical case, we report results obtained with an X-NUCLEO-IKS01A2 (https://www.st.com/en/ecosystems/x-nucleo-iks01a2.html) board, a sensor board from ST equipped, among others, with a gyroscope-accelerometer combined sensor and a magnetometer-accelerometer one. The board can be used with a microcontroller unit, in our case we used a NUCLEO-F411RE (https://www.st.com/en/evaluation-tools/nucleo-f411re.html), with a Cortex-M4F-based microcontroller similar to the one used in the previous experiments.

For this hardware system the firmware has been developed using the STM32CubeIDE development platform, version 1.4.0. An additional software package, X-CUBE-MEMS1, has been installed in the IDE through its integrated software package manager. This package supports the communication with the specific sensors in the X-NUCLEO-IKS01A2 inertial system. In this case the microcontroller (STM32F411RE) is clocked at 84 MHz. Like in the previous experiment, the firmware is “bare-metal”, with no RTOS or other middlewares added. Communication with the board for programming and debugging is entirely managed by the IDE in this case.

Having two accelerometers available, it is convenient to use both data to get a better estimate of the state, given the noisy nature of the device. Mathematically, matrices in Equations (18) and (19) can be expressed as:

$$A=\left[\begin{array}{cccc}1& -\Delta t& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& -\Delta t\\ 0& 0& 0& 1\end{array}\right]B=\left[\begin{array}{cc}\Delta t& 0\\ 0& 0\\ 0& \Delta t\\ 0& 0\end{array}\right]C=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 1& 0\\ 1+\delta & 0& 0& 0\\ 0& 0& 1+\delta & 0\end{array}\right]$$

(20)

and variables as:

$$x=\left[\begin{array}{c}\widehat{\varphi }\\ {\widehat{b}}_{\varphi }\\ \widehat{\theta }\\ {\widehat{b}}_{\theta }\end{array}\right]u=\left[\begin{array}{c}{\dot{\varphi }}_G\\ {\dot{\theta }}_G\end{array}\right]y=\left[\begin{array}{c}{\widehat{\varphi }}_{A1}\\ {\widehat{\theta }}_{A1}\\ {\widehat{\varphi }}_{A2}\\ {\widehat{\theta }}_{A2}\end{array}\right]$$

(21)

Here the system state includes the estimated roll and pitch angles ($\widehat{\varphi }$, $\widehat{\theta }$) along x and y axes respectively and the estimated gyroscope bias on both angles (${\widehat{b}}_{\varphi }$, ${\widehat{b}}_{\theta }$); system inputs include angular rates from gyroscope (${\dot{\varphi }}_G,{\dot{\theta }}_G$); $\Delta t$ is the period of acquired data; system measurements are given by estimated angles from both accelerometer data (${\widehat{\varphi }}_{A1},{\widehat{\theta }}_{A1},{\widehat{\varphi }}_{A2},{\widehat{\theta }}_{A2}$).

The $\delta$ factor accounts for the fact that the two series of accelerometer data are very similar numerically, leading to an ill-formed problem: the matrix is now nearly singular, due to redundancy of system measurement variables. As shown in [22], we can expect that resolution of Kalman filter equations by means of conventional algorithms will suffer from roundoff errors when $\delta$ becomes small enough to be comparable to machine precision. We can also expect that the SVD-based algorithm will work better in such cases.

The firmware on the board has been used to export a batch of measurements (gyroscope, accelerometer 1, accelerometer 2) to the computer, so that the data could be examined in Matlab in a reproducible way. The data have a duration of about 90 s, with a $\Delta t$ of 10 ms, and have been obtained by moving the device through a set of reference positions, namely 0, 90 and −90 degrees on the three axes.

Then the Kalman filter has been computed in Matlab with both algorithms (conventional and SVD-based), repeating the experiment for various decreasing values of $\delta$, until approaching values comparable to machine precision.

Finally the accuracy of the estimated state has been evaluated by computing the mean absolute error (in degrees) with respect to the reference intervals of known orientations, for all the different values of $\delta$.

Table 8. Matlab: mean absolute errors ($\varphi$, $\theta$) of estimated angles (degrees).

Algorithm	$\mathit{\delta }={10}^{-8}$	$\mathit{\delta }={10}^{-9}$	$\mathit{\delta }={10}^{-10}$	$\mathit{\delta }={10}^{-11}$	$\mathit{\delta }={10}^{-12}$
Conventional	1.7337 1.1066	1.7337 1.1066	NaN NaN	NaN NaN	NaN NaN
SVD-based	1.7337 1.1066	1.7337 1.1066	1.7357 1.1065	1.7113 1.3738	23.6834 27.0080

shows that the two algorithms give very similar results down to a certain value of $\delta$, but at lower values the conventional algorithm fails (“NaN” means Not-a-Number, an error condition indicating an invalid numeric value). The SVD-based algorithm, conversely, keeps working with neglectable difference for other two orders of magnitude of $\delta$, starting to degrade at a much lower value.

To perform a similar experiment with the limited capabilities of an embedded system, the same algorithms for the Kalman filter have been implemented in a C program on the same set of off-line data, using the custom SVD algorithms discussed in the previous sections and with floating-point numbers in single precision (32 bit), that is the same precision of the floating-point hardware of the Cortex-M4F architecture. To just focus on the difference between conventional and SVD-based Kalman filter, we used one only SVD algorithm, namely the One-Sided Jacobi, that proved to be the best one under several criteria (Section 3). The other algorithms have shown to give similar results.

Preliminary tests have shown that with limited numeric precision, as in this case, the conventional Kalman algorithm is much more sensible to variations of initial conditions. Actually, the difference between the two Kalman implementations is evident even in the single accelerometer case, by varying the relative values of the covariance matrices involved.

As a clarifying example, a series of tests is shown with fixed values ${\Pi }_0=I_4$ and $Q=I_4$, data from a single accelerometer and varying values of R. Results are shown in

Table 9. Single precision C code: mean absolute errors ($\varphi$, $\theta$) of estimated angles (degrees).

Algorithm	$\mathit{R}={\mathit{I}}_2$	$\mathit{R}=0.8{\mathit{I}}_2$	$\mathit{R}=0.6{\mathit{I}}_2$	$\mathit{R}=0.4{\mathit{I}}_2$	$\mathit{R}=0.2{\mathit{I}}_2$
Conventional	2.7197 1.3642	2.7191 1.3820	2.7124 1.4480	NaN NaN	NaN NaN
SVD-based	2.7186 1.3477	2.7173 1.3494	2.7159 1.3516	2.7143 1.3543	2.7124 1.3580

, which highlights a trend similar to the Matlab case: the two algorithms are very similar for some cases, but the SVD-based one works uniformly on a wider range of parameters.

About the actual performances on a Cortex-M4F microcontroller, the matrices involved in the SVD-based Kalman algorithm are much smaller than the ones in Table 5, having a maximum size of 8x4 in the case of double accelerometer. An extrapolation of the results in Table 5 leads to an estimate of fractions of millisecond, so the computation of Kalman filter using the SVD-based variant should be possible in real time in most applications.

5. Conclusions

This paper presents a comparative study of five of the most common SVD algorithms, namely Golub–Reinsch, Demmel–Kahan, Jacobi rotation, one-sided Jacobi rotation, divide and conquer. The aim of this comparative study is to find the most suitable algorithm for an embedded system. The chosen algorithms have been investigated starting from a theoretical perspective giving a detailed mathematical description, which provides crucial hints for the practical implementations. The algorithms have been initially written in MATLAB to verify their correctness, then implemented on a Cortex-M4F microcontroller to achieve optimized results for low-resource embedded systems. The comparison of the Cortex-M4F implementations shows that the one-sided Jacobi rotation outperforms the other algorithms in terms of speed, accuracy and energy consumption. Moreover the divide and conquer algorithm shows similar accuracy results. Finally the SVD has been applied to an example case, showing that it can improve the accuracy of algorithms where the data are ill-conditioned, while traditional implementations may fail.