1. Introduction
Applications where Toeplitz matrices arise commonly involve computation of inverses, products, and eigenvalues. Although inverses and products of Toeplitz matrices are not in general Toeplitz, inverses and products of circulant matrices are circulant. Moreover, unlike for Toeplitz matrices, an eigenvalue decomposition is known for any circulant matrix. Hence, the substitution of Toeplitz matrices by circulant matrices leads to a notable reduction of the computational complexity in those applications.
A sequence of Toeplitz matrices generated by a function f can be found in any information-theoretic or statistical signal processing application in which a wide sense stationary (WSS) process appears. In those applications, is the sequence of correlation matrices of the WSS process and the function f is its power spectral density (PSD).
The substitution of the sequence
by a sequence of circulant matrices can be done when both sequences are asymptotically equivalent. In [
1], Gray gave an example of such a sequence of circulant matrices
, where each circulant matrix
depends on the entire sequence
. However, in practical situations, what we usually know is
for certain natural number
. For those practical situations, Pearl presented in [
2] a different sequence of circulant matrices
, which is also asymptotically equivalent to the sequence of Toeplitz matrices
, but this satisfies the condition that each circulant matrix
only depends on the corresponding Toeplitz matrix
.
In the present paper, the sequence is generalized to block circulant matrices. This sequence of block circulant matrices is used to obtain a result on the rate-distortion function (RDF) of WSS vector processes that allows us to reduce the complexity of coding those processes. In addition, we use that sequence to reduce the complexity of filtering WSS vector processes.
The paper is organized as follows. In
Section 2, we set up notation and review the mathematical definitions and results used in the rest of the paper. In
Section 3, we define the sequence of block circulant matrices considered in this paper and show its main properties. Finally, in
Section 4 and
Section 5, we use this sequence of block circulant matrices to reduce the complexity of coding and filtering WSS vector processes, respectively.
2. Preliminaries
2.1. Notation
In this paper, , , , and denote the set of natural numbers (i.e., the set of positive integers), the set of integer numbers, the set of (finite) real numbers, and the set of (finite) complex numbers, respectively. If , then , , and are the set of all complex matrices, the zero matrix, and the identity matrix, respectively. The symbol ∗ denotes conjugate transpose, E stands for expectation, is the imaginary unit, denotes trace, δ stands for the Kronecker delta, ⊗ is the Kronecker product, , , are the eigenvalues of an Hermitian matrix A arranged in decreasing order, and , , are the singular values of an matrix B arranged in decreasing order.
Let
be a random
N-dimensional vector process, i.e.,
is a random (column) vector of dimension
N for all
. We denote by
the random vector of dimension
given by
2.2. Block Toeplitz Matrices
We first review the concept of block Toeplitz matrix.
Definition 1. An block Toeplitz matrix with blocks is an matrix of the formwhere with . Consider a matrix-valued function of a real variable
, which is continuous and
-periodic. For every
, we denote by
the
block Toeplitz matrix with
blocks given by
where
is the sequence of Fourier coefficients of
F:
It should be mentioned that
is Hermitian for all
if and only if
is Hermitian for all
(see [
3] (Theorem 4.4.1)). Furthermore, in this case, from [
3] (Theorem 4.4.2) (it was previously given in [
4] (p. 5674) but without a proof) and [
5] (Corollary VI.1.6), we obtain
2.3. Block Circulant Matrices
We first review the concept of a block circulant matrix.
Definition 2. An block circulant matrix with blocks is an block Toeplitz matrix with blocks of the formwhere with . The next result [
6] (Lemma 3) characterizes block circulant matrices.
Lemma 1. Let ; then, the following statements are equivalent:- 1.
C is an block circulant matrix with blocks.
- 2.
There exist such thatwhere and is the Fourier unitary matrix
2.4. Asymptotically Equivalent Sequences of Matrices
We now review the concept of asymptotically equivalent sequences of matrices introduced in [
6] (Definition 2), which is an extension of the original concept given by Gray in [
7] to sequences of non-square matrices.
Definition 3. Consider two strictly increasing sequences of natural numbers and . Let and be matrices for all . We say that the sequences and are asymptotically equivalent, and write , if and are bounded, andwhere and are the spectral norm and the Frobenius norm, respectively (The definition and the main properties of these two matrix norms can be found, e.g., in [3] (Section 2.1)). We finish this section by reviewing [
6] (Lemma 4).
Lemma 2. Let be continuous and -periodic. Then,where is the block circulant matrix with blocks given by When
and
F is in the Wiener class (We recall that a function
is said to be in the Wiener class if it is continuous and
-periodic, and it satisfies
, see, e.g., [
3] (Appendix B)),
is the sequence of circulant matrices defined by Gray in [
1] (Equation (4.32)), see [
3] (Lemma 5.2).
3. Sequence of Block Circulant Matrices Considered
We begin this section by presenting a sequence of block circulant matrices , where each block circulant matrix only depends on the corresponding block Toeplitz matrix .
Definition 4. Let be continuous and -periodic. For every , we define as the block circulant matrix with blocks given by When
and the matrices
are real and symmetric,
is the sequence of circulant matrices defined by Pearl in [
2].
The following result gives an expression for the blocks of the block circulant matrix .
Lemma 3. Consider and let be continuous and -periodic.- 1.
If , thenwhere is the sign function. - 2.
If is real, then is real.
Proof. (1) Fix
. For convenience, we denote
by
. We have
where
denotes the conjugate of
.
(2) It is a direct consequence of Assertion (1). ☐
When
and the matrices
are real and symmetric, from Lemma 3, we obtain [
2] (Equation (7)).
We now show that the block Toeplitz matrix can be approximated by the block circulant matrix for large n.
Lemma 4. If is continuous and -periodic, then- 1.
for all , where .
- 2.
.
Proof. (1) From [
3] (Theorem 4.3)
for all
. Consequently, to prove Assertion (1), we only need to show that
for all
. Fix
. As the spectral norm is unitarily invariant (see, e.g., [
3] (
Section 2.1)) and
and
are unitary matrices, we have
Consider
and
satisfying
Let
with
for all
, then
(2) From Lemma 1,
is an
block diagonal matrix with
blocks:
Hence, since the Frobenius norm is unitarily invariant (see, e.g., [
3] (
Section 2.1)), one obtains
Thus, from Lemma 2, we conclude that
☐
We now show that keeps several properties of for the case in which .
Lemma 5. Consider and let be continuous and -periodic.- 1.
If is Hermitian, then is Hermitian.
- 2.
If is real and symmetric, then is real and symmetric.
- 3.
If is positive semidefinite, then is positive semidefinite.
- 4.
If is positive definite, then is positive definite.
Proof. (1) As is Hermitian, is Hermitian. Therefore, is Hermitian for all , and, consequently, is also Hermitian. Hence, is Hermitian.
(2) If is real and symmetric, then is Hermitian. Applying Lemma 3 and Assertion (1), is real and Hermitian, and hence, is real and symmetric.
(3) For every
, let
with
for all
. If
then
where
and
for all
. Since
is positive semidefinite,
for all
, and, thus,
.
(4) Consider
. Since
is positive definite, it is positive semidefinite, and from Assertion (3), we have
. Suppose that
. As
is positive definite, applying Assertion (
2) yields
for all
. Consequently,
for all
, and, therefore,
for all
. Hence,
, and thus,
. ☐
Finally, we show that Assertion (
1) is also true if
is replaced by
.
Lemma 6. Let be continuous and -periodic. If is Hermitian for all then Proof. Applying Lemma 5
is Hermitian for all
. Fix
and
. From Assertion (
2), we have
Suppose that
is an eigenvector of
with
. Consequently,
for some
. Let
be a unitary diagonalization (i.e., an eigenvalue decomposition where the eigenvector matrix
is unitary) of
for all
. Then,
Since
we obtain
which completes the proof. ☐
4. Coding WSS Vector Processes by Using the Sequence of Block Circulant Matrices Considered
Consider that
is a real zero-mean Gaussian
N-dimensional vector process. From [
8], we know that the RDF of the real zero-mean Gaussian vector
is given by
where
is a real number satisfying
We recall that represents the lowest possible required rate (measured in nats) for jointly encoding (compressing) n symbols from N sources with mean square error (MSE) distortion D.
We assume that
is WSS with continuous PSD
X and
. Fix
, and from Assertion (
1), we obtain that
, and, consequently,
For every
, let us compute the discrete Fourier transform (DFT) of the
N sources as
The correlation matrix of
is given by
It is clear from Assertion (
3) that the lowest possible required rate for encoding
with MSE distortion
D is the same as for
, i.e.,
.
Let us compress the Gaussian vector
as if the samples
and
were uncorrelated for all
with
, or, equivalently, as if its correlation matrix were
instead of
. In this case, from Lemma 6, we obtain that the corresponding rate is given by
We now study the rate loss
. We have
Applying Assertion (
1) yields
and hence, from Lemma 4, we conclude that
The obtained result (
4) on the RDF of WSS vector processes shows that there is no rate loss for large enough
n if we consider the correlation matrix of
as if it were a block diagonal matrix. Obviously, encoding all the samples
separately involves a notably lower computational complexity than encoding them jointly. The complexity of coding a WSS vector process in this way is
if the fast Fourier transform (FFT) algorithm is used.
5. Filtering WSS Vector Processes by Using the Sequence of Block Circulant Matrices Considered
Consider a zero-mean WSS M-dimensional vector process with continuous power spectral density (PSD) X. Let be a zero-mean WSS N-dimensional vector process with continuous PSD Y. Assume that those two processes are jointly WSS with continuous joint PSD Z.
For every
, if
is an estimation of
from
of the form
with
, the MSE per sample is given by
The minimum MSE (MMSE) is given by
, where
whenever
(or, equivalently, whenever
is positive definite). The filter
is known as the Wiener filter.
Consider the following filter:
The filter is well defined because, from Lemma 5, is positive definite, and, therefore, it is invertible.
We now study the effect on the MSE when the optimal filter
is substituted by
. We have
Consequently, applying Assertion (
1) and Lemma 4 yields
If we assume that
, from Assertion (
1), we obtain that
is positive definite for all
. Moreover, applying Assertion (
1) and Lemma 6 yields
and, therefore, from Lemma 4, we conclude that
The obtained result (
6) shows that there is no difference in the MSE for large enough
n if we substitute the optimal filter
by
. Obviously, the computational complexity of the operation (
5) is notably reduced when applying this substitution and the FFT algorithm is used. Specifically, the complexity is reduced from
to
.
6. Conclusions
In this paper, we present a sequence of block circulant matrices and we apply it to reduce the complexity of coding and filtering WSS vector processes. Specifically, in both applications, the complexity is reduced from to , which is the complexity of performing an FFT.