Source Quantization and Coding over Noisy Channel Analysis

Abstract

Recently, lossy source coding based on linear block code has been designed using the duality principle, i.e., the channel decoding algorithm is employed to realize the lossy source coding. However, the quantization structure has not been analyzed in this compression technique, and the codebook design does not match the source characteristics well. Hence, the compression performance is not so good. To overcome this problem, the codebook design is correlated with the quantization structure in this work. It is found that the lossy source coding based on the linear block code can be defined as lattice vector quantization (VQ), which provides a new analytical perspective for the coding methodology. Then, the VQ scheme is generalized with the noisy channel to evaluate the transmission robustness of the continuous source compression. Finally, the codebook of the VQ scheme is optimally designed by uniforming the radiuses of the quantization subspace to reduce the quantization distortion. The proposed codebook outperforms existing codes in terms of its proximity to the rate–distortion limit, while also exhibiting enhanced robustness against channel noise.

1. Introduction

Lossy source coding provides a high-efficiency method for data compression, which is useful for saving communication bandwidth and reducing hardware complexity in the physical layer [1,2,3]. Conventionally, linear block codes, such as low-density parity-check (LDPC) codes, are employed for source and channel coding due to their good hardware realizability [4,5,6]. The code matrices of linear block codes are different since the source coding aims to realize data compression and the channel coding is for error correction. However, this is not a hardware-friendly design in chip manufacturing. Recent works have embraced the principle of “duality”, which suggests repurposing the channel code as a source code and implements channel-decoding techniques for the lossy source coding [7,8]. Based on the principle of duality, the cost of chip manufacturing can be reduced by reusing the source and the channel coding modules in circuit implementation.

It has been demonstrated that LDPC code has a capacity-approaching property, and can be applied for channel coding with good error correction performance. The duality principle indicates that lossy compression based on LDPC code can also approach the rate–distortion limit of the binary source [9,10]. To simplify the parity-check matrix of the LDPC code, the protograph LDPC (P-LDPC) code is introduced to compress the binary sources with equal probability [11]. Compared with the parity-check matrix of the LDPC code, the protomatrix of the P-LDPC code has lower encoding and decoding complexities [12,13,14]. Referring to the duality principle, the belief propagation (BP) algorithm, which achieves good decoding performance, is used to implement the lossy source coding. For instance, the reinforced BP (RBP) algorithm accomplishes the lossy source coding in [15]. Then, the shuffled RBP algorithm was proposed to improve the encoding efficiency for compressing the binary source [11]. Subsequently, the lossy compression of the Gaussian source was considered. Ref. [16] proposes a multi-level coding (MLC) structure to preprocess the floating sequence by mapping one float into a bit string. To further increase the compression efficiency, the BP algorithm is modified to transform one float into one bit in [17]. In addition, the source model is extended to the general source by applying neural networks (NNs) in [18], including the Gaussian and Laplacian source models.

The aforementioned works focus on improving the compression efficiency using different methods which can be classified into two categories; one optimally searches the better codebooks, the other designs the BP-like algorithms. The optimization principle of these works is to improve the code matrices and BP-like algorithms based on the mutual information transfer (MIT) strategy [17]. The MIT strategy assumes the minimum distortion to be the maximum mutual information, and it is an optimization criterion for searching the objective codebook.

However, the rate–distortion performances of these works still need to be improved. The main reasons are the following: First, more coding gains are obtained in channel coding than lossy source coding since the data compression generates more distortion and its structure is more complicated. Second, the MIT strategy is only appropriate for optimizing lossy compression based on the determined source, i.e., it lacks universality for variable sources. Since the codebook and the coding algorithm should be designed based on the source characteristic, it is difficult to achieve compatible performance for different source distributions. Third, it is unreasonable to evaluate data compression systems using only the MIT strategy since the shape of the quantization space should also be another optimization criterion. In consequence, a new strategy should be explored as a supplement to the MIT strategy.

Lossy source coding based on linear block code can be regarded as a VQ scheme since the codeword is N-dimensional. The VQ is a classical quantization technique in signal processing that allows the modeling of probability density functions by the distribution of prototype vectors [19,20,21,22,23,24,25]. Refs. [26,27] indicate that the compression performance mainly depends on the spatial structure of the quantizer. In this case, the rate–distortion performance can be optimized by improving the quantization space. The coding relation between the quantization space and the linear block code is shown in Figure 1. The codebook optimization of the linear block code aims to allocate the quantization space according to the source distribution appropriately since the codebook contains all the central codewords of the quantization subspaces. Hence, the radius of the quantization subspace can be determined by the coding property of the linear block code.

Referring to Figure 1, the lossy source coding based on block code can be defined as a lattice VQ scheme since the shapes of the quantization subspaces are regular. As the lattice VQ scheme has some advantages [28], namely, intuitive architecture and simple realization, the coding methodology will be simplified. The error pattern of the block code is expressed as the radius of a quantization subspace. Usually, all the error patterns are similar in one codeword, so all the quantization subspaces have the same shape. However, the quantization subspaces are inhomogeneous in N-dimensional spaces. In this case, each source codeword has an unequal distance from the center codeword at any dimension, which will increase the quantization distortion.

There is a technical challenge in the existing VQ scheme in terms of compressing the source with variable statistical properties. This source compression needs a more complicated quantization structure, which will exponentially increase the design complexity. To overcome this challenge, ref. [29] designed a VQ scheme based on block code, and showed that the codebook based on the short block code will generate less quantization distortion. However, split and multistage structures are considered in the VQ scheme with higher complexities. In addition, its codebook is not designed using the duality principle. Hence, the structural complexity and performance improvement needed to be solved in [29].

To overcome the above issues, a VQ scheme based on linear block code is proposed and the P-LDPC code is designed based on the quantization structure in this work. The motivations can summarized as follows: First, the preprocessing module is modified with a scalar quantizer and linear decoding is used in the proposed VQ scheme, which will significantly reduce the structural complexity. The VQ structure is employed for the codeword matching module for quantization, which is implemented by the BP algorithm with lower complexity. Second, the codebook is optimized based on the quantization structure, where the error correction property is used to improve the compression performance. With the codebook optimization, all the quantization subspaces become homogeneous to improve the quantization performance. In this case, the proposed VQ scheme is hardware friendly with lower complexity and better compression performance for practical implementation.

In practice, transmission distortion deriving from the noisy channel also needs to be considered in the VQ scheme, while the compression distortion of quantization is influenced by the transmission distortion. If the VQ scheme is robust over the noisy channel, the  transmission power will be reduced. Ref. [30] concludes that the quantization performance based on LDPC code is increased by channel noise. To solve the problem, ref. [30] introduces index allocation to ameliorate the channel robustness. The existing works demonstrate that index allocation is effective for resisting channel noise and for obtaining a better compression performance in the VQ scheme [26,28,31,32]. For instance, ref. [29] designed a block coding method based on linear mapping to implement index allocation by employing linear prediction, and [33] modifies the simulated annealing algorithm to appropriately allocate the indices. Hence, the index allocation is also supplemented to optimally design the P-LDPC code to reduce the noisy influence in this work. It should be noted that reference [30] only considers the lossy compression of the binary source over nosiy channels, while the continuous source is generalized in this work.

Reference [30] finds that the distortion caused by channel noise is reduced when the weight of the check-parity matrix is decreased. Based on this discovery, a search global degree allocation (S-GDA) algorithm is proposed to optimally design the protomatrix of the P-LDPC code with lower weight. The theoretical derivation of the S-GDA algorithm is demonstrated in this work. Furthermore, the P-LDPC codes are optimally searched with the properties of the channel noise resistance and variable source adaption. The validity of the code design based on the quantization structure is evaluated by the relation between the compression distortion and the code rate, and the code robustness is presented by the performance of the compression distortion and the signal-to-noise ratio (SNR). The availability of the proposed scheme is verified by the coding gains of the objective codes. In this work, it is illustrated that the channel distortion tends towards zero in the higher SNR since better compression performance can be obtained in the higher SNR and good robustness performance is present in the lower SNR.

Overall, the three main contributions can be summarized as follows:

(1) The lossy source coding based on linear block code is analyzed by the lattice VQ scheme, which is a new analytical perspective for the coding methodology.

(2) A general VQ scheme over the noisy channel is proposed with lower structural complexity, and it has universality for variable source distributions.

(3) A valid improvement of the coding performance is designed by the quantization structure, and its principle is equalizing the radiuses of the quantization subspace to reduce the quantization distortion.

The rest of this paper is organized as follows: The proposed VQ scheme and its methodology are introduced in Section 2. In Section 3, the P-LDPC codes are designed for different source distributions over the noisy channels. The simulation results and analyses are presented in Section 4. The conclusion is summarized in Section 5.

2. VQ Scheme over Noisy Channel

2.1. Generalized VQ Scheme

The proposed VQ scheme based on P-LDPC code over the noisy channel is shown in Figure 2, and it contains source coding, source decoding, and coding channel modules. The source ${\tilde{\mathbf{S}}}^N$ is a joint independent and identically distributed sequence of length N, and ${\breve{\mathbf{S}}}^N$ presents the reconstructed source.

In the source coding module, the source sequence $\{{\tilde{S}}^1,{\tilde{S}}^2,...,{\tilde{S}}^i\}\in {\tilde{\mathbf{S}}}^N$ is preprocessed by a scalar quantizer, and each variable ${\tilde{S}}^i$ is quantified as a binary $S^i$ of length L as follows:

$$S^i=q\left({\tilde{S}}^i\right),$$

(1)

where q represents the preprocessor of scalar quantization, $S^i$ represents a binary sequence or one bit since its length L can be taken any positive integer, the parameters satisfy $i\in [1,N]$ and $i,N,L\in {\mathbb{N}}^{*}$, and ${\mathbb{N}}^{*}$ is the set of positive integers.

Each local sampling point ${\tilde{S}}^i$ is quantified to achieve the highest bit rate. Since all the scalar quantizers are the same at each sampling point, this provides a feasible condition for constructing a regular VQ scheme based on P-LDPC code. It should be noted that the preprocessor performance will not influence the final design of the P-LDPC codes. The reason is that each sampling point needs to be compressed with the same expected distortion to change the VQ scheme from irregular to regular. In this case, the quantization complexity is simplified and the P-LDPC codes can be compatibly designed.

After that, the binary sequence $\{S^1,S^2,...,S^N\}\in {\mathit{S}}^N$ will pass through the source coding module based on the P-LDPC codes. The quantization codebook is determined by the check-parity matrix $\mathit{H}$ of the P-LDPC code, and the codeword ${\mathit{c}}^N$ should be closest to the center of quantization subspace to obtain the better compression performance. The source is quantified as the legal codeword ${\mathit{c}}^N$. The quantization subspace is determined by the codeword space, which is formed by the weight distribution of the P-LDPC codes.

In the transmission module, the index sequence ${\mathit{c}}^k$ of the codeword ${\mathit{c}}^N$ is extracted, and it is transmitted through the noisy channels as the receiving ${\widehat{\mathit{c}}}^k$. ${\mathit{c}}^k$ is obtained by the information bits of length k. Hence, the compression rate is defined as

$$R={\displaystyle \frac{1}{N}}{log}_2\left(2^k\right)={\displaystyle \frac{k}{N}},$$

(2)

where R is the compression rate, N is signified as the length of ${\mathit{S}}^N$, there are $2^k$ codewords encoded by ${log}_2\left(2^k\right)$ bits, and $k\in {\mathbb{N}}^{*}$.

In the source decoding module, the generator matrix $\mathit{G}$ corresponding to the check-parity matrix $\mathit{H}$ is multiplied with the demodulated ${\widehat{\mathit{c}}}^k$ to accomplish the linear decoding as follows: 

$${\widehat{\mathit{S}}}^N=\mathit{G}·{\widehat{\mathit{c}}}^k.$$

(3)

After that, the source ${\breve{S}}^i$ is reconstructed as

$${\breve{S}}^i=q^{-1}\left({\widehat{S}}^i\right),$$

(4)

where $q^{-1}$ is the post-processer which is a reverse of q, ${\widehat{S}}^i\in {\widehat{\mathbf{S}}}^N$, and ${\breve{S}}^i\in {\breve{\mathbf{S}}}^N$.

Overall, the distortion is mainly generated from two modules, the source coding and the channel noise modules. Hence, the total distortion of the VQ scheme is defined as

$$\begin{array}{c}\hfill D={\displaystyle \frac{\left|\right|{\breve{\mathbf{S}}}^N-{\tilde{\mathbf{S}}}^N{\left|\right|}^2}{N}}.\end{array}$$

(5)

2.2. Coding Methodology of VQ Scheme

Different from the design of conventional quantization, the VQ scheme based on linear block code needs to be appropriately designed with the codebook and the source space. Generally, the compression performance is determined by the codebook and the source subspace. In [20], the locally optimal quantizer is obtained by a training process, which is iteratively updated by the generalized Lloyd algorithm (GLA) based on the initial codebook and subspace.

On the contrary, the proposed VQ scheme does not need the training process. The corresponding codebook is determined by the given parity-check matrix of the linear block code, and the subspace is related to the coding property of this linear block code. Hence, the compression performance of the proposed VQ scheme mainly depends on the employed linear block code. Replacing the training process by designing the linear block code, the structure of the proposed VQ scheme has lower complexity. In addition, source decoding is implemented by the linear decoding method, which reduces the decoding complexity compared with the look-up table operation in the existing methods.

In the proposed VQ scheme, the preprocessing module is necessary since the source statistic distributions are diversiform. The preprocessing will average the source probability distribution and allocates the compression rate to the corresponding probability density. For example, the quantization performance is locally optimized with subspace partitioning to reasonably allocate the compression rate in [34]. In this way, there will be multi-levels in quantization to refine the signal region while the code rate is increased with the constriction of the signal region. Hence, the preprocessing module is useful for improving the compression efficiency.

In the codeword matching module, the subspace is partitioned according to the block code. The source is replaced by the closest codeword, and these source subsets are centered on the codeword form of the quantization subspace. Then, the distance between the subspace with the codeword is calculated as

$$\begin{array}{cc}\hfill \mathit{\xi }& ={\mathit{S}}^N+{\mathit{c}}^N\hfill \\ \hfill & ={(S^1+c^1,S^2+c^2,\cdots ,S^N+c^N)}^{\mathrm{T}}\hfill \\ \hfill & ={({\xi }^1,{\xi }^2,\cdots ,{\xi }^N)}^{\mathrm{T}},\hfill \end{array}$$

(6)

where “+” represents the modulo 2 addition, ${\xi }^i\in \mathit{\xi }$, $\mathit{\xi }$ is defined as the radius vector of the quantization subspace, and the codeword matching relation satisfies

$$\begin{array}{c}\hfill {\mathit{c}}^N={\mathit{S}}^N+\mathit{\xi }.\end{array}$$

(7)

This codeword matching relation indicates the proposed coding methodology as follows:

$$\begin{array}{cc}\hfill & \mathit{H}·\mathit{\xi }=\mathit{H}·{\mathit{S}}^N,\hfill \\ \hfill s.t.& \mathit{H}·{\mathit{c}}^N=\mathbf{0}.\hfill \end{array}$$

(8)

Then, the i-th quantization subspace can be expressed as

$$\begin{array}{c}\hfill {\mathit{S}}_{{\mathit{c}}_i}^N=\left\{{\mathit{S}}^N\right|{\mathit{S}}^N={\mathit{c}}_i^N+\mathit{\xi }\}.\end{array}$$

(9)

Assume that there are $2^k$ quantization subspaces in total, and each subspace contains $2^{N-k}$ source vectors. In this case, the number of radiuses is $2^{N-k}$. For each codeword, the set of radiuses is the same, which is determined by the weight distribution of the parity-check matrix. Specifically, a set of radiuses is signified as $\{{\mathit{\xi }}_1,{\mathit{\xi }}_2,\cdots ,{\mathit{\xi }}_{2^{N-k}}\}$.

The quantization subspace of lattice VQ is regular. Referring to the aforementioned methodology, it can be seen that the quantization based on the linear block code is lattice VQ. This structure is suitable for realizing high-dimensional quantization, and its regular subspace structure is appropriate for uniform quantization. Most of the lattice VQ can be obtained by linear calculations with lower coding complexity. For the lattice VQ, the generation matrix of the linear block code needs to be used as the system matrix, and the source decoding is implemented as

$$\begin{array}{c}\hfill {\widehat{\mathit{S}}}^N=\mathit{G}·{\widehat{\mathit{c}}}^N,\end{array}$$

(10)

where ${\widehat{\mathit{S}}}^N$ and ${\widehat{\mathit{c}}}^N$ represent the reconstructed source and the demodulated codeword, respectively, and the check-parity matrix $\mathit{H}$ can be partitioned as follows:

$$\begin{array}{c}\hfill \mathit{H}=\left[{\mathit{I}}_{M\times M}\left|{\mathit{B}}_{M\times k}\right.\right],\end{array}$$

(11)

where ${\mathit{I}}_{M\times M}$ is an identity matrix of the dimension $M\times M$, $\mathit{B}$ represents a partition matrix of dimension $M\times k$, and $N=M+k$. If the generation and the check-parity matrices are determined, the degrees of information bits can be allocated correspondingly.

In addition, the N-dimensional source is considered in the proposed VQ scheme. For example, the N-dimensional source follows the independent and identically distributed Gaussian distribution, and its rate–distortion function is

$$\begin{array}{c}\hfill \begin{array}{c}\hfill R\left(D\right)=\sum _{i=1}^N{\displaystyle \frac{1}{2}}log{\displaystyle \frac{{\sigma }_i^2}{D_i}},\end{array}\end{array}$$

(12)

where ${\sigma }_i^2$ presents the variance of the ith Gaussian source, $i\in [1,N]$, and the function log is replaced by ln to express the $L\left(D\right)$ as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill L\left(D\right)=\sum _{i=1}^N{\displaystyle \frac{1}{2}}ln{\displaystyle \frac{{\sigma }_i^2}{D_i}}+\lambda \sum _{i=1}^N{D}_i.\end{array}\end{array}$$

(13)

The partial derivative of $D_i$ is calculated as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\displaystyle \frac{\partial L}{\partial D_i}}=\lambda -{\displaystyle \frac{1}{2D_i}}=0.\end{array}\end{array}$$

(14)

This equation indicates that the minimum is a constant. That is, each dimension should be assigned with the same quantization accuracy. Specifically, if each dimension of the linear block code has the same coding ability, the VQ scheme will obtain the overall minimum average distortion. For example, ref. [30] proposes the GDA codes to balance the coding ability by averaging the degree distribution to reduce the distortion.

In summary, the minimum number of codewords is determined by the codeword length and the rate distortion function as follows:

$$\begin{array}{c}\hfill 2^{NR\left(D\right)}=\left({\displaystyle \frac{{\sigma }^2}{D}}\right).\end{array}$$

(15)

2.3. Robust Code Design

In the proposed VQ scheme, the overall distortion is composed of quantization distortion ($D_S$) and channel distortion ($D_C$) [35] as follows:  

$$\begin{array}{c}\hfill D=D_S+D_C,\end{array}$$

(16)

where D is calculated by the mean-squared error (MSE) distortion.

In the VQ scheme, the channel distortion is generated from the noise influence.  Reference [30] finds that the channel distortion will be increased when the generation matrix of the P-LDPC code has a larger weight. The reason is as follows:

$$\begin{array}{cc}\hfill {\overline{w}}_e& =e·\left[1+{\displaystyle \frac{1}{k}}{\displaystyle \sum _{j=1}^k}V\left(j\right)\right]\hfill \\ & ={\displaystyle \frac{e}{k}}V\left(\mathit{G}\right),\hfill \end{array}$$

(17)

where ${\overline{w}}_e$ represents the average distortion generated by the e-bits error of the binary indices, $V\left(j\right)$ is the degree of the j-th variable node, and $V\left(G\right)$ is the total weight of the generation matrix.

Hence, the P-LDPC codes with the lower weights should be considered. For the code design, the weight constraint is supplemented to search the objective code, and the S-GDA algorithm is proposed, shown in Algorithm 1. The parameters can be explained as follows:

$ϵ$: the threshold of quantization distortion reduction;

$V_j$: the degree of the j-th variable node ($j\in {[M+1,N]}^{*}$);

$C_i$: the degree of the i-th check node ($i\in {[1,M]}^{*}$);

d: the total degree of the variable nodes associated with the information bits in the protomatrix;

$V_{iy}$: the number of edges connecting the i-th check node and the y-th variable node ($y\in {[M+1,N]}^{*}$);

$C_{xj}$: the number of edges connecting the j-th variable node and the x-th check node ($x\in {[1,M]}^{*}$).

To suppress the channel noise, the quantization distortion reduction $ϵ$ is given as a small value in the S-GDA algorithm so that the distortion $D_S$ is decreased. Then, the S-GDA codes with the minimum weight will be obtained with the minimum $D_C$. The optimal S-GDA code has good robustness to the channel noise. Furthermore, the tradeoff between the quantization distortion ($D_S$) and the channel distortion ($D_C$) should be balanced.

Algorithm 1 S-GDA

Input:  M, N, d, $ϵ$; Output:  S-GDA codes 1: do 2:      for $j=M+1\to N$ do 3:            $V_j\leftarrow d/(N-M)$ 4:            if $j\ge N-d\%(N-M)$ then 5:                $V_j\leftarrow V_j+1$ 6:            else 7:                $V_j\leftarrow V_j$ 8:            end if 9:      end for      // Allocate the total degree to the variable nodes 10:     for $i=1\to M$ do 11:            $C_i\leftarrow (d+M)/M$ 12:            if $i\ge M-(d+M)\%M$ then 13:                $C_i\leftarrow C_i+1$ 14:            else 15:                 $C_i\leftarrow C_i$ 16:            end if 17:     end for    // Allocate the total degree to the check nodes 18:     for $j=M+1\to N$ do 19:           $C_{xj}\leftarrow V_j/M$ 20:           if $++x\ge M-V_j\%M$ then 21:                $C_{xj}\leftarrow C_{xj}+1$     // Until the degree of the j-th variable node is filled 22:         else 23:              $C_{xj}\leftarrow C_{xj}$ 24:         end if     // Traverse filling: the next filling position starts at adjacent position 25:     end for 26: while  $D_S\left(\mathrm{GDA}(d-1)\right)-D_S\left(\mathrm{GDA}\left(d\right)\right)>ϵ$ 27:  $\mathrm{S}-\mathrm{GDA}\leftarrow \mathrm{GDA}(d-1)$ 28: return objective code

Mathematically, the principle of Algorithm 1 is as follows:

$$\begin{array}{cc}\hfill & \underset{inputd}{\left[{\mathit{I}}_{M\times M}\left|{\mathit{B}}_{M\times k}\right.\right]}\hfill \\ \hfill & \to \left[\mathit{I}\left|{\mathit{B}}_{c1}^{⌊{\displaystyle \frac{d}{k}}⌋}{\mathit{B}}_{c2}^{⌊{\displaystyle \frac{d}{k}}⌋}\cdots {\mathit{B}}_{c(k-1)}^{⌊{\displaystyle \frac{d}{k}}⌋+1}{\mathit{B}}_{ck}^{⌊{\displaystyle \frac{d}{k}}⌋+1}\right.\right]\hfill \\ \hfill & \to \left[\mathit{I}\left|\begin{array}{c}{\mathit{B}}_{r1}^{⌊{\displaystyle \frac{d}{M}}⌋}\\ {\mathit{B}}_{r2}^{⌊{\displaystyle \frac{d}{M}}⌋}\\ ⋮\\ {\mathit{B}}_{r(M-1)}^{⌊{\displaystyle \frac{d}{M}}⌋+1}\\ {\mathit{B}}_{rM}^{⌊{\displaystyle \frac{d}{M}}⌋+1}\end{array}\right.\right],\hfill \\ \hfill & s.t.\min D_S\left(\mathrm{GDA}(d-1)\right)-D_S\left(\mathrm{GDA}\left(d\right)\right)>ϵ,\hfill \end{array}$$

(18)

where $⌊·⌋$ is the floor function, the code rate is defined as $M/N$, $⌊{\displaystyle \frac{d}{k}}⌋$ implies $V_j={∥{\mathit{B}}_{cj}∥}_1=⌊{\displaystyle \frac{d}{k}}⌋$, and $⌊{\displaystyle \frac{d}{M}}⌋$ indicates $C_i={∥{\mathit{B}}_{ri}∥}_1+1=⌊{\displaystyle \frac{d}{M}}⌋+1$. The objective code will be obtained when the quantization distortion reduction is less than the threshold $ϵ$; otherwise, the total degree d will be decreased to search the objective code.

3. Two System Cases: Code Design of VQ Scheme

3.1. Codes Design for Lossy Gaussian Source Compression System

In this paper, the MLC structure proposed in [16] is modified with the noisy channel as shown in Figure 3. First, the source is quantified based on the probability distribution intervals using non-uniform quantization. Then, a set of floating codebooks is transformed as the binary with the corresponding centroid. After that, the obtained binary indices are encoded by the set partition (SP) mapping method, which divides one-dimensional source space into $2^L$ quantization subspaces. Finally, the mapped indices are vectorized by the P-LDPC codes.

The S-GDA algorithm can be utilized in the MLC structure-based VQ scheme to search the objective P-LDPC codes assuming there are L levels in the MLC, and the corresponding P-LDPC codes can be generated with the minimum quantization distortion ($D_S$) at each level. For example, when the compression rate is 1.5, the MLC structure has three levels, and the code rates of each level are 0.15, 0.4, and 0.95, respectively. In this case, the objective codes of three levels will be constructed by the S-GDA algorithm, namely, the S-GDA_17×20$\left(d_1\right)$, S-GDA_3×5$\left(d_2\right)$, and S-GDA_2×40$\left(d_3\right)$.

Considering the bits at the first level is not important for improving the system distortion; therefore, it is possible to discard the bits at the first level. In this case, the code rate is increased to a higher level and it will generate less compression distortion than using the code given in [16].

For example, the code rates are divided into 0.1 and 0.4 at two levels when the total code rate is 0.5. It should be noted that the overall distortion mainly depends on the higher code rate. Hence, the bits are discarded and replaced with random bits at the first level, and the code rate is updated as 0.5 at the second level. The objective code is obtained by the S-GDA algorithm to be

$$\begin{array}{c}\hfill \mathrm{S}-{\mathrm{GDA}}_{3\times 6}\left(7\right)=\left[\begin{array}{ccccccc}1& 0& 0& |& 0& 1& 1\\ 0& 1& 0& |& 1& 0& 1\\ 0& 0& 1& |& 1& 1& 1\end{array}\right],\end{array}$$

(19)

where the dimension of the protomatrix is $3\times 6$, and the total degree is 7, indicating that there are seven linking edges between the variable nodes and the information bits expect the identity matrix.

Assuming there are three levels in the MLC structure, since the dimensions are the same at all levels, the ratio of the code rate is equal to the ratio of the index length at each level as follows:

$$\begin{array}{c}\hfill R_1:R_2:R_3.\end{array}$$

(20)

Referring to Equation (17), the ratio of distortion bits caused by each level is calculated as

$$\begin{array}{cc}\hfill & {\overline{w}}_{e_1}:{\overline{w}}_{e_2}:{\overline{w}}_{e_3}\hfill \\ \hfill & ={\displaystyle \frac{e_1}{k_1}}V\left({\mathbf{G}}_1\right):{\displaystyle \frac{e_2}{k_2}}V\left({\mathit{G}}_2\right):{\displaystyle \frac{e_3}{k_3}}V\left({\mathit{G}}_3\right)\hfill \\ & ={\displaystyle \frac{\zeta ·R_1}{k_1}}V\left({\mathit{G}}_1\right):{\displaystyle \frac{\zeta ·R_2}{k_2}}V\left({\mathit{G}}_2\right):{\displaystyle \frac{\zeta ·R_3}{k_3}}V\left({\mathit{G}}_3\right)\hfill \\ & =V\left({\mathit{G}}_1\right):V\left({\mathit{G}}_2\right):V\left({\mathit{G}}_3\right)\hfill \\ & =f_{{\mathit{G}}_1}^N:f_{{\mathit{G}}_2}^N:f_{{\mathit{G}}_3}^N,\hfill \end{array}$$

(21)

where the bit error ratio (BER) is signified as $\zeta$, the subscripts 1, 2, and 3 indicate three levels, respectively, $f_{\mathit{G}}^N={\displaystyle \frac{V\left(\mathit{G}\right)}{N}}$ is defined as the average row density of the generator matrix, and the channel distortion is expressed as the average Hamming distortion of a single dimension, i.e., $D_C=\zeta ·f_{\mathit{G}}^N$.

In this MLC structure-based VQ scheme, the coding performance cannot achieve the optimum for all levels. However, it is found that the distortion is mainly generated from the higher levels. Hence, the S-GDA algorithm is only implemented to search the objective codes at the higher levels, and the structural complexity of the system is greatly reduced.

3.2. Codes Design for Lossy General Source Compression System

The continuous source has uneven distribution, which is more difficult to compress than the binary source. The Gaussian and Laplacian sources are considered as the general source in this section, and their compression performance can be improved by increasing the dimension. When the dimension is large enough, the statistical properties of the general source will be uniformly distributed in the N dimensions.

The joint probability density of the independent and identically distributed general source is defined as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill f_S\left(\mathit{S}\right)=\prod _{i=1}^N{f}_S\left(S_i\right),\end{array}\end{array}$$

(22)

where the length of source is N. Then, the log product is calculated as

$$\begin{array}{cc}\hfill -{\displaystyle \frac{1}{k}}{\displaystyle \sum _{i=1}^N}log{f}_S\left(S_i\right)& =-{\displaystyle \frac{1}{N}}log{f}_S\left(\mathbf{S}\right)\hfill \\ \hfill & \underset{N\to \infty }{\to }-E[log{f}_S\left(S\right)]\hfill \\ \hfill & =h\left(S\right),\hfill \end{array}$$

(23)

where $h\left(S\right)$ is the differential entropy of the independent variables, and the joint probability density distribution can be expressed as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill f_S\left(\mathit{S}\right)\approx 2^{-N·h\left(S\right)}.\end{array}\end{array}$$

(24)

It can be seen that the independent (memoryless) source obeys an approximately uniform distribution in a high dimension, and the lattice VQ will be suitable when N is taken to be a large value. For example, the Gaussian source follows the distribution of $N(0,{\sigma }^2)$, and the shape of the quantization subspace of the equivalently uniform distribution is regarded as an N-dimensional hypersphere as follows:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\displaystyle \frac{1}{N}}\sum _{i=1}^N{S}_i^2\approx {\sigma }^2.\end{array}\end{array}$$

(25)

Referring to [18], a lossy source coding system is proposed to compress the general source, as shown in Figure 4. In [18], the scalar quantization of the preprocessing is accomplished through the utilization of neural networks (NNs), and the restrict minimum distortion (RMD) is an error correction algorithm based on the BP algorithm. In this work, the proposed system is modified with the noisy channel and supplements the modulation modules. Then, the objective code is designed by the S-GDA algorithm as follows:

$$\begin{array}{c}\hfill \mathrm{S}-{\mathrm{GDA}}_{4\times 8}\left(16\right)=\left[\begin{array}{ccccccccc}1& 0& 0& 0& |& 1& 1& 1& 1\\ 0& 1& 0& 0& |& 1& 1& 1& 1\\ 0& 0& 1& 0& |& 1& 1& 1& 1\\ 0& 0& 0& 1& |& 1& 1& 1& 1\end{array}\right],\end{array}$$

(26)

where the code dimension is $M=4$ and $N=8$.

Compared with the existing codes, the AR3A code needs to be punctured to enlarge the code rate to 0.5, while the S-GDA code can achieve the 0.5 rate without puncturing. Furthermore, according to quantization theory, it is found that the compression performance of the AR3A and the AR4JA codes [36,37,38] cannot be improved with the larger dimensions. However, the S-GDA codes are designed based on the quantization theory and their dimensions can be expanded arbitrarily. Hence, the S-GDA codes will have less compression distortion than the punctured codes with lower design complexity.

4. Simulation Results and Analyses

4.1. Performance Analyses of Code Design for Gaussian Source

In this section, the Gaussian source compression is analyzed based on the MLC structure-based VQ scheme. The protomatrix of the P-LDPC code is lifted to 600. The iteration number is 1500. The channel is the additive white Gaussian noise (AWGN) with binary phase-shift keying (BPSK) modulation. The system distortion D is composed by the quantization distortion $D_S$ and the channel distortion $D_C$. The channel environment is defined so that the system distortion approximates the quantization distortion, i.e., $D\approx D_S$, when the BER is less than ${10}^{-3}$ in the AWGN channel. In this case, the compression distortion will be stabilized without the channel noise impact.

Next, the S-GDA algorithm is used to optimally designed three objective codes at rates of 0.5, 1, and 1.5, respectively, as shown in Figure 5, Figure 6 and Figure 7. The searching process of the S-GDA algorithm is detailed in three steps. First, the number of layers is allocated the given code rate. Second, the GDA code family is obtained according to the rate allocation number. Third, the objective S-GDA codes are determined by selecting the appropriate GDA codes in different layers. Specifically, the selection criterion satisfies the threshold of quantization distortion reduction as

$$D_S\left(\mathrm{GDA}(d-1)\right)-D_S\left(\mathrm{GDA}\left(d\right)\right)>ϵ,$$

(27)

where $ϵ$ is set as a small value according to the code rate and the range of distortion reduction. To balance the channel and the quantization distortions, $ϵ$ is usually taken as suboptimal at the lower layer to achieve the robustness with a lower SNR.

In Figure 5, the compression rate is 0.5, which is divided into 0.1 and 0.4, and the two objective codes are obtained by the S-GDA algorithm as S-GDA_9×10$\left(d_1\right)$ and S-GDA_3×5$\left(d_2\right)$ at the $L_1$ and $L_2$ layers, respectively. The initial degrees are taken as $d_1=12$ at the $L_1$ layer and $d_2=8$ at the $L_2$ layer. The distortion is decreased when the degree is given as $d_1=7,8$ at $L_1$, as shown in Figure 5a. In this way, the code GDA_9×10$\left(7\right)$ with the suboptimal value $d_1=7$ is selected as the objective code S-GDA_9×10$\left(7\right)$ at the $L_1$ layer. At the $L_2$ layer, the optimal GDA_3×5$\left(14\right)$ with the minimum distortion is obtained as the objective code S-GDA_3×5$\left(14\right)$, as shown in Figure 5b. Overall, the objective S-GDA codes consist of S-GDA_9×10$\left(7\right)$ and S-GDA_3×5$\left(14\right)$.

In Figure 6, the compression rate is 1, which is divided into 0.05, 0.2, and 0.75, and three codes need to be obtained using the S-GDA algorithm, S-GDA_19×20$\left(d_1\right)$, S-GDA_4×5$\left(d_2\right)$, and S-GDA_2×8$\left(d_3\right)$ at $L_1$, $L_2$, and $L_3$, respectively. The initial degrees are taken as $d_1=21$ at the $L_1$ layer, $d_2=6$ at the $L_2$ layer, and $d_3=10$ at the $L_3$ layer. Finally, the objective codes are obtained as S-GDA_19×20$\left(8\right)$, S-GDA_4×5$\left(7\right)$, and S-GDA_2×8$\left(25\right)$ at the three layers, respectively.

In Figure 7, the compression rate is 1.5, which is divided into 0.15, 0.4, and 0.95, and the three objective codes are S-GDA_17×20$\left(d_1\right)$, S-GDA_3×5$\left(d_2\right)$, and S-GDA_2×40$\left(d_3\right)$ at $L_1$, $L_2$, and $L_3$, respectively. The initial degrees are taken as $d_1=51$ at the $L_1$ layer, $d_2=6$ at the $L_2$ layer, and $d_3=40$ at the $L_3$ layer. Finally, the objective codes are obtained as S-GDA_17×20$\left(38\right)$, S-GDA_3×5$\left(6\right)$, and S-GDA_2×40$\left(40\right)$ at the three layers, respectively.

Furthermore, the distortion–rate performance based on different P-LDPC codes is shown in Figure 8. The AR3A and AR4JA codes are benchmark codes [37], and the MLC code is proposed in [16] for Gaussian source compression. It is clear that the S-GDA codes present obvious coding gains and they approach the distortion–rate limit compared to other codes.

In Figure 9, the distortion sensitivity of each layer under the channel noise is shown by the distortion–SNR performance at three compression rates, 0.5, 1, and 1.5. The variance of the Gaussian source is given as ${\sigma }^2=1$. The simulation shows that the distortion is increased at the higher layer in the lower SNR region, which indicates that the higher layer is more sensitive to the channel noise and the lower layer is more robust. Compared with the multi-level scheme [16], the distortion can be reduced by the proposed VQ scheme.

4.2. Performance Analyses of Code Design for General Source

In this section, the general source compression is analyzed based on the NN structure-based VQ scheme, mainly including the Gaussian (${\sigma }^2=1$) and Laplacian ($\lambda =\sqrt{2}$) sources. The P-LDPC codes are lifted to 5, 20, and 40 times for the analyses, respectively. The simulation scale is set to $1000\times 1000\times N$. The channel environment is the AWGN with BPSK modulation. The distortion D is composed by the quantization distortion $D_S$ and the channel distortion $D_C$.

First, the Gaussian source is compressed as shown in Figure 10. The AR3A codes with the determined code rates are introduced to be the benchmarks of distortion comparison. The distortion will be increased with the decreased lifting numbers (signified as copy times) of the P-LDPC codes. This satisfies the VQ principle and the characteristics of the lattice VQ for high-dimensional compression. The codes with the lifting number of 40 have fewer distortions. Furthermore, the objective S-GDA codes are obtained at different code rates, namely, S-GDA_3×5$\left(12\right)$, S-GDA_3×6$\left(15\right)$, S-GDA_3×7$\left(16\right)$, and S-GDA_3×9$\left(19\right)$, as shown in Figure 10a, b, c and d, respectively.

In Figure 11, the energy distribution of nodes is compared with the AR3A and the S-GDA_3×5$\left(15\right)$ codes at different lifting numbers, respectively. When the lifting number is 5, the energy distributions of two codes are similar. On the contrary, the code S-GDA_3×5$\left(15\right)$ has more uniform energy distribution than the AR3A when the lifting number is enlarged to 40. This indicates that the code S-GDA_3×5$\left(15\right)$ can uniformly smooth the distortion distributions and reduces the overall distortion in the VQ scheme.

Furthermore, the distortion–rate performance is compared with the AR3A and the S-GDA codes for the Gaussian source compression, as shown in Figure 12. It is obvious that the S-GDA codes present coding gains in any conditions, including the different lifting numbers and code rates.

Table 1. Distortion comparisons between the LTS and S-GDA codes for Gaussian source compression over the AWGN channel with BPSK modulation when SNR = 7 dB and BER = ${10}^{-3}$.

R	0.4	0.5	0.57	0.66
Codes
LTS codes [17]	0.729	0.677	0.603	0.507
S-GDA codes	0.671↓	0.600↓	0.553↓	0.495↓

compares the lattice topological splicing (LTS) codes [17] and the S-GDA codes. It should be noted that the LTS codes are designed for Gaussian source compression, and the S-GDA codes are designed for general source compression. The LTS codes are obtained by the differential evolution (DE) algorithm based on the MIT strategy, which is locally optimal and has a higher coding complexity. Compared with the DE algorithm, the S-GDA algorithm has lower complexity. In addition, since the LTS code is proposed for a Gaussian source over the noise-free environment, the S-GDA codes present the better compression performance and system compatibility.

Second, the Laplacian source is compressed as shown in Figure 13. As in Figure 10, the AR3A code family is introduced as the benchmark. The objective S-GDA codes are obtained at different code rates, namely, S-GDA_3×5$\left(12\right)$, S-GDA_3×7$\left(14\right)$, and S-GDA_3×9$\left(14\right)$, as shown in Figure 13a, b and c, respectively. Figure 14 shows the distortion–rate performance based on the Laplacian source compression. It can be seen that the S-GDA codes have better performance.

The distortion–SNR performances based on the Gaussian and Laplacian source compressions are shown in Figure 15 and Figure 16, respectively. The S-GDA codes are tested at three code rates, namely, 0.4, 0.57, and 0.66. The S-GDA codes have less distortion in the lower-SNR region. Hence, the S-GDA codes have excellent source compression performance and channel noise robustness.

Overall, the limitation of the proposed study is that the source and channel are only formulated as some statistical models, since the compression bound should be presented based on the statistical characteristics. Some complicated signals (including the image, video, or radio) and practical environments (including non-stationary and fading environments) will be tested by this VQ scheme in the future.

5. Conclusions

In this paper, a general VQ scheme over the noisy channel is proposed based on block code for lossy source compression. The compression performance is improved by matching the codebook and quantization structure well. This is a new optimization perspective for codebook design. It is proved that the coding methodology of the duality principle can be validated with lower source distortion and channel noise. The optimal codebook outperforms the existing codes to approach the rate–distortion limit, and it enhances robustness against channel noise. The existing MIT method only focuses on formulating mutual information propagation to improve the compression performance, without considering the quantization structure. In contrast, the proposed scheme constructs a well-matching relation between the codebook and the quantization structure, and it integrates the MIT method to optimize the source code. Next, the proposed VQ scheme will be applied with complicated source models and a non-stationary channel environment.