A Low-Bit-Rate Image Semantic Communication System Based on Semantic Graph

Abstract

In the progress of research in the field of semantic communication, most efforts have been focused on optimizing the signal-to-noise ratio (SNR), while the design and optimization of the bit rate required for transmission have been relatively neglected. To address this issue, this study introduces an innovative low-bit-rate image semantic communication system model, which aims to reconstruct images through the exchange of semantic information rather than traditional symbol transmission. This model employs an image feature extraction and optimization reconstruction framework, achieving visually satisfactory and semantically consistent reconstruction performance at extremely low bit rates (below 0.03 bits per pixel (bpp)). Unlike previous methods that used pixel accuracy as the standard for distortion measurement, this research introduces multiple perceptual metrics to train and evaluate the proposed image semantic encoding model, aligning more closely with the fundamental purpose of semantic communication. Experimental results demonstrate that, compared to WebP, JPEG, and deep learning-based image codecs, our model not only provides a more visually pleasing reconstruction effect but also significantly reduces the required bit rate while maintaining semantic consistency.

1. Introduction

With the rapid advancement of high-resolution visual applications such as streaming media, virtual reality (VR), and smart surveillance, the efficient compression and transmission of images and videos generated by these applications have become a pressing issue. Current lossy codecs, such as JPEG [1], JPEG2000 [2], WebP [3], and BPG [4] based on HEVC, achieve data compression primarily by removing redundant information in images and employing certain approximate representations. Unlike lossless image codecs, lossy codecs may introduce some loss of information during the restoration of images. However, through careful design, significant data compression can be achieved without noticeably impacting visual perception.

At extremely low bit rates, however, codecs designed based on traditional algorithms can significantly degrade image quality, possibly introducing block effects and artifacts. This issue arises from the compression algorithms’ block division and internal compression when processing images, which may lead to noticeable block boundaries and color distortions. This is because traditional evaluation standards tend to favor the preservation of pixel-level local structures rather than semantic information. Image compression methods based on deep learning could also produce blurry, checkerboard-like artifacts, and when relying on MS-SSIM and MSE metrics, the texture reconstruction performance is generally poor [5].

Semantic communication, which emphasizes the fidelity of semantics rather than minimizing bit or symbol error rates, is gaining increasing attention and has become an important research direction in the field of communication [6,7,8]. Semantic communication has shown broad application prospects in various industries, including brain computer interfaces [9], virtual reality [10], augmented reality [11], and mixed reality [12]. Although existing research on semantic communication covers text, voice, image, and even video information, the bit rate usage during the semantic transmission process has rarely been addressed.

Inspired by these considerations, this study introduces a low-bit-rate semantic communication system for image transmission, aimed at capturing the semantic information of images through a feature extraction network and a semantic segmentation network. The system transmits compressed semantic information during the communication process to reduce the bit rate required for transmission, and reconstructs the image on the semantic level at the receiver end using prior knowledge.

The principal contributions of our work are as follows:

-

We propose a deep learning based feature extraction compression network, utilizing the ResNet [13] neural network model combined with various attention mechanisms, to achieve efficient semantic feature extraction. Through the network’s feature extraction capabilities, we manage to capture partial semantic information of images at extremely low bit rates.

-

We employ a conditional diffusion model for image reconstruction. Unlike traditional diffusion models, the conditional diffusion model incorporates conditional information, making the generation process more controllable. This implies that the model’s generation is influenced not only by random noise but also by given conditional information. We use semantic information as this conditional input, thus aligning the generated images more closely with our requirements.

-

We optimize and evaluate the semantic exchange efficiency of the proposed semantic encoding method and combine different perceptual metrics. The reconstruction results are quantitatively assessed using the Fréchet Inception Distance (FID) [14], mean Intersection over Union (mIoU) [15], and classification accuracy of the reconstructed images, aligning with human perception.

2. Related Work

2.1. Semantic Communication

Since Claude Shannon introduced the theory of information, he and Warren Weaver have further developed and refined their theory and model. In this process, they recognized the importance of semantics in communication and proposed three levels of communication: syntactic communication, semantic communication, and pragmatic communication [16]. Semantic communication, focused on a task-oriented “understand first, transmit later” communication strategy, involves selective feature extraction, compression, and the transmission of the original signal, followed by communication using semantic-level information. Semantic communication systems for text transmission have been introduced in references [17,18], achieving intelligent communication between humans and machines, as well as between machines. The theory of the lossless compression of semantic data, which asserts that data can be significantly compressed at the semantic level, was explored in reference [19]. Building on this, we propose an image semantic encoding method for multimedia semantic communication, utilizing prior information such as image distribution models.

2.2. Image Compression

Early image compression techniques primarily utilized entropy coding directly to reduce the redundancy in image encoding, such as Huffman coding [20], arithmetic coding [21], and context-adaptive binary arithmetic coding [22]. In the late 1960s, compression methods based on image transformation were introduced; they compress images by transforming them from the spatial domain to the frequency domain and encoding them in the frequency domain. The transformation techniques used in transform coding mainly include the Fourier transform [23], Hadamard transform [24], and Discrete Cosine Transform (DCT) [25]. In addition to eliminating data redundancy through entropy coding and transformation techniques, prediction and quantization techniques were also proposed to further reduce spatial redundancy and psychovisual redundancy in images.

In recent years, image compression techniques based on deep learning have been extensively studied and applied, including image compression methods based on autoencoders, convolutional neural networks, and Generative Adversarial Networks (GANs). Toderici [26] pioneered an end-to-end optimized image compression method that reconstructs images using Recurrent Neural Networks (RNNs). Ballé [27] proposed an image compression framework based on the Variational Autoencoder (VAE) structure. Rippel et al. [28] were the first to introduce adversarial loss functions in an end-to-end framework to enhance the visual quality of images. To further reduce the number of bits required to store images, researchers have explored various effective probability models for entropy coding, including hierarchical priors and autoregressive models with context.

2.3. Image Generation

In the field of image generation, Generative Adversarial Networks (GANs) and diffusion models have made rapid progress, focusing on learning the distribution of natural images rather than pursuing precision in local reconstruction.

GANs, as pioneers in image generation technology, were introduced by Ian Goodfellow and colleagues in 2014 [29]. The core concept of GANs is to generate new data instances through adversarial training between two networks—a generator network (G) and a discriminator network (D). This approach has been demonstrated by Agustsson et al. [30] and D He et al. [31], who have employed advanced generative networks to provide exceptional reconstruction quality at extremely low bit rates, showcasing the potential of adversarial learning.

As an inheritor of the diffusion model, the conditional diffusion model [32] is an extension that introduces conditional information for more precise control of generation. Compared to the potential instability during the training process of GANs, such as pattern collapse and non-convergence, the training process of conditional diffusion models exhibits greater stability. After 2022, GAN has gradually been replaced by the conditional diffusion model. Conditional diffusion models can constrain multimodal information and promote low-bit-rate compression through contrastive language image pre-training encoding [33], as well as denoising diffusion implicit models, latent diffusion, and stable diffusion [34] reconstruction models. Compared to GANs, conditional diffusion models can reduce the randomness of generated images to a certain extent through reasonable parameter adjustments and conditional control, thereby improving the stability of generated images.

3. Low-Bit-Rate Image Semantic Communication System

This section introduces the overall architecture of the low-bit-rate image semantic communication system proposed in this paper, followed by a detailed discussion of the designed feature extraction module, multi-attention module, and conditional diffusion image generation module.

3.1. Overall Architecture

The system model framework for semantic communication at extremely low bit rates proposed in this document is composed of a multiscale feature extraction module, an image reconstruction module, a channel, and a shared knowledge background between the sender and receiver, as illustrated in Figure 1. The multiscale feature extraction module comprises two parts: a multiscale feature extraction network and a semantic segmentation network. The multiscale feature extraction network is responsible for extracting feature information from images, while the semantic segmentation network is tasked with extracting the semantic distribution of images. The semantic reconstruction module includes a semantic generation network, which reconstructs images using a conditional diffusion model based on the received image feature information and semantic distribution as conditions.

Under the lowest bpp compression configuration proposed in this article, image feature information is compressed at a very high compression rate (<0.01 bpp), and the semantic distribution map is down-sampled to a resolution of 128 × 128 and transmitted as a grayscale image, achieving a satisfactory compression rate (<0.02 bpp). These two pieces of information are transmitted to the receiver end through low-bit-rate lossless encoding.

3.2. Multiscale Feature Extraction Module

As shown in Figure 2, the multiscale feature extraction network we propose follows the ResNet50 structure and is composed of four module stages. The four stages are composed of multiscale residual blocks and attention blocks.

The details of the multiscale residual module are depicted in Figure 3, where the convolutional layers utilize 3 × 3 and 5 × 5 sized depthwise separable convolutions [35]. Compared to conventional convolutional methods, this approach significantly reduces computational complexity and the number of parameters, while still maintaining effective feature extraction capability from the input.

In the multiscale residual module, features are initially processed through 3 × 3 and 5 × 5 sized depthwise separable convolution layers for both $X$ and $Y$ inputs, utilizing the SiLU activation function to obtain features at different scales [36]. Subsequently, the extracted features are concatenated and fed into 5 × 5 and 3 × 3 sized depthwise separable convolution layers in a cross manner to further expand the receptive field and enhance information extraction. The concatenated features are then passed through a 1 × 1 sized conventional convolution layer to adjust dimensions, before being combined with the original residual features to produce the final output. Specifically, the multiscale residual module can be expressed as follows:

$$X_1=\mathsf{\sigma }\left(f_{3\times 3}^{\mathrm{d}1}\right(\mathrm{w}\left)\right),$$

(1)

$$Y_1=\mathsf{\sigma }\left(f_{5\times 5}^{\mathrm{d}1}\right(\mathrm{w}\left)\right),$$

(2)

$$X_2=\mathsf{\sigma }\left(f_{3\times 3}^{\mathrm{d}2}\right([X_1,Y_1]\left)\right),$$

(3)

$$Y_2=\mathsf{\sigma }\left(f_{5\times 5}^{\mathrm{d}2}\right([Y_1,X_1]\left)\right),$$

(4)

$$w^{\prime }=f_{1\times 1}^{\mathrm{n}3}\left(\right[X_2,Y_2\left]\right))+\mathrm{w}$$

(5)

where $f$ represents the weight of the network, $d$ in the superscript represents the deep separable convolution, n represents the ordinary convolution, and the number represents the number of layers. The subscript is the convolution kernel size used in the convolution layer. σ represents the SiLU activation function, and $\left[X_1,Y_1\right],\left[Y_1,X_1\right],\left[X_2,Y_2\right]$ represents the fusion operation, which corresponds to C in Figure 3. The specific operation is the addition on the convolution channel.

3.3. Hybrid Attention Block

Image features exhibit significant spatial locality, yet current feature extraction networks do not fully leverage the potential of attention mechanisms. The common practice of employing a single type of attention mechanism often leads to inaccurate feature extraction, thereby affecting the performance of subsequent tasks. To address this issue, this study proposes a hybrid attention module that integrates multiple types of attention mechanisms. This module applies different attention mechanisms to various component groups, fully utilizing the characteristics of each attention mechanism to significantly enhance the effectiveness of image feature extraction.

First, for the first to second module stages, they are characterized by more spatial information and fewer channels, so this paper uses SA (self-attention) [37] with jump connection to focus on their characteristics. The formula is as follows:

$$q\left(x\right)=W_{q}x,k\left(x\right)=W_{k}x,v\left(x\right)=W_{v}x,$$

(6)

$$\mathsf{\Gamma }\left(i,j\right)=\frac{{q\left(x_i\right)}^{\mathrm{T}}k\left(x_j\right)}{\parallel q\left(x_i\right)\parallel \parallel k\left(x_j\right)\parallel },$$

(7)

$$y=x+W_j\sum _j{softmax}_j\left(\alpha \mathsf{\Gamma }\left(i,j\right)\right)·v\left(x_j\right)$$

(8)

where $x$ is the input of the attention block and y is the output. $W_q{,W}_k{,W}_v{,W}_j\in {\mathbb{R}}^{C\times C}$ represents 1 × 1 in the attention block. $i$ and $j$ are spatial indexes ranging from 1 to H × W.

Secondly, for the third to fourth module stages, as the feature map size decreases and the number of channels increases, the importance of each channel of the feature map is enhanced. Therefore, this study adopts the CA (coordinate attention) [38] mechanism to better focus on spatial position information and channel information, thereby improving the efficiency and accuracy of feature extraction. Specifically, the coordinate attention mechanism is implemented as follows:

$$g^h=\sigma \left(F_h\left(f^h\right)\right),$$

(9)

$$g^w=\sigma \left(F_w\left(f^w\right)\right),$$

(10)

$$y(i,j)=x(i,j)\times g^h\left(i\right)\times g^w\left(j\right)$$

(11)

where $x$ is the input of the attention block and $y$ is the output. $g^h\in {\mathbb{R}}^{C\times H\times 1},g^w\in {\mathbb{R}}^{\mathrm{C}\times 1\times \mathrm{W}} \mathrm{i}\mathrm{s} \mathrm{t}\mathrm{h}\mathrm{e} \mathrm{a}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n} \mathrm{v}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r} \mathrm{o}\mathrm{b}\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{d} \mathrm{b}\mathrm{y} f^h\in {\mathbb{R}}^{\frac{C}{r}\times H\times 1},f^w\in {\mathbb{R}}^{\frac{C}{r}\times 1\times \mathrm{W}}$ using 1 × 1 convolution to increase dimensionality and combining it with sigmoid activation function, $i$ and $j$ are spatial indexes ranging from 1 to H × W.

3.4. Quantitative Transmission

In the feature extraction network, quantizing the output results can significantly reduce the number of bits required for transmission. However, during the model training process, optimization requires backpropagation and gradient descent, necessitating that the computation process be differentiable, while quantization is inherently non-differentiable. To address this issue, we refer to the quantization method described in reference [39] and simplify it following the ideas presented in reference [40]. Specifically, given $w=[w_1,w_2,w_3,\dots ]$, to quantize each element $w_i$ in $w$ to the corresponding symbol $a_j$, it is necessary to select the $a_j$ closest to $w_i$ from a set of candidate symbols $a=[a_1,a_2,a_3,\dots ]$. We employ the nearest neighbor assignment method to accomplish this calculation,

$$w_i^{\prime }=Q\left(w_i\right):={argmin}_j\parallel w_i-c_j\parallel$$

(12)

In backpropagation, to make the network differentiable, quantization can be replaced by:

$$w_i^{\prime }=\sum _{j=1}^L\frac{\mathrm{e}\mathrm{x}\mathrm{p}(-\sigma \parallel w_i-c_j\parallel )}{{\sum }_{l=1}^L\mathrm{e}\mathrm{x}\mathrm{p}(-\sigma \parallel w_i-c_l\parallel )}$$

(13)

This is equivalent to weighted averaging $a=[a_1,a_2,a_3,\dots ]$, with higher weights approaching $w_i$. This method is called soft quantization.

3.5. Conditional Diffusion Image Generation Module

Firstly, a brief introduction to the theory of conditional diffusion models is provided. The objective of conditional diffusion models is to generate samples that conform to the maximum likelihood estimate $m\left(p_0\right|x)$ under a given conditional data distribution $c\left(p_0\right|x)$. In the Conditional Denoising Diffusion Probabilistic Model (Conditional DDPM), two processes are defined: the forward process and the backward process. The forward process $n\left(p_{1:T}\right|p_0)$, also known as the diffusion process, refers to the gradual addition of Gaussian noise to the data until it becomes random noise. The formula for this process is as follows:

$$n\left(p_t|p_{t-1}\right)=\mathcal{N}(p_t;\sqrt{1-{\beta }_t}{p}_{t-1},{\beta }_t\mathit{I}),$$

(14)

With the notation $\alpha :=1-\beta$, there is a formula as follows:

$$n\left(p_t|p_0\right)=\mathcal{N}(p_t;\sqrt{{\alpha }_t}{p}_0,(1-{\alpha }_t\left)\mathit{I}\right),$$

(15)

Reverse process $m_{\theta }\left(p_{0:T}\right|x)$ is defined as a Markov chain that learns the Gaussian distribution $m\left(p_T\right)~\mathcal{N}(0,\mathit{I})$, and its formula is as follows:

$$m_{\theta }\left(p_{0:T}|x\right)=m\left(p_T\right){\prod }_{t=1}^T{m}_{\theta }\left(p_{t-1}|p_t,x\right),$$

(16)

$$m_{\theta }\left(p_{t-1}|p_t,x\right)=\mathcal{N}(p_{t-1};{\mu }_{\theta }\left(p_t,x,t\right),{\sum }_{\theta }\left(p_t,x,t\right))$$

(17)

The purpose of training a conditional diffusion model is to optimize its negative logarithmic likelihood variational upper bound.

Referring to previous research, this study made some modifications on the basis of the Semantic Diffusion Model (SDM) [41] network to meet the image reconstruction requirements of this paper. In SDM, the conditional denoising network is based on the U-Net structure and aims to estimate the noise components in the input noisy image. This network structure can independently process semantic information and noisy images, where noisy images are introduced into the denoising network during the encoder stage.

In order to more effectively utilize semantic information, this article has improved the structure for processing semantic information in SDM—SDDResblock. As shown in Figure 4, this study introduces semantic features obtained from multiscale feature extraction modules as additional conditions in the original SDDResblock structure. Through multi-layer spatially adaptive normalization operation, the semantic label map and semantic features are combined into the decoder of the denoising network.

4. Experiments and Discussion of Results

4.1. Setup

Experimental Environment: The experiments were conducted on a computer running Windows 11 64-bit OS, equipped with an Intel i7-12700H processor, NVIDIA GeForce RTX 3070 graphics card, and 8 GB RAM. The programming environment is based on the PyTorch framework, implemented in Python 3.10.

Dataset: The experiments were performed on the ADE20K dataset [42], which was released by MIT CSAIL Computer Vision Group in 2016. It includes 20,210 training images and 2000 validation images, covering 150 semantic categories such as people, vehicles, plants, and buildings. Each image provides pixel-level semantic annotation. Due to the inconsistent size of each image in the ADE20K dataset, we adjusted the size of all images to a resolution of 256 × 256 for subsequent experiments.

Evaluation Metrics: In evaluating the performance of different algorithms, we employed three crucial metrics: FID (Fréchet Inception Distance), mIoU (Mean Intersection over Union), and accuracy. These metrics measure the differences in authenticity, semantic consistency, and classification accuracy between synthetic and real images from different dimensions.

First, FID, as an indicator for assessing image authenticity, was achieved by calculating the Wasserstein-2 distance between synthetic and real images in the feature space. This metric quantified the perceptual realism of synthetic images, providing us with an intuitive standard for evaluating the performance of image generation algorithms.

Second, to assess the semantic accuracy of synthetic images, we introduced two metrics: mIoU and accuracy. Accuracy was evaluated by computing the average pixel classification accuracy across all categories, reflecting the ability to recognize the semantic content of synthetic images. Specifically, we re-segmented the synthetic images semantically and calculated the pixel classification accuracy between the segmentation results and the original semantic labels, thereby obtaining the overall accuracy. As another significant semantic evaluation metric, mIoU comprehensively measured the performance of synthetic images in semantic segmentation tasks by calculating the ratio of the intersection to the union between the true values and predicted values for each category, and then averaging the values for all categories.

Training method: We used an end-to-end approach to train the model as a whole, where the semantic segmentation network used a pre-trained frozen model that only provides segmentation results. Specifically, the learning rate of the model was 10-4 and the adam optimizer was used. We trained three different bpp size models for comparative experiments, namely, the smallest model with a semantic feature compression rate of 0.01 bpp and a segmentation result compression rate of 0.02 bpp; the medium model with a semantic feature compression rate of 0.03 bpp and a segmentation result compression rate of 0.02 bpp; and the large model with a semantic feature compression rate of 0.03 bpp and a segmentation result compression rate of 0.05 bpp.

Experimental Method: In the experiment, for traditional image compression algorithms such as JPEG and BPG, we controlled the compression ratio of the image to about 0.03 bpp by controlling the compression quality. For network models that use semantic distribution maps to generate images, such as SPADE and Pix2pixhd, we used the segmentation results of the VIT adapter model [43] as input, which showed excellent semantic segmentation performance on the ADE20K dataset. At a resolution of 256 × 256, the compression ratio of the segmentation result was about 0.05 bpp. The segmentation result was down-sampled once to a resolution of 128 × 128, and the compression ratio of the segmentation result was about 0.02 bpp. The model proposed in this article had two inputs. The first part was the semantic features extracted by a deep semantic extraction network, with a minimum compression rate of about 0.01 bpp. The second part was the segmentation result of the Vision Transformer Adapter model, which was added together to achieve a compression rate of about 0.03 bpp. Meanwhile, we tested the performance of the model under different BPPs by extracting semantic features of different sizes.

4.2. Result and Analysis

Through comparative experiments, we comprehensively compared the proposed model with other excellent image processing models at various bpps, and the results are listed in

Table 1. Comparison of different methods used on the Ade20k dataset. ↓ indicates the lower the better, ↑ indicates the higher the better. Bold denotes the best performance.

Method		ADE20K		bpp
	FID↓	mIoU ↑	ACC ↑
Jpg	163.32	8.81	48.39	0.03
Jpg2000	110.66	20.78	63.28	0.03
Webp	59.89	31.23	72.03	0.03
Bpg	41.91	37.57	75.62	0.03
Pix2pixhd [44]	82.31	26.83	72.23	0.02
SPADE [45]	34.27	36.01	76.87	0.02
SDM	30.18	47.08	80.25	0.02
Ours-S	28.89	48.93	81.42	0.03
Jpg	132.86	25.72	50.57	0.05
Jpg2000	97.43	31.37	68.64	0.05
Webp	54.72	35.42	75.36	0.05
Bpg	37.23	40.36	77.48	0.05
Pix2pixhd [44]	81.80	27.27	72.61	0.05
SPADE [45]	33.92	36.28	77.12	0.05
SDM	29.23	47.26	80.75	0.05
Ours-M	28.63	49.27	81.77	0.05
Jpg	83.45	41.15	78.35	0.08
Jpg2000	72.67	43.42	78.94	0.08
Webp	31.37	46.68	79.61	0.08
Bpg	29.64	47.34	81.44	0.08
Ours-L	27.95	49.76	82.04	0.08

. The results indicate that our research model outperforms the comparison model in all indicators compared to the second ranked model at each bpp. The lower FID value and higher accuracy score directly reflect the high visual similarity between the generated image and the real image. Meanwhile, the significant increase in mIoU further validates the consistency between the distribution of generated images and real images in our model, clearly demonstrating the effectiveness of using semantic conditions to guide the diffusion model during the generation process.

To further compare the popularity of images generated by our method versus other coding and decoding methods on the ADE20K dataset, we conducted a user study. In each set of experiments, participants were asked to choose between images generated by our method and those generated by other coding and decoding methods, based on which had better visual performance. According to the data presented in

Table 2. During the user study, participants made their selection between our method and other methods based on the visual quality of the generated images. The numerical values in the table represent the percentage of user preferences that supported our method. The results indicate that images generated by our method were generally more favored by users.

Method		Method
Ours vs. Jpg	82.1%	Ours vs. SPADE	69.6%
Ours vs. WebP	64.4%	Ours vs. SDM	53.3%

, users were more likely to support our results. Figure 5 showcases a comparison of partial results generated using various methods. It is evident that at low compression rates, JPEG images already begin to show blocky artifacts, and WebP images display smearing effects. Meanwhile, images generated by SPADE and SDM methods exhibit boundary distortions, indicating that these approaches are not fully capitalizing on semantic information.

4.3. Ablation Studies

To verify the effectiveness of key designs within our system, we conducted a series of ablation studies. These experiments targeted modules including the multiscale residual blocks and the hybrid attention blocks. Without the multiscale residual blocks, a traditional residual block composed of two convolutional layers was used; without the hybrid attention blocks, the model would directly output to subsequent layers. The quantitative results are summarized in

Table 3. Ablation studies about multiscale residual block and hybrid attention block. ↓ indicates the lower the better, ↑ indicates the higher the better.

MRB 1	HAB 2	FID↓	mIoU ↑	Acc ↑
✕	✕	29.12	48.29	81.07
✕	√	28.98	48.76	81.36
√	✕	28.94	48.47	81.22
√	√	28.89	48.93	81.42

¹ MRB represents multiscale residual block. ² HAB represents hybrid attention block.

. Upon analysis, we observed that both the multiscale residual blocks and the hybrid attention blocks positively contribute to enhancing the performance metrics of our model on the test set.

Under the condition of retaining the multiscale residual blocks and hybrid attention blocks, this study conducted detailed ablation experiments on the attention sub-modules within the hybrid attention blocks and investigated the sequence of these sub-modules. The experimental results are summarized in

Table 4. Sequential testing of attention sub-modules in hybrid attention blocks. ↓ indicates the lower the better, ↑ indicates the higher the better.

Method				FID ↓	mIoU ↑	Acc ↑
SA 1	SA	SA	SA	28.97	48.61	81.35
SA	SA	SA	CA 2	28.91	48.77	81.38
SA	SA	CA	CA	28.89	48.93	81.42
SA	CA	CA	CA	28.93	48.86	81.40
CA	CA	CA	CA	29.03	48.52	81.33

¹ SA represents self-attention with jump connection. ² CA represents coordinate attention.

. We observed that, compared to employing a single attention sub-module, the strategy of using hybrid attention sub-modules has a more significant positive impact on the overall performance of the model.

5. Conclusions and Future Work

In this work, we developed a low-bit-rate image semantic communication system, which reconstructs images by transmitting key information and semantic label maps, aiming at the consistency of semantic information transmission rather than the pixel-level consistency of the image. Compared to previous efforts, our system can receive information and reconstruct satisfactory images at an extremely low bit rate (below 0.03 bits per pixel (bpp)). This advantage not only significantly conserves bandwidth resources but also fundamentally prevents the occurrence of image degradation phenomena that are common in traditional coding and decoding methods under extremely low-bit-rate reconstruction. Exploring semantic communication systems for video content is a promising direction for future work.