Full-Reference Image Quality Assessment Based on Multi-Channel Visual Information Fusion

Abstract

This study focuses on improving the objective alignment of image quality assessment (IQA) algorithms with human visual perception. Existing methodologies, predominantly those based on the Laplacian of Gaussian (LoG) filter, often neglect the impact of color channels on human visual perception. Consequently, we propose a full-reference IQA method that integrates multi-channel visual information in color images. The methodology begins with converting red, green, blue (RGB) images into the luminance (L), red–green opponent color channel (M), blue–yellow opponent color channel (N) or LMN color space. Subsequently, the LoG filter is separately applied to the L, M, and N channels. The convoluted components are then fused to generate a contrast similarity map using the root-mean-square method, while the chromaticity similarity map is derived from the color channels. Finally, multi-channel LoG filtering, contrast, and chromaticity image features are connected. The standard deviation method is then used for sum pooling to create a full-reference IQA computational method. To validate the proposed method, distorted images from four widely used image databases were tested. The evaluation, based on four criteria, focused on the method’s prediction accuracy, computational complexity, and generalizability. The Pearson linear correlation coefficient (PLCC) values, recorded from the databases, ranged from 0.8822 (TID2013) to 0.9754 (LIVE). Similarly, the Spearman rank-order correlation coefficient (SROCC) values spanned from 0.8606 (TID2013) to 0.9798 (LIVE). In comparison to existing methods, the proposed IQA method exhibited superior visual correlation prediction accuracy, indicating its promising potential in the field of image quality assessment.

1. Introduction

Visual perception stands as one of the most crucial senses for humans. Images, serving as an efficient and practical carrier of visual information, are progressively taking on a paramount role in human life. However, the processes of image acquisition, transmission, and storage are subject to various distortions, such as Gaussian white noise, motion blur, and white balance errors [1]. Perceptual image quality assessment (IQA) can be dichotomized into subjective and objective evaluations. The former is constrained by environmental factors and exhibits suboptimal practicality. Conversely, objective IQA predominantly leverages algorithmic methods to predict image quality, with the goal of attaining a high degree of congruence between computed values and human visual perception, ultimately supplanting human vision in the task of image quality assessment. In recent years, as the depth of research continues to expand, the theoretical underpinning of objective IQA research has gradually matured. Generally, objective IQA can be categorized into three types based on the availability of reference images [2]: full reference (FR) [3], reduced reference (RR) [4], and no reference (NR) [5] IQA. The primary focus of this paper lies in the investigation of FR-IQA methods.

Current research on FR-IQA methods is relatively advanced, and depending on their computational approaches, can be broadly classified into three categories: mathematical methods, methods predicated on human vision system (HVS) characteristics, and learning-based methods [6]. Mathematical methods apply direct computations on image pixel gray values, but they overlook the varying contributions of different pixels to human vision. This oversight leads to discrepancies between computed results and subjective perception [7]. In recent years, researchers have started to incorporate HVS characteristics [8]—including brightness adaptation, contrast sensitivity, masking effects, and attention mechanisms—into IQA research. A seminal method in this domain is the structural similarity index (SSIM) proposed by Wang [9], which assesses image quality via the image’s brightness, contrast, and structural information using the RMS algorithm for contrast computation [10]. This method has a high computational efficiency, but the computational accuracy needs to be obviously improved. Several researchers have enhanced the SSIM algorithm. For instance, Li [11] proposed the three-component weighted SSIM (3-SSIM) method, which calculates feature similarities for the edge, texture, and smooth areas across the entire image. Although the calculation accuracy of SSIM has been further improved by 3-SSIM, the generalization cannot be ideal. Zhang [12] introduced contrast and visual saliency similarity (CVSS), which amalgamates image contrast with visual saliency. Liu [13] proposed the gradient similarity metric (GSM) index by masking the structural information. Although the aforementioned methods exhibit high computational accuracy, their generalizability need to be optimized. From an information theory standpoint, Sheikh [14] proposed visual information fidelity (VIF), which boasts a high computational efficiency but exhibits limited generalizability and calculation accuracy. The Laplacian of Gaussian (LoG) operator excels in simulating human visual perception characteristics and is frequently employed in the computer vision-related literature [15]. Zhang [16] proposed the feature similarity index (FSIMc) using LoG wavelets to calculate phase consistency. Zhang [17] presented the visual saliency-induced index (VSI) for perceptual image quality assessment, drawing upon the variable attention of the human eye to different scenes. VSI employs LoG filters to extract image frequency domain features and applies image gradient and chromaticity information to the method. Temel [18] proposed a perceptual similarity index (PerSIM) method using LoG on the luminance channel to mimic the visual system on the human retina. The three aforementioned methods all utilize the LoG filter in the process of establishing IQA methods, and their evaluation results exhibit commendable accuracy. However, these IQA methods still have some deficiencies in generalization. Learning-based methods [19] typically necessitate feature extraction and method training. However, due to limited datasets, a protracted method training process, and a dependence on data information, these methods bear a significant disadvantage in terms of generalizability. Gao [20] employed the VGGnet (ConvNets trained for image recognition) to construct IQA methods by the improved architecture of deep neural networks. Yao [21] used a convolutional neural network to generate a visual difference map mapping, and weighted the visual difference map according to the local spatial characteristics of the distorted image to evaluate the image quality. Chen [22] used several difference of Gaussian (DoG) frequency bands to extract image features, and construct IQA methods through random forest regression. Learning-based methods are also used in the field of NR-IQA, for example, Zhang et al. employed a neural network-based pairwise learning-to-rank algorithm named RankNet [23] to learn the IQA method by incorporating the uncertainty into the loss function. Kim [24] proposed a two-stage NR-IQA method based on CNN. This method uses the local quality score generated by the full-reference image quality assessment as the image patch score label. With the rapid development of IQA, IQA-related researches have been widely used in the fields of face recognition and biometric classification in recent years [25,26,27].

FR-IQA methods are designed to serve human vision, and an ideal computational method should mirror human visual perception in its representation of image texture and structural content. In general, the chromaticity channels of an image also contain structural information, but the way in which to extract this information needs to be explored. In the previous research, LoG filtering was utilized to extract structural information on the image brightness channel. Based on this method, more extensive structural information can also be obtained by LoG filtering processing on luminance channel and chromaticity channels, respectively. In an RGB image, the brightness channel and chromaticity channels are mixed. Hence, it is difficult to extract more extensive structural information. In order to solve this problem, the RGB image can be transformed to a uniform color space. LMN is the widely used uniform color space, and it has been utilized in some IQA research [1,17]. In this article, the image structure information is extracted on the three channels of L, M, and N by LoG filtering. Then, the three filtering components are fused by mathematical formula. This calculation process can be termed as multi-channel LoG filtering (MLG). Since alterations in contrast impact the HVS perception of image quality [28] and the contrast of the filtered image remains consistent, this paper introduces image contrast to offset the aforementioned limitations. The proposed method primarily targets color images, and this paper aims to conduct rudimentary chromaticity similarity calculations on the chromaticity channels [29]. After the completion of the calculations for the three components, the final quality score is computed using the standard deviation (SD)-based [28] pooling summation method. The main contribution of this paper is to propose an IQA method that combines the three image characteristics, e.g., multi-channel structural information by LoG filtering, and contrast and chroma information. Finally, a stable and efficient FR-IQA MVFC is established. The superiority of the proposed method will be substantiated from multiple perspectives in the succeeding sections.

2. Proposed Method

2.1. Construction of Multi-Channel LoG Filtering

During image quality assessment, image features that are consistent with human visual perception are generally introduced, such as image structure and texture. The literature [30] mentions that the difference of Gaussian (DoG) can be used to simulate the visual characteristics of cats. However, when constructing the DoG method, different standard deviations and scale differences are usually tuned to obtain the ideal feature differences, making the fusion of methods at different scales very complex. When adjusting the scale, the DoG operators can be approximated by the second-order derivative of Gaussian, which corresponds to the LoG operator [15]. This operator can simulate the neural mechanism of human vision. Therefore, to simplify the calculation, LoG is used here to calculate the image difference values, with:

$$L\widehat{o}G=\frac{1}{\sqrt{2\pi {\sigma }^2}}\frac{m^2+n^2-2{\sigma }^2}{{\sigma }^4}{\mathrm{e}}^{-(m^2+n^2)/(2{\sigma }^2)}$$

(1)

where $\sigma$ is the standard deviation of $L\widehat{o}G$, and m and n represent the positions of each pixel (for ease of writing, x is used here to represent the pixel position, i.e., x = (m, n)).

Before constructing a multi-channel LoG filter, the image needs to be preprocessed, that is, the RGB image is mapped to the opposite uniform color space. At present, there are many methods for color space conversion, such as CIELab [31], LMN [32], YIQ [33], and YUV [34]. To reduce the errors generated during color space conversion, the evaluation standard values of the color space in four public databases are calculated first, and the ideal uniform color space is determined through their values. The top-ranked evaluation standard values are highlighted in bold, as shown in

Table 1. Overall performance of color spaces in different databases.

Database		CIELab	YIQ	LMN	YUV
TID2013	SROCC	0.7805	0.7983	0.8123	0.7983
	KROCC	0.5828	0.6060	0.6192	0.6061
	PLCC	0.7899	0.8158	0.8241	0.8159
	RMSE	0.7603	0.7169	0.7023	0.7168
TID2008	SROCC	0.7735	0.8103	0.8138	0.8103
	KROCC	0.5717	0.6123	0.6172	0.6123
	PLCC	0.7713	0.8130	0.8161	0.8130
	RMSE	0.8541	0.7813	0.7755	0.7813
CSIQ	SROCC	0.8585	0.9075	0.9093	0.9076
	KROCC	0.6620	0.7269	0.7298	0.7269
	PLCC	0.8595	0.9069	0.9086	0.9069
	RMSE	0.1342	0.1106	0.1097	0.1106
LIVE	SROCC	0.9462	0.9656	0.9665	0.9656
	KROCC	0.8107	0.8586	0.8612	0.8586
	PLCC	0.9152	0.8772	0.8766	0.8772
	RMSE	12.6080	15.0241	15.0581	15.0244
Direct average	SROCC	0.8397	0.8704	0.8755	0.8705
	KROCC	0.6568	0.7010	0.7069	0.7010
	PLCC	0.8340	0.8532	0.8564	0.8533

.

From the data in Table 1, it can be seen that LMN has the best overall performance among the four uniform color spaces. Therefore, the RGB image is transformed into the LMN uniform color space in the proposed method. The transformation formulation is as shown in Equation (2):

$$\left[\begin{array}{l}L\\ M\\ N\end{array}\right]=\left[\begin{array}{l}0.06 0.63 0.27\\ 0.30 0.04 -0.35\\ 0.34 -0.6 0.17\end{array}\right]\left[\begin{array}{l}R\\ G\\ B\end{array}\right]$$

(2)

where L* represents luminance component; M* and N* represent chromaticity components.

Traditional calculation methods generally convolve the operator $L\widehat{o}G$ with the image luminance component, ignoring the importance of color components. In this paper, the operator $L\widehat{o}G$ is convolved with the three color components in the uniform color space, as implemented by Equation (3):

$$\begin{array}{l}LLo{G}_i(x)=L_i(x)\ast L\widehat{o}G(x)\\ MLo{G}_i(x)=M_i(x)\ast L\widehat{o}G(x)\\ NLo{G}_i(x)=N_i(x)\ast L\widehat{o}G(x)\end{array}$$

(3)

where i represents the image type (i.e., original image f₁ and distorted image f₂) and * represents convolution calculation.

Next, after convolution, the three components are fused, and this process is defined as:

$$LMNLo{G}_i(x)={\left(LLo{G}_i{}^2(x)+MLo{G}_i{}^2(x)+NLo{G}_i{}^2(x)\right)}^{\frac{1}{2}}$$

(4)

the MLG similarity can be represented as:

$$SI{M}_{LoG}(x)=\frac{2\cdot LMNLo{G}_1(x)\cdot LMNLo{G}_2(x)+T_1}{LMNLo{G}_1{}^2(x)+LMNLo{G}_2{}^2(x)+T_1}$$

(5)

where T₁ is a constant for increasing formula stability.

To verify the performance of MLG, two groups of original and distorted images with the same specifications are selected from the TID2008 database [35], and the image quality scores of these groups are calculated using different algorithms (i.e., traditional LoG filtering and MLG). The corresponding similar images in this calculation process are listed, as shown in Figure 1.

In Figure 1, it can be seen that there are fewer bright spots in Figure 1g, h compared to Figure 1e,f, indicating that the algorithm in this paper has a strong characterization ability for the differences between reference and distorted images. Moreover, for the score values analysis, the higher the subjective score value, the smaller the difference between the reference and distorted images, and the higher the quality for the distorted image. The lower the objective score, the smaller the difference between the reference and distorted images, and the higher the quality, and vice versa. For example, in Figure 1, the subjective score of Figure 1d is lower than that of Figure 1c, the objective score of Figure 1e is lower than that of Figure 1f, and the objective score of Figure 1g is higher than that of Figure 1h.

These results indicate that MLG outperforms traditional LoG filtering in terms of predictive performance and is more consistent with human eye observation results in characterizing image structural information. The subjective scores of Figure 1c,d show that MLG is more consistent with human visual perception and has a higher prediction accuracy for IQA. It can be seen that MLG can better express the degree of distortion.

2.2. Contrast Similarity Image Calculation

Contrast reflects the brightness variation in images and is a significant visual attribute of image quality. There are many methods for calculating image contrast [16], among which RMS contrast is suitable for capturing natural stimuli with low computational complexity, and performs high consistency with the subjective contrast of natural images [10]. Therefore, this paper adopts RMS contrast as the second feature of the image, and its global contrast expression is:

$$L{C}_i(x)=\sqrt{\frac{1}{H-1}{\displaystyle \sum _{j=1}^H{(x_j-{\mu }_x)}^2}}$$

(6)

where ${\mu }_x=\frac{1}{H}{\displaystyle \sum _{j=1}^H{x}_j}$ is the average intensity of the image.

Based on Equation (6), the global contrast similarity images of the original image f₁ and the distorted image f₂ can be calculated, and the expression is:

$$SI{M}_{LC}(x)=\frac{2L{C}_1(x)\cdot L{C}_2(x)+T_2}{L{C}_1{}^2(x)+L{C}_2{}^2(x)+T_2}$$

(7)

where T₂ is a constant for increasing the stability of the equation.

2.3. Chromaticity Similarity Image Calculation

The change in image colors affects the quality of the image, so chromaticity similarity is introduced as the third feature of the objective method, such as FSIM, VSI, and PerSIM. According to Table 1, the RGB image is mapped to the LMN uniform color space, where M* represents the red–green channel and N* represents the blue‒yellow channel [17]. The chromaticity similarity can be calculated as:

$$SI{M}_{MN}(x)=\frac{2M_1(x)\cdot M_2(x)+T_3}{M_1{}^2(x)+M_2{}^2(x)+T_3}\cdot \frac{2\cdot N_1(x)\cdot N_2(x)+T_3}{N_1{}^2(x)+N_2{}^2(x)+T_3}$$

(8)

where T₃ is a constant for increasing the stability of the equation, M₁, M₂ and N₁, N₂ are the chromaticity channels of f₁ (original image) and f₂ (distorted image), respectively.

2.4. MVFC Method

Based on the quantification results of multi-channel LoG filtering, contrast, and chromaticity, SIM_LoG, SIM_LC, and SIM_MN feature similarity images can be obtained. By pooling the obtained feature similarities in space, an IQA method can be obtained. The index of the proposed method can be calculated using Equation (9):

$$S=w_1\cdot SD\left(SI{M}_{LoG}\right)+w_2\cdot SD\left({\sum }_{\mathsf{\Omega }}SI{M}_{MN}\right)+w_3\cdot SD\left({\sum }_{\mathsf{\Omega }}SI{M}_{LC}\right)$$

(9)

where w₁, w₂, and w₃ are weights representing importance, and w₁+ w₂+ w₃ = 1. Standard deviation (SD) can effectively highlight the feature differences of the image [28]. The SD calculation formula is $SD\left(k\right)=\sqrt{\frac{1}{H}{\displaystyle \sum _{k=1}^H{(k-\overline{k})}^2}}$, $\overline{k}=\frac{1}{H}{\displaystyle \sum _{k=1}^{H}k}$, where k represents the feature similarity value of the image.

In order to express the reliability of this method intuitively, two groups of original distorted image pairs (including subjective scores of images) with the same specifications are selected from the TID2008 database, and the objective scores of the above images are calculated through the proposed method, and then the obtained objective scores are compared with the subjective scores. The higher the subjective score, the smaller the difference between the reference and distorted images and the higher the quality for the distorted image. The lower the objective score, the smaller the difference between the reference and distorted images, and the higher the quality, and vice versa.

As can be seen from Figure 2, the subjective scores of Figure 2b,c decrease in turn, indicating that Figure 2c is more distorted than Figure 2b, while the objective scores calculated by the proposed method decrease in turn, which is in line with the results of human judgment. The subjective and objective scoring results in Figure 2e,f are the same, so they are not repeated here. It can be observed that the proposed method can accurately express the degree of distortion.

The method content described above can be more intuitively expressed in Figure 3, and the calculating process of MVFC is shown in Algorithm 1.

Algorithm 1 MVFC.

Input: The reference image R and 1 distorted image D. Output: An quality score S. % transform into an opponent color space (steps 1–2) H stands for R or D. 1: RHGHBH transfer to LHMHNH 2: Downsample the image % Contrast feature map similarity calculation. (steps 3–9) 3: window ← fspecial(‘gaussian’,2,1.5). 4: window ← window/sum(sum(window)). 5: muH ← filter2(window, LH,’same’). 6: mu_sqH ← muH × muH. 7: sigmaH ← sqrt(abs(filter2(window,i LH× LH,’same’) − mu_sqH)). 8: C2 ← 50. % Define C2 to avoid instabilities when denominator is close to 0 9: SIMLC ← (2× sigmaR×sigmaD + C2)/(sigmaR^2 + sigmaD^2 + C2). % Similarity calculation in chroma channels (M*, N*) (steps 20–22). (steps 10–13) 10: T1 ← 0.00006. 11: mSIM ← abs ((2 × (MR) × (MD) + T1)/((MR)^2 + (MD)^2 + T1)). 12: nSIM ← abs ((2 × (NR) × (ND) + T1)/((NR)^2 + (ND)^2 + T1)). 13: SIMMN ← mSIM×nSIM. % Multi-channel LoG feature map extraction and similarity calculation. (steps 14–19) 14: LHDoG ← imgDoG(LH). 15: MHDoG ← imgDoG(MH). 16: NHDoG ← imgDoG(NH). 17: DoGR ← sqrt(LRDoG^2 + MRDoG^2 + NRDoG^2). 18: DoGD ← sqrt(LDDoG^2 + MDDoG^2 + NDDoG^2). 19: SIMLoG ← abs ((2×DoGR × DoGD + T1)/((DoGR)^2 + (DoGD)^2 + T1)). % Calculate the quality score of D. (steps 20–21) 20: Output ← 0.5 × std2(SIMLoG) + 0.05 × std2(SIMMN) + 0.45 × std2(SIMLC). 21: end

3. Experimental Results and Analysis

3.1. Evaluation Metrics and Protocol

The performance of an FR-IQA method is given by correlation coefficients in the literature, which are measured between predicted and ground-truth quality scores. Four commonly used correlation coefficients are the Spearman rank-order correlation coefficient (SROCC), Kendal rank-order correlation coefficient (KROCC), Pearson linear correlation coefficient (PLCC), and root-mean-squared error (RMSE). The first two can characterize the prediction trend of the IQA method, while PLCC reflects the prediction accuracy of the method, and RMSE reflects the deviation of the method prediction. To calculate the last two indicators, the logistic regression analysis method [17] is adopted, and its mapping function is:

$$f\left(s\right)={\beta }_1\left[\frac{1}{2}-\frac{1}{1+\exp \left({\beta }_2\left(s-{\beta }_3\right)\right)}\right]+{\beta }_{4}s+{\beta }_5$$

(10)

where ${\beta }_1$, …, ${\beta }_5$ are fitting parameters, s represents the original IQA score, and f(s) is the IQA score after regression.

3.2. Databases

Currently, the four commonly used public image databases in the literature are TID2008 [35], CSIQ [36], LIVE [37], and TID2013 [38]. These databases contain many distortion types, rich content information, and noticeable pattern textures. Each image in these databases includes a corresponding mean opinion score (MOS) or differential mean opinion score (DMOS), which is suitable for conducting method-performance testing experiments.

Table 2. IQA benchmark database.

Database	Source Images	Distorted Images	Distortion Types	Observers
TID2013	25	3000	24	971
TID2008	25	1700	17	838
CSIQ	30	866	6	35
LIVE	29	779	5	161

lists the basic information of the databases, including a total of 6345 distorted images and 52 distortion types.

3.3. Parameter Optimization of the Method

As mentioned earlier, the parameter variables of the MVFC are T₁, T₂, w₁, w₂, and w₃. To achieve the optimal performance of the method, it is necessary to determine the ideal variable values. In this research, the method of controlling variables is chosen to calculate the variable values, and the final target value is determined by the corresponding evaluation standard value (PLCC).

First, the values of T₁, w₁, w₂, and w₃ are kept unchanged, and the value of T₂ is set in the interval [20, 70]. If T₂ is too large, it will weaken the feature representation, and if it is too small, it will reduce the calculation stability. As shown in Figure 4, when the T₂ value is in the interval [40, 50], the PLCC value in the four databases is the most ideal. Here, T₂ = 50 is selected. Using the same method, T₁ is set in the interval [0.00003, 0.00008], and when the T₁ value is in [0.00004, 0.00006], the PLCC value of the MVFC in each database is the most ideal, and T₁ = 0.00006 is chosen.

The selection of the numerical values of w₁, w₂, and w₃ is consistent with the above method. Since the values of T₁ and T₂ have been given and w₁ + w₂ + w₃ = 1, it is only necessary to determine the values of two variables. As shown in Figure 5, keeping the value of w₃ unchanged, when the value of w₁ is in the interval [0.44, 0.54], the PLCC value of the MVFC in each database is the most ideal, and w₁ = 0.5 is chosen. Similarly, keeping the value of w1 unchanged, when the value of w₃ is in the interval [0.41, 0.46], the PLCC value of the MVFC is the most ideal, and w₃ = 0.45 is selected.

3.4. Overall Performance Comparison

MVFC method extracts three feature information of the image, namely multi-channel LoG filtering, contrast, and chroma. To demonstrate the performance advantages of the MVFC method, 10 classic algorithm methods are selected for comparison, including SSIM [9], CVSS [12], GSM [13], VIF [14], FSIMc [16], and VSI [17], as well as high-precision methods—RVSIM [39], MAD [40]—high-efficiency methods—GMSD [41]—and recently published methods—CAGS [42]. As shown in

Table 3. Performance comparison of IQA methods on four databases.

Database Criteria		SSIM	VIF	MAD	FSIMc	GMSD	VSI	GSM	RVISM	CVSS	CAGS	Proposed
TID2013	SROCC	0.7417	0.6769	0.7807	0.8510	0.8044	0.8965	0.7946	0.6757	0.8069	0.8316	0.8606
	KROCC	0.5588	0.5147	0.6035	0.6665	0.6339	0.7183	0.6255	0.5146	0.6331	0.6469	0.6810
	PLCC	0.7895	0.7720	0.8267	0.8769	0.8590	0.9000	0.8464	0.7825	0.8406	0.8445	0.8822
	RMSE	0.7608	0.7880	0.6975	0.5959	0.6346	0.5404	0.6603	0.7719	0.6715	0.6639	0.5838
TID2008	SROCC	0.7749	0.7491	0.8340	0.8840	0.8907	0.8979	0.8504	0.7375	0.9001	0.8231	0.9048
	KROCC	0.5768	0.5860	0.6445	0.6991	0.7092	0.7123	0.6596	0.5628	0.7215	0.6289	0.7278
	PLCC	0.7732	0.8084	0.8308	0.8762	0.8788	0.8762	0.8422	0.7954	0.8961	0.8091	0.9000
	RMSE	0.8511	0.7899	0.7468	0.6468	0.6404	0.6466	0.7235	0.8133	0.5956	0.7886	0.5850
CSIQ	SROCC	0.8756	0.9195	0.9466	0.9310	0.9570	0.9423	0.9108	0.8979	0.9580	0.9198	0.9570
	KROCC	0.6907	0.7537	0.7970	0.7690	0.764	0.7857	0.7374	0.7234	0.8173	0.7487	0.8127
	PLCC	0.8613	0.9277	0.9502	0.9192	0.930	0.9279	0.8964	0.9236	0.9589	0.9014	0.9611
	RMSE	0.1334	0.0980	0.0818	0.1034	0.0786	0.0979	0.1164	0.1007	0.0745	0.1137	0.0725
LIVE	SROCC	0.9479	0.9636	0.9669	0.9645	0.9603	0.9524	0.9561	0.9600	0.9672	0.9734	0.9798
	KROCC	0.7963	0.8282	0.8421	0.8363	0.8268	0.8058	0.8150	0.8203	0.8406	0.8658	0.8863
	PLCC	0.9449	0.9604	0.9675	0.9613	0.9595	0.9482	0.9512	0.9570	0.9651	0.9640	0.9754
	RMSE	8.9455	7.6137	6.9073	7.5296	7.6937	8.6816	8.4327	7.9274	7.1573	8.3251	6.9051
Weighted Average	SROCC	0.7942	0.7646	0.8405	0.8847	0.8544	0.9100	0.8452	0.7638	0.8721	0.8588	0.9027
	KROCC	0.6108	0.6049	0.6702	0.7101	0.7022	0.7366	0.6732	0.6006	0.7074	0.6828	0.7414
	PLCC	0.8140	0.8261	0.8619	0.8928	0.8892	0.9033	0.8650	0.8307	0.8869	0.8575	0.9112
Direct Average	SROCC	0.8350	0.8273	0.8821	0.9076	0.9031	0.9223	0.8780	0.8178	0.9081	0.8870	0.9256
	KROCC	0.6557	0.6707	0.7218	0.7427	0.7457	0.7555	0.7094	0.6553	0.7531	0.7226	0.7770
	PLCC	0.8422	0.8671	0.8938	0.9084	0.9127	0.9131	0.8841	0.8646	0.9152	0.8798	0.9297

, the top three evaluation results are highlighted in bold. In addition, the evaluation results in the four databases are weighted averaged and directly averaged, where the weight of each database is determined by the number of distorted images it contains.

As seen from Table 3, MVFC method has good performance in all four databases, and its overall performance ranks first. Especially in the LIVE database, the performance evaluation standard value has a significant advantage compared with the second and third places. Compared with FSIMc and VSI, which also use LoG filtering to extract image structure information, MVFC method has more obvious advantages in the LIVE and CSIQ databases. In the weighted average and direct average values of the first three indicators, the performance of MVFC method ranks first. In addition, in the TID2008 database, the SROCC value of MVFC method is 0.9048, ranking first among the 11 calculation methods. In summary, the MVFC method in this paper has good competitiveness and prediction accuracy.

The methods with the top three places in Table 3 are the MVFC method in this paper (22 times), CVSS (14 times), and VSI (12 times). Moreover, it can be seen from Table 3 that the SROCC and PLCC values of the MVFC method are both greater than 0.8606, indicating that the method not only has a good predictive performance, but also has a better generalization ability.

Then, we will also compare the MVFC method with five learning-based or AI-based IQA methods, namely DoG-ssim, diplQ, Deepsim, DeepFR, and DIQA. Among them, with the SROCC and PLCC values chosen as the performance evaluation criteria. The results are shown in

Table 4. Performance comparison with AI-based IQA methods on three databases.

Database Criteria		DoG-ssim	diplQ	Deepsim	DeepFR	DIQA	Proposed
TID2013	SROCC	0.9070	0.8940	0.8460	0.8760	0.8250	0.8606
	PLCC	0.9190	0.8770	0.8720	0.8940	0.8500	0.8822
CSIQ	SROCC	0.9430	0.9300	0.9190	0.9600	0.8840	0.9570
	PLCC	0.9540	0.9490	0.9190	0.9660	0.9150	0.9611
LIVE	SROCC	0.9610	0.9580	0.9740	0.9810	0.9750	0.9798
	PLCC	0.9630	0.9570	0.9680	0.9840	0.9770	0.9754
Average	SROCC	0.9370	0.9273	0.9130	0.9390	0.8947	0.9325
	PLCC	0.9453	0.9277	0.9197	0.9480	0.9140	0.9396

, with the top three marked in bold.

In Table 4, the top three ranked computational methods based on the total marked count are DeepFR (8 times), MVFC method (7 times), and DoG-ssim (6 times), and their overall performances rank second. As seen from the data, the MVFC method exhibits a high prediction performance. It can be concluded that the MVFC method performs with a higher generalization in the commonly used public image databases.

To further demonstrate the performance advantages of the MVFC method, the fitting performance of different methods is compared using the discrete points provided in the TID2013 database. As shown in Figure 6, the regression curve of the MVFC method has a better correlation with the subjective observation values.

The MVFC integrates HVS characteristics and simultaneously extracts multi-channel LoG filtering, contrast information, and chroma information. It uses deviations and characterizes feature differences. Therefore, it has a better accuracy and generalization compared to other methods. In terms of multi-channel LoG filtering extraction, considering the coordinated changes in luminance, as well as the impact of chroma components on image structure. The accuracy of LoG extraction is improved, and the image structure changes can be better characterized.

3.5. Performance Comparison for Individual Distortion Types

The following analysis focuses on the predictive performance of the method under different distortion types. Since the distortion types in the TID2008 database overlap with those in the TID2013 database, they are not used repeatedly. The performance comparison data for individual distortion methods can be found in

Table 5. PLCC values of IQA methods for each type of distortion.

Database	Distortion Type	SSIM	VIF	MAD	FSIMc	GMSD	VSI	GSM	RVISM	CVSS	CAGS	Proposed
TID2013	AGN	0.8599	0.8555	0.8846	0.9151	0.9494	0.9515	0.9129	0.8674	0.9454	0.9395	0.9537
	ANC	0.7811	0.8169	0.8302	0.8869	0.9133	0.9170	0.8576	0.8331	0.9048	0.8835	0.9138
	SCN	0.8502	0.8680	0.8835	0.8240	0.9389	0.9441	0.9257	0.8474	0.9206	0.8637	0.9470
	MN	0.8231	0.8627	0.7858	0.8478	0.7514	0.811	0.7834	0.8600	0.8039	0.7975	0.8231
	HFN	0.9037	0.9083	0.9223	0.9432	0.9548	0.9639	0.929	0.9161	0.948	0.876	0.8358
	IN	0.7649	0.8346	0.331	0.7216	0.7573	0.8637	0.8168	0.8804	0.7426	0.8344	0.7971
	QN	0.7693	0.8515	0.8365	0.8096	0.9109	0.8726	0.8659	0.7740	0.8944	0.8763	0.9072
	GB	0.9662	0.9602	0.9336	0.9549	0.9094	0.9564	0.9629	0.9733	0.9292	0.9592	0.9028
	DEN	0.9609	0.8892	0.9611	0.9647	0.9756	0.9704	0.9759	0.9343	0.9649	0.9743	0.9802
	JPEG	0.9493	0.9198	0.9612	0.973	0.9834	0.9854	0.9765	0.9626	0.9753	0.9831	0.9846
	JP2K	0.9681	0.9366	0.9738	0.9741	0.9812	0.9831	0.9801	0.9689	0.9699	0.9734	0.9787
	JGTE	0.8859	0.8922	0.8955	0.9178	0.8786	0.9454	0.9217	0.9024	0.8856	0.9098	0.8980
	J2TE	0.8725	0.8650	0.8755	0.8010	0.9087	0.9194	0.8994	0.8835	0.8884	0.8315	0.8884
	NEPN	0.7755	0.7989	0.8552	0.8057	0.8162	0.8154	0.8063	0.7368	0.8405	0.7675	0.8154
	Block	0.5622	0.5620	0.4216	0.5488	0.6351	0.4875	0.6061	0.6393	0.4983	0.6636	0.6253
	MS	0.7849	0.7081	0.6726	0.7870	0.7661	0.8025	0.7721	0.6911	0.7537	0.7577	0.7176
	CTC	0.4691	0.8362	0.2970	0.6838	0.5098	0.6595	0.6470	0.2129	0.4278	0.6346	0.5466
	CCS	0.4998	0.3541	0.1601	0.8377	0.1516	0.8106	0.7384	0.4374	0.0893	0.408	0.7324
	MGN	0.7768	0.7908	0.8528	0.8654	0.891	0.913	0.8430	0.8291	0.8825	0.8768	0.8961
	CN	0.9043	0.8866	0.9343	0.9466	0.9562	0.9544	0.9492	0.9333	0.9480	0.9390	0.9643
	LCNI	0.9141	0.8761	0.9561	0.9564	0.9699	0.9634	0.9570	0.9256	0.9671	0.9639	0.9783
	ICQD	0.7976	0.8147	0.8694	0.8053	0.9192	0.8963	0.900	0.8169	0.9124	0.9176	0.9346
	CHA	0.9599	0.9381	0.9591	0.9798	0.9709	0.9752	0.9683	0.9764	0.9697	0.9713	0.9580
	SSR	0.9690	0.9161	0.9762	0.9766	0.9848	0.9800	0.9840	0.9657	0.9785	0.9739	0.9847
CSIQ	AGWN	0.9148	0.9563	0.9514	0.9383	0.4556	0.9636	0.9505	0.9389	0.9671	0.9646	0.9601
	JPEG	0.9774	0.9833	0.9827	0.9848	0.7621	0.9808	0.9804	0.9823	0.9865	0.9791	0.9860
	JP2K	0.9698	0.9782	0.9842	0.9818	0.7883	0.9745	0.9696	0.9799	0.9856	0.9616	0.9848
	AGPN	0.8916	0.9612	0.9548	0.8172	0.5573	0.8698	0.8499	0.9194	0.9581	0.8514	0.9553
	GB	0.9363	0.9562	0.9713	0.8933	0.6558	0.8761	0.8573	0.9737	0.9815	0.8394	0.9811
	CTC	0.7867	0.9233	0.9306	0.8914	0.8775	0.8686	0.8595	0.8581	0.9449	0.8672	0.9478
LIVE	JP2K	0.8848	0.9392	0.9754	0.9794	0.9771	0.8662	0.8564	0.9619	0.978	0.9776	0.9853
	JPEG	0.9761	0.9817	0.9806	0.9849	0.9832	0.9816	0.984	0.9853	0.9523	0.9861	0.9912
	AWGN	0.9765	0.9907	0.9881	0.9772	0.9655	0.9819	0.8904	0.9809	0.982	0.9857	0.9863
	GB	0.9237	0.9626	0.9415	0.9737	0.9618	0.8544	0.8565	0.9682	0.9709	0.9572	0.9751
	FF	0.8510	0.9527	0.952	0.8515	0.9408	0.8151	0.7925	0.9623	0.9585	0.9541	0.9625

, where the PLCC value is used as the performance evaluation criterion. In each distortion type, the top three PLCC values are highlighted in bold, and the methods with the highest total count of marks are the MVFC method (21 times), VSI (16 times), and CVSS (12 times).

Comparing the data in Table 5, the total count of marks for the MVFC method ranks first among the 11 methods and is significantly higher than VSI and CVSS. It also outperforms other methods in single-distortion evaluation results. The MVFC method performs well in the LIVE and CSIQ databases, with the highest PLCC value of 0.9912, which is significantly better than the computational methods that also use LoG (VSI and FSIMc). A further analysis of the MVFC method, VSI, and CVSS (Table 5) shows that the MVFC method has lower PLCC values for CTC (contrast change) and block (different intensity local block distortions) distortion types, with values of 0.5466 and 0.6253, respectively; VSI has lower results for block (different intensity local block distortions) and CTC (contrast change) distortion types, with results of 0.4875 and 0.6595, respectively; CVSS has lower results for block (different intensity local block distortions), CTC (contrast change), and CCS (color saturation change) distortion types, with results of 0.4983, 0.4278, and 0.0893, respectively. From the above data, it can be seen that the currently more challenging distortion type evaluations are CTC (contrast change) and block (different intensity local block distortions). The MVFC method outperforms CVSS in the evaluation of these two distortion types and performs significantly better than VSI and CVSS in block evaluation. Therefore, the generalization performance of MVFC method is good, both on the TID2013 database and across all databases.

To further validate MVFC method’s performance in evaluating individual distortion types, a comparison was conducted with five recently published journal articles, namely DISIS [43], SG-ESSIM [44], and VSPSI [45]. LGV and SWLGV [46] were tested on the TID2013 database, with the SROCC value chosen as the performance evaluation criterion. The results are shown in Table 5, with the top two marked in bold.

In

Table 6. SROCC values of IQA methods for each type of distortion.

Database	Type	DISTS	SG-ESSIM	VSPSI	LGV	SWLGV	Proposed
TID2013	AGN	0.8450	0.9360	0.9471	0.9210	0.9360	0.9500
	ANC	0.7860	0.8550	0.8722	0.9110	0.9040	0.8859
	SCN	0.8590	0.9350	0.9385	0.8870	0.9300	0.9438
	MN	0.8140	0.7150	0.7712	0.8210	0.8420	0.7480
	HFN	0.8680	0.9200	0.9218	0.9160	0.8740	0.9211
	IN	0.6740	0.8330	0.8758	0.7590	0.7950	0.8082
	QN	0.8100	0.9110	0.8765	0.8660	0.9560	0.9063
	GB	0.9260	0.9690	0.9631	0.9520	0.9610	0.9074
	DEN	0.8990	0.9630	0.9502	0.9840	0.9760	0.9592
	JPEG	0.8970	0.9500	0.9650	0.9700	0.9520	0.9532
	JP2K	0.9310	0.9490	0.9725	0.9450	0.9680	0.9625
	JGTE	0.9060	0.8230	0.9234	0.8950	0.9000	0.8804
	J2TE	0.8650	0.8990	0.9246	0.9180	0.8740	0.8960
	NEPN	0.8330	0.8010	0.8076	0.7190	0.7950	0.8140
	Block	0.3020	0.6230	0.1716	0.6030	0.6010	0.6427
	MS	0.7520	0.7060	0.7715	0.6770	0.7560	0.6566
	CTC	0.4640	0.4520	0.4763	0.6590	0.6670	0.3666
	CCS	0.7890	0.0100	0.8116	0.7500	0.7580	0.7350
	MGN	0.7900	0.9000	0.9135	0.8190	0.8410	0.8903
	CN	0.9070	0.9160	0.9231	0.8380	0.8590	0.9356
	LCNI	0.9320	0.9520	0.9583	0.8730	0.9170	0.9754
	ICQD	0.8320	0.9280	0.8856	0.8450	0.8640	0.9261
	CHA	0.8790	0.8350	0.8923	0.7930	0.7880	0.8396
	SSR	0.9440	0.9640	0.9632	0.8000	0.8100	0.9695
	Average	0.8127	0.8227	0.8532	0.8384	0.8552	0.8531

, the top three ranked computational methods based on the total marked count are VSPSI (20 times), the MVFC method (17 times), and SG-ESSIM (14 times). As seen from the data, the MVFC method exhibits a high prediction performance. Upon further analysis of the data, MVFC method’s average SROCC value across all 24 distortion types is 0.8531, which is very close to the computational methods VSPSI and SWLGV and is significantly higher than the remaining three computational methods. These results also indicate that the MVFC method has good generalizability and strong competitiveness.

3.6. Comparison of Method Computational Efficiency

To compare the computational complexity of the method proposed in this study with that of the other seven methods, a pair of color images with a resolution of 512 × 512 from the TID2008 database were selected as test images. These were run on a computer with a 2.3 GHz Intel Core i5 memory, using MATLAB R2016a as the simulation software.

As shown in

Table 7. Comparison of the efficiency of different IQA methods.

IQA Index	Time Cost/s	IQA Index	Time Cost/s
SSIM	0.126	VSI	0.536
GSM	0.271	RVSIM	0.939
CVSS	0.436	MAD	1.256
CAGS	0.483	Proposed	0.148

, the method in this study only requires a runtime of 0.14 s, demonstrating a relatively low computational complexity. In practical applications, this method can quickly accomplish relevant tasks.

In the comparison of the overall performance in Section 3.4, the MVFC method ranks first. This is because the MVFC method combines the HVS characteristics, that is, the multi-channel structural information, by LoG filtering, and then combines with contrast information and chromaticity information, and the deviation and characteristic difference are used to characterize. The main limitation of this research is to deal with the different distortions in Section 3.5. At present, the IQA method mentioned in this paper does not have a method that can obtain a higher evaluation performance in all types of distortion evaluation. The main reason is that the preprocessing method, extraction method, and pooling method of image features are all different. From Table 5 and Table 6, we can conclude that each IQA method has no means to solve all the distortion type evaluation, which is the main deficiency of current IQA research. Hence, all researchers should pay more attention to solve all the distortion types’ evaluation in future IQA research.

4. Conclusions

In accordance with the high precision and high efficiency requirements of modern FR-IQA methods, this paper proposes a full-reference method (MVFC) based on the fusion of multi-channel visual information from color images. Image structural information is extracted using the multi-channel LoG algorithm and it is pooled with chroma and contrast features using the standard deviation method. The parameters of MVFC are obtained through the control variable method, ultimately resulting in the computational method of this study. During the experimental phase, four classic FR-IQA databases were selected, totaling 109 reference images and 6345 distorted images for testing. The results were compared with those from ten classic FR-IQA methods and five learning-based methods. The experimental results indicate that in terms of the performance evaluation standard of PLCC, the minimum value of this method across the four databases is 0.8822, with the maximum reaching 0.9754. Its stability and precision are superior to most comparison methods. In single-distortion type tests, the highest PLCC value of this method can reach up to 0.9912, with the lowest at 0.5466. Its generalization and prediction performance also outperform most FR-IQA methods. By comparing the computational complexity with seven other methods, the higher computational efficiency of this method is verified. In conclusion, the proposed FR-IQA method in this study demonstrates a stable algorithmic performance. In the future, all IQA methods need to be improved to yield a better performance for the IQA problem in real settings, including the MVFC method.