An Empirical Study of the Quality Governance Level of China’s Civil Aircraft Industry

Abstract

The quality governance level of an industry is a multi-index evaluation problem that must consider multiple dimensions and factors. This study is the first to construct a comprehensive quality governance evaluation model for the civil aircraft industry of China (CAIC). The index system for the quality governance evaluation of CAIC was established using a literature review, enterprise investigation, expert interviews, and questionnaire surveys. An Analytic Hierarchy Process (AHP) was employed to determine index weights. Based on the evaluation model, data from 53 aviation manufacturing enterprises were collected, and the quality governance level of the CAIC was empirically evaluated; thus, quantitative and qualitative evaluation results were obtained. This empirical study shows that the quality governance of the CAIC is currently at a “medium to low” level. Furthermore, critical factors and bottleneck indices restricting the quality governance level of the CAIC were identified.

1. Introduction

Quality governance is a systematic engineering effort involving numerous influencing factors. Quality governance evaluation is a methodical and complex task, as well as a cognitive and decision-making process in management. It is applied in various fields, such as economy, society, technology, education, and engineering. In the quality governance process, the quality governance evaluation system can provide a comprehensive and profound estimation of the risks and benefits expected from the industry. It can also measure the expected objectives and outcomes the industry achieves. Hence, it is essential to objectively understand the industry’s current status and provide critical support for improving industrial processes and optimizing management measures. World-class civil aircraft products have consistently represented the highest technological level in global manufacturing for nearly a century. Ensuring the quality of civil aircraft development and enhancing quality governance in civil aircraft products have become essential topics in studying the civil aircraft industry. Compared with other industries, the highly internationalized civil aircraft industry demonstrates strong monopolistic characteristics, high technology investment, as well as a long cycle and slow return. China currently possesses its own trunk aircraft, C919, regional aircraft, ARJ21, and Modern Ark 60. However, compared to the more mature civil aircraft industries in developed countries, such as Europe and the US, the civil aircraft industry of China (CAIC) is still in its infancy. The complex international market environment and increasingly stringent industry requirements have posed unprecedented challenges for China’s aviation industry. Expediting the enhancement of quality governance in civil aircraft and attaining a prominent position in the industry have become crucial endeavors to ensure the safety of domestically produced civil aircraft and promote the comprehensive and high-quality development of the industry.

The domestic civil aircraft industry guarantees China’s national security strategy and is a strategic resource for addressing challenges and crises. In the era of rapid scientific and technological advancements, integrating modern quality management concepts, constructing an evaluation model and standards for quality governance capability and effectiveness to assess its level in the civil aircraft industry, and enhancing the quality governance effectiveness and quality assurance in aviation manufacturing enterprises are important issues faced by nations worldwide.

This study aims to construct a quality governance evaluation model for the CAIC. Through a literature review, expert interviews, enterprise investigations, and questionnaire surveys, an index system for evaluating the quality governance level of the CAIC is summarized and refined, and the Analytic Hierarchy Process (AHP) is used to determine the index weights. Based on the evaluation model, we conduct empirical research on the quality governance level of the CAIC and obtain preliminary quantitative and qualitative evaluation results. Our study has reference value for analyzing and understanding the quality status of the CAIC and enhancing the quality governance capability of the industry. It also lays the foundation for further standardizing the collection of good quality data on the CAIC and formulating evaluation criteria and standards for the industry’s quality governance level and capability. The contributions of this study are as follows:

(1)

It contributes to establishing a quality governance evaluation model for the CAIC for the first time, including an evaluation index system and the weights of each index.

(2)

It conducts a large-scale enterprise survey and empirical evaluation of the quality governance level of the CAIC, as well as qualitative and quantitative assessment.

(3)

It identifies the critical factors and bottleneck indices that restrict the quality governance level of the CAIC.

For a system, if it cannot be measured, it cannot be managed, let alone improved, and the same is true for the quality government system of the CAIC. The empirical study finds that the quality governance of the CAIC is currently at a “medium to low” level and identifies the important restrictive factors leading to this unsatisfactory quality governance level, among which low coverage of quality laws and regulations, weak government guidance, unsound mechanisms and low technical levels for incident investigation, low standard service capability, very low-order growth rate, low satisfaction with domestic civil aircrafts, and low development quality constitute the fundamental causes. Our study has important practical significance and application value for formulating targeted quality government improvement measures and promoting the high-quality development of the CAIC.

The remainder of this paper is organized as follows. Section 2 reviews the related literature. Section 3 constructs an index system for the quality governance level evaluation of the CAIC. In Section 4, the weights of each index are determined. Section 5 presents an empirical assessment of the CAIC’s quality governance level. Finally, Section 6 summarizes the conclusions of this study and discusses the directions for future research.

2. Literature Review

Quality is a complex, multidimensional concept. Hoyer et al. [1] defined quality from a production perspective as the degree to which a product meets specified requirements. From a customer perspective, Juran and De Feo [2] proposed that quality can be summarized by the term “fitness for use”. They believe that the fitness for use of a product represents its quality, indicating the extent to which it satisfies customer needs. Governance emerged as a concept in the 1990s. Building upon it, Jochem [3] introduced the concept of quality governance, which he defined as a new methodology that enables organizations to establish appropriate quality and performance standards and implement governance over quality management methods. The Commission on Global Governance defines governance as the sum of various ways in which individuals and institutions, both public and private, manage their common affairs and an ongoing process by which conflicting or divergent interests are reconciled, and joint actions are taken [4]. Quality governance encompasses both formal institutional arrangements and rules that have the power to compel individuals to obey and informal institutional arrangements that people agree with or perceive as being in their interests [5]. Evaluating the quality of governance requires a comprehensive assessment of the industry or organization’s institutional arrangements, the effectiveness of their execution, and the implementation of improvement measures. Quality governance evaluation requires selecting appropriate evaluation methods according to the industry or organization’s specific circumstances and evaluation requirements and adopting comprehensive means and analytical tools for evaluation design [6].

In recent years, increasing attention has been paid to research on quality governance, especially in the fields of the environmental protection and food and agricultural product safety, which involve the human living environment and the health industry. In the field of environmental protection, Ali et al. [7] empirically evaluated the role of technological innovation, research and design (R&D), and quality governance in pollution mitigation in the EU economy. Wang et al. [8] explored the compliance relationship between environmental governance attention and environmental quality in the Beijing–Tianjin–Hebei region. Zheng et al. [9] adopted a quasi-natural experimental method to explore the relationship between environmental governance capabilities and water quality. Air and water are natural resources that human beings depend on for survival; thus, in environmental protection governance, the quality governance of air [10,11,12] and water resources [13,14,15] has received particular attention. In food and agricultural product safety, Edelmann et al. [16] empirically evaluated the role of social learning in food quality governance. De Souza et al. [17] investigated the impact of formal (i.e., contracts, standards, processes, and structure) and informal (i.e., social structure, norms, information sharing, value system, and culture) governance instruments on supply chain quality in the dairy industry. Tong et al. [18] explored the quality governance mechanism of imported agricultural products in China. They proposed improving the quality and safety of imported agricultural products through collaborative governance by the government, importers, and consumers.

Some scholars have attempted to construct quality evaluation index systems. Li et al. [19] developed a logistics service quality evaluation index system in the Internet of Things (IoT) context that comprehensively considered enterprise, customer, and IoT technology factors. The index system is divided into three dimensions: enterprise service quality, customer-perceived quality, and remarkable quality, consisting of 8 primary and 24 secondary indices. Li [20] constructed a manufacturing-based product quality capability evaluation index system from a new perspective and conducted an empirical study based on the analysis of manufacturing product quality formation. The author divided the enterprise into seven departments and evaluated 22 influencing factors by the department to construct a quality factor matrix, enabling enterprises to conduct quantitative evaluations from different perspectives. Zhao et al. [21] constructed a macro-quality evaluation index system based on three dimensions: quality conformity, quality fitness for use, and quality externality. They also explored macro-quality evaluation indices and methods to quantitatively describe the quality level, fluctuation trend, and magnitude over different periods. However, the civil aircraft industry, characterized by large investments, long life cycles, high confidentiality, and high complexity, has been rarely addressed. There have been few theoretical and empirical studies on civil aircraft industry quality evaluation. Moreover, given the relatively late development of CAIC, research in this area is scarce.

The evaluation of the quality governance level of the civil aircraft industry is a multi-index evaluation problem involving multiple dimensions and factors. The most widely used multi-index evaluation method is the AHP [22,23]. In the context of balancing adjustment in mortgage credit risk analysis, Ferreira and Santos [24] compared three methods: AHP, Delphi, and model-based analysis of preferences and trade-offs in terms of usability, time consumption, applicability, accuracy, and overall evaluation. They concluded that AHP performed better than the other two methods. Many extended versions of the AHP exist, including the fuzzy AHP (FAHP). To solve the problem of aircraft type selection for airline routes, Dozic et al. [25] proposed an FAHP method to develop a new automated aircraft-type selection system in which logarithmic fuzzy preference programming was used to derive clear priorities from a fuzzy pairwise comparison matrix. Wu et al. [26] adopted the FAHP to establish a competitiveness evaluation model for China’s aviation industry. They assessed the competitiveness of five major Chinese airlines with respect to 5 primary and 17 secondary indices.

The reliability of civil aircraft is an essential metric for measuring the development of the aviation industry and a key factor influencing aviation safety. Current research on civil aircraft quality mainly focuses on pre-control, analysis, post-control, and analysis. Pre-control and analysis refer to data mining techniques and advanced quality management methods used during the production process of civil aircraft to improve their quality. In contrast, post-control and analysis involve exploring the factors affecting quality and safety accidents based on historical data. For example, Cui and Li [27] established a regression model based on panel data to identify the factors influencing civil aviation safety in 10 Chinese airlines. However, only relying on non-textual data is insufficient to represent all the factors influencing civil aircraft quality and safety. Bao et al. [28] collected 17 years of textual records of air traffic control (ATC) incidents from local ATC bureaus of the Civil Aviation Administration of China and divided them into 20 themes using a thematic modeling method. They found that factors affecting ATC incidents in China had gradually transitioned from external to human factors.

Although quality governance has been introduced into the aerospace industry for a long time, such as in the practice of quality governance in space system development [29], there is relatively little research on the quality management of CAIC from the perspective of quality governance.

3. Construction of an Evaluation Index System

First, we refined the existing indices to evaluate the quality governance level of the CAIC. Based on index refinement, we constructed an evaluation index system. To refine the evaluation indices, extensive literature collection and analysis were conducted to establish the evaluation framework. Given the current quality governance situation in the civil aircraft industry, we integrated network data, statistical yearbooks, references, and expert suggestions to focus on the two dimensions of quality governance capability and effectiveness. This approach allowed us to draft a preliminary framework for evaluating the quality governance level of the CAIC.

To establish a preliminary evaluation index system, we further conducted research interviews with the Ministry of Industry and Information Technology (MIIT) and the Civil Aviation Administration of China (CAAC) in April 2022 and June 2022, respectively. The interviewees were the leaders of the Aviation Division of the MIIT and Airworthiness Department of the CAAC, who provided detailed introductions on the roles played by the MIIT and the CAAC in the quality governance of the CAIC. Almost at the same time, from May 2022 to June 2022, we completed research interviews with the headquarters and major manufacturers of four aviation industrial groups: the Aviation Industry Corporation of China (AVIC), Commercial Aircraft Corporation of China (COMAC), Shanghai Aircraft Manufacturing Company (SAMC), and Xi’an Aircraft Industry (Group) Company (XAC). The interviewees were the relevant leaders of the quality management departments at the group headquarters and major manufacturers, totaling more than 10 participants. Through interviews, we learned that the group headquarters and manufacturers in the CAIC bear the primary responsibility of quality governance and play a core role in production, manufacturing, and quality standard implementation, while enterprises play a significant role in their respective fields and drive the development of the CAIC. Subsequently, based on research interviews and the literature analysis carried out in the early stage, combined with the development characteristics and constraints of the CAIC, an initial evaluation framework for the quality governance level of the CAIC was adjusted and refined, and the evaluation indices were further refined to form a preliminary evaluation index system. The index system covers two primary indices of quality governance capability and effectiveness and multiple sub-dimensions, such as Macro-Regulation, Public Services, and Pluralistic Co-Governance. This preliminary index system provides a sound foundation for subsequent index optimization and evaluation model construction.

Exploiting the preliminary evaluation index system, we invited five experts from the CAIC to participate in a questionnaire survey. Through statistical analysis of the frequencies of each index selected by experts, the evaluation index system was eventually determined, as shown in

Table 1. Evaluation index system of the quality governance level.

Primary Index	Weight (Global Weight)	Secondary Index	Weight (Global Weight)	Tertiary Index	Weight (Global Weight)	Quaternary Index	Weight (Global Weight)
Quality Governance Capability	0.7917 (0.7917)	Macro-Regulation	0.2773 (0.2195)	Quality Access Management	0.2166 (0.0476)	Coverage of Access Requirements	0.25 (0.0119)
						Level of Access Management	0.75 (0.0357)
				Quality Supervision Management	0.3372 (0.074)	Coverage of Supervision Requirements	0.375 (0.0278)
						Supervision Management	0.625 (0.0463)
				Quality Incident Investigation	0.3619 (0.0795)	Incident Investigation Mechanisms and Technical Levels	0.7917 (0.0629)
						Investigation Synergy Degree	0.2084 (0.0166)
				Effect of Quality Technology and Method Promotion	0.0844 (0.0185)
		Public Services	0.3261 (0.2582)	Standard Service Capability	0.2658 (0.0686)	Extent to which Standard Service Capability Meets Demand	0.75 (0.0515)
						Service Capability of Standard Service Institution	0.25 (0.0172)
				Metrology Service Capability	0.2788 (0.072)	Extent to which Metrology Service Capability Meets Demand	0.75 (0.054)
						Service Capability of Metrology Service Institution	0.25 (0.018)
				Test and Inspection Service Capabilities	0.2788 (0.072)	Extent to which Test and Inspection Service Capabilities Meet Demand (0.045)	0.625 (0.045)
						International Mutual Recognition of Test and Inspection Service Capabilities	0.375 (0.027)
				Certification Service Capability	0.1052 (0.0272)	Extent to which Certification Service Capability Meets Demand	0.75 (0.0204)
						Service Capability of Certification Service Institution	0.25 (0.0068)
				Quality Information Resources	0.0716 (0.0185)	Extent to which Quality Information Resources Meet Demand	0.2084 (0.0039)
						Satisfaction Degree of Airworthiness Technology	0.7917 (0.0146)
		Pluralistic Co-Governance	0.3967 (0.3141)	Industrial Foundation	0.4995 (0.1569)	Coverage of Quality Laws and Regulations	0.7917 (0.1242)
						Coverage of Quality Systems	0.2084 (0.0327)
				Role of Government	0.2884 (0.0906)	Government Guidance	0.75 (0.0679)
						Government Synergy	0.25 (0.0226)
				Principal Role of Enterprise	0.1465 (0.046)	Role of Prime Manufacturer	0.625 (0.0288)
						Role of Supporting Unit	0.375 (0.0173)
				Role of Social Organizations	0.0656 (0.0206)
Quality Governance Effectiveness	0.2084 (0.2084)	Quality Level	0.4070 (0.0848)	Level of Development Quality	0.6491 (0.0551)	Level of Development Quality: Domestic vs. International	0.7917 (0.0436)
						Level of Development Quality: Military vs. Civil Aircraft	0.2084 (0.0115)
				Level of Operational Quality	0.0719 (0.0061)	Level of Operational Quality: Domestic vs. International	0.7917 (0.0048)
						Level of Operational Quality: Military vs. Civil Aircraft	0.2084 (0.0013)
				Level of Manufacturing Quality	0.2790 (0.0237)	Level of Manufacturing Quality: Domestic vs. International	0.7917 (0.0187)
						Level of Manufacturing Quality: Military vs. Civil Aircraft	0.2084 (0.0049)
		Satisfaction with Domestic Civil Aircrafts	0.2237 (0.0466)
		Economic Benefit	0.3694 (0.077)	Profit Margin	0.1524 (0.0117)
				Revenue Growth Rate	0.2292 (0.0176)
				Order Growth Rate	0.6185 (0.0476)

(due to space limitations, the construction process of the index system is omitted). The index system consists of 2 primary, 6 secondary, 19 tertiary, and 28 quaternary indices. An index with no lower-level indices is referred to as a leaf node index. The measurement methods for each leaf node index can be found in

Table A1. Index measurement and quantification methods.

Primary Index	Secondary Index	Tertiary Index	Quaternary Index	Measurement	Quantification
Quality Governance Capability	Macro-Regulation	Quality Access Management	Coverage of Access Requirements	Compared to the US and Europe, the quality access requirements for the civil aircraft industry stipulated by Chinese laws and regulations are ( ). A. more comprehensive and stricter; B. generally consistent; C. relatively lenient; D. unclear.	1~3 (A = 3, B = 2, C = 1)
			Level of Access Management	The implementation of quality access requirements for the civil aircraft industry by Chinese regulatory authorities is ( ). A. very well; B. well; C. average; D. poor; E. very poor; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
		Quality Supervision Management	Coverage of Supervision Requirements	Compared to the US and Europe, the quality supervision requirements for the aircraft industry stipulated by Chinese laws and regulations are ( ). A. more comprehensive and stricter; B. generally consistent; C. relatively lenient; D. unclear.	1~3 (A = 3, B = 2, C = 1)
			Level of Supervision Management	The implementation of quality supervision requirements for the civil aircraft industry by Chinese regulatory authorities is ( ). A. very well; B. well; C. average; D. poor; E. very poor; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
		Quality Incident Investigation	Incident Investigation Mechanisms and Technical Levels	Compared to foreign countries, the quality incident investigation processes and technical means in China’s civil aircraft industry are ( ). A. more advanced; B. generally consistent; C. Relatively backward; D. Unclear.	1~3 (A = 3, B = 2, C = 1)
			Investigation Synergy Degree	The synergy degree among quality incident investigation departments in China’s civil aircraft industry is ( ). A. very high; B. high; C. average; D. low; E. very low; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
		Effect of Quality Technology and Method Promotion	∕	Compared to foreign countries, the promotion and application of quality technology and method in China’s civil aircraft industry is ( ). A. more effective; B. generally consistent; C. ineffective; D. unclear.	1~3 (A = 3, B = 2, C = 1)
	Public Services	Standard Service Capability	Extent to Which Standard Service Capability Meets Demand	The supporting role of existing standards for the development needs of the civil aircraft industry is ( ). A. very strong; B. strong; C. average; D. weak; E. very weak; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
			Service Capability of Standard Service Institution	Your organization believes that the service level of domestic civil aircraft standard service institutions is ( ). A. very high; B. high; C. average; D. low; E. very low; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
		Metrology Service Capability	Extent to Which Metrology Service Capability Meets Demand	The supporting role of existing metrology standards for the development needs of the civil aircraft industry is ( ). A. very strong; B. strong; C. average; D. weak; E. very weak; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
			Service Capability of Metrology Service Institution	Your organization believes that the service level of domestic civil aircraft metrology service institutions is ( ). A. very high; B. high; C. average; D. low; E. very low; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
		Test and Inspection Service Capabilities	Extent to which Test and Inspection Service Capabilities Meet Demand	The supporting role of existing test and inspection capability for the development needs of the civil aircraft industry is ( ). A. very strong; B. strong; C. average; D. weak; E. very weak; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
			International Mutual Recognition of Test and Inspection Service Capabilities	The international mutual recognition of China’s civil aircraft product test and inspection accounts for ( ). A. over 90%; B. 50–90%; C. 30–50%; D. less than 30%; E. unclear.	1~4 (A = 4, B = 3, C = 2, D = 1)
		Certification Service Capability	Extent to which Certification Service Capability Meets Demand	The supporting role of existing AS9100 certification service for the development needs of the civil aircraft industry ( ). A. very strong; B. strong; C. average; D. weak; E. very weak; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
			Service Capability of Certification Service Institution	Compared to European and American certification institutions, the service of domestic civil aircraft industry certification institutions is ( ). A. better; B. generally consistent; C. relatively backward; D. unclear.	1~3 (A = 3, B = 2, C = 1)
		Quality Information Resources	Extent to which Quality Information Resources Meet Demand	The supporting role of existing quality and airworthiness information resources for the development needs of the civil aircraft industry is ( ). A. very strong; B. strong; C. average; D. weak; E. very weak; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
			Satisfaction Degree of Airworthiness Technology	The supporting role of existing airworthiness technology for the development needs of the civil aircraft industry is ( ). A. very strong; B. strong; C. average; D. weak; E. very weak; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
	Pluralistic Co-Governance	Industrial Foundation	Coverage of Quality Laws and Regulations	Your organization believes that the perfection degree of quality laws and regulations in China’s civil aircraft industry is ( ). A. very high; B. high; C. average; D. low; E. very low; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
			Coverage of Quality Systems	Your organization believes that the perfection degree of quality governance mechanisms (policies, systems, etc.) in China’s civil aircraft industry is ( ). A. very high; B. high; C. average; D. low; E. very low; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
		Role of Government	Government Guidance	Your organization believes that the role of government departments in the quality governance of China’s civil aircraft industry is ( ). A. very strong; B. strong; C. average; D. weak; E. very weak; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
			Government Synergy	Your organization believes that the synergy of government departments in the quality governance of China’s civil aircraft industry is ( ). A. very high; B. high; C. average; D. low; E. very low; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
		Principal Role of Enterprise	Role of Prime Manufacturer	Your organization believes that the role of primary manufacturers in the quality governance of China’s civil aircraft industry is ( ). A. very strong; B. strong; C. average; D. weak; E. very weak; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
			Role of Supporting Unit	Your organization believes that the role of the supporting unit in the quality governance of China’s civil aircraft industry is ( ). A. very strong; B. strong; C. average; D. weak; E. very weak; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
		Role of Social Organizations	∕	Your organization believes that the role of social organizations (industrial associations) currently playing in the quality governance of China’s civil aircraft industry is ( ). A. very strong; B. strong; C. average; D. weak; E. very weak; F. unclear.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
Quality Governance Effectiveness	Quality Level	Level of Development Quality	Level of Development Quality: Domestic vs. International	Compared to the US and Europe, the development quality level of China’s civil aircraft products ( ). A. is higher; B. is basically the same; C. has a gap; D. has a significant gap; E. unclear.	1~4 (A = 4, B = 3, C = 2, D = 1)
			Level of Development Quality: Military vs. Civil Aircraft	Compared to military aircraft, the development quality level of China’s civil aircraft products ( ). A. is higher; B. is basically the same; C. has a gap; D. has a significant gap; E. unclear.	1~4 (A = 4, B = 3, C = 2, D = 1)
		Level of Operational Quality	Level of Operational Quality: Domestic vs. International	Compared to the US and Europe, the operation and support level of China’s civil aircraft products ( ). A. is higher; B. is basically the same; C. has a gap; D. has a significant gap; E. unclear.	1~4 (A = 4, B = 3, C = 2, D = 1)
			Level of Operational Quality: Military vs. Civil Aircraft	Compared to military aircraft, the operation and support level of China’s civil aircraft products ( ). A. is higher; B. is basically the same; C. has a gap; D. has a significant gap; E. unclear.	1~4 (A = 4, B = 3, C = 2, D = 1)
		Level of Manufacturing Quality	Level of Manufacturing Quality: Domestic vs. International	Compared to the US and Europe, the manufacturing quality level of China’s civil aircraft products ( ). A. is higher; B. is basically the same; C. has a gap; D. has a significant gap; E. unclear.	1~4 (A = 4, B = 3, C = 2, D = 1)
			Level of Manufacturing Quality: Military vs. Civil Aircraft	Compared to military aircraft, the manufacturing quality level of China’s civil aircraft products ( ). A. is higher; B. is basically the same; C. has a gap; D. has a significant gap; E. unclear.	1~4 (A = 4, B = 3, C = 2, D = 1)
	Satisfaction with Domestic Civil Aircrafts	∕	∕	As a member of the public, you believe that the role of news reporting in increasing your overall satisfaction with domestic civil aircraft brands is ( ). A. very significant; B. significant; C. average; D. minor; E. very minor.	1~5 (A = 5, B = 4, C = 3, D = 2, E = 1)
	Economic Benefit	Profit Margin	∕	Using the “Operating Profit Margin of Civil Aviation Manufacturing Industry” index with data sourced from the statistical yearbook.	See Note 1
		Revenue Growth Rate	∕	Using the “Revenue Growth Rate of Civil Aviation Manufacturing Industry” index with data sourced from the statistical yearbook.	See Note 2
		Order Growth Rate	∕	Using the index of “Growth Rate of New Subcontracting Orders of Civil Aviation Manufacturing Industry” with data sourced from the statistical yearbook.	See Note 3

Note 1: The normalization method for the Profit Margin index (measured by the operating profit margin of the civil aviation manufacturing industry) is as follows: $Y=\frac{x-X}{X}+0.5$. In this context, $x$ represents the profit margin of the civil aviation manufacturing industry, and $X$ is the profit margin of the overall manufacturing industry in China based on data from 2019. $Y$ represents the normalized profit margin value in the civil aircraft industry. If $Y>1$, $Y=1$; if $Y<0$, $Y=0$. This method maps Profit Margin to the interval [0, 1]. $Y=0.5$ represents a scenario in which the profit margin of the civil aviation manufacturing industry is equal to that of the overall Chinese manufacturing industry. Additionally, if the profit margin of the civil aircraft industry is significantly higher (or lower) than the average level of the Chinese manufacturing industry, it is normalized to one (or zero). Note 2: The normalization method for the index Revenue Growth Rate is similar to that for the Profit Margin. Note 3: The Order Growth Rate index is measured by the growth rate of new subcontracting orders in the civil aviation manufacturing industry owing to data availability. It is challenging to find corresponding data for the overall Chinese manufacturing industry. Therefore, we adopted data from the civil aviation manufacturing industry in China for this index, specifically data from 2019 and the preceding four years, totaling five years. We used x to represent the order growth rate in China’s civil aircraft industry in 2019, and X is the average order growth rate from 2015 to 2019. Then, similar to the Profit Margin method, we normalized the Order Growth Rate index to a value in the [0, 1] interval.

in the Appendix A. It should be noted that the secondary Economic Benefit index under the primary index (Quality Governance Effectiveness) reflects the Economic Benefit of the entire civil aviation manufacturing industry. Based on data availability, the three Economic Benefit tertiary indices are the Profit Margin, Revenue Growth Rate, and Order Growth Rate.

4. Setting Index Weights

We employed the AHP to determine the index weights. To this end, we designed a questionnaire survey. Three experts from the aviation manufacturing industry, academia, and the government–industry regulatory department were invited to complete the questionnaire. They were asked to compare the importance associated with the indices at each level in the evaluation index system and provide their judgments. Based on the questionnaires returned by each expert, the index weights were calculated, and consistency tests were performed following the procedure specified by the AHP (the calculation process is omitted here). A response questionnaire that failed the consistency test was returned to the respondents for revision until the consistency test was satisfied. After obtaining the index weights from the three experts, a simple arithmetic average was calculated and used as the index weight. Table 1 lists the global weights of each index in the evaluation system. The global weight of an index can be obtained by multiplying its local weight at the current level with the local weights of its parent indices at higher levels.

Table 1 reveals that among the two primary indices determining the quality governance level of CAIC, Quality Governance Capability is the most important index, followed by Quality Governance Effectiveness. This result aligns with the fundamental principle that “capability determines effectiveness, which stems from capability”. Among the six secondary indices, the top-priority index is Pluralistic Co-Governance, indicating that the quality governance of the civil aircraft industry is a complex system engineering effort requiring collaboration across multiple sectors and the entire society. Public Services ranked second, followed by Macro-Regulation, Quality Level, Economic Benefit, and Satisfaction with Domestic Civil Aircrafts indices.

Summarizing and ranking the global weights of all leaf node indices in descending order, we sorted the top one-third of the indices as “critical”. The underlying indices located at the leaf nodes require focused attention and improvement to enhance the quality governance level of the industry. In the CAIC, eleven critical factors determine the quality of governance. They are listed in descending order of importance:

(1)

Coverage of Quality Laws and Regulations;

(2)

Government Guidance;

(3)

Incident Investigation Mechanisms and Technical Levels;

(4)

Extent to which Metrology Service Capability Meets Demand;

(5)

Extent to which Standard Service Capability Meets Demand;

(6)

Order Growth Rate;

(7)

Satisfaction with Domestic Civil Aircrafts;

(8)

Supervision Management;

(9)

Extent to which Test and Inspection Service Capabilities Meet Demand;

(10)

Level of Development Quality: Domestic vs. International;

(11)

Level of Access Management.

5. Empirical Evaluation of the Quality Governance Level

5.1. Data Sources

The index system and associated weights together form a comprehensive evaluation model for the quality governance level of the CAIC. Using this model, we empirically evaluated the current quality governance level of the CAIC based on enterprise and industry statistics. The index data served as the basis for empirical evaluation.

In March 2022, we carried out data collection through an enterprise survey. The survey process consisted of a pre-survey and a formal survey. First, one business expert each from Commercial Aircraft Corporation of China (COMAC) and Xi’an Aircraft Industrial Corporation (XAC), as well as two domain professors from academia, were invited to complete the preliminary survey questionnaire and provide suggestions on the setting of the questions. After modifying and refining the questionnaire, a formal survey was carried out. The questionnaires were issued to 53 domestic enterprises involved in different civil aircraft products, including prime manufacturers of mainline aircraft, regional aircraft, helicopters, general aviation aircraft, drones, and engines, as well as component suppliers and subcontractors. Each enterprise provided the final answers to each question after a collective discussion among internal domain experts led by the quality management department or an institution with similar functions. For each enterprise surveyed, questionnaires were individually distributed, collected, and analyzed by a designated contact person. Ultimately, within the specified timeframe, a total of 53 questionnaires were returned, with a response rate of 100%. In this study, we did not make a distinction between the influence of the surveyed enterprises, that is, their responses contributed equally to the evaluation. It should be noted that some indices are not reported by enterprises or reported as “unknown” or “unclear”. However, the missing data were scattered across different indices and enterprises rather than concentrating on specific indices or firms. In other words, data from dozens of enterprises were available for all indices. Therefore, we believe that these 53 enterprises can represent the current status and effectiveness of the quality governance in the entire CAIC in terms of the statistical average.

The three leaf node indices–Profit Margin, Revenue Growth rate, and Order Growth Rate– are based on macro data at the industry level and do not require reporting by individual enterprises. The data for these three indices were sourced from the 2019 Statistical Bulletin of China’s Civil Aviation Manufacturing Industry.

5.2. Index Quantification and Normalization

All indices were quantified and normalized to values within the interval [0, 1]. First, we must quantify all leaf node indices and assign them a numerical value. The methods for index measurement and quantification are presented in the Appendix A. For leaf node indices whose quantized values are not in the interval [0, 1], the normalization method is as follows:

$$\overline{X}=\frac{X-X_{min}}{X_{max}-X_{min}}$$

(1)

where $X$ represents the quantized value of a particular index, $X_{max}$ is the maximum possible value of that index, and $X_{min}$ indicates the minimum possible value of that index. Consequently, the normalized value of the index, denoted by $\overline{X}$, is within the [0, 1]. For instance, for a leaf node index assigned discrete integer values in the range of 1–5, $X_{max}=5 \mathrm{a}\mathrm{n}\mathrm{d} X_{min}=1$. If this index is assigned a value of $3.2$, its normalized value would be $\overline{X}=\frac{3.2-1}{5-1}=0.55$.

It should be noted that the three leaf node indices under Economic Benefit, namely, Profit Margin, Revenue Growth Rate, and Order Growth Rate, are based on actual statistical data for the civil aviation manufacturing industry rather than quantified values of the qualitative indices. These three indices, representing an industry’s economic performance, are normalized to a value within the [0, 1] interval differently from most indices based on their respective characteristics and data availability. Please refer to the notes in the Appendix A for details on how they are normalized.

5.3. Evaluation Results and Analysis

First, the evaluation values for the leaf node indices were determined by taking the industry average represented by the 53 aviation manufacturing enterprises. Knowing the evaluation values for all leaf node indices, by embedding the index weights, we can calculate the weighted evaluation values of each level of index step-by-step in a backward derivation manner until the two primary indices. The evaluation values of the primary indices of quality governance capability and effectiveness are 0.4807 and 0.3528, respectively. The complete evaluation values for all the indices are summarized in Figure 1. Finally, by synthesizing the two primary indices to calculate the weighted average, a comprehensive evaluation value for the quality governance level of the CAIC, denoted as $L_q$, is obtained:

$$L_q=0.4867\times 0.7917+0.3528\times 0.2084=0.4588$$

(2)

Furthermore, considering that a comprehensive evaluation value can never be completely accurate, we adopted a five-level scale to qualitatively evaluate the quality governance level, divided into five grades from low to high: very low, low, medium, high, and very high.

Table 2. Grading of the quality governance level.

$\mathbf{Range} \mathbf{of} {\mathit{L}}_{\mathit{q}}$	$[0,0.2)$	$[0.2,0.4)$	$[0.4,0.6)$	$[0.6,0.8)$	$[0.8,1]$
Grade	Very low	Low	Medium	High	Very high

presents the grade differentiation with respect to the five possible value ranges of $L_q$. According to Table 2, the current quality governance level of CAIC is graded as “medium”. As $L_q<0.5$, we believe that, to be accurate, the current quality governance of the CAIC is at a “medium to low” level.

Similarly, based on Figure 1 and Table 2, the qualitative evaluations for each index level are as follows:

Among the two primary indices, Quality Governance Effectiveness is graded as “low”.

Among the six secondary indices, the two indices of Satisfaction with Domestic Civil Aircrafts and Economic Benefit are graded as “low”.

Among the 19 tertiary indices, four indices are graded as “low”—Standard Service Capability, Industrial Foundation, Role of Social Organizations, and Profit Margin—while Order Growth Rate is graded as “very low”.

Among the 28 quaternary indices, seven indices are graded as “low”. These are the Incident Investigation Mechanisms and Technical Levels; Extent to which Standard Service Capability Meets Demand; International Mutual Recognition of Test and Inspection Service Capabilities; Coverage of Quality Laws and Regulations; Government Guidance; Level of Development Quality: Domestic vs. International; and Level of Manufacturing Quality: Domestic vs. International.

Indices that perform “low” or “very low” constrain the quality governance level of CAIC. These are essential factors contributing to the current quality governance of the CAIC at “medium to low” levels. To improve the quality governance level of the CAIC, special attention must be paid to them, especially the underlying indices at the leaf node.

From the indices that influence the CAIC’s quality governance level, we further identified a few critical underlying indices. In Section 4, we identified 11 critical factors that determined the quality governance level of the CAIC. Among these 11 indices, we selected the leaf node indices rated as “low” or “very low” and referred to them as bottleneck indices. Consequently, we identified seven bottleneck indices ranked in descending order of importance:

(1)

Coverage of Quality Laws and Regulations;

(2)

Government Guidance;

(3)

Incident Investigation Mechanisms and Technical Levels;

(4)

Extent to which Standard Service Capability Meets Demand;

(5)

Order Growth Rate;

(6)

Satisfaction with Domestic Civil Aircrafts;

(7)

Level of Development Quality: Domestic vs. International.

The first four indices are related to quality governance capability, whereas the remaining three indices are associated with quality governance effectiveness. These seven indices represent bottlenecks in the CAIC’s quality governance and are crucial for rapidly improving the quality governance.

6. Conclusions

The civil aircraft industry is a high-end manufacturing industry with strategic significance and broad prospects for development. With the continuous development of China’s economy and the rapid expansion of civil aviation worldwide, market competition in the civil aircraft industry is becoming increasingly intense, bringing higher requirements and challenges to quality governance. Therefore, conducting a scientific quality governance level evaluation is essential for the CAIC to improve core competitiveness and market position and a crucial guarantee for promoting the industry’s high-quality development. This study is the first attempt to evaluate the quality governance level of the CAIC systematically and quantitatively, thus providing support for research and improvement of the industry’s quality governance capability and effectiveness.

This study constructs an evaluation model for the quality governance level of the CAIC. First, we established an evaluation index system for the CAIC’s quality governance level. The AHP method was used to determine the index weights. Exploiting the evaluation model, an empirical study of the quality governance level of the CAIC was conducted based on survey data from 53 aviation manufacturing enterprises and industry statistics. The empirical analysis shows that the comprehensive evaluation value for the CAIC’s quality governance level is 0.4588, indicating that the current quality governance of the CAIC is “medium to low”. We also identified 11 critical factors influencing the quality governance level of the CAIC and seven bottleneck indices for improving the quality governance level.

This study can be regarded as a comprehensive diagnosis of the CAIC’S quality governance level, which reveals the causes of unsatisfactory quality governance level and provides action guidance for improvement. At least corresponding to some deadly bottleneck indices, we can give the following suggestions:

(1)

Government departments should formulate targeted policies and measures to address the identified shortcomings, such as improving the relevant regulatory framework, strengthening the government’s macro-guidance, and optimizing the incident investigation mechanism.

(2)

The industry or academic community should actively organize workshops or forums, inviting enterprises, research institutions, and other relevant parties to engage in in-depth discussion on the root causes of these shortcomings and develop corresponding strategies. They can also conduct targeted in-depth research and analysis, such as examining the gaps between standard service capabilities and actual needs, thus providing more comprehensive policy recommendations for decision-makers.

As an attempt to explore quality governance issues in the CAIC, this study has some limitations. In view of these limitations, future research should focus on the following aspects:

(1)

Expand the scope of the industry survey, and employ statistical methods to improve and optimize the evaluation index system at the quality governance level. In addition, improving the gathering and processing methods for evaluation indices’ data may make the empirical evaluation results more objective and reasonable.

(2)

Introduce a metric that comprehensively represents the quality governance level of the CAIC, widely recognized by the industry, and conduct longitudinal dynamic-analysis-based annual metrics.

(3)

Compare horizontally the quality governance levels of the civil aircraft industry in aviation manufacturing between China and developed countries.