Research on Optimization of VR Welding Course Development with ANP and Satisfaction Evaluation

Abstract

The purpose of this study is to explore the ability indices of VR (virtual reality) technology when it is applied to assist the teaching of a welding practice course, develop a VR welding course as the basis of course planning and teaching design, and implement experimental teaching to verify its effectiveness. On the basis of a literature review and focus group interviews, initial ability indices of a VR welding course were proposed. Then, 15 experts from VR- and welding-related specialties were invited to form a consulting team to determine “the ability indices of the VR welding course” according to the results of a Fuzzy Delphi expert questionnaire. Moreover, the results of an ANP (Analytic Network Process) expert questionnaire were used to understand the relative importance of the ability indices of the VR welding course, as well as the relative feasibility of VR-assisted welding teaching, in order to develop a “VR welding course”, in which 34 first-grade students of the welding practice course were taken as the research objects during the implementation of experimental teaching. The qualitative research and analysis results are as follows: (1) the VR welding course includes 8 ability indices and 30 evaluation indices; (2) the item with the highest feasibility in VR-assisted welding teaching is “welding construction”, followed by “map reading and drawing”; (3) best feasibility of VR technology “Interaction” in assisting welding teaching; (4) the relative importance of the ability indices of the VR welding course is the greatest for “welding construction”, followed by “welding inspection”; (5) the VR welding course students express significant positive responses to the learning of ability indices and ability demonstration; (6) the majority of students express significant positive learning satisfaction with VR-assisted welding course teaching. This study puts forward a set of rigorous models for the construction of ability indices for a VR course and course development. It can provide a reference for introducing VR-assisted teaching to related welding courses that are run by universities of science and technology in Taiwan. Furthermore, such VR courses can offer students a safe, diversified, and efficient learning environment.

1. Introduction

With the rapid growth of the economy and improved industrial technology in Taiwan, welding technology has been widely used as a metal processing method in modern times. Welding can be used to process and repair almost all metal products, such as spacecraft engines, nuclear reactors, household appliances, automobiles, machinery, shipbuilding, and architecture [1]. According to the data of the National Institute for Occupational Safety and Health (NIOSH), the labor population of industrialized countries engaged in welding operations is estimated to be 0.2–2% of the total labor population [2]. In contrast, the number of welding operators in Taiwan’s industry is quite large, the employment scope of those with this skill is very wide, and the demand for human resources is still on the rise, which shows that the training of welding technology professionals is worthy of attention.

Moreover, under the policy of actively promoting the offshore wind power industry in Taiwan, the 104 Job Bank pointed out that such vacancies have risen sharply in Taiwan since 2016. In 2019, the number of job vacancies was 103 more than that in 2017, for a growth rate of 3.6 times. All welding works of fan base, mechanical, and electrical parts of maintenance equipment play significant roles in initial construction [3]. Therefore, the quality of welding technology directly determines the quality of products and construction [4]; thus, improving the welding techniques and construction quality of welders is one of the goals that many traditional industries intend to achieve.

According to the statistics of the Skill Evaluation Center of Workforce Development Agency of the Ministry of Labor (Taiwan) [5], there are more than 1000 applicants for the examination of general manual welding technicians every year, as shown in Figure 1, and there is an increasing trend year by year, with more than 1600 applicants in 2019. Nevertheless, because of the ability-oriented demand of welders, the qualification rate of the examination in 2019 reached a new low of 58.3%, as shown in

Table 1. Statistics for the qualification of the general manual welding examination.

Year of Examination	Number of Applicants	Number of Qualified	Qualification Rate of Applicants
2016	1132	662	58.5%
2017	1221	876	71.7%
2018	1412	858	60.8%
2019	1646	960	58.3%

Occupational class group: welding piping class group, subdivision of occupational class-00400 [5].

, which has led to a serious shortage of welding talent in the industry.

The fundamental solution lies in the reform and practice of training and educating welders. The traditional training mode of welding technicians has a long duration and high costs, such as equipment, manpower, raw materials, and energy costs; it also consumes a lot of electrodes and workpiece materials and produces toxic and harmful gases, dust, ultraviolet light, radiation, electromagnetic pollution, and other harmful substances during welding operations [6]. Therefore, new methods that apply emerging technologies to train welding talents, improve the learning of welding knowledge and ability, reduce the investment cost of welding training, and reduce the physical harm caused by the welding training process are worthy of active consideration and improvement by technical and vocational education and engineering education units.

Although mechanical arms can support welding work at present, their accuracy is not as good as that of professional masters [7]. In addition, as the diversity and complexity of the welding work environment test the welding stability and coordination of welding technicians, the development of welding ability requires long-term basic training and hands-on learning. At present, virtual reality (VR) technology is widely used in education and skills training to provide immersive learning, and it has achieved remarkable results [8,9]; for example, in [10], VR was applied to engineering training and education, and in [11], VR was applied to safety education and training in nuclear power plants. VR can guide trainees to learn in an interactive and exploratory manner without being restricted by the actual environment, and it can assist learners in understanding course knowledge and improving their practical abilities. The above shows that the application of VR technology in engineering-related practical teaching can strengthen the immersive and interactive characteristics of students’ experience and reduce the time of learning and exploration [12]. In addition, it can further provide students with the opportunity to practice repeatedly and increase their skill proficiency without increasing the cost of additional consumables or providing a safe learning field [13].

Specifically, the main research purposes of this study are as follows:

(1)

Construct the ability indices of a welding practice course;

(2)

Discuss the importance of the ability indices of the welding practice course;

(3)

Understand the feasibility of VR technology-assisted welding course teaching;

(4)

Develop a VR welding course;

(5)

Discuss the effectiveness of students’ learning in the VR welding course.

2. Literature Review

2.1. Development and Application of Welding Technology

Welding is the technology of heating more than two ferrous or non-ferrous metal weldments to an appropriate temperature (or melting them) at the point to be joined, adding electrodes (or not adding electrodes), applying pressure (or not applying pressure), and then combining them into a whole after condensation in order to form a permanent connection between atoms [14]. Common welding methods include gas welding, resistance welding, arc welding, laser welding, and e-beam welding, and welding technology is widely used in many industries [15]. At present, there are more than 100 laser welding production lines that are continuously and rapidly growing in the global automobile industry, including BMW and Volkswagen of Germany, General Motors of the United States, Toyota of Japan, and other automobile assembly lines, which produce more than 70 million automobile sheet metal parts every year. In addition, welding technology is widely used in the shipbuilding industry, in which the main welding methods include gas-shielded welding, submerged arc welding, and manual arc welding, which meet the requirements of various production processes and greatly improve welding efficiency.

2.2. Welding Technology Training

As welding is a professional field, it requires learners to possess basic welding knowledge and skills acquired from schools or vocational training schools. The knowledge and skills include the use of common welding equipment, such as the argon welding machine, electric welding machine, welding gun, electrodes, grounding clips, and other operations, as well as the use of hand tools and slag removal tools, such as the steel brush and slag hammer [7]. Furthermore, welding personnel must identify the material characteristics according to the construction drawing and then select the appropriate welding methods, fillers, and electrodes to carry out metal processing. They also visually check the weld beads for defects, such as penetration, corrosion, insufficient penetration, and cracking. Moreover, mechanical testing (e.g., X-ray irradiation, ultrasonic testing) is conducted in accordance with customer requirements for accuracy. The learners should be familiar with welding structure drawings, symbols, performances, the installation methods of various welding machines, the characteristics of various special ferrous and non-ferrous materials, and the selection of electrodes/welding lines/fillers. Moreover, it is also necessary for learners to possess knowledge of welding and cutting metallurgy and know how to prevent the deformation of weldments, preheat/post-heat, and other treatment methods.

In recent years, the exposure hazard for welders has become an important topic in the research of labor safety and health in advanced industrial countries [16]. The welding operation environment must consider whether there are flammables or volatile gases and provide good ventilation equipment to avoid humid or conductive operation environments. During an overhead welding operation, welders must wear sling hooks to ensure their safety. Personal protective equipment, such as safety helmets, respiratory protective equipment, protective gloves, safety belts, safety shoes, and protective clothing, must be used to ensure personal safety. The above protective equipment is used to prevent scalds and burns from flying objects, arc lighting, toxic gases, burns, electric shock, and other disasters [17,18]. Finally, students must acquire basic welding skills and relevant knowledge and cultivate the spirit of hard work, diligence, and responsible service, as well as professional ethics, and be able to engage in general manual welding, inert gas tungsten arc welding, gas flame cutting, and other operation skills after training.

To summarize, the eight ability indices of the welding course, as initially proposed in this study, include basic map reading ability, preparation before welding, welding material processing and combination, welding construction ability, slag cleaning treatment, welding quality inspection, welding safety, and health and professional ethics.

2.3. VR Technology Development

VR is employed to create three-dimensional virtual scenes using computer-related software and hardware technologies and provide visual, auditory, and other sensory simulations to immerse users in the scene. Furthermore, users can interact with virtual objects in the scene through various sensing devices (e.g., 3D stereoscopic glasses, stereoscopic gloves) [19].

A successful VR course should have three important characteristics: interaction, immersion, and imagination [20,21].

2.3.1. Interaction

Interaction refers to the human–computer interaction between users and VR through the operational interface. Liang and Lee [22] pointed out that a VR system should have an interactive function to detect users’ movements in real time and immediately provide corresponding responses; for example, users can use appropriate tools to move, rotate, and scale the objects in the virtual environment [23]. It has been demonstrated that VR with effective interaction can increase learners’ interest in learning and improve the application of VR [24,25,26]. This study posits that human–computer interaction is necessary for VR-assisted teaching; therefore, students’ actions and interface operations in the virtual welding field must be responded to within the shortest time possible to present real-time, dynamic, and interactive scenes.

2.3.2. Immersion

Immersion in VR refers to a 3D virtual environment, as generated by computer-related software and hardware technologies, in which users can be immersed. According to relevant research, the virtual environment should be directly felt and controlled by users so that they are immersed in the virtual world. When VR is applied to teaching, the higher the degree of immersion in the virtual environment, the better the learning effect [27]. This study suggests that VR should apply computer software and hardware technologies to generate a 3D virtual environment so that users can be immersed in the virtual environment. Therefore, this study aims to achieve a visual and auditory pseudo-sensory simulation and effect through the application of various sensing devices (e.g., 3D stereoscopic glasses, stereoscopic stereo) for the purpose of assisted teaching.

2.3.3. Imagination

Imagination means that the production of VR can provide users with an imaginary space. Krueger [28] pointed out that users can exercise their creativity and create their own imaginary space after their visual and auditory senses are stimulated in virtual scenes. According to relevant research, VR has the characteristics of imagination, which enables users to truly understand the spatial relationship of objects in the virtual environment [29], such as welding plants, equipment, steel plates, and protective appliances. Therefore, through the design of virtual scenes, this study provides more virtual spaces in which students can create lively practice modes and usage situations, which enhances students’ intention to use and interest in learning [30].

2.4. Integration of VR into Education

The rapid development of VR technology has stimulated reforms in education. The greatest difference between conventional education and VR is that two-dimensional course content can be presented in real-world 3D scenes from the development and application of the latter. For example, students can visit scenic spots and historical sites across the world in a history class, appreciate the artworks of Vincent van Gogh in an art class, and see the Himalayas or gorges in a geography class. Additionally, surgical procedures can be simulated in a school of medicine. These possibilities are out of reach in conventional teaching and classes.

The aim of integrating VR into education is to allow students to learn from hands-on experience so that they will find learning easier, more enjoyable, and more efficient. The pertinent application effects of engineering education are as follows. Price, Kuttolamadom, and Obeidat [8] proposed the use of a VR welding system and produced satisfactory results in improving manufacturing process education. Furthermore, Stone, Watts, and Zhong [31] probed into the effects of “traditional training (TT)” and the “virtual reality integrated training (VRI)” on the training results, team learning, and material costs of welding training. After two weeks of experiments, the results demonstrated that, in terms of training effects, the team learning of the VRI group outperformed that of students receiving traditional training; material costs of the VRI group were significantly lower than those of the TT group. Furthermore, Byrd, Anderson, and Stone [32] indicated that virtual welding simulators could evaluate the existing skill level of welders from five welding parameters, namely, arc length, position, work angle, travel angle, and travel speed. In another study, for basic courses (at low and medium weld difficulty levels), fully virtual training was able to satisfy training needs and be introduced to welding courses at the beginning. By contrast, for advanced courses (at a high level of difficulty), VR training courses were needed [33].

3. Research Design and Implementation

3.1. Research Design

In this study, a VR welding course was developed, and its implementation effectiveness was evaluated, as shown in Figure 2. First, the ability indices of the VR welding course were determined through literature analysis and expert interviews. Second, the relative importance of the ability indices and the feasibility of VR-assisted welding teaching were established, the teaching mode and course content were constructed, and experimental teaching was implemented to verify the effects. Fuzzy Delphi, ANP, and other methods were used to conduct the literature review, expert interviews, and the questionnaire.

3.2. Research Methods

The methods used in this study include the Fuzzy Delphi method, ANP, and the questionnaire method, which are described below.

3.2.1. Fuzzy Delphi method (FDM)

The FDM, which is an effective method for the construction of indices, is a combination of the Delphi method and fuzzy theory and can compensate for the shortcomings of the traditional Delphi method and solve the limitation of human fuzziness using triangular fuzzy numbers. Therefore, in this study, the Fuzzy Delphi method was adopted to construct the ability indices of a VR welding course.

The Fuzzy Delphi method can express the trend of data concentration in the form of intervals and integrate the opinions of experts according to the concept of the membership function in fuzzy theory, with a range between 0 and 1, where the higher the degree of intention, the closer the membership value is to 1; otherwise, it is closer to 0 [34]. In this study, the method of fuzzy set defuzzification was used to obtain the total membership values of the actual measured indices by assuming the concepts of the membership functions of the maximum set and minimum set. Then, the Max-Min method was used to integrate the experts’ fuzzy weight evaluation values, which are expressed with the right bound value (μR), the left bound value (μL), and the total value (μT), after defuzzification. FDM was performed according to the following steps: (1) select the preference scale of the ability indices; (2) select the triangular fuzzy numbers of the ability indices; (3) determine the right bound value; (4) determine the left bound value; (5) establish the total fuzzy number of the indices; (6) select the indices.

3.2.2. ANP

Satty proposed the analytic network process (ANP) in 1996, which allows complex interrelations between decision levels and attributes in the framework, meaning that the elements in the cluster may be affected by some or all of the other elements in the cluster. The source cluster, the middle cluster, and the bottom cluster are organized into a network with a dependent and feedback relationship [35]. In this study, ANP was used to analyze the data and prioritize the ability indices to determine the relative importance and feasibility weights of the ability indices of the VR welding course.

The main implementation steps of ANP for decision analysis are as follows [36,37]:

(1)

Establish the problem structure

ANP determines the goal according to the characteristics of the problem, determines the decision criteria and sub-criteria contained in each criterion group, and identifies the mutual influence among criteria. If the criteria influence each other, it is outer dependence; if the sub-criteria contained in each criterion group influence each other, it is inner dependence. Then, the overall structure of the decision problem is drawn.

(2)

Pairwise comparison of decision criteria

Pairwise comparison aims to compare the criteria in pairs and can be divided into two parts: pairwise comparison among criteria and the comparison of sub-criteria within the criterion group. The pairwise comparison of sub-criteria further includes pairwise comparisons in the same group and pairwise comparisons of elements in different groups. In pairwise comparison, if two elements are in opposite positions, their values can be substituted by reciprocal values, i.e., $a_{ij}=\frac{1}{a_{ji}}$. In ANP, pairwise comparisons provide a matrix framework and an original priority vector to obtain the evaluation and comparison of the relative importance of the relationship between elements and matrices. Finally, the rationality of weight allocation is tested according to the questionnaire results of each interviewee, which takes the consistency ratio (C.R.) as the judgment basis. If the value of C.R. is less than 0.1, the consistency meets the acceptable standard; otherwise, it is necessary to adjust the judgment matrix of this level in order to reach a satisfactory consistency of the overall ranking of levels.

(3)

Form a supermatrix

A supermatrix is an effective method for solving the dependence of each criterion in a system; it is composed of multiple submatrices, each of which contains the interaction relationship of each group’s own elements, and is compared with the interaction of other group elements in pairs. The value of each submatrix is calculated by pairwise comparison, and the feature vector is used as the weight value of the submatrix to form the final supermatrix. The calculation process of ANP consists of three matrices, namely, the unweighted supermatrix, the weighted supermatrix, and the limit supermatrix. The unweighted supermatrix is the weight obtained from the original pairwise comparison. The weighted supermatrix is the weight of the same element in the unweighted supermatrix multiplied by the relevant community weight so that the sum of each straight column is 1. The limit supermatrix functions to multiply the weighted supermatrix by multiple power until it converges, that is, until the number of each column is equal, as shown in Equation (1).

$$W_{\lim }=\underset{k\to \infty }{\lim }{\left(W_{weighted}\right)}^k$$

(1)

In sum, in this study, ANP analysis was used to obtain the importance of the ability indices of the welding course, as well as the feasibility of VR technology-assisted welding course teaching, which can be used as the basis for the construction of the teaching mode of the welding course.

3.3. Research Subjects

Given the purpose of this study, the research objects are grouped into two main teams: the consulting expert team and the student team. In terms of the Fuzzy Delphi method and the ANP expert questionnaire, 15 experts and scholars with rich experience in welding and VR were invited, including 8 scholars and 7 experts in the industry, as shown in

Table 2. Expert Background Information.

Expert No.	01	02	03	04	05	06	07	08	09	10	11	12	13	14	15	Total
Title *	1	4	2	5	7	2	7	2	3	6	3	6	8	1	2	-
Seniority (Year)	16	18	5	16	8	8	9	6	5	12	7	11	14	17	10	-
Academia	V		V			V		V	V		V			V	V	8
Industry		V		V	V		V			V		V	V			7
Welding Expertise	V	V		V	V	V	V	V		V			V	V		10
VR Expertise			V			V		V	V	V	V	V			V	8

* Code of titles: 1 Professor, 2 Associate Professor, 3 Assistant Professor, 4 General Manager, 5 Factory Manager, 6 Manager, 7 Group Leader, 8 Chief Engineer.

. The interviewees were considered important and representative; they mainly replied to revise and confirm the ability indices, the index network structure, and the importance and feasibility of applying VR technology in the assisted teaching of the welding course. In terms of quasi-experimental teaching, 34 first-grade students of a welding practice course at the case University of Science and Technology were taken as the research objects. An 18-week VR welding course was conducted, a learning satisfaction questionnaire was implemented, and statistical analysis was performed to explore the performance of students’ welding ability.

3.4. Research Tools

In the first step of this study, the key points of teaching a welding course and the application of VR in engineering and teaching were summarized on the basis of literature analysis. Then, interviewees were invited to express their opinions regarding the first revision to develop the framework of the ability indices of the VR welding course, as shown in

Table 3. Suitability Score Table of Ability Indices of the Main Dimensions of the Welding Course.

No	Items of the Main Dimensions	Leftmost Value	Rightmost Value	Total Value	Ranking
W-1	Map reading and drawing	0.361	0.832	0.735	2
W-2	Operation preparation	0.418	0.783	0.683	8
W-3	Test material processing and combination	0.374	0.820	0.723	6
W-4	Welding construction	0.321	0.869	0.774	1
W-5	Weld bead slag removal	0.370	0.823	0.727	5
W-6	Welding inspection	0.366	0.828	0.731	3
W-7	Industrial safety and health	0.375	0.820	0.723	6
W-8	Professional ethics of welding personnel	0.368	0.826	0.729	4

, in order to independently develop the research tools and compile the expert questionnaire. The content of the questionnaire includes three parts: introduction to completing a questionnaire and the answering methods, basic information of respondents, and questionnaire title. Among these three parts, the questionnaire title is the research focus, which includes eight major dimensions, namely, map reading and drawing, operation preparation, test material processing and combinations, welding construction, weld bead slag removal, welding inspection, industrial safety and health, and professional ethics of welding personnel, for a total of 30 evaluation indices, as shown in

Table 4. Suitability Score Table of 30 Evaluation Indices of the Welding Course.

NO.	Main Dimension	Detailed Item of Evaluation Index	Leftmost Value	Rightmost Value	Total Value	Ranking	Total Ranking
W-1	Map reading and drawing	(1-1) Skill code and welding position	0.361	0.832	0.735	1	12
		(1-2) Welding construction drawing	0.361	0.832	0.735	1	12
W-2	Operation preparation	(2-1) Material preparation	0.425	0.776	0.676	1	27
		(2-2) Equipment preparation	0.427	0.774	0.674	2	28
		(2-3) Site preparation	0.464	0.739	0.638	3	30
W-3	Test material processing and combination	(3-1) Test material cleaning	0.381	0.814	0.716	1	16
		(3-2) Weld filing and grinding	0.429	0.772	0.672	4	29
		(3-3) Combination	0.383	0.812	0.714	2	20
		(3-4) False welding (temporary welding)	0.404	0.793	0.695	3	24
W-4	Welding construction	(4-1) Construction preparation	0.396	0.801	0.703	9	23
		(4-2) Current adjustment	0.326	0.865	0.769	1	1
		(4-3) Flat welding butt joint	0.418	0.797	0.690	10	25
		(4-4) Horizontal welding butt joint	0.372	0.826	0.727	8	15
		(4-5) Vertical welding butt joint	0.349	0.844	0.747	7	9
		(4-6) Overhead welding butt joint	0.341	0.850	0.754	3	4
		(4-7) Vertical fixed butt joint of pipe axis	0.341	0.850	0.754	3	4
		(4-8) Horizontal fixed butt joint of pipe axis	0.341	0.850	0.754	3	4
		(4-9) 45° fixed butt joint of pipe axis	0.334	0.857	0.762	2	3
		(4-10) Defect prevention and improvement	0.342	0.851	0.754	3	4
W-5	Weld bead slag removal	(5-1)Slag removal between weld beads and surface slag removal	0.344	0.848	0.752	1	8
W-6	Welding inspection	(6-1) Visual inspection	0.396	0.804	0.704	3	22
		(6-2) Mechanical inspection	0.379	0.816	0.719	2	17
		(6-3) Ray examination	0.374	0.820	0.723	1	16
W-7	Industrial safety and health	(7-1) Understand occupational safety and health regulations	0.419	0.779	0.680	4	26
		(7-2) Understand the protection of safety and health	0.381	0.814	0.716	3	18
		(7-3) Electric shock prevention and first aid	0.326	0.865	0.769	1	1
		(7-4) Disaster prevention and control	0.365	0.828	0.732	2	14
W-8	Professional ethics of welding personnel	(8-1) Professional dedication	0.351	0.841	0.745	1	10
		(8-2) Work environment maintenance	0.385	0.811	0.713	3	21
		(8-3) Professional quality	0.359	0.835	0.738	2	11

. Furthermore, a students’ learning feedback sheet and a learning satisfaction questionnaire were developed on the basis of these ability indices, and mixed quality analysis was conducted to explore the students’ learning status in the welding course. In addition, this study was supplemented with a scoring function built into the VR welding equipment to evaluate the performance of students’ welding ability.

3.5. Validity and Reliability

The validity of the research tools of the questionnaire was evaluated by adopting the content validity method. After reviewing relevant theories, research, and other literature, a preliminary draft of the questionnaire was constructed, and three experts with rich experience in welding and VR application were interviewed to obtain their opinions on the initial version, and their comments were used to amend and design the questionnaire. Therefore, this questionnaire has sufficient expert content validity.

The reliability of the research tools was also evaluated. The Cronbach’s α value of the reliability of the overall dimension is 0.739, and the reliability test results of each dimension are as follows: 0.752 for map reading and drawing, 0.721 for operation preparation, 0.712 for test material processing and combination, 0.764 for welding construction, 0.717 for weld bead slag removal, 0.725 for welding inspection, 0.731 for industrial safety and health, and 0.719 for professional ethics of welding personnel. Cronbach’s α values are all greater than 0.7, which indicates that the questionnaire has high reliability.

4. Results and Discussion

Given the purpose of this study and the results of the literature review, the ability indices, course content, and teaching activity design of the “VR welding course” for engineering students of the University of Science and Technology were constructed, and the expert questionnaire survey of the Fuzzy Delphi method and the ANP method were carried out. The analysis results are described in the following subsections.

4.1. Analysis of Fuzzy Delphi Method Expert Questionnaire

On the basis of the literature review regarding VR and welding knowledge, the “Fuzzy Delphi method expert questionnaire regarding the ability indices of the welding course” was developed for this study. Through focused interviews with three welding-related experts and scholars, the main structure of the appropriate composite-type ability indices of the VR welding course was obtained, as shown in Table 3, including eight main dimensions and 30 detailed evaluation indices were added (modified), as shown in Table 4.

4.1.1. Analysis of Suitability of the Ability Indices of the Main Dimensions

To obtain the evaluation values of experts and scholars for each question, 15 experts and scholars were invited to assign scores of subjective value judgment according to their welding knowledge, the current teaching situation, and the application of the practice course at the University of Science and Technology. According to the 10 valid questionnaires (the recovery rate of valid questionnaires is 66%), the Fuzzy Delphi method expert questionnaire survey and analysis results show that the suitability scores of the ability indices of the main dimensions range from 0.683 to 0.774, as shown in Table 3. The item with the highest suitability score is “W-4”, which scored 0.744, followed by “W-1” with a score of 0.735, while “W-6” scored 0.731, “W-8” scored 0.729, “W-5” scored 0.727, both “W-7” and “W-3” scored 0.723, and “W-2” scored 0.683. In conclusion, the results of the expert questionnaire are all higher than 0.6, and thus within the acceptable range. All of the ability indices of the main dimensions were evaluated by the research team and retained.

4.1.2. Analysis of the Suitability of Detailed Evaluation Indices

The suitability scores of the 30 evaluation indices of the welding course were analyzed, and the Fuzzy Delphi method expert questionnaire and analysis results show that the suitability scores of the evaluation indices range from 0.638 to 0.769, as shown in Table 4. The results of the expert questionnaire are all higher than 0.6, and thus within the acceptable range. The research team suggested that all the evaluation indices be retained, and the details are as follows.

In the dimension of “map reading and drawing”, the evaluation indices of “1-1” and “1-2” are the highest, with a score of 0.735, and thus equally important.

In the dimension of “operation preparation”, the evaluation index of “2-1” is the most suitable, with a score of 0.676, while the evaluation index of “2-2” is the second most suitable, with a score of 0.674, and the evaluation index of “2-3” is the third most suitable, with a score of 0.638.

In the dimension of “test material processing and combination”, with a score of 0.716, the evaluation index of “3-1” is the most suitable; the second most suitable is the evaluation index of “3-3”, with a score of 0.714; the third most suitable is the evaluation index of “3-4”, with a score of 0.695; and the evaluation index of “3-2” is the fourth most suitable, with a score of 0.672.

In the dimension of “welding construction”, the evaluation index of “4-2” is the most suitable, with a score of 0.769. The second most suitable is the evaluation index of “4-9”, with a score of 0.762. The evaluation indices of “4-6”, “4-7”, “4-8”, and “4-10” are the third most suitable, with a score of 0.754. The seventh most suitable is the evaluation index of “4-5”, with a score of 0.747; the eighth most suitable is the evaluation index of “4-4”, with a score of 0.727. The ninth most suitable is the evaluation index of “4-1”, with a score of 0.703; and the evaluation index of “4-3” is the tenth most suitable, with a score of 0.690.

In the dimension of “weld bead slag removal”, the evaluation index of “5-1” is the most suitable, with a score of 0.752.

In the dimension of “welding inspection”, the evaluation index of “6-3” is the most suitable, with a score of 0.723; the second most suitable is the evaluation index of “6-2”, with a score of 0.719; and the evaluation index of “6-1” is the third most suitable, with a score of 0.704.

In the dimension of “industrial safety and health”, the evaluation index of “7-3” is the most suitable, with a score of 0.769; the second most suitable is the evaluation index of “7-4”, with a score of 0.732; the third most suitable is the evaluation index of “7-2”, with a score of 0.716; and the evaluation index of “7-1” is the fourth most suitable, with a score of 0.680.

In the dimension of “professional ethics of welding personnel”, the evaluation index of “8-1” is the most suitable, with a score of 0.745; the second most suitable is the evaluation index of “8-3”, with a score of 0.738; and the evaluation index of “8-2” is the third most suitable, with a score of 0.713.

The overall suitability of the evaluation indices of the welding course was further assessed, as shown in Table 4. The first seven evaluation indices with the best overall suitability ranking are as follows: the evaluation index of “7-3” in the dimension of “industrial safety and health” and the evaluation index of “4-2” in the dimension of “welding construction” are the most suitable (0.769); “4-9” in the same dimension is the second most suitable (0.762); and “4-6”, “4-7”, “4-8”, and “4-10” in the same dimension are the third most suitable (0.754). The above assessment shows that the welding course not only teaches the students welding-related knowledge and skills but also attaches great importance to industrial safety and health issues, especially electric shock prevention advocacy and emergency treatment methods, as related to the physical safety of the students. The seven detailed evaluation indices were used as an important reference for the development of the course.

4.2. ANP Expert Questionnaire Survey and Analysis

In this study, the “VR welding course” applied VR technology to a welding practice course, which included three major characteristics: immersion, interaction, and imagination; the network structure was planned, as shown in Figure 3. The “VR-assisted welding course teaching ANP expert questionnaire” was developed, including two parts, i.e., weighted analysis of the feasibility of VR-assisted welding teaching and weighted analysis of the importance of the ability indices. The analysis results are described below.

4.2.1. ANP Weighted Analysis of the Feasibility of VR-Assisted Welding Teaching

To understand the feasibility of using VR technology for assisted welding course teaching, ANP was used to calculate the relative feasibility weight values of the eight ability indices of the welding course, and the dependence relations of the ability indices were compared in pairs. Super Decision software was used for ANP evaluation and analysis to obtain three supermatrices: the unweighted supermatrix (

Table 5. Unweighted Supermatrix of Feasibility of VR-Assisted Welding Course Teaching.

		VR			Welding
		V-1	V-2	V-3	W-1	W-2	W-3	W-4	W-5	W-6	W-7	W-8
VR	V-1	0.000	0.000	0.000	0.343	0.327	0.335	0.337	0.346	0.355	0.336	0.337
	V-2	0.000	0.000	0.000	0.346	0.350	0.353	0.341	0.352	0.331	0.336	0.320
	V-3	0.000	0.000	0.000	0.311	0.323	0.312	0.322	0.302	0.314	0.328	0.343
Welding	W-1	0.201	0.201	0.204	0.000	0.167	0.168	0.168	0.000	0.144	0.168	0.000
	W-2	0.000	0.195	0.198	0.188	0.000	0.156	0.000	0.157	0.134	0.156	0.157
	W-3	0.198	0.197	0.000	0.199	0.164	0.000	0.166	0.166	0.142	0.165	0.166
	W-4	0.211	0.210	0.213	0.213	0.175	0.177	0.000	0.177	0.152	0.177	0.178
	W-5	0.000	0.000	0.189	0.200	0.165	0.166	0.166	0.000	0.143	0.000	0.167
	W-6	0.197	0.197	0.000	0.201	0.166	0.167	0.167	0.168	0.000	0.167	0.168
	W-7	0.193	0.000	0.196	0.000	0.164	0.165	0.166	0.166	0.142	0.000	0.166
	W-8	0.000	0.000	0.000	0.000	0.000	0.000	0.167	0.167	0.143	0.167	0.000

), the weighted supermatrix (Appendix

Table A1. The weighted supermatrix of the feasibility.

		VR			Welding
		V-1	V-2	V-3	W-1	W-2	W-3	W-4	W-5	W-6	W-7	W-8
VR	V-1	0.000	0.000	0.000	0.178	0.170	0.174	0.175	0.180	0.184	0.175	0.175
	V-2	0.000	0.000	0.000	0.179	0.181	0.183	0.177	0.183	0.172	0.175	0.166
	V-3	0.000	0.000	0.000	0.162	0.168	0.162	0.167	0.157	0.163	0.170	0.178
Welding	W-1	0.201	0.201	0.204	0.000	0.080	0.081	0.081	0.000	0.069	0.081	0.000
	W-2	0.000	0.195	0.198	0.090	0.000	0.075	0.000	0.075	0.064	0.075	0.075
	W-3	0.198	0.197	0.000	0.096	0.079	0.000	0.080	0.080	0.068	0.079	0.080
	W-4	0.211	0.210	0.213	0.102	0.084	0.085	0.000	0.085	0.073	0.085	0.085
	W-5	0.000	0.000	0.189	0.096	0.079	0.080	0.080	0.000	0.069	0.000	0.080
	W-6	0.197	0.197	0.000	0.097	0.080	0.080	0.080	0.081	0.000	0.080	0.081
	W-7	0.193	0.000	0.196	0.000	0.079	0.079	0.080	0.080	0.068	0.000	0.080
	W-8	0.000	0.000	0.000	0.000	0.000	0.000	0.080	0.080	0.069	0.080	0.000

), and the limited supermatrix (summary shown in

Table 6. Summary of the limited supermatrix of Feasibility and Importance.

No.	Item	Feasibility of VR-Assisted Welding Teaching			Importance of Index
		Weighted Score	Standard Deviation	Ranking	Weighted Score	Ranking
W-1	Map reading and drawing	0.105	0.160	2	0.129	4
W-2	Operation preparation	0.079	0.121	6	0.107	6
W-3	Test material processing and combination	0.092	0.139	4	0.134	3
W-4	Welding construction	0.119	0.180	1	0.165	1
W-5	Weld bead slag removal	0.062	0.095	7	0.087	8
W-6	Welding inspection	0.093	0.141	3	0.157	2
W-7	Industrial safety and health	0.080	0.122	5	0.128	5
W-8	Professional ethics of welding personnel	0.027	0.042	8	0.093	7
V-1	Immersion	0.116	0.340	2	N/A
V-2	Interaction	0.117	0.343	1
V-3	Imagination	0.109	0.318	3

). The relative weight values of the feasibility of the ability indices of the welding course were ranked, as shown in Table 6; the higher the weight value, the higher the feasibility of VR-assisted welding course teaching. After standardization, the item with the highest relative weighted score of feasibility is “W-4” with a score of 0.180, followed by “W-1” with a score of 0.160, “W-6” with a score of 0.141, “W-3” with a score of 0.139, “W-7” with a score of 0.122, “W-2” with a score of 0.121, “W-5” with a score of 0.095, and “W-8” with a score of 0.042. These results show that the application of VR technology for assisted welding course teaching can be planned according to welding construction items, including flat welding, horizontal welding, vertical welding, overhead welding, pipe axis butt joint, and other welding methods.

In terms of VR technology application, the item with the highest relative weighted score of feasibility after standardization is “interaction” with a score of 0.343, followed by “immersion” with a score of 0.340, and “imagination” with a score of 0.318. These results show that VR welding course planning can focus on the interaction between VR and learners and their immersion in the welding field.

4.2.2. ANP Weighted Analysis of the Importance of the Ability Indices of the Welding Course

To understand the importance degree of the ability indices of the welding course, ANP was used to calculate the relative feasibility of the weight values of eight ability indices, and the dependence relationships of these ability indices were compared in pairs. Super Decision software was used for ANP evaluation and analysis to obtain the three supermatrices: the unweighted supermatrix (Appendix

Table A2. The unweighted supermatrix of the importance.

		Welding
		W-1	W-2	W-3	W-4	W-5	W-6	W-7	W-8
Welding	W-1	0.205	0.153	0.183	0.208	0.000	0.176	0.000	0.000
	W-2	0.183	0.000	0.154	0.000	0.188	0.000	0.188	0.235
	W-3	0.195	0.139	0.000	0.158	0.199	0.163	0.199	0.000
	W-4	0.218	0.150	0.174	0.000	0.213	0.174	0.213	0.266
	W-5	0.000	0.137	0.163	0.155	0.000	0.161	0.000	0.000
	W-6	0.199	0.142	0.164	0.163	0.201	0.000	0.201	0.251
	W-7	0.000	0.138	0.163	0.159	0.199	0.161	0.000	0.248
	W-8	0.000	0.140	0.000	0.157	0.000	0.165	0.200	0.000

), the weighted supermatrix (Appendix

Table A3. The weighted supermatrix of the importance.

		Welding
		W-1	W-2	W-3	W-4	W-5	W-6	W-7	W-8
Welding	W-1	0.205	0.153	0.183	0.208	0.000	0.176	0.000	0.000
	W-2	0.183	0.000	0.154	0.000	0.188	0.000	0.188	0.235
	W-3	0.195	0.139	0.000	0.158	0.199	0.163	0.199	0.000
	W-4	0.218	0.150	0.174	0.000	0.213	0.174	0.213	0.266
	W-5	0.000	0.137	0.163	0.155	0.000	0.161	0.000	0.000
	W-6	0.199	0.142	0.164	0.163	0.201	0.000	0.201	0.251
	W-7	0.000	0.138	0.163	0.159	0.199	0.161	0.000	0.248
	W-8	0.000	0.140	0.000	0.157	0.000	0.165	0.200	0.000

), and the limited supermatrix (summary shown in Table 6). The relative weight values of the importance of the ability indices of the welding course were ranked, as shown in Table 6, where the higher the weight value, the greater the importance of the ability index. The item with the highest relative weighted score of feasibility after standardization is “W-4” with a score of 0.165, followed by “W-6” with a score of 0.157, “W-3” with a score of 0.134, “W-1” with a score of 0.129, “W-7” with a score of 0.128, “W-2” with a score of 0.107, “W-8” with a score of 0.093, and “W-5” with a score of 0.087.

4.3. Planning and Development of the VR Welding Course

The above analysis results of a questionnaire of experts were regarded as the basis for designing the VR welding course. The course planning and development are set out in the following subsections.

4.3.1. Comprehensive Analysis of Ability Indices in the Welding Course

The comprehensive analysis method used by Chung, Chao, and Lou [38] was used in this study. The importance and feasibility of the welding course’s ability indices were comprehensively compared, as shown in Figure 4, and are important grounds for the planning and development of a VR welding course. The comprehensive analysis is presented below.

(1)

First quadrant: priority planning

As can be observed from the items distributed in the first quadrant, the ability indices that should be prioritized in planning the course are W-1, W-3, W-4, and W-6. Among them, the ability of “W-4 Welding construction” is the most important and highly feasible, so many related courses are planned. More specifically, the “unit of VR welding practice” is developed in order to improve the exactness and stability of students’ basic welding actions. In this way, students can prepare themselves for the advanced practice course of welding application.

Regarding the “W-3 Test material processing and combination”, the course plans to match it with the advanced welding course (after the mid-term exam) in order to cultivate students’ welding application ability of material processing and combination.

Regarding the “W-1 Map reading and drawing”, students must arm themselves with a basic map reading ability so that they can finish welding correctly and smoothly. In principle, priority should be given to compulsory courses of the school, including “engineering graphics” and “mechanical drawing”, while this course is for conceptual teaching and application only.

“W-6 Welding inspection” concentrates on the visual inspection of weld beads so as to furnish students with the ability to validate the quality of weld beads on the front line. The development of more advanced knowledge and ability concerning weld beads can be facilitated by encouraging students to take optional courses on nondestructive testing offered by the school, for example, liquid penetrant testing (PT), magnetic particle testing (MT), radiographic testing (RT), and ultrasonic testing (UT).

(2)

Second quadrant: feasibility planning

No items of the ability index are distributed in the second quadrant.

(3)

Third quadrant: importance planning

As can be observed from the items distributed in the third quadrant, the ability indices of additional planning in the course are W-2, W-5, W-7, and W-8. Among them, the ability of “W-7 Industrial safety and health” is the most important and feasible (close to the third quadrant). The majority of experts maintained that it was important to impart knowledge about welding safety to students when developing their welding ability. Furthermore, a “unit for safety and health in and operation preparations for VR welding” can be developed in combination with the “W-2 Operation preparation”, “W-5 Weld bead slag removal”, and “W-8 Professional ethics of welding personnel”. VR technology can be used to assist operation preparations for welding and to simulate the correct operation steps. Moreover, students are asked to implement such preparations and steps in each welding exercise so that they can adopt the correct job attitude and professional ethics.

(4)

Fourth quadrant: additional planning

No items of the ability index are distributed in the fourth quadrant.

4.3.2. VR Welding Course Development

As an important reference for the development of the “VR welding course” in this study, and according to the analysis results of the first two sections, the core of the course design is “student-centered”, as shown in

Table 7. VR Welding Course Schedule.

Week No.	Course Outline	VR Integration
Weekly	Site arrangement, including turning off the power supply to the machines; maintaining and returning tools; cleaning up workbenches, waste materials, and the environment; using the equipment carefully; and respecting public property	***
Weeks 1–2	1. Briefly introduce AC welding machines, as well as the functions and comparison of AC welding machines 2. Briefly introduce the numbering principle of CNS mild steel electrodes and the application of mild steel electrodes 3. Introduce the selection of personal appliances and safety lens for welding operation 4. Explain the safety rules of welding operation and introduce relevant equipment in the welding field	VR-assisted welding safety simulation
Weeks 3–4	1. Protective equipment and correct welding postures (including electrode chuck holding and electrode holding methods) 2. Welding machine operation, welding current adjustment, arc initiation method, and straight welding for flat welding 3. Arc welding damage and protective measures and electric shock hazard and prevention methods 4. Demonstration explanation and practice	VR-assisted welding skills and stability
Weeks 5–8	1. Appropriate welding arc length 2. Angle between base material and electrode during welding 3. Key points of flat welding, horizontal welding, vertical welding, and overhead welding 4. Demonstration explanation and practice	VR-assisted welding skills and stability
Week 9	Midterm exam: 1. Review the finished products; teachers and students evaluate them together 2. Answer students’ questions and summarize the key teaching points of this course	VR-assisted simulated welding test
Weeks 10–17	1. Types and causes of weld bead defects 2. Prevention and improvement of weld bead defects Operation practice (teacher provides individual guidance and corrects students’ welding postures): 1. Strengthen the practice of welding actions, as demonstrated by the teacher 2. Complete the works on the work list	VR-assisted welding remedial teaching
Week 18	Midterm exam: 1. Review the finished products; teachers and students evaluate them together 2. Answer students’ questions and summarize the key teaching points of this course	VR-assisted simulated welding test

. Then, with reference to the analysis results in Section 4.2.1 and Section 4.3.1 regarding the importance of the ability indices of the welding course, a proportion of the traditional 18-week welding practice course content was adjusted to place emphasis on the planning of the “welding construction” unit in order to improve students’ abilities according to the indices of the welding course. Moreover, by referring to the feasibility analysis of VR-assisted welding teaching in Section 4.2.2 and Section 4.3.1, VR technology was integrated and applied to the welding course units, which mainly focused on the “welding construction” and “industrial safety and hygiene” units, in order to provide students with diversified learning stimulation in the assisted teaching activity design. The course contents include weekly site arrangement, a basic welding knowledge course in weeks 1–2, a basic welding practice course in weeks 3–4, an advanced welding practice course in weeks 5–8, a welding project course in weeks 10–17, a midterm exam in week 9, and a final exam in week 18. During the implementation of the course, VR welding equipment simulation teaching was combined according to the unit attributes of each week, including VR-assisted welding safety simulation, VR-assisted welding skills and stability, VR-assisted simulated welding test, and VR-assisted welding remedial teaching.

4.4. Analysis of VR Welding Course Development and Implementation Effect

Feedback from students was collected, and a questionnaire survey was carried out during the 18-week experimental teaching. Analyses of the effects of and feedback from students are detailed in the following subsections.

4.4.1. Analysis of Learning Effects of Students Taking the VR Welding Course

After the 18 weeks of the VR integrated welding practice course, the learning and implementation effects of students who took the course were analyzed. The analysis consisted of the scales of “welding ability indices” and the “VR equipment-assisted learning satisfaction”, which are presented below.

(1)

Analysis of the demonstration of students’ welding ability

The results of the questionnaire survey were analyzed using the one-sample t-test, and the verification value was 3. The results are shown in

Table 8. Analysis of the demonstration of students’ welding ability.

No.	Dimension	Mean Value	Standard Deviation	t	Ranking
W-1	Map reading and drawing	3.87	0.75	7.16 ***	8
W-2	Operation preparation	3.99	0.80	7.64 ***	3
W-3	Test material processing and combination	3.90	0.84	6.62 ***	7
W-4	Welding construction	3.95	0.77	7.61 ***	4
W-5	Weld bead slag removal	3.91	0.73	7.63 ***	5
W-6	Welding inspection	3.91	0.80	7.01 ***	5
W-7	Industrial safety and health	4.05	0.78	8.27 ***	2
W-8	Professional ethics of welding personnel	4.18	0.75	9.63 ***	1

Verification Value = 3; *** p-value < 0.000.

. Specifically, the students’ average scores of welding ability indices ranged from 3.87 to 4.18, which were all positively and significantly different (p-value < 0.001), indicating that most students had positive enhancement effects in the performance of the 8 principal welding ability indices after this “VR welding course”. Among these indices, the score of “W-8” was the highest, with an average score of 4.18 (SD = 0.75) and t-value of 9.63. The average score of “W-7” was the second highest, at 4.05 (SD = 0.78) with a t-value of 8.27; the average score of “W-2” was 3.99 (SD = 0.80) with a t-value of 7.64. In addition, the students’ average score of “W-4” reached 3.95 (SD = 0.77) with a t-value of 7.61; “W-5” received a score of 3.91 (SD = 0.73) and t-value of 7.63; “W-6” received a score of 3.91 (SD = 0.80) and t-value of 7.01; “W-3” received a score of 3.90 (SD = 0.84) and t-value of 6.62; “W-1” received a score of 3.87 (SD = 0.75) and t-value of 7.16.

(2)

Scores of VR equipment-assisted learning satisfaction

After 18 weeks of the VR integrated welding practice course, the 34 students participating in the course were asked to complete the “VR equipment-assisted learning satisfaction” questionnaire in order to understand their feelings regarding the teaching of the VR-assisted welding course and its influence. The results of the questionnaire were analyzed with single-sample t-testing, as shown in

Table 9. Satisfaction analysis of VR equipment-assisted learning.

Dimension	Mean Value	Standard Deviation	t
VR Experience Feeling	4.42	0.51	13.89 ***
VR Equipment Use Interface and Operation Experience	4.23	0.58	10.63 ***
VR Equipment Advantages	4.41	0.46	15.26 ***
VR Equipment Use Satisfaction	4.45	0.57	12.66 ***

Verification Value = 3; *** p-value < 0.000.

, where the scores of each dimension are between 4.23 and 4.45 and in the following order: “VR equipment use satisfaction”, “VR experience feeling”, “VR equipment advantages”, and “VR equipment use interface and operation experience”. These results are detailed below.

The score of the dimension of “VR experience feeling” is 4.42 with a t-value of 13.89, and the individual scores of the 5 items range from 4.24 to 4.56, which are all significantly different, indicating that most students expressed positive feedback for the VR experiential feeling. Among the five items, item 4, “Using VR equipment for operation learning has better learning effect than traditional learning”, received the highest score of 4.56 and a t-value of 15.40. Followed by item 1, “Using VR equipment can help to simulate a factory and other field spaces”, which received a score of 4.48 and t-value of 12.63.

The dimension score of “VR equipment usage interface and operational experience” is 4.23, the t-value is 10.63, and the individual scores of the five items range from 4.08 to 4.36, which are all significantly different, indicating that most students expressed positive feedback for the VR equipment usage interface and operational experience. Among the five items, item 4, “I think the Chinese and English language patterns of VR interactive context development software meet the needs”, received the highest score of 4.36 and a t-value of 8.39. Followed by item 5, “I think the VR interactive context development software supports cross-platform collaboration with high compatibility and stability”, which received a score of 4.32 and t-value of 9.56.

The dimension score of “VR equipment advantages” is 4.41, the t-value is 15.26, and the individual scores of the seven items range from 4.32 to 4.48, which are all significantly different, indicating that most students expressed positive feedback for VR equipment advantages. Among the seven items, item 2, “Using VR equipment in the practice courses allows me to experience the environment and practice repeatedly”, and item 4, “Using VR equipment virtual simulation technology can provide users with exquisite entertainment games”, received the highest scores of 4.48 and t-values of 12.63 and 11.33, respectively.

The dimension score of “VR equipment use satisfaction” is 4.45, the t-value is 12.66, and the individual scores of the 4 items range from 4.36 to 4.56, which are all significantly different, indicating that most students expressed positive feedback for the VR experiential feeling. Among the four items, item 3, “I support the development of VR equipment assisted course learning and course teaching”, received the highest score of 4.56, and the t-value is 13.38. Followed by item 2, “I would like to recommend teachers to use VR equipment to assist teaching”, and item 4, “I think the scene layout and design of VR equipment are very realistic”, which received scores of 4.44 and t-values of 11.07.

4.4.2. VR Welding Course Implementation and Student Feedback

In terms of the implementation of the VR welding course, in order to cultivate professional ethics and quality, students were required to complete the arrangement and cleaning of the internship site every week. The basic welding knowledge course in weeks 1–2 mainly introduced the basic knowledge, functions, and operational precautions of welding-related equipment. During this course, the implementation of safety and health practices related to welding must be emphasized, as shown in Figure 5 and Figure 6, as this index is easily ignored by students. S0203: “In the first welding course, the teacher taught us a lot of knowledge about welding-related equipment, safety regulations for welding operation, and relevant protective measures, and required us to comply with them.”

In week 2, this course applied VR technology to present the safety rules related to welding in a virtual manner, such as wearing protective clothing, gloves, and masks and confirming the automatic electric shock prevention device, so that students could experience and implement the requirements one by one and teachers could design the clearance test. S0401: “The teacher asked us to play VR games to experience the correct safety measures before welding operations. If we did not wear protective gloves according to the regulations, we would be scalded (as shown in Figure 7); the right tool, meaning a slag hammer, should be used to remove slag from the weld bead (as shown in Figure 8).” Through VR experience, the students improved their interest in learning and understood the correct implementation procedures of welding operations [9].

Weeks 3–4 demonstrated the basic welding practice course. Because the welding equipment was limited, students were divided into groups, which took turns practicing welding in sequence after the demonstration of the teaching assistant. As most students were exposed to welding for the first time during the course, the sparks, arc lights, toxic gases, and noises produced during welding often caused fear, rejection, and resistance. In addition, as both arcing and welding operations require high skill and stability, in the first welding experience, most students received negative evaluations, as they resulted in wasted costs, such as electrodes and steel, as shown in Figure 9. S0301: “In the first practical welding, although there was a demonstration by a teaching assistant, I was still afraid of welding by myself. The weld bead was crooked, and the welding rod was often stuck.” Therefore, this course designed VR welding construction unit experience activities before students’ actual welding practice so that students could immerse themselves in the VR welding operation environment with sound and light, as shown in Figure 10, where the key aim was to make students feel safe. S0602: “VR welding equipment enabled us to experience welding operations and get familiar with welding steps and postures. The most important thing was that we could practice repeatedly without fear of injury or damage to the equipment.” Furthermore, the practice mode of VR welding equipment has a prompt function, as represented by the symbol “+” and shown in Figure 11 and Figure 12, which can guide students’ posture, welding angle, welding walking speed, etc. Moreover, it can allow students to repeatedly experience welding operations and learn correct welding skills and stability [12].

The advanced welding practice course in weeks 5–8 included key welding points, such as horizontal welding, vertical welding, and overhead welding. After basic practice, as conducted in the first two weeks, most students had acquired basic arcing skills and welding stability and thus could advance to learning different welding positions. Before course implementation, as preparation for the subsequent implementation of actual welding, the VR welding course designed VR simulations of different welding positions to provide students with preliminary practice and welding experience in different welding positions. S0503: “In addition to flat welding and horizontal welding, VR welding equipment can also set plate thickness, so that we can practice a variety of welding modes. In the actual welding, practice makes perfect, and we can weld out good works (as shown in Figure 13).”

Week 9 was the midterm exam week, during which teachers assessed students’ welding work results, reviewed them according to the weld bead quality, as shown in Figure 14, and offered suggestions and answered students’ questions. S0104: “In addition to the practice mode, VR welding equipment also has a test mode, which can test the stability, walking speed, and welding angle of our welds.” For the students who failed to meet the examination standard, this course offered welding remedial teaching and provided extra exercises and tests matched with VR welding equipment to help students improve their welding skills [19].

Weeks 10–17 focused on the welding project application course, during which students completed the works on the work list according to the teachers’ requirements, including cutting steel according to the size and assembling and welding according to the design drawing. The final project required the student groups to design products themselves, construct them according to their drawings, and complete the finished products with their welding skills.

Week 18 was the final exam week, during which teachers assessed students’ finished welding products, as shown in Figure 15 and Figure 16. The teachers evaluated the finished products according to their composition construction difficulty, welding defects, and weld bead quality and offered suggestions to strengthen students’ welding ability.

4.5. General Discussion

The discussions presented below are based on the above qualitative and quantitative analysis results.

4.5.1. The Student-Centered VR Integrated Welding Training Course Design Can Provide Students with a Safe and Efficient Learning Environment

The 15 experts from industry and academic circles evaluated the suitability of the “ability indices of the VR welding course” constructed using Fuzzy Delphi, which comprised 8 ability indices and 30 evaluation indices. Moreover, ANP expert questionnaires were employed for comprehensive comparison and analysis of the “relative importance” and the “relative feasibility of VR-assisted teaching” of the ability indices to develop the student-centered “VR welding course”. The planning of the “VR welding course” consisted of two major units, “welding construction” and “industrial safety and health”, and employed a course that integrated VR with the practical operation. The VR course, therefore, enables students to learn welding skills in a safe and secure environment and to familiarize themselves with the correct operation steps and matters that need attention. Next, hands-on welding exercises were conducted, and the welding postures and stability were corrected through VR integrated interactive exercise and comparison. In this way, a student can learn effectively, and the consumption of materials can be reduced [31]. The approach to running the welding course can be adjusted flexibly to offer a better, safer learning environment [39].

4.5.2. VR Integrated Welding Training Facilitates Students’ Learning and Demonstration of Welding Ability

After 18 weeks of the practice course, most of the 34 students expressed positive responses for the demonstration of welding ability. In particular, they had the best performance in industrial safety and health, professional ethics, and operation preparation among the standard operation procedures. Moreover, students also thought highly of the learning effects of hands-on welding [33]. The feedback data on VR welding simulation testing, including arc length, position, work angle, travel angle, and travel speed, can provide students with appropriate suggestions on welding. Additionally, they unanimously approved of the integration of VR technology into early welding teaching, which could boost the learning effects [32].

4.5.3. Students’ High Acceptance of Integrating VR Technology into the Teaching of the Welding Course

In sum, after completing the VR welding course developed in this study, most of the students generally considered VR-assisted welding teaching to be helpful for simulating a welding factory, ensuring the safety and environmental protection of personnel, providing a good user interface and interaction, and accurately grasping the learning situation of students [13]. In addition, it can provide the functions of game-style practice and repeated learning to improve students’ learning efficiency of welding skills. Finally, the students expressed positive responses in support of the development of VR equipment-assisted course teaching and recommended its use by teachers. The results illustrate that most students readily accept the application of VR welding simulation to a training course for welding professionals, which will become a trend in the coming years [32].

4.5.4. High Interaction of the VR Welding Course Effectively Enhanced Students Welding Ability

The analysis results of the expert questionnaire of this study showed that “Interaction” was most feasible for the VR (virtual reality) technology to assist in welding teaching. As seen from the feedback of most students, the virtual welding equipment has the function of prompting the welding speed, the distance between the electrode and the plate, and the welding angle (Front, Back, Left, Right), which can facilitate students to self-examine, familiarize themselves with welding standard actions, and increase the stability of welding quality. It shows that the interactive nature of VR technology can enhance students’ learning interest and learning effectiveness of welding ability [25,39], matching the results of the questionnaire survey. Furthermore, teachers and students can review the actual learning of students’ welding skills and give objective and substantive suggestions via real-time performance feedback of the virtual welding system [32]. In addition, for those students who have failed to pass their mid-term exams, this virtual welding practice course combined with real welding equipment (which implements remedial teaching and increases repeated practice opportunities) can help freshman students acquire basic welding skills [33].

5. Conclusions and Suggestions

In this study, the importance and feasibility of the ability indices of VR technology-assisted teaching of welding were evaluated using Fuzzy Delphi and an ANP expert questionnaire in order to develop a VR welding course and teaching activity design and implement experimental teaching to verify its effectiveness. The conclusions and suggestions are provided below.

5.1. Conclusions

5.1.1. The VR Welding Course Includes 8 Ability Indices and 30 Evaluation Indices

According to the results of the Fuzzy Delphi expert questionnaire, the ability indices of this VR welding course are map reading and drawing, operation preparation, test material processing and combinations, welding construction, weld bead slag removal, welding inspection, industrial safety and health, and professional ethics of welding personnel. Among these, in order of suitability, the top three indices scores are “welding construction”, “map reading and drawing”, and “welding inspection”. In terms of the suitability of the 30 evaluation indices, “skill code and welding position” and “welding construction drawing” scored highest in the ability index of “map reading and drawing”; “material preparation” is the most suitable evaluation index for the ability index of “operation preparation”; “test material cleaning” is the most suitable evaluation index for the ability index of “test material processing and combination”; “current adjustment” is the most suitable evaluation index for the ability index of “welding construction”; the evaluation index “slag removal between weld beads and surface slag removal” is suitable for the ability index of “weld bead slag removal”; “ray examination” is the most suitable evaluation index for the ability index of “welding inspection”; “electric shock prevention and first aid” is the most suitable evaluation index for the ability index of “industrial safety and health”; “professional dedication” is the most suitable evaluation index for the ability index of “professional ethics of welding personnel”.

5.1.2. Assisted “Welding Construction” Ability Index Teaching Is Best for the Relative Feasibility of ANP of VR-Assisted Welding Teaching

According to the analysis of the ANP expert questionnaire, in terms of the relative feasibility of VR technology-assisted welding course teaching, the ability index of “welding construction” has the highest feasibility among the eight ability indices, followed by “map reading and drawing”, “welding inspection”, “test material processing and combination”, and “industrial safety and health”, which shows that welding construction has the highest applicability in VR-assisted welding teaching.

5.1.3. Best Feasibility of VR Technology “Interaction” in Assisting Welding Teaching

In terms of the VR technology application as concluded from the ANP expert questionnaire analysis, “Interaction” is the most feasible, followed by “Immersion” and “Imagination”. Focusing on the characteristics of “Interaction” supplemented by “Immersion” and “Imagination”, this VR welding course is able to design virtual welding teaching activities, including teaching of industrial safety and hygiene, virtual welding prompt function, and virtual welding performance feedback.

5.1.4. The Ability Index of “Welding Construction” Is Most Important for the Relative Importance of ANP of the Ability Indices of the VR Welding Course

According to the analysis of the ANP expert questionnaire, in terms of the relative importance of the ability indices of the VR welding course, “welding construction” has the highest feasibility among the eight ability indices, followed by “welding inspection”, “test material processing and combination”, “map reading and drawing”, and “industrial safety and health”, which shows the importance of welding construction in VR welding course planning and design.

5.1.5. The VR Welding Course Allows Students to Express Significant Positive Responses to the Learning of Ability Indices and Ability Demonstration

In this study, a VR welding course was developed, and the two units of “welding construction” and “industrial safety and health” were designed on the basis of the comprehensive analysis results of ability indices. After an 18-week VR and hands-on operation course of welding, most students expressed significant positive responses in the analysis results of the learning effects of ability indices. Specifically, the top five ability demonstrations are “professional ethics of welding personnel”, “industrial safety and health”, “operation preparation”, “welding construction”, and “weld bead slag removal”, roughly consistent with the design and planning of the course. Accordingly, the course benefits students’ learning of welding ability.

5.1.6. The Majority of Students Expressed Significant Positive Feedback for Learning Satisfaction with VR-assisted Welding Course Teaching

In this study, the importance analysis results for the ability indices of the welding course and the feasibility analysis results of VR-assisted welding teaching were integrated in an ANP expert questionnaire, and an 18-week “student-centered” VR welding course was developed. The course content included site arrangement, a basic welding knowledge course, a basic welding practice course, an advanced welding practice course, a welding project application course, a midterm exam, and a final exam. During the implementation of the course, VR welding equipment simulation teaching was appropriately integrated according to unit attributes, and questionnaires regarding students’ learning feedback and learning satisfaction were collected. The results are positive with significant differences, which shows that most students think that VR-assisted welding course teaching can accurately meet their learning requirements and improve their learning efficiency.

5.2. Suggestions

On the basis of the above research conclusions, the following suggestions are proposed for schools, teachers, and future research.

5.2.1. Take the Importance and Feasibility of the Ability Indices Summarized in this Study as an Important Reference for VR Welding Course Planning

The conclusions of this study propose eight ability indices and 30 evaluation indices of the welding course and discuss their relative importance and feasibility. According to Fuzzy Delphi, ANP expert interviews, questionnaire surveys, and other empirical research results, the results of this study have reference value. It is suggested that the University of Science and Technology take these results as a blueprint for planning and implementing a welding practice course and carry out course content planning and teaching activity design according to the most important indices in order to meet the training needs of students and industrial talent.

5.2.2. Train Teachers in the Application of VR Knowledge in Assisting Practical Teaching

The future teaching trend is the integration of emerging and related technologies. According to the findings from the VR technology-assisted welding course in this study, the index of welding construction is the most important among the ability indices of the VR welding course and has the highest feasibility in VR-assisted teaching. From the perspective of students’ satisfaction with learning results, in addition to the improved learning results, VR-assisted training can improve personal safety and reduces the cost of consumables compared with traditional teaching. Therefore, the innovation of practical teaching in the University of Science and Technology must start with changing how teachers think and improving their abilities. To lay a foundation for improving the effectiveness of innovative teaching, teachers should be encouraged to participate in more innovative teaching practice workshops or forums to increase their knowledge and ability related to emerging technologies.

5.2.3. Discuss the Experimental Teaching of Related Research Topics

As VR technology is applied to the teaching of practical skills, which is one of the important issues in the application of emerging technologies in education, it is suggested to refer to the results of this study, apply them to the welding course, plan experimental and control groups, and compare the effects of different teaching methods on students’ learning results. By referencing the research process of developing the VR-assisted welding practice course in this study, VR technology can be applied to other practice courses to explore the effectiveness of VR-assisted teaching.