Construction and Application of VR-AR Teaching System in Coal-Based Energy Education

Abstract

Coal-based energy has provided strong support and made outstanding contributions in the process of China’s economic development. Coal mining in China has gradually developed into intelligent, refined and green mining. However, due to the lack of effective science popularization and propaganda in coal mining for a long time, people’s understanding of coal mining often stays in the stereotype of dirty, messy and very dangerous. Based on this fact, this paper firstly discusses the difficulties and pain points of the popularization of science in coal mining based on the questionnaire survey. And then a VR-AR system for intelligent coal mining was developed. Finally, popular science teaching activities based on VR-AR system were carried out during the “Open Day” activity and “Entering Campus” activity. It is found that the long-term negative reports of coal mining and the complexity of coal mining system make the science popularization and propaganda in coal mining less effective. The proportion of primary and secondary school students with bad impression reached 85.0% and 90.3%, respectively, and 63.1% for college students. With our VR-AR system in coal-based energy education, the impression of the coal industry has increased significantly, the proportion of bad impression decreased to 23.4%. This helps to form the nationwide coal mining science popularization and justifies China’s coal mining.

1. Introduction

Since the 18th century, with the invention and use of steam engine, coal has been widely used as “industrial food”, which has greatly promoted the progress of human civilization [1]. China is the world’s second largest economy, as well as the world’s largest producer and consumer of coal according to bp (British Petroleum)’s Statistical Review of World Energy (2022). Coal plays an important role in maintaining China’s energy security and long-term stable supply [2,3,4]. According to bp’s Statistical Review of World Energy (2022) [5], in 2021, coal accounted for 75% of China’s energy production and 56.0% of its energy consumption. The coal production exceeded 4 billion tons for the first time in 2021. Figure 1 shows China’s energy consumption structure and coal production in recent 10 years. The “coal–dominated” energy structure determined by China’s resource endowment will not change in the short term [6,7].

In the process of China’s economic development, as a strong support of energy supply, coal has made remarkable contributions. From 2000 to 2013, the average annual growth rate of coal was over 10%, which supported the development of China’s economy with an average annual growth rate of over 9% [8]. Since the new round of economic adjustment in China in 2014, the coal quantity has changed from the long-term insufficient total quantity to the stage of rapid construction to adapt to economic development to the stage of relatively loose total quantity [9]. Since the 21st century, the research and development of safe and efficient high-end mining technology and equipment in China has made rapid progress. The total number of coal mines in China has decreased from 25,000 in 2005 to 5000 in 2020, reducing by 80% [10]. Coal production increased from 1.09 billion tons in 2001 to 4.13 billion tons in 2021, rising by 3.79 times. The coal field vigorously promote intelligent efficient safe mining, clean utilization and conversion, which realizes the safe high-efficient exploitation of coal and the transformation of the clean coal technology, also forms a series of transformational coal mining technology such as the intelligent mining technology, precision drilling technology, the green mining technology, water conservation mining technology, etc. 5G, artificial intelligence, internet of things, cloud computing and other new technologies were used in coal mining [11,12,13,14,15,16,17,18]. A large number of important achievements have been made, greatly promote happening in the scale, mechanization and automation of the coal industry [19,20,21,22], and significantly reducing the total number of coal mine accidents, major accidents and the death rate per million tons. For example, the death toll of coal mine accidents dropped from 1384 in 2012 to 179 in 2021, decreasing by 87% in recent ten years, as shown in Figure 2, which statistics from the China coal safety net accessed on 15 January 2022 [23]. (https://www.mkaq.org/html/2022/01/15/606203.shtml). With the transformation of China’s economic development mode, the coal industry has been vigorously transformed from extensive production mode to intensive, refined and intelligent direction.

However, because of the lack of effective science popularization and propaganda, people often think coal-based energy mining and using are extensive, scientific and technological backward and very dangerous [24,25,26]. Meantime, the surface subsidence, groundwater damage, casualties, caused by coal mining further make people believe that coal mining is a sunset industry, backward industry and dirty industry [27,28,29,30]. However, the environmental and safety problems caused by coal mining are worldwide problems, various countries have done a vast of work in this regard [31,32,33,34]. In particular, primary and secondary school students’ scientific understanding of coal only focuses on the formation and simple utilization of coal, and lack of understanding and recognition of the new technology, new development and ideas in the safe mining of coal. Moreover, some we media chasing hot spots (environmental pollution, casualty accident) and irresponsible reports have exacerbated the public’s negative impression of the coal industry. Thus, it is necessary to carry out popular science activities in coal mining to change the negative impression of the public.

Most of the objects for the teaching and popular science activities on coal-based energy industry are the coal mine workers and the relevant staff in the coal-based field, and the students of this major. However, there is a lack of teaching and popular science activities for primary and secondary school students and non-professional public. The existing teaching and popular science activities for those school students only focus on the display of teaching models and scientific research instruments, the animation demonstration of underground coal mining and equipment as well. In such popular science activities, due to the lack of modern technology immersion experience system, it is difficult for primary and secondary school students to feel real scenes of intelligent mining. And it is more difficult to understand the relevant operation process of coal mining. At the same time, considering the safety of students, it is difficult to truly feel the disasters in the process of underground mining of coal mines (such as roof falling, gas, water inrush, ignition, rock burst, etc.), which results in the understanding of coal mine disasters only staying in my mind or picture. Therefore, with the use of Augmented Reality (AR) technology to recognize the panorama of intelligent mining and Virtual Reality (VR) technology to feel the prevention and control of mine disasters. VR-AR technology has developed rapidly in recent years, and has been widely used in the fields of medical treatment, manufacturing and maintenance, entertainment, aerospace and so on. As a new type of teaching media, VR-AR technology has been gradually valued and favored by educators because of its strong teaching advantages and potential. And it is widely used in practical teaching and plays an important role [16,35,36,37,38,39,40,41]. Thus, it is of great significance to the construction of mass intelligent mining science popularization mode in primary and secondary schools with VR and AR technology.

Accordingly, this paper first summarizes the difficulties and pain points of popular science education in coal mining. After that, the VR-AR system based on 5G suitable for popular science teaching in coal mine is constructed. Finally, the popular science teaching mode based on VR-AR system were proposed.

2. Difficulties in Teaching and Popularization of Coal-Based Energy Industry

2.1. Questionnaire Survey for the Impression of the Coal Mining Industry

With the development of information technology, livestreaming taking popularity, the irresponsible reports for coal mining spread into public life. It is causing the public and non-coal mining majors have a very bad impression on coal mining. We conducted a survey on the impression of the coal mining industry among more than 1000 primary and secondary school students and 1000 college students of non-related majors, the questionnaire for this survey is shown in

Table 1. The questionnaire for this survey on coal based energy industry.

Education	Primary School □ Secondary School □ College □	Coal Related Majors
		Yes □  No □
Impression	Good □   Average □   Bad □
Technological level	Good □   Average □   Bad □
Working environment	Good □   Average □   Bad □
Safety	Good □   Average □   Bad □
Importance to the economy	Good □   Average □   Bad □
Environment	Good □   Average □   Bad □
Do you want to seek this job	Yes □  No □
Keywords you can think

. The survey results were listed in

Table 2. The survey results on coal based energy industry.

Education	Primary School			Secondary School			College (No-Related Majors)			College (Related Majors)
	613			725			1355			212
Option	G	A	B	G	A	B	G	A	B	G	A	B
Impression	5	87	521	2	68	655	88	412	855	78	112	22
Technological level	1	22	590	4	45	676	10	185	1160	87	115	10
Working environment	0	3	610	0	5	720	1	12	1342	5	41	166
Safety	0	1	612	0	6	719	3	22	1330	22	134	56
Importance to the economy	20	182	411	121	372	232	459	688	208	178	32	2
Environment	2	3	608	5	24	696	15	67	1273	24	38	150
Do you want to seek this job	Yes		No	Yes		No	Yes		No	Yes		No
	2		611	0		725	0		1355	153		59

.

It can be seen from Table 2 that except for college students in related majors, others have a poor impression of coal mines. The proportion of primary and secondary school students with bad impression reached 85.0% and 90.3%, respectively. College Students’ impression of coal based energy has improved due to their higher education, However, there are still 63.1% chose a bad impression. For technical level, working environment, safety and environmental pollution, are all negative impressions. Even 78.3% of students in related majors think that the working environment in coal mines is bad. Coal based energy is China’s basic energy, which promotes the development of China’s economy. However, for the option of “Importance to the economy”, only 3.3%, 16.7%, and 33.9% for primary, secondary school and college students (no-related majors) chose important. Except for college students in related majors, few people want to work in this industry. Even 27.8% of students in this major do not want to engage in related work. For the option of “Keywords you can think”, they think that the coal industry is dirty, messy, prone to accidents, damaging the environment, poorly trained people (coal bosses), backward technology, and even blamed for air quality (smog), as shown in Figure 3. In this case, it is difficult for them to accept when we tell about the important position of coal in China’s national economy, the development of coal mining technology and equipment modernization, the quality of employees, the number of deaths and other real circumstances in the process of popular science teaching. They even think that this is our science personnel deliberately touted the coal industry.

2.2. The Mining System of Coal Mine Is Complex and Hierarchical, and the Atmosphere of Teaching and Popular Science Is Poor

The application of knowledge related to coal mining is practical and should be closely combined with on-site production practice. Coal mining generally divided into open and underground mining. Open-pit mining system is relatively simple, and underground coal mining have a large buried depth (the maximum depth has reached 1500 m) [42]. The system of mine is very complicated, strong spatial effects, concrete as shown in Figure 4. Especially, it is hard for students to understand by relying on nothing more than a few pictures or simple animation. In the process of teaching and popularization of science in coal mining in the past, most of the contents displayed by the popularization personnel came from textbooks, which made it difficult for the audience to place themselves in the real mine. People was unable to intuitively feel the environment and equipment of the mining operation site, resulting in the popularization atmosphere was poor. Public interest in science popularization activities on coal mining is also very low. Thus, using VR and AR technology to restore the coal mine production system in proportion can not only immerse the popular science audience, but also improve the enthusiasm of the audience to participate.

2.3. It Is Difficult to Reproduce Mine Disasters and the Relevant Concepts of Audiences Are Vague

Major mine disasters, such as mine fire, gas explosion, gas outburst, rock burst, roof collapse and other phenomena, are dangerous, sudden and complex, as shown in Figure 5 [43,44,45,46,47]. It is difficult to simulate these extreme accidents in the laboratory. Primary and secondary school students also cannot go to the site to learn knowledge and skills, which makes it difficult for imagining mine disasters. Thus, this requires the use of VR-AR technology to restore the coal mine disaster.

3. VR-AR Teaching System in Coal-Based Energy Development

In terms of the new technology of intelligent and safe mining of coal, China’s coal mine industry will take intelligence, networking and digitalization as the core, and strive to promote the development of coal production from mechanization and automation to intelligent direction. At present, industry-leading new technologies such as 5G, WiFi6, AI, cloud computing, VR-AR and others have been introduced and rich professional practical experience in the construction of intelligent mines (Figure 6) [48]. Upstream and downstream coal-based energy industries, such as coal power, coal coking, coal chemical industry, will also vigorously develop. Thus, in this paper, with the help of the existing technology of the mine, we have made the corresponding VR and AR teaching popular science system for coal mining to provide means support for teaching popular science activities.

3.1. AR Panoramic Video Production

AR panoramic video is mainly shot in real time by the panoramic camera in the underground coal mines. The panoramic camera is mainly composed of six cameras in the shape of regular hexahedron, which are responsible for shooting videos in six directions respectively. Under the guidance of mine instructors, panoramic cameras were set up to take photos (Figure 7). At the same time, on-site technicians introduced the layout of roadway, roadway support methods, equipment models and parameters. In the low brightness area underground settings, we used explosion-proof lamps to increase the brightness. After shooting videos, Autopano video pro is used for video production. The specific process is as follows: Import the required video materials; Synchronous video; Splicing video; Video stability; Color calibration; Video mixing and batch rendering, as shown in Figure 8.

On the basis of panoramic video, with the help of 5G technology, remote video live can be further realized and the actual production situation of underground coal mine can be remotely projected to the room, as shown in Figure 6. Students or the audience of science popularization can carry out remote interactive operation, whose direct feeling of the intelligent mining of coal mine greatly improve the interest of science popularization (Figure 6).

3.2. Construction of the VR Roaming System of the Mine

According to the typical scenes shot by the panoramic video and the whole mine system in Figure 9, the VR roaming system of the whole mine is constructed. Video shooting mine is an inclined shaft development mine. The mine field is divided into two stages along the inclined plane, with two mining levels. Each mining level is divided into several mining areas along the strike direction. Each mining area is arranged with one stope working face and one preparatory working face. The system, restoring all scenes in equal proportion, includes all roadway and related equipment and carry out human-computer interaction at the same time [47]. The construction of roaming system is similar to the production of VR games. The specific steps are as follows: System construction and planning; Modeling based on panoramic video; Animation of modeled content; Engine construction; Scene optimization and interaction design. Some of the completed scenarios are shown in Figure 10. Every device in the VR system can be operated. The construction process of VR mine roaming system is shown in Figure 11.

3.3. Mine Dynamic Disaster Demonstration System

For cannot experience of coal mine disasters and accidents, VR technology is used to build the demo model dynamic disaster in coal mines. The model includes the rock burst and coal and gas outburst accident, which highly restore the destructive power disaster and let students experience personally feel the power of dynamic disaster in coal mine. The scene of dynamic disaster occurrence is shown in Figure 12. After the occurrence of dynamic disaster, the deformation and damage of roadway are serious, and the casualties are large.

After VR and AR systems for coal mining are built, Steam VR software can be used for human-computer interaction based on HTC Vive VR head display (Figure 13). The user can enter any area according to the map in Figure 9. In each scene, the user can walk freely and control the equipment in the scene. Therefore, the user is equivalent to actually visiting the mine. Moreover, this system is portable and can be independent of the site, it can also be carried out outdoors and especially suitable for popular science education activities.

4. Popular Science Teaching Activities Based on VR-AR System and Its Effect

4.1. Popular Science Teaching Activities Based on VR-AR System

Relying on the relevant equipment of the State Key Laboratory of “Coal Resources and Safe Mining” and the Virtual simulation center of coal mining in university, we set up the “Open Day” activity in primary and secondary schools of the State Key Laboratory. Students in primary and secondary schools and their parents are invited to carry out activities. During the Open Day activities, VR-AR mining system is used to explain the intelligent mining of coal mines and display the results.

With the help of the developed AR-VR system of intelligent coal mines, we regularly go to primary and secondary schools to carry out popular science activities (entering campus activity). Meantime, parents of students are invited to participate in the event, so as to promote the public simultaneously. Meanwhile, combined with 5G panoramic live broadcasting technology, primary and secondary school students can remotely experience intelligent mining, so that students and the public have a new understanding of mine mining (Figure 6).

4.2. Investigation on the Effect of Popular Science Teaching

After the periodic science popularization activities, we conducted a questionnaire survey again on primary and secondary school students and their parents who had received science popularization activities based on VR-AR system. The questionnaire is the same as Table 1, for parents of children, an additional item is added as “Would you like your child to engage in relevant work”. The survey results show that the impression of the coal industry has increased significantly, the proportion of bad impression decreased to 23.4%. Their parents’ impression of coal mines has also generally improved, nearly 50% of parents accept their children to work in the coal industry. In addition, we also conducted a questionnaire on popular science methods, Options include VR-AR technology, video, pictures and text content. Only 15 of the 1132 questionnaires chose video, and the rest chose VR-AR technology. Three main advantages of our VR-AR system are abtained from the questionnaires are “① Lifelike, immersive, visual impact”, “② It increases the interest of learning” and “③ Vivid, intuitive and comprehensive”. We also found some shortcomings of the system in science popularization activities, the number of people who can use at the same time is limited. Thus, the number of simultaneous users of the system will be further expanded in the later development process.

5. Conclusions

Science popularization, as one of the two wings of innovation and development, plays a very important role in innovation. In the process of popular science, if scientific researchers are absent the non-professional ‘popular science’ will ‘fill in’. Then rumors have space and conditions for existence. As a strong support of China’s energy supply, coal mining has made outstanding contributions. But due to the long-term lack of coal mining science education and the mass media bias reports, the coal mining industry in the national impression is very poor. The proportion of primary and secondary school students with bad impression reached 85.0% and 90.3%, respectively, and 63.1% for college students. Except for college students in coal-related majors, few people want to work in this industry.

Only 3.3%, 16.7%, and 33.9% for primary, secondary school and college students (no-related majors) believed that coal based energy is crucial to China’s economy. However, China’s economic and social development will still be dependent on coal in the short term due to the energy resource endowment and the stage of economic and social development. China’s coal production and utilization is also grasping the “carbon peak” and “carbon neutral” goal to intelligent high-quality development transformation development gradually. Therefore, only if people learn more about the public understand coal mining, more talents come into coal mining and the industry can get better develop.

This paper is first summed up the difficulties in teaching and popularization of coal-based energy industry: How to transform professional systematic knowledge into popular scientific knowledge accepted by the public; How to increase exposure like entertainment news and public interest in science popularization; How to improve the media threshold to make up some irresponsible and nonstandard media science.

The VR-AR system of coal mining is developed; the real intelligent production situation of coal mine is introduced into the process of popular science activities. AR panoramic video is mainly shot in real time by the panoramic camera in the underground coal mines. And with the help of 5G technology, remote video live can be further realized and the actual production situation of underground coal mine can be remotely projected to the room. Based on AR panoramic video, the VR roaming system of the whole mine is constructed. Besides, VR technology is used to build the demo model dynamic disaster in coal mine.

Popular science teaching activities based on VR-AR system were carried out by the “Open Day” activity and “Entering Campus” activity. After popular science teaching activities, the impression of the coal industry has increased significantly, the proportion of bad impression decreased to 23.4%. The children’s parents also reduced their prejudice against the coal industry. Children are more receptive and like science popularization activities based on VR-AR technology.