When Taekwondo Meets Artificial Intelligence: The Development of Taekwondo

Abstract

This study explores the comprehensive understanding of taekwondo, the application of fourth industrial revolution technologies in various kinds of sports, the development of taekwondo through artificial intelligence (AI), and essential technology in the fourth industrial revolution while suggesting advanced science directions through a literature review. Literature was sourced from six internet search electronic databases, consisting of three English databases and three Korean databases, from January 2016 to August 2023. The literature indicated cases of sports convergence with the application of fourth industrial revolution technologies, such as the game of go, golf, table tennis, soccer, American football, skiing, archery, and fencing. These sports not only use big data but also virtual reality and augmented reality. Taekwondo is a traditional martial art that originated in Republic of Korea and gradually became a globally recognized sport. Since taekwondo’s competition analysis is an analysis in which researchers manually write events, it takes a very long time to analyze, and the scale of the analysis varies depending on the researcher’s tendencies. This study presented the development of an AI Taekwondo performance improvement analysis and evaluation system and a metaverse-based virtual Taekwondo pumsae/fighting coaching platform through an AI-based motion tracking analysis method.

1. Introduction

Modern society is rapidly changing in the fourth industrial revolution era. In particular, technologies of the fourth industrial revolution (e.g., artificial intelligence [AI], big data, Internet of Things [IoT], and drones), discussed for the first time in 2016 at the World Economic Forum (WEF), are rapidly transforming society as a whole while having a profound impact on all aspects of human society, including culture, the arts, and education [1]. This impact is changing individuals’ lifestyles and altering the structure of fundamental human productive activities, while an increasing number of cases involving the integration and convergence of fourth industrial technology are found in various fields. Accordingly, competition among companies to establish their presence in new markets is intensifying, and the field of sports is no exception.

Since sports have undergone significant changes over the past two decades, sports have served as a key economic driver, while influencing social and cultural structures [2]. Fourth industrial technologies have been actively adopted in sports because they can effectively demonstrate the excellence of technological capabilities and have a low risk of errors. Fourth industrial technologies are used in various sports events to improve athletic performance, match viewing, data analysis, training, and injury management [3,4,5,6].

In particular, taekwondo was selected as a core sport in the 2016 Rio Olympics and 2020 Tokyo Olympics, contributing to the improvement of Republic of Korea’s image and its advancement, and it was firmly established as one of the best cultural content brands in Republic of Korea [7]. However, frequent disputes have taken place concerning the fairness of decisions in taekwondo matches, raising issues regarding referee decisions. Accordingly, in 2009, the Korea Taekwondo Association adopted the use of electronic chest protectors in taekwondo, which incorporates various technology trends to assist in preventing errors for hit scores due to the increase in quantified objective evaluation indicators in international sports competitions and provide objective criteria for effectively coping with referee shortages [8]. The adoption of electronic chest protectors can increase the integrity of referee decisions, minimize in-match delays due to video replays, and address scoring and decision-making distrust [7]. However, repeated mechanical defects and errors in the early stage of adoption warrant the need to identify the electronic protectors’ problems and to continuously develop software (S/W). Moreover, some individuals are demanding accuracy and consistency assurances in the measurement of hit intensity to ensure decision fairness by promoting competitive development through fair competition among different electronic chest protector brands. Accordingly, it is necessary for World Taekwondo (WT) to clearly define the specifications for electronic chest protectors.

Based on the example of using electronic protectors and some technical issues that remain to be dealt with and optimized, it becomes clear that taekwondo would benefit from integrating the innovative technologies of the fourth industrial revolution. In fact, Republic of Korea ranked the lowest among the 35 countries compared in the analysis of preparedness for the changes of the fourth industrial revolution, indicating that Republic of Korea is unprepared for fourth industrial technologies and necessitating discussion regarding the development of cutting-edge technologies in taekwondo to keep up with the times [9,10]. This study presents the development direction of taekwondo based on AI, a core technology of taekwondo and the fourth industrial revolution, and the development direction of taekwondo through technology convergence.

These technologies have been adopted early in sports because they can demonstrate the excellence of technological capabilities and have a low risk of errors [11] (

Table 1. Application cases of fourth industrial revolution technologies.

Event	Case
Golf	LDRIC, the golf robot, is a robot capable of making accurate error-free shots and was manufactured to teach optimal golf motions to humans [12]. In addition, screen golf using virtual reality (VR) has also become very popular [13].
Go	In Go, AI-based robots AlphaGo and HanDol have demonstrated greater capabilities than humans through competitions with humans [14].
Table tennis	Agilus, a table tennis robot, calculates where the ball would land, using machine-learning technology to identify the direction and trajectory of the ball. It won a match against Timo Boll, who is ranked 9th in the world [15].
Soccer	The German national team won the World Cup for the first time in 24 years after using fourth industrial technology sensors and IoT technology for real-time collection and analysis of data regarding work rate, instantaneous velocity, heart rate, shooting motion, and direction for their training [16].
Football	In National Football League matches, RFID sensors are placed inside the uniforms to accurately identify and analyze player movements, which assists in preventing injuries and improving performance [17].
Yachting	The US yachting team installed several sensors on their yacht to collect and analyze wind direction, wind speed, and information on the mast and vessel and sent the data to the PDA of each player, which helped them achieve a dramatic ‘come-from-behind’ victory [18].
Baseball	Major League Baseball is also using the convergence of IoT and big data technologies to analyze ball speed and direction while collecting data regarding all situations occurring inside the stadium [5]. Since various strategies can be established by using the collected data, significant technological developments are occurring to improve game performance. In addition, VR-based screen baseball has become popular owing to its ‘anytime’, ‘anywhere’ availability.
Skiing	Because ski events are significantly affected by space and the environment, several national ski teams are training their athletes to overcome surrounding impediments using VR [19].
Archery	The Republic of Korea national archery team created its training ground to have a similar environment to the actual competition venue in preparation for the Tokyo Olympics, showing preparedness to adapt to the route through VR. In addition, personalized grips for each athlete were created using AI-based postural analysis and 3D printing [20].
Fencing	The national fencing team conducted virtual training with VR by creating 3D models with the skills of world-class athletes [21].
Refereeing	More accurate refereeing decisions are being made in sports such as soccer, tennis, and volleyball through advanced technologies [22].

).

• Archery—Deep learning vision technology AI coach

Hyundai Motor Group’s shooting machine and AI coaching technology were behind the gold medal sweep in the 2021 Tokyo Olympics. Arrow selection program, unaffected by the environment, and deep learning vision AI coaching provided the environment for in-depth analysis of players’ habits and weaknesses (Figure 1).

• Baseball—Korea Baseball Organization (KBO) robotic umpire

Umpire qualifications and decisions have always been an issue in the KBO. Cases of strike calls being different have been observed even in the same game. To prevent this, the KBO has been using an automatic ball-strike call system (robotic umpire) in minor league games since 2020 as a pilot program. A robotic umpire tracks all pitches thrown by a pitcher in real time by considering pre-entered ground location data (mound, home plate, etc.). When the ball passes through the strike zone, AI identifies the location of that pitch and determines whether it is a strike or a ball. The final decision is relayed to the umpire through a voice system, and if fairness is recognized, it is expected to be implemented in major league games as well.

• E-sports—The development of a virtual taekwondo sparring system

In anticipation of the market potential of e-sports, WT has partnered with Refract Technologies, a gaming and IT company based in Singapore, for the past several years to develop a virtual taekwondo sparring system (Figure 2).

Unlike other computer games played with hands, the virtual sparring system is similar to actual taekwondo matches and uses motion tracking technology to delicately track whole-body movement to allow head-to-head sparring with an avatar as an actual opponent. The body of the player is transformed into the game controller. This has opened a new avenue for non-face-to-face competition with no constraints concerning sex, age, physical barrier, or region.

Sports have always changed with the times. Taekwondo has also continued to change to maintain its global popularity, and taekwondo remains an official Olympic event. At the center of change is the revision of rules for taekwondo sparring. The historical rules of taekwondo are presented in

Table 2. Explanation of historical taekwondo rules.

Year	Existing Rule(s)	Revised Rule(s)
1970	− Creation of rules for taekwondo sparring matches
1989 Adoption of electronic technologies	− Paper scoring: displays scores at the end of each round − Video: no	− Electronic scoring: displays scores in real time during matches − Use of video: for appeals
2001 First implementation of differential scoring system	− Kick to the chest protector, kick to the head: 1 point	− Kick to the chest protector: 1 point; kick to the head: 2 points
2005 Shortened length of matches	− Three 3 min rounds	− Three 2 min rounds (sudden death in case of a tie)
2009 Implementation of electronic chest protector	− Kick to the chest protector: 1 point; kick to the head: 2 points − Regular chest protector: referee scoring	− Kick to the chest protector: 1 point; spinning kick to the chest protector: 2 points; kick to the head: 3 points − Implementation of electronic chest protector: electronic scoring − Implementation of video replays − Implementation of ranking system
2014 Implementation of electronic headgear	− Kick to the chest protector: 1 point; spinning kick to the chest protector: 2 points; kick to the head: 3 points	− Kick to the chest protector: 1 point; spinning kick to the chest protector: 3 points; kick to the head: 3 points − Deduction penalty for falling motion − Implementation of octagonal court − Implementation of electronic headgear
2017 Strengthened differential scoring	− Kick to the chest protector: 1 point; spinning kick to the chest protector: 3 points; kick to the head: 3 points − Warning: −0.5 point	− Punch: 1 point; torso: 2 points; spinning kick to the chest protector: 3 points; kick to the head: 3 points; spinning kick to the head: 4 points − Penalty after warnings: −1 point
2018 Strengthened differential scoring	− Punch: 1 point; kick to the chest protector: 2 points; spinning kick to the chest protector: 3 points; kick to the head: 3 points; spinning kick to the head: 4 points	− Punch: 1 point; kick to the chest protector: 2 points; kick to the head: 3 points; spinning kick to the chest protector: 4 points; spinning kick to the head: 5 points − Penalty when just one foot goes outside the boundary line − Implementation of second weigh-in
2022 Implementation of best-of-three system	Win based on total scores from three 2 min rounds	Win determined by adding scores from each of the three 2 min rounds (the winner of two rounds is the winner)

.

A review of taekwondo, which has changed over the past four decades, showed that all changes, apart from the implementation of electronic scoring system, electronic chest protector, electronic headgear, and 4D video replay, were related to match rules. Moreover, technologies that were implemented for taekwondo competitions were defined as conventional electronic technologies. Therefore, applying the advanced technologies of the fourth industrial revolution, which surpass the technologies currently in use, can bring about revolutionary changes to the taekwondo industry.

2. Core Technologies for Improving Taekwondo Performance

For a systematic and reasonable scoring system for events that require accurate and clear judgment, such as taekwondo pumsae, this study aimed to develop an analysis and evaluation system on the basis of AI motion analysis, which is a key fourth industrial revolution technology. In addition, this study also aimed to develop a metaverse-based virtual taekwondo pumsae/sparring coaching platform that is similar to real life and is expected to have a positive impact on the development of taekwondo pumsae competitions.

2.1. Motion Recognition Technology (Human Pose Estimation)

• Motion recognition technology refers to technology that recognizes and interprets human motion. This technology is predominantly used in the fields of computer vision, machine learning, and neural networks [23].
• Motion recognition technology is predominantly used in sports for creating personalized exercise programs or teaching proper poses or motions by analyzing taekwondo or yoga motions.
• In taekwondo, motion recognition technology can be used to develop a scoring system that automatically recognizes the athletes’ motions during competition or a system that provides personalized training programs on the basis of detailed analyses of poses and motions in taekwondo-learning applications.

2.2. Three-Dimensional Reconstruction Technology (3D Reconstruction)

• 3D reconstruction technology is a technology that can reconstruct 3D models of objects or buildings from the past. Through this technology, the shape, size, structure, and color of an object or building can be digitally reconstructed (Figure 3) [24].
• 3D reconstruction technology is used in various fields. In the fields of history, culture, and the arts, the appearance of objects or buildings from the past can be reconstructed for use in research, exhibition, and education [25]. The Partymonio Project is a project that uses 3D scans to reconstruct historical sites in the Mediterranean region that have been designated as UNESCO World Heritage Sites.
• Moreover, 3D reconstruction technology is used for designing, simulating, and testing products or buildings in the fields of engineering, construction, and manufacturing. In architectural design, 3D reconstruction can be used to digitally verify the façade, internal structure, and material of buildings, on which the design, simulation, and construction of the building can be based.
• Prime examples of 3D reconstruction technology include 3D scanning, 3D modeling, VR, and augmented reality (AR). 3D scanning is a technology used to scan the façade of objects or buildings to create 3D models. 3D modeling is a technology used to create detailed 3D models by processing 3D scanning data. VR is a technology that creates a virtual world using computer graphics technology [26]. AR is a technology that adds virtual information to real-world environments [27].
• Three-dimensional reconstruction technology may be useful for taekwondo athletes. Taekwondo organizations and researchers are using 3D scanning and motion capture technology to create 3D models of taekwondo athletes, analyze athletes’ movements during training and competitions to reconstruct their movements in a 3D space, assist coaches and trainers in identifying skills and poses that require improvement, and develop personalized training programs to help the athletes achieve their goals. Moreover, 3D reconstruction technology can be used to track athletes’ progress and compare their movements to those of other elite athletes to gain insight into performance improvement.

2.3. Metaverse Technology (Real-Time Interaction and Multi-Avatar Control)

• Real-time projection in a metaverse refers to technology that can show various contents in real-time in a virtual space [28]. This allows various experiences that are impossible in the real world to be experienced in virtual space.
• Having taekwondo competitions in a metaverse allows for taekwondo matches to be viewed in virtual space without creating an actual arena, while 3D modeling of the movements of taekwondo athletes can show projections of their movements in real-time in a virtual space (Figure 4).
• Real-time projection technology is one of the essential technologies for providing various contents in a metaverse. In the field of taekwondo, such technology can be used to provide athletes with an actual competition experience in virtual space.
• Taekwondo training that occurs in a metaverse environment involves the detection of movement (motion) through the devices of the user (trainee), followed by real-time 3D streaming of the recognized motions in a metaverse.
• The user’s camera tracks their movements in real-time and reconstructs the generated images in 3D. For this, the Neural Recon algorithm is used for real-time 3D reconstruction of the scenes and predicts the position of each pose that has been detected using pose estimation and camera calibration information. Therefore, the virtual space and reconstructed 3D objects are mapped to reproduce the motions in real-time.

Figure 4. Example of taekwondo sparring in real-time in a metaverse.

Figure 4. Example of taekwondo sparring in real-time in a metaverse.

A list of key trends in technology use in taekwondo and their status is shown in Table 3.

Table 3. Key technology trends and status.

Technology	Product	Description
3D reconstruction	Pix4D (ver. 4.7.3)	− Cloud-based photogrammetry S/W that uses photos to create 3D models [29] − Enables real-time collaboration with easy data sharing, and collaborating teams can remotely choose popular options
	Autodesk Recap (Pro 2022)	− Recap, developed by Autodesk, is a photogrammetry S/W that uses photos and laser scans to create 3D models − User-friendly interface − Used in various industries, including architecture, engineering, and construction
	Zephyr photogrammetry (Zephyr 5.0.1)	− Photogrammetry S/W developed by LFM Software(ver. 5.2) that uses photos to create 3D models − Advanced processing features including texture mapping and digital surface modeling
3D data creation process	Luma AI (ver. 2.1.0)	− AI-based 3D scanning and modeling application − Creates a 3D model after 150 multi-angle scans of an object or individual using a smartphone camera − Provides an easy and accessible method of digitizing physical objects by using computer vision algorithms to process photos of objects and create 3D models − Used in applications such as product design, interior design, and gaming
	in3D: Avatar Creator Pro (ver. 1.10.68)	− Acquires the user’s facial image with a camera and uses an image processing algorithm to create a virtual character based on facial features − Various attributes of the created avatar, such as clothes, hair, and accessories, can be added or changed − Share videos, photos, messages, etc. − Link to other platforms, such as game characters or AI chat
Real-time multi-avatar interaction and control technology	Decentraland (ver. 2.0.0)	− An example of real-time multi-avatar interaction technology; VR platform that allows players to enter a virtual world and interact with other players using avatars [30] − Players can move their avatars and engage in various activities, including talking to each other via voice chat and exploring the virtual world together through mini-games [30] − Avatars of all players are displayed in real-time and all movements and motions are synchronized to provide seamless and immersive experiences
3D human motion tracking technology (Republic of Korea)		− Motion recognition research using open platforms − Software solutions that can identify body joint positions using videos are available as open platforms from several global companies and organizations − In Republic of Korea, motion recognition solutions are provided by Kakao and Naver − Compared with overseas solutions, domestic technologies have technical limitations in providing relatively accurate pose recognition because they can only provide 2D information, which does not include depth information concerning joint positions − Limited commercialization due to the REST API form, which charges according to usage
3D human motion tracking technology (overseas)		− Most use AI technology, which provides more accurate solutions for pose recognition by providing 3D coordinates of joints displayed in 2D images − As the provider of Android, Google provides ML Kit optimized for mobile environments with Tensorflow-Lite [31], making it suitable for commercialization − ML Kit is a mobile version of MediaPipe, which provides a variety of vision-related solutions in addition to pose recognition − Carnegie Mellon University, one of the leading universities in computer vision, provides a development solution called OpenPose [32]; however, its validation for technical stability is lacking − Facebook provides its motion recognition solution, called DensePose, which provides 3D surface information for estimating joint positions and muscle movements (Figure 5) − DensePose has a high computational burden, making it unsuitable for use as a mobile solution
Human pose estimation technology	OpenPose (ver. 1.7.0) CPAlphaPose (ver. 0.3.0) HRNet (ver. 2.0)	− Human pose estimation and tracking is a computer vision technology that can detect semantic key points, find the relevance of key points, and continuously track them; semantic key points, as the word implies, refer to points that have semantic meaning. For human pose estimation, the “right shoulder” and “left knee” are examples of semantic key points (Figure 6). − Pose estimation can be divided into 2D and 3D. Of the two, 2D can be divided into single-person and multi-person; it can be divided into direct regression and heat map regression as a method for predicting key points based on body joints [33]. − Direct regression is a method for estimating coordinates using 2D images’ pixel values as input values, which enables fast learning [33]. However, mapping has limitations due to the key point positions being non-linear; it is difficult to apply this technology when there are multiple individuals on a single screen. − Heat map regression calculates the probabilistic positions of key points on the body through heat maps; methods that estimate key point positions using this are visually intuitive and can be applied to multiple individuals [33]. − Recent deep learning-based approaches have achieved significant performance improvements for individual and multi-person pose estimation. Well-known 2D human pose estimation methods include OpenPose, CPN, AlphaPose, and HRNet.
Sports motion 3D key point extraction technology	MediaPipe solution from Google (ver. 0.8.8)	− Pose solution from MediaPipe Solution extracts a total of 33 body landmarks and provides 3D coordinates for each [34] − Uses standard dance videos to store 33 body 3D coordinates at a speed of 15 frames per second (FPS) or higher to collect 3D joint movement during sports dance as standard dance movement data (Figure 7)

Table 3. Key technology trends and status.

Technology	Product	Description
3D reconstruction	Pix4D (ver. 4.7.3)	− Cloud-based photogrammetry S/W that uses photos to create 3D models [29] − Enables real-time collaboration with easy data sharing, and collaborating teams can remotely choose popular options
	Autodesk Recap (Pro 2022)	− Recap, developed by Autodesk, is a photogrammetry S/W that uses photos and laser scans to create 3D models − User-friendly interface − Used in various industries, including architecture, engineering, and construction
	Zephyr photogrammetry (Zephyr 5.0.1)	− Photogrammetry S/W developed by LFM Software(ver. 5.2) that uses photos to create 3D models − Advanced processing features including texture mapping and digital surface modeling
3D data creation process	Luma AI (ver. 2.1.0)	− AI-based 3D scanning and modeling application − Creates a 3D model after 150 multi-angle scans of an object or individual using a smartphone camera − Provides an easy and accessible method of digitizing physical objects by using computer vision algorithms to process photos of objects and create 3D models − Used in applications such as product design, interior design, and gaming
	in3D: Avatar Creator Pro (ver. 1.10.68)	− Acquires the user’s facial image with a camera and uses an image processing algorithm to create a virtual character based on facial features − Various attributes of the created avatar, such as clothes, hair, and accessories, can be added or changed − Share videos, photos, messages, etc. − Link to other platforms, such as game characters or AI chat
Real-time multi-avatar interaction and control technology	Decentraland (ver. 2.0.0)	− An example of real-time multi-avatar interaction technology; VR platform that allows players to enter a virtual world and interact with other players using avatars [30] − Players can move their avatars and engage in various activities, including talking to each other via voice chat and exploring the virtual world together through mini-games [30] − Avatars of all players are displayed in real-time and all movements and motions are synchronized to provide seamless and immersive experiences
3D human motion tracking technology (Republic of Korea)		− Motion recognition research using open platforms − Software solutions that can identify body joint positions using videos are available as open platforms from several global companies and organizations − In Republic of Korea, motion recognition solutions are provided by Kakao and Naver − Compared with overseas solutions, domestic technologies have technical limitations in providing relatively accurate pose recognition because they can only provide 2D information, which does not include depth information concerning joint positions − Limited commercialization due to the REST API form, which charges according to usage
3D human motion tracking technology (overseas)		− Most use AI technology, which provides more accurate solutions for pose recognition by providing 3D coordinates of joints displayed in 2D images − As the provider of Android, Google provides ML Kit optimized for mobile environments with Tensorflow-Lite [31], making it suitable for commercialization − ML Kit is a mobile version of MediaPipe, which provides a variety of vision-related solutions in addition to pose recognition − Carnegie Mellon University, one of the leading universities in computer vision, provides a development solution called OpenPose [32]; however, its validation for technical stability is lacking − Facebook provides its motion recognition solution, called DensePose, which provides 3D surface information for estimating joint positions and muscle movements (Figure 5) − DensePose has a high computational burden, making it unsuitable for use as a mobile solution
Human pose estimation technology	OpenPose (ver. 1.7.0) CPAlphaPose (ver. 0.3.0) HRNet (ver. 2.0)	− Human pose estimation and tracking is a computer vision technology that can detect semantic key points, find the relevance of key points, and continuously track them; semantic key points, as the word implies, refer to points that have semantic meaning. For human pose estimation, the “right shoulder” and “left knee” are examples of semantic key points (Figure 6). − Pose estimation can be divided into 2D and 3D. Of the two, 2D can be divided into single-person and multi-person; it can be divided into direct regression and heat map regression as a method for predicting key points based on body joints [33]. − Direct regression is a method for estimating coordinates using 2D images’ pixel values as input values, which enables fast learning [33]. However, mapping has limitations due to the key point positions being non-linear; it is difficult to apply this technology when there are multiple individuals on a single screen. − Heat map regression calculates the probabilistic positions of key points on the body through heat maps; methods that estimate key point positions using this are visually intuitive and can be applied to multiple individuals [33]. − Recent deep learning-based approaches have achieved significant performance improvements for individual and multi-person pose estimation. Well-known 2D human pose estimation methods include OpenPose, CPN, AlphaPose, and HRNet.
Sports motion 3D key point extraction technology	MediaPipe solution from Google (ver. 0.8.8)	− Pose solution from MediaPipe Solution extracts a total of 33 body landmarks and provides 3D coordinates for each [34] − Uses standard dance videos to store 33 body 3D coordinates at a speed of 15 frames per second (FPS) or higher to collect 3D joint movement during sports dance as standard dance movement data (Figure 7)

3. Results

3.1. Description of the System Process for the Development of the Metaverse-Based Virtual Taekwondo Coaching Platform

This study aimed to promote improved performance among taekwondo athletes by developing a platform that collects their motion data for real-time analysis in a metaverse (Figure 8). Accordingly, the ultimate goal was to develop and operate the following five major systems (Figure 9):

• A management system that generates multiple real-world objects at a photorealistic level that can go beyond the spatiotemporal limitations of extant technologies (volumetrics, motion capture, etc.).
• A pipeline system that enables fast and accurate analysis by converting 2D objects in videos into 3D objects.
• A system that can track, analyze, and predict the pose (motion) of an object (athlete) in real time by integrating multiple perspectives from multiple cameras.
• A system that can analyze and simulate matches by projecting them into a metaverse through real-time 3D reconstruction.
• A system that can use athletes’ biometrics, physical condition, athletic performance, and motion data to analyze their strengths and weaknesses.

The target system configuration of the program reconstructs and projects 2D (2D image) input data of multiple objects that actively move in a wide space, such as a taekwondo arena, from cameras or images into 3D objects in a metaverse (virtual space) in real-time to provide a system that can analyze performances by freely replaying scenes from different perspectives.

In addition, it implements a proprietary Convolutional Neural Network (CNN) architecture to improve the advanced performance of proprietary AI CNN architecture (speed, time, accuracy, resources) for excellent performance (Figure 10). The composition of this is as follows:

• Fast inference speed: a function that detects as many objects as possible in the shortest possible time and infers the poses;
• High 3D reconstruction accuracy: a function that increases the accuracy of reconstructing multiple 3D objects in various environments (wide open spaces where overlapping of individuals’ faces can occur rapidly);
• Fast and accurate motion tracking: accurate motion tracking function for multiple fast-moving 3D objects;
• Fast and accurate motion prediction: accurate motion prediction function for multiple fast-moving 3D objects.

3.2. The Process of Technical Development for the System

3.2.1. Development of 3D Reconstruction Technology; Server-Based S/W

Application of 3D technology analysis technologies is required for motion analysis. For this, photorealistic video image-based 3D object generation technology can be developed to reconstruct 2D objects into 3D objects using sensor-less cameras (Figure 11). Applied technologies for this include photogrammetry technology that can generate 3D stereoscopic models on the basis of 2D flat images and convert long-distance and wide-area objects into 3D objects (Figure 12). Moreover, neural radiance fields (NeRF) technology can build avatars or an entire virtual world on the basis of images from a single camera by using a multi-resolution hash encoding method, pioneered by NVIDIA (Figure 12). This can be provided as open-source to open locations, such as hospitals, schools, and sports venues, and Point Cloud Library technology can be used (Figure 12) [35]. With this, the existing limitations of limited space and the number of objects can be overcome, and the cost and time of high-quality 3D objectification services and architecture can be reduced (Figure 13).

3.2.2. Development 3D Data Creation Process (Server-Based S/W + Algorithm)

A 3D data creation process is required for taekwondo motion analysis. Subsequently, the development of video image-based 3D model texture mapping technology is required [36]. This requires geometry reconstruction and texture geometry reconstruction of 2D images acquired from a monocular camera and no other equipment (sensor, various other types of cameras, etc.). For this, the NeRF model can be applied to multi-perspective images to create 2D textures and calculate the geometric correlation for texture blending through texture transfer. Subsequently, 3D composition is used for reconstructions that are the closest to reality.

Applied technologies are suitable for situations that require real-time processing and are used by converting extracted point clouds into 3D mesh. Moreover, simultaneous localization and mapping (SLAM) can use single cameras (wide-angle, fisheye, and spherical cameras), compound eye cameras (stereo and multi-cameras), and RGB-D cameras (depth and ToF cameras), which is expected to reduce time and cost through various algorithms [37]. By using geometry reconstruction algorithms to extract 3D features, texture geometry reconstruction algorithms for fast and accurate texture reconstruction, texture transfer algorithms that can apply high-frequency components of the target image (texture) to the source image, texture blending algorithms to obtain special effects by applying various textures, and a perspective warping algorithm that flexibly adjusts the perspective from which an object is viewed to transform the perspective according to the vanishing point, not having to use expensive sensors such as those used in extant methods can be overcome, and the time and cost can be reduced. Consequently, the limitations of limited space and the number of objects can be overcome, while the cost of high-quality 3D objectification can be reduced compared with existing services. Furthermore, the rendering time for 3D objectification can be reduced from that of existing architecture.

3.2.3. Development 3D Human Motion Features (3D Human Motion Tracking Technology; Server-Based S/W + Algorithm)

For taekwondo motion analysis, the development of recognition technologies for real-time positioning, pose, forms, and behavior of fast-moving objects is required. This requires the development of 3D human motion features to extract 3D information from 2D images for analysis and prediction. This allows for quick and accurate reconstructions, including occlusions, and reproducing them as smooth motions.

Owing to occlusions, extant technologies require improvements in processing methods. An example of this is a hugging scene, where one object overlaps another object. Therefore, the 3D object motion must be improved for a faster and more accurate 3D reconstruction; however, the multi-person motion capture function also requires improvement. For this, AI-based learning is being used to solve the occlusion problem, and attempts are being made to solve the existing performance problem with multi-person motion capture of ten or more individuals in real-time and data transmission of ≥30 FPS. Moreover, fast 3D reconstruction of objects, accurate pose estimation, natural movements of object poses, validity of the degree of freedom of joints, and accuracy of determining whether the identity of an individual appearing in a scene is an existing individual or a new individual can be improved using algorithms.

Applied technologies that can be used to improve this include (1) a fast and accurate pose inference CNN architecture capable of fast and accurate performance and assurance of cost competitiveness through network architecture module design (Figure 14 and Figure 15) [38]; (2) global context vision transformers capable of fast and accurate 3D pose reconstruction through CNN; (3) skip connection capable of fast and accurate processing by ensuring accuracy and eliminating memory (speed) bottlenecks associated with DenseNet; (4) human motion comparison (comparison of similarities in human motions); (5) multi-object tracking (MOT) identity tracking; and (6) encoding maps for immediate 3D pose encoding (Figure 15). In addition, human motion tracking is required to resolve the natural movements of object poses and the validity of joint induction [39]. Through this, obscured parts can be accurately predicted to improve the accuracy of tracking and reconstructing complex scenes by simultaneously tracking multiple objects’ movements (ten or more individuals). Furthermore, smooth motions can be created through a fast and smooth process of realistic movements, and the identification of objects can be improved to create smooth and diverse scenes.

3.3. Expected Final Project

The AI-powered, metaverse-based XR training simulator, which was developed using the technologies described above, is a virtual taekwondo pumsae/sparring coaching platform based on metaverse and AI analysis and evaluation system to improve taekwondo performance (Figure 16). The sparring platform evaluates motions to improve performance, and AI provides suitable training solutions. The pumsae platform evaluates pumsae motions to improve performance and provides AI-based training solutions for accurate motions and environment composition. Specific functions are as follows:

• Provides a virtual taekwondo arena where competition is held.
• AI observes athletes’ motions to analyze the athletic ability of each athlete.
• Reproduces match videos as 3D poses in real time to analyze them using pose analysis.
• Analyze athletes’ data in a virtual space to simulate sparring with self for analysis of outcome and prediction of win or loss.
• Analyzes the actions of each national team athlete to provide match strategies.
• Provides various data and information by connecting with various IoT devices used by other national team athletes.

AI taekwondo: athletic performance improvement analysis and evaluation system (Figure 17):

• Analysis and evaluation of physical abilities.
• Analysis and evaluation of BMI.
• Analysis and evaluation of athletic performance.
• Motion analysis and evaluation.
• Performance analysis and evaluation.
• Comparison with opponents.
• Suggestion of personalized training methods.
• Suggestion of personalized match strategies.

Sparring judgment and coaching platform (Figure 18):

• AI-based athletic performance analysis and evaluation.
• Analysis of athletic performance of self and opponent.
• Prediction of win probability through athletic performance analysis.
• Identification of opponent’s strengths and weaknesses.
• Improvement of win probability through intensive and repeated simulation training for the opponent’s target points.

Pumsae judgment and coaching platform (Figure 19):

• AI-based pumsae evaluation.
• Accuracy of motion evaluation.
• Motion evaluation according to scoring criteria.
• Motion analysis of opponent.
• Improvement of weaknesses by identifying deduction penalty motions.

Creativity and Innovativeness of the Project

First, athletes are not required to wear any additional equipment; accurate analysis and predictions can be made by using only cameras and AI technology. This ensures athletes’ freedom of movement.

Second, characters of each national team athlete are created, and virtual matches based on analysis data are provided, while the strengths and weaknesses of athletes from each country are identified and provided on the basis of the analysis of match video data. Moreover, the appearances of actual athletes are processed as 3D models to enable virtual simulation matches with any opponent at any time.

Third, the feeling of being at the competition arena is recreated using metaverse (XR) to provide an environment equivalent to the actual competition venue.

4. Discussion

This study aimed to identify the convergence of advanced technologies in the sports industry to keep up with the fourth industrial revolution era and proposed the direction of application of fourth industrial technologies for the performance advancement of Republic of Korea taekwondo. Existing AI-based technologies have focused on taekwondo education rather than improving performance. However, to improve the performance of national team athletes, it is necessary to accurately analyze the match data of national team athletes from each country with AI and provide a training method capable of providing a systematic training program and simulation. In addition, the implementation of advanced technologies to predict match outcomes and technological changes that provide analysis through virtual simulation closest to reality and the training environment are required. In particular, precise analysis of athletes’ strengths and weaknesses and those of their opponents is important to winning in advanced competitions. The findings indicated that the development of advanced equipment is required because establishing personalized match strategies for each opponent and running enough simulations before matches through advanced fourth industrial technologies can increase the win probability. Accordingly, this study presented an AI taekwondo performance improvement analysis and evaluation system and a metaverse-based virtual taekwondo pumsae/sparring coaching platform for the performance advancement of taekwondo (

Table 4. The present and future of project technology.

Division	Currently	Future
Data (match) analysis	Image-based data analysis	3D transformation-based data analysis
Training equipment	Sensor data analysis with training equipment	Image data analysis without training equipment
Virtual training	Training without virtual players	Allows actual player-based virtual training
Professional coaching	Subjective person-based coaching	Objective AI-based coaching
Feeling of being at a competition	No feeling of being at a competition	Provides feeling of being at a competition using metaverse

, Figure 20).

It is believed that the use of these systems can contribute to improving taekwondo performance in various ways (

Table 5. Quantitative objectives of the project.

Division	Quantitative Goals
Taekwondo skill analysis	For pumsae, detailed analysis of motions according to scoring chart For sparring, detailed analysis of hand and leg motions
Athlete evaluation	Analysis for multidimensional evaluation when selecting athletes for the national team Evaluation and analysis by analyzing matches of Republic of Korea and foreign national team athletes
Development of match strategies	Establishment of match management strategies through analysis of opponent’s motions
Development of training program	Establishment of personalized training programs that identify the strengths and weaknesses of each athlete’s motions to emphasize the strengths and reduce the weaknesses
Real-time match analysis	Match analysis from multi-angular positions by converting match videos into 3D data

).

First, AI technology can be used to analyze taekwondo techniques. Analyses can be performed by observing various athletes’ taekwondo match videos and extracting characteristics of taekwondo techniques or identifying match strategies. Through this, the strengths and weaknesses of each athlete can be identified, and methods for improving such strengths and weaknesses can be found.

Second, AI technology can be used to evaluate the performance of athletes objectively. For example, the skill levels of athletes can be measured, or the skills of athletes can be improved by providing feedback.

Third, AI technology can be used to develop match strategies through simulations. Match situations can be analyzed in real-time to suggest appropriate strategies, or the strategies of the opponent can be predicted before the match to identify the appropriate response.

Fourth, AI technology can be used to develop personalized training programs. Match management methods of athletes and taekwondo training data can be collected and analyzed to identify vulnerabilities and, subsequently, provide training programs, thereby increasing athletes’ efficiency of training.

Fifth, AI technology can analyze match situations in real time. By analyzing athletes’ in-match movements in real-time, appropriate strategies can be suggested. The analysis of matches and athletes with AI technology can significantly improve taekwondo athletes’ performance; athletes can enhance their skills and strategies, which can help them produce better results.

Fourth industrial revolution technologies (e.g., AI, big data, IoT, and drones), discussed for the first time in 2016 at the WEF, are rapidly transforming society while having a profound impact on all aspects of human society, including culture, the arts, and education. This impact is changing individuals’ lifestyles and altering the structure of fundamental human productive activities, while an increasing number of cases involving the integration and convergence of fourth industrial technology are found in various fields. Accordingly, competition among companies is intensifying to establish their presence in new markets, including the field of sports.

Match analysis using AI-based motion tracking is used for various purposes. Motion tracking analysis is a method that recognizes and analyzes bone joints, which enables the identification of body angles, speeds, and positions [40,41]. Consequently, motion tracking analysis is being used in various sports (baseball, hurdles, basketball, soccer, etc.) for effective athletic motion analysis [42,43,44].

The academic community has identified the importance of match analysis and has continued to research match analysis. However, the analysis of taekwondo matches requires researchers to input events manually. Consequently, the analysis time is long, and different scales are being used depending on the researcher’s propensity, resulting in differences in the accuracy of the analysis results, data information, etc. Therefore, this study supported match analysis using motion tracking instead of analysis using variables such as scoring, win/loss, valid kicks, and location.

This study presented the development of an AI taekwondo performance improvement analysis and evaluation system through AI-based motion tracking analysis and a metaverse-based virtual taekwondo pumsae/sparring coaching platform. Since this is one of the key technologies that is globally observed, countries and sports industries should have an active interest in and invest in enhancing fourth industrial technologies to ensure that they are not left behind in the changing environment. However, many countries have yet to adopt innovative technologies and are not prepared for the changing times. The reality is that they are falling behind not only regarding the country’s development but in all environments, such as daily life and training. Efforts to respond to and converge on rapidly growing fourth industrial technologies and the advancement of taekwondo are needed. Therefore, the research on measures for advancing the fourth industrial revolution and taekwondo presented in this study may be significant.