Reliability of Wu Huanqun’s Table Tennis Game Analysis Method in Authors’ Own Modification

Abstract

The observation and analysis of the game or the player’s behavior and actions is a very important aspect of optimizing sports training in table tennis. The aim of this study was to present the applied method of observation, developed by Professor Wu Huanqun and modified by the authors, together with an assessment of its reliability. The method consists in observing the winning actions of a given player and the same actions of his opponent in parallel, on three levels of complexity. Levels 1, 2, and 3 are concerned with identifying increasingly detailed winning strokes and stroke combinations. To assess the reliability of the method, seven experts independently analyzed the men’s final game of the World Championship 2019. Kendall’s coefficient of concordance, with an χ² significance test, was used to assess the compliance of the experts. Cronbach’s alpha was used to assess internal consistency. The obtained results indicate the high compliance and internal consistency (thus, high reliability) of the assessed method: the highest at the Level 1 observation level and the smallest, but significant, at Level 3. The performed statistical analysis indicates a high agreement between the experts’ opinions, which was taken as a measure of high reliability for the described method of observation.

1. Introduction

Table tennis is a well-known sports game that, according to the literature, is played by around 300 million people [1]. This figure proves its high popularity, but also its high availability, ease, and the possibility of cultivating it (simple and understandable rules; affordable equipment prices; the possibility of playing indoors in small spaces, or even recreationally in the garden, park, etc.). As a competitive sport, however, it is a very complex game that requires players to have a high level of many factors, including: physical fitness, mental toughness, technical skills, and tactical skills [2,3,4]. A large number of types of strokes, their combinations into combinations of strokes, the possibility of using different types of strokes, and the variety of the equipment used (blades and rubbers), along with a very short duration of the action, make this game difficult to perceive (difficult to observe). The observation and analysis of the game or the player’s behavior and actions are very important aspects of optimizing sports training in table tennis. Performance analysis is an important part of an athlete’s development and coaching process, and it is a significant competitive advantage [5]. Notational analysis, the purpose of which is to obtain objective data from the course of the game or the actions of the player (team), allows one to gain knowledge about the tactics used, techniques used, and activity in various periods of sports competition [6].

Many authors dealing with this subject in table tennis have used a variety of tools for technical, tactical, or biomechanical observation and analysis. Malagoli, Lanzoni, Di Michele, and Merni [3], summarizing the work performed and published in this area, listed several factors, called analysis indicators, e.g., the diversity of players (gender, age, advancement, etc.), technical indicators (strokes, footwork), tactical indicators (types and number of errors, effectiveness, position in relation to the table during the strike, duration of the game and individual phases, etc.), the equipment used, etc. Depending on the indicators used, different areas of the game were analyzed. Malagoli, Lanzoni et al. [3] analyzed, inter alia, the places where players serve, the type of shots used and their score, and the footwork used during the game. In their research, the authors compared players from Asia and Europe, pointing to the elements of the game in which Asians showed an advantage. Tamaki, Yoshida, and Yamada [7] proposed the introduction of an observation and analysis method consisting of recording won or lost service actions and the number of strokes. Jie Wang [8] assessed the shots used, their effectiveness, and the positions taken depending on the period of the game. Fuchs et al. [9], in their work, presented an overview of various approaches in the field of game analysis over several decades, indicating in more detail the research procedures used in China and Japan. Zhang, Zhou, and Yang [10] reviewed the methods of analysis used in China. They indicated the importance of obtaining current knowledge about the best solutions for the game and understanding it. The authors also indicated a model for dividing the game into three phases (service–attack, return–attack, stalemate), commonly used in China, as the most frequently used by Chinese trainers and researchers, which is also supported and developed in many studies with computer software [11]. In Chinese-language literature, a lot of space is devoted to issues related to research work and its meaning, as well as methods of game analysis. In the collective work edited by Qiu Zhonghui [12], the authors presented the basic methods of game analysis developed by Professor Wu Huanqun. Various variants of these methods, related to the comprehensive analysis of win shares, have been used by one of the authors of this work for the last decades as part of the training process for players, mainly national teams [13]. Many analyses were also carried out, the conclusions of which found their application in the training of coaches under the proprietary program “FUNdamentals” and training conducted by the Polish Table Tennis Association [14,15].

Various tools used to analyze the game provide various information related to the technique or tactics. The enormous variety and technical and tactical complexity of table tennis, as well as the use of “fake” actions by players to increase the illegibility of the game, mean that the data collected by analysts or observers may be misidentified. Therefore, it seems very important to create tools that will enable deep, but also reliable, information gathering. In the literature, it is difficult to find works on the discussed scope that propose or use tools with a specific high reliability, providing a high degree of certainty that the collected data are objective and properly identified. The few studies that are available concern evaluating the repeatability of research on selected game elements [16,17,18].

The aim of this study was to present the applied method of observation, developed by Professor Wu Huanqun and modified by the authors, together with an assessment of its reliability. Confirming the high reliability of the method described in this paper may provide an objective tool, thanks to which it is possible to obtain very important information on the technical and tactical structure of the gameplay of the best players in the world and, above all, on the combinations of strokes used that will result in winning an action (scoring a point). The presented method could also be used in the analysis of players’ games at any sports level in order to monitor, diagnose, and delineate training plans. The developed reliable method can also answer questions of how to win in table tennis, how the game has changed over the last decades, how women and men win, how different types of playstyles win, etc. It could also guide the discipline’s development trends.

2. Materials and Methods

Fifteen experts—mainly medalists of large international seniors’ competitions and currently working trainers—with national and international achievements were invited to assess the reliability of the method. Each of them was asked to analyze the game according to the proposed method. Seven of the experts responded and delivered their observations and calculations. The observation and analysis concerned the final of the Table Tennis World Championship, played in Budapest in 2019—Ma Long (China) vs. Mathias Falk (Sweden). The match was recorded and the material was cut into pieces—that is, rallies won by the individual players (51 rallies won by Ma Long; 37 rallies won by Falk). Despite the fact that the method can be partially used live, recording and, thus, the possibility of replaying the actions is a better method. Each of the experts viewed the same material, prepared and pre-sorted by the authors of the paper. Every expert watched the full match (all rallies) and the number of replays was unlimited.

The method consists of observing the winning actions of a given player and the same actions of his opponent in parallel. The method takes into account the division, used and described in the literature, of the entire action (rally) into three phases: service–attack, return–attack, and stalemate. The observation sheet (

Table 1. Observational sheet.

Observational Sheet			Number of Points Won by Player 1	Number of Points Won by Player 2
Level 1 Identification of Won Action	Level 2 Identification of Winning Shot or Shots in Action	Level 3 Identification of Shots in Combination
1. Serve
	1.1. Fh sidespin serve
	1.2. Fh backspin-nonspin serve
	1.3. Bh sidespin serve
	1.4. Fh “hook” serve
	1.5. Fh reverse serve
	1.6. Other serves
2. Serve+Counterattack
	2.1. Fh sidespin serve+Counterattack
		2.1.1. Fh sidespin serve+Fh-topspin
		2.1.2. Fh sidespin serve+Bh-attack
		2.1.3. Fh sidespin serve+Bh-topspin
		2.1.4. Fh sidespin serve+Fh-attack
		2.1.5. Fh sidespin serve+Bh-block
		2.1.6. Fh sidespin serve+Fh-push
		2.1.x. Fh sidespin serve+other
	2.2. Fh-back-nonspin-serve+Counterattack
		2.2.1. Fh-back-nonspin-serve+Fh-topspin
		2.2.2. Fh-back-nonspin-serve+Bh-attack
		2.2.3. Fh-back-nonspin-serve+Bh-topspin
		2.2.4. Fh-back-nonspin-serve+Fh-attack
		2.2.5. Fh-back-nonspin-serve+Bh-block
		2.2.6. Fh-back-nonspin-serve+Fh-push
		2.2.7. Fh-back-nonspin-serve+Fh-flick
		2.2.8. Fh-back-nonspin-serve+Bh-push
		2.2.x. Fh-back-nonspin-serve+Other
	2.3. Bh sidespin serve+Counterattack
		2.3.1. Bh sidespin serve+Fh-topspin
		2.3.x. Bh sidespin serve+Other
	2.4. Fh “hook” serve+Counterattack
		2.4.1. Fh “hook” serve+Fh-topspin
		2.4.x. Fh “hook” serve+Other
	2.5. Fh reverse serve+Counterattack
		2.5.1. Fh reverse serve+Fh-topspin
		2.5.x. Fh reverse serve+Other
	2.6. Other serves+Counterattack
3. Return
	3.1. Fh push
	3.2. Fh topspin-attack
	3.3. Bh-push
	3.4. Bh-flick
	3.5. Bh-topspin-attack
	3.6. Fh-flick
	3.x. Other return
4. Return+Counterattack
	4.1.Fh-push+Counterattack
		4.1.1. Fh-push+Fh-topspin
		4.1.2. Fh-push+Bh-block
		4.1.3. Fh-push+Bh-topspin
		4.1.4. Fh-push+Fh-block
		4.1.5. Fh-push+Bh-attack
		4.1.x. Fh-push+Other
	4.2. Bh-push+Counterattack
		4.2.1. Bh-push+Fh-topspin
		4.2.2. Bh-push+Bh-block
		4.2.3. Bh-push+Bh-topspin
		4.2.x. Bh-push+Other
	4.3. Fh-topspin-attack+Counterattack
		4.3.1. Fh-topspin+Fh-topspin
		4.3.2. Fh-topspin-fh+Bh-atttack
		4.3.3. Fh-attack+Counterattack
	4.4. Bh-topspin-attack+Counterattack
		4.4.1. Bh-attack+Bh-attack
		4.4.x. Bh-attack-topspin+Other
	4.5. Bh-flick+Counterattack
		4.5.1. Bh-flick+Bh-attack
		4.5.x. Bh-flick+Other
	4.x. Other return+Counterattack
5. Attack+Counterattack
	5.1. Fh-Topspin+Counterattack
		5.1.1. Fh-topspin+Fh-topspin
		5.1.2. Fh-topspin-fh+Bh-atttack
		5.1.3. Fh-topspin-fh+Fh-atttack
		5.1.4. Fh-topspin+Bh-topspin
		5.1.5. Fh-topspin+Bh-block
		5.1.x. Fh-topspin+Other
	5.2. Bh-attack+Counterattack
		5.2.1. Bh-attack+Bh-attack
		5.2.2. Bh-attack+Fh-attack
		5.2.3. Bh-attack+Fh-topspin
		5.2.x. Bh-attack+Other
	5.3. Fh-attack+Counterattack
		5.3.1. Fh-attack+Fh-attack
		5.3.2. Fh-attack+Bh-attack
		5.3.x. Fh-attack+Other
	5.4. Bh-topspin+Counterattack
		5.4.1. Bh-topspin+Fh-topspin
		5.4.2. Bh-topspin+Bh-topspin
		5.4.3. Bh-topspin+ Bh-attack
		5.4.x. Bh-topspin+Other
	5.5. Flick+Counterattack
6. Block+Counterattack
	6.1. Bh-block+Counterattack
		6.1.1. Bh-block+Bh-block
		6.1.2. Bh-block+Fh-topspin
		6.1.3. Bh-block+Fh-attack
		6.1.4. Bh-block+Fh-block
		6.1.5. Bh-block+Bh-attack
		6.1.x. Bh-block+Other
	6.2. Fh-block+Counterattack
		6.2.1. Fh-block+Bh-block
		6.2.2. Fh-block+Fh-topspin
		6.2.x. Fh-block+Other
	6.3. “Fishing”-lob+Counterattack
7. Push-Counterattack
	7.1. Bh-push+Counterattack
		7.1.1. Bh-push+Fh-topspin
		7.1.2. Bh-push+Bh-block
		7.1.3. Bh-push+Bh-push
		7.1.4. Bh-push+Fh-attack
		7.1.x. Bh-push+Other
	7.2. Fh-push+Counterattack
		7.2.1. Fh-push+Fh-topspin
		7.2.2. Fh-push+Bh-block
		7.2.x. Fh-push+Other
8. Attack-against-Chop+Counterattack
9. Chop+Counterattack
10. Other

Note: Fh—forehand; Bh—backhand; +—combination of shots.

) takes this division into account and contains information on three levels. Level 1 concerns the observation of the number of actions and the identification of the actions won directly with the serve (1), the return (2), the serve and counterattack (3), the return and counterattack (4), the actions won in the furthest part of the exchange (the last two strokes), the attack–counterattack (5), the block–counterattack (6), the push and counterattack (7), the attack-against-chop and counterattack (8), the chop and counterattack (9), and others, meaning the so-called “lucky balls” (10). Level 2 concerns a further, more detailed identification of the actions in individual phases of the game (defining a part of the game), and Level 3 is a detailed description of Level 2 (division into subcategories, taking into account the type of strokes used by the player). The record in the observational sheet (see Supplementary Materials) is made during the analysis of the footage from a given match. All actions played in the match were presorted according to the victories of individual players. It is best to do so by cutting this material with a suitable tool (Bandicut in the present study). The material obtained in this way was sorted into appropriate folders corresponding to individual levels, such as fragments of the game and strokes, as in the observational sheet. Then, numbers corresponding to the number of data used in the won actions of game fragments and strokes were entered into the spreadsheet. The use of the spreadsheet and the individual observation steps are presented in the Instructions for the Method (Appendix A), which was made available to each of the experts.

Statistical Analysis

Basic descriptive statistics (median quartile deviation and mean standard deviation) were calculated for the results of the process of evaluating individual observations (winning actions of players). The normality of the distribution of variables was verified with the Shapiro–Wilk test. As the normal distribution was not confirmed for most of the data, nonparametric methods were used. The differences between the experts’ assessments were checked using the Friedman test, as was their differentiation at 3 levels using the Kruskal–Wallis test (K-W). Kendall’s coefficient of concordance (W) was used with a significance test, χ², assuming a good agreement between the experts’ scores for W ≥ 0.75, moderate for 0.5 ≤ W < 0.75, and low agreement for 0.2 ≤ W < 0.5, and no consensus was used to assess the compliance of the experts using W < 0.2 [19,20]. For the assessment of internal consistency, Cronbach’s alpha was used; a value above 0.70 was adopted as an acceptable consistency [21]. The level of statistical significance was established for α = 0.05, and in cases of analyses in the area of the 3 levels of the assessment process (Levels 1–3), the Bonferroni correction was applied.

3. Results

The number of actions won by Ma Long, classified and ordered by experts, is presented in

Table 2. Descriptive statistics and reliability assessment—the number of individual actions (game fragments) classified by experts (n = 7)—for the observations of Ma Long.

Level of Observation	Fragment of the Game	M ± SD	Me ± SQ	Kendall’s W	p	Cronbach’s Alpha
Level 1				0.991	<0.001	0.999
	1. Serve	10.0 ± 0.0	10.0 ± 0.0
	2. Serve+Counterattack	9.9 ± 0.4	10.0 ± 0.0
	3. Return	10.0 ± 0.0	10.0 ± 0.0
	4. Return+Counterattack	11.1 ± 0.4	11.0 ± 0.0
	5. Attack+Counterattack	6.0 ± 0.0	6.0 ± 0.0
	6. Block+Counterattack	1.9 ± 0.4	2.0 ± 0.0
	7. Push+Counterattack	2.0 ± 0.6	2.0 ± 0.0
	8. Attack after chop+Counterattack	0.0 ± 0.0	0.0 ± 0.0
	9. Chop+Counterattack	0.0 ± 0.0	0.0 ± 0.0
	10. Others	0.0 ± 0.0	0.0 ± 0.0
Level 2				0.783	<0.001	0.966
	1.1. Fh sidespin serve	5.4 ± 2.1	5.0 ± 1.5
	1.2. Fh backspin-nonspin serve	2.0 ± 2.0	3.0 ± 1.5
	1.5. Fh reverse serve	2.1 ± 0.4	2.0 ± 0.0
	2.1. Fh sidespin serve+Counterattack	8.1 ± 2.2	9.0 ± 2.5
	2.2. Fh backspin service+Counterattack	1.7 ± 2.3	1.0 ± 2.5
	3.1. Fh push	5.3 ± 0.8	5.0 ± 0.0
	3.2. Topspin+fh attack	1.7 ± 0.8	2.0 ± 0.0
	3.4. Bh flick	2.0 ± 0.0	2.0 ± 0.0
	3.6. Fh flick	1.0 ± 0.0	1.0 ± 0.0
	4.1. Fh push+Counterattack	7.3 ± 0.5	7.0 ± 0.5
	4.2. Bh push+Counterattack	0.9 ± 0.4	1.0 ± 0.0
	4.3. Fh topspin-attack+Counterattack	1.0 ± 0.0	1.0 ± 0.0
	4.4. Bh topspin-attack+Counterattack	1.9 ± 0.4	2.0 ± 0.0
	5.1. Fh topspin+Counterattack	5.1 ± 1.1	5.0 ± 0.5
	5.2. Bh attack+Counterattack	0.4 ± 1.1	0.0 ± 0.0
	5.3. Fh attack+Counterattack	0.1 ± 0.4	0.0 ± 0.0
	6.1. Bh block+Counterattack	1.9 ± 0.4	2.0 ± 0.0
	7.2. Fh push+Counterattack	2.0 ± 0.6	2.0 ± 0.0
Level 3				0.603	<0.001	0.946
	2.1.1. Fh sidespin serve+Fh-topspin	4.1 ± 1.2	5.0 ± 1.0
	2.1.2. Fh sidespin serve+Bh-attack	1.3 ± 0.8	1.0 ± 0.5
	2.1.3. Fh sidespin serve+Bh-topspin	0.3 ± 0.8	0.0 ± 0.0
	2.1.5. Fh sidespin serve+Bh-block	2.4 ± 1.3	3.0 ± 0.5
	2.2.1. Fh-back-nonspin-serve+Fh-topspin	0.7 ± 1.3	0.0 ± 1.0
	2.2.2. Fh-back-nonspin-serve+Bh-attack	0.6 ± 1.1	0.0 ± 0.5
	2.2.5. Fh-back-nonspin-serve+Bh-block	0.5 ± 0.8	0.0 ± 0.5
	4.1.1. Fh-push+Fh-topspin	4.3 ± 0.5	4.0 ± 0.5
	4.1.2. Fh-push+Bh-block	0.3 ± 0.5	0.0 ± 0.5
	4.1.3. Fh-push+Bh-topspin	0.6 ± 0.5	1.0 ± 0.5
	4.1.x. Fh-push+Other	2.1 ± 0.4	2.0 ± 0.0
	4.2.x. Bh-push+Other	0.9 ± 0.4	1.0 ± 0.0
	4.3.1. Fh-topspin+Fh-topspin	1.0 ± 0.0	1.0 ± 0.0
	4.4.1. Bh-attack+Bh-attack	0.9 ± 0.4	1.0 ± 0.0
	4.4.x. Bh-attack-topspin+Other	1.0 ± 0.6	1.0 ± 0.0
	5.1.1. Fh-topspin+Fh-topspin	3.6 ± 0.8	4.0 ± 0.5
	5.1.3. Fh-topspin-Fh+Fh-atttack	0.9 ± 1.1	1.0 ± 0.5
	5.1.x. Fh-topspin+Other	1.0 ± 0.6	1.0 ± 0.0
	5.2.3. Attack+bh+Topspin+fh	0.4 ± 1.1	0.0 ± 0.0
	5.3.1. Fh-attack+Fh-attack	0.1 ± 0.4	0.0 ± 0.0
	6.1.1. Bh-block+Bh-block	0.7 ± 0.5	1.0 ± 0.5
	6.1.2. Bh-block+Fh-topspin	0.7 ± 0.5	1.0 ± 0.5
	6.1.3. Bh-block+Fh-attack	0.1 ± 0.4	0.0 ± 0.0
	6.1.5. Bh-block+Bh-attack	0.1 ± 0.4	0.0 ± 0.0
	6.1.x. Bh-block+Other	0.1 ± 0.4	0.0 ± 0.0
	7.2.1. Fh-push+Fh-topspin	1.0 ± 0.6	1.0 ± 0.0
	7.2.2. Fh-push+Bh-block	0.9 ± 0.4	1.0 ± 0.0
	7.2.x. Fh-push+Other	0.1 ± 0.4	0.0 ± 0.0

Note: M—arithmetic mean, SD—standard deviation, Me—median, SQ—quartile deviation, Fh—forehand, Bh—backhand.

.

After analyzing the results, it was found that the tested competitor scored the most points directly from service (10) and return (10), in actions after his service (service–counterattack: 10), and from counterattack actions after his return (11). The obtained results also show the great use of the forehand serve in winning actions (directly and in the first stroke after his service) and in the forehand push, both as a return (5) and in the exchange in play immediately after the return (7).

There were no significant differences between the judges’ assessments for individual observations (χ² (53.6) = 12.302, p = 0.056), while the data for each expert differed according to the three levels of the experiment (K-W (53.3) > 13.83, p with Bonferroni correction <0.008). Post hoc tests showed differences between the first and third level and the second and third level for individual experts, except for experts 4, 6, and 7, and for all experts, there were no differences between the first and second level.

The reliability of the judges’ (experts’) grades was checked in each of the analyzed levels of the game. At the first two levels, the agreement of the assessments was good (W > 0.78, p < 0.001), and it was moderate at the third level (W = 0.60, p < 0.001). A very high internal consistency was obtained (Cronbach’s alpha > 0.90). At subsequent levels, the procedures for assessing the values of the reliability indicators were minor and decreasing.

The number of won actions by Mathias Falck, classified and ordered by experts, is presented in

Table 3. Descriptive statistics and reliability assessment—the number of individual actions (game fragments) classified by experts (n = 7)—Mathias Falck’s observation.

Level of Observation	Fragment of the Game	M ± SD	Me ± SQ	Kendall’s W	s	Cronbach’s Alpha
Level 1				0.989	<0.001	0.997
	1. Serve	2.0 ± 0.0	2.0 ± 0.0
	2. Serve+Counterattack	7.0 ± 0.0	7.0 ± 0.0
	3. Return	7.0 ± 0.0	7.0 ± 0.0
	4. Return+Counterattack	5.9 ± 0.4	6.0 ± 0.0
	5. Attack+Counterattack	8.0 ± 0.6	8.0 ± 0.0
	6. Block+Counterattack	4.7 ± 0.5	5.0 ± 0.5
	7. Push+Counterattack	2.1 ± 0.4	2.0 ± 0.0
	8. Attack after chop+Counterattack	0.0 ± 0.0	0.0 ± 0.0
	9. Chop+ Counterattack	0.0 ± 0.0	0.0 ± 0.0
	10. Others	0.0 ± 0.0	0.0 ± 0.0
Level 2				0.778	<0.001	0.966
	1.1. Fh sidespin serve	2.0 ± 0.0	2.0 ± 0.0
	2.1. Fh sidespin serve+counterattack	2.7 ± 1.0	3.0 ± 0.5
	2.2. Fh backspin serve+counterattack	1.0 ± 0.6	1.0 ± 0.0
	2.3. Bh sidespin serve+counterattack	1.0 ± 0.0	1.0 ± 0.0
	2.5. Bh riverse serve+counterattack	2.4 ± 0.8	2.0 ± 0.5
	3.1. Fh push	1.0 ± 0.0	1.0 ± 0.0
	3.2. Topspin Fh attack	1.0 ± 0.0	1.0 ± 0.0
	3.4. Bh flick	0.7 ± 0.5	1.0 ± 0.5
	3.5. Topspin Bh-attack	2.3 ± 0.5	2.0 ± 0.5
	3.6. Fh flip	2.0 ± 0.0	2.0 ± 0.0
	4.1. Fh push+Counterattack	5.9 ± 0.4	6.0 ± 0.0
	5.1. Fh topspin+Counterattack	3.9 ± 0.4	4.0 ± 0.0
	5.2. Bh attack+Counterattack	2.0 ± 1.2	2.0 ± 1.0
	5.3. Fh attack+Counterattack	1.1 ± 0.4	1.0 ± 0.0
	5.4. Bh topspin+Counterattack	1.0 ± 1.2	1.0 ± 1.0
	6.1. Bh block+Counterattack	3.0 ± 0.0	3.0 ± 0.0
	6.2. Fh block+Counterattack	1.7 ± 0.5	2.0 ± 0.5
	7.2. Fh push+Counterattack	2.4 ± 1.1	2.0 ± 0.0
Level 3				0.456	<0.001	0.792
	2.1.1. Fh sidespin serve+Topspin-fh	0.7 ± 0.5	1.0 ± 0.5
	2.1.2. Fh sidespin serve+Attack-bh	0.4 ± 0.5	0.0 ± 0.5
	2.1.3. Fh sidespin serve +Topspin-bh	0.3 ± 0.5	0.0 ± 0.5
	2.1.4. Fh sidespin serve+Attack-fh	0.1 ± 0.4	0.0 ± 0.0
	2.1.6. Fh sidespin serve +Push-fh	1.1 ± 0.4	1.0 ± 0.0
	2.2.2. Fh backspin serve+Attack-bh	0.1 ± 0.4	0.0 ± 0.0
	2.2.6. Fh backspin serve+Push-fh	0.7 ± 0.5	1.0 ± 0.5
	2.2.8. Fh backspin serve+Push-bh	0.1 ± 0.4	0.0 ± 0.0
	2.3.x. Bh sidespin serve+Others	1.0 ± 0.0	1.0 ± 0.0
	2.5.1. Fh-riverse serve+Topspin-fh	1.3 ± 0.5	1.0 ± 0.5
	2.5.x. Fh-riverse serve+Others	1.1 ± 0.4	1.0 ± 0.0
	4.1.1. Fh push+Topspin-fh	1.4 ± 0.8	1.0 ± 0.5
	4.1.2. Fh push+Block-bh	1.1 ± 0.4	1.0 ± 0.0
	4.1.3. Fh push+Topspin-bh	1.1 ± 0.9	1.0 ± 1.0
	4.1.4. Fh push+Block-fh	0.6 ± 0.8	0.0 ± 0.5
	4.1.5. Fh push-fh+Attack-bh	0.7 ± 1.0	0.0 ± 1.0
	4.1.x. Fh push-fh+Others	0.9 ± 0.9	1.0 ± 1.0
	5.1.1. Fh topspin+Topspin-fh	0.3 ± 0.5	0.0 ± 0.5
	5.1.2. Fh topspin+Attack-bh	1.7 ± 1.3	1.0 ± 1.0
	5.1.3. Fh topspin+Attack-fh	0.7 ± 0.5	1.0 ± 0.5
	5.1.4. Fh topspin+Topspin-bh	1.1 ± 1.2	1.0 ± 1.0
	5.2.1. Bh attack+Attack-bh	0.7 ± 0.5	1.0 ± 0.5
	5.2.2. Bh attack+Attack-fh	1.1 ± 0.7	1.0 ± 0.5
	5.2.3. Bh attack+Topspin-fh	0.1 ± 0.4	0.0 ± 0.0
	5.3.1. Fh attack+Attack-fh	1.0 ± 0.6	1.0 ± 0.0
	5.3.x. Fh attack+Others	0.1 ± 0.4	0.0 ± 0.0
	5.4.2. Bh topspin+Topspin-bh	0.3 ± 0.5	0.0 ± 0.5
	5.4.3. Bh topspin+Attack-bh	0.1 ± 0.4	0.0 ± 0.0
	5.4.x. Bh topspin+Others	0.6 ± 0.8	0.0 ± 0.5
	6.1.2. Bh block+Topspin-fh	0.1 ± 0.4	0.0 ± 0.0
	6.1.3. Bh block+Attack-fh	1.9 ± 0.4	2.0 ± 0.0
	6.1.5. Bh block+Attack-bh	0.4 ± 0.5	0.0 ± 0.5
	6.1.x. Bh block+Others	0.6 ± 0.5	1.0 ± 0.5
	6.2.1. Fh block+Block-bh	0.1 ± 0.4	0.0 ± 0.0
	6.2.x. Fh block+Others	1.6 ± 0.8	2.0 ± 0.5
	7.2.1. Fh push+Topspin-fh	1.0 ± 0.0	1.0 ± 0.0
	7.2.2. Fh push+Block-bh	0.1 ± 0.4	0.0 ± 0.0
	7.2.x. Fh push+Others	1.3 ± 0.8	1.0 ± 0.0

Note: M—arithmetic mean, SD—standard deviation; Me—median, SQ—quartile deviation; Fh—forehand; Bh—backhand.

. The proportions in the number of points scored and applied strokes by this player are very similar to the one discussed earlier, although he scored fewer points directly from the serve (only two).

There were no significant differences between the judges’ assessments in terms of individual observations (χ² (63.6) = 0.709, p = 0.994), while the data for each expert differed according to the three levels of the experiment (K-W (63.3) > 25.94, p with Bonferroni correction <0.001). Post hoc tests showed differences between Level 1 and Level 3 and Level 2 and Level 3 for all experts; there were no differences between Level 1 and Level 2.

The reliability of the judges’ grades was checked got each of the analyzed levels of the game. In the first two levels, the agreement of assessments was good (W > 0.77, p < 0.001), while it was slightly lower in the third level (W = 0.46, p < 0.001). A very high internal consistency was obtained (Cronbach’s alpha > 0.90) in the first two levels, and a relatively lower but acceptable consistency was obtained at Level 3 (Cronbach’s alpha = 0.79).

4. Discussion

The aim of this study was to assess the consistency of judges’ (experts’) assessments in order to determine the reliability of our own modification of Wu Huanqun’s game analysis method. An analysis was made of the use of individual strokes and the combinations of strokes that won in the men’s final of the World Table Tennis Championships 2019. As a result of the method used, what strokes and combinations of strokes participating players scored with were assessed. The observations made allow for the conclusion that Ma Long, the winner of this competition (2019 World Champion) scored the most points directly with service and directly with the return of service. This may prove the importance of these strokes in table tennis, and it highlights the need to work on these strokes. Likewise, Ma Long won a lot of actions by applying the winning combination of serve plus the third ball counterattack and the combination of return plus the next fourth ball counterattack. In these elements, he had the greatest advantage over his opponent. This was characteristic, as the second of the observed players won his points in similar fragments of the game (except for points scored directly with serves; in these fragments, Ma Long had a significant advantage). This also confirms the observations of other authors, specifically regarding the way that, in table tennis, most points are scored (at the global level) with the first two strokes [9,22]. This is a clear indication of how training may need to emphasize these factors when working with table tennis players. Service (especially the forehand side-spin serve), return (especially the forehand push), the counterattack after service, and counterattack after service return—these are technical and tactical elements, which, as the analysis performed in this paper showed, play a special role in relation to the player’s performance.

The very high value of Cronbach’s alpha, exceeding the maximum value of 0.90 suggested by Tavakol and Dennick [21], may be caused by the redundancy of positions. This would mean that, in practice, the number of expert evaluators could be limited in this procedure.

Analyses conducted separately at subsequent levels allowed the new observations to be compared with the reliability of the experts’ assessments. A slight reduction in compliance and internal consistency in the subsequent stages of “judging” may be an additional indicator of a loss of credibility. At the second and third levels of the spreadsheet observation, the value of the compliance index is slightly lower, but still, in the adopted interpretation, these values indicate a significant agreement between experts’ assessments. The differences between the experts’ judgments are that they classify the same strokes (combinations of strokes) into different categories, e.g., sidespin serve as backspin–nonspin, fast attack as topspin, block as attack, flip as attack, etc. In particular, the slight discrepancies between experts in the results of Falck’s game analysis may have resulted from the specific style of the player’s fast attack on the forehand side (with the use of short pimples rubber). It seems that these differences may disappear after learning how to use the method and practicing it. Experts, according to oral declarations, needed between 4 and 12 h to complete the task. They reported that, when performing the analysis, they, again and again, made corrections to the previously classified data. Apparently, the method requires practice. Then, the analysis will become more reliable. Nevertheless, the high and significant values of the statistical analyses carried out in the study indicate the high reliability of the observation method.

Therefore, it seems that the described method of observation and analysis of the game is an appropriately reliable and objective tool for obtaining very important information about the technical and tactical structure of the gameplay of the best players in the world, as well as about the combinations of strokes they use that result in winning an action (scoring a point). It can be used in the analysis of player gameplay at any sports level in order to monitor, diagnose, and define training plans.

It also seems that the proposed method of observation will lead ambitious trainers and training staff to colloquial and intuitive (sometimes accurate, but usually based on false assumptions and fantasies) conclusions. Therefore, the systemic application of the proposed method of observation and analysis of the game may lead to an increase in the level of training in each table tennis club, center, or federation, as well as the implementation of an evidence-based training control process. This requires further research and verification.

In the course of interacting with the experts asked to perform the analysis, we discovered that they encountered various difficulties. When conducting the analysis at Levels 2 and 3, different interpretations of the same action appeared. Therefore, the following additional instructions were created, facilitating the work of the analyzer:

• The analysis should be performed with the greatest care. Some mistakes seem to be due to inattention or fatigue. For example, not counting the strokes, not noticing whether the observed player served or received the serve, etc.
• Identify the technique in detail; for example, the fh-side-spin service is visible for a fraction of a second when the racket is in “face down” contact, while with the backspin–nonspin service, the racket is flat. In the analysis, imperfect material is used; most often, these are videos available on the Internet, lack high resolution, are not recorded with three cameras, etc. Therefore, please view the video clip several times and sometimes in slow motion.
• Remember that game fragments 1 and 3 are serves and returns. These are one-stroke actions. Game fragments 2 and 4 are two-stroke actions: serve and counterattack and return and counterattack. Fragments 5–9 are long actions (rallies) with three strokes or more. Attack–counterattack, block–counterattack, and push–counterattack are long actions (three strokes or more). In the analysis of long actions, we are interested in the last two strokes in each action.
• Note if there are two-stroke actions among the long actions (5–9) (for example, return and counterattack). Are there only “edges”, “nets”, missed own-serves, illegal serves called by the referee, etc., among the “10. Others” actions? If not, these actions must belong to categories 1–9.
• In attack–counterattack, the penultimate hit is the attack, and the last hit can be attacking (attack, topspin), defensive (block, “fishing”, lob, or chop), or push.
• You need to distinguish between a fast attack and a topspin. Each attacking stroke has an “impact force” and “friction force” component. Both the half-volley and the smash have a certain rotation. Perhaps someday we will have technology with which we will be able to determine the power of rotation or the speed of a flying ball “live”. Currently, we have to observe the direction of force use, listen to the impact/rub, etc. If the emphasis of the strike is on speed, assume it is a fast attack; if the emphasis is on rotation, assume it is a topspin.
• In a block–counterattack action, the penultimate hit is a block (“fishing”, lob) and the last hit is attacking (attack, topspin), blocking (“fishing”, lob), or pushing.
• You must distinguish between attack and block. If the opponent is attacking with great force or rotation, and we “borrow” the force - we play “block”. A block may have a downspin (this is not a chop defense), but it can also have a bit of a topspin (a kuaidai is not a topspin). Attacking is giving the ball its own strength; blocking is “borrowing” the strength of the ball that arrives.
• In a push–counterattack action, the penultimate hit is a push, and the last can be a push, block (“fishing”, lob), or attacking hit (attack, topspin). An error may also result from the fact that, for example, some actions, e.g., “4. Return–counterattack (e.g., push + push) were classified as long actions, e.g., as “7. Push–counterattack”.
• “Chop” should be reserved for a “Chop defense” against the opponent’s attack (or block). A “Push”, on the other hand, is a “shortcut” after a short serve, after a push by the opponent, etc. For example, a long, slow, spinning push is not a chop. A push is a hit with more or less bottom spin; it is not defensive, but rather, a transitory stroke. If so, you should categorize it in the “7. Push–counterattack” category.
• Stick to the established definitions we propose here, for example, even if you prefer to call a topspin a “flip with topspin” or a “kuaidai”. Classify an attacking short ball attack as a flip, not a topspin. Likewise, for borrowing power strokes, classify them as blocks. For example, with side-spins or chop-blocks, classify them as blocks, not as chops. A kuaidai is classified as a block, not a topspin.

More than 400 winning strokes and stroke combinations were identified in a pilot study of the world’s top players [13,14,15]. If they were included in one sheet, it would become very large. Therefore, we can simplify it. The worksheet on Level 1 may be the same for each game, but developing an analysis of Levels 2 and 3 should be generative and relate to the knowledge of previous analysis. Knowing which strokes and combinations of strokes occur most frequently for a given player or a representative of a given type of play, we can prepare a new sheet (containing the most frequently won strokes and combination of strokes). This is what generativity is all about, e.g., in the case of a defensive player, we should categorize the most common strokes and combinations as “9. Chop–counterattack”. Another example would be the observation sheet of an attacking player versus a defensive player. In this case, the observation sheet at Levels 2 and 3 should include the expanded combinations found in “8. Attack-against-chop and counterattack”. Each subsequent analysis can be based on the results of previous observations. With a new observation sheet, especially at Levels 2 and 3, we can add the new most frequent hits and hit combinations as needed, along with new discoveries. For example, taking into account all the types and styles of play and over 100 techniques (i.e., for the shakehand and penhold grips), a trainer/analyst must identify them efficiently. The trainer/analyst should be knowledgeable about each playstyle, so he needs complete technical–tactical training in other playstyle types than his own. China and other leading Asian countries have the greatest knowledge and experience in this field [22,23]. This is the basis for creating the so-called “small world” and the possibility of implementing so-called “system training”.

This method has some limitations. It is time-consuming, requiring several hours to fully analyze one match. Apparently, it can also be too difficult at times, even for table tennis experts. It must also be learned, requiring analysts to quickly learn to distinguish between different strokes (more than 100 in shakehand and penhold grip) and combinations of strokes (more than 400). Computer versions of the analysis and the further development of already-existing and used technology, for example, in Asia (but not readily available for all) may significantly improve the process of observation and analysis of the game in table tennis [24,25]. An example is a computer program for game analysis according to Professor Wu Huanqun’s method [26]. One could probably achieve much higher reliability if one had the highest quality footage, with the game recorded from different perspectives. One method that may revolutionize the observation and analysis of the game in the future is the telemetric identification of the quality elements of individual shots and their combinations (speed, rotation, placement, power, and confidence). It seems that the further development of this and other methods of observation and analysis of the game may also be strongly related to the creation of organizational, training, and research staff (cooperation between organizers, trainers, scientific researchers, IT specialists, statisticians, etc.). The creation of a central training and research center in our country, in accordance with the assumption of the National Table Tennis Development Program 2018-33, should also take into account the full application and further development of methods of observation and analysis of the game of table tennis and the creation of an appropriate team of people.

This work can also be treated as an introduction to evaluating the validity of the method. By determining the compliance and internal consistency of expert assessments in the applied observation method (and, thus, its high reliability—in this case, “inter-rater reliability”), we have demonstrated an important basis for directing future research on the validation of this method. Making more observations and further evaluations of the method validation can and should be an issue for further research.

5. Conclusions

The performed statistical analysis indicates a high agreement between the experts’ opinions and high compliance and internal consistency between their assessments, which was taken as a measure of the high reliability of the described method of observation. At the second and third levels of the observational spreadsheet, the value of the compliance index was slightly lower. Thus, the proper use of the method involves the process of learning it, therefore developing the appropriate knowledge and skills of the trainer/analyst. Therefore, cognitive development is required in the process of applying the method of observation and analysis. The proposed method of observing and analyzing the game is a good enough tool to build quantitative game models: what is the structure of winning shares, how does the structure of winning shares of men and women change, what is the structure of winning shares for different types and styles of play, how has the structure of winning shares changed on over the years, etc.