**1. Introduction**

In the future information systems featuring rich sensing, wireless inter-connection, and data analytics, which are usually denoted as Internet of Things (IoT) and cyber-physical systems (CPS) [1,2], human motions and activities can be recognized to perform intelligent actions and recommendations. The monitoring, recognition, and in-depth analysis of human gesture, posture, gait, motion, and other daily activities has received growing attention in a number of application domains, and significant progress has been made, attributing to the latest technology development and application demands [3]. As a result, new sensing techniques and mathematical methods are dedicated to the numerical analysis of the posture and activities of human body parts, and plenty of related studies are found in the literature.

The significant progress in human activity recognition (HAR) and motion analysis may be driven by the following powers: (1) The new powerful information systems require more efficient

human–machine interactions for potential intelligence; (2) the development of integrated circuits (ICs) and circuits fabrication techniques has prompted the emergence of low-power, low-cost, miniature-sized, and flexible wearable sensing devices; (3) there have been recent advances in data analytics techniques, including filtering and machine learning techniques in dealing with data processing and classification; and (4) some new emerging applications fields, including wearable sensor-based medical care, sports analysis, physical rehabilitation, assisted daily living (ADL), etc., have also provided new opportunities for the techniques to revolutionize the traditional techniques in these fields. As a result, much research effort has been devoted to this area, and a number of technical solutions have been proposed.

There have been plenty of innovative studies that introduce novel sensing techniques to human activity recognition and motion analysis. Human–machine interaction (HMI) may be a critical area, where human gesture, posture, or motion can be recognized for the efficient interaction between machines covering a wide field of applications, including human–object interaction, virtual reality, immersive entertainment, etc. [4–6]. Human activity recognition and motion analysis has also been an effective way for sports analysis evaluation, while peer investigations are identified for golf swing analysis [7], swimming velocity estimation [8], and sports training [9]. Medical care has become a new research interest in the recent years, where precise and quantitative representation of human motion can help physicians in diagnosis, treatment planning, and progress evaluation; typical applications include gait analysis for stoke rehabilitation therapy [10], clinical finger movement analysis [11], Parkinson's disease treatment [12], etc. In addition, many other innovative applications are also found in assisted daily living, elderly and children care, etc. It is easy to see how through the applications, the innovative usage of the mentioned techniques has revolutionized the traditional approaches and resulted in convenience, efficiency, and intelligence that have never been seen before.

Many different techniques are employed to obtain the raw sensor data for monitoring human activities. The most commonly used techniques are the smartphone built-in inertial measurement units (IMUs) and optical cameras [13,14]. Since smartphones have almost become a must-have assistant for people's everyday life, a lot of human body and human activity related studies are carried out by taking advantage of smartphones. An optical camera, as a widely used sensing device, is a mainstream solution for human activity recognition. A depth camera, due to the one added dimension of depth information, has its unique strength compared to normal optical cameras [15]. Then, electromagnetic waves, such as frequency modulated continuous wave (FMCW) radar [16], impulse radio ultra-wide band (IR-UWB) radar [17], WiFi [18], and capacitive sensor array [19], featuring non-contact and non-line-of-sight (NLOS) traits, have also been introduced to human activity recognition. For the sensing techniques, accuracy, safety, privacy protection, and convenience are key factors for their applications. The low-cost, lightweight, and miniature-sized wearable IMU sensor devices have been prevalent techniques employed by peer investigations for human activity recognition, and the integration of two or more techniques may result in a better performance [20]. In addition, discrete sensing devices may be combined to constitute a wireless body area network (WBAN) for the establishment of wearable computing in many high-performance systems [21].

In addition to sensing techniques and networking techniques, data processing techniques including filtering, segmentation, feature extraction and classification are also indispensable enablers. For the pre-processing of sensing data, moving average filter (MAF), Kalman filter (KF) and complementary filter (CF) are the common approaches [22–24]. For classification, different machine learning algorithms have been used to create recognition models such as decision tree (DT), Bayesian network (BN), principle component analysis (PCA), support vector machine (SVM), artificial neural networks (ANNs), logistic regression, hidden Markov model (HMM), K-nearest neighbors (KNN) and deep neural networks (DNNs) [25–28]. Deep learning methods, such as the convolutional neural network (CNN) and recurrent neural network (RNN), due to their performance and wide acceptance in data analysis, have also recently gained interest in being used as tools [29–31]. The data processing techniques play an important role in guaranteeing the efficiency and accuracy of the recognition and analysis of human activities.

Admittedly, due to the potential in the different application fields, there has been a grea<sup>t</sup> demand for accurate human activity recognition techniques that can lead to the convenience, efficiency, and intelligence of information systems, new opportunities of medical treatment, more convenient daily living assistance, etc. However, there lacks a timely report on the recent contributions of recent technical advances with an in-depth analysis of the underlying technical challenges and future perspectives. Motivated by both the grea<sup>t</sup> demand for a more efficient interaction between human and information systems and the lack of investigations about the new contributions to knowledge in the field, this paper aims to provide a comprehensive survey and in-depth analysis of the recent advances in the diverse techniques and methods for human activity recognition and motion analysis.

The rest of this paper is organized as follows: Section 2 summarizes the innovative application regarding human activity recognition and motion analysis; Section 3 illustrates the fundamentals, including the common methodology, the modeling of human parts, and identifiable human activities. Sections 4 and 5 present the novel sensing techniques and mathematical methods, respectively. Then, Section 6 gives the underlying technical challenges and future perspectives, followed by Section 7, which concludes the work.

### **2. Latest Progress in HAR-Related Applications**

The past few decades have witnessed unprecedented prosperity of electronics techniques and information systems, which have resulted in a revolutionary development in almost all aspects of technological domains, including aeronautics and astronautics, automotive industry, manufacturing and logistics, consumer electronics and entertainment, etc. Human activity recognition and motion analysis, due to its potential in wide areas of applications, has attracted much research interest and made remarkable progress in recent years. This section gives an overview of the latest technical progress and a summary of the application domains of human activity recognition and motion analysis.

### *2.1. Overview of the Latest Technical Progress*

An overview of the latest technical progress of human activity recognition is given in Figure 1. The technical progresses of HAR are found to focus on the following aspects: new sensing device and methods, innovative mathematical methods, novel networking and computing paradigms, emerging consumer electronics, and convergence with different subject areas.

**Figure 1.** Overview of the latest technical progress of human activity recognition (HAR) investigations.

1. New sensing devices and methods: The acquisition of raw data is the first step for accurate and effective activity recognition. The cost reduction of the electronic devices has accelerated the pervasive sensing and computing systems, such as location and velocity tracking [32]. On account of the sensing techniques, many new techniques that were previously not possible for their cost, size, and technical readiness are now introduced for human activity related studies in addition to the traditional video cameras (including depth cameras), FMCW radar, CW-doppler radar, WiFi, ultrasound, radio frequency identification (RFID), and wearable IMU and electromyography (EMG) sensors, etc. [15–18]. Among the above candidates, FMCW radar, CW-doppler radar, WiFi, ultrasound, and RFID are both NLOS and contactless. Wearable IMU is NLOS and body-worn, for which the applications are not limited to specific areas. The micro-electro-mechanical system (MEMS) IMUs, due to their advantages in being low-power, low-cost, miniature-sized, as well as able to output rich sensing information, have become a dominant technical approach for HAR studies [33,34].


HAR has also been introduced in sports for sports analysis for the purpose of enhancing athletes' performance [7,8]. HAR-assisted daily living is another example as a field of application, where power consumption, home appliance control, and intelligent recommendations can be implemented to customize the living environment to people's preferences [2,47].

## *2.2. Widespread Application Domains*

Driven by the subtantial technical progress in the related fields, HAR has been extended to a wide spectrum of application domains. Since human involvement is the most critical part for many information systems, the introduction of HAR can potentially result in greater efficiency and intelligence of the interaction between human and information systems, which also creates new opportunities for the human body or human activity related studies and applications. The fields of applications/domains are summrized as follows:


Although many new techniques and methods have been introduced in HAR for different applications and remarkable progress has been made, there is still large room for improving the performance of the methods and systems. More convenient, lightweight, powerful computation sensing devices and systems, more accurate classification algorithms, and some application-oriented studies will still continually gain attention and play a more important role in various information systems and in people's daily lives.
