*3.6. Summary*

Depending on the targeted application, researchers have explored different sensing modalities to accomplish their tasks in HAR. Table 2 summarized the mainstream of the sensing modalities and compared them with aspects of cost, power consumption level, working type (active or passive), privacy concern, computing load, typical applications, and their critical advantages and shortcomings. We also supplied some of the publicly available datasets of each sensing modality for HAR tasks in the table, so that readers can check and have a better understanding of the data properties of each sensing technique, or try their own mining approach on the dataset. The cost and power consumption express the practicability of a sensing modality, such as the IMU as a low-cost and low-power-consumption approach, which is the most widely explored aspect of HAR tasks. The compute load and robustness, ranging from high to low, were ranked with specific references. Computations that require large memory (over hundreds of megabytes) and complex instruction (such as multiplication of float point data) are regarded as having a high compute load. A low compute load needs simply a few instructions for one inference on weak devices such as the micro-controller. High robustness indicates that the signal could hardly be interfered with

by surroundings, such as the gravitational field. Bluetooth signal strength, as an example, could be easily affected by a variation in the nearby environment. The typical application lists the activities coarsely at a high level, such as the activity of daily living (ADL), which includes all fundamental actions of a human in everyday life, such as sleeping, eating, dressing, etc. The positioning includes the location of the whole body and the body part such as the hand and finger. Gesture Recognition implies gestures performed by hand, finger, arm, etc. Active/passive sensors indicate the complexity of the sensing modality because of the existence of the signal sources. A privacy-respect sensor does not abstract identity-sensitive messages from users, thus being more acceptable. The computing load and robustness show the sensor's working performance and are categorized into three levels: "low", "medium", and "high". Depending on the usage scenarios, each sensing modality could be deployed targeting different tasks among "where", "what", and "how". A passive electric field, as an example, can be used for both positioning and action sensing.

IMU sensor and optic approaches (mainly video-based) are the two most popular sensing modalities in the community, since the IMU sensor is pervasively deployed in smart devices and outperforms in power consumption, cost, size, and the visual modality can supply high accuracy for activity recognition benefiting from the advanced deep neural network models for feature abstraction. They both are utilized to target a much wider range of human activity recognition tasks than other sensors. However, there are still certain limitations, such as that they both suffer from computational load. Especially for the vision-based approach, which deals with 2D or 3D high resolution and high frame data stream with hundreds of thousands of conventional operations challenging the hardware resources, the computational load is high compared with other sensing modalities. Since the images from a video capture massive identity messages, the privacy issues need to be considered. The IMU sensors face accumulated errors, which results in the configuration demand for each new start for positioning applications with the demand of high accuracy.

Wave-based sensing modalities (RF waves and acoustic waves) are active approaches demanding signal sources from the sensing system and are mainly used for ambient intelligence. The corresponding systems are generally weak in robustness since the wave signal could be affected by the multipath effect (except for the UWB) and environmental noises. However, they are particularly efficient in privacy-respect scenarios since no other information beyond the wave property is collected. The cost and power consumption of such systems are much higher than the IMU-based solution, but still lower than the visual approach.

The electrophysiological signals (ExG) are perceived mostly by devices with highresolution analog-to-digital chips for healthy monitoring such as mental state, stress level, sports quality evaluation, etc. The cost of such a system is relatively high compared with IMU and most field-based approaches. Since the signal sensing units are mainly at the chip-level design, the power consumption is an obvious advantage of those approaches. Depending on the channel numbers, the computational load of electrophysiological signals is distinct. The ECG signal, as an example, a simple rule-based approach that needs only a few computer instructions, can be used to detect the critical features from it efficiently. The EEG signal, on the other hand, requires a more complex algorithm to abstract the features from multiple channels to uncover the messages behind it.

Pressure sensing is versatile since the sensing unit (mainly composed of conductive layers) is highly customizable. Since the weight signal perceived from such a sensing system is quite straightforward, the detection accuracy of a certain human actions is high. However, maintaining such systems is costly because of the deployment complexity and the limited lifetime caused by the long-term stressful contact.


**Table 2.** Sensing techniques in HAR tasks.

A magnetic field is a robust distance-based approach that can deliver reliable distance information with a lower computational load. The approach is low-cost and wearable (after minimizing), without limitation of multipath and line-of-sight. More importantly, it can be used for positioning in the underwater environment, which blocks most of the positioning techniques because of the quick attenuation of the adopted medium (such as RF-signal) in water. However, the detection range is limited by a few meters with the active magnetic field and a few decimeters with the passive magnetic field.

The electric field has recently become a novel sensing approach for HAR tasks, distinguished by its ability in full-body motion sensing and environmental electric sensing. It also enjoys the advantages of low power consumption and wearability. Since electrons exist anywhere in the environment where people live, including the human body, the body's motion will deform the distribution of the electric field. Therefore, human activity could be deduced by perceiving the electric field variation, either on the environmental side or on the body side. However, the environmental noise is a big challenge for electric field-based sensing and is hard to overcome because of the pervasiveness of the surrounding objects acting as noise sources.
