Lightweight Stacked Hourglass Network for Human Pose Estimation

Featured Application

The proposed lightweight hourglass network can be applied as an alternative to existing methods that use the hourglass model as a backbone network.

Abstract

Human pose estimation is a problem that continues to be one of the greatest challenges in the field of computer vision. While the stacked structure of an hourglass network has enabled substantial progress in human pose estimation and key-point detection areas, it is largely used as a backbone network. However, it also requires a relatively large number of parameters and high computational capacity due to the characteristics of its stacked structure. Accordingly, the present work proposes a more lightweight version of the hourglass network, which also improves the human pose estimation performance. The new hourglass network architecture utilizes several additional skip connections, which improve performance with minimal modifications while still maintaining the number of parameters in the network. Additionally, the size of the convolutional receptive field has a decisive effect in learning to detect features of the full human body. Therefore, we propose a multidilated light residual block, which expands the convolutional receptive field while also reducing the computational load. The proposed residual block is also invariant in scale when using multiple dilations. The well-known MPII and LSP human pose datasets were used to evaluate the performance using the proposed method. A variety of experiments were conducted that confirm that our method is more efficient compared to current state-of-the-art hourglass weight-reduction methods.

1. Introduction

Human pose estimation is a fundamental method for detecting human behavior, and it is applied in virtual cinematography using computer graphics, human behavior recognition, and building security systems. Joint position varies greatly depending on a variety of factors, such as camera angle, clothing, and context. The traditional method estimates or tracks the human pose using additional equipment, such as depth sensors. However, with the advent of convolutional neural networks (CNNs), it is possible to efficiently infer the entire spatial feature from a single image without the need for additional equipment. Accordingly, many studies on human pose estimation using convolutional neural networks are currently underway, and these have achieved great results in terms of their accuracy [1,2,3].

The stacked hourglass network [2] is one of the best-known methods for resolving performance problems in human pose estimation. It has a stacked structure of hourglass modules composed of residual blocks [4]. Since the hourglass network performs promisingly in resolving the human pose estimation problem, a number of studies have used it as a backbone or modified the original hourglass network to improve performance [5,6,7,8,9,10]. Ning et al. [11] developed a stacked hourglass design and inception-resnet module with encoded external features for human pose estimation. Ke et al. [12] introduced the multiscale supervision network with a multiscale regression network using the stacked hourglass module to increase robustness in keypoint localization for complex background and occlusions. Zhang et al. [13] suggested a method to overcome information loss in a repetitive cycle of down-sampling and up-sampling in a feature map. They used a dilated convolution and skip connections applied to a stacked hourglass network to optimize performance while adding extra subnetworks and large parameter sizes.

However, the residual block used in the hourglass network consists of a relatively small kernel with a fixed size. This may not be conducive to extracting the relationship between joints in the entire human body, and performance may significantly deteriorate depending on the size of the person in the input image. Additionally, since the network is stacked, very large memory capacity and computational powers are required. In this paper, we focus on how to reduce the parameter size and achieve the best performance, while others [5,6,7,8,9,10,11,12,13,14] deploy an extra subnetwork or layers to the stacked hourglass network. The purpose of this paper is to illustrate an optimized hourglass network with minimized parameter size without sacrificing the quality of the network.

Previous studies investigating human pose estimation [2,6,15] have confirmed that the size of the convolutional receptive field is a major factor in understanding the whole human body. If the receptive field is too small, it is difficult for the network to understand the relationship between each joint. Conversely, if it is too large, information that is not relevant to pose estimation is used for calculation, leading to impaired performance. In addition, if the convolution operation using a large kernel size is used several times to increase the receptive field, the size of the network may be too large.

The goal of this study is to improve the efficiency of the stacked hourglass network [2] in human pose estimation. We propose a method for designing an efficient hourglass network that is lightweight and greatly reduces the number of network parameters. We also propose a residual block that, by expanding the convolutional receptive field of the residual block, enables performance to be maintained on a multiscale basis through multidilation. To verify the performance, we used the MPII dataset [16] and Leeds Sports Poses (LSP) dataset [17], which are widely used human pose estimation datasets, and demonstrated the effectiveness of our approach through various experiments. In summary, our main contributions are threefold:

• A proposed stacked hourglass network structure improves performance in human pose estimation with fewer parameters (Section 3.1, Section 3.2.1 and Section 3.2.2).
• A new structured residual block, known as a multidilated light residual block, which expands the receptive field of convolution operation, effectively represents the relationship of the full human body, and supports multiscale performance through multiple dilations (Section 3.2).
• An additional skip connection in an encoder of the hourglass module that reflects the low-level features of previous stacks on a subsequent stack. This additional skip connection improves performance in pose estimation without increasing the number of parameters. (Section 3.1).

2. Related Work

2.1. Network Architecture

The structure of the original hourglass network is shown in Figure 1. The network consists of stacked hourglass modules. An hourglass module is composed of residual blocks [4], and there is a skip connection between each stack. Each module has an encoder–decoder architecture. Loss can be calculated using heat maps obtained from each stack, and the network can perform more stable learning by adjusting to repeated predictions; this process is known as intermediate supervision. The hourglass network has been used as a backbone network for many other studies that have since been shown to perform efficiently to overcome human pose estimation problems. We designed a new residual block to reduce the parameters and improve the performance of the network. In addition, we propose a new network architecture based on an additional skip connection.

2.2. Residual Block

An hourglass module basically consists of residual blocks [4]. In [8,18], a hierarchical structure of residual blocks was proposed, and the convolution operation was binarized to increase the efficiency of the network. Compared to problems such as object detection, a key problem in human pose estimation is analyzing the full human body by extracting global spatial information. In [9], performance was improved by expanding the receptive field of convolution in residual blocks. Researchers in [5,9] also proposed a multiscale residual block model.

2.3. Human Pose Estimation

It has been well established that convolutional neural networks (CNNs) represent a significant step forward in recognizing spatial features from images. Accordingly, numerous studies have been conducted that use CNNs in human pose estimation where spatial information recognition is critical. The researchers in [19] made the first attempt to use CNNs for human pose estimation problems and showed a dramatic improvement in performance compared to traditional computer vision methods, such as [20,21]. Initial pose estimation methods using CNNs [19,22,23] predicted the coordinates of joints using CNNs and fully connected layers (FCLs). In [3], a method using a heat map generated by a CNN was proposed; this is currently used in most human pose estimation research [1,2,8,15,24,25,26,27].

3. Proposed Method

3.1. Network Architecture

Several studies have confirmed that the encoder–decoder architecture makes the network lighter and improves performance [28,29,30]. The hourglass network [2] used in our study is a productive approach to solving problems in human pose estimation because the network is able to learn more complex features by stacking modules. An encoder first extracts features by reducing the image resolution, while a decoder increases the image resolution and reassembles features. In an hourglass network, the encoder function is connected to the decoder using a skip connection so that the decoder can restore features well. According to recent work [31,32], extracting features is a more important process than simply restoring them. An hourglass network is structured so that input from a previous stack ($n-1$) is reflected in the current stack ($n$), in addition to the output of the previous stack ($n-1$) via the skip connection, as shown in Figure 1. In this structure, only relatively high-level features reconstructed by the decoder are reiterated in the next stack. Since our goal is to resolve this problem while also making the network lighter, we propose a method that enhances feature extraction performance while requiring minimal modification.

Figure 2 shows the proposed hourglass module. A feature extracted by the encoder is transferred to the next stack by the simple addition of parallel skip connections, as shown in Figure 2; this does not require a significant increase in computation. The proposed structure improves the encoder’s extraction performance by transferring features to subsequent stack encoders. This structure produces better performance than the original hourglass network. An architecture with additional skip connections improves performance by increasing the performance of the encoder, even though the size of the network itself remains substantially unchanged.

3.2. Residual Block Design

3.2.1. Dilated Convolution

In human pose estimation, it is important to increase the receptive field so that the network can learn to recognize the features of the full human body. However, if the kernel size is increased to widen the receptive field, the computational cost also increases. Since our goal was to design an efficient hourglass network, we constructed a residual block using dilated convolution [33]. The number of parameters in the standard convolution is shown in Equation (1), where the kernel size is $K$, the input channel size is $C$, and the output channel size is $M.$ If the size of the input and output is the same as $H\times W$, the required computational cost is shown by Equation (2):

$$\# param=K^{2}CM$$

(1)

$$Computational Cost=K^{2}CMHW$$

(2)

In the case of the dilated convolution, if the kernel size remains the same, the number of parameters and the computational cost remain the same as the standard convolution; however, the receptive field is wider, depending on the dilation size $D$. For dilated convolutions with a $3\times 3$ kernel size when $D_1=2$, the kernel size and computational cost are the same as for the $3\times 3$ standard convolution, but the receptive field is the same as for the $5\times 5$ standard convolution. Additionally, as shown when $D_1=2 \mathrm{and} D_2=3$ in Figure 3, dilated convolution has zero padding inside the kernel; thus, the computational cost is slightly lower than it is for standard convolution with the same kernel size.

3.2.2. Depthwise Separable Convolution

We used depthwise separable convolution, as proposed in [34], with dilated convolution for our residual block. Depthwise separable convolution performed pointwise ($1\times 1$) convolution after depthwise convolution, which was performed using an independent kernel for each channel. Figure 4 shows the concept of the depthwise separable convolution. This method shows lower performance than it does with standard convolution, but the number of parameters was significantly reduced, and the computation speed was faster. We interpolated the reduced performance due to depth-size separable convolution into a new residual block with dilated convolution.

3.2.3. Proposed Multidilated Light Residual Block

The original stacked hourglass network constructed an hourglass module using a preactivated residual block, as shown in Figure 5a. The structure of the preactivated residual block is [ReLU→Batch Normalization→Convolution]; this is unlike that of a conventional residual block, which is designed as [Convolution→Batch Normalization→ReLU]. This structure is advantageous in building a deep network and improves the training speed [35]. However, residual blocks were originally designed for image classification or object detection problems (in which it is important to learn local features) where the convolutional receptive field is relatively small. Moreover, in the deep network architecture, although a bottleneck structure is used to reduce the number of parameters and computation cost, the residual block with a bottleneck structure is still large when designing a stacked multistage network, such as an hourglass network. Therefore, there is a need for a residual block with a new structure that can improve performance in human pose estimation by reducing the size of the network and expanding the receptive field.

In this study, to design a residual block with a new structure capable of solving the aforementioned problems, experiments were performed on residual blocks with various structures. Figure 5b shows a structure in which the middle layer of the preactivated residual block shown in Figure 5a has been changed to a depthwise separable convolution. Using this residual block, we carried out experiments to observe the effect of the depthwise separable convolution on the network size and performance in human pose estimation. Since preactivated residual blocks are bottlenecks with the first and last layers having $1\times 1$ convolutions, it did not make sense to reduce the number of parameters by changing the layer to a depthwise separable convolution. The researchers in [34] declared that if a nonlinear function is used between a depthwise convolution and a $1\times 1$ convolution, the performance is significantly reduced. Therefore, all of the depthwise separable convolutions used in this paper consist of a structure that does not use an activation function between the depthwise convolution and $1\times 1$ convolution, such as [ReLU→Batch Normalization→Depthwise Convolution→1 × 1 Convolution].

To evaluate the effect of the bottleneck structure of residual blocks while using a depthwise separable convolution, we designed the new module shown in Figure 5c. Figure 5c is a modified structure of Figure 5b, where we changed the standard convolutions of the first layer [256→128, 1 × 1] and the last layer [128→256, 1 × 1] into depthwise separable convolutions of [256→128, 3 × 3] and [128→256, 3 × 3], respectively. Figure 5d is our proposed multidilated light residual block, where a residual block of a new structure was used for improving the performance and reducing the number of parameters.

Table 1. Proposed multidilated light residual block structure.

Type		Input Size (C, H, W)	Output Size (C, H, W)	Kernel Size	Dilation Scale (D)
$I$	INPUT	(256, H, W)	-	-	-
Layer 1
ReLU		(256, H, W)	(256, H, W)	-	-
Batch Norm		(256, H, W)	(256, H, W)	-	-
Conv		(256, H, W)	(128, H, W)	$1\times 1$	1
Layer 2
$B_1$	ReLU	(128, H, W)	(128, H, W)	-	-
	Batch Norm	(128, H, W)	(128, H, W)	-	-
	Conv	(128, H, W)	(128, H, W)	$3\times 3$	1
$B_2$	ReLU	(128, H, W)	(128, H, W)	-	-
	Batch Norm	(128, H, W)	(128, H, W)	-	-
	Depthwise Conv	(128, H, W)	(128, H, W)	$3\times 3$	2
	Conv	(128, H, W)	(128, H, W)	$3\times 3$	1
$B_3$	ReLU	(128, H, W)	(128, H, W)	-	-
	Batch Norm	(128, H, W)	(128, H, W)	-	-
	Depthwise Conv	(128, H, W)	(128, H, W)	$3\times 3$	3
	Conv	(128, H, W)	(128, H, W)	$3\times 3$	1
ADD ($B_1+B_2+B_3$)		(128, H, W)	(128, H, W)	-	-
Layer 3
$B_4$	ReLU	(128, H, W)	(128, H, W)	-	-
	Batch Norm	(128, H, W)	(128, H, W)	-	-
	Conv	(128, H, W)	(256, H, W)	$1\times 1$	1
ADD ($B_4+I$)		(256, H, W)	(256, H, W)	-	-
OUTPUT		-	(256, H, W)	-	-

below shows the detailed structure of the proposed residual block.

In this study, the multidilated light residual block was used to greatly lighten the stack hourglass network, while multidilated convolution was used to expand the receptive field to increase the immutability of scale, resulting in improved performance in human pose estimation.

4. Experiments and Results

4.1. Dataset and Evaluation Matrix

The well-known human pose estimation datasets MPII and LSP were used to evaluate the performance of the proposed additional interstack skip connection and multidilated light residual blocks. The MPII dataset contains over 40,000 images of people with joint information, of which around 25,000 images were collected in real-world contexts. For human pose estimation, 16 coordinates for each joint were labeled for each person. In addition, we conducted experiments using the LSP and its extended training datasets [36] for objective evaluation. The LSP dataset contains 12,000 images with challenging athletic poses. In this dataset, each full body is annotated with a total of 14 joints.

To evaluate the performance of our method, we compared the performance with the state-of-the-art lightweight method for the stacked hourglass network [8] with various experiments. As an evaluation method, we used the percentage of correct key-points (PCK) on the LSP datasets and the modified PCK measure, which is the percentage of correct key-points on the head (PCKh) with the MPII dataset, as used in [32]. [email protected] uses 50% of the ground-truth head segment’s length as a threshold. If the error rate is lower than the threshold value when comparing the predicted value with the ground truth, it is determined to be the correct answer.

4.2. Training Details

We followed the same training process as used for the original stacked hourglass network [2] with an input image size of $256\times 256$. For the data augmentation required for training, rotation (±30°), scaling (±0.25), and flipping were performed. The model used in all experiments was written using PyTorch software [37]. We used the Adam optimizer [38] for training with a batch size of eight. The number of training epochs was 300, and the initial learning rate was $2.5\times {10}^{-4}$, which was reduced to $2.5\times {10}^{-5}$ and $2.5\times {10}^{-6}$ in the 150th and 220th epochs, respectively. The network was initialized by a normal distribution $\mathcal{N} (m, {\sigma }^2)$ with mean m = 0 and standard deviation σ = 0.001.

$$ℒ=\frac{1}{N}{\displaystyle \sum }_{n=1}^N{\displaystyle \sum }_{ij}{\Vert H_n(i, j)-{\widehat{H}}_n(i, j)\Vert }^2$$

(3)

The ground-truth heat map $H={\{H_k\}}_{k=1}^K$ was generated by applying a Gaussian around $k$ body joints, as shown in [3]. The loss $ℒ$ between the heat map $\widehat{H}={\{{\widehat{H}}_k\}}_{k=1}^K$ and $H$ predicted by the network used the mean squared error (MSE). Losses were calculated using the predicted heat maps from each stack and summed up by intermediate supervision. Figure 6a visualizes the loss in the training process, and Figure 6b visualizes the accuracy of [email protected] in the MPII validation set.

4.3. Lightweight and Bottleneck Structure

To evaluate the network weight-reduction performance, we used depthwise separable convolution and looked at the effect of the bottleneck structure.

Table 2. Results for MPII validation sets in a single-stack hourglass network, depending on the type of residual block. (# Params is total number of parameters in a single-stack hourglass network.).

Residual Block	# Params	[email protected] (Mean)
Bottleneck (Original) (Figure 5a)	3.6M	86.21
Bottleneck (Original) + Depthwise Separable (Figure 5b)	1.4M	85.41
Bottleneck ($3\times 3$) + Depthwise Separable (Figure 5c)	1.6M	85.67
Multidilated Light Residual Block (Ours) (Figure 5d)	2.0M	86.12

shows the experimental result obtained by constructing a single-stack hourglass network with each residual block in Figure 5; the number of parameters in the table represents the total number of parameters in a single hourglass network.

In general, in a problem involving localization, such as human pose estimation, performance reduction occurs when using residual blocks in a bottleneck structure that uses $1\times 1$ convolution to reduce the size of the network [18]. However, using $1\times 1$ convolution was inevitable in this study because the network was made lighter by using depthwise separable convolution. Therefore, in order to confirm the effect of the bottleneck structure using $1\times 1$ convolution in this experiment, the 1 × 1 convolutions of the first and last layers of the original residual block (Figure 5a) used $3\times 3$ kernels. We experimented with a residual block (Figure 5c) that increased the kernel size to $3\times 3$ and applied the depthwise separable convolution. Although the accuracy and parameters increased slightly, our result confirmed that the $1\times 1$ convolution had no significant effect on our experiment.

Through this experiment, we confirmed that the multidilated light residual blocks proposed in this paper showed improved performance in terms of achieving a more lightweight network through a 56% reduction in the number of parameters. In addition, it was confirmed that the [email protected] performance was reduced by approximately 0.09, despite the large reduction in the number of parameters, as compared to the original residual block (Figure 5a). This slight reduction in accuracy was overcome by using the additional skip connection structure described in Section 3.1.

4.4. Additional Skip Connection

To confirm the effect of the additional skip connection (Section 3.1) on network performance, experiments were conducted on a dual-stack hourglass network (

Table 3. Comparison of different network architectures with the MPII validation dataset. (# Params is the total number of parameters in a dual-stack hourglass network).

Network Architecture	Residual Block	# Params	[email protected] (Mean)
Hourglass (Original)	Bottleneck (Original) (Figure 5a)	6.7M	87.72
Ours	Bottleneck (Original) (Figure 5a)	6.7M	88.68
Ours	Ours (Figure 5d)	3.9M	87.60

). When we applied only the proposed method, without using a modified residual block, we observed that the number of parameters remained the same, but the accuracy was greatly increased. The number of parameters in the dual-stack network, using both the proposed network structure (Section 3.1) and the residual block (Figure 5d), was reduced to the level of the original single-stack hourglass network. However, it was confirmed that the accuracy was similar to that of the original dual-stack hourglass network. From this experiment, it was confirmed that the proposed hourglass network using an additional skip connection showed significant results.

4.5. Effect of the Dilation Scale

The dilated convolution in our residual block used zero padding equal to the dilation value $D$ to fit the size of the input and output. Therefore, to check the effect of zero padding on the pose estimation problem according to dilation size, the dilation sizes of $D_1=2 \mathrm{and} D_2=3$ and the increased sizes of $D_1=3 \mathrm{and} D_2=5$ (proposed in this work) were compared (

Table 4. Comparison of different dilation scales with the MPII validation dataset.

Method $[\times \mathrm{Stack}]$	[email protected] (Mean)
Ours ($D_1=2, D_2=3$) [×1]	86.12
Ours ($D_1=3, D_2=5$) [×1]	85.67
Ours ($D_1=2, D_2=3$) [×2]	87.89
Ours ($D_1=3, D_2=5$) [×2]	87.19

).

The receptive field of the 3 × 3 dilated convolution extended by $D=2, 3, \mathrm{and} 5$ was the same as the standard convolution using kernels of $5\times 5, 7\times 7, \mathrm{and} 11\times 11$, respectively. In this experiment, optimum dilation enhanced the performance of pose estimation; however, when the dilation size became too large, too much zero padding caused the network to fail to learn the spatial features, resulting in a loss of the ability to localize joints. In the human pose estimation problem, it was confirmed that zero padding due to the size of the receptive field and dilation had a significant effect on performance, while the optimum dilation size was determined to be $D_1=2 \mathrm{and} D_2=3$.

4.6. Results and Analysis

To evaluate our method, we compared it with current state-of-the-art lightweight hourglass network methods. The authors of [8] proposed a new hourglass architecture using hierarchical residual blocks and evaluated the performance of network binarization in human pose estimation. Single-stack and an eight-stack networks are implemented for comparison.

As shown in

Table 5. Comparison with state-of-the-art lightweight hourglass methods with the MPII validation dataset.

Method $[\times \mathrm{Stack}]$	[8]-Real $[\times 1]$	[8]-Real $[\times 8]$	Ours $[\times 1]$	Ours $[\times 8]$
Head	96.8	97.4	96.3	98.1
Shoulder	93.8	96.0	94.1	96.2
Elbow	86.4	90.7	85.7	90.9
Wrist	80.3	86.2	80.4	87.2
Hip	87.0	89.6	85.6	89.8
Knee	80.4	86.1	80.3	87.3
Ankle	75.7	83.2	76.0	83.5
# Params	6M	25M	2M	14.8M
[email protected] (Mean)	85.5	89.8	86.1	90.8

, our method enhanced performance in pose estimation, despite an approximately 40% reduction in the number of parameters, as compared to state-of-the-art lightweight hourglass networks. It can be seen that the human pose estimation performance using our method was superior.

We compared our results with those of existing methods on the MPII and LSP datasets.

Table 6. Accuracy comparison with existing methods using the MPII validation dataset ([email protected]).

Method	Head	Sho.	Elb.	Wri.	Hip	Knee	Ank.	Mean
Pishchulin et al. [39]	74.3	49.0	40.8	34.1	36.5	34.4	35.2	44.1
Tompson et al. [3]	95.8	90.3	80.5	74.3	77.6	69.7	62.8	79.6
Carreira et al. [40]	95.7	91.7	81.7	72.4	82.8	73.2	66.4	81.3
Tompson et al. [41]	96.1	91.9	83.9	77.8	80.9	72.3	64.8	82.0
Hu et al. [42]	95.0	91.6	83.0	76.6	81.9	74.5	69.5	82.4
Pishchulin et al. [43]	94.1	90.2	83.4	77.3	82.6	75.7	68.6	82.4
Lifshitz et al. [44]	97.8	93.3	85.7	80.4	85.3	76.6	70.2	85.0
Gkioxary et al. [45]	96.2	93.1	86.7	82.1	85.2	81.4	74.1	86.1
Rafi et al. [46]	97.2	93.9	86.4	81.3	86.8	80.6	73.4	86.3
Belagiannis et al. [24]	97.7	95.0	88.2	83.0	87.9	82.6	78.4	88.1
Insafutdinov et al. [47]	96.8	95.2	89.3	84.4	88.4	83.4	78.0	88.5
Wei et al. [15]	97.8	95.0	88.7	84.0	88.4	82.8	79.4	88.5
Bulat et al. [8]	97.9	95.1	89.9	85.3	89.4	85.7	81.7	89.7
Ours	98.1	96.2	90.9	87.2	89.8	87.3	83.5	90.8

presents the PCKh scores from different methods with the MPII dataset.

Table 7. Accuracy comparison with existing methods using the LSP validation dataset ([email protected]).

Method	Head	Shoulder	Elbow	Wrist	Hip	Knee	Ankle	Mean
Lifshitz et al. [44]	96.8	89.0	82.7	79.1	90.9	86.0	82.5	86.7
Pishchulin et al. [43]	97.0	91.0	83.8	78.1	91.0	86.7	82.0	87.1
Insafutdinov et al. [47]	97.4	92.7	87.5	84.4	91.5	89.9	87.2	90.1
Wei et al. [15]	97.8	92.5	87.0	93.9	91.5	90.8	89.9	90.5
Bulat et al. [8]	97.2	82.1	88.1	85.2	92.2	91.4	88.7	90.7
Ours	98.1	93.1	89.2	86.1	92.7	92.8	90.4	91.7

shows the PCK scores with the LSP dataset. The results confirmed that the proposed method shows improved human pose performance compared to the existing methods.

Table 8. Parameter and accuracy comparison with existing methods using the MPII validation dataset ([email protected]).

Method	# Params	[email protected] (Mean)
Bulat et al. [1]	58.1M	89.7
Insafutdinov et al. [47]	42.6M	88.5
Bulat et al. [8]	25.0M	89.8
Newell et al. [2]	25.1M	90.9
Ours	14.8M	90.8

shows results from the experiment comparing the number of parameters for the MPII dataset. Figure 7 presents a schematic diagram of Table 8. This experiment confirmed that the method proposed in this paper significantly reduced the number of parameters while enhancing pose estimation performance.

The number of parameters and accuracy according to the number of stacks used are summarized in

Table 9. Results for MPII validation datasets by number of stacks.

Method	# Params	[email protected] (Mean)
$[\times 1]$	2.0M	86.12
$[\times 2]$	3.9M	87.89
$[\times 8]$	14.8M	90.78

. Figure 8 visualizes the pose estimation results for the MPII dataset in the eight-stack network, in which the joints in the areas covered or crossed by the body are correctly estimated. These experiments also confirmed that the proposed method represents an improvement over existing methods in terms of efficiency and performance.

5. Conclusions

In this paper, we proposed a lightweight stacked hourglass network for human pose estimation. The problem with existing stacked hourglass networks is that they continuously transmit only relatively high-level features from one stack to the next. To solve this problem, we proposed a new hourglass network structure utilizing additional interstack skip connections at the front end of the encoder. These improve the performance by reflecting relatively low-level features extracted by the encoder in the next stack to allow the gradient to flow smoothly during the learning process, even in the case of a deep stack. Moreover, since the skip connection involves a simple elementwise sum operation, performance can be improved without increasing the number of parameters, which assists in constructing a lightweight network.

To maintain accuracy, a multidilated light residual block was also proposed to reduce the number of parameters in the network by about 40% compared to an existing hourglass network. Using a multidilated light residual block improves performance by expanding the receptive field using dilated convolution, significantly reducing both the number of parameters and the computational load by applying depthwise separable convolution. In this paper, a variety of experiments was conducted for objective performance evaluation of the proposed methods, and the results confirmed that our proposed methods demonstrate an effective step forward in meeting the challenges of human pose estimation.