Next Article in Journal
Impact of Supersonic Flow in Scintillator Detector Apertures on the Resulting Pumping Effect of the Vacuum Chambers
Next Article in Special Issue
Full-Body Kinematics and Vertical Ground Reaction Forces in Elite Ten-Pin Bowling: A Field Study
Previous Article in Journal
A Systematic Solution for Moving-Target Detection and Tracking While Only Using a Monocular Camera
Previous Article in Special Issue
A New Approach to Quantifying Muscular Fatigue Using Wearable EMG Sensors during Surgery: An Ergonomic Case Study
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Wearable Multi-Modal Digital Upper Limb Assessment System for Automatic Musculoskeletal Risk Evaluation

1
Department of Electronic Systems, Aalborg University, 9220 Aalborg, Denmark
2
Department of Mechanical, Mechatronics, and Manufacturing Engineering, University of Engineering & Technology Lahore, Faisalabad Campus, Faisalabad 38000, Pakistan
3
Department of Materials and Production, Aalborg University, 9220 Aalborg, Denmark
*
Author to whom correspondence should be addressed.
Sensors 2023, 23(10), 4863; https://doi.org/10.3390/s23104863
Submission received: 21 March 2023 / Revised: 12 May 2023 / Accepted: 16 May 2023 / Published: 18 May 2023

Abstract

:
Continuous ergonomic risk assessment of the human body is critical to avoid various musculoskeletal disorders (MSDs) for people involved in physical jobs. This paper presents a digital upper limb assessment (DULA) system that automatically performs rapid upper limb assessment (RULA) in real-time for the timely intervention and prevention of MSDs. While existing approaches require human resources for computing the RULA score, which is highly subjective and untimely, the proposed DULA achieves automatic and objective assessment of musculoskeletal risks using a wireless sensor band embedded with multi-modal sensors. The system continuously tracks and records upper limb movements and muscle activation levels and automatically generates musculoskeletal risk levels. Moreover, it stores the data in a cloud database for in-depth analysis by a healthcare expert. Limb movements and muscle fatigue levels can also be visually seen using any tablet/computer in real-time. In the paper, algorithms of robust limb motion detection are developed, and an explanation of the system is provided along with the presentation of preliminary results, which validate the effectiveness of the new technology.

1. Introduction

Physical workload assessment is critical but complex and multi-dimensional, which requires careful consideration of various physical workload modalities, e.g., intensity, duration, frequency, and variability, together with environmental and individual factors, e.g., temperature, lighting conditions, age, sex, and physical fitness [1]. Workers in factories, individuals performing activities of daily life (ADL) with poor posture, and even computer and mobile phone users are prone to develop MSDs [2]. MSDs include temporary or permanent injuries in the inner body parts, e.g., muscles, tendons, joints, and spinal disks. Industrial workers often have to perform repetitive movements, heavy lifting, and body twisting, which can lead to a variety of problems, e.g., chronic pain, decreased mobility, and MSDs. Such jobs require workers to work in conditions that are not conducive to good posture and body alignment [3], hence carrying high risks of developing MSDs. A recent survey for working conditions in Europe indicates that 42% of European workers complain of pain around the neck and upper limbs [4].
To ensure the safety of workers, it is important to assess the ergonomic risks associated with these tasks and implement controls and interventions to improve working conditions. Several risk assessment methods have been developed, such as rapid entire body assessment (REBA) [5], novel ergonomic postural assessment (NERPA) [6], strain index (SI) [7], Ovako working posture analyzing system (OWAS) [8], and RULA [9]. Of them, RULA is the most widely used subjective tool to quickly assess the ergonomic risk factors associated with upper limbs, incorporating both muscle utilization and limb postures. Basahel et al. used RULA to assess ergonomic risk factors associated with lifting and pulling tasks in an industrial environment [10]. In another study, a modified RULA method was proposed for two posture risk quantification methods, namely event-based and time-based, for upper-extremity MSDs [11]. Additional similar works using RULA and its modified forms to assess ergonomic risk factors associated with industrial workers can be found in references [12,13]. Since RULA requires deployment in the field, therefore these non-automatic methods require tedious setup and a medical expert to continuously observe workers’ body movements. To empower devices to take on the burden from humans, body-mounted sensors can be used for computing RULA scores automatically, e.g., electromyograph (EMG) based methods [14,15,16] and inertial-sensor-based methods [17,18,19]. However, it is essential to note that EMG-based methods have higher costs and longer deployment times. The past works utilizing inertial-based methods do not explicitly provide kinematic details and use graphic software, e.g., CATIA or AnyBody, to show limb movements, which put a limitation on the design and deployment of an automatic RULA assessment system in the form of an embedded wearable device.
As a graphical assessment technique, a digital human model (DHM) can be used to visualize and analyze the movements of limbs and human posture in computer software [20,21]. Such techniques are particularly useful to identify and mitigate potential risks before designing a product or workstation. This technology is suitable for early ergonomic analysis but cannot be utilized in real-time, as performing a certain task is majorly dependent upon an individual’s preference. Therefore, the utilization of muscles or body posture would be different for the same task from time to time and person to person. Hu et al. worked on predicting the real-world ergonomic measurements by simulating the scenario in a virtual environment (VE) while a subject did the movements in a real environment [22]. Nevertheless, VE-based methods can only be deployed and tested in a laboratory setup. Another mode of risk assessment is based on video recording [23,24,25]. Optical assessment methods are tedious to set up with the constraint of having direct sight of the worker. Other assessment methods, e.g., questionnaires and interviews [26,27], limit their practicality due to the substantial need for human resources, and such methods can be biased.
For the people working in environments where the setup is already installed, non-obstructive and intrinsic or body-mounted sensors, e.g., EMG, inertial sensors, gyroscope, and gravimeters, can be used for recording and analyzing motion data, and further for ergonomic risk assessments [17,18,19,28]. Padilla et al. designed a web-based risk-factor assessment system using inertial sensors and displayed limb movements [29], but they did not provide any details for limb kinematics. Employing graphic software to display limb movements is common but not very helpful in assessing working conditions due to the lack of kinematic information or evaluating exoskeleton design, particularly passive exoskeletons [16,30] in industrial usages. A comparison of past works is listed in Table 1.
In this study, a multi-modal digital upper limb assessment system—DULA—is presented along with robust kinematics, which continuously tracks and records muscle activities and limb motions to digitally fill the RULA chart and automatically generate risk levels. DULA consists of multi-modal sensor bands having wireless sensors interfaced with a mobile or tablet application. Two modalities, (i) limb posture in space using IMUs, and (ii) muscle activity through a force myograph (FMG), are devised for the proposed system. A robust kinematic model is developed to accurately determine upper limb motion. Furthermore, the data from these wearable sensors is continuously recorded and stored in Google Firebase, which can be retrieved for visualization, examination, and further analysis by a medical expert. The contributions of this work are:
  • Multi-modal wireless wearable sensors are used for digitally computing RULA scores.
  • A robust kinematic model has been developed for upper limbs motion tracking.
  • Digital implementation of the new method has been validated by comparison with an online RULA calculator.
The balance of the paper is organized as follows: Section 2 describes RULA, kinematics to detect arm posture, frame transformations, and load identification. Hardware and software implementation is presented in Section 3. Results are presented in Section 4 and discussed in Section 5. Finally, conclusions are drawn in Section 6.

2. DULA Method

The DULA system utilizes battery-powered wireless multi-modal sensor bands, which are wirelessly connected to a mobile application through Bluetooth. As will be presented in Section 3, each sensor band is equipped with an IMU and force-sensitive resistors (FSRs) for FMG, which are primarily used for ergonomic risks analysis, whereas gyroscopes and gravimeters are used to set up coordinate frames with the upper limbs for kinematic analysis. The system automatically generates the RULA score, which will be henceforth referred to as the DULA score.

2.1. RULA

DULA is a digital implementation of RULA. We will, therefore, briefly describe the RULA method for completeness’ sake. RULA is a subjective observation method of posture analysis to assess the risks of developing MSDs for individuals who perform physical tasks using their upper extremities. Illustrations for different upper limb postures, the load being lifted, and associated numerical scores are shown in Figure 1 [9]. The DULA system finds the orientation of the upper arm and forearm and the load being lifted, assigns the corresponding scores, and yields the aggregated score. The aggregated score has the range of 1–9, which determines the overall risk of MSDs for upper limbs [9] and the corresponding action level as shown in Table 2. A lower RULA score corresponds to a lower risk of developing MSDs indicating that work posture is good enough. Hence, no change in the working scheme is required. In contrast, a higher RULA score corresponds to a severe risk of developing MSDs, and therefore, the working scheme should be immediately changed.

2.2. Kinematics for DULA

DULA, i.e., automatic computation of RULA score, requires first determining the accurate forearm and upper arm orientation. IMUs in the sensor band are primarily used for this purpose. However, raw data from IMUs do not provide any useful kinematic information without the attachment of reference coordinate frames on the upper arm and forearm; doing so will provide a coordinate reference to measure the limb’s orientation. Setting up the coordinate frame is explained in Section 2.2.1 while frame transformations of the upper arm and forearm movements with respect to the attached coordinate frames are explained in Section 2.2.2 and Section 2.2.4.

2.2.1. Reference Coordinate Setup

The reference coordinate frames for the forearm and upper arm are shown in Figure 2. To attach a frame to the forearm, a sensor band is worn on the dorsal side of the arm, i.e., near the wrist. A certain posture and a certain movement are performed so that a coordinated setup can be established. Each course of action identifies one unit vector of the forearm as each movement projects one unit vector of the forearm reference frame in the coordinate frame identified by the current orientation of IMU referred to as { I } . The cross-multiplication of two orthogonal unit vectors from the two actions yields a third unit vector or axis and completes the procedure. The right forearm frame is settled with the following steps:
  • Stretch the arm forward while the palm is facing downwards. This is considered the right arm base posture.
  • Record the gravity vector g at base posture, upon which the z F * ^ for the forearm frame is determined.
    z F * ^ = g g .
  • Perform supination motion and record angular velocity ω during the motion. The y F * ^ for the forearm frame is hence obtained as:
    y F * ^ = ω ω .
  • The cross product of y F * ^ and z F * ^ results in x F * for the forearm frame. However, it is not humanly possible to have y F * ^ and z F * ^ exactly perpendicular to each other, hence, x F * is normalized.
    x F * = y F * ^ × z F * ^ ,
    x F * ^ = x F * x F * .
  • Finally, z F * ^ is obtained to complete right hand coordinate frame convention.
    z F * ^ = x F * ^ × y F * ^ .
  • Thus, the computed rotation matrix belongs to a special orthogonal group S O 3 . The forearm reference frame in the IMU frame has the following form:
    I F * R = x F * ^ y F * ^ z F * ^ .
The method to attach a reference coordinate frame on the right upper arm follows the same steps as the forearm, except that the sensor band is worn on the biceps muscles. The same steps are repeated for the left forearm and left upper arm; however, for the left limbs, pronation motion will be performed as in Step 3, and the angular velocity direction is taken positively.

2.2.2. Frame Transformations

A robust frame transformation methodology is developed, which provides the limbs orientations with respect to corresponding reference coordinate frames. A scenario, as shown in Figure 3, is presented for the right forearm, where the frames, shown in curly braces, are randomly oriented to formulate the mathematical expression of the rotation matrix to represent the orientation of the forearm { F } with respect to the forearm reference frame { F * } . The matrix R represents a rotational transformation from the frame in the subscript to the frame in superscript, whereas the subscripts under the axes identify the frame they belong to.
Consider the posture when the sensor band is worn on the forearm. Raw data from the IMU represents the forearm orientation in global reference frame G F R . A reference frame { F * } is attached to the forearm according to the method described in Section 2.2.1. The objective is to find F * F R , however, it should be noted that { F * } does not necessarily exhibit the same orientation as that of { G } . Furthermore, the reference frame { F * } constitutes a new coordinate system. Hence, the objective is to find the forearm frame { F } orientation which is defined with respect to { G } , in a new coordinate system established by { F * } . The method is described as follows:
  • Record IMU orientation G I R when the arm is in the base posture.
  • I F * R is computed according the method in Section 2.2.1.
  • G F R is continuously acquired from the IMU sensor in real-time.
  • To find the forearm orientation with respect to IMU frame, i.e, I F R , the following equations are presented:
    G F R = G I R I F R ,
    I F R = G I R T G F R .
  • Finally, we find the orientation of the forearm with respect to the forearm reference frame { F * } ,
    F * F R = I F * R T I F R I F * R .
The same methodology is applied to find upper arm transformation to have a rotation matrix describing upper arm { U } orientation with respect to upper arm reference frame { U * } .

2.2.3. Upper Arm Orientation

The rotation matrix defining upper arm orientation with respect to the upper arm reference frame is considered to be formed by the upper arm extension/flexion ψ u , abduction/adduction θ u , and internal/external rotation ψ u . Mathematically:
U * U R = R X Z Y = R X ϕ u R Z θ u R Y ψ u = r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ,
where, starting with a frame coincident with upper arm reference frame { U * } , R X , R Z θ u , and R Y ψ u represent three consecutive rotations following XZY convention. The angles ϕ u , θ u , and ψ u are obtained as:
ϕ u = arctan r 32 , r 22 ,
θ u = arcsin r 12 ,
ψ u = arctan r 13 , r 11 .

2.2.4. Forearm Orientation

To compute the RULA score for a forearm, the relative rotation of the forearm with respect to upper arm is required. However, the orientations of the forearm and upper arm are defined in { F * } and { U * } , respectively, while both constitute different coordinate systems. Therefore, we map the forearm orientation in the coordinate frame { U * } and then find the relative rotation.
  • Find the orientations of upper arm base frame { U * } and forearm base frame { F * } with respect to global reference frame { G } as:
    G F * R = G I f R I f F * R ,
    G U * R = G I u R I u U * R ,
    where I u and I f are upper arm IMU and forearm IMU frames, respectively.
  • Compute the relative rotation R 1 between both frames { F * } and { G * } :
    G F * R R 1 = G U * R ,
    R 1 = G F * R T G U * R .
  • The forearm orientation { F } defined in { F * } is mapped in the upper arm reference frame { U * } using similarity transformation technique:
    U * F R = R 1 T F * F R R 1 .
  • Since both forearm and upper arm orientations are now established in the same coordinate system defined by { U * } , the relative rotation matrix R 2 is found as:
    U * U R R 2 = U * F R ,
    R 2 = U * U R T U * F R .
  • The forearm orientation with respect to the upper arm is considered to be formed by the combination of two rotations, the forearm extension/flexion α f about the current Y, and the forearm pronation/supination β f about the current Z axis.
    R 2 = R Y Z = R Y ( α f ) R Z ( β f ) = s 11 s 12 s 13 s 21 s 22 s 23 s 31 s 32 s 33 .
    The composition of orientation is illustrated in Figure 4. Inverse kinematics is used to find the forearm angles:
    α f = arctan s 21 , s 22 ,
    β f = arctan s 13 , s 33 .

2.3. Load Identification

FMG is a technology that can measure changes in muscle volume during limb motion, which arise from muscle contractions and expansions. An FSR is a special resistor that exhibits a change in resistance when a force is applied to it. Therefore, FMG is primarily done using FSRs, and each sensor band is equipped with eight FSRs. Upper limb muscles hold the weight carried by an individual since the sensor band is worn over bicep muscles; hence the change in measured resistance corresponds to the muscle activity, which is interpreted as force being exerted or load being lifted by an individual. Various research works are available to identify payload using FMG [31,32]. When the sensor band is worn over the upper arm, the maximum value for each FSR channel is recorded by intense elbow flexion, later used to normalize FSR data. Normalized FSR values for 2 kg and 10 kg loads, as two thresholds for RULA, are important according to Figure 1c and are marked experimentally to identify the load category. Finally, the identified load is scored according to Figure 1c.

3. DULA System Development

A portable and easy-to-deploy multi-modal digital upper limb assessment system, DULA, has been developed for automatic musculoskeletal risk assessments as shown in Figure 5. DULA continuously tracks and records upper limbs motion as well as upper arm muscle activity. The system generates single-digit risk assessment scores, ranging from 1 to 9, which indicates the risk of developing MSDs. A lower score means that human postures for carrying out physical tasks are appropriate, implying that no change in the working pattern is required, while a higher score corresponds to higher chances of developing MSDs implying that the working pattern should be immediately changed. The system also stores human motion data and muscle activity data in Google Firebase for visualization and in-depth analysis by medical experts.

3.1. Hardware

The hardware of the designed system consists of wireless, battery-powered, multi-modal sensor bands and a mobile phone or tablet. The bands are connected to mobile/tablet applications through Bluetooth. Each sensor band has various sensors, e.g., an IMU consisting of a gravity sensor, gyroscope, linear and angular accelerometers, and eight FSRs. All the sensors provide kinematic information except FSRs, which are used for FMG to determine the weight being lifted by an individual. FSRs are embedded in flexible armbands to have a tight grip over muscles. Data from FSRs are amplified and passed through low pass filters before feeding to analog to onboard digital conversion (ADC) modules, while data from IMU are acquired using the I2C protocol of the ESP32 chip. This chip then sends the data to the mobile phone in real-time through the onboard Bluetooth module.

3.2. Software

A mobile application is developed to receive data from sensor bands and to perform the matrix operations for digital upper limb assessment in real-time. On startup, this application guides the user through the procedure of attaching the reference coordinate frames with left and right forearms and upper arms. Once coordinate setup is complete, multi-modal data is acquired, and matrix operations are performed in the background of the mobile application. Risk assessments, limb movements, and muscle usage are graphically shown on the screen. The application has an additional feature of triggering an alarm/notification if an individual is at a high risk of developing MSDs. The application is developed using Flutter and vector_math_64 library is used for matrix operations.

4. Results

With the developed algorithm and system, a DULA score has been generated for the right upper limbs as a test scenario where a person picked an electrical screwdriver from a table and tightened a screw on the wall. Figure 6 shows the body posture for the screw-tightening task and corresponding limb orientations. Angular velocity and gravity data are used only for coordinate frames setup as described in Section 2.2.1 and are not shown here. The orientation of the upper arm and forearm, upper arm abduction, forearm working out to the side of the body or across body mid-line, wrist twist, and carried weight are the factors considered to digitally fill up the RULA score chart. The DULA score is generated in accordance with Figure 1, leading to the generation of the final RULA score according to the method described in Reference [9]. Figure 6 shows that the designed experiment consists of five major movements. The DULA score for each phase shows that there are no ergonomic risks associated with standing still (SS), while approach phases (A1 and A2) show DULA scores of 3 and 4, respectively, indicating that these postures and movements do not carry high risks for developing MSDs.
Correlating the DULA score for the screw tightening task with Table 2 indicates that the working routine will soon require changes, and the long time exposure in carrying out this task in the same pattern will certainly lead to developing MSDs. Hence investigations to improve the task procedure or workstation are suggested. To validate the proposed methodology, a web-based ergonomic assessment tool for RULA [33] is used to manually input upper limb postures to generate corresponding RULA scores. Ten different body postures as shown in Figure 7, and three different weights (1.5 kg, 6 kg, and 12 kg) are set for an individual, and RULA scores from the web-based tool and from the DULA system are compared as shown in Table 3. The verification of DULA is ensured by matching the generated scores, as the difference in the RULA scores was 0.3 ± 0.49.

5. Discussion

RULA is well developed ergonomic assessment tool, effective for MSD prevention. In this study, a new system, named DULA, is designed for digital rapid upper limb assessment for MSD early prevention and intervention. The system can be used by physical workers and can be easily deployed in any working setup. In addition, the system has the potential to be used with assistive devices, e.g., wearable exoskeletons [30], for their performance assessment, ensuring their effective use. For ergonomic risk assessments in industrial environments, different studies have evaluated ergonomic risks by recording workers’ movements, through questionnaires, or manual analysis by medical experts, which are time-consuming and delicate evaluations. The DULA system effectively addresses these challenges using wearable sensors linked with a mobile application and performing the risk assessment in real-time. However, RULA may not be suitable for assessing ergonomic risks associated with very complex or specialized tasks, which may require a more tailored approach to indicate a correct risk evaluation and would also require more wearable sensor bands. Nevertheless, RULA combined with fuzzy logic has the potential to indicate a better risk level. The same methodology can be further enhanced by tracking and scoring lower limbs, wrists, and necks.
It has to be noted that the risk of developing MSDs is also influenced by various body-related factors, such as age, height, weight, and any comorbidities that the worker may have. RULA provides quick and valuable insights into the potential for injury or discomfort associated with a particular job or task. However, it has limitations in considering the time for performing the task and environmental conditions, e.g., temperature, lighting, noise, etc. Upper arm muscles are responsible for lifting weight. Their role is particularly significant when there is elbow flexion. However, when the elbow is not flexed, then instead of upper arm muscles, weight is carried by bones or by shoulder muscles. In such circumstances, FSRs over bicep muscles are not helpful in identifying the right value of the picked load. Regarding the measurement of muscle activation to determine the lifted load, limitations occur when FSRs become saturated or FSRs experience external force. Consequently, FMG data becomes erroneous. Anti-saturation is required for the post-processing of FMG data. Another limitation is related to an inherent singularity with Euler angles. Euler angles have the intrinsic property that they undergo singularity when the rotation about the middle axis makes the first and third axes parallel to each other.

6. Conclusions

The proposed system, DULA, digitally records and automatically analyzes upper limb motions and muscle activities to determine the RULA score for the upper limbs without any intervention from a medical expert. Preliminary results have shown that DULA comprehends RULA. For the ten different postures and three different weights (1.5 kg, 6 kg, and 12 kg), the difference between RULA and DULA is 0.3 ± 0.49.
DULA is a portable system that can be easily deployed in industry and possesses great potential to protect workers from developing temporary or permanent MSDs through early intervention and prevention. In future research, the following issues will be considered to further enhance the methodology: (a) exploring the impact of external forces on FMG, (b) incorporating the analysis of wrist, neck, and lower limbs to enable comprehensive and automated whole-body posture analysis, and (c) integrating task duration into RULA to enhance the accuracy of the assessment. These research directions hold great potential in advancing the field of workplace ergonomics and contributing to the development of effective interventions to prevent musculoskeletal disorders.

Author Contributions

Conceptualization, S.B.; methodology, A.T. and S.B.; validation, A.T.; formal analysis, A.T. and S.B.; investigation, A.T. and S.B.; resources, S.B.; writing—original draft preparation, A.T.; writing—review and editing, S.B. and M.S.; visualization, A.T., S.B. and M.S.; supervision, S.B. and M.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received funding from Interreg North Sea Region.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to extend their sincere gratitude to the Interreg North Sea Region-funded project EXSKALLERATE. The authors thank Pascal Madeleine from the Department of Health Science and Technology, Aalborg University, for his discussion of and comments on the paper and Mathias Mosskov Jacobsen from the Department of Computer Science, Aalborg University, for developing the mobile application.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
MSDMusculoskeletal disorders
RULARapid upper limb assessment
DULADigital upper limb assessment
IMUInertial measurement unit
F or fForearm
U or fUpper arm

References

  1. Winkel, J.; Mathiassen, S.E. Assessment of physical work load in epidemiologic studies: Concepts, issues and operational considerations. Ergonomics 1994, 37, 979–988. [Google Scholar] [CrossRef]
  2. Dockrell, S.; O’Grady, E.; Bennett, K.; Mullarkey, C.; Mc Connell, R.; Ruddy, R.; Twomey, S.; Flannery, C. An investigation of the reliability of Rapid Upper Limb Assessment (RULA) as a method of assessment of children’s computing posture. Appl. Ergon. 2012, 43, 632–636. [Google Scholar] [CrossRef] [PubMed]
  3. Lei, L.; Dempsey, P.G.; Xu, J.G.; Ge, L.N.; Liang, Y.X. Risk factors for the prevalence of musculoskeletal disorders among Chinese foundry workers. Int. J. Ind. Ergon. 2005, 35, 197–204. [Google Scholar] [CrossRef]
  4. Parent-Thirion, A.; Biletta, I.; Cabrita, J.; Vargas Llave, O.; Vermeylen, G.; Wilczyńska, A.; Wilkens, M. Sixth European Working Conditions Survey–Overview report. Eurofound 2016. [Google Scholar] [CrossRef]
  5. Hignett, S.; McAtamney, L. Rapid entire body assessment (REBA). Appl. Ergon. 2000, 31, 201–205. [Google Scholar] [CrossRef]
  6. Sanchez-Lite, A.; Garcia, M.; Domingo, R.; Angel Sebastian, M. Novel ergonomic postural assessment method (NERPA) using product-process computer aided engineering for ergonomic workplace design. PLoS ONE 2013, 8, e72703. [Google Scholar] [CrossRef]
  7. Moore, J.S.; Vos, G.A. The strain index. In Handbook of Human Factors and Ergonomics Methods; CRC Press: Boca Raton, FL, USA, 2004; pp. 109–114. [Google Scholar]
  8. Karhu, O.; Kansi, P.; Kuorinka, I. Correcting working postures in industry: A practical method for analysis. Appl. Ergon. 1977, 8, 199–201. [Google Scholar] [CrossRef]
  9. McAtamney, L.; Corlett, E.N. RULA: A survey method for the investigation of work-related upper limb disorders. Appl. Ergon. 1993, 24, 91–99. [Google Scholar] [CrossRef]
  10. Basahel, A.M. Investigation of work-related musculoskeletal disorders (MSDs) in warehouse workers in Saudi Arabia. Procedia Manuf. 2015, 3, 4643–4649. [Google Scholar] [CrossRef]
  11. Bao, S.; Howard, N.; Spielholz, P.; Silverstein, B. Two posture analysis approaches and their application in a modified rapid upper limb assessment evaluation. Ergonomics 2007, 50, 2118–2136. [Google Scholar] [CrossRef]
  12. Intranuovo, G.; De Maria, L.; Facchini, F.; Giustiniano, A.; Caputi, A.; Birtolo, F.; Vimercati, L. Risk assessment of upper limbs repetitive movements in a fish industry. BMC Res. Notes 2019, 12, 1–7. [Google Scholar] [CrossRef] [PubMed]
  13. Valentim, D.P.; de Oliveira Sato, T.; Comper, M.L.C.; da Silva, A.M.; Boas, C.V.; Padula, R.S. Reliability, construct validity and interpretability of the Brazilian version of the Rapid Upper Limb Assessment (RULA) and Strain Index (SI). Braz. J. Phys. Ther. 2018, 22, 198–204. [Google Scholar] [CrossRef] [PubMed]
  14. Balasubramanian, V.; Swami Prasad, G. An EMG-based ergonomic evaluation of manual bar bending. Int. J. Ind. Syst. Eng. 2007, 2, 299–310. [Google Scholar] [CrossRef]
  15. González, A.; Barrios-Muriel, J.; Romero-Sánchez, F.; Salgado, D.; Alonso, F. Ergonomic assessment of a new hand tool design for laparoscopic surgery based on surgeons’ muscular activity. Appl. Ergon. 2020, 88, 103161. [Google Scholar] [CrossRef]
  16. Kong, Y.K.; Choi, K.H.; Cho, M.U.; Kim, S.Y.; Kim, M.J.; Shim, J.W.; Park, S.S.; Kim, K.R.; Seo, M.T.; Chae, H.S.; et al. Ergonomic assessment of a lower-limb exoskeleton through electromyography and Anybody modeling system. Int. J. Environ. Res. Public Health 2022, 19, 8088. [Google Scholar] [CrossRef]
  17. Huang, C.; Kim, W.; Zhang, Y.; Xiong, S. Development and validation of a wearable inertial sensors-based automated system for assessing work-related musculoskeletal disorders in the workspace. Int. J. Environ. Res. Public Health 2020, 17, 6050. [Google Scholar] [CrossRef]
  18. Moore, J.; Stuart, S.; McMeekin, P.; Walker, R.; Celik, Y.; Pointon, M.; Godfrey, A. Enhancing free-living fall risk assessment: Contextualizing mobility based IMU data. Sensors 2023, 23, 891. [Google Scholar] [CrossRef]
  19. Akhmad, S.; Arendra, A.; Findiastuti, W.; Lumintu, I.; Pramudita, Y.D. Wearable IMU Wireless Sensors Network for Smart Instrument of Ergonomic Risk Assessment. In Proceedings of the 6th Information Technology International Seminar (ITIS), Surabaya, Indonesia, 14–16 October 2020; pp. 213–218. [Google Scholar]
  20. Mohamad, D.; Md Deros, B.; Ismail, A.R.; Daruis, D.D.I.; Sukadarin, E.H. RULA analysis of work-related disorder among packaging industry worker using digital human modeling (DHM). In Advanced Engineering Forum; Trans Tech Publications Ltd.: Kuala Lumpur, Malaysia, 2013; Volume 10, pp. 9–15. [Google Scholar]
  21. Hussain, M.M.; Qutubuddin, S.; Kumar, K.P.R.; Reddy, C.K. Digital human modeling in ergonomic risk assessment of working postures using RULA. In Proceedings of the International Conference on Industrial Engineering and Operations Management, Bangkok, Thailand, 5–7 March 2019; pp. 5–7. [Google Scholar]
  22. Hu, B.; Ma, L.; Zhang, W.; Salvendy, G.; Chablat, D.; Bennis, F. Predicting real-world ergonomic measurements by simulation in a virtual environment. Int. J. Ind. Ergon. 2011, 41, 64–71. [Google Scholar] [CrossRef]
  23. Cao, W.; Zhong, J.; Cao, G.; He, Z. Physiological Function Assessment Based on Kinect V2. IEEE Access 2019, 7, 105638–105651. [Google Scholar] [CrossRef]
  24. Yu, Y.; Yang, X.; Li, H.; Luo, X.; Guo, H.; Fang, Q. Joint-level vision-based ergonomic assessment tool for construction workers. J. Constr. Eng. Manag. 2019, 145, 04019025. [Google Scholar] [CrossRef]
  25. Forsman, M.; Hansson, G.Å.; Medbo, L.; Asterland, P.; Engström, T. A method for evaluation of manual work using synchronised video recordings and physiological measurements. Appl. Ergon. 2002, 33, 533–540. [Google Scholar] [CrossRef] [PubMed]
  26. David, G.C. Ergonomic methods for assessing exposure to risk factors for work-related musculoskeletal disorders. Occup. Med. 2005, 55, 190–199. [Google Scholar] [CrossRef] [PubMed]
  27. Cheng, H.Y.K.; Cheng, C.Y.; Ju, Y.Y. Work-related musculoskeletal disorders and ergonomic risk factors in early intervention educators. Appl. Ergon. 2013, 44, 134–141. [Google Scholar] [CrossRef] [PubMed]
  28. McDevitt, S.; Hernandez, H.; Hicks, J.; Lowell, R.; Bentahaikt, H.; Burch, R.; Ball, J.; Chander, H.; Freeman, C.; Taylor, C.; et al. Wearables for biomechanical performance optimization and risk assessment in industrial and sports applications. Bioengineering 2022, 9, 33. [Google Scholar] [CrossRef]
  29. Padilla, B.E.O.; Glushkova, A.; Menychtas, D.; Manitsaris, S. Designing a web-based automatic ergonomic assessment using motion data. In Proceedings of the 12th ACM International Conference on Pervasive Technologies Related to Assistive Environments, Rhodes, Greece, 5–7 June 2019; pp. 528–534. [Google Scholar]
  30. Balser, F.; Desai, R.; Ekizoglou, A.; Bai, S. A novel passive shoulder exoskeleton designed with variable stiffness mechanism. IEEE Robot. Autom. Lett. 2022, 7, 2748–2754. [Google Scholar] [CrossRef]
  31. Islam, M.; Bai, S. Payload estimation using forcemyography sensors for control of upper-body exoskeleton in load carrying assistance. Model. Identif. Control (Online) 2019, 40, 189–198. [Google Scholar] [CrossRef]
  32. Tahir, A.; An, Z.; Bai, S.; Shen, M. Robust payload recognition based on sensor-over-muscle-independence deep learning for the control of exoskeletons. IEEE Trans. Circuits Syst. II Express Briefs 2023. [Google Scholar] [CrossRef]
  33. Ergonomics, O. RULA—Rapid Upper Limb Assessment. 2019. Available online: https://www.rula.co.uk/assessment.html (accessed on 12 May 2023).
Figure 1. DULA diagram for ergonomic assessment. (a) Upper arm postures and corresponding scores, (b) forearm postures and corresponding scores, (c) load being lifted and corresponding scores, (d) wearable sensor band.
Figure 1. DULA diagram for ergonomic assessment. (a) Upper arm postures and corresponding scores, (b) forearm postures and corresponding scores, (c) load being lifted and corresponding scores, (d) wearable sensor band.
Sensors 23 04863 g001
Figure 2. Reference coordinate frames setup for the forearm { F * } and the upper arm { U * } .
Figure 2. Reference coordinate frames setup for the forearm { F * } and the upper arm { U * } .
Sensors 23 04863 g002
Figure 3. Frame setup for the kinematics of arm motion. The coordinate frames are shown in curly braces, whereas rotation matrices adjacent to arrows connecting two frames represent their transformation. { G } is the global reference frame. { F * } and { I } are the forearm reference frame and IMU frame, respectively, and they are attached to the initial configuration of the forearm. Frame { F } moves with the forearm.
Figure 3. Frame setup for the kinematics of arm motion. The coordinate frames are shown in curly braces, whereas rotation matrices adjacent to arrows connecting two frames represent their transformation. { G } is the global reference frame. { F * } and { I } are the forearm reference frame and IMU frame, respectively, and they are attached to the initial configuration of the forearm. Frame { F } moves with the forearm.
Sensors 23 04863 g003
Figure 4. Moving from the upper arm frame to the forearm frame, when forearm orientation is already defined with respect to frame { U * } .
Figure 4. Moving from the upper arm frame to the forearm frame, when forearm orientation is already defined with respect to frame { U * } .
Sensors 23 04863 g004
Figure 5. System hardware and software overview.
Figure 5. System hardware and software overview.
Sensors 23 04863 g005
Figure 6. Screw tightening task, corresponding data from the sensors, and RULA score.
Figure 6. Screw tightening task, corresponding data from the sensors, and RULA score.
Sensors 23 04863 g006
Figure 7. Ten scores for ten different postures and lifting three different weights.
Figure 7. Ten scores for ten different postures and lifting three different weights.
Sensors 23 04863 g007
Table 1. List of related works.
Table 1. List of related works.
ReferenceAssessment ToolData Acquisition MethodWearableNon-ObstructiveLoad Identification
[10,11,12,13,23,24]RULA/Modified RULA/SIOptical/Self-reportNoNoNo
[14,15,16]Muscle activity levelEMG/AnyBody softwareYesYesYes
[20,21,22]RULADHM/VR/CATIA softwareYesNoNo
[17,19]RULA/REBA/NIOSHInertialYesYesNo
[18]Artificial Intelligence basedInertial and opticalYesNoNo
Our workRULAInertialYesYesYes
Table 2. RULA score, action level, and corresponding action.
Table 2. RULA score, action level, and corresponding action.
RULA ScoreAction LevelAction
1–21Acceptable working pattern; no changes are required.
3–42Working pattern may be changed; hence, further investigations are suggested.
5–63Working pattern will soon require changes; therefore, further investigations are needed.
>74Working pattern highly risks health and should be immediately changed.
Table 3. RULA and DULA scores.
Table 3. RULA and DULA scores.
PostureDULARULAPostureDULARULA
166666
255776
354855
444966
5651077
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Tahir, A.; Bai, S.; Shen, M. A Wearable Multi-Modal Digital Upper Limb Assessment System for Automatic Musculoskeletal Risk Evaluation. Sensors 2023, 23, 4863. https://doi.org/10.3390/s23104863

AMA Style

Tahir A, Bai S, Shen M. A Wearable Multi-Modal Digital Upper Limb Assessment System for Automatic Musculoskeletal Risk Evaluation. Sensors. 2023; 23(10):4863. https://doi.org/10.3390/s23104863

Chicago/Turabian Style

Tahir, Abdullah, Shaoping Bai, and Ming Shen. 2023. "A Wearable Multi-Modal Digital Upper Limb Assessment System for Automatic Musculoskeletal Risk Evaluation" Sensors 23, no. 10: 4863. https://doi.org/10.3390/s23104863

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop