Next Article in Journal
Adaptive Hybrid Beamforming Codebook Design Using Multi-Agent Reinforcement Learning for Multiuser Multiple-Input–Multiple-Output Systems
Previous Article in Journal
A Human Body Simulation Using Semantic Segmentation and Image-Based Reconstruction Techniques for Personalized Healthcare
Previous Article in Special Issue
Kinematic Modeling of a Trepanation Surgical Robot System
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Assessment of Exoskeletons for Work Activities: The Dilemma behind the Product

by
Jérémy Lefint
1,* and
António B. Moniz
2
1
Institute for Technology Assessment and Systems Analysis, Karlsruhe Institute of Technology, 76133 Karlsruhe, Germany
2
CICS.NOVA, Nova University of Lisbon, 2829-516 Caparica, Portugal
*
Author to whom correspondence should be addressed.
Appl. Sci. 2024, 14(16), 7108; https://doi.org/10.3390/app14167108
Submission received: 27 May 2024 / Revised: 6 August 2024 / Accepted: 8 August 2024 / Published: 13 August 2024

Abstract

:
The introduction of exoskeletons by many companies has often resulted in potential users not wanting to wear them. Evaluations of the exoskeleton reveal benefits of use, including ergonomic advantages and a reduction of work-related musculoskeletal disorders. How can this contradiction be explained? By searching the available literature, we can identify the methods used to develop these devices and the methods employed to evaluate their acceptance and benefit. By looking at the product from different viewpoints and involving different disciplines, we will get to the root of this discrepancy. Our findings indicate that the product definition, development focus, and evaluation methods do not sufficiently and adequately address the primary goal of exoskeletons. The development of such devices should be prioritized not only for the delivery of support but also for the motivation of the user. Finally, we put forward the proposal to discuss the establishment of a novel development method for the prospective elaboration of exoskeletons.

1. Introduction

Today, the exoskeleton market for working purposes is well established. Whether by big-name companies, start-ups, or research institutes, a large number of exoskeletons have been developed in recent years. Such exoskeletons are becoming more common and can be found in different contexts of use, including the medical, consumer, military, and industrial sectors [1]. Exoskeletons are presented on the market as physical support devices or ergonomic solutions and promise to be the answer to a lot of current challenges in our society, e.g., the aging society, socio-demographic changes, the healthcare crisis, or “work-related musculoskeletal disorders” (WRMSDs) [2].
The concept of exoskeletons, designed to augment human strength and endurance, has a rich history that spans several centuries. The earliest recorded ideas of human augmentation through external support date back to the 19th century, with Nicholas Yagn’s development of a mechanical exoskeleton in 1890 [3]. This initial design, which utilized the user’s kinetic energy for propulsion, can be seen as a marker of human efforts to enhance physical capabilities through mechanical means. The mid-20th century saw further advancements, notably with General Electric and the United States military’s Hardiman project in the 1960s [4]. Although Hardiman was never fully operational, it set a foundational precedent for future developments by demonstrating the potential of powered exoskeletons.
The late 20th and early 21st centuries ushered in significant technological advancements, particularly in the fields of robotics, materials science, and computer engineering. These innovations have led to the creation of more sophisticated and practical exoskeletons. For instance, in 2000, the University of California, Berkeley, introduced the Berkeley Lower Extremity Exoskeleton, one of the first functional load-bearing exoskeletons capable of enhancing human mobility and strength in a practical manner [5]. Today, exoskeleton technology has evolved into a diverse field encompassing medical rehabilitation, industrial applications, military uses, or consumer use. Modern exoskeletons, such as the Dnsys X1 (https://www.kickstarter.com/projects/dnsys/dnsys-x1-exoskeleton-unleash-superhuman-athletic-potential?lang=de (accessed on 26 May 2024)), SuitX (https://www.suitx.com/de/produkte/ix-back-exoskelett (accessed on 26 May 2024)), or Rewalk (https://golifeward.com/products/rewalkpersonal-exoskeleton/ (accessed on 26 May 2024)), demonstrate the significant progress in making these devices more accessible and effective in everyday use.
The journey of exoskeleton development is not only a testament to technological innovation but also reflects a deeper cultural and psychological narrative. Sigmund Freud, in his 1930 work “Das Unbehagen in der Kultur” (Civilization and Its Discontents), coined the term “Prothesengott” (prosthetic god) to describe humanity’s increasing reliance on technology to extend its capabilities beyond natural limits. This idea resonates profoundly with the evolution of exoskeletons, which embody the human aspiration to transcend physical constraints through mechanical augmentation. In order to gain insight into the impact of exoskeletons on both humans and wider society, this particular product will be subject to detailed examination from a variety of perspectives.
An exoskeleton is essentially a device worn by a human user to support and assist movements of the body to accomplish a task, e.g., by overhand work. The fact that this involves coupling the user to a system that uses the skills of the former and the capabilities of the latter, as shown in Figure 1 (inspired by [6,7]), makes it clear that developing such devices encompasses many different disciplines, such as the physical sciences, social sciences, and even industrial design and ergonomics.
Unfortunately, and despite the existence of standards such as ISO 9241-110:2020 or the “Afnor” NF X35-800, the development of exoskeletons does not appear to follow a multidisciplinary approach [8] but seems to be still driven just by technology [9].
This symbiosis between humans and machines can be seen as having different uses. One goal might be to assist, another might be to rehabilitate, and a third might be to serve as a preventive measure, as Kuhn and Freyberg-Hanl proposed [10]. With this in mind, it is easy to understand the advantages such a symbiosis would entail. Figure 1 illustrates this connection, as well as the advantages and disadvantages of these two actors.
Solutions that support the status of people in an industrial environment play a decisive role in our society, especially in Industry 4.0 [11], and are therefore welcome. Consequently, the expectations of exoskeletons are very high. Their qualities, often extolled in many reports [12,13,14], promote the image of a fitting solution to problems related to profitability and generation change [15]. But how can the fact that exoskeleton developers struggle to achieve market acceptance be explained? The areas most courted by manufacturers in the industry are the automotive and logistics sectors. However, it is precisely companies in these sectors that are the most critical of such systems [16,17,18].
The reasons for this are many and varied; users may find them uncomfortable to wear, too heavy, user-unfriendly, excessively complex, or simply unsatisfactory [19,20,21]. These facts can contribute to lower acceptance in the work environment. Several authors approached this concept of technology acceptance [22,23,24,25], but that means that if this technology does not consider the users’ needs in its development, it may cause a conflict or resistance from potential human users to wear such devices.
There is considerable evidence to suggest that exoskeletons are products developed for the user when conventional ergonomic solutions fail to resolve a physical problem related to a specific activity. This implies that development should focus on the user. However, given the comparison of the number of publications on exoskeleton development and their acceptance, it is not difficult to recognize the focus of development, as shown in Figure 2.
The research field of exoskeleton + (motor/control/actuator/sensor), which is typical for engineering sciences, is much more covered than the field of acceptance (exoskeleton AND acceptance OR adoption).
Another fact highlighting the lack of research into the acceptance or adoption of exoskeletons is that the word “acceptance” does not appear in the title of any of the most cited articles listed in the Web of Science until line position 1906 by searching the topic “exoskeleton”. In contrast, the word “adoption” appears as early as line 118 and again at line 912. Out of the first 2000 most cited articles on exoskeletons in the Web of Science, only three mentions either “adoption” or “acceptance”. This could indicate a significant gap in the literature that needs to be addressed.
The aim of this paper is therefore to answer the following questions:
RQ.1: 
Why do exoskeletons need a specific method for development?
RQ.2: 
Are the evaluation methods suitable for exoskeletons?

2. Background

What makes an exoskeleton good, and what distinguishes a good exoskeleton from a bad one? The answer we will choose is maybe the modest one: the simple fact that it is voluntarily used on a daily basis by the user; in other words, that it is accepted and even adopted by its user. The answer may appear simple, but is it really? As far as the exoskeletons currently being developed are concerned, however, it seems this question has several valid answers.
We very often hear and read the terms acceptance, acceptability, or adoption being used without much differentiation, especially in the context of exoskeletons [26,27].
The terms “to consent, to suffer/to undergo, to admit, to tolerate or to endure”, which are used to define acceptance, imply that the user is bound to a certain resignation or even submission on their own behalf to using the object in question. This suggests that the “acceptance” process is not a decision fully taken by the user but supposes that other factors directly influence it.
This implies that a person, well before they first use the product (or other services), will evaluate it. This judgment can therefore form both the basis for and the start of the acceptance process. Once this process has been completed, the person will be able to decide, even if this judgment is made for the most part unconsciously, whether or not acceptance is given. Once the decision to accept has been made, the next stage, namely adoption [28], can begin, which should be the objective for any product, and particularly for exoskeletons. This theory is generally the basis for different acceptance model types.
To understand how the acceptance of exoskeletons will be evaluated, it is useful to understand how these models are formed and, in particular, how the “intention to use” is constructed. In fact, self-predicted future usage, or intention, has been a better predictor [29]. The acceptance models have been developed to study the use of information systems [30,31], but they have also been used to study other technologies, such as exoskeletons. To this end, we will use the UTAUT2 model, as shown in Figure 3. Used in many studies, this is one of the most recent and valid models [32].
Even if UTAUT2 was developed for a consumer-use context, according to Venkatesh et al. [33], it is nevertheless also used in many other contexts, as mentioned by Ain et al. [34].
This model is based on the fact that the use behavior of a technology depends on the intention to use it, and it considers that this intention is influenced by many factors from different sources. As far as UTAUT2 is concerned, Venkatech et al. agree on seven factors that influence intention [33].
The seven factors influencing an “intention to use “(or, directly, “use behavior”) are defined by the authors as follows:
Performance expectancy is defined as the degree to which using a technology will provide benefits to consumers in performing certain activities; effort expectancy is the degree of ease associated with consumers’ use of technology; social influence is the extent to which consumers perceive that important others (e.g., family and friends) believe they should use a particular technology; and facilitating conditions refer to consumers’ perceptions of the resources and support available to perform a behavior”.
Hedonic motivation is defined as the fun or pleasure derived from using a technology. The Price value (is defined) as consumers’ cognitive trade-off between the perceived benefits of the applications and the monetary cost for using them and the Habit is a perceptual construct that reflects the results of prior experiences”.
As the name suggests, technology acceptance models like UTAUT 2 provide information about the potential acceptance of a technology by its users. While it is possible to evaluate an exoskeleton based on its technical characteristics, such as support strength or joint mobility, it is clear that assessing the overall effectiveness and usability of an exoskeleton must also consider user acceptance. Consequently, it is necessary to examine the current methodologies for evaluating exoskeletons and their development processes to understand the current status quo.

3. Materials and Methods

The nature of the research question can be explored through a review of existing literature. Notably, there has been a significant increase in publications on the development and evaluation of exoskeletons in recent years, making a wealth of material available in international databases. Therefore, this literature review aims to provide an overview of the current state of the art in exoskeleton development and evaluation, utilizing a multidisciplinary approach for comprehensive reflection. The search is divided into two sections, each corresponding to a specific research question.

3.1. Method of Development

The first literature search was carried out using Scopus, IEEE, and Web of Science to find out which development methods are being used to develop exoskeletons.
The keywords used were: (development AND (process OR method)) AND exoskelet*. It should be noted that the obtained result does not encompass all the publications on this subject. Indeed, the term “product development process” comes from the terminology used by experts in product development but is probably not used by all the authors who study the development of exoskeletons.
Given the considerable number of publications, several filters were applied to refine the results and enhance their accuracy. When filtering by title and abstract, a different analytical approach was employed (qualitative), which considers the semantic meaning of the titles and/or abstracts. The developmental context of the articles was examined, and articles lacking an industrial context were excluded. Similarly, articles lacking clear methodologies or focusing exclusively on the development of specific components were also excluded. Finally, relevance to the content presented here was used as the final criterion for exclusion.
As shown in Table 1, the following filters can be described as follows:
  • Filtering term: “(development AND (process OR method)) AND exoskelet*”. These terms are central to this research. Given our focus on achieving the process of exoskeleton development, they were specifically chosen for this study. “Process” and “method” are often used interchangeably as synonyms and are therefore explored together in this context.
  • We limit the topic to focusing on the product itself. We are now looking at the following topics:
    Engineering.
    Rehabilitation.
    Computer science.
    Neurosciences neurology.
    Robotics.
    Behavioral sciences.
    Social sciences.
    Health professions.
    Multidisciplinary.
    Psychology.
  • Filtering is performed by focusing on keywords found in the title.
  • Filtering of publications older than 2020 was not considered (to ensure the timeliness of this article).
  • Filtering only articles.
  • Double-checking via the software Mendeley Desktop 1.19.8.
  • Filtering content based solely on the title.
  • Filtering is conducted based on the abstract content.
  • Finally, 44 articles were read in depth. In the end, we used only a total of two articles for this study because none of the others demonstrated a process behind their development. This raises the question of how exoskeletons are typically developed and whether a formal method (as known in the discipline of product development) is used.
  • The objective of this work is not to deliver an exhaustive overview of the literature but to offer valuable insights and highlight the most relevant publications. The research indicates that most of the articles found in this research concentrate on the technical development of specific components, functions, or software aspects of exoskeletons. In contrast, only a limited number of studies address the holistic development of systems intended for practical, real-world applications.
Table 1. Search for: (development AND (process OR method)) AND exoskelet*.
Table 1. Search for: (development AND (process OR method)) AND exoskelet*.
Web of ScienceScopusIEEE
Filtering term: “(development AND (process OR method)) AND exoskelet*”6454513744527
Filtering by the topics:3994358174527
Filtering based on the keyword in the title27068466
Filtering publications not older than 202013833440
Filtering article only13818033
Double-checking via the software Mendeley13617615
Filtering based on the title26322
Filtering based on the abstract12292

3.2. Evaluation of Exoskeletons

The second literature search in this article focuses on the evaluation of exoskeletons (Table 2). This research was also conducted using Scopus, IEEE, and the Web of Science. As in the previous paragraph, several filters were applied to the results in order to refine them and improve their accuracy. When filtering by title and abstract, a qualitative analysis approach was used, considering the semantic meaning of the titles and/or abstracts. The focus was explicitly on finding methodologies that were clearly explained and where users interacted with the exoskeletons in real-world settings. Furthermore, the articles were filtered based on their relevance to the research question and on methods that appeared suitable for exoskeletons. Additionally, attention was paid to ensure that the articles added value to the content presented here.
The keywords exoskelet* and (acceptance OR adoption OR evaluation OR assessment) were used.
Table 2. Search for evaluation of exoskeletons.
Table 2. Search for evaluation of exoskeletons.
Web of ScienceScopusIEEE
Exoskelet* and (acceptance OR adoption OR evaluation OR assessment)149729071073
Filtering by topics111122161073
Include in title205447187
Limited to 2020148298104
Article13617633
Doppelcheck via Mendeley12314333
Filter by title24325
Filter by abstract6123
The following results can be outlined as follows:
  • Filtering term: “exoskelet* and (acceptance OR adoption OR evaluation OR assessment)”: These terms are central to this research. Given our focus on achieving the evaluation of the exoskeleton, they were specifically chosen for this study. Acceptance, adoption, evaluation, and assessment are often used interchangeably as synonyms and are therefore explored together in this context.
  • The topics are limited to the following:
    Engineering rehabilitation.
    Computer science.
    Neurosciences neurology.
    Robotics.
    Behavioral sciences.
    Social sciences.
    Health professions.
    Multidisciplinary.
    Psychology.
  • Filtering is performed by focusing on keywords found in the title.
  • Filtering publications that are not older than 2020.
  • Filtering only articles.
  • Double-checking via the software Mendeley.
  • Filtering content based solely on the title.
  • Filtering is conducted based on the abstract content.
Finally, 21 articles were read in depth, and we added 6 articles from previous research on this topic. In total, we screened 28 articles for this study. From these, we selected 8 articles that clearly demonstrated an evaluation process, included a relevant step for evaluation, or provided significant arguments that contribute to understanding how exoskeletons are evaluated.
Similar to the previous research, the same filtering process was employed due to the high volume of publications. In summary, it can be stated that very few articles adopt a holistic approach to the evaluation of exoskeletons. Most focus on specific aspects of evaluation, covering only certain factors relevant to their adoption. However, some papers specifically address the question of a comprehensive analysis of exoskeletons. These papers will be discussed in detail in a subsequent section.

4. Analysis of the Previous Literature Research

As previously mentioned, the results are structured into two parts corresponding to the research questions. The first part consists of a review of the scientific literature on development methods considered “suitable” for exoskeletons, aiming to understand which methods developers are using. The second part focuses on evaluation methods and how the results are presented. While the presented results do not claim to encompass the entire body of literature, they provide a basis for discussing various perspectives to address the R.Q.s.

4.1. Method of Development

After a review of the selected articles, it was found that none showed any really new development methods specially adapted to exoskeletons. Only one article reflects on the method used to develop an exoskeleton. It is interesting, too, because it was written by the developers themselves. They reveal to us a useful starting point for exoskeleton design. Bornmann et al. [35] mention that “focus on an in-depth analysis of the user context and what they try to accomplish should be the starting point of the development” [35] (p. 2).
They also propose involving the various stakeholders in interdisciplinary workshops or interviews to best define the use case in question. The same authors continue their process with the aid of a “rough understanding of the general conditions and ergonomic problems” and by reproducing the task to be accomplished in the laboratory in order to better observe it and to highlight the mechanical and biomechanical aspects of the product to be developed. Moreover, they recommend testing the prototypes under real-world use conditions as well as carrying out comfort tests. They use objective and subjective scientific evaluation tests (these are not clearly stipulated) to manage the evolution of the prototype in the sense of its transition from prototype to product.
The same authors insist, however, that the results of both objective and subjective evaluation tests must be integrated into the product development process, but that they “[…] can be hardly transferred to other studies or exoskeletons due to the large variety of evaluation methods and the heterogeneous exoskeletal systems” [35] (p. 3).
A further article by Drees and colleagues [36] is based on VDI 2221 and develops the first three phases of the product development process with the aim of adapting these three phases to the specific needs of exoskeletons [36]. They begin by carrying out phase 1 of the VDI 2221, namely Clarification of Problem or Task, and also by analyzing the user’s movements during the execution of a specific task and transcribing them into a list of requirements. The second task focuses on the basic movements performed in the specific task, which allows the movements to be structured or linked to their function. These movements are then classified in a matrix called “ExoMotion”. In the next task, this data will be translated so that a framework for prototype design can be built. The authors insist that the generated solutions must be classified to be able to fulfill future needs. The same authors also note that the use of personas could be a good way of responding to the users’ need for the exoskeleton that is to be developed to be customized. Nevertheless, they consider the persona to be “one” method of analyzing the needs of users. They also note the necessity of verifying whether these methods are suitable for generating requirements.

4.2. Evaluation of Exoskeletons

The first study by Elprama et al. from 2020 involved the testing of two exoskeletons in an industrial environment and using the UTAUT questionnaire to understand and highlight the significant aspects governing the acceptance of exoskeletons [37].
Thanks to this research, the authors were able to show that effort expectancy plays a primary role in the intention to use and, thus, in the acceptance of exoskeletons. They explain that this effect is because exoskeletons “are especially developed to reduce efforts whereas performance increase is only of secondary importance” [37] (p. 344). Furthermore, they demonstrate that “people with high levels of perceived usefulness and perceived ease-of-use will have a more positive attitude toward exoskeletons” [37] (p. 345). They also point out that the intention to use by young users is particularly influenced by performance expectations. They caution, however, that the UTAUT is not suitable for this specific research and requires some modification. They also note that the UTAUT can only be applied to technologies already in use, i.e., where some user experience is already available, and not to research on the imminent introduction of exoskeletons.
The second example is based on research conducted at a German car manufacturer by a group of researchers employed by the same company. Hensel and Keil used different methods to evaluate the exoskeleton in the company with 51 participants (all male) [16]. Firstly, they recorded work-related physical discomfort and then used the body map (Body Part Discomfort Scale by Corlett and Bishop) to localize the uncomfortable areas of the exoskeleton. To identify the intention of use and the usability (defined by the authors as the ease of use and the meeting of requirements), they used the UMUX lite (Usability Metric for User Experience from Lewis, Utesch, and Maher). These analyses were conducted at the beginning of the test period over a period of 4 weeks in total.
Hensel and Keil conclude that discomfort is the source of a negative intention to use. They also highlight the correlation between usability and acceptance, as well as its opposite. At the same time, however, the authors point out that the exoskeletons had a positive effect on physical discomfort, even though the issue could not really be validated.
Thirdly, we find a research group (Ferreira et al., 2020) that is studying the relationship between and the importance of the intention of use (I.U.) and user acceptance (U.A.) [22]. They have focused on six workstations in a car assembly plant over a period of four weeks with 88 volunteers. They report a positive correlation between U.A. and I.U. but also a negative correlation between discomfort/perceived effort and U.A./I.U. They also note that U.A. and I.U. are influenced by the type of task performed, and that perceived utility will be high and discomfort will be lower if the task is in accordance with the primary function of the exoskeleton. They also conclude that I.U. and U.A. decline as the user uses the product.
The fourth example comes from a study by Hoffmann et al. conducted during an EU-funded project that used structured interviews to highlight the barriers created by exoskeletons with the aim of facilitating their integration into small and medium enterprise (SME) environments. These researchers first revealed the need for exoskeletons in the SME environment [38], because, in their opinion, the risks of musculoskeletal diseases are higher there than in industrial or automotive environments. They also stress that tasks in the many different branches where SMEs operate tend to be very varied and therefore not suitable for the use of exoskeletons. Hoffmann et al. claim that most exoskeletons provide support only for a specific movement and can often cause discomfort. They also point out that the price–performance ratio of these devices can be very negative if the active exoskeleton is not comfortable enough. They believe that the acceptance of exoskeletons can only be increased if an exoskeleton is chosen that is specifically tailored to the working conditions, the environment in which it is to be used, and, above all, the target result. They warn that the benefits of exoskeletons are often extolled too euphorically due to a lack of knowledge. This same lack of knowledge can, in some cases, also lead to distrust of the devices. Hoffmann et al. emphasize that acceptance can have many facets. They give the example of a bad experience with a lifting system that, because of the complexity of use and the time required to operate it, was not accepted by the employees and could have repercussions for the integration of new technologies such as exoskeletons. On the other hand, they also report that the opposite outcome is also possible.
The upcoming publication by Wioland et al. (2024) explores a method to prevent occupational risks by examining the adoption process of exoskeletons [39]. It highlights the use of a special questionnaire to assess the interaction quality between humans and the device at various stages, from initial acceptability to ongoing acceptance. This approach should identify potential issues early on, allowing companies to make necessary adjustments before full implementation. During “acceptance”, the questionnaire should help to uncover new issues and track changes in previously identified problems. Long-term use of the questionnaire may enable continuous monitoring and corrections until the adoption process stabilizes. The authors, Wioland et al., explain that “persistent tensions” identified through the questionnaire can indicate risks, making it a valuable tool for early risk detection and management throughout the exoskeleton’s adoption.
A recent article by Hussain et al. also proposes a new Exoskeleton Usability Questionnaire (UEQ) to assess the usability of lower limb exoskeletons [40]. For the first experiment using this questionnaire, only 20 people participated. Hussain et al. aim to assess subjective feedback and argue that a questionnaire specifically addressing exoskeletons is still missing and needs to be developed. Therefore, they have proposed a new questionnaire focused on the specific attributes of exoskeletons. The researchers identified four crucial factors for the usability and acceptance of these devices: mobility, adjustability, handling, and safety. The description of these factors exclusively focuses on the immediate use of the device. The resulting questions from these factors primarily address usability aspects such as wearing and removing the exoskeleton, storage, comfort during movement affecting body joints, and device safety. They also acknowledge self-imposed limitations in their study, mentioning their use of students as participants and conducting tests exclusively in a laboratory setting.
The next article is not directly related to the evaluation of exoskeletons themselves but aims to highlight a significant aspect of their evaluation through user adoption [41]. The authors, Gonsalves et al., claim that identifying and integrating various stakeholders into the development process is crucial for the adoption and implementation of exoskeletons in environments such as construction companies. They argue that understanding the facilitators and barriers will help developers and the scientific community design exoskeletons that users are more likely to adopt. Their work identifies eight stakeholders, with six considered important: construction workers, safety professionals, supervisors, corporate management, ergonomists, and certifying bodies. Furthermore, they present two lists detailing the facilitators and barriers to adoption. The facilitators are categorized into six groups: applications, challenges, modifications, benefits, compatibility, and implementation. Although no classifications were made for the barriers, the authors highlight the most relevant barrier as “catch and snag risks” and the least relevant as “cultural beliefs”. These comprehensive lists provide valuable insights for developers.
The last article (also by Elprama et al., but from 2021) extends the results already obtained here and focuses on the factors influencing the acceptance of exoskeleton use in industrial settings. This empirical study was conducted using a systematic search of the literature on the individual use of exoskeletons [42]. This study, which included 35 articles, reported that 14 of them used “real” users to wear the device and that 80 of the 850 participants were female. The researchers illustrate their results in a graph, as shown in Figure 4: It is divided into four parts, which can be regarded as a framework for exoskeleton acceptance.
Depicted in the outer part are the various characteristics influencing the use of an exoskeleton, which are divided into five blocks. In the third part, each block is named: physiological factors, psycho-social factors, work-related factors, policy-related factors, and implementation-related factors.
The second dimension relates to the I.U., while in the middle—and thus in the center of interest—we finally have the exoskeleton.
Elprama et al. wish to motivate developers to design their product from more of a user point of view than from an engineering perspective and hope that their work will serve as the basis for further research.

5. Answer to the Research Question

RQ1: 
Why do exoskeletons need a specific method for development?
This research question raises the issue of the need for a method specifically designed for those products that require a deep understanding of human/machine interaction. Indeed, it is possible to develop an exoskeleton like any current mechanical system. An exoskeleton is (from a purely material viewpoint) merely a set of mechanical components, textiles, and electronics (if active). In other words, it makes sense that an exoskeleton can only be considered a technical system, like many others.
Some of the authors, like Bornmann et al., mentioned that exoskeletons are, contrary to this idea, products that require an interdisciplinary approach and therefore an adapted development [35]. They suggest, therefore, that their development should begin only after an exact analysis of the target task and the biomechanical requirements, and only once the use context has been fully understood. Though this can certainly be taken as an adequate starting point, it nonetheless leaves some questions unanswered, for example, “how” it should be developed. Furthermore, is it valid to question whether brief inclusion (in the form of workshops or interviews) of the stakeholders at a certain point in the design process and laboratory tests are sufficient to produce a socio-technical product whose purpose is to assist a user in everyday tasks [8,43]? To answer this question, it is necessary to clarify some basic components of exoskeletons.
  • The product and its attributes
It is of the utmost importance that the product in question, the “wearable assistive physical device” (WAPD), be precisely described, as this is a fundamental aspect in the process of selecting and designing its characteristics, as proposed by Karafillidis and Weidner [44]. To illustrate this concept more clearly, we will employ the following exaggerated example: While a pair of high-heeled shoes is not ideal for hiking, walking shoes undoubtedly will be. If my goal is to walk 20 km with a backpack, then I will probably choose hiking shoes. But what if we ask ourselves the question about the differences between these products by focusing only on the main function of the product itself? After all, both products are shoes whose main function is to be worn on the feet in order to protect the skin. They are made for the most part from textiles, both have heels and a sole, are anthropomorphic, available in various sizes, and aim to provide (bare) feet with a certain comfort. A distinction will quickly be made when using them in their respective contexts. Recognizing the main function and the context of use may be sufficient to develop a new product but may nonetheless fail to satisfy the user. In fact, there are many different types of hiking boots: low, high, white, black, sole density, price, different materials (vegan, leather), etc. Ultimately, these factors represent the individual details of the product. They are the ones that developers will design, and they are the important features that will help a potential buyer decide to buy (or wear) this rather than any other model. Incidentally, high heels tend to be regarded as uncomfortable shoes yet are still worn by numerous people, so does this imply that the primary purpose of such shoes is not solely “functional”? May a similar distinction be made with exoskeletons?
These WAPDs are also characterized by their intense relationship with the wearer. They allow the user to allocate, transfer, assign, or even reduce some of their constraints. In an industrial context, it is clear that these systems, being a special kind of tool, are intended to provide their users with assistance and thereby increase their competence. Although the aspect of assistance is largely debatable and even misused, as pointed out by Krings and Weinberger in 2017 [23], it is quite correct to describe exoskeletons as socio-technical systems. Gransche proposes an interesting vision in which [45], in addition to refining the definition of exoskeletons, submits the idea that these devices aim to afford the wearer a certain “comfort”. Without it, a task can be performed, but with difficulty. By relating comfort to assistance, he explains that assistance (a system) will be used to perform a task (even for non-needy people) easier and, in this way, more comfortably than without assistance, and therefore will be used.
If the exoskeletons used in an industrial environment are objects that serve only to increase the user’s comfort, as the author defines it, and are therefore not indispensable to the execution of a task, then it is quite understandable that they should come equipped with more than utilitarian aspects. To put it differently, it means that hiking shoes do not need to be only functionally appropriate to be worn/wanted by the user.
The difficulties involved in dealing with these aspects are also widely discussed by Savolainen and Hyysalo [24], who, through their work, highlight the complexity of this issue and the fact that it also deserves further research.
The qualities of a product are directly influenced by the way in which it is defined by its designer.
  • Subjective factors
Some authors, such as Law et al. in 2007, discuss in a UX manifesto what qualities a product should have to satisfy the user, or more precisely, the interaction between them, talking in this context about the pragmatic and hedonic dimensions [46].
In this manifesto, Hassenzahl defines the pragmatics dimension as “the product’s perceived ability to support the achievement of “do-goals”” [47]. By contrast, he declares the hedonics dimension to be “the product’s perceived ability to support the achievement of ‘be-goals’”. He explains that the be-goal is “being competent”, “being related to others”, “being special”, etc. Typical pragmatic qualities are stimulation and identification, while pragmatic qualities include functionality and effectivity.
He also informs that developers are the ones who, “by choosing and combining specific product features, i.e., content, presentational style, functionality, and interactional style” [47] (p. 2), will determine the final character of the product. He also notes that this “desired character” of the developers does not always reflect the character that the user will experience when using the product. He further adds that “people value products on the basis of how they satisfy needs in particular situations” [47] (p. 11).
Qualities such as stimulation or identification are insufficiently represented in the context of exoskeletons and require more research. The fact that these qualities are subjective in nature [47] may be one of the reasons for their absence in the context of exoskeleton development.
Identification and stimulation may well be the very qualities that would play a key role in the adoption of these devices. Elprama et al. (2023) list a variety of requirements relevant to the development and evaluation of exoskeletons. They clearly emphasize the importance of factors such as the appearance of the exoskeleton during use: “[…] even if exoskeletons are effective and can support workers, workers might not wear them if they feel embarrassed to be seen wearing one.” [48] (p. 13). Similarly, the authors support the idea that non-instrumental characteristics improve the perceived function of the product [49,50].
It therefore makes sense that a development method should be used that considers both a product’s pragmatic and hedonic qualities, i.e., not only objective but also subjective factors.
  • User/developer relationship
Another study by Fischer et al. (2020) on user involvement by older adults highlights the importance of the user/developer relationship [51]. Indeed, it is developers who decide what knowledge from users is to be included or not in the product development. Moreover, the authors underline the fact that developers have some control over “which role, stage, level, and images prevail, and how participants are selected”. They also state that user choices (during testing or data collection) directly influence the requirements. Indeed, if you take people who all agree that hiking should be done in the mountains, then it is likely that the requirements collected will be aimed at developing a shoe suitable for mountain walking, with the result that those who prefer to walk in the countryside will probably not be satisfied with the product.
It is therefore understandable that the choice of users has a huge influence on the requirements and should not be underestimated by the developers. Furthermore, this stresses once again how important it is for the users surveyed to be the ones who will actually use the product in the future.
Despite the consensus that the user should be integrated into the product development process, it should also be highlighted that, according to Ármannsdóttir et al. [52], there is no formal evidence that this in fact happens. This naturally raises the question of whether the needs or wishes of the users are properly recognized and therefore used as the basis for development. In contrast, as described by Lowdermilk [53], “knowing when to listen to users and when not” is important because they do not always know which solution is possible or not. Nielsen adds that users only know how to use a product and that they cannot predict (before they actually begin using it) whether a future product will be useful to them or not and thus cannot yet know which product they will need [54].
A certain additional and inevitable ambiguity therefore exists. The authors recommend observing user/product interaction under actual use conditions and while performing real tasks [54,55]. As already described above and widely reported by Endter [43], it is indeed vital to carry out tests in real conditions of use and, of course, with those who are supposed to wear the WAPDs; without any knowledge or experience of socio-qualitative methods [56], however, it appears difficult to understand how this ensemble (the assistance provided as well as the object itself) is really perceived by the users and, even more importantly, whether it corresponds to their expectations.
As Hoffmann et al. (2021) point out, developers focusing on aspects such as “workplace, work organization, the support device, or (economic) company objectives” [57] (p. 527) forget the importance of the relationships between all these aspects and those that are important to the user. Effectively, by focusing on the problem and the solution, the development strategy can overlook the needs, wishes, desires, etc. of the users.
The user/developer relationship is an important factor that influences development.
Answer to RQ1: On the one hand, we have the final product, an exoskeleton, which is still not worn by its users, while on the other, no complete method, especially adapted for the development of exoskeletons as a socio-technical product, can currently be found in the literature. Indeed, any discipline dealing with the evaluation or development of exoskeletons will have to take many factors from different sources into account; there are disciplines that attempt to understand the intrinsic acceptance mechanisms of the final product, while there are those that have the tools to develop exoskeletons. A solution to this challenge has already been proposed by the real-time technology assessment team and must now be validated for use with exoskeletons, as mentioned by Guston and Sarewitz [58].
In fact, a new development method could fill this gap and help to take this special product into account by systematically integrating the necessary subjective factors into the development process.
RQ2: 
Are the evaluation methods suitable for exoskeletons?
Ergonomic problems and work-related musculoskeletal disorders, identified by ergonomics experts in various studies (e.g., Peters and Wischniewski [2]), seem to justify the development and use of devices like exoskeletons. According to many authors (e.g., [8,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61], these devices are frequently cited as potential solutions to these issues.
Working on evaluations and assessments, several authors (e.g., [8,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65]) concluded that a lack of comfort experienced by users when wearing an exoskeleton is an important reason for its rejection.
This raises the question of whether comfort can be a sufficient reason to reject the exoskeleton and whether the need was recognized by the expert. In addition, how can the evaluation method yield such positive results yet still not indicate acceptance of the exoskeleton?
Three key elements form the basis for ergonomics, namely physical, work-related, and cognitive factors. All three are equally important, and all play their part in achieving a good ergonomic outcome, as referred to by the Association for International Ergonomics [66]. One aspect of ergonomics that is increasingly used in the jargon of developers is usability. While ergonomics focuses on the task to be accomplished, usability includes aspects of effectiveness, efficiency, and satisfaction (example, DIN EN ISO 9241-210:2010) [67]. Satisfaction seems not to be emphasized enough and should be given higher priority in usability analyses.
This focus on the aspects of effectiveness and efficiency may be due to the fact that they can be objectively measured by increasingly precise tests, as Daub et al. point out in their article [61]. These results are subsequently cited as evidence of a system’s effectiveness. Even though certain studies, like Gonsalves et al. (2023), gather a variety of highly relevant factors for the adoption of exoskeletons [41], aspects such as fun/joy, “gamification”, or satisfaction during use seem to have been overlooked. The same applies to the newly developed Exoskeleton Usability Questionnaire (UEQ) referred to by Hussain et al. [40].
To answer RQ2, we have to more thoroughly understand how the exoskeleton will be evaluated and which disciplines are able to achieve this.
  • Focus on objective measurement
One of the most widely used tests in usability analyses is the questionnaire known as the System Usability Scale (SUS). However, rather than enabling a system’s usability to be analyzed directly, it merely permits a comparison between different systems and is only supposed to give an idea of whether the system in question was easy to use or not. On the other hand, this questionnaire only appears suitable for measuring the performance of a system [68]. The use of the SUS for individual analysis or to demonstrate the system’s usability is still widespread and validated by many studies. Nonetheless, it is unclear whether the SUS questionnaire is correctly used and whether it is really able to analyze the usability (effectiveness, efficiency, and satisfaction!) of exoskeletons. One answer to this question, as far as the “satisfaction” aspect of usability is concerned, is proposed by Hassenzahl et al. (2000):
“We are aware that user satisfaction is a part of the usability concept provided by ISO 9241-11. However, it seems as if satisfaction is conceived as a consequence of user experienced effectiveness and efficiency rather than a design goal in itself. This implies that assuring efficiency and effectiveness alone guarantees user satisfaction”.
[69] (p. 202)
Consequently, it can be argued that the satisfaction recorded by these questionnaires, resulting from the interaction between the user and the system, reflects only part of a system’s usability since it merely focuses on its functional qualities. On the other hand, the testing period of these systems is relatively brief [70]. It is therefore questionable whether such a short period of time is sufficient to assess the usability of an “everyday” exoskeleton. To return to the shoe analogy, it is not uncommon to get blisters from new favorite shoes, even if they seemed very comfortable during the short trial period in the store. This misinterpretation is possible, even though we are all experts when it comes to wearing shoes.
Research conducted to examine exoskeleton use by older people highlighted psychosocial considerations, which can have a major impact on the acceptance of exoskeletons [71]. De Bock et al. also note this problem [8]: tests carried out in the laboratory, on a test circuit, or as mentioned by this author as “parcours” [70], even if they reflect the task performed, are not sufficiently representative of the conditions experienced when performing a task in actual use conditions. In addition, the evaluation of the exoskeleton using sensors can introduce numerous uncertainties that may lead to inaccurate results, as pointed out by Li-Baboud et al. [72].
This discrepancy may be since users in the laboratory tests focus on the device itself, meaning that for a short period of time they may ignore factors that are important for the execution of the task in an everyday context. This may also explain why exoskeletons are initially accepted and then rejected some time later [73].
Besides the issue of usability not being fully analyzed, it can be argued that usability, even if it is judged to be favorable, does not seem to be a good indicator of acceptance or adoption of a system, even though it certainly increases the chances of a system being adopted. If the lack of acceptance or non-adoption is regarded as a symptom (the lack of comfort being simply an expression of the latter), it is questionable whether the developers, who will ultimately come up with ergonomic exoskeleton solutions by analyzing the ergonomic problem experienced by users [51], have really answered the (real) problem.
Then again, is it truly this physical need analyzed by experts that will lead to exoskeletons being adopted by their users, or are there other factors that will determine whether an exoskeleton will be used or not? The initial step may be to inquire about the user rather than measure them.
It therefore makes sense to conclude that an evaluation method should be employed that considers all aspects of usability, with a particular focus on the satisfaction factor (after a genuine trial in a genuine context for a genuine “worktime” with a genuine user).
  • User Feedback
According to the general statement “A good exoskeleton is one that is used”, the true value of an exoskeleton lies in its practical application and utility rather than its theoretical capabilities or design features. If “use” is the only criterion for regarding an exoskeleton as good, would imposing a rule making its use mandatory then make it good? Probably not. Perhaps the best exoskeletons are those that people decide to use of their own accord because of the advantages they offer.
In other words, there are two possible reasons why exoskeletons might be worn by their users: either they are obliged by some regulation to wear an exoskeleton (as is the case with construction helmets, goggles, or ear protectors), or users are themselves motivated by an exoskeleton’s qualities to wear it. Before requiring employees to wear exoskeletons, however, it is certainly sensible to attempt to get users to appreciate or even desire them, as many companies are in fact trying to do.
Diefenbach and Hassenzahl (2017) wonder whether people might be willing to pay for hedonic qualities. They suggest that certain pragmatic aspects are more justifiable than hedonic aspects (own translation from German): “That it is appropriate and justifiable to pay for pragmatic quality is beyond question. Paying for hedonic quality, on the other hand, seems questionable” [73] (p. 69).
If it is indeed easier to justify one’s choice of pragmatic qualities than hedonic ones, is there also evidence of this same behavior in the user feedback? Is it possible that user feedback only focuses on pragmatic aspects such as comfort because these are easier to justify? In the literature review, we found a similar pattern. For example, “ease of use, including putting on/off and comfort”, was rated as the most critical facilitator for the adoption of exoskeletons [41]. Indeed, much of the literature [8,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61] presents mainly instrumental feedback and very little subjective feedback. In fact, we only rarely find user feedback criticizing, e.g., the aesthetic aspects or expressing fear of the new technologies in relation to excessive workloads [74], loss of employment, loss of salary, etc. However, these aspects, as discussed by Siedl and Mara [75], may have a crucial impact on the user’s entire socio-technical environment. In some cases, subjective aspects will even be mentioned, yet the reasons given for not wearing the exoskeleton will be objectified in a second step, according to Theurel [76].
The author points out that expectancy performance has less influence on behavioral intention than hedonic motivation (in a non-organizational context). It can therefore be concluded that exoskeletons should have hedonic qualities [33].
  • Technology Acceptance Model
Furthermore, it can also be argued that using conventional acceptance methods such as TAM in Davis et al. [77] or UTAUT2 to perform a product analysis reveals only whether the object in question will be used or not, but not whether it will actually be adopted. Moreover, some authors, such as Renaud and Van Biljon (2008), also mention the fact that TAMs are not adapted to special needs, such as those that senior citizens may have in the context of product use. The Senior Technology Acceptance Model (STAM) [78] was developed in response to the need for the TAM to be modified for people over 65 whose needs cannot be included in the TAM [79]. This enabled them to obtain a more accurate and therefore more valid TAM by adding certain specific categories, as shown in Figure 5. In the Almere model [80], they added factors such as anxiety, trust, cognitive ability, etc., as well as physical aspects such as state of health. This raises the question of whether a TAM adapted to devices such as exoskeletons would perhaps be an effective way of predicting their acceptance, or better yet, their adoption. Indeed, aspects such as “a usage decision not taken by the user”, physical needs, or monetary advantages/disadvantages are difficult to represent in a TAM; the user could, for example, lose a bonus granted to compensate for the difficulty of performing a given task.
  • The Internal Balance of Use of Technology Ha/P.V.
In the same way that the adoption of a system (e.g., an exoskeleton) is a process that results from favorable acceptance [78], it can be assumed that the user will weigh up the advantages and disadvantages of using a specific technology. This intrinsic process can be described as a comparison between the value of wearing an exoskeleton and the handicap it causes. If the perceived value (P.V.) is higher (or weighs more heavily, as represented in Figure 6 below) than the handicap (Ha), then the user will see a benefit/advantage, i.e., a motivation to wear an exoskeleton.
The Ha that is inevitably generated when an exoskeleton is worn can stem from many different sources: comfort, weight, anthropomorphism, aesthetics, beauty, etc. P.V. is an intrinsic factor for the user and can likewise take many different forms. For example, a user is provided with an exoskeleton to help them cope with heavy and repetitive loads. However, the user considers carrying these loads to be a good way of building up muscles or staying physically fit while working. For this particular user, wearing an exoskeleton will not entail any P.V. (if the P.V. offered by the exoskeleton is of the muscle relief type). Another possibility is that the user felt privileged to wear an exoskeleton, as his colleagues did not have this opportunity. The P.V. for him was that his colleagues recognized that he had been chosen to wear an exoskeleton. It is also important to consider that P.V. is an evolving factor over time. On the other hand, if the user acknowledges the need to wear an exoskeleton, then it may be used and later accepted, as Baltrusch et al. studied [81]. To answer the question “Can an exoskeleton therefore be described as accepted or adopted simply by the fact that it is used?” we must understand the relationship between P.V./Ha, technology acceptance models, and use.
With the UTAUT2, we have already introduced an instrument used to evaluate the acceptance of exoskeleton systems. In the same principle, the UTAUT 2 builds like the TAM, with the assumption that before the use of a technology, a process takes place. Due to the simplicity of the TAM, we will use it here to explain this concept. As depicted in Figure 7, there are two distinct stages: the intrinsic stage, which takes place before use, and the actual use.
The P.V./Ha balance can only be established once the user actually begins using the product (indicated as “Use” in Figure 7). Following the very first experience of the product itself, the balance will begin to shift to one side or the other after a phase of reflection. It may eventually become positive even if it was negative at the beginning, as demonstrated by Norman [82]. It can be argued that this balance is neutral before use and therefore becomes established only after a period t of use, maybe about a month, as in Carroll et al. [83].
The illustration below (Figure 8) depicts a view of the intrinsic process of adoption, acceptance, or rejection of an exoskeleton, set in relation to the user experience, technology acceptance models, and the internal balance of use of technology.
Before use: The user gains an initial impression of the use of the product with the help of external variables. With increasing knowledge, the user recognizes its usefulness and/or ease of use. This stage can be modified according to the chosen TAM, provided that the latter gives some idea of an intention before use.
At this stage, the user cannot yet weigh up the Ha/P.V.
Use: The user begins using the product and tests it a priori, exploring the benefits of the product. At the same time, the user notes the handicaps involved in wearing it.
A first impression of the Ha/P.V. is formed.
Reflection: This is indeed the stage at which the pros and cons will be weighed against one another; though this decision may be made quickly, it may also require several “trial runs”.
The decision is made, with the balance shifting one way or the other.
This model also considers the fact that most exoskeletons will not be “chosen” by the user but imposed upon the user by others. In such cases, the decision to wear the exoskeleton is no longer up to the user, and the Ha/P.V. balance will not be considered. This step will eventually lead to acceptance or resignation if the user does not want to wear the product. It is, of course, conceivable that the user will recognize the benefits of the product and voluntarily choose to wear it even during this phase. However, the imposition of use precludes the choice made when use is voluntary. This can be referred to as the “filter” before the process of appropriation [83].
After use: At this stage, the adoption process can begin. It is important to note that, although the process is underway, it can be interrupted or changed at any time.
Note 1: External variables can influence the process at any time.
Note 2: Theoretical models of acceptance aim to understand the steps needed to achieve favorable acceptance but provide little or no information about the post-use phase.
Note 3: User Experience (UX) focuses on the user experience in all its phases and adds aspects such as stimulation and identification. It also looks at the emotions felt by the user during the product experience.
As shown in Figure 8, only the user experience can cover all the phases that the user undergoes. In other words, this method appears suitable for evaluating and designing exoskeletons. This stage is also similar to Level 2 in the model of technology appropriation [83].
  • Development Context
The next point describes the development context. Indeed, if we define an exoskeleton as a product that claims to provide support for its wearer, then the product should be developed to achieve this. On the other hand, if it is primarily a question of technological advances, then it is reasonable to ask what role the user plays in the development process. A better definition of the product, as argued above, can provide an adequate solution here.
The multidisciplinary approach should not end with the physical or biological sciences, however. A full ergonomic evaluation could lead to genuinely better exoskeletons, yet unfortunately, only a few such evaluations can claim to be complete. Most fail to track user satisfaction or use any psycho-social evaluation methods. Even if we find the terms acceptance or user integration being used, and even if this is shown to be at the heart of the ergonomic process, it seems doubtful that a truly multidisciplinary approach will be possible on account of the considerable homogeneity of the disciplines mentioned (e.g., Daub et al. [61]). A multidisciplinary approach to exoskeleton development should incorporate the social sciences as well as industrial design and other disciplines if necessary. Although many authors, such as Ármannsdóttir et al., focus on testing exoskeletons to understand the user/exoskeleton relationship [84], we could not find any article that clearly articulated a development method that would help developers understand and act upon the subjective factors discussed in this article. The difficulties involved in integrating these factors reveal how the process should ideally be, as described by Bai et al. [9], though it does not ultimately achieve the desired results. Furthermore, the search for a scenario in which this device could actually be used does not match the methods employed. On the other hand, it is possible to read about a result that seems to be driven by research focused on the technology itself rather than on the users’ needs.
In their paper “Roadmap Toward Large-Scale Adoption of Occupational Exoskeletons (O.E.s)”, Crea et al. (2021) describes the expectations of the different actors involved in the development of exoskeletons. They explain that “As different stakeholders have different interests, they may need different types of information to build their opinion on O.E.s” [85] (p. 12).
However, the fact that 97% of the developers claim to incorporate the user into their development process and 71% claim to determine the users’ requirements through functional testing [52] suggests that there is a mismatch between the development method used and the exoskeletons developed. This highlights the need for a paradigm shift in exoskeleton development.
Answer to RQ2:
RQ2 can be answered in three distinct ways:
Firstly, the evaluation tool appears not to be suitable because it focuses only on the objective part of the interaction or on the device itself.
Secondly, P.V. is not accessible to developers, which may lead to a divergence between the qualities that exoskeleton users expect and those they experience. It is therefore necessary to involve disciplines that are able to understand and analyze this kind of knowledge. Social sciences and user experience could be suitable for adding new input to the development process.
Thirdly, acceptance models may not be accurate enough to clearly predict future adoption and are therefore not usable for exoskeleton development.
It therefore makes sense that an evaluation method should be used that considers both a product’s pragmatic and hedonic qualities, i.e., objective and subjective factors. It seems equally important to us that the correlations described above in Figure 8 are used to determine the results and are not considered in isolation to draw conclusions about a certain degree of acceptance that is apparently measured incorrectly or incompletely.

6. Conclusions

Exoskeletons are devices that will undoubtedly become fully integrated into numerous applications. Given their significant advancements in recent years, we can confidently anticipate continued and impressive improvements in their development.
Looking to the future, advances in artificial intelligence, materials science, and biomechanics will dramatically expand the applications and understanding of exoskeleton technology. Future exoskeletons are likely to incorporate advanced AI algorithms to provide personalized rehabilitation programs that adapt in real time to the user’s progress and provide the individualized support needed. Progress in brain-machine interfaces could enable direct communication between the user’s nervous system and the exoskeleton—both afferent and efferent—providing more natural and intuitive control that could significantly benefit patients.
In industry, exoskeletons have the potential to become standard equipment in sectors where heavy lifting and repetitive tasks are required, thereby reducing the physical strain on workers. Future exoskeleton models may be equipped with smart sensors that monitor fatigue levels and environmental conditions, providing real-time feedback to enhance safety or even demonstrate the newer process. The integration of exoskeletons with collaborative robots could revolutionize manufacturing and construction by combining the benefits of both human and robotic workers.
In the military sector, exoskeletons have the potential to enhance the strength, endurance, and protection of soldiers, for instance, through the provision of real-time health monitoring, as proposed recently by Koch and Keisinger [86]. The integration of trained capabilities into exoskeletons could facilitate the fulfillment of novel tasks in the context of logistics and supply missions. As the technology becomes more affordable, exoskeletons could also be introduced to the consumer market, where they could provide support for people with mobility issues. The potential for lightweight, unobtrusive designs to make these devices practical for everyday use is significant, as it could help users maintain or improve their independence and quality of life. This is analogous to the way in which bicycles with electric motors have become a common sight on our roads. In addition to their applications in medicine and industry, exoskeletons could also be used to support everyday human activities, from recreational sports to housework. As exoskeletons become more widespread, it is important to consider the legal framework that will regulate their use and development. The integration of exoskeletons into the world of work will require adjustments to labor markets and vocational training programs, necessitating the acquisition of new skills and potentially displacing certain job roles while generating others [87].
It has become clear that such devices are special products because they are closely linked to humankind and personhood. This intense connection means that their development requires a holistic and multidisciplinary approach. The lack of adequate tools for their development, as well as for their assessment, requires further work. The development methods used take only limited account of the balance between objective and subjective factors (hedonic vs. pragmatic qualities) and do not appear to be suitable for multidisciplinary approaches.
Furthermore, the instruments used to evaluate exoskeletons reflect only a small number of their advantages and are therefore not precise enough to evaluate the product as a whole [88]. Besides the physical advantages, satisfaction is a quality that is too often neglected on account of its subjective/qualitative value. The TAMs do not consider the “after-use” of the exoskeleton, which leads to an incomplete evaluation of the system with respect to its future adoption. Utilization of retrospective interviews like the CORPUS or a method like Real-World Lab may help to gain a better understanding of what kind of quality is still missing to achieve adoption [89,90]. Hence, there is a need for a truly multidisciplinary approach (physical sciences, biological sciences, social sciences, industrial design, marketing, etc.), as already underlined by the authors [41], and the integration of different stakeholders along the development process.
The utilization of tools such as the “action sheet” (as Kunz et al. in 2016) is a promising approach to addressing the deficiency in stakeholder integration [91] and could serve as a foundation for the development of the proposed method.
The need for a development method that fosters mutual understanding between the different actors involved and their need for a common language are equally important.
This method, whose aim is to develop (or NOT!) (In fact, it should be possible, using the method, to cancel, modify, or redirect the solution being developed if it does not meet the requirements, with full transparency.) A solution (eventually, a new exoskeleton type) must be based on the needs, expectations, and possibilities that each actor can offer to ultimately achieve a very specific (and commonly understood) goal. This method should be aimed primarily at developers, as it is they who generate development projects. This could involve a pedagogical approach with a view to broadening the perspective to include all the needs and resources required for the appropriate development of an exoskeleton.
In addition, the methodologies employed in the context of exoskeleton assessment should be re-evaluated and refined. Given that exoskeletons are not conventional devices, they must be evaluated with a comprehensive understanding of all their aspects, facilitators, barriers, primary goals, and the various stages of user interaction. The objective of exoskeleton development should be to focus on the outcomes of the device’s interaction with the user, the tasks to be accomplished, and the context of use.
Another significant consideration is the way this novel method should be tested and validated. Test and validation criteria must be handled with the utmost care, as they can lead to erroneous expectations or results. Therefore, the method should align with the development goals rather than the other way around.

Funding

This research was partially funded by Portuguese national funds through Foundation for Science and Technology (FCT), I.P., within the scope of the project ‘UIDB/04647/2020’ of Interdisciplinary Centre of Social Sciences of Nova University of Lisbon–CICS.NOVA. The APC was funded by KIT.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Borislav, M.; Tri, D. Exoskeleton Catalog. Available online: https://exoskeletonreport.com/product-category/exoskeleton-catalog/?filter_exoskeleton-category=industrial (accessed on 26 May 2024).
  2. Peters, M.; Wischniewski, S. The Impact of Using Exoskeletons on Occupational Safety and Health. Dortmund, 2019. Available online: https://osha.europa.eu/sites/default/files/2021-11/Exoskeletons%26OSH.pdf (accessed on 26 May 2024).
  3. Yagn, N. Apparatus for Facilitating Walking, Running, and Jumping. U.S. Patent 440684, 18 November 1890. [Google Scholar]
  4. Makinson, B.J. Research and Development Prototype for Machine Augmentation of Human Strength and Endurance: Handiman I Project of General Electric Company Specialty Materials Handling Products Operation; National Technical Information Service: Springfield, VA, USA, 1971. [Google Scholar]
  5. Zoss, A.; Kazerooni, H.; Chu, A. On the mechanical design of the Berkeley Lower Extremity Exoskeleton (BLEEX). In Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, Edmonton, AB, Canada, 2–6 August 2005; pp. 3465–3472. [Google Scholar] [CrossRef]
  6. Hoffman, R.R.; Ford, K.M.; Feltovich, A.; Woods, D.D.; Feltovich, P.J.; Klein, G. A Rose by Any Other Name…Would Probably Be Given an Acronym. IEEE Intell. Syst. 2002, 17, 72–80. [Google Scholar] [CrossRef]
  7. Müller, R.; Vette, M.; Mailahn, O. Process-oriented Task Assignment for Assembly Processes with Human-robot Interaction. Procedia CIRP 2016, 44, 210–215. [Google Scholar] [CrossRef]
  8. De Bock, S.; Ghillebert, J.; Govaerts, R.; Elprama, S.A.; Marusic, U.; Serrien, B.; Jacobs, A.; Geeroms, J.; Meeusen, R.; De Pauw, K. Passive Shoulder Exoskeletons: More Effective in the Lab Than in the Field? IEEE Trans. Neural Syst. Rehabil. Eng. 2021, 29, 173–183. [Google Scholar] [CrossRef]
  9. Bai, S.; Islam, M.R.; Power, V.; OŚullivan, L. User-centered development and performance assessment of a modular full-body exoskeleton (AXO-SUIT). Biomim. Intell. Robot. 2021, 2, 100032. [Google Scholar] [CrossRef]
  10. Kuhn, D.; Freyberg-Hanl, B. Exoskelett und Mensch. Orthopädie Technik, no. Exoskelette, pp. 43–44, 2019. Available online: https://360-ot.de/exoskelett-und-mensch-ein-resuemee-zum-einsatz-von-exoskelett-systemen-in-deutschland/ (accessed on 26 May 2024).
  11. El-Haouzi, H.B.; Valette, E.; Krings, B.J.; Moniz, A.B. Social dimensions in cps & iot based automated production systems. Societies 2021, 11, 98. [Google Scholar] [CrossRef]
  12. DGUV Fachbereich Handel und Logistik. Fragen und Antworten zum Thema Exoskelette. Available online: https://www.dguv.de/fbhl/sachgebiete/physische-belastungen/faq_exo/index.jsp (accessed on 26 May 2024).
  13. Gilotta, S.; Spada, S.; Ghibaudo, L.; Isoardi, M.; Mosso, C.O. Acceptability beyond usability: A manufacturing case study. In Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018), Florence, Italy, 26–30 August 2018; Volume 824. [Google Scholar] [CrossRef]
  14. Spada, S.; Ghibaudo, L.; Gilotta, S.; Gastaldi, L.; Cavatorta, M.P. Investigation into the Applicability of a Passive Upper-limb Exoskeleton in Automotive Industry. Procedia Manuf. 2017, 11, 1255–1262. [Google Scholar] [CrossRef]
  15. Tröster, M.; Kopp, V.; Holl, M.; Daub, U.; Maufroy, C.; Schneider, U.; Bauernhansl, T. Arbeitsgesundheit mit digitalen Exoskeletten/IPA Ergo-Solution with digital assessment for industrial exoskeletons based on real application data. Wt Werkstattstech. Online 2022, 112, 137–141. [Google Scholar] [CrossRef]
  16. Hensel, R.; Keil, M. Subjective Evaluation of a Passive Industrial Exoskeleton for Lower-back Support: A Field Study in the Automotive Sector. IISE Trans. Occup. Ergon. Hum. Factors 2019, 7, 213–221. [Google Scholar] [CrossRef]
  17. Hensel, R.; Steinhilder, B. Bewertung von Exoskeletten für industrielle Arbeitsplätze. In Dritte Transdisziplinäre Konferenz-Technische Unterstützungssysteme, die die Menschen Wirklich Wollen; Weidner, R., Karafillidis, A., Eds.; Helmut-Schmidt-Universität: Hamburg, Germany, 2018. [Google Scholar]
  18. Schmalz, T.; Bornmann, J.; Schirrmeister, B.; Schändlinger, J.; Schuler, M. Prinzipstudie zur Wirkung eines Industriellen Exoskeletts bei Überkopfarbeit; Orthopädie Technik: Dortmund, Germany, 2019. [Google Scholar]
  19. Sposito, M.; Toxiri, S.; Caldwell, D.G.; Ortiz, J.; De Momi, E. Towards design guidelines for physical interfaces on industrial exoskeletons: Overview on evaluation metrics. In Wearable Robotics: Challenges and Trends, Proceedings of the 4th International Symposium on Wearable Robotics, WeRob2018, Pisa, Italy, 16–20 October 2018; Springer: Cham, Switzerland, 2019; Volume 22. [Google Scholar] [CrossRef]
  20. Harbauer, C.M.; Fleischer, M.; Nguyen, T.; Kopfinger, S.; Bos, F.; Bengler, K. Too Close to Comfort? A New Approach of Designing a Soft Cable-Driven Exoskeleton for Lifting Tasks under Ergonomic Aspects. In International Journal of Mechanical Engineering and Robotics Research; IEEE: Piscataway, NJ, USA, 2020; pp. 99–106. [Google Scholar] [CrossRef]
  21. Chen, T.H.; Lin, C.L.; Wang, M.J.J. The evaluation of double-layers clothing on clothing microclimate, physiological responses and subjective comfort. In Advances in Human Factors, Ergonomics, and Safety in Manufacturing and Service Industries; CRC Press: Boca Raton, FL, USA, 2010; pp. 741–749. [Google Scholar] [CrossRef]
  22. Ferreira, G.; Gaspar, J.; Fujão, C.; Nunes, I.L. Piloting the Use of an Upper Limb Passive Exoskeleton in Automotive Industry: Assessing User Acceptance and Intention of Use. In Advances in Human Factors and Systems Interaction; AISC; Springer: Cham, Switzerland, 2020; Volume 1207. [Google Scholar] [CrossRef]
  23. Krings, B.; Weinberger, N. Kann es technische Assistenten in der Pflege geben? Überlegungen zum Begriff der Assistenz in Pflegekontexten. In Assistive Gesellschaft; Springer Fachmedien Wiesbaden: Wiesbaden, Germany, 2017; pp. 183–201. [Google Scholar] [CrossRef]
  24. Savolainen, K.; Hyysalo, S. User Representations as a Design Resource. Sci. Technol. Stud. 2021, 34, 25–45. [Google Scholar] [CrossRef]
  25. Gan, W.Y.; Ghazilla, R.A.R.; Yap, H.J.; Selvarajoo, S. Industrial practitioner’s perception on the application of exoskeleton system in automotive assembly industries: A Malaysian case study. Heliyon 2024, 10, e26183. [Google Scholar] [CrossRef]
  26. Larousse. 2024. Available online: https://www.larousse.fr/dictionnaires/francais/accepter/415 (accessed on 26 May 2024).
  27. Dictionnaire de l’Académie Française, “Accepter”. Available online: https://www.dictionnaire-academie.fr/article/A9A0233 (accessed on 26 May 2024).
  28. Rogers, E.M. DIFFUSION of INNOVATIONS 3RD E REV. In Diffusion of Innovations; Free Press: New York, NY, USA, 1983; Available online: https://books.google.de/books?id=pXRkAAAAIAAJ (accessed on 26 May 2024).
  29. Turner, M.; Kitchenham, B.; Brereton, P.; Charters, S.; Budgen, D. Does the technology acceptance model predict actual use? A systematic literature review. Inf. Softw. Technol. 2010, 52, 463–479. [Google Scholar] [CrossRef]
  30. Börstler, J.; bin Ali, N.; Svensson, M.; Petersen, K. Investigating acceptance behavior in software engineering—Theoretical perspectives. J. Syst. Softw. 2023, 198, 111592. [Google Scholar] [CrossRef]
  31. Börstler, J.; bin Ali, N.; Petersen, K.; Engström, E. Acceptance behavior theories and models in software engineering—A mapping study. Inf. Softw. Technol. 2024, 172, 107469. [Google Scholar] [CrossRef]
  32. Lampo, A. The Role of Habit in UTAUT-2 Research: A Study of BEV Users. In Proceedings of the 2023 9th International Conference on Industrial and Business Engineering, Beijing, China, 22–24 September 2023; pp. 20–25. [Google Scholar] [CrossRef]
  33. Venkatesh, V.; Thong, J.Y.L.; Xu, X. Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology. MIS Q. 2012, 36, 157–178. [Google Scholar] [CrossRef]
  34. Ain, N.U.; Kaur, K.; Waheed, M. The influence of learning value on learning management system use: An extension of UTAUT2. Inf. Dev. 2016, 32, 1306–1321. [Google Scholar] [CrossRef]
  35. Bornmann, J.; Schirrmeister, B.; Parth, T.; Gonzalez-Vargas, J. Comprehensive development, implementation and evaluation of industrial exoskeletons. Curr. Dir. Biomed. Eng. 2020, 6, 20202001. [Google Scholar] [CrossRef]
  36. Drees, T.; Kunnen, S.; Pluhnau, R.; Nagarajah, A. Methodology for a task-specific and personalised development of an initial exoskeleton design. Proc. Des. Soc. 2021, 1, 2077–2086. [Google Scholar] [CrossRef]
  37. Elprama, S.A.; Vannieuwenhuyze, J.T.A.; De Bock, S.; Vanderborght, B.; De Pauw, K.; Meeusen, R.; Jacobs, A. Social Processes: What Determines Industrial Workers’ Intention to Use Exoskeletons? Hum. Factors 2020, 62, 337–350. [Google Scholar] [CrossRef]
  38. Hoffmann, H.; Pitz, I.; Adomssent, B.; Russmann, C. Association, expectations and barriers of the use of exoskeletons in small and medium-sized enterprises. Zentralbl Arbeitsmed Arbeitsschutz Ergon. 2022, 72, 68–77. [Google Scholar] [CrossRef] [PubMed]
  39. Wioland, L.; Kouadio, J.J.A.; Bréard, H.; Clerc-Urmès, I.; Paty, B. The Adoption of Occupational Exoskeletons: From Acceptability to Situated Acceptance, Questionnaire Surveys. Int. J. Hum. Comput. Interact. 2024, 1–13. [Google Scholar] [CrossRef]
  40. Hussain, M.; Kong, Y.K.; Park, S.S.; Shim, H.H.; Park, J. Exoskeleton Usability Questionnaire: A preliminary evaluation questionnaire for the lower limb industrial exoskeletons. Ergonomics 2023, 1–10. [Google Scholar] [CrossRef]
  41. Gonsalves, N.; Akanmu, A.; Shojaei, A.; Agee, P. Factors influencing the adoption of passive exoskeletons in the construction industry: Industry perspectives. Int. J. Ind. Ergon. 2024, 100, 103549. [Google Scholar] [CrossRef]
  42. Elprama, S.A.; Vanderborght, B.; Jacobs, A. An industrial exoskeleton user acceptance framework based on a literature review of empirical studies. Appl. Ergon. 2022, 100, 103615. [Google Scholar] [CrossRef] [PubMed]
  43. Endter, C. Assistiert altern. Die Entwicklung eines Sturzsensors im Kontext von Ambient Assisted Living. In Assistive Gesellschaft; Springer: Wiesbaden, Germany, 2017. [Google Scholar] [CrossRef]
  44. Karafillidis, A.; Weidner, R. Taxonomische Kriterien technischer Unterstützung. Auf dem Weg zu einem Periodensystem Taxonomische Kriterien technischer Unterstützung Auf dem Weg zu einem Periodensystem. In Technische Unterstützungssysteme, die die Menschen Wirklich Wollentzungssysteme; no. December; Weidner, R., Ed.; Helmut-Schmidt-Universität Hamburg: Hamburg, Germany, 2016; pp. 233–247. [Google Scholar]
  45. Gransche, B. Wir assistieren uns zu Tode. In Assistive Gesellschaft; Springer Fachmedien Wiesbaden: Wiesbaden, Germany, 2017; pp. 77–97. [Google Scholar] [CrossRef]
  46. Law, E.L.-C.; Vermeeren, A.P.O.S.; Hassenzahl, M.; Blythe, M. Towards a UX Manifesto. In Proceedings of the HCI 2007 the 21st British HCI Group Annual Conference University of Lancaster, UK (HCI), Lancaster, UK, 3–7 September 2007; pp. 10–14. [Google Scholar] [CrossRef]
  47. Hassenzahl, M. The Thing and I: Understanding the Relationship between User and Product. In Funology; vol. 3, no. January 2005; Blythe, M.A., Monk, A.F., Overbeeke, K., Wright, P.C., Eds.; Kluwer Academic: Dordrecht, The Netherlands, 2003; Chapter 3; pp. 31–42. [Google Scholar] [CrossRef]
  48. Elprama, S.A.; De Bock, S.; Meeusen, R.; De Pauw, K.; Vanderborght, B.; Jacobs, A. Design and Implementation Requirements for Increased Acceptance of Occupational Exoskeletons in an Industrial Context: A Qualitative Study. Int. J. Hum. Comput. Interact. 2023, 1–16. [Google Scholar] [CrossRef]
  49. Mahlke, S. Visual aesthetics and the user experience. Dagstuhl Semin. Proc. 2008, 8292, 1–5. [Google Scholar]
  50. Tractinsky, N.; Katz, A.S.; Ikar, D. What is beautiful is usable. Interact. Comput. 2000, 13, 127–145. [Google Scholar] [CrossRef]
  51. Fischer, B.; Peine, A.; Östlund, B.; Heyn, P.C. The Importance of User Involvement: A Systematic Review of Involving Older Users in Technology Design. Gerontologist 2020, 60, E513–E523. [Google Scholar] [CrossRef] [PubMed]
  52. Ármannsdóttir, A.L.; Beckerle, P.; Moreno, J.C.; van Asseldonk, E.H.F.; Manrique-Sancho, M.-T.; Del-Ama, A.J.; Veneman, J.F.; Briem, K. Assessing the Involvement of Users During Development of Lower Limb Wearable Robotic Exoskeletons: A Survey Study. Hum. Factors 2020, 62, 351–364. [Google Scholar] [CrossRef] [PubMed]
  53. Lowdermilk, T. User-Centered Design: A Developer’s Guide to Building User-Friendly Applications; O’Reilly, Ed., O’Reilly Media, Inc.: Sebastopol, CA, USA, 2013; p. 152. [Google Scholar]
  54. Nielsen, J. Don’t Listen to the Customers, (2020). [Online Video]. Available online: https://www.youtube.com/watch?v=OARVjXqBI2A&ab_channel=NNgroup (accessed on 26 May 2024).
  55. Herrmanny, K.; Analytics, P. Wissen die Menschen, was sie wirklich wollen? In Technische Unterst{ü}tzungssysteme, die die Menschen Wirklich Wollen; Weidner, R., Ed.; Helmut-Schmidt-Universität Hamburg: Hamburg, Germany, 2016; pp. 193–201. [Google Scholar]
  56. Rudlof, C. Neue Curricula braucht die Hochschule–Ingenieur * innen zur Arbeitsgestaltung befähigen. In Dritte Transdisziplinäre Konferenz-Technische Unterstützungssysteme, die die Menschen Wirklich Wollen; Weidner, R., Karafillidis, A., Eds.; Helmut-Schmidt-Universität: Hamburg, Germany, 2018; pp. 19–24. [Google Scholar]
  57. Hoffmann, N.; Ralfs, L.; Weidner, R. Leitmerkmale und Vorgehen einer Implementierung von Exoskeletten. Z. Für Wirtsch. Fabr. 2021, 116, 525–528. [Google Scholar] [CrossRef]
  58. Guston, D.H.; Sarewitz, D. Real-time technology assessment. Technol. Soc. 2002, 24, 93–109. [Google Scholar] [CrossRef]
  59. Kim, S.; Moore, A.; Ojelade, A.; Gutierrez, N.; Harris-Adamson, C.; Barr, A.; Srinivasan, D.; Nussbaum, M.A. A data-driven approach to understand factors contributing to exoskeleton use-intention in construction. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 2023, 67, 801–802. [Google Scholar] [CrossRef]
  60. Ciullo, A.S.; Catalano, M.G.; Bicchi, A.; Ajoudani, A. A Supernumerary Soft Robotic Limb for Reducing Hand-Arm Vibration Syndromes Risks. Front. Robot. AI 2021, 8, 650613. [Google Scholar] [CrossRef] [PubMed]
  61. Daub, U.; Bölke, N.; Kopp, V.; Brück, M.; Schneider, U. Effective physical stress reduction. WT Werkstattstech. 2021, 111, 617–621. [Google Scholar] [CrossRef]
  62. Sanchez-Villamañan, M.D.C.; Gonzalez-Vargas, J.; Torricelli, D.; Moreno, J.C.; Pons, J.L. Compliant lower limb exoskeletons: A comprehensive review on mechanical design principles. J. Neuroeng. Rehabil. 2019, 16, 55. [Google Scholar] [CrossRef]
  63. Luger, T.; Cobb, T.J.; Seibt, R.; Rieger, M.A.; Steinhilber, B. Subjective Evaluation of a Passive Lower-Limb Industrial Exoskeleton Used During Simulated Assembly. IISE Trans. Occup. Ergon. Hum. Factors 2019, 7, 175–184. [Google Scholar] [CrossRef]
  64. Wolff, J.; Parker, C.; Borisoff, J.; Mortenson, W.B.; Mattie, J. A survey of stakeholder perspectives on exoskeleton technology. J. Neuroeng. Rehabil. 2014, 11, 169. [Google Scholar] [CrossRef] [PubMed]
  65. Andrade, C.; Nathan-Roberts, D. Occupational Exoskeleton Adoption and Acceptance in Construction and Industrial Work: A Scoping Review. Proc. Human. Factors Ergon. Soc. Annu. Meet. 2022, 66, 1325–1329. [Google Scholar] [CrossRef]
  66. What Is Ergonomics? Association International Ergonomics. Available online: https://iea.cc/what-is-ergonomics/ (accessed on 26 May 2024).
  67. DIN e.V. DIN EN ISO 9241-210:2010; Ergonomie der Mensch-System-Interaktion–Teil 210: Prozess zur Gestaltung Gebrauchstauglicher Interaktiver Systeme. ISO: Geneva, Switzerland, 2011.
  68. Peres, S.C.; Pham, T.; Phillips, R. Validation of the system usability scale (sus): Sus in the wild. Proc. Human. Factors Ergon. Soc. 2013, 57, 192–196. [Google Scholar] [CrossRef]
  69. Hassenzahl, M.; Platz, A.; Burmester, M.; Lehner, K. Hedonic and ergonomic quality aspects determine a software’s appeal. In Proceedings of the Conference on Human. Factors in Computing Systems–Proceedings, The Hague, The Netherlands, 1–6 April 2000; pp. 201–208. [Google Scholar] [CrossRef]
  70. Kopp, V.; Holl, M.; Schalk, M.; Daub, U.; Bances, E.; García, B.; Schalk, I.; Siegert, J.; Schneider, U. Exoworkathlon: A prospective study approach for the evaluation of industrial exoskeletons. Wearable Technol. 2022, 3, e22. [Google Scholar] [CrossRef]
  71. del Rio Carral, M.; Bourqui, V.; Vuilleumier, N.; Ortieb, A.; Bouri, M. Are Functional Measures Sufficient to Capture Acceptance? A Qualitative Study on Lower Limb Exoskeleton Use for Older People. Int. J. Soc. Robot. 2021, 14, 603–616. [Google Scholar] [CrossRef]
  72. Li-Baboud, Y.S.; Virts, A.; Bostelman, R.; Yoon, S.; Rahman, A.; Rhode, L.; Ahmed, N.; Shah, M. Evaluation Methods and Measurement Challenges for Industrial Exoskeletons. Sensors 2023, 23, 5604. [Google Scholar] [CrossRef] [PubMed]
  73. Diefenbach, S.; Hassenzahl, M. Psychologie in der Nutzerzentrierten Produktgestaltung; Springer: Berlin/Heidelberg, Germany, 2017; Volume 63. [Google Scholar] [CrossRef]
  74. Krings, B.-J.; Moniz, A.B.; Frey, P. Technology as enabler of the automation of work? Current societal challenges for a future perspective of work/A tecnologia como facilitadora da automação do trabalho? Desafios sociais atuais para uma visão do futuro do trabalho. Rev. Bras. De Sociol.–RBS 2021, 9, 206–229. [Google Scholar] [CrossRef]
  75. Siedl, S.M.; Mara, M. What Drives Acceptance of Occupational Exoskeletons? Focus Group Insights from Workers in Food Retail and Corporate Logistics. Int. J. Hum. Comput. Interact. 2023, 39, 4080–4089. [Google Scholar] [CrossRef]
  76. Theurel, J. Exosquelettes au travail: Intérêts et limites pour la prévention des TMS? In Hygiène et Sécurité du Travail; INRS: Quebec,, QC, Canada, 2020; Volume 259. [Google Scholar]
  77. Davis, F.D.; Bagozzi, R.P.; Warshaw, P.R. User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Manag. Sci. 1989, 35, 982–1003. [Google Scholar] [CrossRef]
  78. Renaud, K.; Van Biljon, J. Predicting technology acceptance and adoption by the elderly: A qualitative study. ACM Int. Conf. Proceeding Ser. 2008, 338, 210–219. [Google Scholar] [CrossRef]
  79. Shore, L.; de Eyto, A.; O’Sullivan, L. Technology acceptance and perceptions of robotic assistive devices by older adults–implications for exoskeleton design. Disabil. Rehabil. Assist. Technol. 2020, 17, 782–790. [Google Scholar] [CrossRef]
  80. Heerink, M.; Kröse, B.; Evers, V.; Wielinga, B. Assessing acceptance of assistive social agent technology by older adults: The almere model. Int. J. Soc. Robot. 2010, 2, 361–375. [Google Scholar] [CrossRef]
  81. Baltrusch, S.J.; Houdijk, H.; van Dieën, J.H.; Kruif, J.T.C.M. Passive Trunk Exoskeleton Acceptability and Effects on Self-efficacy in Employees with Low-Back Pain: A Mixed Method Approach. J. Occup. Rehabil. 2021, 31, 129–141. [Google Scholar] [CrossRef] [PubMed]
  82. Norman, D. Emotion & design. Interactions 2002, 9, 36–42. [Google Scholar] [CrossRef]
  83. Carroll, J.; Howard, S.; Peck, J.; Murphy, J. From Adoption to Use: The process of appropriating a mobile phone. Australas. J. Inf. Syst. 2003, 10, 38–48. [Google Scholar] [CrossRef]
  84. Ármannsdóttir, A.; Manrique-Sancho, M.T.; Moreno, J.C.; del-Alma, A.J.; Beckerle, P.; van Asseldonk, E.H.F.; Veneman, J.F. User Involvement, Device Safety, and Outcome Measures During Development of Walking Exoskeletons: Current Practices. In Inclusive Robotics for a Better Society: Selected Papers from INBOTS Conference 2018, 16–18 October, Pisa, Italy 2018; Springer International Publishing: Cham, Switzerland, 2020; Volume 25. [Google Scholar] [CrossRef]
  85. Crea, S.; Beckerle, P.; De Looze, M.; De Pauw, K.; Grazi, L.; Kermavnar, T.; Masood, J.; O’sullivan, L.W.; Pacifico, I.; Rodriguez-Guerrero, C.; et al. Occupational exoskeletons: A roadmap toward large-scale adoption. Methodology and challenges of bringing exoskeletons to workplaces. Wearable Technol. 2021, 2, e11. [Google Scholar] [CrossRef] [PubMed]
  86. Koch, W.; Keisinger, F. How can responsible AI be implemented? In Responsible Use of AI in Military Systems; CRC Press: Boca Raton, FL, USA, 2024; pp. 37–58. [Google Scholar] [CrossRef]
  87. Kirpestein, F.; Bax, L.; Chadima, D.; Van Breevoort, N.; Dobbs, H. EXOSKELETONS Future Exoskeleton Technology Application by 2030. 2023. Available online: https://northsearegion.eu/media/24407/exskallerate_foresight-1.pdf (accessed on 26 May 2024).
  88. Gneiting, E.; Schiebl, J.; Tröster, M.; Kopp, V.; Maufroy, C.; Schneider, U. Model-Based Biomechanics for Conceptual Exoskeleton Support Estimation Applied for a Lifting Task. In Biosystems and Biorobotics; Springer International Publishing: Berlin/Heidelberg, Germany, 2022; Volume 27, pp. 395–399. [Google Scholar] [CrossRef]
  89. Parodi, O.; Steglich, A.; Bylund, J. Real-world lab. In Handbook Transdisciplinary Learning; Transcript Verlag: Bielefeld, Germany, 2023; pp. 287–296. [Google Scholar] [CrossRef]
  90. Von Wilamowitz, M.; Hassenzahl, M. Dynamics of User Experience: How the Perceived Quality of Mobile Phones Changes over Time, no. JANUARY 2006. 2006. Available online: https://www.researchgate.net/publication/259823112_Dynamics_of_user_experience_How_the_perceived_quality_of_mobile_phones_changes_over_time_In_User_experience-towards_a_unified_view (accessed on 26 May 2024).
  91. Kunz, A.; Pohlmann, S.; Heinze, O.; Brandner, A.; Reiß, C.; Kamradt, M.; Szecsenyi, J.; Ose, D. Strengthening interprofessional requirements engineering through action sheets: A pilot study. JMIR Hum. Factors 2016, 3, e5364. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Human and technical systems (inspired by [2,3]).
Figure 1. Human and technical systems (inspired by [2,3]).
Applsci 14 07108 g001
Figure 2. A total of 104 publications were selected from ScienceDirect. Subject areas: Engineering (accessed on 22 May 2024).
Figure 2. A total of 104 publications were selected from ScienceDirect. Subject areas: Engineering (accessed on 22 May 2024).
Applsci 14 07108 g002
Figure 3. Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) Venkatesh et al. [33].
Figure 3. Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) Venkatesh et al. [33].
Applsci 14 07108 g003
Figure 4. Framework of exoskeleton acceptance (original picture [42]).
Figure 4. Framework of exoskeleton acceptance (original picture [42]).
Applsci 14 07108 g004
Figure 5. Senior technology acceptance and adoption model, STAM.
Figure 5. Senior technology acceptance and adoption model, STAM.
Applsci 14 07108 g005
Figure 6. Internal balance of use of technology.
Figure 6. Internal balance of use of technology.
Applsci 14 07108 g006
Figure 7. Two stages of TAM.
Figure 7. Two stages of TAM.
Applsci 14 07108 g007
Figure 8. Links between different points of view.
Figure 8. Links between different points of view.
Applsci 14 07108 g008
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lefint, J.; Moniz, A.B. Assessment of Exoskeletons for Work Activities: The Dilemma behind the Product. Appl. Sci. 2024, 14, 7108. https://doi.org/10.3390/app14167108

AMA Style

Lefint J, Moniz AB. Assessment of Exoskeletons for Work Activities: The Dilemma behind the Product. Applied Sciences. 2024; 14(16):7108. https://doi.org/10.3390/app14167108

Chicago/Turabian Style

Lefint, Jérémy, and António B. Moniz. 2024. "Assessment of Exoskeletons for Work Activities: The Dilemma behind the Product" Applied Sciences 14, no. 16: 7108. https://doi.org/10.3390/app14167108

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop