In the face of the high number of accidents at work in the EU-28 [
1], companies adopted several actions to improve safety at work, such as risk assessment, targeted campaigns, and training programs for risk prevention and safety promotion, especially currently, where many workplaces are high-risk environments for COVID-19 [
2]. The interventions on human factors (henceforth, HF) assume critical value in the safety field. Indeed, statistics show that human error is one of the primary causes of failure and accidents in a variety of work contexts, such as the road and transportation [
3], chemical and petrochemical [
4], marine [
5], aviation [
6], construction [
7] sectors, and engineering and security [
8]. In this regard, training is a tool that can improve the impact of HF and occupational health, and can be particularly useful in times such as the COVID-19 situation [
9].
A meta-analytic study by Burke and colleagues [
11] found that not all training methods have the same levels of effectiveness. It shows, for instance, that as the methods become more engaging, based on learners’ participation in the training process, the safety courses demonstrate greater effectiveness. Therefore, methods based on passive learning, such as lectures, achieve lower levels of effectiveness than engaging methods with active learning, such as behavioural simulations, where interactions between trainees and trainers are frequent and based on reciprocal feedback [
11]. This type of study draws on the principle that an active method improves the trainee engagement, without considering the actual engagement that can be developed. It is important to integrate the perspective of Burke and colleagues [
8], focusing on the perceptions, impressions, sensations, and involvement of the trainees who attend a safety course.
Therefore, the present study intends to deepen the exploration of the concept of engaging training methodology from the trainees’ perspective. It proposes a measure that analyses participants’ level of engagement and involvement in a safety course. Based on previous measures of engagement, the scale was developed to help trainers and researchers evaluate the levels of participant engagement in training, specifically in the safety field, in a practical, valid, and reliable way.
This study lies in a scientific field that sees the presence of various instruments for training evaluation (e.g., [
12,
13,
14,
15,
16]), yet not specifically focused on the engagement and/or validated in the safety training program. Indeed, although many studies emphasize the critical value of engagement in safety training [
11,
17], and there are scales on trainee reactions (e.g., [
12,
14]) and various aspects of training from a broader perspective, no measurement tools have been developed and validated for assessing the trainees’ engagement in this field, meaning that a short, valid, and reliable measure designed to assess participants’ engagement in safety training is necessary.
The present study attempts to fill these gaps, and aims to contribute to the literature with a valid, reliable, and practical measurement tool, specifically tested in safety training. Our scale is supposed to add to the training evaluation scholarly field. It is meant to be complementary and integrated, because it provides a training engagement analysis with a brief measure of a few items. It also intends to be specific to the safety context because it is developed and tested in this field.
1.1. Theoretical Background
The importance of engagement in safety training comes to light in an essential review by Burke and colleagues [
11]. The researchers analyzed the relative effectiveness of different training methods considering three outcomes: (a) safety knowledge, (b) safety performance, and (c) safety and health outcomes (e.g., accidents, illnesses, or injuries). The researchers distinguished three levels of engagement in safety training: (1) least engaging training, which relies on methods such as lectures, videos, pamphlets, and written materials; (2) moderately engaging training, which adopts techniques such as feedback interventions, performance information provided in small groups, and computer-based interactive instruction; and (3) most engaging training methods, where there are hands-on demonstrations associated with behavioural stimulation and active participation of the trainee. The findings underline that the more engaging the training is, the greater the effects on knowledge acquisition are. Furthermore, training will have a more significant impact on reducing adverse safety and health outcomes if the course adopts highly engaging methods. Concerning safety performance [
18], the results outlined by the review are mixed, but suggest the effectiveness of more engaging training methods. Overall, Burke and colleagues [
11] argued that the most engaging methods in safety training are approximately three times more effective than the least engaging in promoting knowledge and skill acquisition. Robson and colleagues [
7] provided a second review where the level of trainees’ engagement categorized training interventions. The review shows mixed findings: on the one hand, if we consider attitudes and health as outcomes, more engaging training methods are more effective than low-engagement ones. On the other hand, when the authors analyzed outcomes, the findings were not in line with the Burke and colleagues’ review [
5]. Indeed, the effectiveness of interventions is strong regarding the effect on behaviours, but for health outcomes, the results are not so consistent; however, it should be noted that the latest results are derived from three studies only, two of which consist of very brief interventions. The study by Namian and colleagues [
19], conducted in the construction industry and based on interviews and questionnaires with experts and the analysis of empirical data gathered from 51 case projects, presents a picture of findings where high-engagement training methods maximize safety training outcomes.
Having underlined the value of adopting engaging methods in safety training, we now present an analysis of the concept of engaging methods. Burke and colleagues [
20] describe a method as more engaging when it incorporates elements of action, dialogue, and reflection, and encourages trainees to infer relations among concepts, events, and actions to develop new ways of thought and action. On a different level, by adopting a subjective perspective, the concept of engagement is broader and extensively studied in organizational psychology. Its relevance rises in the positive psychology that studies human resource strengths, resources, and psychological capacities that can be measured, managed, and effectively developed in the workplace [
21]. In this regard, ‘work engagement’ consists of a sense of energetic and effective connection with employees’ work activities, and reflects the ability to deal well with the job demands. The literature [
22] defines it as a positive, fulfilling, work-related state of mind experienced by employees, and characterized by vigor, dedication, and absorption. Work engagement is widely studied, and it is considered a result of an energetic process, as opposed to an energy-consuming process leading to burnout [
22]. Work engagement is presented as similar to having “flow” (i.e., experiencing a sense of total harmony), but it is more stable than flow, which tends to be a peak experience [
23]. Similarly, in learning contexts, situational interest and affective engagement can play important roles, acting as positive motivational states of learners [
24,
25]. A recent study found that behavioural engagement (defined as effort and perseverance in learning) and emotional engagement (defined as a sense of belonging) significantly predicted academic performance [
26].
Recently, Casey and colleagues [
27] proposed a specific framework for the safety field that considers training engagement as a combination of pre-training factors (e.g., individual, organizational, and contextual factors) and the training program factors (e.g., learning environment, trainer characteristics, as a trust). Based on the educational psychology literature [
13], the authors conceptualized safety training engagement using three-component psychological states (affective, cognitive, and behavioural), which drive the motivation to learn and other training approach behaviours. Affective or emotional engagement refers to a positive mental state concerning the learning task. Cognitive engagement refers to the mental effort invested in the training to think about and attend to the materials, and behavioural engagement as actively participating in the training program.
Eventually, for the definition of the safety training engagement, we followed the definitions provided by Ben-Eliyahu and colleagues [
13], and Hallberg and Schaufeli [
19]. We consider safety training engagement in terms of dedication and absorption. Dedication reflects how much one is involved in the safety training program, experiencing a sense of significance and enthusiasm (affective state). Absorption pertains to how much the trainee is entirely concentrating on and interested in the safety program (cognitive state), where time passes quickly, and without detaching themselves from the course [
28]. So we focus on a mix of affective and cognitive elements [
16], analyzed in a synergic way, during a safety training program. This is an additional element that distinguishes the safety training engagement scale (hereafter, STE-S) from the other measures in the literature.
1.2. Aims and Hypotheses of the Research
Despite the importance of adopting engaging methods in safety training [
11], there are no validated scales in a safety training program that measure trainee engagement with a theoretical perspective that has been described in the previous paragraph, to the best of our knowledge. However, there are many examples in the literature of scales evaluating subjective reactions towards training, which justifies the development of a safety training engagement scale to complement these tools. Morgan and Casper [
12] built scales on general trainee reactions in different courses. Ritzmann and colleagues [
13] developed the training evaluation inventory, which measures subjective enjoyment, usefulness, difficulty, knowledge gain, and attitude toward training. Grohmann and Kauffeld [
14] built a questionnaire for professional training evaluation (Q4TE), which measures short- and long-term training outcomes. Moreover, O’Brien and Toms [
15] developed a scale on engagement that focuses only on the peoples’ experiences with technology, from a perspective of the design of interactive systems.
Lastly, the literature presents a specific scale of engagement in training by Ben-Eliyahu and colleagues [
16], which provides an analytic and articulate measurement of the construct with satisfactory psychometric proprieties. Yet, it presents some limitations that may affect its application in practical contexts. Indeed, it has not been developed in the safety context, as it was tested among high school students. Moreover, due to its length (it consists of 17 items), it may not be suitable for application in situations where agility is required, such as at the end of a training session. A more agile and less demanding tool is recommended for gathering more reliable data.
Drawing on these bases, our research’s first aim is to develop a short instrument to assess trainees’ engagement in terms of absorption and dedication to the safety training. The study focuses on creating a brief and practical scale (STE-S) that measures this construct in a reliable, valid, and efficient manner, and can be adopted by both researchers and trainers of safety programs. The second aim is the validation of STE-S. Many researchers underlined the relevance of developing valid and usable training evaluation tools [
29]. For instance, following this research line, scholars validated some questionnaires: Morgan and Casper [
12] studied the factor validity of scales on trainee reactions in different courses. Ritzmann and colleagues [
13] analyzed the internal validity of the measures. Grohmann and Kauffeld [
14] studied the stability of the questionnaire for professional training evaluation (Q4TE) factor structure.
We wanted to study the psychometric properties by testing the construct in terms of internal and external validity, according to the American Psychological Association [
30]. Some essential aspects of internal validity were analyzed, such as dimensionality and reliability, while external validity was tested through criterion-related validity. Given that internal validity focuses primarily on the internal relationships of the test, the first step (Study 1) was to analyze whether the theoretical uni-dimensionality of the scale is supported.
Therefore, our first research questions address the factorial structure and reliability:
Research question 1: Do STE-S items construct a uni-dimensional scale?
Research question 2: Does STE-S present fair internal consistency?
Furthermore, we aimed to evaluate external validity with a second study. Specifically, the criterion-related validity was tested to assess the degree to which scores from our test correlated expectedly with a network of previously validated measures [
30]. Herein, three variables were tested as connected with the STE construct: work engagement, the type of training methods and techniques (as antecedents), and the perceived quality and usefulness of safety training in terms of safety performance (as an outcome). The conservation of resources theory [
31] suggests a relationship between work engagement and STE. COR theory asserts that people strive to gain, retain, and enhance valuable resources to protect themselves from psychological harm, and achieve the desired goals. As such, people must invest resources to preserve existing resources and activate resource gain spirals. Those who dispose of resources can invest them and better sustain the resource acquisition process. Work engagement is typically seen in the literature as the outcome of an investment of personal and job resources [
22,
32]. However, some evidence shows that it may also represent the initiator of a motivational investment process toward resource acquisition [
33,
34]. In other words, work engagement may predict STE positively. Those engaged may seek to reinforce their resource acquisition and consider safety training an investment towards better skills for safety performance, increasing their cognitive involvement with the training. A similar idea is expressed by the model of Casey and colleagues [
27], which lists work engagement among the pre-training factors that can influence training engagement.
Therefore, this study hypothesizes:
Hypothesis 1. Work engagement is positively associated with STE-S.
This study compares STE scores across trainees who attended training sessions with different engaging training methods and techniques. As outlined previously, Burke and colleagues [
8], in a meta-analytic study, distinguished between high-engagement training activities that stimulate the active participation (i.e., observation of a role model, feedback, hands-on demonstrations with the active participation of the trainees), and more passive and less engaging ones (i.e., passive trainees’ attending lectures to obtain health and safety-related information). Moreover, they found that the former type of training is more effective in knowledge acquisition and transfer than the latter. This suggests that a more participative and engaging training technique may influence trainees’ engagement. Indeed, some evidence exists [
35,
36] that situational factors (type of content, content delivery, training environment setting, trainers’ behaviors) affect trainees’ engagement in training. In relation to this, Casey and colleagues [
23] model lists the type of training method among the training factors that can influence training engagement.
A valuable contribution to external validity is establishing whether training characteristics impact STE scores. To ascertain the impact of different training methods, we tested whether there were differences in the STE in trainees attending less engaging safety training sessions (i.e., classical lectures) vs. more engaging safety training sessions (i.e., behavioral stimulations, active trainees’ participation). Therefore, the following research question is posited:
Hypothesis 2. Trainees attending a more engaging safety training session will report higher STE scores compared to trainees attending a less engaging safety training session.
The relationship between STE and perceived quality is suggested by previous research in the field of learning and training. Evidence exists that forms of involvement (e.g., cognitive, emotional) experienced by trainees are associated with some positive outcomes in terms of subjective reactions of the trainees towards training quality, such as perceived usefulness [
37,
38,
39]
Hypothesis 3. STE-S is positively associated with the perceived usefulness of safety training.
The two studies conducted are presented below to describe the development and construct validity evaluation (in line with the recommendations of Grimm and Widaman, [
30]) of the newly developed STE-S scale. The first study pertains to the development of the items for the scale. Moreover, this study seeks initial evidence of the scale construct (internal) validity in terms of the dimensional structure of the scale and internal consistency. The second study seeks to provide further evidence for construct validity by testing the external validity in terms of criterion-related validity.
Figure 1 displays a synthesis of the process for the STE scale development.