Methodology for Complex Efficiency Evaluation of Machinery Safety Measures in a Production Organization

Abstract

Even though the rules for the free circulation of machinery within the European Union (EU) market have existed for more than 30 years, accidents related to their activities have constantly been reaching significant value. When designing a machine, the design must stem from a risk assessment, where all stages of its life cycle and the ways to use it must be taken into consideration. In industrial operations with old machinery, despite fulfilling its function reliably, the safety level is below the developing requirements for safe operations. The proposed methodology to assess machinery safety conditions comes from the assumption of the proper application of risk assessment steps and their effectiveness in risk reduction mainly through implementing both effective and efficient preventive measures. The objective of the research applied in three operations was to verify the methods concerning machinery safety and its management. The created methodology, based on 19 requirements for safety, evaluates the level of current measures using a criterion of the current safety status and the total effectiveness of safety measures. Its output is the assessment of the efficiency level of the implemented safety measures of each machine as well as of the whole operation.

1. Introduction

Within the last ten years, the number of accidents related to activities connected with the use of machinery equipment has been decreasing, however, the reality is that machinery, including lifters, conveyors, and similar equipment, is, according to statistics [1,2], a source of as much as 25% of all serious occupational injuries annually.

According to an official report of the National Labor Inspectorate of the Slovak Republic-(NLI) [3], there was a total of 9022 occupational injuries registered in 2019. Out of the stated amount, 31 were fatal and 57 were serious with a heavy bodily injury. In its reports, the NLI states that the main source of fatal occupational injuries was vehicles (mainly motor and road vehicles, 45.2%). The proportion of serious occupational injuries when using machinery equipment was 9.7%.

The most frequent reason for injuries was the so-called undetected reason that was related to the occurrence of a high amount of road traffic accidents. The other main reason for injuries (up to 29%) were workers themselves, using dangerous processes or methods of work including acting without authorization, against orders, a prohibited act, or they remained within an endangered area.

Many authors analyze accident rate statistics and deal with the causes. They emphasize inappropriate processes for equipment design [4,5] or non-observance of safety procedures throughout their operation [6]. It is interesting that increasing the efficiency level by implementing preventive measures may depend also on the level of both the cognitive and emotional approach of managers to safety [7,8]. In industrial practice, the methods of Behavior-Based Safety (BBS) have tried to be asserted over the last decade, successful implementation of which requires a change in human behavior by means of a positive influence that aims to prevent hazardous acts. However, the program requires a certain level of commitment and implementation at the managerial and occupational health and safety (OHS) levels (e.g., ISO 45001) and it must be understood as providing “ added value to the management system” [9,10].

OHS management represents a complexity of activities related not only to the construction and standards of technical equipment but also to the work environment and care of employees’ behavior in order to minimize serious accidents and incidents. In 2017, during the XXI World Congress on Safety and Health at Work: A Global Forum for Prevention, the effort to create such a platform and support within the OHS field was declared, so that it is possible to reduce the level of injuries and damage to health to “almost” 0 (Singapore’s Vision Zero Journey) [11].

In 1989, the legal framework for determining the mutual relation between machinery design and its safe use in practice was created in Europe. These two basic regulations were 89/391/EEC, known as the “OHS Directive” [12], and 89/392/EEC, known as the “Machinery Directive”. Requirements for construction of “safe machinery” were developed and the Machinery Directive is currently known as the European Directive 2006/42/EC [13]. Conditions for maintaining a safe state of machines during their operation with emphasis on the operating conditions are included in Directive 2019/104/EC, which is bound by the OHS directives within the meaning of Article 16(1). Basically, this created a framework in the EU to establish the interconnection of requirements for machinery safety between its manufacturer or an authorized representative and the user.

Knowledge and experience from accidents confirm the inevitability of integrated prevention based on legal responsibilities [12,13,14]. Safety management is based on a holistic approach, the basis of which is risk assessment and effective and efficient prevention that is achieved by the cooperation of everyone.

In regard to this development and trends not only in Europe but all over the world, machinery safety appears as a field that is still not sufficiently solved and further development supported by new regulations and harmonized standards is expected.

The condition of machinery, the methods for performing activities, and the environment in which the machine is located are the basic factors that can influence the occurrence of hazardous events. Although the unified EU market expects the observance of the actual rules for machinery construction, it is, however, mainly the organizations that have out-of-date machinery and make changes to the machinery without adequate consideration of the basic requirements of the Machinery Directive [13]. The next issue is the education and training of technical workers/designers who have no knowledge of the principles and requirements for safe designing [14,15,16]. Some industry studies have identified up to 50% of the root causes of serious industry accidents come from mistakes caused by machinery designs [17].

A risk assessment performed early in the stage of the machine’s design is an essential requirement for safe machinery construction, so that the residual risks are at the lowest possible level. Although modernization of old machinery provides higher effectivity and enables a significant increase in productivity by means of new automated functions, this will only happen when the right principles of safe design are used [18].

Modernization may also be performed to increase reliability and safety, meet environmental requirements, and for maintenance improvement purposes. When implementing new safety principles into old machinery, the knowledge and implementation of logical steps of risk assessment are necessary, targeting risk reduction in both an effective and efficient way. Consideration of all activities related to the machine’s operation throughout its entire life cycle (pursuant to ISO 12100 [19]) is mainly related to an intended use assessment as well as to reasonably foreseeing any misuse, however, the actual trend also requires consideration of every kind of intentional violation, which can affect the integrated machinery’s safety system (e.g., applied pursuant to the ISO 13849-1 requirements) [20].

This article contributes to the analysis of current methods and requirements for safe machinery construction. It describes a methodology of assessment of safety conditions and the efficiency of measures implemented throughout the machine’s life cycle. It stems from the fact that the machine’s condition and its operation changes during its life cycle, which is not always considered in safety management practice. For this reason, the foundation of the methodology is risk assessment, as a tool to determine the level of machine safety in any operation. This risk assessment was performed in accordance with the principles described in the harmonized standard ISO 12100:2010 within a particular production plant (using their internal risk matrix for risk evaluation). In the past, risk reduction measures had already been implemented on the machines (either by the manager or operator depending on the machine’s age and/or requirements of the operation itself), these measures and the machine’s state were assessed by experts in the field of safety by means of a questionnaire (19 criteria) stemming from the requirements of the directives (Directive 2009/104/EC but mainly Directive 2006/42/EC). The objective of the proposed methodology was to create relevant indicators that would make it possible to evaluate whether the implemented risk reduction measures were efficient.

In applying the methodology in practice on 60 machines (three operations-plants), it provides instructions on how to assess the fulfillment of the regulatory requirements and implement the right procedures to meet the requirements for safe machinery operations while meeting the business objectives of the organization.

2. Machinery Risk Assessment Methodology

No compliance with the requirements of the regulations may be proved without having performed the documented procedure of analysis and risk reduction, and without having proved all effective and efficient measures in accordance with the current knowledge and possibilities of injury prevention and harm to health when working with the equipment [21,22,23].

2.1. Basic Principles in Risk Assessment Procedure

Despite the fact that it was mainly the machinery safety that initiated the development of a number of harmonized standards to formulate the algorithm of risk assessment and risk reduction, as a basis for the design and construction of the safest possible machinery, not even 30 years makes a sufficiently long enough period for the principles in question to be implemented properly in technical practice.

The reason for this is obvious. Even with a basic knowledge of EN ISO 12100 [14,24,25,26], the harmonized standards went through several alterations, from EN 292-1, 2, EN 1050, EN ISO 14121-1, to EN ISO 12100-1, 2), and there still happens to be incomprehension of the importance of the individual steps of risk assessment (Figure 1):

Risk analysis starts with the classification of a system into elements (Step 1). The aim is to describe and identify the source of accidents as clearly as possible (Step 2), thereby highlighting what in the system is a hazard to a human, and provide a description of hazardous situations that may occur in a given activity (harm) and how (hazardous event), i.e., the accident scenario. Then, an estimation of probability and consequence follows, which requires the selection of a suitable methodology for the assignment of probability and consequence parameters (e.g., risk matrix, risk graph, fault tree analysis (FTA), event tree analysis (ETA) etc., and this methodology may exist in the form of qualitative, semi-quantitative or quantitative approaches [26,27]. The next important step in the logical procedure is the expression of the relation of probability and consequence by a combination of their values, the so-called risk estimation (Step 3).

Risk evaluation (Step 4)—is the step following risk analysis, which compares the estimated amount of risk (related to an identified hazard) with the ‘set’ parameters of its acceptability (or tolerability).

Taking of measures—is related to the risk values which either exceed or reach the unacceptable level (Step 5). It may happen that even though the estimated risk level is within the area of acceptability, the manager (designer, producer) decides to take certain measures to reduce it further. Such an approach is marked as ALARP (As Low As Reasonably Practicable)—it represents a reduction of risk to the best reasonable level, i.e., effectively and efficiently [28,29,30]. In designing the machinery, this process goes through the so-called three steps (Step 6), where first, is the hazard source elimination (replacement of hazardous properties with safe alternatives), after is consideration of whether the acceptance of a particular measure is possible or is effective to ensure the protective measures are applied. The third and last step of risk reduction is the provision of information on the residual risk in the machinery use instructions.

The whole process is considered as risk reduction but also risk control, and it requires repeated assessment in order to verify whether the proposed and implemented measures are effective.

2.2. Risk Assessment Methods

Description of the area, steps of the risk assessment, what suitable methods should be used for what particular purpose (system), is a very extensive issue. This is where mistakes regarding the ideas of risk assessment arise. For example, there are methods that are sufficient for enabling a risk assessment of the mechanical hazards of a sharp machine edge, however, the methods for a risk assessment of a hazardous substance leak (e.g., gas) from a high-pressure pipeline, including its initiation and elements such as “jet fire” type are completely different [31].

Each system that requires a risk assessment for either legislative or other reasons has certain recommended methods and procedures that accurately describe its individual application, scenarios, and potential causality [32]) for a hazardous event that may lead to damage (e.g., to life, health, property, environment, etc.). There is a difference in the risk assessment of simple machinery and the risk assessment of a potential explosion in a paint shop or a serious industrial accident of an ammonia leak from a tank. Scenarios involving a burning building in a built-up area are described differently than scenarios involving a blast-furnace gas leak in an industrial plant.

For this reason, the application of measures relates mainly to ensuring reliable and safe technical conditions that prevent the origination of a hazardous event (e.g., prevention of a leak by preventive maintenance), i.e., the so-called preventive measures [33,34,35,36,37]. To reduce hazard consequences, tools aimed at reducing the extent of the damage after a hazardous event (e.g., sprinklers, extinguishers, alarms, etc.) i.e., the so-called reactive measures, are applied.

The simplest methods often applied in practice at the level of machinery risk assessment and the work activities incorporated into the risk assessment are called the “risk matrix” [25,26,32,34,37,38]. It could be said that it is a framework method that enables the identification of hazardous properties and hazards that may result from them, an assessment of the risks and their evaluation according to the “own” rules of the reviewer (e.g., the organization or its management) considering the determined acceptance rate.

The basis of this method is following the aforementioned risk assessment algorithm (Figure 1). The harmonized standards for machinery construction EN ISO 12100 [19] stem from the assumption that the construction in question may contain certain hazardous properties concerning its future use: sharp edges, electric parts, thermal sources, ionizing radiation sources, worn parts, liquids, dust, etc. For this reason, to make easier the identification (to ISO 12100 standards) of hazardous situations that may arise during the performance of a given activity in the proximity of hazardous properties (Step 2 in Figure 1), the hazards are categorized into 10 basic groups: mechanical, electric hazards, thermal hazards, noise and vibration hazards, radiation hazards, material and substances hazards, ergonomic hazards, hazards connected with the environment in which the machinery will be used, or a combination of these hazards.

The risk matrix [38,39] enables the estimation of the probability level of an analyzed hazardous situation, e.g., touching the cutting part when cutting materials and the severity of harm, such as cutting the hand or finger off, in a simple and usually qualitative way. It does this by assigning values to the probability and consequence levels, i.e., a framework for risk level estimation related to the respective hazardous property, e.g., a mechanical hazard that may arise during a performed activity (hazardous situation) when touching the cutting part of machinery (hazardous event), as shown in

Table 1. Example of a constructed “risk matrix” according to the algorithm for risk assessment (source: own design).

Estimation of Probability (P) and Consequence (C), and Risk. (Step 3, Figure 1)		Consequence (C)
		Level of C	1—minor	2—serious	3—severe
		Description	Almost accident, or injury with the sick leave duration max. 3 days	Sick leave more than 3 days	Severe occupational injury
Probability (P)			Risk evaluation (Step 4, Figure 1)
Level of P	Description		R = P × C
1—low	The harm after exposure to a hazard is almost impossible.		R(1)	R(2)	R(3)
2—medium	The harm after exposure to a hazard is almost possible.		R(2)	R(4)	R(6)
3—high	The harm after exposure to a hazard is almost certain.		R(3)	R(6)	R(9)
Acceptable risk assessment (Risk category) (Step 5, Figure 1)		Taking measures to reduce risk (Step 6, Figure 1)
R(1); R(2)	Negligible	No measures for risk level reduction are necessary.
R(3); R(4)	Acceptable (ALARP)	It is suitable to accept and/or increase the efficiency of measures to reduce the risk level, by reducing the P level or C level.
R(6); R(9)	Inacceptable	It is inevitable to immediately take and increase the efficiency of measures to reduce the risk level, by reducing the P level and/or C level.

.

Estimation of the probability and consequence level in Table 1 represents a qualitative expression, which does not specify the frequency or probability of harm, e.g., per year. With the methodology applied for the assessment of consequences of severe industrial accidents (e.g., SEVESO III Directive), the consequence is expressed mainly by the number of affected people (within but also out of plant) and the so-called social acceptability (e.g., 1 death per 1000 persons) is evaluated [28,40,41].

3. Analytical Methodology for the Evaluation of Machinery Safety

Within the running research, experts from the safety field (two from practice and three from the university) cooperated on the creation of the methodology, the aim of which was based on the risk assessment results of operated machines (for the activities of operators and maintenance according to the specific methodology of an organization), with regard to the phase of their operation, to analyze the status and efficiency of current measures on particular equipment, or the total efficiency level of protective/safety measures utilized for all equipment in the operation (see Figure 2).

The main parameters of the model were the following assumptions: there are n operation machines at each production facility; the number of safety requirements (SR) for machinery safety is m. Then the status of the i-th requirement ($i=1,2,3,\dots ,m$) is assessed on each k-th machine ($k=1,2,\dots ,n$) by means of the suitable safety measures that are implemented.

3.1. Evaluation of the Status of Current Measures

The current status of safety requirements on an assessed machine is expressed by means of the so-called coefficient of current measure status $w_{k,i}$, for which holds $w_{k,i}\in \left\{0,1,2\right\}$. The $w_{k,i}$ coefficient represents a categorical variable, which can reach three possible values (see

Table 2. Coefficient of current measure status $w_{k,i}$

$\mathbf{Coefficient}{\mathit{w}}_{\mathit{k},\mathit{i}}$	Explanation
$w_{k,i}=0$	Measures for the fulfillment of the i-th requirement on the k-th machine are not introduced.
$w_{k,i}=1$	Measures for the fulfillment of the i-th requirement on the k-th machine are introduced but not followed.
$w_{k,i}=2$	Measures for the fulfillment of the i-th requirement on the k-th machine are introduced and fully followed.

).

3.2. Evaluation of Current Measures Status in Operations or Organizations

The level of measures efficiency ${\Delta }_k$ expresses the fulfillment of safety requirements by means of the realized measures on the k-th machine and is expressed by the relation:

$${\Delta }_k=\frac{{{\displaystyle \sum }}_{i=1}^m{w}_{k,i}}{S_{MAX}}\times 100\%, k=1,2,\dots ,n, i=1,2,3,\dots , m,$$

(1)

where n represents the number of machines, m is the number of safety requirements, and $w_{k,i}$ is the coefficient of current measures status of the i-th requirement on the k-th machine. The $S_{MAX}$ variable represents the coefficient of maximum reached efficiency of all measures on a given machine and is determined by the relation:

$$S_{MAX}=w_{max}m,$$

(2)

where $m$ is the number of safety requirements and $w_{max}$ is the maximum value of measures evaluation, in our case $w_{max}=2$.

The total efficiency level of measures $\Delta$in the given operation is expressed by the relation

$$\Delta =\frac{{{\displaystyle \sum }}_{k=1}^n{\Delta }_k}{n}\times 100\%, k=1,2,\dots ,n,$$

(3)

where ${\Delta }_k$ is the level of measures efficiency in the case of the k-th machine and n is the total number of machines located in the given operation. The total level of measures efficiency $\Delta$ in the given operation takes on values from the interval $0,100$.

For a complex safety level of a given operation conditioned by the status of introduced measures on machinery, evaluation according to the following scale was suggested:

• if $0\%<\Delta <29\%$, then the level of safety measures efficiency in a given operation is low;

• if $30\%<\Delta <49\%$ then the level of safety measures efficiency in a given operation is negligible;

• if $50\%<\Delta <69\%$, then the level of safety measures efficiency in a given operation is middle;

• if $\mathrm{if}70\%<\Delta <89\%$ then the level of safety measures efficiency in a given operation is high;

• if $90\%<\Delta$, then the level of safety measures efficiency in a given operation is very high.

If a production organization has p operations, then the complex level of safety measures efficiency $\stackrel{=}{\Delta }$would be possible to be expressed by the relation:

$$\stackrel{=}{\Delta }=\frac{{{\displaystyle \sum }}_{j = 1}^p{\Delta }_j}{p} \times 100\%, j = 1,2,\dots ,p,$$

(4)

where $p$ is the number of operations (in our case 3) and ${\Delta }_j$is the level of measures efficiency in the case of j-th operation, which holds the relation:

$${\Delta }_{j }= \frac{{{\displaystyle \sum }}_{k = 1}^n{\Delta }_{k,j}}{n} \times 100\%, k = 1,2,\dots ,n, j = 1,2,\dots ,p$$

(5)

where ${\Delta }_{k,j}$ is the level of measures efficiency in the case of the k-th machine in the j-th operation and n is the total number of machines within the given operation.

4. Results

The aim of the suggested and applied methodology to assess the safety level of machines in a phase of their use was to check the status of the machinery that was operated in a given organization from 1 to more than 30 years. Within the particular operations (three operations for the production of plastic products, marked I, II, III), there were new as well as older machines, such as automatic or semi-automatic assembly workstations, with one or a maximum of two control places (loading of components, checking and unloading of ready products).

For the research purposes, a questionnaire was created stemming from the requirements from the Directive on machinery (Annex I), which consisted of 19 safety requirements (criteria) (marked SR), the introduction thereof should efficiently prevent the occurrence of hazardous events and harm (SR1—SR19—see Annex A).

The experimental research was carried out in several parts:

• risk assessment of each machine based on the unified methodology in accordance with ISO 12100 (risk matrix—specific methodology of the organization),
• status evaluation of already established (current) safety measures for each machine according to the determined safety requirements (SR1—SR19) by means of the coefficient of current measures status,
• evaluation of the efficiency level of such measures (with regard to the outcomes from risk assessment) for each machine and for each operation,
• evaluation of complex efficiency level of introduced protection/safety measures for the whole organization.

A simplified example of the process of risk assessment and safety measures implementation evaluation is presented in

Table 3. Simplified process of the applied methodology of risk assessment and safety measures implementation evaluation on machine A003 (source: own design).

Type of Hazard	Machinery A003: Risk Assessment							Risk Reduction	→	SafetyRequirements
	n.	Hazardous Situation	Potential Consequence	P	C	R	Level of Risk	Protective Measures (Recommended)	VERIFICATION OF CURRENT SAFETY MEASURES	i = 1–19	${\mathit{w}}_{\mathit{A}\mathbf{003}\mathbf{,}\mathit{i}}$
Mechanical	M1	acceleration, deceleration (working regime, equipment)	impact, crushing, being thrown	4	3	12	H	safety devices (emergency stop, safeguard, LOTO), following SWP, PPE		SR1	2
	M2	moving elements of a machine	drawing in or trapping, crushing	3	3	9	H			SR2	1
	Mn	…								SR3	1
Electrical	E1	contact with live parts	burns, death	3	5	15	STOP	safety devices (emergency stop, LOTO, safeguard, covering live parts), following WP, PPE, regular maintenance-revisions, training		SR4	2
	E2	arc flash	burns, death	4	5	20	STOP	safety devices (emergency stop, LOTO, safeguard), maintain safe distance, following SWP, PPE, regular maintenance-revisions, training		SR5	2
	En	….								SR6	1
Thermal	T1		burns	3	3	9	H	safety devices (emergency stop, safeguard, LOTO), following SWP, PPE		SR7	2
	Tn	…								SR8	1
Noise	N1	moving elements (mutual interaction)	discomfort, tiredness	3	3	9	H	following SWP, using PPE		SR9	1
	Nn	…								SR10	1
Radiation	R1	high-frequency electromagnetic radiation	burns, headache	3	4	12	H	following SWP, using PPE, status check, and regular maintenance		SR11	1
	Rn	…								SR12	2
Material/substance	S1	gas (during bag welding)	fire, explosion	3	4	12	H	following SWP, using PPE, exhaustion		SR13	1
	Sn	…								SR14	2
Ergonomic	Eg1	position at work (operator, maintenance worker)	musculoskeletal disorder, discomfort	3	3	9	H	following SWP and work pauses, construction solution		SR15	2
	Egn	…								SR16	2
Environmental impact	En1	low or high environment temperature	tiredness, stress	3	3	9	H	following SWP, suitable work climate		SR17	0
	En2	electromagnetic disruption	tiredness, stress	3	3	9	H	construction solution, line placement		SR18	0
	En3	…								SR19	2
$\mathrm{Level}\mathrm{of}\mathrm{measures}\mathrm{efficiency}\mathrm{A}003\mathrm{machine}:{\Delta }_{k = A003}$[%] = 76.32 (High)

Explanations: R—risk (H—high, STOP—very high); P—probability; C—consequence; PPE—personal protective equipment; SWP—safe working procedure; SR—safety requirements; LOTO—Lockout/Tagout devices.

, for machine A003, a fully-automated assembly machine for the production of plastic bags in hall I. The following activities were performed on this machine: manipulation of material, gluing, welding, and cutting plastic waste.

Each i-th safety requirement of the assessed k-th machine was assigned the coefficient of current measures status $w_{k,i}$. The assigned value (0, 1, or 2) was the result of a consensus of five reviewers with the aim of decreasing the uncertainty rate of a subjective evaluation (mainly when assessing the safety status of older machines).

The results of the status evaluation of current measures in the first assessed operation (17 machines) are shown in Figure 3.

From the results of the machinery and introduced measures evaluation in operation I, it is obvious that safety requirements SR1, SR4, SR5, SR11, SR14, SR16, and SR19 were fulfilled on all machines. Requirements SR17 and SR18 were not fulfilled on any of the machines, i.e., the requirements for the application of devices for hazardous energy isolation and application of LOTO (Lockout, Tagout) meant for the machinery [42,43], they were not fulfilled.

The evaluation of the current status of introduced measures on particular machines of the operation I are graphically shown in Figure 4.

The analysis of the evaluation showed that in operation I, there was no machinery that would meet all safety requirements (see Figure 3). Each machine met on average only 52% of requirements, where measures were implemented and followed. For almost 39% of the requirements, safety measures were introduced but not followed.

For each k-th machinery equipment, the requirement fulfillment efficiency ${\Delta }_k$ was determined according to the relation (1). The total level of measures efficiency $\Delta$ in operation I (marked ${\Delta }_I$) was determined according to the relation (3). Graphical illustration of the current measures efficiency level on 17 machines of operation I are shown in Figure 5. The red horizontal line represents the average value of the total efficiency of current measures in the operation (${\Delta }_I = 69.66\%$).

Three criteria were analyzed and evaluated in a similar way (criteria, ${\Delta }_k$, $\Delta$) in two further operations: operation II (20 machines) and operation III (23 machines). The basic descriptive statistics of total efficiency of current measures $\Delta$ in particular operations are shown in

Table 4. Descriptive statistics of the total efficiency of measures $\Delta$ in operations [%].

Operation	Number	Average	Max	Min	Range	Stand. Deviation	95% Confidence Interval
I	17	69.66	78.95	63.16	15.79	4.57	(67.24, 72.02)
II	20	75.66	78.95	47.37	31.58	6.77	(72.49, 78.83)
III	23	53.89	81.58	52.63	28.95	6.03	(51.28, 56.50)
Complex efficiency of organization	60	65.61	81.58	47.37	34.21	11.25	(48.66, 68.52)

.

The analysis showed that the total evaluation of the current applied safety measures in operation I reached a value of 69.66% of the total efficiency rate, which represented the middle-efficiency level of the current safety measures.

Operation II reached a value of 75.66% of the total efficiency rate, which represented the high-efficiency level of measures. On each machine of operation II, there were on average only 62% of requirements, where the measures were introduced and followed. For almost 27% of the requirements, measures were only introduced but not followed.

Operation III reached only 53.89% of the total efficiency level, which meant it was on the lower borderline of the middle level of measures efficiency. On each machine of operation III, there was on average only 18% where the required measures were introduced and followed. For almost 71% of the requirements, the measures were only introduced but not thoroughly followed.

The results of the evaluation of the current status of measures in the whole organization (a total of 60 machines) are shown in

Table 5. Status of current measures for safety requirements in an organization (n = 60) [%].

${\mathit{w}}_{\mathit{k},\mathit{i}}$	Safety Requirements SR/Current Status [%]
	SR1	SR2	SR3	SR4	SR5	SR6	SR7	SR8	SRS9	SR10
0	0.0	0.0	0.0	1.6	0.0	0.0	0.0	0.0	0.0	0
1	32.0	68.0	61.7	36.7	70.0	100	1.7	100	100	60.0
2	68.0	32.0	38.3	61.7	30.0	0.0	98.3	0.0	0.0	40.0
${\mathit{w}}_{\mathit{k},\mathit{i}}$	SR11	SR12	SR13	SR14	SR15	SR16	SR17	SR18	SR19
0	0.0	0.0	0.0	1.7	1.7	0.0	100	100	0.0
1	56.7	38.3	58.3	0.0	73.3	36.7	0.0	0.0	38.3
2	43.7	61.7	41.7	98.3	25.0	63.3	0.0	0.0	61.7

.

It can be seen from the results of the evaluation that as much as 98.3% (59 out of 60 machines) of machines within the organization met safety requirements SR7 and SR14. On the contrary, safety requirements SR17 and SR18 were not met on any machine in the whole organization, i.e., the requirements for the application of LOTO for machinery were not fulfilled. All machines lacked markings concerning hazardous energy isolation, and the methodology for the assignment of LOTO devices for maintenance activities or specific operator procedures were not elaborated (even though the organization declared TPM—Total Production Maintenance introduction) [44].

To evaluate the status of machinery safety in the entire organization, a complex efficiency level of introduced measures $\stackrel{=}{\Delta }$ (4) was determined, according to safety requirements SR1–SR19 (see Table 3). Based on the results of the applied methodology of efficiency level evaluation of the introduced safety measures on machines and the evaluation of the total efficiency level in the entire organization, it was possible to state that the value of complex efficiency level $\stackrel{=}{\Delta }$ was valued at 65.61%, which meant the middle-efficiency level.

5. Discussion

The performed research and development of the methodology for complex efficiency evaluation of machinery safety measures in a production organization stemmed from the fact that despite the changes related to the length of operation and machine operating conditions themselves, the outcomes from the risk assessment were not taken into account during their implementation. The machinery directive requires the application of the risk assessment with every considered change on the machinery during its life cycle. For this reason, a group of five safety experts (two from practice and three from the university) focused on the development of a complex methodology. The first step was the repeated risk assessment for all identified hazards (in accordance with ISO 12100:2010) on each machine and a risk evaluation using the risk matrix tool. Based on the risk evaluation results, risk reduction measures were recommended. As these machines had been operating (three operations, 60 machines) already for several years, the team of experts, based on the created criteria (see Appendix A) and their evaluation ($w_{k,i}$), assessed whether the current measures were sufficient to be able to provide an efficient risk reduction (${\Delta }_k$) for each machine by comparing measures recommended against the risk assessment. At each operation (I, II, III), there was a certain group of machines (17, 20, 23) that were very similar from a production point of view, which is why the efficiency of measures on every machine is necessary to contribute to the overall complex efficiency of the implemented measures on all machines in the operation ($\Delta ).$To be able to evaluate this complex efficiency, criteria were chosen—from low (0–29%) up to very high (more than 90%). This parameter made it possible to compare several different operations and to determine whether the machinery safety regarding its age and the operating conditions were at the required level and that safety measures were implemented efficiently to reduce the risks connected with its use.

The benefits of the methodology are that it supports O.H.and S. management in the determination of particular objectives—priorities during the improvement of the safety of machines themselves but also the entire operation based on e.g., safety hazards but also machinery availability, quality of production, etc. The proposal of the complex efficiency level of safety measures efficiency parameter $\stackrel{=}{\Delta }$for the whole organization has a few advantages—to continuously evaluate the improvement of machinery safety in the whole plant, e.g., with every change of technology, work organization, production program, and, in the case of a corporate company, it also makes it possible to compare similar production plants with each other.

During the application of this methodology to a specific organization, a complex efficiency level of implemented measures $\stackrel{=}{\Delta }$ was determined (4), and according to Safety requirements SR1–SR19, it was assessed to be at the level of 65.61%, which represented the middle-efficiency level. It is possible to reach a higher level of complex efficiency measures $\stackrel{=}{\Delta }$ by introducing additional safety measures at those operations, e.g., III, which had the lowest efficiency level, $\Delta =53,89\%$, by improving the efficiency of the safety measures on the particular machine ${\Delta }_k$.

The selection of Safety requirements SR1-SR19 can be extended and modified based on the development of future requirements for machinery safety regulations, but also based on the specific business objectives of an organization to reach the highest possible level of machine operational safety. The successful application of this methodology requires a systematic risk assessment of machines in a unified way (use of the same tool) and sufficient knowledge of efficient operating conditions.

6. Conclusions

An applied methodology for the complex assessment of machine operational safety levels, that promoted efficiency and made it possible to check the status of the introduced machinery safety measures against current legislative regulation requirements in three production facilities. By means of evaluation parameters, the safety level of a machine itself, the safety level of the operations (production hall), as well as the complex level of safety management of the machinery in the whole facility (value of complex efficiency level $\stackrel{=}{\Delta }$) were checked.

By classifying the efficiency into levels (low, negligible, middle, high, and very high), the basic framework for machinery safety status improvement was created. Getting OHS managers to decide on the effectiveness and efficiency of introduced measures is not always a simple task and requires a suitably applied tool. The methodology in question enables, in future, to complete safety requirements with machinery critical criterion, while also considering the business objectives of an organization (e.g., productivity, quality, efficiency, maintenance costs, etc.).

Another research step in the field of machinery safety will be the analysis and evaluation of new machine readiness status for “smart productivity/factory” requirements. Machinery safety management systems must be able to react to an attack from the outside so that their functionality is not threatened in the case of requirements for their safety (safety function). Actual trends in machinery design show minimal respect for the current holistic approach to safety management [45,46] (Safety and Security) utilizing the requirements and principles of ISO/TR 22100-4 (IT security aspects) [47]. For this reason, the objective of further research will be the extension of a machinery safety complex assessment by a security measures check as an integral part of the safety measures on machinery operated in the so-called Industry 4.0 plants [48,49].