3.1. Analysis of Data
Before presenting the results of a survey, their credibility must be determined, and an analysis of the data themselves needs to be conducted. Thus, an analysis of variance (ANOVA) and Cronbach’s alpha were performed on the received answers. ANOVA is one of the most widely used statistical techniques, with applications in areas that include biology, medicine, industry, and finance [
21]. The ANOVA test has long been a valuable tool for researchers conducting studies on multiple experimental groups and one or more control groups [
22]. The main goal of an ANOVA is to identify whether the null hypothesis is accepted or rejected. A null hypothesis is the proposition that a study’s findings will not demonstrate an effect when subjected to a scientific experiment. If the null hypothesis is true, there will be no differences between the intervention and control groups at the end of the study [
23]. An ANOVA uses the F-test to statistically evaluate the equality of the means. The F-value is the ratio of the between-group variation and the within-group variation. The given F-value is compared with the critical value of the F ratio derived from the table for a given significance level. The significance level determines the risk of concluding that a difference exists where there is no actual difference. The most common significance level of 0.05 was used. If the given value of F is greater than the F ratio obtained from the tables, we can reject the null hypothesis with a 95% confidence level, as shown in
Figure 2.
Equation (1) describes how the value of F calculated is obtained:
where:
Equation (2) describes how to calculate the standard deviation:
where:
In this case, n represents the number of questions, x_i represents the answer of one respondent to one of the questions, represents the arithmetic mean of the total number of answers to each question for a specific company, and the degrees of freedom is the number of variables (questions) minus one. After the F-value was calculated for each of the companies in the study, the values obtained were compared with the critical value of F_0 obtained from the table of the critical values for the F distribution for given degrees of freedom. The values were as follows:
= 2.0837 > = 1.4591.
= 4.1361 > = 1.4591.
= 3.7868 > = 1.5543.
= 5.8746 > = 1.4591.
It was concluded that the null hypothesis is rejected, the results of the study are credible, and there is a difference among the populations.
Another approach to the verification of data is to use the Cronbach’s alpha (α) and McDonald’s omega (ω) coefficients. Cronbach’s alpha is the most common estimate of the internal consistency of responses on multi-item bipolar scales [
25]. It can be defined as measuring the extent to which item responses (i.e., answers to survey statements) correlate with each other. In other words, Cronbach’s alpha estimates the proportion of variance that is systematic or consistent in a set of survey responses. Often, when skew items are present, it is preferable to use the omega coefficient, as it is more robust than the alpha. Therefore, both coefficients were calculated. Equation (3) describes how to calculate Cronbach’s alpha:
where:
The value of Cronbach’s alpha is usually expressed as a number between 0 and 1. A value of 0 means there is no consistency in the measurements. The closer the Cronbach’s alpha coefficient is to 1.0, the greater the internal consistency of the items on the scale. An acceptable range is between 0.70 and 0.90, or higher [
26].
Omega estimations are based on confirmatory factor analysis (CFA). A CFA model fits the data first, and then the omega is calculated based on the factor loadings and the error variances [
27]. Equation (4) describes how to calculate McDonald’s omega:
where:
The coefficients were calculated using the freeware software “JASP 0.18.3.0.” An analysis was conducted for each individual group of participants (i.e., each company). The results of the analysis are shown in
Table 3.
The cutoff for both coefficients is 0.7. All of the above scores are considered reliable. Because all of the Cronbach’s alpha and McDonald’s omega coefficients are greater than 0.70, it can be concluded that the data from the surveys are consistent.
3.2. Results
For a better understanding of the given data, the results were grouped both by company and by hierarchical position within the company. This way, it can be observed whether the size of the company and the hierarchical position had any influence on the perception of challenges during the implementation and maintenance of TPM and lean philosophies in general. First, the results were filtered by company. Company 1 is a small enterprise that focuses on innovative solutions in automation and robot technology. Company 2 is a medium-sized company that provides engineering solution services. Companies 3 and 4 are large enterprises with over five hundred employees. The goal was to determine whether the size of an enterprise influenced the perception of potential problems and challenges related to the implementation of lean tools and TPM.
The survey participants were also asked to provide information about their role at the company, so in the second part of the data analysis, the data were grouped according to job function within the company. It is beneficial to see the potential drawbacks from different points of view. As shown in
Table 4, the data were grouped into four categories based on roles (i.e., hierarchical levels): operators, technologists, leaders, and managers.
For better results of the data analysis, answers such as operator, worker, manipulator, and production employee were grouped under the “operators” category. Operators’ points of view are crucial for understanding the challenges of implementation, because most often it is this level that performs the time-consuming lean tasks, such as data collection, organising, cleaning, and autonomous maintenance. Roles such as technologist, designer, and operating engineer were grouped under “technologists.” Individuals in this group are often those who are well educated and among the first to help operators understand the principles. Leaders and heads of departments were all grouped under the hierarchical level of “leaders.” They are the decision-makers in a department, often also educated alongside technologists, or they decide who is going to receive the training. Leaders coordinate tasks with the leaders of other departments. Finally, managers, directors, supervisors, and executives were grouped in the “managers” category, which is the highest level of management in an enterprise. They make strategic decisions and are expecting to be briefed on the results of the implementation of lean tools and TPM. They are usually only introduced to the lean philosophy in the form of the vision and goals related to lean tools and TPM.
The numbers below the x-axis (representing statements) are the sums of all of the total answers (each on a scale from one to five) for a selected group. Because a different number of surveys were received from each company, the values were normalised for comparison. The higher the number, the higher the level of agreement with each statement. On the contrary, the lower the number, the lower the level of agreement with a statement. The lowest cumulative answers of the individual groups are emphasised below, and the results are shown in
Figure 3.
For Company 1, these were statements Q19 (job security is not threatened by lean), Q23 (lean tools are considered as important as other activities), Q24 (lean activities are not performed only before an audit), and Q20 (there is no resistance to change). Employees at Company 1 (a small enterprise) expressed worry about job safety when performing lean tasks. This can result in poor job performance, the masking of real problems, hiding true results, and conducting lean tasks only before an audit, as recognised by the workers themselves. Management must assure employees that their job is not threatened by lean processes. On the contrary, the aim of lean activities is to ease the workload. The fact that employees at the small company did not recognise the importance of lean tools is a failure of the training program. The employees of Company 1 also thought that there was a certain amount of resistance to change, which is common when implementing new practices but can be reduced using correct implementation methods.
Company 2 is a medium-sized enterprise with approximately 150 employees. Their primary business is the provision of engineering solution services. Employees at Company 2 had the lowest level of agreement concerning statements Q25 (an audit is a good use of time), Q12 (performed on suitable equipment), and Q16 (lean activities are well aligned with other tasks). The employees of this medium-sized enterprise did not recognise audits as an important part of TPM and lean practices in general. It may appear that once a system is established, there is no need for regular reviews and checks. However, auditing any system is crucial for ensuring the stability, continuity, and quality of implemented practices. A common consequence of a lack of audits is that employees tend to become complacent in performing their tasks, inevitably leading to a significant drop in quality. Additionally, employees at Company 2 believed that TPM was not conducted on suitable equipment. This could indicate a lack of strategic selection of equipment suitable for TPM implementation. Typically, a pilot project is developed at the beginning of activities to demonstrate TPM principles with a suitable and compatible system. This approach allows for the philosophy to be understood in an easy and comprehensive way, using suitable equipment as an example. The third major concern raised by employees in the medium-sized company was that the lean activities were not aligned with their regular workloads. This issue can arise in medium- and large-sized enterprises, particularly when multiple trainers and departments are involved. In such cases, it becomes crucial to synchronise activities to ensure alignment among all workers involved.
Employees at Company 3 had the lowest level of agreement with statements Q25 (an audit is a good use of time), Q12 (performed on suitable equipment), Q24 (lean activities are not performed only before an audit), and Q19 (job security is not threatened by lean). As did employees at the medium-sized enterprise, employees of this large-sized enterprise expressed their biggest concerns about lean audits and their benefits. This is another example of why an emphasis on auditing practices during training is vital. Educators must take the time to explain the role of audits in the sustainability of implemented practices. Employees must be included in the audit process so that everybody involved in lean activities is audited at least once and is an auditor themselves. It is important to clarify during training that an audit is not used to “catch” someone not performing planned activities but as a tool to increase the success of established processes. In connection with the questioned importance of the audits, employees of this large-sized enterprise thought that lean and TPM activities were only performed before audits. This is a cause for concern. Lean activities should not be performed for the sole purpose of passing an audit. If this is the case, the leaders of those processes must be reminded and, if needed, retrained. Interestingly, even though employees at Company 3 did not consider lean and TPM activities important, as indicated by the statement that most activities are only done before audits, they still perceived them as threats to their job security if they did not engage in them. This is a failure of implementation, and its effect is the opposite of the intention.
Company 4 is also a large enterprise, with 800 employees. They had the lowest agreement with statements Q23 (lean tools are considered as important as other activities), Q17 (there is enough time), Q19 (job security is not threatened by lean), and Q20 (there is no resistance to change). People at this large-sized company did not see lean tools as important for the company as other everyday tasks and activities. Also, they thought there was not enough time provided for these activities. It is often the case that lean activities are added on top of the daily tasks of workers. In the process of planning implementation, time for additional activities must be included as well. Leaders need to take into account the amount of time needed for these activities and adjust workloads accordingly. Job security was also an issue at this company, and people thought there was great resistance to change. This is often the case at companies with a higher average age and with workers who have been at the company for multiple decades. Some of them have performed their tasks for years in the same way. It is natural to feel resistance to change. It is the responsibility of an educator to thoroughly explain the benefits of lean and TPM practices.
When the results were compiled for all of the companies, the least agreed upon statements were Q19 (job security not threatened by lean), Q23 (lean tools considered as important as other activities), and Q25 (an audit is a good use of time). An overview of the statements with the lowest levels of agreement among the companies is shown in
Table 5.
Another approach to examining the results is to filter the data according to the job position within a company. This way, different opinions and points of view can be examined with respect to the hierarchical level. Therefore, data were evaluated according to operators, technologists, leaders, and managers. As with the comparison by company, these data were also normalised to compare the results among the groups. The results are visually presented in
Figure 4.
Operators indicated less agreement with statements Q19 (job security is not threatened by lean), Q23 (lean tools are as essential as other activities), and Q24 (lean activities are not performed only before an audit). Naturally, operators were most concerned about job safety because it was not within their control. Their performance depends on an evaluation by their leaders and managers. However, when it comes to lean activities and tasks, the operators do not recognise their importance and usually perform them before checks and audits. It is the responsibility of technologists and leaders during training and implementation processes to avoid such outcomes.
Technologists showed less agreement with statements Q19 (job security is not threatened by lean), Q25 (an audit is a good use of time), Q12 (performed on suitable equipment), and Q16 (activities of lean are well aligned with other tasks). The results indicate that technologists share concerns about job security and find audits to not be a valuable use of time. This contradiction is notable because technologists are typically the most knowledgeable about the desired lean tools and philosophies. It is possible that their opinion is linked to the third-lowest agreed-upon statement, suggesting that the implementation of these practices is not conducted on suitable equipment. This may occur if technologists are not responsible for selecting the equipment on which the pilot project is executed. Although it seems obvious, the selection of equipment or part of a production line for the pilot project must always be performed by the personnel that are most educated about the tool that is going to be introduced, and there are examples where this has not been the case.
Leaders showed less agreement with statements Q24 (lean activities are not performed only before an audit), Q25 (an audit is a good use of time), Q12 (performed on suitable equipment), and Q23 (lean tools are as essential as other activities). It is concerning that the leaders also recognise that lean activities are performed more actively, mostly before audits. Because internal audits are friendly, meaning their purpose is to sustain good practices, they are always announced. Therefore, it would be beneficial to perform unannounced internal audits. It would provide a better picture of the status of a lean tool’s performance. Another concern of the leaders was that an audit itself was not a good use of time. Often, lean is imposed on leaders by management, which expects results. Because leaders are not often the ones performing the tasks, it is easy for them to underestimate these activities, as recognised in the survey.
Managers showed lower levels of agreement with the statements Q25 (an audit is a good use of time), Q12 (performed on suitable equipment), and Q10 (implementation of lean tools was performed within a reasonable amount of time). For executives, the results of their investment, rate of savings, and return on investment time are usually the most important. High-level management often only understands the philosophy superficially, just enough to decide whether to invest. Management also thought that time spent on audits was not so useful and, in their opinion, it was not performed on suitable equipment. It is interesting to note that their third-lowest level of agreement was on the statement related to lean and TPM tools being implemented within a reasonable time frame. Often, the implementation of lean tools can take up to two years (depending on the size of the enterprise, the number of personnel involved, the scope of the implementation, and the number of lean tools being implemented). So, management, in this case, showed concern about the time span of the implementation. Therefore, when planning the introduction of lean tools, it is crucial to prepare and present a period, timeline, or Gantt chart to investors. Establishing clear milestones and time-bound goals is essential for maintaining a transparent dashboard to monitor activities. This approach helps prevent issues related to execution deadlines for specific activities or the entire project.
Table 6 provides an overview of the statements with the lowest level of agreement according to job position within a company.