**Stage 2:**

Analysis of questionnaire results showed no significant difference between average scores before the designated systems thinking course (Time 1) and after its completion (Time 2) (*t* = −0.61, Sig. = 0.5476).

Table 3 illustrates independent samples *t*-test outcomes.

**Table 3.** *t*-Test Outcomes.


Figure 3 shows the normal theoretical quantiles at both time points (course onset on the left and course completion on the right). According the results of the Q-Q plots, the difference scores were not normally distributed.

**Figure 3.** Q-Q Plots.

Since normal distribution was not met, we used the Wilcoxon non-parametric test to compare the scores before and after the course.

The results of Table 4 clearly illustrate no significant difference between scores at both time points (Sig. = 0.6881).

This result may be explained in several ways:



**Table 4.** Wilcoxon two-sample test.

In order to examine the last hypothesis, questionnaires were distributed to two groups whose members did not study the course: one group of students studying mechanical engineering and a second group of students studying towards their bachelor's degree in psychology.

Table 5 presents the analysis of variance when comparing total questionnaire scores among mechanical engineering, industrial engineering and management, and psychology students. This comparison was carried out during Time 1 only.


**Table 5.** ANOVA comparing total score among three groups (Time 1 only).

Table 5 shows a significant difference exists between the three groups in total questionnaire scores at Time 1 (Sig. = 0.0010). Figure 4 indicates that the average scores of industrial engineering and managemen<sup>t</sup> students were higher compared to those of mechanical engineering and psychology students.

**Figure 4.** Grade distribution of the three groups.

Multiple comparisons were carried out to determine whether significant differences exist in total scores among the three groups, as presented in Table 6.


**Table 6.** Tukey studentized range (HSD) test.

Since normal distribution was not met, we also used a Kruskal-Wallace non-parametric test to compare the three groups' total scores. Table 7 shows that a significant difference was found among the three groups (Sig. = 0.0028).


**Table 7.** Kruskal-Wallace Test (Time 1 only).

Figure 5 presents score distribution among the different groups. Industrial engineering and managemen<sup>t</sup> students demonstrated an innate tendency for higher levels of systems thinking capability compared to mechanical engineering and psychology students.

One possible explanation for this finding may be related to the structured differences between industrial engineering and managemen<sup>t</sup> students and those students who study other fields. It is reasonable to assume that among people who have a systems thinking approach, there is a tendency to choose a multi-disciplinary profession which, by its very definition, requires systems thinking. This means that people with an innate systems thinking approach will prefer a profession that is systems-oriented, such as industrial engineering and management, while those whose natural tendency is to see details will choose a profession that requires paying attention to the small details. Similar findings to these results were also presented is Kordova's previous study [28,36].

**Figure 5.** Grade distribution of the three groups—Time 1.

#### **5. Summary and Conclusions**

This study examined whether it is possible to train engineers and graduates for a systems-oriented position in a formal teaching environment such as systems engineering courses or systems design. The different courses teach the engineering design process, and during the course, a systems model is built; a model based on a structure related to requirements, functions, components and tests. The full model also includes systems scenarios, material and design interfaces, as well as outputs and inputs.

The main goals of these courses are: to provide knowledge about product design and development processes; to provide knowledge about different technologies in different business environments; to learn about methodologies and tools used for product and services design and development; to give

students self-confidence in their personal ability to initiate and design new products/services, as well as to present and "sell" their ideas and products to clients for design and development projects.

These subjects are a main part of systems engineering studies; however, according to the results of the first part of the study, these courses focused primarily on specific engineering design processes and did not provide sufficient tools for developing systems thinking skills among the course participants. One of the study groups participated in an engineering design course, which was taught over a two-semester period as part of the master's degree program for systems engineering. The course lasted two semesters, and respondents completed the CEST questionnaire [39] at three different time points: at the beginning of the course, the end of the first semester, and the end of the second semester.

The results showed that there was no significant difference between systems thinking skills before the course and after the first semester. However, a significant difference was found between the students' average score at the end of the first semester and their average score at the end of the course. In addition, a significant difference was found between the students' average score before the course and their average score at the end of the second semester.

From these results, we can conclude that the second part of the course provides systems thinking tools to a greater extent than the first part of the course, which mainly focused on specific engineering design.

These results also show that it is necessary to create a systems thinking study course that deals with specific methodologies and systems thinking tools. These findings are in line with the results of Davidz and Nightingale [31] and Kasser [32], which showed that it is possible to acquire systems thinking through education and training.

The study also examined the systems thinking skills of systems engineers as opposed to other engineers who are partners in systems projects. It was found that the systems engineers' score on the systems thinking questionnaire was significantly higher than the other engineers' scores (Sig. = 0.000).

In addition, these engineers' managers were asked to evaluate the engineers' tendency towards systems thinking and to rank them on an ordinal scale. A significant correlation was found between this ranking and the score on the questionnaire that evaluated the engineers' system thinking (*r* = 0.855, Sig. = 0.000).

In contrast to these results, no correlation was found between the systems thinking score and number of years' experience acquired by the engineer.

These findings stress that systems engineers with high systems thinking skills are capable of understanding the general/big picture—functionally and conceptually—even without understanding all of the small details.

The study findings show that despite the difficulty to define systems thinking, people know how to evaluate the systems thinking skills of their work colleagues, and to identify those who immediately see the big picture compared to those who tend to look at the small details.

The finding that shows no correlation between systems thinking skills and number of years' experience may indicate that additional factors exist which foster this ability.

The fact that it is often possible to distinguish a capacity for engineering systems thinking, even after only a few years of work experience, proves that apparently there are additional factors that strengthen systems thinking acquisition. Among these factors, there is also the notion of innate potential—which seems to be an inseparable part of those candidates who received a high CEST score, even though they had little work experience (in years) [28].

This finding supports Frank's claim [38] that systems thinking is a combination of an acquired ability and an innate talent.

The current study found that a link exists between personality type and systems thinking skills. The study found that 57.9% of the engineers in the sample belonged to the sensing, thinking, judging (STJ) personality type, according to the MBTI questionnaire. This finding emphasizes that a large percentage of engineers have unique personalities and traits.

The second part of the study presented a preliminary attempt to develop a systems thinking study course. Since this is a pilot course, additional studies are needed with diverse sample groups in order to strengthen the claim that this type of course is likely to improve its participants' systems thinking skills.

**Author Contributions:** S.K.K. was the main researcher of this study who prepared the literature review, developed the study design and analyzed the results. M.F. developed the CEST (capacity for engineering systems thinking) assessment questionnaire and conducted several studies for assessing the reliability and validity of the questionnaire. A.N.M. conducted the pilot course presented at this paper. The main goal of this course was to develop systems thinking capability.

**Funding:** This research was supported by Gordon Center for Systems Engineering at the Technion—Israel Institute of Technology.

**Acknowledgments:** The authors would like to thank Guy Ribnikov for his contribution for the study findings.

**Conflicts of Interest:** The authors declare no conflict of interest.
