STEM Graduate Students’ Systems Thinking, Modeling and Scientific Understanding—The Case of Food Production

Featured Application

Model-Based Systems Engineering is an edX Professional Certificate Program comprised of two five-week courses that teach principles and applications of model-based systems engineering using Object-Process Methodology—OPM ISO 19450.

Abstract

Modeling and systems thinking skills, as well as scientific understanding, are necessary for comprehending complex, food-related processes. The aim of this research was to evaluate the effect of food-related learning units on graduate students’ systems thinking and modeling skills, as well as on their understanding of science, technology, engineering, and mathematics (STEM) issues. In this research, six STEM experts constructed a conceptual model of the codfish tracking process using Object-Process Methodology. Next, 15 STEM graduate students, who are prospective teachers, participated in a graduate course, which includes four online units on food production processes based on their respective models. Research tools included an expert focus group, student assignments, and questionnaires. Modeling and scientific understanding rubrics were adapted and validated for analysis of the assignments. We found a significant difference in the scores of systems thinking and modeling skills between students with modeling background and those without. Based of students’ feedback along the course, learning in context of food and sustainability also contributed to developing these skills. The contribution is the combination of food production and conceptual models for developing STEM teachers’ systems thinking and modeling skills, and their scientific understanding of food processes and sustainability issues.

1. Introduction

In an age where problems and systems are becoming ever more interdisciplinary, system thinking is an increasingly sought-after ability. This is especially true when it comes to science, technology, engineering, and mathematics (STEM) disciplines. In response to this need, we have developed two Model-Based Systems Engineering courses as an edX Professional Certificate Program [1]. As STEM-based systems are becoming more complex, the requirement for systems thinking is increasing. Both the Next Generation Science Standards—NGSS [2] and the criteria of Accreditation Board for Engineering and Technology—ABET, 2018–2019 [3] for accrediting engineering programs include concepts related to systems thinking as part of engineering education. Developing and using models is the second of eight key practices which NGSS has recognized as the 21st-century skills [2]. The first practice is asking questions and the other six are planning and performing investigations, analyzing data, implementing computational thinking, constructing explanations, drawing arguments, and obtaining and evaluating information. According to the scientific understanding theory, background knowledge is a key to explaining phenomena [4]. The nature of science is considered a critical component of scientific literacy and has been a dominant area of research in the field of science education. Learning science has unique features, including required background knowledge, practices, methods, and methodological rules [5,6].

Citizens of the modern world are required to apply trans-disciplinary abilities that enable them to project a broad perspective and gain deep understanding of complex systems [7,8]. This makes systems thinking an increasingly important skill not only for STEM experts but also for school students, who are the citizens of tomorrow.

Beside the general trans-disciplinary knowledge and systems thinking skills, citizens should be familiar with sustainability, food production, and food nutritional values, as these subjects are critical to the future of each individual and to the planet as a whole. Therefore, learning these topics and understanding them is crucial not only for food engineers and environmental professionals but also for the community at large [9,10,11]. Consequently, increasing awareness of sustainability and food-related issues might serve as a good platform to develop systems thinking.

Project-based learning promotes meaningful learning that focuses on finding and deploying a solution to a real-life problem [12]. Learning through projects can bridge between the theory learned in class and the real world. This learning method typically integrates several disciplines and contributes to the development of various thinking skills [13,14]. Sustainability, a prominent trans-disciplinary domain of human research and activity, can promote active learners through project-based learning as they face real-life problems, and provide opportunities to incorporate project-based learning as part of the curriculum [15,16].

This work aims to facilitate communication and synergy between seemingly different domains: Food quality and food production on one hand and system thinking on the other hand. In this study, we focus on STEM teachers, who are expected to understand and address complex, interdisciplinary, real-world problems, so they can convey it effectively to their students. Such synergy is key to the teaching and learning process in STEM disciplines, and it explains the criticality of developing STEM teachers’ systems thinking and scientific understanding skills.

To achieve the goal of raising teachers’ systems thinking and scientific understanding, we have used model-based systems engineering and conceptual modeling with Object-Process Methodology—OPM [17,18,19] ISO 19450 [20] to model holistically the function, structure, and behavior of the codfish and other food production processes and value chain, ultimately enabling real-time view of the physical objects and processes along this food chain at increasing levels of detail through their digital twins. The fact that OPM is recognized as ISO 19450 is key to its international dissemination, as it can be accessed globally via the ISO online platform [20]. This specification provides a basis for understanding OPM and complying with its syntax and semantics.

Table 1. Abbreviations used for Object-Process Methodology—OPM representations.

Abbreviation	Explanation
OPM	Object-Process Methodology
OPD	Object-Process Diagram
OPL	Object Process Language
SD	System Diagram
SD1	System Diagram—first detail level

presents a few abbreviations used in OPM for its two complementary, bimodal representations: the textual and the visual.

1.1. Systems Thinking

Systems thinking is an approach for examining complex situations and systems in a holistic manner [8]. Systems thinking as a concept has been developed in many disciplines, including social science [21], natural sciences [22], and engineering [23]. Systems thinking in science and engineering has a dual meaning: In the context of engineering, systems thinking implies understanding of the system’s structure and behavior and the interactions between them to achieve the function, which provides value—benefit at cost—to the system’s beneficiaries [19,24]. In the context of natural sciences, systems thinking entails a description of natural phenomena, preferably through a model, which can be used to explain and analyze it both qualitatively and quantitatively, e.g., via cause-and-effect of physical, chemical, or biological events or circumstances [25]. Usually those assessments are based on attitudes or on system thinking personality traits [26,27,28]. Assessing systems thinking based on a system description, model, or representation is rare. It is important to create and use a common, unified language across disciplines for systems thinking [29]. OPM with its universal minimal ontology of objects as things that exist and processes as things that transform objects caters to systems thinking promotion. In this research, it serves as the conceptual modeling language and methodology, with models created by the experts and students serving as the basis for systems thinking assessment.

1.2. Conceptual Models and Object-Process Methodology

As the content complexity of STEM subject matter is rising, students in the various STEM disciplines are required to engage with more data, information, and knowledge. One approach to deal with this rising complexity is conceptual modeling with graphical languages and tools that support them [30].

A concept can be considered as the basic unit of knowledge [31]. Learners can use concept maps to represent their declarative knowledge structures. Concept maps comprise blocks that describe the concepts and lines linking them to informally describe relationships among the concepts by writing them along the links. Such maps can support learners in constructing new knowledge, as new information is linked to prior knowledge [32,33]. In model-based systems engineering, conceptual models are the central artifact that represents systems and provides the source of authority throughout the system lifecycle. Unlike concept maps, conceptual models use a formal graphical language, in which there may be a differentiation between several kinds of concepts or entities and several kinds of relations between the various entity kinds [19].

Figure 1 is an example of the difference between a concept map (left) and an OPM conceptual model (right). Both diagrams express the fact that a train and an aircraft are vehicles. Since concept maps are limited to concepts (graph nodes) and relations between them (graph edges), one has to specify textually the relation along each edge. In Figure 1, the two relations are “is a”. In the OPD on the right, a dedicated symbol, the blank triangle, is used to denote the “is a” relation, which in OPM is called generalization-specialization. Moreover, since OPM is bimodal, implying that each fact is modeled both graphically and textually, using the OPM modeling software OPCloud [34], OPM provides an automatic textual interpretation of the graphical representation that the modeler provides. Each symbol has a specific syntax and semantics. In this example, the blank triangle in the OPM model, which expresses the generalization-specialization relation, provides for using inheritance. Bold indicates names of objects or processes in the OPM model. Thus, if we assign to the object Vehicle the attribute Traveling Medium, this attribute is automatically inherited to its Train and Aircraft specializations, so the respective attribute values land and air can be assigned to them (not shown). The formality of conceptual models provides a more rigorous knowledge representing than that of concept maps, but proper use of a conceptual modeling language requires some learning to use it properly.

Object-Process Methodology (OPM) is a formal conceptual modeling and language methodology for model-based systems engineering and conceptual modeling of systems, with a recognition by ISO 19450:2015 [20]. Each OPM element is defined as either a thing or a link. Things are objects, which can be either physical or informatical, and processes—things that happen to objects and transform them. A process transforms one or more objects by creating new objects, consuming existing objects or affect existing objects by changing their state. Objects may have states that are changed by processes. Links define structural or procedural relations between things, and each kind of link has well-defined semantics. OPM models are represented in a tree-structured hierarchy of object-process diagrams–OPDs. SD, the system diagram, is the root of the OPD tree and the highest level of abstraction. It depicts the main system’s function—a combination of a main process and one or more objects—which the process transforms. SD also includes the beneficiary—a person or a group of people—and the benefit providing attribute—the beneficiary’s attribute whose change from its input state to the output state provides the benefit that the system delivers. The top-level diagram, SD, is refined into the first detail level, SD1, which is a more detailed view of SD, elaborating on the combination of the structural, behavioral, and functional aspects of the system. The refinement can continue similarly to further levels of detail, but deeper levels beyond SD1 were not used in this research. The OPM modeling process enables systematic thinking as it is conducive to a top-down approach, in which one has to first model the beneficiary, purpose, and benefit of the system as achieved by its function, which, in turn, is a combination of a process and the operands it transforms. Figure 2 depicts an OPM model of the Chocolate Producing process. The top level, called the system diagram (SD), appears on the left of Figure 2, and the first level of detail (SD1) appears on the right, in which the process is in-zoomed. SD1 exposes its three inner subprocesses and interim object (Cocoa Beans Batch).

As noted, OPM is bimodal: Any OPM model consists of two modalities, which express the same set of model facts: (1) The graphical modality–the OPD set, which is the hierarchically organized set of one or more Object Process Diagrams (OPDs) and, (2) The textual part–the OPL spec: A collection of sentences in Object Process Language (OPL)—a subset of English (or any other natural language). OPL is the counterpart textual representation of the OPD set. Each OPD construct—two or more things connected by one or more links—is reflected textually in one or more OPL sentences. This bimodal representation caters to the dual channel assumption [35,36].

1.3. Assessment of System Thinking

Science and engineering education have had distinct assessment tools for systems thinking, each emphasizing the relevant aspects and insights of its domain. The trans-disciplinary nature of modern systems and their rising complexity have been nurturing the need for a unified systems thinking assessment [26]. Sustainability and food nutritional values are topics with scientific foundations. Sustainability has global societal consequences to the future of the planet and its inhabitants, while food nutritional values impact the health of individuals and communities. Modern food production is based on agricultural and engineering principles and knowledge. The level of systems thinking for understanding of systems in these domains needs to be assessed holistically, clearly pointing to the need for a unified method and approach. Creating a common language for systems thinking across science and engineering disciplines is important [8], and it requires a high level of scientific understanding.

In the context of the natural sciences, systems thinking can be used to describe and analyze natural phenomena, events or circumstances—be they physical, chemical, biological or some combination of the three—as if they were systems [25,37,38,39]. Since OPM integrates the system’s structure, function, and behavior in one kind of diagram, and it is bimodal, it is simple enough for teaching and learning of system concepts, and for these reasons it is also suitable for assessing STEM students’ systems thinking.

1.4. Science Understanding Levels

Understanding complex, wide-scale processes requires more than acquisition of facts and memorizing terms related to these processes. Science, and specifically scientific understanding, is not exceptional in this context [40,41]. Food production and awareness of food nutritional value are issues of major concern. Learning and understanding these topics are crucial not only for food engineers and environmental professionals but for the community at large [11]. Educators should therefore teach food-related issues and systems thinking since both are of concern to society. Systems thinking, modeling, and scientific understanding skills are taught to prospective teachers and by them to their future students. These skills can strengthen the society’s control and involvement in production processes, hence contributing to sustainability. To achieve this goal, STEM teachers should be able to understand and address complex, interdisciplinary, real-world problems, such as assessing food quality and understanding complex food value chains, including their production and distribution, and their environmental effects. Equipped with knowledge and understanding of these topics, the teachers can foster their students’ systems thinking and provide them with a wide perspective of scientific topics that develop their scientific understanding.

Systems thinking in science and engineering is a dual skill: In the context of science, it entails the ability to accurately and methodically describe natural phenomena and generalize them into laws of nature. In the context of technological, engineered systems, it can be considered as holistic understanding of the system’s function, structure, and behavior, and how the latter two interact to deliver the former. With this in mind, we have combined this dual science and engineering skill in a graduate course, which was attended by graduate students from various STEM backgrounds, who are prospective teachers. The objective of this research is to evaluate the effect of educating STEM professionals, with focus on STEM educators, to apply systems thinking to food-related systems by teaching them to create and assess conceptual models with OPM.

The research questions are as follows:

(1) How can a complex food production process be modeled and evaluated?

(2) What is the effect, if any, of an OPM-based conceptual modeling of food-related systems course on graduate students’ systems thinking, modeling, and scientific understanding skills?

(3) What are the challenges and contributions that participants report during the learning processes in the course?

The rest of the paper is organized as follows. In the Material and Methods section, we introduce the EIT Food project, the research settings, population, and phases. In the Results section, findings are presented according to the research phases. The findings are discussed, and conclusions are drawn in the Discussion (Section 4) and Conclusions (Section 5).

2. Materials and Methods

This research is performed as part of an EIT Food project TRACOD—Model-Based Tracking of Cod and Other Fish Value Chain for Consumer Confidence Boosting and Food Engineers Education. EIT Food is Europe’s leading food innovation initiative, working to make the food system healthier and more trusted. The mission of EIT Food is building an inclusive and innovative community, where the consumer is actively involved [42].

To address the research questions mentioned above, the research was conducted in two phases. In Phase I, to answer the first questions, conceptual modeling experts collaborated with food professionals and modeled a food production system. Based on this model, a modeling assessment rubric was adapted and validated, as detailed in the Data Analysis section. In Phase II, relating to research questions 2 and 3, STEM students who participated in a graduate course submitted learning products. These were analyzed to determine the students’ systems thinking and modeling skills, as well as scientific understanding and their feedback on the learning process.

2.1. Research Setting—The Context of the Study and the Assignments

Global food production processes start from the region in which the raw materials, such as coffee, chocolate beans, and codfish, are collected processed, packed, and shipped worldwide. Such trans-disciplinary processes, as modeled in Phase I, are taught in the course titled Assessment of Educational Projects at the Faculty of Education in Science and technology at the Technion, Israel Institute of Technology. The course gives rise to awareness of authentic problems, which relate to sustainability, economical, engineering, and societal aspects.

2.2. Participants

The research participants in each one of the phases are presented in

Table 2. Research participants.

Phase	N	Description
I	6	Experts in conceptual modeling, food, and STEM education
II	15	STEM graduate students who are prospective teachers

.

In Phase I, the six experts included two conceptual modeling experts, including one of the authors, three food professionals, and a STEM education expert who is also one of the authors. In Phase II, the participants were 15 graduate students, 11 females and four males, with diverse STEM backgrounds, who took part in the course. Three of the students had prior experience with OPM modeling while the other 12 were novice. During the course, participants were introduced to OPM while learning about food nutritional values and production systems, such as the codfish supply chain and the chocolate production process.

2.3. Research Tools

The research was conducted in two phases, with a variety of research tools to answer the three research questions.

Table 3. Research plan.

Phase	Tool	Model Generated or Data Collected
I	OPM modeling and food engineering focus group	OPM models of chocolate and codfish food production and distribution systems
II	Four assignment sets, one for each learning unit	participants’ systems thinking skill
	A final assignment including: OPM models textual and visual design	participants’ modeling skill participants’ scientific understanding
	Feedback questionnaire	Students’ feedback

A detailed description of each of the research tool in each phase follows.

presents the research plan, including tools and data collected.

2.3.1. Phase I

Phase I of the research included OPM modeling and food engineers focus group, in which the model of the codfish supply chain was gradually constructed. The model was created in several iterations and discussions as suggested by other engineering educators and experts [43,44]. Our model is based on input of the food engineering experts and their responses to questions asked by the modeling experts. This model is presented in the Results section.

The model was created using OPCloud [34,45]—a collaborative, cloud-based software environment for conceptual and computational modeling in OPM. The codfish ’farm to fork’ model was verified by the food professionals who were actively involved in the modeling. The iterations were necessary to build a complex model depicting several layers of understanding.

2.3.2. Phase II

In Phase II, we assessed participants’ deliverables using the three different research tools described in Table 3: (1) The course assignments were used to assess the participants’ systems thinking, (2) The participants’ final assignment included two components: Models, which were used for assessing their modeling skill, and textual with visual design, which were used for assessing their scientific understanding, and (3) feedback questionnaires, from which we gleaned participants’ perceived challenges and views on their learning process. In what follows, we describe the three tools used in Phase II.

Course Assignments

The assignments comprised of four learning units which included video clips, provided as Supplementary Materials. Two examples are: 111B-Model-Based Systems Engineering and Object-Process Methodology [46] and 311-Refinement and why we need it [47]. The difficulty level of the learning units increased incrementally to enable gradual learning of modeling concepts and to enhance scientific literacy and systems thinking. Each learning unit has three parts.

The first part of each learning unit is instructional, and includes tutorial videos, food production descriptions, specifically the chocolate making processes, and questions regarding general modeling and specific application of the production process using the learned modeling concepts. Questions at different understanding levels are included throughout the learning unit. About half of the questions are closed-ended. The open-ended questions require participant to provide explanations and construct small conceptual models of increasing systems thinking levels.

The second part includes a quiz, in which transfer of knowledge from the chocolate production process to a different food domain—the codfish supply chain—is required. The objective here is to extend students’ systems thinking and modeling skills.

The third part includes a feedback questionnaire with closed- and open-ended items regarding the participants’ perception about the learning process and their challenges and contributions.

Final Assignment

At the end of the course, each student submitted a final assignment that served for assessing scientific understanding. In this assignment, the participants were asked to design for their future students online OPM modeling assignments that are related to a specific food supply or production process of their choice, combined with one or more sustainability issues.

Feedback Questionnaire

Students’ feedback questionnaires included closed- and open-ended questions formulated to understand students’ perceptions about the challenges and contributions regarding the learning process of modeling and food production issues.

2.4. Data Analysis

In Phase I, we adapted Systems Thinking Assessment Rubric (STAR), developed by Lavi and colleagues [39] for evaluating engineering students, into STAR*, a simplified version of STAR for evaluating prospective STEM teachers. The three main system aspects evaluated in STAR are function, structure, and behavior. The function aspect describes how the beneficiaries of human-made systems benefit from the operation of those systems, while structure and behavior refer to the static and dynamic elements of any given system, respectively. STAR* is adapted to our research participants, STEM prospective teachers, whose OPM conceptual modeling knowledge level is low-to-medium. The main differences between STAR and STAR* are (1) using two slightly different criteria for two levels of OPM diagrams—the top abstract level, SD, and the first detail level, SD1, (2) omission of Attribute 8 in STAR, “Temporary objects and decision nodes”, which is beyond the modeling level of our research population, and (3) addition of examining OPL sentences (the textual modality of OPM), whose soundness and correctness provides indication of participants’ metacognitive reflection while constructing the model OPD.

Table 4. Systems Thinking Assessment Rubric* (STAR*).

Aspect	System Thinking Attribute	Expected Implementation of the Attribute	Scoring (0–2 or 0–1)
Function	A1-Intended Purpose	Beneficiary and benefit are linked with the correct link (Exhibition-Characterization), and both phrased correctly according to the context.	Both beneficiary and benefit are absent: Zero points. Only one of them (beneficiary/benefit) is used or both of them without a correct link: 1 point. Both beneficiary and benefit are used, with a correct link: 2 points.
	A2-Main Function	Exactly one systemic main process, which transforms at least one object with the relevant transformation link, all of them phrased correctly according to the context. For SD1, At least three sub-processes, with the same specification as detailed above.	No main process, or a main process which is totally irrelevant to the context: 0 points. Main process is correct but transforms no object(s), or is wrongly phrased: 1 point. Main process transforms at least one object, phrased and linked correctly: 2 points.
Structure	A4-Main Object	One main object (or more, depends on the context). The main object(s) must be defined and phrased according to the subtext.	No main object, or a main object which is totally irrelevant to the context: 0 points. Relevant main object(s): 1 point.
	A5-Structural relations	Correct use of at least one type of link between objects and/or between processes.	Less than one link: 0 points. One link or more: 1 point.
Behavior	A3-Complexity level	Two diagrams are included: one Top-level and one SD1 of any kind.	Only an SD without an SD1 or vice versa: 0 points. Two models are included: SD + SD1: 1 point.
	A6-Procedural relations	Correct use of at least two types of links between objects and processes.	Less than two links: 0 points. Two links or more: 1 point.
OPL	A9-OPL main process functional sentences	The presence of OPL sentence(s) depicting the link between the beneficiary and its relevant benefit(s), using an exhibition-characterization link, e.g., “Producer exhibits profit”.	At least one of the following is missing or wrongly phrased (in case of the objects): Beneficiary, benefit, or exhibition-characterization link: 0 points. Both beneficiary and benefit are present and phrased correctly, and linked with an exhibition-characterization link: 1 point.
	A10- OPL main process procedural sentences	The beneficiary is linked with an agent link. e.g., “Winemaker handles Harvesting.” The operand is linked with a correct result/effect link, e.g., “Bread Making yields Bread Loaf.”, or: “Harvesting changes status of Grape from on tree to picked.” An instrument or consumption link, used correctly, e.g., “Bread Making requires Mixing Machine.”, or: “Bread Making consumes Flour, Yeast and Water.”	No more than one of the three sentences is present: 0 points. Two or three sentences (out of three) are present: 1 point.

presents detailed explanations for using STAR* to score the modeling skill.

The main difference between the criterion for SD and SD1 is that for SD the attribute “Complexity level” is used (1 point for including both SD and SD1 in the model), while for SD1 “Procedural sequence”, is used, as it is relevant only for SD1, where in most cases there is a sequence of subprocesses within the main process from SD that is in-zoomed in SD1.

In Phase II, we analyzed the graduate students’ learning products. The participants’ systems thinking and the modeling levels were determined by their performance in the four learning unit assignments, which were validated by the STEM education experts.

The final assignment was analyzed by two different rubrics: The OPM model in the final assignment was assessed using STAR*, while the Scientific Understanding Rubric (SUR), presented in

Table 5. The Scientific Understanding Rubric (SUR).

Criterion	Scoring Scheme
C1 Digital and visual design	1 point is added for each of the following: Graphical interface with proper division of the assignment form into sections and use of colors Use of images Use of illustrations and models Use of information video clips or animations
C2 Food processing and sustainability (F&S) context	0—The questions are general with no or partial connection to sustainability 1—The questions include one aspect of food processing or one aspect of sustainability 2—The questions integrate one aspect of food processing and one aspect of sustainability or integration of at least two aspects of food processing or sustainability 3—The questions integrate at least two aspects of food processing and one aspect of sustainability or complex scientific analysis of the process 4—The questions integrate several issues of food processing and sustainability
C3 Variety	0—all the activities are of the same type 1—a variety of activities, e.g., completing sentences, answering questions, building models, and completing models 2—variety of activities and a special assignment that did not appear in previous exercises

, was used to assess the scientific understanding. The modeling skill was evaluated using STAR*. The scores rated by the three experts based on STAR* received 84% interrater reliability [48,49].

To evaluate the participants’ scientific understanding, we analyzed the mini-projects participants composed using the SUR, which consists of three categories (see Table 5): (C1) digital and visual design, (C2) context related to food processing and sustainability issues [50,51,52], and (C3) a variety of activities. Each SUR criterion can score up to 2 or 4 points, yielding a maximum score of 10.

The SUR rubric received 90% interrater reliability [48,49] between three experts. To understand students’ challenges and contributions in relations to systems thinking, modeling, and food processes, we analyzed their feedbacks, which included students’ ranking (1–5) of the learning units’ aspects in a Likert scale and identified themes in the open-ended questions [53].

2.5. Ethics

All subjects gave their informed consent for inclusion prior to their participation in the study. This research is part of the EIT Food project TRACOD, titled Model-Based Tracking of Cod and Other Fish Value Chain for Consumer Confidence Boosting and Food Engineers Education. This project was reviewed and approved by the Behavioral Sciences Research Ethics Committee of the Technion—Israel Institute of Technology. Approval number 2020–165 was received on 16 July 2020.

3. Results

We present the results according to the research phases.

3.1. Phase I—Case Study: Experts’ Modeling of Food Production Processes

Phase I, related to the first research question, resulted in a detailed model of the codfish supply chain. At the point of sale, fresh fish can be measured by the vendor or even the consumer, with immediate response presented and printed on a sticky label that accompanies the fish fillet all the way to the consumer’s kitchen. In this project, we used fish articles as food examples, of which relevant data including food quality and authenticity (FQA; e.g., species, nutritional value, freshness level) along the supply chain is exposed. We empower the consumer to ensure that the specific fish she or he is buying is authentic, what its various nutritional values are, and additional details of interest, such as the place and time of catch and possibly even details of the fishing vessel and captain. The comparison of the freshness value at the point of sale to that at the point of packaging will inform the customer about the level of degradation in quality, which should be minimized and can be a new factor in determining the price of the fish, in addition to the species and the weight. Following are the diagrams presenting the model. Figure 3 presents the SD level—model and OPL.

The main process in the model in Figure 3 is Cod Tracking, which changes the Nutrition Value Knowledge of the Household Member Group from low to high. This is the benefit of the Cod Tracking process. Links in OPM are of several kinds. For example, in Figure 3, the double arrowhead link between the process Cod Tracking and the object Supply Chain Group is an effect link, implying that the process affects the object. The “white lollipop” between TRACOD System and Cod Tracking is an instrument link, implying that TRACOD System is required for Cod Tracking. These facts are also expressed textually in the same figure.

This model was analyzed according to STAR*.

Table 6. STAR* analysis of codfish supply chain model—SD level.

Attribute	Scoring	Examples in the Model
A1-Intended Purpose	2 points. The beneficiary–Household Member Group, exhibits the benefit- Nutrition Value Knowledge at state high.
A2-Main Function	2 points. Cod Tracking transforms (in this case-changes) Nutrition Value Knowledge.
A3-Complexity level	1 point. Two diagrams are included.	NA
A4-Main Object	1 point. The object Nutrition Value Knowledge is phrased in a meaningful way.
A5-Structural relations	1 point. Two different links: exhibition-characterization and tagged link.
A6-Procedural relations	1 point. Two different links: Instrument link and effect link.
A9-OPL main process functional sentences	1 point.
A10-OPL main process procedural sentences	1 point.
	Total Scoring: 10/10 points

presents the scoring of the SD level according to this rubric.

Figure 4 presents SD1 level of the codfish supply chain model. This is the first detail level of the model in Figure 3. It presents the sub-processes of the Cod Tracking process and the objects that are involved in this process.

The OPL and analysis of the model in Figure 4 (based on the STAR* for assessing SD1) appear in Appendix A. The modeling stage and focus group interactions raised questions with which the food professionals have never dealt with before, which caused rethinking the process and improving it. The process of creating the models enables each food professional group member to express the functions and roles of the different parts within the process.

3.2. Phase II—Assessing Students Systems Thinking, Modeling and Scientific Understanding

Phase II, which is related to research questions 2 and 3, included three steps: (1) Assessing students’ systems thinking, modeling and scientific understanding, and statistical analysis; (2) detailed analysis of several examples; and (3) feedback analysis.

We classified the participants according to their systems thinking, modeling, and scientific understanding levels. Participants whose systems thinking and modeling levels were at least 8 were classified as high while others were classified as low. Similarly, participants whose scientific understanding level scored at least 8 were classified as high while others were classified as low. Figure 5 presents the distribution of students according to their systems thinking and modeling skill levels and their scientific understanding level in four groups.

In order to test the differences between students with prior experience in OPM modeling and novice students, we ran a Man–Whitney test. The experienced students had higher modeling (Median = 9) and higher systems thinking (Median = 9) skills than the novice students’ modeling (Median = 5.5) and systems thinking (Median = 6) skills. The Mann–Whitney test indicated that these differences were statistically significant. For modeling: U_{(experience = 3, novice = 12}) = 5, z = −1.959, p = 0.05. For systems thinking, U_{(experience = 3, novice = 12)} = 3.5, z = −2.11, p < 0.05. No significant correlations were found between the levels of systems thinking and modeling skills and scientific understanding.

Following are in-depth analysis examples of two participants. The first example, presented in Figure 6, is of a participant who is experienced in using OPCloud for OPM-based modeling, while the second is of a novice modeling participant.

The modeling score of this participant was calculated based on STAR* (Table 4). The explanations for the detailed scoring are presented in

Table 7. Explanation of participant #722122205 modeling score.

Attribute	Explanation of Scoring	Score
A1-Intended Purpose	Supplier group, Employee, and Market Group are possible beneficiaries. Market Group exhibits Pasta Product at state packaged.	2/2
A2-Main Function	Pasta Manufacturing and Pasta Product are connected with result link between the process and the operand.Unpackaged/packaged states in the object Pasta Product are not necessarily relevant to this diagram, considering the whole manufacturing process being described, including the packaging sub-process. Could have been, instead, “raw materials” state as initial state and “packaged product” state as final state.	1/2
A3-Complexity level	An SD1 diagram is included	1/1
A4-Main Object	Paste Product	1/1
A5-Structural relations	Two types: Exhibition-characterization and Aggregation-Participation links.	1/1
A6-Procedural relations	Three types: Result/consumption, instrument and agent links	1/1
A9-OPL main process functional sentences	“Market Group exhibits Pasta Product at state packaged”	1/1
A10-OPL main process procedural sentences	“Pasta Manufacturing yields…and Pasta Product at state packaged” “Pasta Manufacturing requires Electrical Energy and Quality Control”
Total		9/10

below. The model itself appear in Figure 6, and appears in full scale in Appendix B.

The scientific understanding score was calculated based on the rubric presented in the Materials and Methods section, Table 5.

The experienced student’s assignment presented in Figure 6 was evaluated based on the following evaluation:

• Digital and Visual Design (4/4)—The assignment contains designed interface including colors, models, image, and video clips.
• Food processing and sustainability context (4/4)—The questions combine several issues of food processes and sustainability.
• Variety (1/2)—There is a variety of activities but no special assignment.

In summary, the total score is 9/10.

The second example, presented in Figure 7, is of a student who is novice in modeling.

The modeling score of this participant was calculated based on the rubric presented in Section 2, Table 4. Detailed explanation for the modeling score is presented in

Table 8. Explanation of participant # 112122201 modeling score.

Attribute	Explanation of Scoring	Score
A1-Intended Purpose	Only a benefit (Baked Bread) without its context: beneficiary and a suitable link.	1/2
A2-Main Function	In Bread Making Process, Wheat Dough is consumed and Bread is produced. This main process is linked wrongly to a series of asynchronous sub-processes. “Bread Making process” should have been phrased “Bread Making”, but no points were taken off for that matter.	1/2
A3-Complexity level	An SD1 diagram is included	1/1
A4-Main Object	Wheat Dough	1/1
A5-Structural relations	Aggregation-Participation link appears twice, with one correct use. (the minimal number required)	1/1
A6-Procedural relations	Only one type: result/consumption	0/1
A9-OPL main process functional sentences	No such sentence	0/1
A10-OPL main process procedural sentences	Only one of the three sentences—“Bread Making Process consumes Wheat Dough”	0/1
Total		5/10

below. The model itself appear in Figure 7.

The scientific understanding score was calculated based on the rubric presented in Section 2, Table 5. According to Figure 7 the evaluation of this assignment is based on the following reasoning:

• Digital and Visual Design (4/4)—contains designed interface including colors, models, image, and video clips
• Food processing and sustainability context (4/4)—the questions include assimilation of a few issues of both food processes and sustainability. In addition, the assignment integrates multiple complex scientific analysis of the chemical processes
• Variety (2/2)—a variety of activities and a special assignment that did not appear in previous exercises

In summary, the total score is 10/10.

We analyzed the participants’ feedback evaluate the challenges and benefits students faced during their learning process. In the open-ended questions, participants raised several issues, including cognitive, technical, and social issues, as well as challenges with the systems thinking and modeling aspects. For example, one student stated: “In the practical aspect of replying to the codfish assignment, I had trouble building a model that will include the components in a meaningful way.” [222122201]. Another student stated: “The first difficulty was thinking about asynchronous processes in the food production.” [72212205].

Students’ reflections on the final assignment raised additional aspects, including the following.

• The need for iterations to clarify issues: “We had an iterative discussion until we found the optimal solution for the evaluation tool” [212122201].
• Acquiring a broad viewpoint: “The process made me think in a broader, systematic viewpoint and to relate to the links between the different components” [112122201].
• Applying reverse engineering: “The challenge in creating an OPM model made me use reverse engineering to articulate the assignment” [412122201].

Figure 8 presents analysis of the students’ feedback regarding contributions of the course to the understanding of modeling and systems thinking as well as understanding of food processes, by the learning units, based on their Likert scale statements. The graphs show that students’ perceptions of the contribution of the course to their systems thinking and modeling is consistently higher than to understanding food processes, with both showing a similar trend.

4. Discussion

Guaranteeing food quality and authenticity is a challenging task with common interest for all students, as they are also consumers of food, making this topic suitable for context-based learning. At these times, while the COVID19 pandemic exerts impact on production and transportation of food worldwide, citizen understanding of food production process is of particular importance.

The first research question was: How can a complex food production process be modeled and evaluated? Modeling these processes requires system thinking which is developed while learning conceptual modeling with Object Process Methodology (OPM) using OPCloud. A trans-disciplinary team is needed to build such a model. Once built, the model can serve as part of learning materials for modeling context-based narratives focusing on environmental and food-related issues [7,44,52].

The second research question was: What is the effect, if any, of an OPM-based conceptual modeling of food-related systems course on graduate students’ systems thinking, modeling, and scientific understanding skills? We found a significant difference between students with modeling experience and those who were novices. As found before, gradual long-term processes are needed in order to develop these skills [26,27]. This indicates that practicing the modeling skill might improve systems thinking. More research is needed to determine the long-term effect of the link between modeling and systems thinking skills. We recommend using these skills in studies of complex processes and sustainability issues, in line with some of the NGSS recommendations [2].

The third research question was: What are the challenges and contributions that participants report during the learning processes? The students’ feedback indicated that understanding food processes and modeling have similar trends. These similarities provides an indication that food and sustainability can be effectively used to develop systems thinking and modeling skills as they enable learning in context which contributes to the learning process [50,51,52].

Overall, we found that the course contributed to the students, who are prospective teachers, in both the systems thinking and modeling skill perspectives, as well as to their scientific understanding and awareness to nutritional values and sustainability issues.

5. Conclusions

In this research we evaluated the effect of four food-related learning units, which were based on a part of an online model-based systems engineering course with OPM [1], on graduate students’ systems thinking and modeling skills, as well as on their understanding of STEM.

A limitation of the study is the small number of participants. Future research should include a larger sample. We suggest replicating this study with larger groups and continue validating the two rubrics used in this research.

This research has methodological, practical, and theoretical contributions. The methodological contributions are the two rubrics we adapted. The first, STAR*, enables assessing the modeling skill of prospective teachers who have low-to-medium modeling and system thinking skills in many cases. The second, SUR, enables assessing teachers’ scientific understating based on their self-designed online assignments. The practical contributions and findings application include the development of four learning units with questions at various difficulty levels, which exposes teachers and students to conceptual modeling using the OPCloud modeling tool. The learning materials and rubrics can assist STEM students and professionals in developing their systems thinking and modeling skills, as well as their scientific understanding. The theoretical contribution of this study is the innovative combination of food production processes and OPM-based conceptual modeling for developing STEM teachers’ system thinking skill, their modeling skill, and their scientific understanding of food processes and sustainability issues.