Integration of SysML and Virtual Reality Environment: A Ground Based Telescope System Example

Abstract

In recent years, Model Based Systems Engineering (MBSE) has continued to develop as a standard for designing, managing, and maintaining increasingly complex systems. Unlike the document centric approach, MBSE puts the model at the heart of system design. Among the various MBSE language development efforts, “Systems Modeling Language (SysML)”, is the most anticipated and broadly utilized in the research and in industrial practice. SysML originated from Unified Modeling Language (UML) and follows the Object-Oriented Systems Engineering Method (OOSEM). SysML diagrams help users create various systems engineering artifacts, including requirements, use cases, operational concepts, system architecture, system behaviors, and parametric analyses of a system model. In the early days of implementation, MBSE languages, including SysML, typically relied on static viewpoints and limited simulation support to depict and analyze a system model. Due the continuous improvement efforts and new implementation approaches by researchers and organizations, SysML has advanced vastly to encompass dynamic viewpoints, in-situ simulation and enable integration with external modeling and simulation (M&S) tools. Virtual Reality (VR) has emerged as a user interactive and immersive visualization technology and can depict reality in a virtual environment at different levels of fidelity. VR can play a crucial role in developing dynamic and interactive viewpoints to improve the MBSE approach. In this research paper, the authors developed and implemented a methodology for integrating SysML and VR, enabling tools to achieve three dimensional viewpoints, an immersive user experience and early design evaluations of the system of interest (SOI). The key components of the methodology being followed in this research paper are the SysML, a VR environment, extracted data and scripting languages. The authors initially developed a SysML for a ground-based telescope system following the four pillars of SysML: Structure, Requirements, Behavior and Parametrics. The SysML diagram components are exported from the model using the velocity template language and then fed into a virtual reality game engine. Then, the SysML diagrams are visualized in the VR environment to enable better comprehension and interaction with users and Digital Twin (DT) technologies. In addition, a VR simulation scenario of space objects is generated based on the input from the SysML, and the simulation result is sent back from the VR tool into the model with the aid of parametric diagram simulation. Hence, by utilizing the developed SysML-VR integration methodology, VR environment scenarios are successfully integrated with the SysML. Finally, the research paper mentions a few limitations of the current implementation and proposes future improvements.

1. Introduction

According to the International Council on Systems Engineering (INCOSE), “Systems Engineering is a transdisciplinary and integrative approach to enable the successful realization, use, and retirement of engineered systems, using systems principles and concepts, and scientific, technological, and management methods” [1]. Model Based Systems Engineering (MBSE) focuses on formalized applications of modeling to support systems engineering artifacts development from the conceptual design phase throughout the end of the system of interest (SOI) lifecycle [2]. In his 1993 book, “Model Based Systems Engineering”, Dr. Wayne Wymore first introduced the term “MBSE” [3]. Systems engineers are living in a generation wherein modern systems are growing in complexity [4,5]. MBSE is the proposed solution by the researchers to cope with system complexity in a variety of SOI [6,7]. Various MBSE methodologies have been developed in recent years. The most notable ones are Object Oriented Systems Engineering Method (OOSEM), Object Process Methodology (OPM), State Analysis, IBM Harmony and Arcadia-Capella [8,9]. Among these, OOSEM-based SysML has better tool support than any other MBSE methodologies available [10,11]. In fact, SysML has emerged as the de facto standard system modeling language for MBSE [12,13]. According to Friedenthal, Moore, and Steiner, “SysML is a general-purpose graphical modeling language that supports the analysis, specification, design, verification, and validation for complex systems” [14]. SysML is owned and published by the Object Management Group, Inc. (OMG) [15]. SysML is a derivation of UML, developed in the 1990s as a general-purpose language for software engineering [16]. SysML consists of nine diagram types, namely Package Diagram, Requirements Diagram, Block Definition Diagram (BDD), Internal Block Diagram (IBD), Use Case Diagram, Activity Diagram, State Machine Diagram, Sequence Diagram, and Parametric Diagram (see Figure 1) [15].

VR is one of the major technologies with the utmost anticipated potential for growth, especially in the fields of engineering, education, gaming, cinematography and healthcare [18,19,20,21,22,23,24,25]. For example, different engineering fields leveraged VR technologies to accomplish various tasks, such as VR based Civil Engineering training, Industry 4.0 implementation, and Aerospace training [26,27,28,29,30,31,32]. Coates et al. defined VR as an electronic simulation of environments with head mounted display and wired outfit, which allows the end user to interact with lifelike 3D environments [33]. A typical VR system consists of the following key components: VR implementation software, Head Mounted Display (HMD), base station, tracking sensors, feedback devices, on board/external computers and users [34,35,36].

Though SysML application has been widespread in recent years, it has not been fully adopted by organizations. In addition, stakeholders may not have knowledge of SysML [37]. External product suppliers do not have SysML software knowledge and often ask for Excel/Word documents for data exchanges. Hence, system engineers still need to generate documents from the SysML system model to present the work in front of non-technical audiences. Therefore, there is a need for automatic data extraction from SysML models to facilitate model demonstration to technical and non-technical stakeholders alike.

To date, SysML models have mostly been used to define requirements, use cases, system architectures, system behaviors, and task sequences through static viewpoints and elements. In recent years, researchers have incorporated simulation analysis within SysML models to enable system requirements verification and design evaluation [38,39]. However, very few studies have focused on the integration of VR tools with SysML models. Recent studies indicate that VR-supported immersive design review allows participants to identify more issues/faults in design, improve collaborative engagement, focus and presence as compared to non-immersive methods [40,41]. Moreover, VR environments are beneficial for visualizing complex structures and provide higher levels of understanding and knowledge retention than the conventional approaches [42,43,44]. Recent case studies/evaluations by the researchers showed that VR environment translation of visual formalisms (similar to SysML) resulted in enhanced collaboration, learning and understanding among the users/stakeholders. For example, multiple studies have confirmed that 3D VR environments significantly increase the processes of knowledge retention and the collaborative process modeling experience of Business Process Model and Notation (BPMN) [45,46,47]. Similarly, modeling 3D UML diagrams in a virtual environment resulted in faster model comprehension and a more enjoyable modeling experience than 2D diagrams [48,49]. Based on the above study results, it can be elicited that visualization of 2D SysML diagrams in a 3D VR environment can improve model understanding, especially of complex systems, and facilitate a better collaborative environment among users and stakeholders.

VR technology can help visualize the system architecture, facilitate virtual storytelling, verify system requirements and evaluate product/service performance early in the lifecycle of a SOI. For example, a system engineer can evaluate system architecture (e.g., Telescope Mount System) in a 3D virtual environment and, based on their understanding of the system, modify/refine the architecture early in the design phase. In addition, external stakeholders with minimal technical knowledge can be immersed into a 3D virtual environment and experience use case scenarios of proposed designs. As a result, there is a need for interoperability between 3D VR models and SysML models to enable data exchange and review to systems engineering artifacts such as use cases, requirements, operational concepts, system architecture, etc.

In addition, use of Digital Twin (DT) technologies in systems engineering projects have been increasing in recent years. DT as a concept was first introduced by Michael Grieves as a virtual representation of an actual component that can be used to emulate the same behavior as that of its real world counterpart [50]. However, the term “Digital Twin” first appeared in National Aeronautics and Space Administration’s (NASA) draft technological roadmap published in 2010 [51]. Over the last two decades, literature and research in the field has evolved to produce a wide variety of DT definitions and methodologies [52]. To simplify the ambiguity, a DT can be defined as a digital or virtual model of a real-world object (system, component, part, process, human, etc.) representative of the exact or future state of its physical twin via real time data exchange as well as keeping records of historical data [53]. Virtual models can be developed using a variety of methods, including VR, Augmented Reality (AR), Mixed Reality (MR), dashboard technologies and 2D simulations, etc. [54,55,56,57,58,59]. For example, developers can fabricate an entire machine in a digital environment and test their high-level control mechanism in a VR environment; manufacturing engineers can utilize dashboards to analyze the production process in real time and make necessary adjustments. Data exchanges between real and virtual counterparts of a DT infrastructure are achieved through internal/external sensors connected to physical systems and communication networks [58]. Recently, few DT frameworks have emerged where VR environments co-exist with other DT infrastructure and support validation of the digital/virtual configurations before the physical twin implementation [60,61]. The integration of MBSE tools and DT infrastructure can present several opportunities for systems engineers to test and visualize large models [62,63]. Both the physical and virtual models can be integrated with a shared MBSE repository (e.g., SysML model), which will act as a communication hub between DT infrastructure and systems engineering activities [63]. Moreover, MBSE efforts can introduce early DT development in a program’s lifecycle [64]. Therefore, integration of SysML models with VR tools is one of the key steps in establishing synergy between DT and MBSE.

Hence, the authors have developed a methodology to integrate a SysML model with VR environments for a ground-based telescope system. The authors have chosen the above-mentioned system since telescope systems heavily rely on digital data products and generate virtual scenarios to analyze different aspects of scientific findings. The key contributions of this research are summarized as follows:

• A SysML model for a ground-based telescope system is developed, which will act as a reference model for the telescope MBSE practitioners.
• A detailed methodology to integrate SysML models with VR environments.
• Data interchange between SysML parametric diagram simulation and the VR scenario, which will enable DT and MBSE co-existence.
• SysML model diagram visualization in the interactive VR tool, which will enhance the systems modeling experience, model understanding and collaboration among stakeholders.

2. Literature Review

The authors believe there are limited studies available in MBSE research which emphasize integration of VR environments with SysML tools. In his thesis paper, Kande proposed an integration methodology of Virtual Engineering (VE) suite with the SysML model, providing a graphical interface to demonstrate the system of interest and operations [65]. He created a fermenter analysis model using Computer Aided Design (CAD) data and analyzed the effects of changes in design parameters on system performance. To enable effective communication of systems engineering artifacts to the non-engineering disciplines, Madni mapped systems engineering (SE) artifacts modeled in SysML to virtual worlds, within which different storylines can unfold [66]. In addition, he combined MBSE + frameworks (MBSE and storytelling in VR) and Experimental Design Language (EDL) to enable early participation and collaboration of the stakeholders in the system design [67]. Abidi et al. proposed an interactive VR simulation to encompass real time simulation of production flows in a lean manufacturing industry through communication between ARENA and Virtual Environment by utilizing SysML based transformation and Real Time Infrastructure (RTI) [68]. Mahboob et al. proposed a concept for a user- and task-centric model for product development in the VR environment. In that research proposal, SysML was used to separate isolated model components into products, actors and behaviors of a specific use case, and then those model components were combined to build the use case-specific scenario in VR [69]. Sanvordenker visualized a self-driving truck’s SysML model in a VR environment by extracting SysML model diagrams, representing them in a browser, and finally embedding the web browser inside a Unity scene through a custom plugin [70]. Oberhauser demonstrated a solution concept for visualizing and interacting with SysML models as stacked hyperplanes in a VR environment [71].

However, in the above research there are some limitations. Namely, requiring extensive manual effort to set up each of the simulation components and frequent changes in the interface files (e.g., dynamic link library) to enable seamless data exchange. Furthermore, the VR diagrams used in these studies were more complex (e.g., stacked hyperplanes) instead of being easily comprehendible and their methodologies involved far-reaching conversions of models (XMI conversion, MASCARET, etc.). Those complex design techniques may hinder the mass use of VR-enabled SysML due to the time and effort needed to set up those systems.

3. Methodology

The overview of this study’s methodology is summarized in Figure 2. Key components of the methodology are: SysML System Model, Virtual Environment, Scripting Languages and Extracted Data. Details of the components are described below.

3.1. SysML System Model

An SysML system model functions as the central hub for all the systems engineering artifacts. In this research paper, the authors selected eight SysML diagram types to represent system hierarchy, requirements, structure, and behavior, which are as follows: Package Diagram, Requirement Diagram, Block Definition Diagram (BDD), Internal Block Diagram (IBD), Use Case Diagram, Activity Diagram, State Machine Diagram and Parametric Diagram. A Requirement Diagram represents different types of requirements (system level, interface, etc.) and their traceability with other requirements, model elements that satisfy them, and the test cases which verify them. BDDs are primarily used to depict a system’s physical/logical architecture through the hierarchical relationship between the subsystem level components and classifiers. IBD is a diagram type which is owned by a particular block/system context and shows parts, properties, ports, interfaces and connectors within it. Use Case Diagrams define use cases and operational concepts by visualizing the relationship between the system’s key functions and its actors (human/nonhuman). An Activity Diagram is a key SysML tool to represent system, subsystem and component level scenarios, and to represent behaviors through the sequence of data/control flows between actions. State Machine Diagrams are utilized to present information about dynamic system behaviors, transitions between system states, lifetime of a block through states, event triggers and transition elements. Parametric Diagrams are a specialized IBD diagram that help to combine structural and behavior model elements with engineering, simulation, or mathematical models. Package Diagrams facilitate the organization of a complex system model through container packages and their relationships. The authors used Cameo Systems Modeler (CSM) as the SysML tool because it has Cameo Simulation toolkit incorporated [72]. Cameo Simulation toolkit and Report Wizard tools support a wide variety of scripting languages including JavaScript, Python, MATLAB, Velocity Template Language (VTL), Modelica, etc. [73].

3.2. VR Environment

The authors propose to use Unity as the VR environment tool to be integrated with CSM [72]. The VR environment can be configured by utilizing the input from the SysML system model. The VR scene can incorporate human interaction/feedback, which can be returned to the SysML model using two-way communication. Unity can read the automatically generated excel files with the aid of a scripting language to extract the SysML model elements and import them into the VR environment. The major elements of a Unity scene that one must have knowledge of to understand this research are as follows: GameObject: fundamental objects in Unity that represent characters, props and scenery. Components: added to the game objects to provide real functionality. Transform: determines the Position, Rotation, and Scale of each object in the scene Prefab: acts as a template from which one can create new object instances in the scene [74]. For example, in this research paper, the authors used Cube Game Objects to represent the blocks and requirements. So, the authors attached scripts and collider components to a single instance of Cube. Then, the authors created a prefab of the Cube so that the authors can automate the production of similar game objects in the scene.

3.3. Extracted Data

For presentation purposes and communication of model data with external tools, reusable generic scripts are developed to automate the generation of SysML model output files in various formats (Word, Excel, XML, etc.) containing model elements and diagrams/viewpoints. The extraction is done through a custom script which can be generalized/modified for other project usage. Similarly, Unity simulation results are extracted as Excel files, which are in turn imported into the SysML model for analysis.

3.4. Scripting Languages

The authors have used VTL to create the report wizard template file in the CSM to export the SysML model diagram components (see Figure 3 and Figure 4) [75]. CSM is also integrated with MATLAB to exchange simulation parameter values with the Unity engine. On the other hand, Unity installation comes with built-in C# scripting capabilities (see Figure 5). Hence, C# scripts are used to define the behavior of the game objects, import/export data values and user interaction methods (see Figure 5).

4. Ground Based Telescope System Model

The Ground Based Telescope System Model developed for this research is a large model consisting of multiple instances of the SysML diagram types. Hence, each of the diagram type examples are included here to demonstrate how SysML diagrams are used to model different system artifacts for the SOI.

Figure 6 shows the SysML package diagram of the system model, which is consists of “requirements”, “structure”, “behavior” and “parametric” packages. The requirements package consists of the requirements diagrams and tables. The structure package includes system architecture and communication interfaces. The behavior package contains use cases, operational scenarios, and system states. The parametric package includes the data exchange and integration components with the VR environment.

In this study, SysML requirement diagrams are used to represent the requirements specifications and their relationships with lower-level requirements. Relationship types between requirements and lower-level requirements and/or other related elements are “containment”, “derive”, “refine”, “satisfy”, “verify”, and “trace”. Figure 7 shows a SysML requirement diagram with a containment relationship between parent and child requirements for a telescope data collection system. Requirements can be displayed using a requirements table as well (Figure 8).

SysML BDDs are used to depict the system architecture and relationship between the components of the different subsystems of the ground-based telescope. Figure 9 and Figure 10 demonstrate how the authors have decomposed different subsystems to a lower level with the BDD. Also, users can specify the multiplicity to define the number of components necessary for a given subsystem. The authors also included a navigator to the previous diagram (see Figure 10).

Communications and data flow between different subsystems are represented by SysML IBDs. Specifically, the IBDs represent interconnection, interfaces, and ports between the parts of a block. For example, Figure 11 depicts the telescope control system data flows. Signals are sent from one port to another through connectors to depict the transfer of data between the system components.

SysML use case diagrams are a good way of visualizing operations concepts and use cases for different systems through the visualization of the interaction between actors and use cases. In Figure 12, a data analysis operational concept (OpsCon) is presented that consists of use cases, actors, and their relationships. As the actors are human in this case, they are represented by a human icon. Non-human actors are usually represented through block-like actors.

Activity diagrams are another type of behavior diagram used in this model to visualize different operational scenarios for a ground-based telescope system. Usually, an activity diagram starts with an initial node (a solid circle) to symbolize the start of an activity. The diagram connects different action types with control flows or object flows. Control flows are used to model the flow of control between actions, whereas object flows are used to model the flow of data. In Figure 13, an operational scenario on how to disable the telescope motions are depicted through an activity diagram.

SysML state machine diagrams help to model different operational states and transitions between states triggered by events/signals/values, etc. State machine diagrams can be also used to define/build verification constraints of a system to verify the system requirements [76]. In Figure 14, different states and their transition paths with signals are displayed. For instance, when an errrorIsTrue signal is triggered, the system transitions to FaultState from DisabledState or EnabledState. If there is maintenance activity required, a maintenance signal will trigger a transition to MaintenanceState from FaultState.

SysML parametric diagrams help the modeler to combine system models with engineering analysis models (e.g., performance, reliability, cost analysis, simulation). SysML activity diagrams with the aid of the opaque action can serve a similar purpose of integration with external analysis tools [77]. However, to serve large, complex system models, the using a parametric diagram is a more efficient way of trading with a high volume of data and calculations as compared to an activity diagram. In Figure 15, the authors incorporated a MATLAB function into a constraint block by dragging it into the parametric diagram. The function takes the numObjects (the number of space Objects to be created in a unity scene) variable as input and returns the xPos of a selected game object in the same unity scene by a user.

5. Integration with the VR Environment

In this research paper, a SysML system model is integrated with the VR environment in Unity using two different methods. First, the authors utilized CST in the CSM tool to simulate the parametric diagram in order to achieve data exchange between the model and the VR environment. The value properties SpaceObjects and xPosition are mapped to the numObjects and xPos variables from the MATLAB script, respectively (see Figure 15). Both the MATLAB function and Unity engine access the same excel file to read and write the data values being exchanged. As shown in Figure 16, the SpaceObjects value was set to 10 and the authors received a returned xPosition value of −2.1438 from the Unity scene. It is apparent from Figure 17 that C# script from Unity received the input from the model and generated 10 space objects in the Unity scene by instantiating the Space Objects prefab. Then, the xPosition value becomes populated in the CST console of the system model once the user selects a space object in the Unity scene by hovering the mouse over the object (see Figure 18). The value can be confirmed by comparing it to the unity console showing the xPosition value of the selected object (see Figure 19). The color of the object changes to red once the user selects a particular space object in the Unity scene (see Figure 18). Thus, two-way data exchange between the system model and the Unity environment is facilitated.

Secondly, the authors used VTL and report wizard from CSM to export the SysML diagram components and their relationships to different Excel files which are shared with Unity C# scripts. The scripts written for different diagram types can take the information from the Excel files and generate visualization of the same diagram exported in the Excel files in the Unity scene. In Figure 20, the exact same BDD as modeled in the system model shown in Figure 10 can be seen. The authors used a Cube prefab to depict the blocks and cylinder prefabs to visualize the relationship in the Unity scene. Parent blocks are shown in a different color (cyan) compared to the child blocks (see Figure 20). Users can visualize the requirements diagram utilizing the same prefabs used in a BDD, as both are hierarchical diagrams. However, for a requirements diagram visualization, the authors only show the requirement details when the user selects a particular requirement by hovering the mouse over the selection to keep the diagram less cluttered (see Figure 21). This selection of a requirement also shows how users can design user interactions with diagrams inside the VR environment. Using a similar method, users can use different types of prefabs and organization patterns to visualize the different types of SysML diagrams in the VR environment (e.g., using human prefabs to depict actors in the use case diagram). Hence, with the aid of VTL and C# scripts attached to custom prefabs, users can visualize SysML diagrams inside Unity scenes to enable improved user interaction and communication.

6. Discussion

Telescope systems depend heavily on vast amounts of data collection and scenario visualizations to analyze different science aspects of existing space objects, consider the possibility of new discoveries and run educational programs for different institutions. So, integration of VR environments with the SysML system model will enable design/analysis of those visualization systems with reduced cost and effort, as alternative architectures can be evaluated within the system model. The system model developed in SysML will contribute to the literature on the application of MBSE in ground-based telescope systems. Model based system engineers will be able to use different SysML diagrams appropriately to model similar telescope systems. The system’s integration capabilities with VR environments also facilitates future integration of DT technologies with MBSE tools. As DT’s virtual components can be designed in a VR environment, SysML models can potentially communicate with the physical twin through the digital counterpart in the VR environment. Design/analysis parameters from SysML models can be fed into the VR environment. Then, those changes in VR environment will dictate the physical twin’s behavior, as twins can be connected through data communication networks and sensors. In an alternative configuration, SysML models can be shared with both digital and physical twins, where the system receives sensor fed data from the physical twin and makes changes/updates in the digital twin or vice versa. As a majority of the VR environment tools are also equipped to develop AR/MR models, the proposed methodology can be expanded to enable integration of SysML models with AR/MR based DT infrastructure [55,78]. In addition, with the aid of predesigned prefab databases and ontological dictionaries, it will be possible to create DT scenes of more complex alternatives with multiple game objects seamlessly in the VR environment from the system model. Hence, the time, cost, and effort necessary to design the individual components of the whole MBSE-DT paradigm will be greatly reduced.

Early prototypes can be generated from the system model by utilizing the proposed methodology. Then, the responsible engineer can test the design of the planned system by analyzing the interaction results with the potential customer and other stakeholders. Finally, SysML diagrams are not easily comprehendible to all types of stakeholders involved in the development of a system. An end user may not have enough expertise to understand the different types of diagrams and their relationships within a SysML tool. By recreating the SysML diagrams in an interactive virtual environment, the learning curve present in understanding SysML models will be improved.

The developed methodology to integrate VR environments with SysML has few scopes for extension. The communication of SysML diagram visualizations in VR environments is implemented as one way, i.e., users can only update in the SysML model and then export for interactive visualization in the VR environment. However, the user in VR environments may want to change some blocks of a BDD and propose additional blocks to be added into the diagram. These updates can be exported from the VR scene and incorporated back to the system model upon approval of the systems engineer. Next, the authors have utilized Excel format for extracting the SysML diagrams to visualize in the VR environment. As researchers frequently use JSON and/or XML data interchange format as mediums for data interoperation between tools, the authors will consider these formats in the future expansion of this research. Moreover, since the scope of this research is to develop an easily replicable integration approach, the authors did not design the VR scenario generation scripts with the consideration of VR design parameters such as an end user’s engagement level, fatigue, lighting conditions, etc. Finally, although the benefits of integration of VR and SysML are perceivable through the illustration of the proposed methodology and related studies (see introduction) discussed in this paper, formal measurement studies need to be performed to measure the benefits of the integrated MBSE approach.

7. Conclusions

This research paper demonstrated modeling of a ground-based telescope system utilizing the widely accepted MBSE language SysML. The system model was organized by the four pillars of SysML, which are the Requirements, Structure, Behavior and Parametrics aspects of the SOI. From a methodological viewpoint, this study is useful in illustrating step-by-step how SysML diagrams can be applied to model various design aspects of a telescope system’s system artifacts including requirements, architectures, interfaces, use cases, operational concepts, operational scenarios, and system states. The system model was integrated with a virtual reality environment (Unity engine) to enable data exchange between SysML and VR scenarios through MATLAB and C#. Finally, the SysML diagrams were exported using VTL and recreated automatically through C# scripts as Unity scenes, where users can interact with the SysML diagrams. The proposed methodology fills the gap in the literature by achieving direct communication between SysML and VR environments through real time data exchange. In addition, unlike the previous efforts by the researchers, the proposed methodology did not involve intensive conversion of SysML components to multiple file types to facilitate the integration. In this research, the SysML diagrams visualized in the VR environment are more comprehendible and interactable. Hence, the developed integration methodology will be easy to understand and replicate by future users. The authors plan to expand upon this research in the future to overcome the limitations mentioned in the previous section. The capability to update SysML model diagrams from VR environments based on user feedback will be incorporated. In addition, the integration approach will be evaluated by analyzing the selected benefit types from the MBSE value and benefits review performed by Henderson and Salado [79].