*4.2. Multi-Scale Division of Ship Behavior in Cognitive Space*

## 4.2.1. Action

Considering the practical needs of users for ship behavior, when describing and calculating the microscopic behavior of a ship, this paper avoids the situation that causes the redundancy of successive division of equal time interval or equal distance interval trajectories and the complexity of calculating topological relations. In this paper, from the perspective of behavior semantics, the concept of action is introduced to represent the cognition results of the micro-semantic behavior features, which is to represent the behavior that the ship motion characteristic, involving both speed and course, remains unchanged during the current behavior stage, such as keep course and deceleration (KC\_DE), turn left and deceleration (TL\_DE), turn right and deceleration (TR\_DE), etc. Action behavior is a behavior to characterize the basic motion characteristics of the trajectory without additional semantic information related to environment. The behavior enriched with rich semantic can be formed based the action behavior.

## 4.2.2. Activity

Activity is the cognition results of the behavior represented by the trajectory of action behavior, enriched with topological and geographical semantics, which represents the behavior based on the topological interaction and geographical semantic enrichment. The behavior of the activity occurs on the trajectory of action, which interacts with the entities in the environment, such as anchorage, berth, etc., which is the basic semantic unit of ship behavior, and complex semantic behavior can be expressed by the combination of a set of consecutive activity behaviors.

The behavior difference between the activity and the action is that the action only reflects the semantic of motion characteristics of the trajectory and do not include the semantics of the interaction between the trajectory and its surroundings. In contrast, the activity has more semantics than the action but also involves the semantics of spatial topological calculations and geographic semantic enrichment performed by the action trajectory with environmental entities.

#### 4.2.3. Process

A process represents the behavior of a ship in which the spatial topological relationship between its trajectory and environmental entities remains constant while keeping its speed or course unchanged. That is, when the characteristics of the speed or course of ships change or the spatial topological relationship between its trajectory and environmental entities changes, the current process behavior turns to the next process behavior.

In contrast to activity behaviors, process behaviors are the extension of activity behaviors, which describe the interaction behaviors that occur between trajectories with constant speed and constant course and entities in geographic space, respectively. For example, the behavior of anchoring preparing can be regarded as a process behavior, which is usually accompanied by a series of action behaviors of deceleration, while the trajectory of this behavior keeps the same spatial topology relationship with the geographic entity of anchorage during this process until the behavior of deceleration is stopped, at which point the action behavior of the ship changes, which also means that the activity of the ship changes. Therefore, the process behavior of anchoring preparing indicates that a process behavior occurs for the ship, and its connection with the next process behavior is made through the activity behavior of beginning to anchor, and the moment when the act of beginning to anchor begins is the moment when the behavior of anchoring preparing ends and the moment when the next process behavior begins.

Similarly, when the spatial topological relationship between the ship trajectory and the geographic entity changes, it means that the ship experiences an activity behavior, which indicates the beginning of the next process behavior.

A process behavior tends to have a larger temporal and spatial scope than an activity behavior, and it represents that the ship is executing a certain task, such as the process behavior of anchoring represents that a ship is anchoring at anchor, and the activity of anchoring does not change during this period. In contrast to the activity behavior, the process behavior does not consider the change or not of the combined motion characteristics of speed and course, and its focus is on the change of spatial and topological state between the trajectory that remains constant either on the speed or the course and the surrounding environment.

The behavior of activity, which can be considered as one of the components of the behavior of process, is a trigger element condition between different processes and serves to trigger the end of the previous process and the beginning of the next one.

#### 4.2.4. Event

Event behaviors, which represent the overall behavior of the vessel occurring in the current observation view or macroscopic behavior relative to the current reference target, describe the logical and temporal relationships between the behaviors. For example, the entire behavior of berthing and unberthing can be considered as one event, including the three process behaviors of decelerating for preparing to berth, berthing, and accelerating for unberthing. Event behavior can also denote the behavior that occurs in a larger temporal and spatial context, which is extensible. For example, the behavior of a ship sailing from one port to another can be regarded as a whole event containing several sub-events, including the departure event from port A, several subsequent events, and the arrival event at port B.

The event behavior corresponding to the macro behavior is a semantic aggregation of activities and processes or, alternatively, can represent a semantic aggregation of multiple events with related relationships. As can be seen from Figure 2, the goal of transforming from trajectories to multi-scale spatial-temporal semantic behaviors can be achieved and multi-dimensional modeling and representation of behaviors is realized.

4.2.4. Event

event at port B.

**Figure 2.** Representation process and construction model of ship multi-dimensional behavior. **Figure 2.** Representation process and construction model of ship multi-dimensional behavior.

Event behaviors, which represent the overall behavior of the vessel occurring in the current observation view or macroscopic behavior relative to the current reference target, describe the logical and temporal relationships between the behaviors. For example, the entire behavior of berthing and unberthing can be considered as one event, including the three process behaviors of decelerating for preparing to berth, berthing, and accelerating for unberthing. Event behavior can also denote the behavior that occurs in a larger temporal and spatial context, which is extensible. For example, the behavior of a ship sailing from one port to another can be regarded as a whole event containing several sub-events, including the departure event from port A, several subsequent events, and the arrival

The event behavior corresponding to the macro behavior is a semantic aggregation of activities and processes or, alternatively, can represent a semantic aggregation of multiple events with related relationships. As can be seen from Figure 2, the goal of transforming from trajectories to multi-scale spatial-temporal semantic behaviors can be achieved and multi-dimensional modeling and representation of behaviors is realized.

#### **5. Ontology Modeling for Ship Behavior 5. Ontology Modeling for Ship Behavior**

Based on the analysis of multi-scale behavior, this paper proposes an ontology model for ship semantic behavior modeling in a cognitive manner. The model introduces several concepts in cognition to model the cognitive framework of ship behavior, involving ship, behavior, time, and place, which supports a systematic interpretation of ship behavior by a computer, as shown in Figure 3. Figure 4 presents the ontology of ship behavior created according to our cognitive framework of ship behavior. Figure 4a, Figure 4b and Figure 4c show the class, object properties, and data properties interfaces in the ontology software Protégé, respectively. Based on the analysis of multi-scale behavior, this paper proposes an ontology model for ship semantic behavior modeling in a cognitive manner. The model introduces several concepts in cognition to model the cognitive framework of ship behavior, involving ship, behavior, time, and place, which supports a systematic interpretation of ship behavior by a computer, as shown in Figure 3. Figure 4 presents the ontology of ship behavior created according to our cognitive framework of ship behavior. Figures 4a, 4b and 4c show the class, object properties, and data properties interfaces in the ontology software Protégé, respectively. *J. Mar. Sci. Eng.* **2022**, *10*, x FOR PEER REVIEW 9 of 22

**Figure 4.** Display of cognition ontology of ship behavior in port. (**a**) Classes; (**b**) Object properties;

(**c**) Data properties

**Figure 3.** Ontology model of cognition for ship behavior. **Figure 3.** Ontology model of cognition for ship behavior.

**Figure 3.** Ontology model of cognition for ship behavior.

**Figure 4.** Display of cognition ontology of ship behavior in port. (**a**) Classes; (**b**) Object properties; (**c**) Data properties **Figure 4.** Display of cognition ontology of ship behavior in port. (**a**) Classes; (**b**) Object properties; (**c**) Data properties.


The ship is the generator of trajectory and the occurrence object of behavior. Therefore, it is necessary to identify the object of trajectory in behavior cognition. As a unique code for the ship, the number of the Maritime Mobile Service Identity (MMSI) can be used to determine the object that generates the behavior.

• Behavior

The class of behavior is one of core classes of the cognitive ontology. Based on previous work [2], ship semantic behaviors are divided into four categories, including atomic behaviors, topological behaviors, traffic behaviors, and advanced semantic behaviors. For a better understanding, The class of atomic behavior proposed in [2] are extended to refined divisions, including first-order and second-order atomic behaviors. The second-order

atomic behavior corresponds to action, including 10 categories, while the first-order atomic behavior corresponds to the process where either the speed or the course keeps ships maintaining constant, as can be seen in Figure 2.

As the unit of semantic behavior, the traffic behavior is the basic element to describe advanced semantic behaviors, corresponding to the behavior of activity.

The advanced semantic behavior refers to high-level behaviors, such as the behavior of process and event, that can be formed combining multiple consecutive sort of traffic behaviors in specific application scenarios.

• Environment

The class of environment represents the set of surrounding spatial objects existing in the form of physical or virtual entities, such as anchorages, channels, control areas, and infrastructure.

• Time

In order to represent the temporal relationship of behaviors, we introduce an existed time ontology to our work, which is available online: http://www.w3.org/2006/time# (accessed on 27 September 2020), that includes both instant and interval that can fulfill the temporal functions, which is beneficial to describe complex behaviors that are temporal and logical. Specifically, the class of instant is to describe the transient state of behavior, and time interval represents the time quantum, which lasts for a certain period, including start time and end time to express the duration of behaviors.

According to the theory of interval algebra proposed by Allen [31], basic time relationship includes before, after, and equal. Furthermore, 10 types of relationships between instant and interval can be depicted with 3 kinds of basic relationships. It is the temporal and logic features of ship behavior that can be depicted in this way.

Besides the abovementioned, we connect them with their relationships to link this network. There are three kinds of arrows for that, including solid arrow, dashed arrow, and solid arrow with empty end. As for solid arrow, it refers to the relationship between two classes either same classes or different classes. For example, there is the relationship of "occur object" that has the domain—"traffic behavior" and the range—"ship". For the second one, it means what the own data properties the entity have. An example can be taken to illustrate that the dashed arrow pointing to "Atomic Trajectory" from "Position" represents that the former one has the data of the latter one. The final one means there is the relationship of parent–subclass relationship, such as the arrows of the three behaviors in the top yellow round box pointing to "Advanced Semantic Behavior".

Now that we have extracted the different classes, the next step is to connect them to form knowledge graph. For that reason, we then need to add the relationship properties with each other to them.

#### *5.2. Property-Constrained Axiom*

• The class of Atomic Behavior

Atomic behavior can be recognized and annotated by pre-processing and calculation of trajectories. Therefore, we import atomic behavior as instances of ship behavior directly into the ontology via its interface. On this basis, first-order atomic behavior can be expressed by second-order atomic behavior. For example, several instances can be illustrated using the property-constrained axiom as follows.

acceleration = (KC\_AC or TL\_AC or TR\_AC) keepSpeed = (KC\_KS or TL\_KS or TR\_KS) stop= stay

Furthermore, since the move behavior is one of the general behaviors, including all the first-order atomic behaviors, we likewise use the axiomatic expression of the property constraint to define the move behavior, which indicates that the ship is in the move state, including all the second-order atomic behaviors, which can be expressed as follows.

move = (reflects value KC\_AC) or (reflects value KC\_DE) or (reflects value KC\_KS) or (reflects value TL\_AC) or (reflects value TL\_DE) or (reflects value TL\_KS) or (reflects value TR\_AC) or (reflects value TR\_DE) or (reflects value TR\_KS) or (reflects value TL\_AC) or (reflects value TL\_DE) or (reflects value TL\_KS) or (reflects value TR\_AC) or (reflects value TR\_DE) or (reflects value TR\_KS) • The class of Topological Behavior

Atomic behavior can be recognized and annotated by pre-processing and calculation of trajectories. Therefore, we import atomic behavior as instances of ship behavior directly into the ontology via its interface. On this basis, first-order atomic behavior can be expressed by second-order atomic behavior. For example, several instances can be illus-

Furthermore, since the move behavior is one of the general behaviors, including all the first-order atomic behaviors, we likewise use the axiomatic expression of the property constraint to define the move behavior, which indicates that the ship is in the move state, including all the second-order atomic behaviors, which can be expressed as follows.

move = (reflects value KC\_AC) or (reflects value KC\_DE) or (reflects value KC\_KS)

• The class of Topological Behavior According to definition and semantic computing results for topological behavior,

*5.2. Property-Constrained Axiom*  • The class of Atomic Behavior

stop= stay

According to definition and semantic computing results for topological behavior, each topological behavior can be expressed with ontology via axiomatic expression method of attribute constraint. For example, Topo1 represents the topological relationships between trajectory and navigation environment, which can be expressed with spatial topological relationships as follows: each topological behavior can be expressed with ontology via axiomatic expression method of attribute constraint. For example, Topo1 represents the topological relationships between trajectory and navigation environment, which can be expressed with spatial topological relationships as follows: Topo1 = PL1 some trafficRule

Topo1 = PL1 some trafficRule

• The class of Traffic behavior and Advanced Semantic behavior • The class of Traffic behavior and Advanced Semantic behavior

*J. Mar. Sci. Eng.* **2022**, *10*, x FOR PEER REVIEW 11 of 22

trated using the property-constrained axiom as follows. acceleration = (KC\_AC or TL\_AC or TR\_AC) keepSpeed = (KC\_KS or TL\_KS or TR\_KS)

The traffic behavior class corresponds to the activity behavior. The traffic behavior in port water areas can be divided into nine types of activity behavior, and the advanced semantic behavior can be divided into five types of process behavior and three types of event behavior, which can be seen in Figure 5. The traffic behavior class corresponds to the activity behavior. The traffic behavior in port water areas can be divided into nine types of activity behavior, and the advanced semantic behavior can be divided into five types of process behavior and three types of event behavior, which can be seen in Figure 5.

**Figure 5.** Results of ship semantic behavior in port water traffic areas. **Figure 5.** Results of ship semantic behavior in port water traffic areas.

Representation of behaviors with different semantic scales in the port water traffic areas can be done in different ways using ontology. Simple behaviors, such as active behaviors, can be expressed using property-constrained axioms based on atomic and topo-Representation of behaviors with different semantic scales in the port water traffic areas can be done in different ways using ontology. Simple behaviors, such as active behaviors, can be expressed using property-constrained axioms based on atomic and topological behaviors.

logical behaviors. Taking the behavior of Entering the Fairway as an example, the sufficient and neces-Taking the behavior of Entering the Fairway as an example, the sufficient and necessary conditions for cross\_into\_lane should be as follows:

sary conditions for cross\_into\_lane should be as follows: Trajectory T intersects with the line of fairway or the line between end points of the former, resulting in an intersection point located on an atomic trajectory AT which belongs to T. The beginning point of AT is located on the inside of the fairway and the endpoint is located on the outside of the fairway.. Likewise, the behavior of approach\_pier can also be presented in a same way. The activity behavior of cross\_into\_lane and approach\_pier can be represented as Figure 6.

Trajectory T intersects with the line of fairway or the line between end points of the former, resulting in an intersection point located on an atomic trajectory AT which belongs to T. The beginning point of AT is located on the inside of the fairway and the endpoint is located on the outside of the fairway.. Likewise, the behavior of approach\_pier can also be presented in a same way. The activity behavior of cross\_into\_lane and approach\_pier

Trajectory T intersects with the line of fairway or the line between end points of the former, resulting in an intersection point located on an atomic trajectory AT which belongs to T. The beginning point of AT is located on the inside of the fairway and the endpoint is located on the outside of the fairway.. Likewise, the behavior of approach\_pier can also be presented in a same way. The activity behavior of cross\_into\_lane and approach\_pier

**Figure 6.** Knowledge representation of activity behaviors. **Figure 6.** Knowledge representation of activity behaviors. *5.3. Complicated Behavior Expressions Using SWRL* 

*J. Mar. Sci. Eng.* **2022**, *10*, x FOR PEER REVIEW 12 of 22

#### *5.3. Complicated Behavior Expressions Using SWRL 5.3. Complicated Behavior Expressions Using SWRL* Complicated behaviors, such as process behaviors and event behaviors, are difficult

can be represented as Figure 6.

can be represented as Figure 6.

Complicated behaviors, such as process behaviors and event behaviors, are difficult to express directly with property-constrained axioms due to complex intrinsic behavioral logic. For this reason, we introduce Allen's algebraic theory to model the temporal relationship of behaviors and express their complex behavioral intrinsic logical relations leveraging SWRL. Specifically, advanced semantic behaviors, such as event behaviors, consist of ordered specific activities and process behaviors, and as these behaviors occur, it can be triggered and inferred whether the advanced semantic behavior occurs or not. As shown in Figure 7, the event behavior of Anchor is explicitly temporal and logical, in Complicated behaviors, such as process behaviors and event behaviors, are difficult to express directly with property-constrained axioms due to complex intrinsic behavioral logic. For this reason, we introduce Allen's algebraic theory to model the temporal relationship of behaviors and express their complex behavioral intrinsic logical relations leveraging SWRL. Specifically, advanced semantic behaviors, such as event behaviors, consist of ordered specific activities and process behaviors, and as these behaviors occur, it can be triggered and inferred whether the advanced semantic behavior occurs or not. As shown in Figure 7, the event behavior of Anchor is explicitly temporal and logical, in which behaviors of the blue rectangular box and the gray arrow box make up the Anchor event. to express directly with property-constrained axioms due to complex intrinsic behavioral logic. For this reason, we introduce Allen's algebraic theory to model the temporal relationship of behaviors and express their complex behavioral intrinsic logical relations leveraging SWRL. Specifically, advanced semantic behaviors, such as event behaviors, consist of ordered specific activities and process behaviors, and as these behaviors occur, it can be triggered and inferred whether the advanced semantic behavior occurs or not. As shown in Figure 7, the event behavior of Anchor is explicitly temporal and logical, in which behaviors of the blue rectangular box and the gray arrow box make up the Anchor event.

**Figure 7.** Anchor event of ship occurred in anchorage. **Figure 7.** Anchor event of ship occurred in anchorage.

**Figure 7.** Anchor event of ship occurred in anchorage. For an explicit explanation, the process of the Anchor preparing process is selected to illustrate how to formalize the behavior, which is described below. For an explicit explanation, the process of the Anchor preparing process is selected to illustrate how to formalize the behavior, which is described below.

For an explicit explanation, the process of the Anchor preparing process is selected to illustrate how to formalize the behavior, which is described below. Anchor preparing process: The behavior from the instant the ship enters the anchorage until the start of anchoring, it consists of a series of activities in *preparing to anchor*. It is based on the existing knowledge to infer the advanced behavior, but the activity of preparing to anchor is not easily identified. Therefore, we do not use it to deduce the process. However, it is also worth noting that the trajectory corresponding to the behavior of the process contains a series of trajectories of the activity behavior, i.e., from the trajectory Anchor preparing process: The behavior from the instant the ship enters the anchorage until the start of anchoring, it consists of a series of activities in *preparing to anchor*. It is based on the existing knowledge to infer the advanced behavior, but the activity of preparing to anchor is not easily identified. Therefore, we do not use it to deduce the process. However, it is also worth noting that the trajectory corresponding to the behavior of the process contains a series of trajectories of the activity behavior, i.e., from the trajectory reflecting the *cross\_into\_anchorage* behavior to the trajectory reflecting the activity of the first anchor activity. Since these two behaviors can be obtained computationally from the Anchor preparing process: The behavior from the instant the ship enters the anchorage until the start of anchoring, it consists of a series of activities in *preparing to anchor*. It is based on the existing knowledge to infer the advanced behavior, but the activity of preparing to anchor is not easily identified. Therefore, we do not use it to deduce the process. However, it is also worth noting that the trajectory corresponding to the behavior of the process contains a series of trajectories of the activity behavior, i.e., from the trajectory reflecting the *cross\_into\_anchorage* behavior to the trajectory reflecting the activity of the first anchor activity. Since these two behaviors can be obtained computationally from the AIS-based preprocessing module, it is possible to determine whether the process occurs by judging whether the two behaviors occur sequentially, which can be expressed in SWRL as:

reflecting the *cross\_into\_anchorage* behavior to the trajectory reflecting the activity of the first anchor activity. Since these two behaviors can be obtained computationally from the Anchor preparing process = cognition2:ship (?s) ˆ cognition2:trajectory (?t) ˆ cognition2:hasTraj (?s, ?t) ˆ cognition2:metaTraj (?stra) ˆ cognition2:comprises (?t, ?stra) ˆ cognition2:Point (?p1) ˆ cognition2:Point (?p2) ˆ cognition2:hasBeginPoint (?stra, ?p1) ˆ cognition2:hasEndPoint (?stra, ?p2) ˆ cognition2:LA5 (?stra, ?p) ˆ cognition2:anchorage (?p) ˆ cognition2:hasSpeed (?p2, ?x) ˆ swrlb:lessThanOrEqual (?x, "0.5" ˆˆ xsd:float) → cognition2:hasBehavior (?s, cognition2:anchor\_preparing)

Similarly, other behaviors occurring in port areas can be stated in the same way, as shown in Table 1.

**Table 1.** Selected SWRL rules for reasoning about advanced behavior.


#### **6. Case Study**

In order to investigate the feasibility of the cognitive model, we take the scenario as the experiment case where ship behavior, such as *arrival* and *departure* events, occurred in a port to show how complicated behaviors can be deduced in a cognitive way.

Firstly, based on the navigational experience of seafarers in port traffic areas, the most common ship behaviors occurring in order in these areas can be divided into three layers in which they occur, as shown in Figure 8. The overall behavior can be considered as an event of ship arrival-departure, in which the event of anchoring, entering fairway, berthing and unberthing, and departure are most commonly occur in an orderly manner. Likewise, the process layer and the activity layer can be extracted and depicted as follows.

#### *6.1. Data Processing*

The paper uses the AIS data and geographic data from Xiamen port for March and April 2016, including ship trajectory, fairways, anchorages, and piers. First, we pre-process ship trajectories, including data sorting and interpolation. Then, the dynamic AIS data are matched with the ship name, MMSI, and ship type in the static database to achieve the acquisition of ship attributes. Furthermore, the name, functional attributes, and location information of geographical objects can be obtained from www.chinaports.com (accessed on 18 September 2020).

Protégé is an ontology modeling tool [32] and is used here to construct an ontology model of ship behavior perception. We use version 5.5.0 of the software, version 2.2.0 of the Pellet reasoner and version 2.0.9 of SWRL. In addition, the model imports the time ontology abovementioned to support reasoning about complex behavioral temporal relationships.

**Figure 8.** Cognition graph of the behavior of ship arriving and leaving the port. **Figure 8.** Cognition graph of the behavior of ship arriving and leaving the port.

#### *6.1. Data Processing 6.2. Trajectory Segments and Semantic Annotation*

The paper uses the AIS data and geographic data from Xiamen port for March and April 2016, including ship trajectory, fairways, anchorages, and piers. First, we pre-process ship trajectories, including data sorting and interpolation. Then, the dynamic AIS data are matched with the ship name, MMSI, and ship type in the static database to achieve the acquisition of ship attributes. Furthermore, the name, functional attributes, and location information of geographical objects can be obtained from www.chinaports.com (accessed on 18 September 2020). Protégé is an ontology modeling tool [32] and is used here to construct an ontology model of ship behavior perception. We use version 5.5.0 of the software, version 2.2.0 of In order to reduce the computational complexity and improve the ontological reasoning efficiency, 20,000 AIS data of ships are extracted for validation in this experiment. Firstly, the trajectories of different ships are sorted out in order to obtain the trajectories of each ship. Secondly, the trajectories are divided into "moving-stop" segments based on the recognition of stopping points to realize the labeling of moving trajectories. On this basis, we separate the moving trajectory from the stop trajectory to complete the annotation of atomic trajectory and the further recognition of atomic behavior. Finally, the start and end points of the trajectory are marked according to the stop, start, and end points of the atomic trajectory.

berthing and unberthing, and departure are most commonly occur in an orderly manner. Likewise, the process layer and the activity layer can be extracted and depicted as follows.

the Pellet reasoner and version 2.0.9 of SWRL. In addition, the model imports the time ontology abovementioned to support reasoning about complex behavioral temporal relationships. *6.2. Trajectory Segments and Semantic Annotation*  In order to reduce the computational complexity and improve the ontological rea-In order to calculate the spatial topological relationship between trajectories and the environment, the paper introduces a library for topology calculation based on Python programming language—Shapely. Firstly, various geographical objects and ship trajectories are converted into the format of spatial data. Then, the topological relationships of these converted objects are calculated to obtain the DE-9IM metrics of the relationships between trajectories and geographic objects. Finally, the computed results are mapped with the corresponding trajectories to prepare for the semantic annotation of ship behavior.

soning efficiency, 20,000 AIS data of ships are extracted for validation in this experiment. Firstly, the trajectories of different ships are sorted out in order to obtain the trajectories of each ship. Secondly, the trajectories are divided into "moving-stop" segments based on the recognition of stopping points to realize the labeling of moving trajectories. On this basis, we separate the moving trajectory from the stop trajectory to complete the annota-After the data level preparation is completed, the semantic information needs to be added to the ontology. In order to realize the combination of data and semantic information in the Python environment, the paper introduces Owlready2, a python-oriented ontology programming module that adds the already computed semantic information to the data layer and can load and save ontology files for modification and inference.

tion of atomic trajectory and the further recognition of atomic behavior. Finally, the start and end points of the trajectory are marked according to the stop, start, and end points of the atomic trajectory. Figure 9 shows the overall process of behavioral cognitive computation, semantic reasoning and querying, which can support knowledge queries of behaviors with different semantic scales.

In order to calculate the spatial topological relationship between trajectories and the

#### environment, the paper introduces a library for topology calculation based on Python pro-*6.3. Semanticization of Ship Behavior*

gramming language—Shapely. Firstly, various geographical objects and ship trajectories After importing the data related to ship behavior cognition into the ontology, including the ship, its trajectory segments, and the relationship between them, ship behavior can be clearly depicted. Figure 10 shows the importing results of ship RENLONG and the details of its trajectory segments, such as the place and time of occurrence.

**Figure 9.** The process of cognitive computation, semantic reasoning, and querying of ship behavior. **Figure 9.** The process of cognitive computation, semantic reasoning, and querying of ship behavior.

are converted into the format of spatial data. Then, the topological relationships of these converted objects are calculated to obtain the DE-9IM metrics of the relationships between trajectories and geographic objects. Finally, the computed results are mapped with the corresponding trajectories to prepare for the semantic annotation of ship behavior.

After the data level preparation is completed, the semantic information needs to be added to the ontology. In order to realize the combination of data and semantic information in the Python environment, the paper introduces Owlready2, a python-oriented ontology programming module that adds the already computed semantic information to the data layer and can load and save ontology files for modification and inference.

Figure 9 shows the overall process of behavioral cognitive computation, semantic reasoning and querying, which can support knowledge queries of behaviors with differ-


ent semantic scales.

(**a**) (**b**)

As mentioned in the previous section, the atomic and topological behaviors can be


(**c**) (**d**)

to the channel, etc.


**Figure 10.** The importing result of ship behavior. (**a**) The trajectories occurred by RENLONG; (**b**) The sub-trajectories and key points of trajectory RENLONG\_100; (**c**) The spatial topological relationships with surroundings for RENLONG\_10010135\_begin\_point; (**d**) The spatial topological relationships with surroundings for RENLONG\_100\_9851; (**e**) The instant properties of RENLONG\_100\_interval; (**f**) The XSD date timestamp property corresponding to the moment 1460149090. **Figure 10.** The importing result of ship behavior. (**a**) The trajectories occurred by RENLONG; (**b**) The sub-trajectories and key points of trajectory RENLONG\_100; (**c**) The spatial topological relationships with surroundings for RENLONG\_10010135\_begin\_point; (**d**) The spatial topological relationships with surroundings for RENLONG\_100\_9851; (**e**) The instant properties of RENLONG\_100\_interval; (**f**) The XSD date timestamp property corresponding to the moment 1460149090.

For an intuitive comprehension of deduced behaviors shown in Figure 11, the trajec-

annotation. Figure 12a, Figure 12c, and Figure 12d show the ship's semantic behavior at anchorage, fairway, and berth, respectively, while Figure 12b is a zoomed-in view of the behavior in Figure 12a. The segmented ship trajectory segments can be clearly identified in these images, as well as the annotated advanced ship semantic behavior, such as the ship's approach to the anchorage, the ship's exit from the anchorage, the ship's approach

Specifically, Figure 10a shows the movement and stationary trajectory segmentation identified for the ship RENLONG based on the "move-stop" trajectory segmentation method. Figure 10b shows the different atomic trajectory segments contained in the motion trajectory segmentation RENLONG\_100 and the end point of this trajectory. Figure 10c shows the topological and temporal properties of RENLONG\_ 10010135\_begin\_point as the starting point of RENLONG\_10010135, in relation to the geographic region. Figure 10d represents the topological properties of the RENLONG\_100\_9851 atomic trajectory segment with respect to the geographic region around it, as well as its beginning and end points. Figure 10e shows the temporal properties of the time period in which the RENLONG\_100 trajectory segment occurs, where the property of "has beginning" indicates that the beginning point of the trajectory segment occurred at the moment 1460149090. Figure 10f shows the XSD date timestamp property corresponding to the moment 1460149090.

As mentioned in the previous section, the atomic and topological behaviors can be stated based on the property-constrained axioms. The first-order atomic and topological behaviors are defined in terms of sufficient and necessary conditional constraint axioms. When the second-order behavior or topological features satisfy the definition of the class of the corresponding behavior, they will be automatically derived and classified to the corresponding first-order atomic behavior. As shown in Figure 11, the trajectory segment XINHAIXIU\_49\_3699 is classified as the class of cross\_into\_lane. On the basis of simple semantic behaviors, complicated behaviors, such as behaviors of process and event, can be further deduced based on the rules stated using SWRL, as described in Section 5.3. *J. Mar. Sci. Eng.* **2022**, *10*, x FOR PEER REVIEW 17 of 22

**Figure 11.** The reasoning results of activity behavior cross\_into\_lan*e.*  **Figure 11.** The reasoning results of activity behavior cross\_into\_lane.

For an intuitive comprehension of deduced behaviors shown in Figure 11, the trajectory marked with corresponding behavior is visualized in Figure 12 that shows ship semantic behaviors after trajectory segmentation, spatial topology calculation, and semantic annotation. Figure 12a,c,d show the ship's semantic behavior at anchorage, fairway, and berth, respectively, while Figure 12b is a zoomed-in view of the behavior in Figure 12a. The segmented ship trajectory segments can be clearly identified in these images, as well as the annotated advanced ship semantic behavior, such as the ship's approach to the anchorage, the ship's exit from the anchorage, the ship's approach to the channel, etc.

(**a**) (**b**)

(**c**) (**d**)

**Figure 12.** The visualization of the ship trajectories marked with corresponding behaviors. (Note: in order to facilitate the identification of ship behavior at the pier, a rectangular area of the pier in the

**Figure 11.** The reasoning results of activity behavior cross\_into\_lan*e.* 

**Figure 12.** The visualization of the ship trajectories marked with corresponding behaviors. (Note: in order to facilitate the identification of ship behavior at the pier, a rectangular area of the pier in the **Figure 12.** The visualization of the ship trajectories marked with corresponding behaviors. (Note: in order to facilitate the identification of ship behavior at the pier, a rectangular area of the pier in the port was extracted roughly from the ship *berthing* behaviors.). (**a**) Ship's semantic behavior occurring at anchorage; (**b**) Zoomed-in view ship's semantic behavior occurring at anchorage in (**a**); (**c**) Ship's semantic behavior occurring at fairway; (**d**) Ship's semantic behavior occurring at pier.

#### **7. Results**

#### *7.1. Semantic Query*

Based on this ontology model, users can execute semantic queries on behavior cognition, such as ship trajectory, behavior, occurrence time, and occurrence place. In addition, the behavior of changing speed, changing course, stopping, and so on can be obtained based on the query. The SPARQL language of the query is shown below, and the results of the query can be seen in Figure 13.

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX owl: <http://www.w3.org/2002/07/owl#> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

**7. Results** 

*7.1. Semantic Query* 

PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> PREFIX cog: <http://www.semanticweb.org/song/cognition2#> SELECT ?ship ?behavior ?trajectory ?metatraj WHERE { ?ship cog:hasTraj ?trajectory. optional {?trajectory cog:reflects ?behavior. ?trajectory cog:comprises ?metatraj.}} PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> PREFIX cog:<http://www.semanticweb.org/song/cognition2#> SELECT ?ship ?behavior ?trajectory ?metatraj WHERE { ?ship cog:hasTraj ?trajectory. optional {?trajectory cog:reflects ?behavior . ?trajectory cog:comprises ?metatraj.}}


port was extracted roughly from the ship *berthing* behaviors.). (**a**) Ship's semantic behavior occurring at anchorage; (**b**) Zoomed-in view ship's semantic behavior occurring at anchorage in (**a**); (**c**) Ship's semantic behavior occurring at fairway ; (**d**) Ship's semantic behavior occurring at pier.

Based on this ontology model, users can execute semantic queries on behavior cognition, such as ship trajectory, behavior, occurrence time, and occurrence place. In addition, the behavior of changing speed, changing course, stopping, and so on can be obtained based on the query. The SPARQL language of the query is shown below, and the

**Figure 13.** The SPARQL query results of ship behavior. **Figure 13.** The SPARQL query results of ship behavior.

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

#### *7.2. Semantic Reasoning 7.2. Semantic Reasoning*

results of the query can be seen in Figure 13.

*J. Mar. Sci. Eng.* **2022**, *10*, x FOR PEER REVIEW 18 of 22

Semantic reasoning is the process of acquiring implicit knowledge by driving the reasoning function of an ontology. Figure 14 shows an example of semantic reasoning about the behavior of a ship at anchorage. The attributes with yellow background of RENLONG are based on the inference results, including the implicit behaviors generated by RENLONG, its trajectory segments, and the place where and the time when these be-Semantic reasoning is the process of acquiring implicit knowledge by driving the reasoning function of an ontology. Figure 14 shows an example of semantic reasoning about the behavior of a ship at anchorage. The attributes with yellow background of RENLONG are based on the inference results, including the implicit behaviors generated by RENLONG, its trajectory segments, and the place where and the time when these behaviors took place. *J. Mar. Sci. Eng.* **2022**, *10*, x FOR PEER REVIEW 19 of 22



<sup>(</sup>**b**)

**Figure 14.** Reasoning results of behavior occurred in anchorage. (**a**) Deduced behaviors occurred by RENLONG; (**b**) The time of occurrence of the behavior Cross\_into\_anchorage, involving begin and end moments. **Figure 14.** Reasoning results of behavior occurred in anchorage. (**a**) Deduced behaviors occurred by RENLONG; (**b**) The time of occurrence of the behavior Cross\_into\_anchorage, involving begin and end moments.

*7.3. Discussion*  The behaviors of ships navigating around anchorages, fairways, and piers are selected for property-constrained axiom-based reasoning with SWRL for complicated be-The result of the inference shows that RENLONG has the behavior of an anchor event. The start time is the beginning moment of the cross\_into\_anchorage behavior, and the end time is the end moment of the cross\_out\_of\_anchorage behavior. Likewise, the behaviors

havioral reasoning, respectively. The results show that desired semantic behaviors can be recognized, leveraging the inference mechanism of behavior ontology, including from simple semantic behaviors, such as atomic behavior to large scale ship behavior, such as

proving the advantage and effectiveness of the model in recognizing ship semantic be-

In addition, the behavior of a ship can be expressed by the object property 'has behavior', and the behavior with different scales can be characterized by setting the corresponding object property to achieve the multi-scale behavior of the ship. On this basis, the SWRL rule can be used to achieve the progressive reasoning of behavior between different scales, which is in line with the human habit of behavior cognition. However, such an approach is too cumbersome and all the rules need to be added manually by the people

To enable autonomous objects in waterborne transport systems to have the capability of reasoning about and recognize historical complicated ship behavior semantically based on the historical AIS trajectory data, this paper proposes a framework for constructing

with expert knowledge, which consumes a lot of resources.

**8. Conclusions** 

haviors, especially complicated temporal behaviors.

occurring in the fairway and the pier of the ship can be reasoned out like the reasoning process of ship behavior in the anchorage.

As can be seen in Figure 13, the value of the object property of RENLONG *has behavior* is cross\_into\_anchorage, Anchor, and anchoring\_event, but it cannot be deduced to the scale to which the behavior specifically belongs, such as activity and event. For this reason, the ontology sets different scales of behavior for object properties describing the scale of ship behavior, such as the properties of leave\_pier and crossIntoAnchorage, which can provide a computational basis for reasoning about complicated behaviors.

#### *7.3. Discussion*

The behaviors of ships navigating around anchorages, fairways, and piers are selected for property-constrained axiom-based reasoning with SWRL for complicated behavioral reasoning, respectively. The results show that desired semantic behaviors can be recognized, leveraging the inference mechanism of behavior ontology, including from simple semantic behaviors, such as atomic behavior to large scale ship behavior, such as event in port waters. Key information of ship behavior cognition can be characterized, proving the advantage and effectiveness of the model in recognizing ship semantic behaviors, especially complicated temporal behaviors.

In addition, the behavior of a ship can be expressed by the object property 'has behavior', and the behavior with different scales can be characterized by setting the corresponding object property to achieve the multi-scale behavior of the ship. On this basis, the SWRL rule can be used to achieve the progressive reasoning of behavior between different scales, which is in line with the human habit of behavior cognition. However, such an approach is too cumbersome and all the rules need to be added manually by the people with expert knowledge, which consumes a lot of resources.

#### **8. Conclusions**

To enable autonomous objects in waterborne transport systems to have the capability of reasoning about and recognize historical complicated ship behavior semantically based on the historical AIS trajectory data, this paper proposes a framework for constructing semantic models of multi-scale ship behavior in cognitive space to achieve automatic extraction of semantic behavior of ships from the data layer to the semantic layer. On the basis of multi-scale characteristics of ship behavior reflected in ship trajectories, combined with the logical way humans perceive complicated behaviors, the cognition of ship behavior by an intelligent supervision system can be seen as an all-encompassing cognition involving the object, time, place, and behavior of the occurrence of ship behavior. Therefore, based on our previous work, this paper introduces a multi-scale behavioral semantic representation model to support the intelligent supervisory system's cognition of ship behavior in a multidimensional and multi-scale space. Using the logical reasoning capabilities of the ontology and the temporal ontology's modelling basis for time, ship behavior, including both simple and complex behaviors, can be accessed driven by the knowledge representation and logical reasoning capabilities of ontology. This suggests that it is feasible and reasonable to model the behavior of ships at multiple scales in a human cognitive manner.

However, there are some points that need further improvement. First, the model relies heavily on domain knowledge and needs to be constructed by domain experts, leading to inefficient application in practical scenarios. In addition, the paper does not consider the probability of ship behavior, especially in the continuous process, which limits the effectiveness of behavior implementation. In addition, there are various navigation scenarios where infrastructure exists that needs to be identified by the autonomous objects themselves or be considered as variables for human operator input for further analysis, which also needs to be addressed or clarified in the future. What needs to be done in the future is how to quickly extract and transform the textual information obtained from the website for various navigation scenarios, such as navigation modes, mooring information, etc., into knowledge that can be processed and understood by the autonomous system and

expand it into a knowledge base with some scenario migration capability to make it highly reusable in different scenarios.

Future work can focus on the following points: firstly, online modelling, and identification of ship semantic behavior based on ship trajectory data; secondly, based on the semantic annotation results of historical ship trajectory data, combined with data mining algorithms, further mining of ship behavior at different semantic scales in port waters from the semantic layer to obtain implicit knowledge of high level ship behavior semantics. Finally, extending the individual semantic behavior model to interactive behaviors between two or more vessels can support the safety supervision of the waterborne transport system.

**Author Contributions:** R.S. and Y.W. carried out the experiment. R.S. wrote the manuscript with support from Y.W., R.S., W.T. and Q.Z. planned and carried out the experiment. E.P. helped supervise the project. R.S. and Y.W. conceived the original idea. Y.W. and P.v.G. supervised the project. All authors have read and agreed to the published version of the manuscript.

**Funding:** The study would express great gratitude to the support of China Scholarship Council under Grant 202106950002 and the National Natural Science Foundation of China (NSFC) through Grant No. 52072287.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest and the funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

#### **References**

