1. Introduction
In the global economy, marine transportation is becoming increasingly significant. More than two thirds of all freights in international trade are transported by sea. Therefore, the ability of marine transportation to link the globe and support technical advancement in maritime equipment is extremely important. In recent years, progressive intelligence based on digitalization and aiming for autonomy has emerged as a new trend and hot spot in the development of the shipbuilding industry, driven by the development of cutting-edge theories and technologies such as the Internet of Things, big data, and artificial intelligence (AI). The world’s major shipbuilding and shipping nations have expanded their investments in the creation and use of autonomous ships. Autonomous ships, also known as maritime autonomous surface ships (MASS), have been developed and are increasingly being used [
1]. The desire to avoid human error—which contributes significantly to the bulk of maritime catastrophes—led to the creation of MASSs. Additionally, crewed ships have also been linked to high operating costs. Therefore, the requirement to avoid the financial expenditures and human mistakes linked to crewed ships serves as the main driving force behind developments in autonomous shipping [
2].
The autonomous degree of MASS has been divided into L1–L4 in the 100th Maritime Safety Committee (MSC) by the International Maritime Organization (IMO) [
3]. The mode of the perception module in a ship’s autonomous navigation system changes from a shared view of mariners and machines into a total machine perception as the degree of autonomy develops, which results in an increase in the number and variety of sensors carried by MASS. Therefore, research into multi-sensor fusion technology in autonomous navigation systems is crucial to raising the level of ship autonomy.
The perception data of a ship’s autonomous navigation system are mostly composed of information about the ship’s motion state (such as its position, heading, speed, attitude, etc.), as well as information about the surrounding environment (such as the status of other ships, pontoons, and other obstacles that may threaten the navigation safety of the ships). Nevertheless, the accuracy of the former’s perception serves as the primary guarantee of the latter’s perception correctness. For instance, if the perception of a ship’s own motion status is erroneous, even though the relative azimuth information of an obstruction perceived by the shipborne sensor has adequate precision, the absolute coordinates of the obstacle will be guessed incorrectly.
The perception of a ship’s own motion status is mostly based on the fusion information provided by shipboard positioning systems and inertial navigation systems (INSs), such as the global positioning system (GPS), global navigation satellite system (GLONASS), Galileo positioning system, BeiDou, a compass, and a gyroscope. In the past decade, researchers have carried out various research works on developing efficient multi-sensor fusion algorithms to estimate the motion status of ships. For instance, a fusion technique based on the Kalman filter (KF) and particle filter (PF) was developed [
4]. This method accurately estimated the status of a ship by fusing its position and attitude data. To further improve the accuracy of ship motion attitude estimates, Ref. [
5] proposed a novel transfer alignment method for a gimbaled inertial navigation system (GINS) and strap-down inertial navigation system (SINS) based on an iterative computation approach. By using this technique, the alignment complexity is reduced. Additionally, a dynamic positioning ship state estimation technique combining the unscented Kalman filter (UKF) and PF was also presented in [
6]. In this approach, the UKF optimizes each particle state update while the PF serves as the general framework. Then, using the particles’ importance distribution, the low-frequency condition of a ship’s motion was determined. Ref. [
7] focused on the least-squares linear fusion filter design for discrete-time stochastic signals from a multi-sensor. A covariance-based approach was used to derive easily implementable recursive filtering algorithms under centralized, distributed, and sequential fusion architectures. The year after, using the linear minimum variance optimality criterion, the local least-squares linear filter obtained with each sensor was improved as a distributed fusion filter with a matrix-weighted linear combination while taking into account the autocorrelation and cross-correlation of multi-sensor measurements [
8]. Furthermore, considering the random packet loss in the transmission process of sensor measurement signals, a recursive filtering algorithm was designed by using a method based on covariance and two compensation strategies based on measurement prediction [
9]. In this algorithm, an alternative method based on direct estimation measurement noise and innovative technology was used to compensate for the packet loss system. To make the navigation system more reliable, Ref. [
10] proposed an improved RISS-GPS ship navigation approach. Modern magnetometer and azimuth calibration technology served as the foundation for this technique.
The research on the fusion technology of a ship’s own motion state has gradually matured and is widely used. The following are its main steps: The mathematical model of a ship’s motion is first discretized, and then the measurements from each sensor are used as the input to perform the fusion estimation of the step with an equal time interval based on the framework of the Kalman filter and its enhanced method. Nevertheless, sampling frequency varies widely since there are so many different types of sensors on board. The sampling rate of ambient sensors, such as cameras and lidar, is typically between 10 and 30 Hz, whereas the sampling rate of GPS is typically between 1 and 5 Hz. As a result, the latest estimate of a ship’s state of motion at the time ambient sensors take samples is often tens or even hundreds of milliseconds prior to the sampling time. The higher the speed of a ship, the more significant the difference in motion state will be. As a consequence, matching the sample from an external environment detecting sensor with the outcomes of a ship motion state estimate in the time dimension is the key technology in fusion technology.
Currently, ship motion status prediction makes considerable use of the long short-term memory (LSTM) model. It may learn the features of ship motion through historical observations, and then predict the motion state at a specific future time. However, this method is often used to predict a ship’s trajectory with strong regularity and is rarely used to predict a ship’s irregular motion directly, such as roll and pitch motion. The method, however, does not account for the modification of the motion law brought on by varying speeds. This paper proposes a non-equal time interval incremental prediction (NETIIP) method. First, a collection of cruise data is trained offline using the LSTM architecture. The time deviation and state increment are used as inputs, and the increment of actual state at the moment to be estimated is taken as the output of the network. Then, using the cubature Kalman filter (CKF) estimator, the position, attitude, speed, and other characteristics of the ships are projected at regular intervals. The estimation result of the ship motion state at this moment is finally derived based on the network estimation from offline training after receiving the time stamp of the environmental measurement sensor. The advantages of this approach are as follows:
- (a)
NETIIP uses the CKF estimates as the input for the LSTM prediction network rather than the sensor’s original data. On the one hand, it can effectively avoid the problem of reduced prediction accuracy caused by sensor measurement error signals as the object of network learning. On the other hand, the amplification of sensor measurement noise caused by first-order state difference can be suppressed.
- (b)
NETIIP adopts a semi-supervised learning mode, which not only learns the changes in a ship’s position and attitude but also incorporates the changes in the ship’s speed into the learning features of the network, which can minimize the impact of the poor learning performance caused by ship speed differences. It merely needs to learn annotated datasets of changes in ship movement at any speed. It is feasible to foresee the ship’s motion status at various speeds.
- (c)
NETIIP employs the technique of learning the properties of state increments rather than directly learning the features of motion states. As opposed to the non-incremental LSTM prediction (NI-LSTM) method, it avoids the shortage of the poor network learning rate caused by the difference between the state characteristics of various speeds or sailing modes and the state characteristics of the training set.
The rest of this paper is organized as follows. The related works on the key technologies for matching the information in an asynchronous system are introduced in
Section 2. In
Section 3, the NETIIP process and algorithm modules are provided.
Section 4 contrasts the NETIIP approach and the NI-LSTM method using contrast model experiments, and the NETTIP method is proved to be effective and feasible. Some conclusions are given in
Section 5.
4. Results and Discussion
The experimental verification work is based on the ship model of the unmanned ship platform “CHCNAV APACHE 6”, which is equipped with a dual antenna GPS (installed at the fore and aft of the ship) for real-time measurement of the ship’s position information and an attitude instrument (installed close to the center of gravity of the ship) for real-time measurement of the ship’s attitude information. The experimental verification work was carried out in the Yanxi Lake experimental site, which is open water in a natural environment with an unknown random environment.
Figure 6 displays the ship’s model and experimental site.
The navigation of an intelligent ship can be divided into two modes in the application: manual remote control and autonomous navigation. Different modes have different changing rules for the ship motion state. Furthermore, the characteristics of a ship’s motion state that change with speed vary. To verify the effectiveness and reliability of the NETIIP method, six sets of experiments were carried out at low speed, medium speed, and high speed, respectively, under the manual remote mode (MRM) and autonomous navigation mode (ANM).
Figure 7 shows the results of the real-time estimation of the ship’s position information (taking the north coordinates as an example) in the six trials by using the CKF estimator and UKF estimator. Herein, the longitude and latitude measured with the dual-antenna GPS are converted using the Universal Transverse Mercator (UTM) projection method. Meanwhile, the initial values of the error covariance
P in Formulas (13), (29) and (30), the process noise matrix
Q in Formulas (18) and (36), and the measurement noise matrix
R in Formulas (23) and (39) are all set to the same value in order to compare the estimation accuracy of the two estimation techniques.
The CKF and UKF both have their own benefits and drawbacks for estimating ship positioning data in real-time. The initial convergence rate of the CKF is marginally quicker than the UKF’s for the same set of algorithm parameters (
a in
Figure 7a–e). Nevertheless, the tracking effect of the CKF is worse than the UKF when a ship is turning with a high speed (
b in
Figure 7c and
a in
Figure 7f).
Figure 8 shows the results of the real-time estimation of the ship’s attitude information (taking the pitch angle of ships as an example) in the six experiments by using the UKF estimator. Meanwhile,
Figure 9 shows the CKF estimation results.
It can be seen that the UKF estimations have a low accuracy when the attitude angle is close to zero and fluctuates often (see
Figure 8a). However, the estimated results can fundamentally converge towards the original data when the attitude angle is far from zero and changes relatively regularly. Comparatively, the CKF produces more accurate results than the UKF for the parameters given, and the CKF estimation method’s assessment of the ship’s attitude angle generally converges around the observed value. We are unable to determine which approach performs better in terms of estimation as the parameter settings may not be optimal for them. However, it is clear from a comparison of
Figure 8 and
Figure 9 that the UKF is more susceptible to changes in the features of the item to be estimated, making it more challenging to perform tasks such as parameter modification.
The results of the two approaches’ real-time estimations of the ship forward speed u are shown in
Figure 10. The figure shows that while the UKF estimation results are nearly non-convergent, the CKF estimation results are smoother. Meanwhile, Equations (58) and (59) are used to calculate the resultant velocities and original observations to accurately evaluate the estimation accuracy for ship velocity with the two approaches, which is depicted in
Figure 11.
Equation (58) is the method for calculating the ship’s resultant velocity in the shipboard coordinate system, which is used for the vector addition of the estimated velocity. Meanwhile, Equation (59) is the method for calculating the resultant velocity in the geodetic coordinate system, which is appropriate for calculating the ship’s resultant velocity under the sensor’s original observation.
In contrast,
Figure 10 and
Figure 11 show that the results of the CKF estimator for the first-order state (ship speed) essentially converge towards the observations, while the results of the UKF estimator practically diverge.
The root-mean-square error (RMSE) coefficient (Equation (60)) is used to calculate the estimation results of each variable for assessment in order to more effectively evaluate the estimation accuracy of the estimated state variables using the UKF and CK.
where
represents the number of estimated cycles,
represents the variable estimated for the
ith period, and
represents the truth value. However, because the true value of the quantity to be estimated cannot be acquired in the experiment, we accept the sensor’s observed value as the genuine value. The RMSE coefficient reflects the convergence and credibility of the estimated result. The smaller the value, the more the estimated result converges towards the observed value, and the higher its credibility. The RMSE coefficient of the estimated results is shown in
Table 1.
Combined with
Figure 7,
Figure 8,
Figure 9,
Figure 10 and
Figure 11 and the data in
Table 1, under the same set of parameters, the UKF has a higher estimation accuracy than the CKF for ship position estimation. On the other hand, the UKF has better parameter robustness than the CKF. With the increase in ship speed, the estimation accuracy of the CKF decreases gradually; thus, for different ship speeds, the CKF must change various parameters to provide appropriate accuracy. However, the accuracy of the UKF estimation will not decrease with the increase in ship speed, that is, a set of parameters can adapt to the estimation of ship position at different speeds.
However, the CKF’s accuracy is substantially greater than the UKF’s for ship attitude. Even worse, the attitude estimated using the UKF does not steadily converge towards the observation (see
Figure 8). Moreover, since attitude and ship velocity have a strong coupling connection, the UKF’s estimation for ship velocity virtually exhibits a diverging trend (see
Figure 11). Thus, the UKF is not appropriate for the estimation of the multidimensional autocorrelation state.
Ship speed is one of the inputs for the prediction network in this study (Formula (55)), which has high requirements for the speed-related estimate accuracy of the pre-state estimation module. Therefore, the CKF is chosen as the pre-state estimate module rather than the UKF.
In order to verify the prediction accuracy of the NETIIP, the CKF estimations of the ship in automatic tracking mode with a six-knots set speed are first collected as the training set. The above six groups of data are used as the test set for prediction and compared with the prediction results of the NI-LSTM algorithm. The ship position prediction results are shown in
Figure 12, and the attitude prediction results are shown in
Figure 13. Therein, the prediction time is set to a random number with an upper limit of the estimation period of the CKF.
Comparing the prediction results of the two methods, it can be seen that the prediction result of NETIIP is closer to the truth data than NI-LSTM. On the other hand, with the change in ship speed, the accuracy of the NI-LSTM prediction also changes greatly. The prediction of the position (
a in
Figure 12d) and attitude (
Figure 13a,d) of the low-speed sailing mode almost diverges. In contrast, the NETIIP technique nearly converges towards the truth data and is less impacted by speed.
To better explore the degree of influence of the ship speed, the absolute value of the prediction error–velocity distribution is plotted, as shown in
Figure 14 and
Figure 15.
The prediction error of NI-LSTM tends to grow with the increase in ship speed in both the manual mode and automatic tracking mode, as can be seen from the distribution charts in
Figure 14 and
Figure 15, while the error of the NETIIP prediction method scarcely changes with ship speed. Since the training set chooses data from the automatic tracking mode, the prediction errors of the two approaches for predicting ship attitude are comparable for the test set in MRM, generally speaking, the NETIIIP algorithm is slightly superior to the NI-LSTM algorithm. In ANM, NETIIP maintains a high prediction accuracy, and its prediction error is essentially kept within 1° for the ship attitude. However, the prediction accuracy of NI-LSTM is poor, and its prediction error varies with the change in ship speed. The prediction results of the NI-LSTM algorithm almost exhibit a divergent trend throughout the range of 1–3 m/s, as observed when combined with
Figure 13d and
Figure 14b.
The RMSE coefficients under each mode are calculated in
Table 2.
According to Equation (61) and the data in
Table 2, the decrease ratios of the RMSE coefficient of the two prediction methods were calculated. The improving effect of the suggested prediction method on prediction accuracy is more noticeable the higher the value.
The decrease ratios of the RMSE coefficients of all the prediction state variables of each mode at the three speeds in
Table 3 were averaged, as shown in
Table 4, which represents the degree of improvement of the proposed prediction methods at different speeds.
The maximum ship velocities in the CKF estimator under each mode and those in the training set were obtained in accordance with
Figure 11 in order to more accurately reflect the relationship between the degree of prediction accuracy improvement of NETIIP in order to more accurately reflect the relationship between the degree of prediction accuracy improvement and the change in ship speed, as shown in
Table 5.
The accuracy of the prediction results of NETIIP is not significantly higher than the traditional NI-LSTM prediction method when the ship speed in the test set approaches that in the training set, as can be seen by comparing
Table 4 and
Table 5 and combining with the results of the prediction error distribution in
Figure 14 and
Figure 15. This is because the prediction accuracy of the traditional method is sufficient. However, the conventional NI-LSTM prediction technique has a low learning rate for the state change rule, and even fails to learn the proper rule, when the test set’s ship speed significantly differs from the training set’s. Therefore, the prediction error increases with the increase in the speed difference. Compared with the traditional NI-LSTM method, the prediction accuracy of the NETIIP prediction method is less affected by the speed. Therefore, compared with the traditional NI-LSTM prediction method, the improvement rate of the prediction accuracy increases with the increase in the speed difference.
It is vital to determine whether the time consumption of this technique is less than the need in order to confirm that the real-time performance of the suggested method satisfies the actual engineering requirements. According to
Figure 1, the time of each cycle of the algorithm should not be longer than the sampling period of the sensor GPS, which is set to 0.1 s in this experiment. The NETIIP and NI-LSTM prediction methods are performed using software named MATLAB on a Windows 10 system with an Intel
® Core(TM) i5-3210m 2.50 Ghz processor. The average time consumption of the methods is calculated as shown in
Table 4.
Because NETIIP uses a single LSTM network structure, which is the same as that of the NI-LSTM prediction technique, the algorithm time is comparable to that of NI-LSTM, and the average time is considerably less than the upper limit of the forecast time 0.1 s (see
Table 6). Therefore, the real-time performance of the proposed prediction method meets the engineering needs.
5. Conclusions
This paper studied a non-equal time interval incremental prediction method for ship motion state to solve the problem of ship state estimation at different rates and sensor sampling times in intelligent ship navigation. First, as the method’s time state estimation modules, the estimation results of the CKF and UKF were first compared and studied, and the justifications for selecting the CKF estimator were given. Then, the prediction results of the NETIIP method and the traditional NI-LSTM prediction method were compared, and the impact of a change in ship velocity on the two prediction methods’ accuracy was examined. The comparative findings demonstrate that the suggested approach has good prediction accuracy for ship condition prediction under various modes and speeds when compared with the conventional NI-LSTM prediction methods. Meanwhile, the algorithm time is almost the same as that of the traditional prediction methods, and both can satisfy the requirements of actual engineering.
Nevertheless, the following are some drawbacks of the suggested approach in this paper: (a) In order to guarantee the algorithm’s prediction accuracy, the CKF estimation algorithm’s state estimation accuracy must first be verified. The CKF algorithm needs to adjust various sets of parameters for different speeds of the ships, which restricts how simple the prediction algorithm can be. (b) Different from the traditional NI-LSTM prediction method, the NETIIP algorithm needs to use time intervals as inputs for training and prediction, necessitating high-precision time stamps in the training set and test set. Inaccurate time stamps will lower the algorithm’s forecast accuracy.
As a result, future studies should take into account a more straightforward and efficient state estimation method as the prediction method’s equal time interval state estimation module. Furthermore, in order to lessen the effects of inadequate timestamp accuracy, a novel predictive compensating approach must be created.