A New Model Uncertainty Measure of Wave-Induced Motions and Loads on a Container Ship with Forward Speed

Abstract

A new uncertainty quantifier is presented for linear transfer functions of wave-induced ship motions and loads obtained by various seakeeping codes. The numerical simulations are conducted for the high-speed Flokstra container ship in regular waves at various heading angles, and the results are compared with existing experimental data. The study employs five numerical codes that are based on three different seakeeping theories, namely strip theory, 3D frequency-domain method, and 3D time-domain method. Multiple measures are applied to quantify the uncertainty in the calculated transfer functions, such as frequency-independent model error, coefficient of determination, and the total difference. In addition, a new measure of uncertainty, termed modified total difference, is proposed for determining the uncertainty of individual seakeeping codes based on experimental data rather than the mean of results obtained by numerical codes. Results show that the uncertainty measures can identify differences between the codes. The predicted wave-induced loads have higher uncertainties compared to motions. The uncertainty assessment shows that none of the applied codes can produce accurate estimates for all wave-induced motions and loads at all heading angles at the same time.

1. Introduction

The predictions of wave-induced ship motions and loads are crucial for ship structural design, operation, and safety. The ship structural design procedure has always been based on the assessment of the lifetime maximum values of still water and wave-induced loads, which have to be resisted by the strength of the structure [1]. The more modern approach that relies on a reliability-based format also keeps these two main load components [2] and typically the wave-induced loads account for about 60% of the loads while the other 40% are due to the still-water loads [3].

The design values of the wave-induced loads are determined by the long-term formulation which weights the maximum wave-induced loads experienced in a sea state by the probability of its occurrence [4], a procedure adopted by the various Classification Societies [5]. The long-term model or several of the existing improvements [3] are based on the response amplitude operators that describe the wave-induced loads in each sea state and the long-term models of waves, which have been surveyed recently [6]. The response amplitude operator represents the loads that are associated with the motions that waves induce in ships, which is the subject of this paper.

Analytical or numerical methods, model testing, and full-scale measurements can all be used to calculate motions and loads. The experimental approach is efficient, although it is expensive and time-consuming. Therefore, a viable alternative is the use of seakeeping numerical methods. The emergence of new ship types or hull forms, along with the current trend and interest in expanding the size of conventional ships to reduce transportation costs, necessitates the investigation and enhancement of the accuracy of the seakeeping numerical methods.

The seakeeping numerical methods are categorized based on body geometry and free-surface nonlinearities, which are associated with the hull shape, instantaneous wetted surface area, as well as incident and disturbed wave characteristics [3,7,8,9,10]. Accordingly, depending on the significance placed on incorporating nonlinear effects, different seakeeping codes have been developed on a wide range of mathematical models of varying complexity, from purely linear to fully nonlinear methods, as well as hybrid approaches that combine both. Linear seakeeping methods assume small wave heights, but in practice, they are frequently applied to relatively larger waves. Based on the linear theory, the estimated transfer functions, and spectral analysis, the extreme values of the wave-induced response and loads can be estimated.

The linear seakeeping models are generally based on slender-body theory, strip theory, and the three-dimensional boundary element method (3D BEM) to assess wave-induced ship motions and loads. Where the hydrodynamic problem is formulated for the still water-line wetted body surface. Multiple approximations can be employed to model the interaction between the steady flow and the unsteady wave field to evaluate the effects of forward speed on ship motions, loads, and wave-added resistance [11,12,13,14,15], leading to a variety of seakeeping models. However, it is challenging to claim one approximation superior to all others [7]. Many versions of slender-body and strip theories are commonly employed in the frequency domain [16,17,18,19]. PDSTRIP [20], MAXSURF [21], and IST-CENTEC, which is the frequency domain component of the nonlinear code in [22] are examples of strip theory codes that are implemented in the frequency domain.

To properly account for the 3D interactions and speed effects, various linear three-dimensional panel methods are adopted in the frequency domain. These techniques either apply the wave Green’s function on the wetted hull surface [23,24,25] or Rankine Green’s function on both the wetted hull surface and free surface [26,27,28] to solve the hydrodynamic problem. Saito et al. [29] used a frequency-domain Rankine panel method to calculate the wave-induced motions and loads for a high-speed vessel. Using the wave Green’s function technique, appropriate fundamental singularities are distributed across the body’s surface while satisfying the linearized free surface boundary condition. The singularities can be either translating-pulsating which satisfies the forward speed-free surface condition or pulsating with speed correction as they satisfy the zero forward speed-free surface condition [24,25,30]. For Froude numbers below 0.3, the response calculation differences between the pulsating source method and the translating-pulsating method are minimal [31]. It is worth noting that in the 3D frequency-domain ship motion problem, the speed-dependent oscillatory free-surface Green’s function is extremely complicated to compute compared to its zero-speed case [22,25,32]. WAMIT [33], WADAM [34], and AQWA [31] are examples of three-dimensional panel codes that are implemented in the frequency domain using the wave Green’s function.

The time-domain potential flow methods are often used to account for nonlinearity sources in seakeeping analysis. These methods can be categorized as Froude–Krylov nonlinear methods and body-exact methods [7]. Similar to linear frequency domain formulation, the hydrodynamic disturbance potentials are approximately evaluated over the still water-line wetted body surface in the nonlinear Froude–Krylov methods. However, the fluid memory effects are incorporated through the application of convolution integrals of impulse response functions (IRFs) based on the Cummins [35] formulation to account for hydrodynamic diffraction and radiation forces in the time domain, while the nonlinear Froude–Krylov and restoring forces are exactly evaluated over the time dependant wetted body surface below the incident wave. Fonseca and Guedes Soares [22,36] implemented a partially nonlinear time-domain strip theory code to account for the non-linearities of hydrostatic and Froude–Krylov forces. This code was shown to produce reasonably good results even with abnormal waves [37]. Ballard et al. [38] presented a similar formulation in the time domain in which impulse response functions are calculated using frequency domain results determined by a three-dimensional panel method, and the non-linear incident wave and restoring forces are calculated over the instantaneous underwater surface. The performance of several other codes using similar approximations is shown in [39].

In the body-exact approach, the hydrodynamic problem is solved directly in the time domain. The body boundary condition is satisfied on the instantaneous wetted surface of the body while the linearized free surface boundary condition is retained on the mean free surface. Many researchers claimed that evaluating the forward speed Green function in the time domain is computationally simpler than in the frequency domain [7,32,36,40,41,42]. Lin and Yue [43] applied a transient Green function to satisfy the exact body boundary condition on the instantaneous free surface, while the free surface boundary condition remained linearized on the calm water surface. Lin et al. [44] extended this method to solve for a free surface boundary condition linearized about the incident wave. Datta and Sen [45] and Datta et al. [32] proposed a similar method to evaluate the seakeeping characteristics of various hull forms advancing with a constant forward speed. Datta and Guedes Soares [46] presented a coupled BEM–FEM solver with the free surface transient Green function to estimate the hydroelastic response of a container ship.

Rajendran et al. [47] extended the strip theory code proposed by Fonseca and Guedes Soares [22,36] to include body nonlinearity in the calculation of radiation and diffraction forces. Additionally, surge motion is incorporated through a semi-empirical method to investigate its effect on vertical responses. This numerical code was further extended by Rajendran et al. [48], to analyse hydroelastic loads on a ship in large amplitude waves.

Several authors [49,50,51] applied the Rankine source method in the time domain. Kring and Sclavounos [52] extended this method to a non-linear solution of ship responses and wave-induced loads. Kim and Kim [53] introduced the time-domain 3D Rankine panel program WISH to estimate the nonlinear responses. Later, the solver was extended to WISH-FLEX to consider hydroelastic effects on hull-girder structure [54]. Liu and Papanikolaou [55] created a nonlinear time domain hybrid method that combines a transient Green’s function method and a Rankine source method to simulate large amplitude motions. WASIM is an example of a time domain seakeeping code that is implemented based on the Rankine source method [56]. Luo et al. [57] extended WASIM’s applicability to shallower waters and steeper waves by incorporating additional non-linearities in the incident waves and the free-surface boundary conditions. Pan et al. [15] further developed WASIM to calculate the ship-added resistance in waves. Fully non-linear formulations are also available, although still too heavy computations for routine applications [58,59].

Recently, Reynold Averaged Navier Stokes Computational Fluid Dynamics RANS CFD methods are used for seakeeping studies to account for viscous effects [60,61,62]. Nam et al. [63] studied the seakeeping performance of a barge using a CFD-Modified potential flow hybrid model. The CFD methods are, however, outside the scope of this study. The aforementioned models demonstrate that there are numerous levels of complexity in seakeeping numerical approaches. These seakeeping models may not always produce identical results for the same ship and test conditions. Therefore, uncertainties will exist during the predictions of the transfer function obtained by various numerical codes.

Several benchmark studies have been conducted to evaluate the accuracy of various seakeeping codes in the computations of wave-induced motions and loads. Schellin et al. [64] determined, based on an analysis of the wave-induced loads on a fast container ship, that neither the strip theory nor the panel method produced more accurate results when compared to experiments. Bunnik et al. [65] concluded, based on a comparison of the state-of-the-art seakeeping prediction tools for a container ship and a ferry, that CFD models do not outperform linear potential codes as long as there are no strong nonlinearities or viscous effects. The outcomes of the study revealed substantial discrepancies between theoretically identical mathematical models. Therefore, not only the model selection but also the implementation details are crucial. Kim and Kim [66] presented a benchmark evaluation of the performance of seventeen seakeeping analysis codes for estimating linear and nonlinear responses of a container ship. It was determined that theoretical constraints may be linked to the existence of scattered responses in larger waves or at high forward speeds. Husser and Brizzolara [67] presented a comparative analysis of various fidelity methods for estimating ship responses and loads on a container ship in waves. Parunov et al. [68] presented a benchmark study and uncertainty assessment for numerical predictions of linear wave loads on a damaged surface combatant ship. Ley and El Moctar [69] compared the performance of several advanced nonlinear numerical seakeeping codes for evaluating the wave-induced motions, loads, and hydroelastic effects of various ship types in regular and random irregular severe waves. Potential flow theory codes were found to deviate significantly from measurements in large waves, while Unsteady Reynolds Averaged Navier-Stokes URANS-Unsteady Reynolds Averaged Navier-Stokesbased predictions agreed with measurements at much higher computation times. The majority of the aforementioned comparative studies, however, relied solely on visual comparisons of results or a single uncertainty measure. Parunov et al. [70] conducted a benchmark study based on fifteen numerical seakeeping codes provided by seven institutions. The benchmark study presented a theory-based quantified uncertainty assessment for the Flokstra ship’s heave and pitch motions at the centre of gravity, as well as vertical bending moments at amidship.

Numerous methods have been developed for quantifying the model uncertainty of transfer functions. Guedes Soares [71] introduced the frequency-independent model error (FIME) to assess the systematic bias between measurements and predictions of wave-induced responses. Jafaryeganeh et al. [72] used the linear frequency-dependent model error presented by Guedes Soares [71] to investigate the uncertainties in wave-induced bending moment transfer functions derived by a three-dimensional linear panel method.

Guedes Soares [71] proposed estimating uncertainty based on the integration of the area under the response spectrum to represent the shape of the transfer function and the wave spectrum as well as the relative position between their peaks which eliminates individual frequency sensitivity in transfer functions. Ando [73] predicted the total factored error (TFE) between observed and predicted responses by weighting the error of the predicted transfer function at a specific wave frequency based on its significance in the wave spectrum. However, this uncertainty measure does not indicate whether seakeeping codes underestimate or overestimate experimental results. In addition, the wave spectra themselves and the associated peak periods are uncertain [74,75]. Guedes Soares [75] showed that it is important to quantify the uncertainty in the response variance due to uncertainties in wave spectrum shape. Later, this method was utilized for wave energy converters [76] and floating wind turbine platforms [77]. The uncertainty of the long-term distribution of the wave-induced bending moments in ships was studied by Guedes Soares and Moan [4] using a weighted probability of occurrence of the various sea states that a ship may encounter during its lifetime. Guedes Soares and Trovão [78,79] showed that wave climate uncertainty may have a significant effect on the long-term predictions of wave-induced ship structure responses. Recently, Hirdaris et al. [80] investigated the uncertainties in hull girder response and wave load predictions that are influenced by flexible fluid-structure interactions.

The International Towing Tank Conference (ITTC) provided guidelines on the verification and validation of frequency-domain and time-domain seakeeping numerical codes [9]. Guedes Soares [81] emphasised the importance of quantifying and reducing experimental uncertainties before validating numerical models. ITTC [82] established guidelines for quantifying the various sources of uncertainty in seakeeping experiments. These guidelines have been applied in various studies that looked at the experimental uncertainty in various types of model tests [83,84,85,86]. Abdelwahab and Guedes Soares [87] presented a comprehensive assessment of experimental uncertainty in a physical model of a moored tanker in waves following the ITTC guidelines.

Kim and Kim [66] presented the total difference (TD) measure as a method for determining the degree to which an individual numerical model deviates from the overall average of all models in a benchmark study. Since it is completely unconnected to experimental results, this measure does not represent numerical model bias. A new uncertainty measure is proposed here, termed modified total difference, which uses the experimentally measured transfer functions as the best estimate at each frequency rather than the average of computations of all numerical models adopted in [66].

In addition to proposing a new uncertainty measure, this paper presents a code-based quantified uncertainty assessment of the motions and sectional loads caused by regular wave action on a fast container ship. It is part of a larger benchmark study that was carried out within the scope of the ISSC–ITTC joint committee on uncertainty modelling in waves and wave-induced responses [70].

This paper emphasizes the significance of employing multiple uncertainty measures in comparative studies to precisely characterize the accuracy of different numerical codes. In this paper, five different seakeeping codes are used to perform numerical simulations at three different wave heading angles. This research extends the findings of the benchmark study by reporting several new results such as vertical relative motions at the bow, vertical and horizontal shear forces, torsional moments, and horizontal bending moments at three different locations in the Flokstra ship (stern, midship, and bow), as well as vertical bending moments at bow and stern. The numerical simulation results are then compared to experimental data. Finally, multiple measures are used to assess the uncertainty in individual codes concerning both available experimental results and the mean of the computed results, in this way demonstrating the applicability of the new uncertainty measure proposed in this study.

2. Comparative Study

2.1. Model Tests and Mass Distribution

With a full set of model test measurements necessary for comparisons publicly accessible in the literature [88,89], the container ship Flokstra is selected for this study, as in [70]. Figure 1 depicts the ship’s body plan, and

Table 1. Main characteristics of the container ship Flokstra.

Specifications	Units	Flokstra Ship (Full Scale)
Length (Lpp)	m	270
Breadth (B)	m	32.2
Draft (T)	m	10.85
Displacement volume (∇)	m3	56,097
Longitudinal centre of gravity 1 (LCG)	m	10.12
The vertical centre of gravity (KG)	m	13.49
Metacentric height (GMT)	m	1.15
Pitch radius of gyration (Kyy)	m	0.248 Lpp
Roll radius of gyration (Kxx)	m	0.375 B
Natural heave period (Tz)	s	8.7
Natural pitch period (Tθ)	s	8.6
Natural roll period (Tϕ)	s	24.9

¹ Measured aft of mid-ship.

provides a summary of the characteristics of the investigated full-scale ship. The experiments were conducted at the seakeeping facility of the Maritime Research and Innovation Netherlands (MARIN), formerly the Netherlands Ship Model Basin (NSMB).

On a scale of 1:55, the model was fabricated as a free-running self-propelled model made from glass-reinforced polyester. It was divided into four segments that were linked by three strain gauges installed near stations 5 (at the aft engine room bulkhead), station 10 (amidships), and station 15 (a quarter ship length forward of amidships). The intersegmental gaps were sealed. The model was ballasted following the mass distribution depicted in Figure 2, which was comprised of masses, mass centres, and roll and pitch radii of gyration for each of the four segments.

The self-propelled model was tested with a bilge keel while being guided by an automated rudder. The tested speed corresponded to a Froude number Fr = 0.245. The experiments were conducted in 7 regular waves whose lengths ranged from λ/L_pp = 0.35 to 1.4 and a wave height of 1/60 of ship length, which is not very small compared to the draft [89]. Although the tests were conducted at various heading angles, only three headings are reported in this paper as μ = 45° quartering stern waves, 180° head waves, and 225° oblique bow waves. Heave and pitch motions at the centre of gravity, as well as vertical relative motions measured concerning the undisturbed wave at station 20 in the bow, are among the model test results used in this study. Furthermore, the strain gauges’ measurements of shear forces and bending moments in both vertical and horizontal planes, as well as torsional moments are used. Another experimental study for the same ship conducted by China Ship Scientific Research Centre (CSSRC) at a scale of 1:80, produced the results for heave and pitch motions at a lower speed that corresponds to Fr = 0.1 [90].

2.2. Numerical Simulations

The study employs five numerical codes (PDSTRIP, MAXSURF, IST-CENTEC, ANSYS-AQWA, and HydroD-WASIM) to predict wave-induced motions and sectional loads on the aforementioned container ship that is moving with a forward speed. Strip theory, 3D frequency-domain method, and 3D time-domain method are the three distinct types of numerical codes used in the comparative study. Flokstra’s body lines shown in Figure 1. are used to model the 3D geometric model in IGES format, and the mass is determined in all numerical codes using the distribution shown in Figure 2. All numerical simulations are conducted on full-scale without the bilge keel. While some of the codes only show results for motions, others show results for both motions and loads, as shown in

Table 2. List of numerical codes utilized in the study.

Numerical Code	Method	Results
PDSTRIP	Strip Theory	motions
IST-CENTEC	Strip Theory	motions and loads
MAXSURF	Strip Theory	motions
ANSYS-AQWA	3D BEM (WGF)	motions and loads
HydroD-WASIM	3D BEM (Rankine)	motions and loads

.

2.2.1. PDSTRIP Simulations

The open-source code PDSTRIP is a seakeeping numerical code that was developed by Söding [20] to compute the response of monohull ships based on the 2D strip theory method. The strip theory method assumes that the fluid flow can be represented by a series of 2D problems in the crossflow plane due to the ship’s slenderness. The numerical code is implemented in the frequency domain, and 21 strips with equal spacing are extracted from the 3D hull model of the Flokstra ship.

2.2.2. MAXSURF Simulations

MAXSURF-Motions is a commercial seakeeping analysis software proposed by Bentley Systems [21] as a part of the MAXSURF software suite. It provides rapid and reliable estimation of vessel response and seakeeping characteristics for a variety of floating bodies. There are two methods for calculating the vessel’s response, the linear strip theory method, and the panel method. In this study, the 3D hull form is imported into the software, and simulations are performed using the linear strip theory approach described by Salvesen et al. [16] to determine the coupled heave and pitch response of the vessel at its centre of gravity.

2.2.3. IST-CENTEC Simulations

The IST-CENTEC code is a frequency-domain numerical code developed in-house at Instituto Superior Técnico (IST) as a linear version of Fonseca and Guedes Soares [22,36]. The numerical code utilized the 2D linear strip theory proposed by Salvesen et al. [16] to predict wave-induced motions and vertical sectional loads. The hydrodynamic coefficients are computed using the Frank Close Fit method, which divides the hull shape into 21 transverse strips with equal spacing.

2.2.4. AQWA Simulations

AQWA is commercial software developed by Ansys [31] to cover the majority of analysis requirements related to hydrodynamic evaluation for floating and fixed offshore and marine structures. The software utilizes a pulsating wave Green’s function (WGF) source over the mean wetted surface, as well as a forward speed correction in the body boundary condition, to numerically solve the diffraction and radiation potential components in the frequency domain using the three-dimensional panel method. It also accounts for the forward speed effect in the Bernoulli equation when calculating wave exciting forces, additional mass, and damping coefficients. AQWA employs a Green function database to efficiently compute the Green’s function in finite depth and its first-order derivatives.

As shown in Figure 3, when the IGES 3D model is imported into AQWA, the mean wetted surface body is discretized into 2780 panels with a maximum element size of 3.5 m to account for the shortest wavelength. Since the pulsating Green’s function source satisfies the zero forward speed free surface condition, there is no need to apply a mesh to the free surface. The mass is distributed in the form of four mass points. However, to estimate accurate ship motions and sectional loads, the available point masses are utilized to assume a uniformly distributed mass along the length affected by each point of mass while keeping the overall centre of gravity and roll radius of gyration constant. The viscous roll damping in the numerical simulations is calibrated by 20% of the critical damping to agree with experimental results.

2.2.5. WASIM Simulations

The current study employs the HydroD-WASIM model developed by Det Norske Veritas DNV -Det Norske Veritas [56], which applies a linear three-dimensional time-domain Rankine panel approach to predict motions and sectional loads for the Flokstra ship at a forward speed. The Rankine panel approach accounts for the forward speed in its formulation. The method relies on a Rankine source that does not satisfy any boundary conditions and requires only the Laplace equation to be solved. Therefore, Rankine sources are distributed on both the hull and the free surface except the sea bottom, which can be satisfied by mirroring. The increase in CPU cost caused by a large number of unknowns on all boundaries is less noticeable as it is easy to calculate. The radiation/diffraction problem is estimated using linear theory, and the corresponding pressures are integrated along the mean wetted surface, considering the quadratic term in the Bernoulli equation included. The restoring and Froude–Krylov pressures are computed on the instantaneous wetted surface below the incident wave.

The discretization meshes are created on both the hull of the Flokstra ship and the free surface as shown in Figure 4. Convergence analysis is performed in WASIM at different mesh densities. The 3D hull is discretized into 2300 panels on the mean wetted surface body. This mesh guarantees at least eight panels on the ship hull at the shortest wavelength. The free surface mesh is extended 5 times the length of the ship to ensure that it accounts for the longest radiated wave, while 42 panels are used on the free surface near the hull to account for the shortest wave.

Due to the lack of stiffness in the horizontal motions, there is no mechanism to prevent the ship from drifting away in the time domain simulation. Therefore, WASIM employs an artificial active rudder and autopilot model to control sway and yaw motions. The autopilot model is configured to respond to motions with frequencies close to the natural frequencies for motions, as shown in the following equation.

$${\delta }_R=k_1{x}_6+k_2{\dot{x}}_6+k_3{x}_2+U{k}_3{\int }_0^t{x}_{6}dt,$$

(1)

where δ_R is the rudder deflection, x₂ is the sway motion, x₆ is the yaw motion and ${\dot{x}}_6$ is the yaw angular velocity, k₁ = 1, k₂ = 100 and k₃ = 0.01.

The simulations are performed in regular waves with a wave height of 1/60 of the ship length. The total simulation time is 300 s, with a 50 s transient and a 0.2 s time step. The numerical simulations are calibrated with the non-dimensional roll-damping coefficient obtained from experiments.

2.3. Uncertainty Measures

This section describes the various approaches used in this paper to quantify the model uncertainty. The calculated linear transfer functions acquired by different numerical models are compared with measurements or the average of the results obtained from different numerical codes to quantify model uncertainty. The investigated transfer functions represent the amplitude of motions or sectional loads induced on a ship by unit amplitude waves. Three uncertainty measures are used in this study. First, the Frequency Independent Model Error FIME proposed by Guedes Soares [71] with the associated coefficient of determination CoD. The total difference measure TD^M proposed by Kim and Kim [66] is then evaluated. Lastly, this study proposes a new uncertainty measure, modified total difference TD^E, to quantify the uncertainty of individual seakeeping codes against available experimental results.

2.3.1. Frequency-Independent Model Error

The relationship between calculated $H$ and measured $\widehat{H}$ transfer functions at each encounter frequency ${\omega }_i$ was observed by Guedes Soares [71] to describe various common scenarios as follows:

$$\widehat{H}\left({\omega }_j\right)=\varphi \left({\omega }_j\right)·H\left({\omega }_j\right)+ϵ\left({\omega }_j\right),$$

(2)

where $\varphi$ represents the model error of theoretical predictions, $ϵ$ is the random error in measurements with a zero mean value and j denotes j-th measurement frequency.

It is possible to continue using a theory or a numerical model and even improve its predictions if the model error is known. Guedes Soares [71] suggested that the model error $\varphi$ can be formulated in a general form as follow:

$$\varphi \left({\omega }_j\right)=a+b{\omega }_j+c{\omega }_j^2,$$

(3)

where a, b, and c are constants that define the accuracy of the model error.

The preceding equation demonstrates that there are multiple ways to express model error. It can be expressed as a full quadratic function of the encounter frequency if, for instance, the numerical model overpredicts measurements at both low and high frequencies and underpredicts measurements in the middle range of frequencies. If the constant c is set to zero, the model error may have a simple linear relationship with frequency, such as when the numerical results underpredict the measurements at low frequencies and gradually overpredict them at high frequencies by a growing amount.

The simplest formulation of the model error of the transfer functions can be expressed as a constant that is independent of encounter frequencies, as demonstrated by Guedes Soares [71]. This holds when the theoretical curve of the model predictions can be adjusted to coincide with the mean curve of the measurements by multiplying the model predictions by a constant factor $\varphi =a$, as shown in the following equation:

$${\widehat{H}}_j=a{H}_j+{ϵ}_j,$$

(4)

where ${\widehat{H}}_j$ are measured transfer functions, $H_j$ is the predicted transfer function and ${ϵ}_j$ is the random error in experimental measurements at each frequency ${\omega }_j$.

The experimental error ${ϵ}_j$ can be represented by the variation of the experimental measurements around the adjusted curve. The difference between the original and adjusted curves of the model predictions represents the systematic error or the average bias of the numerical model known as the frequency-independent model error FIME. Guedes Soares [71] showed that it can be approximated by minimizing the sum of the squares of the experimental errors. The FIME of a theory or a numerical model can be quantified as shown in the following equation:

$${\widehat{a}}_i=\frac{{{\displaystyle \sum }}_j\left|{\widehat{H}}_j\right|\left|H_{ij}\right| }{{{\displaystyle \sum }}_j{\left|H_{ij}\right|}^2},$$

(5)

where i represents the applied numerical model.

The benefit of employing this uncertainty measure is that it provides immediate feedback on whether the theory or numerical model overestimates or underestimates measurements. As indicated by the aforementioned equation, a FIME value greater than 1 indicates that the numerical model underestimates measurements, while a FIME value less than 1 indicates that the numerical model overestimates experimental results.

As previously stated, FIME represents the average bias between experimental and numerical model results. A comprehensive uncertainty description requires, in addition to bias, random dispersion of experimental measurements around the regression line represented by FIME. The coefficient of determination CoD is thus used to express this dispersion or variance in the error to assess how accurately the regression prediction matches the discrete data points as shown in the following equation:

$$R_i^2=1-\frac{{{\displaystyle \sum }}_j{\left({\widehat{a}}_i{H}_{ij}-{\widehat{H}}_j\right)}^2}{{{\displaystyle \sum }}_j{\left({\widehat{a}}_i{H}_{ij}-\overline{H}\right)}^2},$$

(6)

where $\overline{H}$ is the average measured transfer function across all frequencies.

When the coefficient of determination $R^2=1$, the regression predictions perfectly match the data, and all measurement points lie on the regression line. In this instance, FIME is adequate to represent the model uncertainty. In other cases, $R^2$ decreases when measurement points are distributed around the regression line, indicating that random uncertainty exists in addition to the systematic error provided by FIME for the estimated transfer functions. A baseline model always forecasts the average of the measurements $\overline{H}$ and has $R^2=0$. Models that perform worse than this baseline model will have a negative $R^2$. In practice, a coefficient of determination greater than 0.9 can be considered an excellent fit to data points, and the calculated FIME is an appropriate measure of uncertainty.

Estimating the parameters of a frequency-dependent model, such as the linear model error or the quadratic model error model, necessitates large experimental data to obtain a more accurate representation of uncertainty. In this work, we adopt only the frequency-independent model error FIME and its associated coefficient of determination CoD due to the limited availability of experimental data (only seven measurements).

2.3.2. Total Difference Approach

Kim and Kim [66] introduced the total difference as a measure of uncertainty in the computational results of transfer functions obtained by different numerical codes in a benchmark study on a container ship. The total difference measure takes into consideration both bias and dispersion of individual transfer functions. It is a measure used to evaluate the deviation of an individual numerical model from the mean value of results obtained by all numerical models.

The mean value of calculated transfer functions $H_{ij}$ by N numerical models at each frequency can be obtained as shown below.

$$Mea{n}_j=\frac{{{\displaystyle \sum }}_{i=1}^N{H}_{ij}}{N},$$

(7)

where i denotes the result obtained from the i-th numerical model at frequency j.

The total difference uncertainty measures of the i-th individual numerical codes to the mean value of predictions from all numerical codes at given frequency j are defined as a percentage following the equation below.

$$T{D}_i^M=\frac{{{\displaystyle \sum }}_{j=1}\left|H_{ij}-Mea{n}_j\right|}{{{\displaystyle \sum }}_{j=1}Mea{n}_j}\times 100,$$

(8)

The definition of the total difference as a percentage yields this uncertainty measure uninformative regarding whether the applied numerical code underestimates or overestimates the results.

2.3.3. The Modified Total Difference Approach

Since the total difference uncertainty measure only considers the calculated transfer functions and disregards the measured ones, it lacks information beyond visual comparisons about the discrepancies of computations or its mean with respect to experimental measurements. In addition, it is challenging to distinguish between different numerical codes or to identify the primary source of mean differences when compared to experimental measurements. Moreover, it is challenging to precisely evaluate a single numerical model when the dispersion of numerical results is large.

This study proposes a new approach, called modified total difference, to quantify the uncertainty of individual seakeeping codes against experimental measurements by assuming the measured experimental transfer function as the best estimate at each frequency rather than the mean value of all numerical models, as shown below.

$$T{D}_i^E=\frac{{{\displaystyle \sum }}_{j=1}\left|H_{ij}-{\widehat{H}}_j\right|}{{{\displaystyle \sum }}_{j=1}{\widehat{H}}_j}\times 100,$$

(9)

where ${\widehat{H}}_j$ is the measured transfer function at frequency j.

This uncertainty measure has the advantage of revealing discrepancies between numerical methods and experimental measurements. It does not, however, provide information about understated or overstated predictions. The general drawback of the Total Difference measures shown in Equations (8) and (9) is that they vary with the number of calculation points (e.g., measured frequency components), because of their cumulative nature. Consequently, only the Total Difference of transfer functions with the same number of measurement frequencies can be compared mutually. It should be noted that FIME is not directly sensitive to the number of measurement frequencies as it represents a regression line passing through the origin regardless of the number of calculation points [3].

3. Results and Discussion

3.1. Linear Response Comparisons

This section compares the results of the model test with linear transfer functions obtained by numerical simulations in regular waves using five different numerical codes. The full-scale simulations were conducted on the container ship Flokstra, which has a draft of 10.85 m and is moving at a speed corresponding to Froude number Fr = 0.245. Part of the results is provided at a slower speed, corresponding to Fr = 0.1.

Transfer functions are presented for heave and pitch motions about the centre of gravity, vertical relative motion (VRM) at the bow, and wave-induced sectional loads such as vertical and horizontal shear forces (VSF and HSF), vertical and horizontal bending moments (VBM and HBM), and torsional moment (TM) at three stations 5, 10, and 15. The numerical simulation results are presented at three different heading angles. Moreover, calculated results are averaged and compared to experiments. For horizontal shear forces, horizontal bending moments, and torsional moments, the results obtained from AQWA and WASIM are compared with experiments for only two wave headings.

The transfer functions are presented in nondimensional form as follows: heave x₃ and vertical relative motions per wave amplitude ζ and pitch motion x₅ per wave slope kζ. Furthermore, sectional forces and moments are nondimensionalized by the terms ρgζBL and ρgζBL², respectively, where ρ is the density of water, g acceleration of gravity, and B ship breadth. The nondimensional transfer functions are plotted versus the non-dimensional wavelength, defined as λ/L_pp, where λ is the wavelength, while L_pp is the ship length between perpendiculars. Figure 5 shows the non-dimensional heave, pitch, and vertical relative motions, whereas Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10 show the non-dimensional sectional loads (VSF, HSF, VBM, HBM, and TM) at stations 5, 10, and 15.

The visual comparison reveals a reasonable approximation in the general behaviour of the computational result depicted in Figure 5. Due to a reduction in excitation forces when buoyancy forces alternate along the hull and the increased significance of dynamic effects at high frequencies, results in head waves reveal a low response to short waves. The motion approaches unity in long waves, which can be explained by the fact that the motion is entirely attributable to buoyancy changes caused by the passage of the waves over the hull. Thus, the ship behaves similarly to a water particle on the surface, with heave motion equal to wave amplitude ζ and pitch equal to wave slope kζ. In long oblique waves, the pitch motion is equal to the effective wave slope amplitude kζ cosμ.

Figure 5 demonstrates the transfer functions of the VRM approach unity in short waves, indicating that the wave motion is the only significant contributor to the relative motion, as the ship appears stationary. Sharply peaked resonances occur at intermediate frequencies, most likely due to synchronisation at the bow between the upward absolute motion and wave depression, producing the largest relative motion at these frequencies.

The average of predicted motions agrees better with experiments in head seas than in oblique waves as shown in Figure 5. Predicted pitch motions, on average, are more consistent with the experiment than heave motions. The majority of numerical codes significantly underestimate the heave motion in oblique stern waves.

All numerical codes give results that agree well with the measured motions in the vast majority of the simulated cases when λ/L ≤ 0.75. whereas the numerical codes tend to overestimate motion at lower frequencies. The motions obtained by the PDSTRIP code generally show the smallest deviation from experiments when compared with other numerical codes in head waves.

The sectional load results are calculated using three numerical models: IST-CENTEC, AQWA, and WASIM. The comparison of computed VSF presented in Figure 6 displays a broad scatter of computational outcomes. Comparing the averages of the calculated VSF results reveals a greater tendency of agreement with experiments in head and oblique stern waves than in oblique bow waves. In addition, the average calculated results obtained at stations 10 and 15 are more in line with the experiments than those obtained at station 5. Among other numerical models, WASIM’s results for vertical shear force show the smallest deviation from experimental results.

Horizontal shear force predictions are also compared with model test data in this study, as shown in Figure 7. Since nonlinear effects due to bow and stern shape tend to be less significant under horizontal loads, the average numerical predictions appear to be closer to the model test for horizontal shear force than for vertical shear force. The results show that WASIM’s numerical predictions for horizontal shear forces are more in line with experiments than those obtained using AQWA.

Figure 8 illustrates the significant discrepancy in the calculated vertical bending moments, particularly at peaks, as different numerical models do not necessarily confirm these peaks in the same manner. Experimental results indicate that the greatest vertical bending moments occur amidships in head seas. The general agreement between VBM computations and measurements is insufficient.

The average predictions of the VBM correlate reasonably well with model test results in oblique stern waves, while measurements in head and oblique bow waves were underestimated. The underestimation of the bending moment predictions may be due to linear numerical model assumptions that do not account for nonlinear effects such as instantaneous wetted surface bodies, which were evaluated during model testing. Results in Figure 8 indicate that it is challenging to determine which of the applied numerical models provides the most accurate estimate of the vertical bending moment.

The findings in Figure 9 demonstrate that the peaks in the horizontal bending moment are related to the natural frequency of roll motion. The agreement for the horizontal bending moment appears to be slightly better than for the vertical bending moment Wasim’s predictions for HBM, especially at midship, are generally superior to AQWA’s.

The calculated torsional moments in Figure 10 have less discrepancy than the other sectional loads. Particularly in the midship region, the average of the calculated torsional moments agrees better with experiments. In oblique bow waves, the torsional moment reaches its maximum value. It should be noted that viscous roll damping is included in the numerical simulations, which influences the roll motion at resonance and, consequently, the peaks of the torsional moments. AQWA appears to correlate better with measured midship torsional moments in oblique bow waves.

In general, visual observation of the sectional loads reveals that the discrepancies between the calculated results depicted in Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10 are larger than that in the predicted motions indicated in Figure 5.

Since large discrepancies are observed between various numerical codes for motions and loads at a speed corresponding to Fr = 0.245, the speed effect may be responsible for these discrepancies. Fortunately, experimental measurements at a lower speed for heave and pitch motions on the Flokstra ship are available in the literature as part of another model test campaign [90]. Therefore, this study simulated in full-scale the same Flokstra container ship at a slower speed, corresponding to Fr = 0.1, to determine if the simulated ship speed could affect the discrepancies in wave-induced motions and loads.

Wave-induced motions and loads predicted by various numerical models at the slower Fr = 0.1 speed are averaged and compared with measured data in Figure 11 and Figure 12, respectively. Heave and pitch motions in head waves show better agreement with the measurement at low speed compared to results at high speed. However, relying on visual comparisons alone can make it difficult to draw firm conclusions about speed’s impact.

3.2. Comparisons of Uncertainty Measures

Numerical codes and theories may not be evaluated accurately through visual comparisons between calculated results or their average and experiments. Quantitative analysis is the only method for determining the degree to which variations in seakeeping numerical codes will affect the linear transfer function calculations. The reliability of various codes can be compared with the help of model test data. Consequently, it is convenient to assess the consistency of numerical codes with experiments using quantified uncertainty analysis.

Having established systematic approaches for dealing with uncertainty in linear transfer functions, the current study addresses multiple measures for uncertainty (FIME, CoD, TD^M and TD^E) in transfer functions computed using the aforementioned five numerical codes. Due to the limited number of applied codes utilized in the study, it is not possible to investigate the uncertainty in the transfer function based on groups employing the same seakeeping method or theory.

Various seakeeping numerical codes may have different uncertainties when compared to model test results. Even numerical codes that may share a common theoretical background may exhibit different uncertainties due to different mathematical model assumptions. Furthermore, multiple users may treat the same numerical code differently. Consequently, it is beneficial to quantify the uncertainty in transfer function with a variety of approaches based on comparisons of the individual numerical codes with either experiment (${\widehat{a}}_i, R_i^2, T{D}_i^E$) or the mean of the predictions obtained by all numerical codes ($T{D}_i^M$). In addition, they can be determined by comparing the mean of all numerical codes with experimental results $\left({\widehat{a}}_{mean}, R_{mean}^2,T{D}_{mean}^E\right)$.

The average of any uncertainty measure at multiple heading angles may also provide an overall measure of the discrepancy between numerical code and measurements, termed the code average index, which is associated with the quality or the accuracy of the code. Although these uncertainty measures are applicable to all the response components, the scope of this study is limited to heave, pitch, VRM, VSF, HSF, VBM, HBM, and TM transfer functions.

Table 3. Measures of uncertainty in heave and pitch transfer functions obtained by different codes at Fr = 0.245.

Measure	Heading	AQWA		WASIM		IST-CENTEC		PDSTRIP		MAXSURF		Mean of Codes
		Heave	Pitch	Heave	Pitch	Heave	Pitch	Heave	Pitch	Heave	Pitch	Heave	Pitch
$\widehat{a}$	45°	1.28	0.86	1.21	1.04	1.23	1.04	0.99	0.81	1.07	0.66	1.15	0.86
$R^2$		0.93	0.99	0.96	0.98	0.97	0.96	0.99	0.99	0.98	0.97	0.99	0.98
$T{D}^E$		32.77	16.57	23.82	7.97	24.64	11.00	7.40	27.02	9.98	53.97	17.83	19.17
$T{D}^M$		17.52	6.86	10.08	16.75	9.40	16.11	23.20	7.02	12.62	29.74	-	-
$\widehat{a}$	180°	0.88	0.92	0.90	0.82	0.92	0.88	0.97	1.01	0.86	0.86	0.91	0.90
$R^2$		0.96	0.98	0.95	1.00	0.96	0.97	0.96	0.98	0.95	0.97	0.96	0.98
$T{D}^E$		17.49	14.27	17.80	24.39	16.85	20.74	15.75	12.36	20.86	22.18	15.42	17.43
$T{D}^M$		4.36	3.24	4.75	12.12	5.50	3.55	12.08	12.37	7.84	4.86	-	-
$\widehat{a}$	225°	0.82	0.86	0.83	0.78	0.82	0.83	0.78	0.85	0.87	0.77	0.83	0.82
$R^2$		0.95	0.98	0.96	0.99	0.96	0.96	0.98	0.95	0.98	0.95	0.97	0.97
$T{D}^E$		24.24	18.72	21.06	29.48	24.77	24.83	24.90	24.28	16.08	35.12	21.79	26.42
$T{D}^M$		3.31	6.13	2.54	8.77	4.32	3.24	7.53	6.87	5.19	6.99	-	-
$\widehat{a}$	Code Average Index	0.99	0.88	0.98	0.88	0.99	0.92	0.91	0.89	0.93	0.76	0.96	0.86
$R^2$		0.95	0.98	0.96	0.99	0.96	0.96	0.98	0.97	0.97	0.96	0.97	0.98
$T{D}^E$		24.83	16.52	20.89	20.61	22.09	18.86	16.02	21.22	15.64	37.09	18.35	21.01
$T{D}^M$		8.40	5.41	5.79	12.55	6.41	7.63	14.27	8.75	8.55	13.86	-	-

and

Table 4. Measures of uncertainty in vertical relative motion (VRM) obtained by different numerical codes at Fr = 0.245.

Measure	Heading	AQWA	WASIM	PDSTRIP	Mean of Codes
$\widehat{a}$	45°	1.04	1.02	0.67	0.95
$R^2$		0.76	0.81	−0.25	0.01
$T{D}^E$		8.11	8.69	57.32	19.69
$T{D}^M$		19.91	17.16	37.08	-
$\widehat{a}$	180°	0.84	0.73	0.93	0.83
$R^2$		0.97	0.99	0.84	0.98
$T{D}^E$		19.60	32.64	18.56	21.58
$T{D}^M$		3.11	12.58	13.04	-
$\widehat{a}$	225°	0.85	0.75	0.85	0.82
$R^2$		0.98	0.91	0.97	0.98
$T{D}^E$		15.93	30.41	18.95	21.76
$T{D}^M$		5.28	10.62	8.12	-
$\widehat{a}$	Code Average Index	0.91	0.83	0.82	0.87
$R^2$		0.90	0.90	0.52	0.66
$T{D}^E$		14.55	23.91	31.61	21.01
$T{D}^M$		9.43	13.45	19.41	-

display the estimated values for the various uncertainty measures that resulted from comparing numerical predictions and measurements for wave-induced motions using Equations (5)–(9).

The estimates of FIME or bias $\widehat{a}$ vary between 0.78–1.28 for heave motion, 0.66–1.04 for pitch motion, and 0.67–1.04 for VRM. The reason for the scatter in the results may be due to the limited number of frequencies at which the results are estimated. However, it may be seen that the associated coefficient of determination R² is greater than 0.9 in all cases except for VRM in quartering stern waves. Therefore, the bias measure $\widehat{a}$ is sufficient to represent the uncertainty in the transfer functions of motions. Pitch motion predictions are generally better than heave motion predictions. Table 3 and Table 4 show that bias and coefficient of determination for pitch motion are closer to unity than for heave and VRM.

The estimates of the total difference for heave, pitch, and VRM transfer functions are also presented in Table 3 and Table 4. The estimated total differences with respect to the average of calculated results obtained by numerical codes TD^M vary between 2.5 to 23.2 % for heave motion, 3.2 to 29.7 % for pitch motion, and 3.1 to 37 % for VRM. On the other hand, the estimates of the total differences with respect to the experimental data (modified total difference) TD^E vary between 7.4 to 32.7 % for heave motion, 7.9 to 53.9 % for pitch motion, and 8.1 to 57.3 % for VRM. It is noticed in many cases that the estimated total difference with respect to the experimental data TD^E is higher than the estimated total difference with respect to the average of calculated results obtained by numerical codes TD^M.

Assessing the uncertainty based solely on TD^M estimations may not reflect the numerical code’s accuracy. Table 3 shows, for example, that PDSTRIP predictions for heave motion in quartering oblique waves yield a TD^M of 23.2% representing the highest uncertainty among all codes, whereas TD^E is 7.4%, the lowest uncertainty among all codes. Therefore, when experimental measurements are available, it is more reasonable to evaluate the accuracy of numerical models based on comparisons with experiments than comparisons with the average of numerical results. In addition, estimates of the total difference with respect to the average of the computed results may be influenced by a variety of factors, including the number of numerical codes and frequencies used in the study, as well as the accuracy of each code.

Based on the analysis of various measures of uncertainty provided in Table 3 at multiple headings, it is clear that the PDSTRIP code may provide the lowest uncertainty in heave, pitch, and VRM transfer functions in head waves. In quartering stern waves, the PDSTRIP code has the lowest uncertainty for the heave results, whereas WASIM has the lowest uncertainty for the pitch and VRM transfer functions. In oblique bow waves, the MAXSURF code is the most accurate code to predict the heave transfer function, while the AQWA code gives the best predictions for Pitch and VRM transfer functions. The comparison of the mean of numerical codes with experiments indicates that all numerical codes predict heave and pitch motions in head waves better than oblique waves.

Table 3 and Table 4 show the results for the code average index for each uncertainty measure, which indicates the mean value of any uncertainty measure obtained at different heading angles using the same numerical code. According to this index, there is no evidence that one code leads to lower uncertainty than other codes for heave motions at all headings concurrently. The MAXSURF code may give the highest uncertainty by overestimating pitch motions, whereas the AQWA code may predict VRM motions with low uncertainty.

Table 5. Measures of uncertainty in VSF and VBM transfer functions obtained by different codes at station 5 and Fr = 0.245.

Measure	Heading	AQWA		WASIM		IST-CENTEC		Mean of Codes
		VSF	VBM	VSF	VBM	VSF	VBM	VSF	VBM
$\widehat{a}$	45°	0.73	0.71	0.84	0.88	0.70	1.11	0.77	0.88
$R^2$		0.03	0.60	0.88	0.84	0.55	0.96	0.84	0.85
$T{D}^E$		35.74	41.46	19.22	15.79	39.66	10.18	30.13	15.99
$T{D}^M$		19.03	21.96	9.46	2.90	16.81	21.78	-	-
$\widehat{a}$	180°	0.95	1.19	1.20	1.12	0.92	1.38	1.04	1.28
$R^2$		0.35	0.56	0.84	0.14	0.31	0.50	0.51	0.74
$T{D}^E$		37.80	26.64	17.82	16.32	30.15	40.65	21.05	24.19
$T{D}^M$		18.79	10.25	22.93	27.93	13.39	23.66	-	-
$\widehat{a}$	225°	0.68	0.99	1.06	1.11	1.15	1.22	0.97	1.14
$R^2$		−0.56	0.72	0.56	0.15	0.93	0.46	0.17	0.79
$T{D}^E$		56.60	16.05	13.98	18.68	14.99	29.52	19.60	15.09
$T{D}^M$		38.53	14.34	21.69	23.26	18.00	22.48	-	-
$\widehat{a}$	Code Average Index	1.18	1.45	1.55	1.55	1.38	1.85	1.39	1.65
$R^2$		−0.09	0.94	1.13	0.56	0.90	0.96	0.76	1.19
$T{D}^E$		65.07	42.07	25.52	25.39	42.41	40.18	35.39	27.64
$T{D}^M$		38.17	23.27	27.04	27.05	24.10	33.96	-	-

,

Table 6. Measures of uncertainty in VSF and VBM transfer functions obtained by different codes at station 10 and Fr = 0.245.

Measure	Heading	AQWA		WASIM		IST-CENTEC		Mean of Codes
		VSF	VBM	VSF	VBM	VSF	VBM	VSF	VBM
$\widehat{a}$	45°	0.81	0.98	0.95	1.07	0.53	1.00	0.75	1.03
$R^2$		−0.30	0.85	0.98	0.97	0.73	0.67	0.84	0.95
$T{D}^E$		33.23	15.29	7.02	7.46	71.09	14.26	33.34	7.44
$T{D}^M$		27.98	13.07	20.26	5.74	37.10	12.89	-	-
$\widehat{a}$	180°	0.86	1.69	0.89	1.51	0.83	1.05	0.87	1.38
$R^2$		0.05	0.81	0.97	0.79	0.76	0.89	0.83	0.91
$T{D}^E$		20.90	46.83	12.94	35.82	23.46	12.03	17.69	29.89
$T{D}^M$		15.19	24.16	8.06	9.34	11.11	32.63	-	-
$\widehat{a}$	225°	0.61	1.49	0.96	1.56	0.94	1.47	0.91	1.55
$R^2$		−2.73	0.67	0.94	0.60	0.21	0.80	−4.09	0.85
$T{D}^E$		82.76	36.05	10.07	37.25	25.98	35.16	37.01	36.15
$T{D}^M$		40.99	9.24	31.11	19.19	16.17	13.85	-	-
$\widehat{a}$	Code Average Index	0.76	1.39	0.93	1.38	0.77	1.17	0.84	1.32
$R^2$		−0.99	0.78	0.96	0.79	0.57	0.79	−0.81	0.90
$T{D}^E$		45.63	32.72	10.01	26.84	40.18	20.48	29.35	24.49
$T{D}^M$		28.05	15.49	19.81	11.42	21.46	19.79	-	-

,

Table 7. Measures of uncertainty in VSF and VBM transfer functions obtained by different codes at station 15 and Fr = 0.245.

Measure	Heading	AQWA		WASIM		IST-CENTEC		Mean of Codes
		VSF	VBM	VSF	VBM	VSF	VBM	VSF	VBM
$\widehat{a}$	45°	1.12	1.11	0.97	1.31	0.62	0.71	0.90	0.99
$R^2$		0.91	0.06	0.88	0.85	−0.60	0.26	0.47	0.50
$T{D}^E$		11.87	18.47	10.92	24.18	50.00	42.57	16.56	14.80
$T{D}^M$		20.28	14.55	14.97	25.92	30.78	39.31	-	-
$\widehat{a}$	180°	1.64	1.15	1.16	1.68	0.72	0.70	1.07	1.09
$R^2$		0.78	0.57	0.92	0.01	0.83	0.72	0.94	0.73
$T{D}^E$		43.47	35.40	14.53	44.61	40.14	37.31	9.13	20.75
$T{D}^M$		37.80	23.82	11.16	35.36	46.97	52.12	-	-
$\widehat{a}$	225°	1.73	0.90	1.24	1.53	1.20	0.98	1.36	1.16
$R^2$		0.52	0.56	0.67	0.62	0.88	0.39	0.78	0.85
$T{D}^E$		41.23	25.64	21.29	38.03	17.25	20.45	25.75	16.66
$T{D}^M$		21.62	32.00	9.54	27.99	12.28	24.86	-	-
$\widehat{a}$	Code Average Index	2.25	1.58	1.69	2.26	1.27	1.20	1.66	1.62
$R^2$		1.10	0.59	1.24	0.74	0.55	0.69	1.09	1.04
$T{D}^E$		48.29	39.75	23.37	53.41	53.70	50.17	25.71	26.10
$T{D}^M$		39.85	35.19	17.83	44.63	45.01	58.14	-	-

,

Table 8. Measures of uncertainty in HSF, HBM, and TM transfer functions by different codes at station 5 and Fr = 0.245.

Measure	Heading	AQWA			WASIM			Mean of Codes
		HSF	HBM	TM	HSF	HBM	TM	HSF	HBM	TM
$\widehat{a}$	45°	0.87	0.67	0.91	1.03	0.74	0.90	0.97	0.73	0.91
$R^2$		−0.31	−21.24	0.80	0.77	−0.38	0.76	0.22	−4.32	0.82
$T{D}^E$		38.40	50.16	10.11	15.34	43.86	12.55	25.33	40.39	10.91
$T{D}^M$		13.87	13.85	3.91	13.87	13.85	3.91	-	-	-
$\widehat{a}$	225°	0.87	1.06	1.87	1.06	1.54	1.03	0.97	1.30	1.35
$R^2$		0.22	−0.82	0.56	0.92	0.75	−0.53	0.70	0.29	0.21
$T{D}^E$		32.15	34.32	47.79	11.63	37.64	20.31	20.43	31.07	27.63
$T{D}^M$		14.47	21.81	30.21	14.47	21.81	30.21	-	-	-
$\widehat{a}$	Code Average Index	0.87	0.87	1.39	1.04	1.14	0.97	0.97	1.02	1.13
$R^2$		−0.05	−11.03	0.68	0.85	0.19	0.12	0.46	−2.01	0.52
$T{D}^E$		35.28	42.24	28.95	13.48	40.75	16.43	22.88	35.73	19.27
$T{D}^M$		14.17	17.83	17.06	14.17	17.83	17.06	-	-	-

,

Table 9. Measures of uncertainty in HSF, HBM, and TM transfer functions by different codes at station 10 and Fr = 0.245.

Measure	Heading	AQWA			WASIM			Mean of Codes
		HSF	HBM	TM	HSF	HBM	TM	HSF	HBM	TM
$\widehat{a}$	45°	1.10	0.69	0.99	0.94	1.03	0.75	1.02	0.84	0.85
$R^2$		−0.25	−0.66	−0.15	0.41	0.19	−0.42	0.17	−0.41	−0.29
$T{D}^E$		15.44	56.22	18.48	13.67	20.89	33.37	10.07	38.07	20.82
$T{D}^M$		9.60	17.02	14.16	9.60	17.02	14.16	-	-	-
$\widehat{a}$	225°	1.30	0.92	1.08	1.20	1.22	0.91	1.26	1.06	0.99
$R^2$		0.75	0.70	0.95	0.87	0.95	0.91	0.83	0.88	0.95
$T{D}^E$		33.66	24.04	9.85	26.87	19.80	14.30	29.49	15.08	10.29
$T{D}^M$		9.10	15.31	10.04	9.10	15.31	10.04	-	-	-
$\widehat{a}$	Code Average Index	1.20	0.81	1.04	1.07	1.13	0.83	1.14	0.95	0.92
$R^2$		0.25	0.02	0.40	0.64	0.57	0.25	0.50	0.24	0.33
$T{D}^E$		24.55	40.13	14.17	20.27	20.35	23.84	19.78	26.58	15.56
$T{D}^M$		9.35	16.17	12.10	9.35	16.17	12.10	-	-	-

and

Table 10. Measures of uncertainty in HSF, HBM, and TM transfer functions by different codes at station 15 and Fr = 0.245.

Measure	Heading	AQWA			WASIM			Mean of Codes
		HSF	HBM	TM	HSF	HBM	TM	HSF	HBM	TM
$\widehat{a}$	45°	0.87	0.85	1.00	0.96	1.31	0.69	0.93	1.06	0.82
$R^2$		0.25	−2.16	0.96	0.93	0.93	0.70	0.75	0.15	0.86
$T{D}^E$		29.95	44.14	3.33	10.03	23.98	43.40	17.79	27.49	21.47
$T{D}^M$		13.53	21.79	18.05	13.53	21.79	18.05	-	-	-
$\widehat{a}$	225°	1.01	0.91	0.79	1.21	1.29	0.84	1.13	1.08	0.82
$R^2$		−1.37	0.14	0.87	0.96	0.92	0.91	0.48	0.75	0.93
$T{D}^E$		37.66	22.82	26.81	18.51	21.96	19.52	26.44	16.86	22.37
$T{D}^M$		15.62	18.75	10.09	15.62	18.75	10.09	-	-	-
$\widehat{a}$	Code Average Index	0.94	0.88	0.90	1.09	1.30	0.77	1.03	1.07	0.82
$R^2$		−0.56	−1.01	0.91	0.94	0.93	0.81	0.61	0.45	0.89
$T{D}^E$		33.81	33.48	15.07	14.27	22.97	31.46	22.12	22.18	21.92
$T{D}^M$		14.58	20.27	14.07	14.58	20.27	14.07	-	-	-

show a comparison of uncertainty estimates for the transfer functions of loads computed by individual codes at three different stations. The analysis of results at station 10 shows that most FIME values $\widehat{a}$ (0.53 to 0.96 for VSF, 0.98 to 1.69 for VBM, 0.94 to 1.3 for HSF, 0.69 to 1.22 for HBM, and 0.75 to 1.08 for TM) are seen to be dispersed from unity. Moreover, there are several relatively low coefficients of determination (−2.73 to 0.98 for VSF, 0.6 to 0.97 for VBM, −0.25 to 0.87 for HSF, −0.66 to 0.95 for HBM, and −0.42 to 0.95 for TM). It is worth mentioning that the number of codes utilized in the sectional load comparisons is lower than in the motion analysis since not all codes provided results for loads.

The total difference estimates for the transfer functions of wave-induced loads at different stations are also provided in Table 5, Table 6, Table 7, Table 8, Table 9 and Table 10. Again, the majority of estimated modified total differences with respect to experimental data TD^E (7.0 to 82.7 % for VSF, 7.5 to 46.8% for VBM, 13.7 to 33.7 % for HSF, 19.8 to 56.2 % for HBM and 9.8 to 33.4 % for TM transfer functions) are greater than the estimated total differences with respect to the average of results obtained by numerical codes TD^M (8.1 to 40.9 % for VSF, 5.7 to 24.2 % for VBM, 9.1 to 9.6 % for HSF, to 15.3 % 17.2 for HBM and 10.0 to 14.2 % for TM). Since the uncertainty estimates of two codes with the total difference approach are identical and mainly dependent on the average, the newly proposed modified total difference approach has an additional advantage over the total difference measure.

The evaluation of various measures of uncertainty at different heading angles reveals that the WASIM code has minimal uncertainty in VSF at all headings, with a coefficient of determination R² close to one. In quartering stern waves, WASIM also provides the lowest uncertainty VBM transfer functions. In oblique waves, the AQWA code produces lower uncertainty in TM transfer functions versus WASIM and the opposite for HSF and HBM.

The comparison of the mean of numerical codes with experiments based on different measures of uncertainty, as shown in Table 5, Table 6, Table 7, Table 8, Table 9 and Table 10, reveals relatively large uncertainties in the predicted midship wave-induced sectional loads compared to motions shown in Table 3 and Table 4. Moreover, the associated coefficient of determination potentially falls below 0.9. WASIM code can compute VSF at various heading angles with minimal uncertainty based on code average index results. While it is challenging to find a code that may provide minimal uncertainty in VBM at all headings at the same time. WASIM code can give better predictions for HSF and HBM in oblique stern waves when compared with AQWA.

The uncertainty analysis for heave and pitch predictions from different numerical models at a slower speed with Fr = 0.1 are summarised in

Table 11. Measures of uncertainty in heave and pitch transfer functions obtained by different numerical codes at Fr = 0.1.

Measure	Heading	AQWA		WASIM		IST-CENTEC		PDSTRIP		MAXSURF		Mean of Codes
		Heave	Pitch	Heave	Pitch	Heave	Pitch	Heave	Pitch	Heave	Pitch	Heave	Pitch
$T{D}^M$	45°	17.60	9.85	4.49	6.86	14.57	12.43	14.82	2.18	14.37	26.23	-	-
$\widehat{a}$	180°	0.96	0.99	1.00	0.95	1.03	0.96	1.02	1.01	1.00	0.94	1.01	0.97
$R^2$		0.85	0.98	0.83	0.99	0.90	0.97	0.67	0.99	0.83	0.97	0.84	0.98
$T{D}^E$		16.31	10.75	16.07	9.06	14.80	13.02	23.36	9.54	15.12	14.06	14.06	10.79
$T{D}^M$		5.91	2.63	7.81	3.98	10.97	3.15	11.09	5.89	5.35	4.97	-	-
$\widehat{a}$	225°	0.89	0.96	0.88	0.91	0.88	0.95	0.89	0.93	1.02	0.87	0.91	0.92
$R^2$		0.92	0.98	0.95	0.99	0.85	0.98	0.95	0.99	0.94	0.96	0.96	0.98
$T{D}^E$		14.60	7.37	15.95	12.35	18.43	7.97	15.42	9.34	7.43	19.45	11.20	11.09
$T{D}^M$		4.88	4.15	5.18	1.86	7.64	2.93	4.17	2.53	16.21	7.53	-	-
$\widehat{a}$	Code Average Index	0.93	0.98	0.94	0.93	0.95	0.96	0.95	0.97	1.01	0.91	0.96	0.95
$R^2$		0.89	0.98	0.89	0.99	0.87	0.98	0.81	0.99	0.89	0.97	0.90	0.98
$T{D}^E$		15.45	9.06	16.01	10.70	16.61	10.49	19.39	9.44	11.27	16.75	12.63	10.94
$T{D}^M$		9.46	5.54	5.82	4.23	11.06	6.17	10.03	3.53	11.98	12.91	-	-

. The lack of experimental data for motions in oblique stern waves and VBM necessitates the use of the total difference measure relative to the mean of calculated results to quantify the uncertainty. The analysis of results shows that the estimates of bias $\widehat{a}$ vary between 0.88–1.03 for heave motion, and 0.87–1.01 for pitch motion. In head waves, the coefficient of determination for pitch motion is closer to unity than for heave motion. This means that the bias alone is adequate to capture the motion uncertainty.

The estimated total difference relative to the average of calculated results obtained by numerical codes TD^M ranges between 4.2 to 17.6% for heave motion and between 1.8 to 26.2% for pitch motion. On the other hand, the modified total differences TD^E are estimated to be between 7.4 and 23.4 % for the heave motion and 7.4 to 19.4 % for the pitch motion. It appears that the numerical codes may predict heave and pitch motions with less uncertainty at lower speeds, as shown by a comparison of the uncertainty estimates in the mean of numerical codes shown in Table 3 and Table 11.

Table 12. Measures of uncertainty in VBM transfer functions obtained by different numerical codes at Fr = 0.1.

Measure	Heading	AQWA	WASIM	IST-CENTEC
		VBM	VBM	VBM
$T{D}^M$	45°	17.40	5.40	13.10
	180°	17.04	6.85	12.63
	225°	32.50	8.30	35.23
	Code Average Index	22.31	6.85	20.32

summarises the uncertainty analysis of VBM predicted by various numerical models at a slower speed with Fr = 0.1. The total difference estimates demonstrate that WASIM is capable of predicting the VBM with less uncertainty than other codes. Results indicate that the predicted VBM at Fr = 0.1 does not demonstrate a significant decrease in uncertainty estimates when compared to the results provided in Table 6 at Fr = 0.245.

Table 13. Summary of numerical codes with low uncertainty at different wave headings and speeds.

Fr = 0.245
Result	Location	Heading
		45°	180°	225°
Heave	CG	PDSTRIP	PDSTRIP	MAXSURF
Pitch	CG	WASIM	PDSTRIP	AQWA
VRM	Bow	WASIM	PDSTRIP	AQWA
VSF	St.5	WASIM	WASIM	IST-CENTEC
	St.10	WASIM	WASIM	WASIM
	St.15	WASIM	WASIM	IST-CENTEC
HSF	St.5	WASIM	-	WASIM
	St.10	WASIM	-	Non
	St.15	WASIM	-	WASIM
VBM	St.5	IST-CENTEC	WASIM	AQWA
	St.10	WASIM	IST-CENTEC	Non
	St.15	Non	Non	IST-CENTEC
HBM	St.5	Non	-	Non
	St.10	WASIM	-	WASIM
	St.15	Non	-	Non
TM	St.5	AQWA	-	Non
	St.10	AQWA	-	AQWA
	St.15	AQWA	-	WASIM
Fr = 0.1
Result	Location	Heading
		45°	180°	225°
Heave	CG	WASIM	IST-CENTEC	MAXSURF
Pitch	CG	PDSTRIP	PDSTRIP	AQWA
VBM	St.10	WASIM	WASIM	WASIM

summarizes the selected codes at various headings based on motions and sectional loads with the lowest uncertainty measures. It is noticed that none of the applied numerical codes can produce accurate estimates for all wave-induced motions and loads at all heading angles at the same time. An overview of the uncertainty analysis suggests that the PDSTRIP code may offer the least uncertainty in heave, pitch, and VRM transfer functions at head waves. WASIM provides the lowest uncertainty for the majority of VSF and HSF estimates. While it is challenging to establish a code with low uncertainty for VBM and HBM. For most TM estimates, AQWA provides the most reliable results.

The discrepancy between calculated and measured outcomes can be explained by several factors. The numerical codes utilized in this study are mainly based on linear techniques with various assumptions to solve a problem that is basically nonlinear. Each numerical code adopts a different approximation for solving the hydrodynamic problem at a forward speed. Moreover, the time-domain code WASIM employs an artificial active rudder and autopilot model to control the horizontal response; the autopilot model necessitates calibration at each speed and wave direction to achieve stable time-domain simulations.

Furthermore, the investigated container ship’s mass is only described as four points of mass, which may not be enough to yield accurate results, particularly for wave-induced sectional loads, because each numerical code treats the detailed mass distribution across each of the four segments differently. On the other hand, the model test results for the Flokstra container ship represent reality by incorporating nonlinear effects such as flare bow and stern and viscous damping, particularly when tested at a wave height of 1/60 of the ship length, which is not small.

4. Conclusions

This paper proposes a new measure of model uncertainty and demonstrates its applicability in a comparative study and uncertainty assessment for linear transfer functions of wave-induced ship motions and loads obtained by various seakeeping codes. The high-speed Flokstra container ship is numerically simulated in regular waves at various heading angles using five numerical codes, and the computed results are compared with experimental data. Later, several uncertainty measures are applied to quantify the uncertainty in individual seakeeping codes with respect to both the available experimental results and the average of the computed results.

It is crucial to select multiple uncertainty measures that can identify differences between the utilized seakeeping codes. Assessing the uncertainty based solely on total difference TD^M estimations may not reflect the numerical code’s accuracy. It is more beneficial to evaluate the accuracy of numerical models based on comparisons with experiments (modified total difference) TD^E than comparisons with the average of numerical results TD^M. Since the uncertainty estimates of the total difference measure for only two codes are identical and mainly dependent on the average, the newly proposed modified total difference approach has an additional advantage over the total difference measure.

The benchmark study shows that the applied codes may give results with low uncertainties and better agreement with experiments at one heading compared to the others. However, none of the applied codes can produce accurate estimates for all wave-induced motions and sectional loads at all heading angles at the same time.

The calculated midship wave-induced sectional loads have higher uncertainties compared to motions. The discrepancies in the computed sectional loads may be attributable to assumptions in linear seakeeping numerical codes that fail to account for critical nonlinear effects that were easily included in experiments, as well as insufficient knowledge or inaccurate representation of the detailed features of the investigated ship model.

Theory-based quantified uncertainty analysis for wave-induced motions and loads may provide a better idea about the accuracy of a numerical code than code-based analysis. However, this kind of study requires the availability of a wide range of well-described experimental data in regular and irregular waves for different hull forms, detailed mass distribution, quantified experimental uncertainty, the application of a large number of numerical models that are based on different theories (for example, five numerical codes for each theory and different users may present results based on the same numerical code). Furthermore, the model uncertainty should be evaluated based on various standard comparable uncertainty measures to have a complete understanding of the calculated results.