**5. Results**

ˆ

This section evaluates the performance of the proposed models. The first subsection evaluates the performance of the AKF model and then compares the AKF with the KF model (Section 5.1). The second subsection presents the performance of the NN model used for estimating the LMP of probe vehicles at the exit of the link (*ρout*) (Section 5.2). The third subsection compares the performance of AKF with the AKFNN approach (Section 5.3). The fourth subsection investigates the sensitivity of the AKF estimation model to the initial conditions (Section 5.4). The accuracy of the proposed models was evaluated based on the root mean square error (RMSE) as shown in Equation (24). The RMSE has been frequently used in the literature to measure the difference between the model estimates and the actual values.

$$\text{RMSE (veh)} \;= \sqrt{\sum\_{t=1}^{n} \left[ \hat{N}^+(t) - N(t) \right]^2} \Big/ n \tag{24}$$

where *N* +(*t*) represents the estimated vehicle count values, *N*(*t*) represents the actual vehicle count values, and *n* is the total number of estimations. All simulation scenarios start with the following initial conditions: an initial vehicle count estimate of zero (*N* ˆ +(0) = 0 veh), which is the same value of the actual vehicle count, and initial mean and the prior covariance estimates of the state system (*m*(0) = 2 veh and *P* ˆ <sup>−</sup>(0)= 75 veh2) if the LMP scenario is less than or equal 60%, and (*m*(0) = 9 veh *<sup>P</sup>*<sup>ˆ</sup>−(0)= 120 veh2) if the LMP scenario is greater than 60%. The proposed models were evaluated using different probe vehicle LMPs, including 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, and 90%. For each scenario, a Monte Carlo simulation was conducted to create 300 random samples of probe vehicles from the full data set.

#### *5.1. Comparison of the KF and the AKF Models*

This section evaluates the proposed AKF model with real-time estimates of the error statistical parameters for the state and the measurement. This section also compares the proposed AKF model with the developed KF model in [5], as shown in Table 2. Results show that the AKF outperforms the KF model in most scenarios except for the scenarios with high LMPs (i.e., LMP of 80% and 90%). Results demonstrate the need to provide real-time estimates for the mean and variance error values in the state and measurement when dealing with low/medium LMPs. This happened due to high error in the fixed *ρ* value that was used, which then produced high error in the vehicle count estimate. The AKF improved the traditional KF vehicle-count estimation accuracy by up to 29%. In contrast, for high LMPs, the user may proceed with predefined statistical values for the state and measurement (mean and variance error values), due to low errors in the vehicle count estimates (low error in the *ρ* value). In conclusion, a simple KF can be used with high LMPs without the need to change statistical noise parameters at every estimation step.


**Table 2.** Root mean square error (RMSE) values using the Kalman filter (KF) and the adaptive Kalman filter (AKF) models.

#### *5.2. Developed NN Model*

The NN model was employed to predict the (*ρout*) value, which is used to reflect the total number of vehicle departures from the given number of probe vehicle departures. The data set was divided into 70% for training, 15% for validation, and 15% for testing. The validation data set is used to measure network generalization and to avoid any over fitting problems [38]. The developed NN performance is shown in Table 3. The mean square error (MSE) is 0.01 and the R value is close to 1.0. The R value measures the correlation between model outputs and desired outputs. A value close to 1.0 means that the model outputs are very close to desired outputs. Figure 3 shows the error histogram for the training, validation, and testing data and their deviations from the zero error bar. Most of the errors lie around the zero error bar, which means that the developed NN model appropriately addressed the research goal (i.e., estimating *ρout*). Figure 4 presents the predicted and actual values for the *ρout* at different LMPs.

**Table 3.** Developed neural network (NN) model performance measures for the training, validation, and testing data set.


**Figure 3.** Error histogram for the training, validation, and testing data set.

**Figure 4.** Actual and estimated values of *ρout* for different level of market penetration (LMP) scenarios: (**a**) 10%, (**b**) 20%, (**c**) 30%, (**d**) 40%, (**e**) 50%, (**f**) 60%, (**g**) 70%, (**h**) 80%, and (**i**) 90% LMP.

#### *5.3. Comparison of the AKF and the AKFNN Models*

This section demonstrates the impact of using two *ρ* values rather than using one predefined *ρ* value. The average predefined *ρ* value is defined as the value for the entire tested link. The average *ρ* value remains constant for the entire simulation for each LMP scenario. For instance, if the scenario of 10% LMP is tested, the *ρ* value in both the state and measurement is treated as a value of 0.1. In this study, the authors proposed the use of two *ρ* values; one at the entrance and one at the exit of the link to reflect the total number of arrivals and departures from the given total number of probe arrivals and departures, respectively.

*ρin* is measured directly using the installed loop detector at the entrance of the link. The developed NN model is used to predict the *ρout* values (Section 5.2). Then, the *ρin* and *ρout* values are utilized in the AKF equations. Recall that the AKF model relies only on probe vehicle data, while the AKFNN model uses a fusion of probe vehicle and single-loop detector data.

In Table 4, the RMSE values using the AKF and the AKFNN models are presented. The results demonstrate the benefits of using the AKFNN approach rather than the AKF approach, where the estimation accuracy is improved by up to 26%. This finding proves what was recommended by Aljamal et al.'s previous study [5] to consider two *ρ* values rather than one value. As a result, the proposed AKFNN approach is robust and produces reasonable errors even with low LMPs. For instance, the estimated vehicle count values are off by 3.7 veh when the LMP is equal to 10%. Figure 5 presents the vehicle count estimation for different LMPs using the proposed AKFNN Approach.

**Table 4.** RMSE values using the AKF and the AKFNN models.


**Figure 5.** *Cont.*

**Figure 5.** Actual and estimated vehicle counts over estimation intervals for different LMP scenarios: (**a**) 10%, (**b**) 20%, (**c**) 30%, (**d**) 40%, (**e**) 50%, (**f**) 60%, (**g**) 70%, (**h**) 80%, and (**i**) 90% LMP.

#### *5.4. Impact of the Initial Conditions on the AKF Model*

The KF model, traditional and adaptive, is sensitive to the initial condition parameters, such as the posterior state estimate (*Ni* = *N* ˆ +(0)), the mean of state noise (*mi* = *m*(0)), and the prior error covariance estimate (*Pi* = *P* ˆ −(0)). These parameters are tuned by a trial-and-error technique to find the best initial condition values for seeking better KF estimation outcomes. However, in real applications, trial-and-error is not realistic and not easy to achieve. Hence, this section investigates the impact of initial conditions on the accuracy of the vehicle count estimation.

#### 5.4.1. Impact of Initial Estimate of the Vehicle Count (*Ni*)

For the initial estimate value of the vehicle count (*Ni*), different values were evaluated (ranges from 0 to 10 at increments of 1). In this study, remember that all simulation scenarios start with an initial estimate of zero (*Ni* = 0 veh), which is the same value as the actual vehicle count. Figure 6a presents the RMSE values for different *Ni* values for the scenario of 10% LMP. As shown in the figure, the values of 8 and 10 produce the lowest RMSE. The RMSE value is equal to 4.3 veh when *Ni* is equal to 0. In contrast, theRMSE value is equal to 3.9 veh when *Ni* is equal to 8. As a result, starting the AKF model with the best initial estimate (e.g., *Ni* = 8 veh) would reduce the errors and therefore improve the estimation accuracy.

#### 5.4.2. Impact of Initial Mean Estimate of the State System (*mi*)

Another critical initial parameter in the AKF model is *mi*. This parameter represents the mean value of the noise in the state equation. This paper tests 16 different *mi* values (i.e., 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, and 15). Figure 6b presents the vehicle count estimation RMSE values for different *mi* values. The RMSE value is equal to 4.7 veh when the simulation starts with a 0 value of *mi*. In contrast, the RMSE value is 3.9 veh when the value of *mi* is equal to 11.

5.4.3. Impact of Initial Prior Covariance Estimate of the State System (*Pi*)

The last parameter tested in this study is the initial prior estimate of error covariance *Pi*. The error covariance parameter describes the accuracy of the state system. For instance, if the covariance value is low, then the state outcome is accurate and close to the actual value. As stated in the literature, the initial parameters should always be tuned to achieve accurate estimation accuracy. Thirteen different *Pi* values were tested (i.e., 5, 10, 15, 20, 25, 50, 75, 100, 120, 150, 200, and 250). Figure 6c presents the RMSE values using different *Pi* values. The *Pi* value of 150 veh<sup>2</sup> produces the lowest RMSE values.

**Figure 6.** Impact of the initial conditions on the AKF model: (**a**) Initial estimate values *Ni*, (**b**) Initial mean estimate values *mi*, and (**c**) Initial covariance estimate values *Pi*.

The research presented in this study evaluates the proposed approaches as they should be in real-world applications. Therefore, the trial-and-error technique was avoided since it is not a valid solution in the field. However, it was noticed that previous research always tunes the initial parameters to determine the best initial conditions when testing their estimation approaches [2,3,17]. If that is the case, let us assume that the proposed AKFNN approach always starts with the best initial value of *Pi*, which would produce less errors. Table 5 presents the RMSE when considering the trial-and-error technique (Tuned AKFNN). The AKFNN and the Tuned AKFNN approaches used the same values of *Ni* and *mi*, but they used different *Pi* values. *Ni* is assumed to be zero, while *mi* has two values based on the tested scenario: a value of 2 veh when low LMP scenarios are tested (LMP <= 60%), and a value of 9 veh with high LMP scenarios (LMP > 60%). From the table, tuning the *Pi* value significantly improves the estimation accuracy for all scenarios (by up to 27%). For instance, at 10% LMP, the estimation error dropped from 3.7 to 3.3 vehicles. On the other hand, the estimated vehicle count values are off by 2.8 vehicles instead of 3.6 vehicles for the scenario of 20% LMP.


**Table 5.** Impact of applying the trial-and-error technique for the initial value of covariance *Pi*.

In conclusion, the AKF model was proven to be very sensitive to the initial conditions (*Ni*, *mi*, *Pi*). Hence, starting the simulation with good assumptions of the initial conditions can significantly improve the estimation accuracy, as shown in Table 5. Finally, Table 6 presents the performance of the models discussed in the paper.


**Table 6.** RMSE values for the KF, the AKF, the AKFNN, and the tuned AKFNN models.
