*4.2. Framework Accuracy*

The accuracy of the system proposed in this study is evaluated using the inverse results of the actual vehicles. The accuracy is evaluated using the DBC files of the test vehicle, which were determined to be the truth.

The accuracy of message filtering is shown in Table 6. All CAN traces are taken from the OBD-II interface, so the accuracy is expressed using the percentage of filtered quantities in the OBD-II. Among all the results, only the brake-related messages have an accuracy of 66.67 %, while all other messages are filtered at 100%. The false-positive result for 0x202 for brakes is due to the fact the brakes are velocity-dependent to some extent. According to the DBC file, 0x202 does contain velocity information, which causes *R*<sup>2</sup> to be higher than the threshold. In addition, message 0x240 in the description of the DBC, describes the vehicle's torque information. Although it is a steering-related message, it cannot be inverted at the bit level because the torque measurement information is not directly available. It is also worth noting that the messages defined in the DBC file do not fully appear in the OBD-II interface. This phenomenon is due to a gateway in the vehicle CAN-bus network, which does not forward all bus traces to OBD-II, but only a portion of the traffic to the OBD-II interface. The rest of the CAN bus data, especially the traffic related to assisted driving and vehicle control, flows only within the vehicle and cannot be captured externally.


**Table 6.** Message filtering accuracy results for vehicle behavior.

The bit-reverse accuracy is shown in Figure 18, which compares the bit reverse results of this framework with the vehicle behavior defined in the DBC file. Figure 18a shows the bit-inverse accuracy of the speed-dependent messages. It is observed that bits are written with speed in 0x202, 0x215, and 0x217 are partially reversed to obtain 9 bits for 16 bits in 0x202, 52 bits for 64 bits in 0x215, and 14 bits for 16 bits in 0x217. The bit reversal accuracy of the two steering-related messages, 0x082 and 0x086, is shown in Figure 18b. The proposed framework in this study correctly reverses 9 of the 16 bits in 0x082 and 18 of the 27 bits in 0x086. The accuracy of gas-related message reversal is shown in Figure 18c. 0x0FD gets 7 out of 8 bits, 0x167 completely reverses 8 bits, 0x202 gets 9 out of 16 bits, and both 0x21F and 0x165 have only one bit that is not reversed. Only bits 38 to 39 of 0x078 were not found in the brake-related messages' reverse results, as shown in Figure 18d. For the gear and wiper-related messages, the bits indicating the gear and wiper switches are both correctly reversed, which can be seen in Figure 18.

(**b**) 

**Figure 18.** *Cont*.

**Figure 18.** Bit reverse accuracy: (**a**) Speed reverse result; (**b**) Steer reverse result; (**c**) Gas reverse result; (**d**) Brake reverse result; (**e**) Gear reverse result; (**f**) Wiper reverse result.

The overall bit-reverse accuracy of the proposed framework for vehicle behavior is shown in Table 7. The overall reverse accuracy is over 76%, especially for gear, and wiper reversion can reach 100% because CAN messages and sensor data are discrete and not easily disturbed by other data. The reverse accuracy for vehicle speed, gas pedal, and accelerator pedal are all about 80% because these behaviors are difficult to reach the limit state during vehicle sampling, such as vehicle speed of 255 km/h, gas, and brake pedals kept at the maximum angle. Therefore, when reversing the messages related to these behaviors, their high values can barely be detected (i.e., the high value of *β* does not satisfy the two-fold relation), which results in poor accuracy. The steering-related information

performs the worst, with only 65%. Due to the low degree of steering wheel variability in daily driving, the linear regression model is easily disturbed by irrelevant bits, resulting in poor accuracy of bit reversals.


**Table 7.** Bit reverse result with DBC file description.
