Assessing the Performance of Highway Safety Manual (HSM) Predictive Models for Brazilian Multilane Highways

Abstract

This paper assesses the performance of Highway Safety Manual (HSM) predictive models when applied to Brazilian highways. The study evaluates five rural multilane highways and calculates calibration factors (C_x) of 2.62 for all types of crashes and 2.35 for Fatal or Injury (FI) crashes. The Goodness of Fit measures show that models for all types of crashes perform better than FI crashes. Additionally, the paper assesses the application of the calibrated prediction model to the atypical year of 2020, in which the COVID-19 pandemic altered traffic patterns worldwide. The HSM method was applied to 2020 using the C_x obtained from the four previous years. Results show that for 2020, the observed counts were about 10% lower than the calibrated predictive model estimate of crash frequency for all types of crashes, while the calibrated prediction of FI crashes was very close to the observed counts. The findings of this study demonstrate the usefulness of HSM predictive models in identifying high-risk areas or situations and improving road safety, contributing to making investment decisions in infrastructure and road safety more sustainable.

1. Introduction

Road safety is a global concern that has prompted nations to implement measures to reduce the fatalities and injuries resulting from road crashes. Despite some success in reducing the number of deaths in road crashes [1], the problem persists, with the proportion of fatal crashes increasing in recent years, causing more than 15 deaths per 100 thousand inhabitants yearly [2]. This number is about three times higher for emerging countries than developed countries [3], which might be related to the rise in motorization across Latin American countries that has led to a significant increase in exposure to traffic risks [4].

Therefore, countries must devise strategies to decrease this figure, including implementing stricter regulations to manage key risk factors and allocating greater resources to initiatives and studies that enhance road safety. By comprehending the factors that significantly influence the likelihood of accidents, it becomes feasible to forecast the probability of their incidence [5,6,7]. Establishing standardized definitions and methodologies for collecting comprehensive data on accidents, risk factors, and exposure occurrence is imperative to facilitate global and regional comparisons [4]. As a result, there is a lack of uniformity in the organization and collection of crash data across different regions and municipalities within the country. Each state and municipality may have its system for collecting crash data, leading to inconsistencies and challenges in data management and analysis [4,8].

Despite a decrease in the total fatalities on federal highways in Brazil over the past ten years, there has been an alarming increase in the proportion of fatal crashes [2,9]. This discrepancy may be attributed to changes in the crash reporting system since 2015, particularly the introduction of self-reporting for non-injury crashes. This could have led to an underreported number of property damage only (PDO) crashes [8,10]. Additionally, Brazil’s technological backwardness resulting from the economic and political crisis that began in 2014 may have contributed to this trend [11]. Thereby, further investigation is needed to address road safety on Brazilian highways, including investments in infrastructure and technology for accident prevention [12].

Developing effective strategies to address road safety requires a comprehensive understanding of contributing factors, which can be achieved through data-driven approaches like safety performance functions (SPF). The Highway Safety Manual (HSM) offers predictive models that integrate SPF with crash modification factors to estimate the crash frequency and identify high-risk areas and scenarios. However, it is crucial to assess the transferability of HSM predictive models when applied to an international context, particularly on Brazilian highways where data availability is limited and local SPFs are lacking. This study aims to bridge this gap by evaluating the performance of HSM predictive models on Brazilian rural multilane highways, thereby contributing to developing effective road safety strategies and advancing the United Nations Sustainable Development Goals.

Additionally, the COVID-19 pandemic had a significant impact on mobility in Brazil, leading to a reduction in the use of public transportation and an increase in individual transport [13,14]. This shift and the overall decrease in mobility during the pandemic resulted in a heterogeneous mobility pattern over time. Therefore, this study aimed to evaluate the performance of the calibrated prediction model under these atypical conditions, offering insights into its resilience and accuracy when confronted with significant changes in traffic patterns and volumes caused by the COVID-19 pandemic.

2. Literature Review

2.1. The Highway Safety Manual Predictive Model

The existing literature on crash prediction models primarily attributes crashes to inadequate driving performance with the demands of the road environment. Factors such as traffic flow, geometric attributes, road signs, and vehicle characteristics have been identified as contributing to this mismatch [15,16,17,18,19,20]. Moreover, SPFs have been developed to estimate crash rates within a specific timeframe or exposure [5,21,22,23,24,25,26]. These SPFs utilize statistical models that analyze risk indicators, including absolute numbers, frequency, and crash rates, as defined by Equation (1).

λ = N × p,

(1)

where λ is the expected crash number, N is the exposure, and p is the crash rate. The introduction of the HSM has provided a systematic approach to assessing crashes by employing analytical techniques and tools that quantify the impacts of road network planning, design, operation, and maintenance decisions. In research-based studies, the HSM has played a significant role in evaluating crashes.

The SPFs included in the HSM were developed using negative binomial (NB) regression models. These models were constructed using a generalized linear modeling (GLM) procedure, as outlined by Srinivasan et al. [27]. The SPFs consider both the infrastructure and operational characteristics.

Equation (2) illustrates how the predicted number of crashes (N_predicted) is determined using the SPF [28]. The SPF equation is specific to each facility, considering its base conditions, and adjusted by a calibration factor (C_x) and multiple crash modification factors (CMFs). Each CMF accounts for the operational and geometric characteristics (y) of the facility (x).

N_predicted = N_SPFx × C_x × (CMF1_x × CMF2_x × … × CMFy_x)

(2)

To determine C_x, Equation (3) provides the necessary calculation. The observed crashes are summed up across all sites and divided by the predicted crashes across all sites. The resulting C_x value is rounded to two decimal places and applied to the predictive model.

C_x = ∑ observed crashes/∑ predicted crashes

(3)

Calculating the corresponding C_x value for each facility type and year is advisable to customize the model. By substituting default values with locally derived values, the reliability of the predictive model can be improved. To apply this methodology, the HSM recommends a minimum desirable sample of 30 to 50 sites, representing at least 100 crashes annually [28]. Following the initial calibration, the HSM suggests utilizing the Empirical Bayes (EB) method to enhance the reliability of results and account for the regression-to-the-mean effect.

However, the model has limitations, particularly regarding its failure to consider speed limits. A study by Shirazinejad et al. [29] demonstrated that increasing the speed limit from 70 mph to 75 mph led to a significant 27% increase in total crashes and a notable 35% increase in fatal and injury crashes. Additionally, the HSM methodology fails to account for factors such as road infrastructure damage and unreasonable road design, all of which have been identified as impacting traffic safety [30].

2.2. Previous Studies on the Transferability of the HSM Model

Numerous studies have investigated the transferability and calibration of the HSM predictive model in different countries and regions. Over the past decade, researchers have explored the performance and parameters of the HSM model to assess its applicability and effectiveness in various contexts. The following studies shed light on the transferability and calibration challenges and the practical solutions and results in different countries.

Sun et al. conducted a statewide calibration of the HSM model for rural divided multilane highways in the US [31]. Their findings indicated that the HSM model reasonably predicted crashes in Missouri, with a calibration factor (C_x) of 0.98. In a study on rural two-lane roads in Arizona, Srinivasan et al. identified limitations in applying the HSM predictive models [32]. They emphasized the importance of gathering a larger sample and exploring the estimation of calibration functions to fit local data better. The overall calibration factor in this study was 1.079, indicating the success of the HSM model for US cases. D’Agostino examined the calibration factor for Italian motorways and found that the HSM model underestimated observed crash counts, with a C_x of 1.26 [33]. La Torre et al. concluded that a jurisdiction-specific base model derived from the HSM’s SPF provided a solid and reliable tool for crash prediction on the Italian freeway network [34].

In Brazilian studies, Rodrigues-Silva applied the HSM predictive model to two-lane highways in São Paulo State and found a calibration factor of 3.73 [35]. Barbosa et al. developed SPFs for intersections in Belo Horizonte, Brazil, with a calculated C_x of 2.06 [36]. Another study in Fortaleza city found a calibration factor of 0.65, highlighting the challenges in developing a nationwide SPF. Waihrich & Andrade investigated the calibration of the HSM model for multilane highways in the states of Minas Gerais and Goiás, Brazil. The resulting C_x values were 2.37 and 1.58 for each region, respectively, indicating a lack of transferability of the original HSM model in these scenarios [37]. Rodrigues-Silva compared the transferability between the HSM method and a local SPF for two-lane highways in different regions of Brazil. The calculated calibration factors were 3.67, 3.77, and 2.60 for São Paulo, Minas Gerais, and Paraná, respectively. This study highlighted the need for more parameters and knowledge in models for different facility types [38].

Studies conducted in Egypt by Elagamy et al. and in California, Maine, and Washington by Matarage & Dissanayake found that the HSM model overpredicted crash occurrences on multilane rural roads [39,40]. These studies emphasized the importance of considering local conditions and conducting calibration to improve the accuracy of predictions. Dadvar et al. proposed a method to adjust the HSM crash prediction model to provide a better fit for local data, as misallocating resources due to incorrect calibration factors can be problematic [41]. Al-Ahmadi et al. studied multilane rural highway segments in Saudi Arabia [42]. They found C_x values ranging from 0.63 to 0.78, emphasizing the need for in-depth local calibration and assessment of SPF quality. Researchers agree that the transferability of a model is dependent on the similarity of site characteristics to base conditions, and models must be built by associating regions with similar characteristics. The effectiveness of the local calibration factor as a method for transferring SPFs is widely discussed, considering socio-economic characteristics, traffic safety data distributions, and traffic flow influences on the transferability process. Kronprasert et al. compared different regression models for prediction accuracy, and the calibrated HSM SPF was the most effective model in predicting crashes on horizontal curve segments, underscoring its usefulness [43]. In a comprehensive overview, Heydari S. et al. [44] addressed road safety in low-income countries (LICs). They stressed the importance of accurate and complete road crash data for effective road safety interventions. They acknowledged that traditional sources such as police records suffer from varying levels of under-reporting, especially in LICs. They also emphasized the need to improve the quality and accuracy of road crash data through techniques like combining police and hospital records.

Countries like Brazil, with comprehensive databases integrating crash counts, traffic volume, and infrastructure data, must evaluate the performance of crash prediction models to shape investment planning strategies effectively. Conducting local calibration exercises considering regional peculiarities is crucial to enhance the transferability and precision of the HSM model across diverse countries and regions. These efforts aim to optimize the reliability of crash predictions, facilitating sustainable and informed interventions in transportation systems.

2.3. Goodness of Fit Measures

Assessing the accuracy of crash prediction models is essential in enhancing road safety measures. One approach to improve model performance is incorporating a local calibration factor (C_x) that considers the specific conditions of the target region. However, it is equally important to evaluate the model’s goodness of fit (GOF) and examine how well it aligns with observed data. In this regard, two widely used measures of forecast accuracy, the mean absolute percentage error (MAPE) and the mean absolute deviance (MAD), are commonly employed for comparative analysis.

Table 1. Works of HSM method application in Brazil [35,36,37,38,45].

Author	Region	Facility Type	Cx	GOF
Rodrigues-Silva (2012) [35]	SP	Two-lane Rural Highways	3.73	Chi square test and Kolmogorov-Smirnov
Barbosa et al. (2014) [36]	CE	Urban Intersection	0.65	AIC, R2 statistic, and CURE plots
			2.06
Cunto et al. (2015) [45]	Fortaleza (CE)	Urban Roads	0.98	$\mathrm{MAD}, \mathrm{MAPE}, \mathrm{CURE}, \mathrm{Pearson} {\mathrm{ꭓ}}_p^2$ statistics and z-score
			2.15
Waihrich & Andrade (2015) [37]	MG	Multilane Rural Highways	2.37	MAD, MAPE and R2Efron
	GO/DF		1.58
Rodrigues-Silva (2017) [38]	SP	Two-lane Rural Highways	3.67	MAD, MAPE, R2Efron, and CURE plots
	PR		3.77
	MG		2.60

summarizes recent studies applying the Highway Safety Manual (HSM) method to the Brazilian road network. The table provides information on the geographical region, facility type, estimated C_x values, and the GOF tests employed in the prediction models. Various studies have focused on different types of highways, including multilane, rural, and urban roads, while also investigating the influence of road geometry and traffic characteristics on crash frequency.

Moreover, in recent research, the root mean square error (RMSE) has emerged as another evaluation metric for prediction accuracy in studies conducted by Li et al., Yao et al., and Yehia et al. [46,47,48]. However, it should be noted that the effectiveness of CURE plots in assessing model performance may be limited in studies with smaller sample sizes, as highlighted by Dadvar et al. [41].

2.4. The Impact of COVID-19 on Traffic Safety

The COVID-19 pandemic has brought about significant changes in traffic patterns and increased interest in investigating its impact on traffic safety globally. During the period of lockdowns and restrictions, there was a noticeable reduction in traffic flow in many affected countries [49]. However, studies have revealed a concerning increase in the severity of crashes during this period [50].

Research suggests that implementing nonpharmaceutical interventions (NPIs) and the higher percentage of people staying at home have had mixed effects on traffic safety. On the one hand, these measures have been associated with potential improvements in pedestrian and cyclist safety but have also increased crash risk for motor vehicle drivers [51]. Surprisingly, the average number of cyclists killed or injured per crash has tripled compared to previous years [52].

It is important to note that simply reducing traffic volume during the pandemic does not necessarily lead to improved traffic safety. This can be attributed to the homeostasis effect, wherein drivers compensate for reduced traffic by engaging in risky driving behaviors such as speeding and failure to signal [47]. Furthermore, crashes resulting in severe injuries are more likely to occur on highways due to, i.a., increased speeding, reduced law enforcement, lack of seat belt usage, and alcohol and drug abuse [49]. Therefore, effective law enforcement mechanisms should focus on preventing these behaviors [53].

Another significant pandemic effect was the shortened trip lengths and decreased travel frequency as people engaged in more online activities as an alternative to physical travel [50]. These changes in transportation characteristics and reduced traffic intensity on the roads, driven by the rise of e-commerce, have had implications for traffic patterns.

The sudden disruptions in traffic behavior caused by the pandemic offer a valuable opportunity to broaden the understanding of risk factors and the application of SPFs. As such, in this study, the calibrated HSM SPF for 2020 is compared to the crash data count in 2020 to assess its capability in evaluating the impacts of COVID-19 on the studied highways. This analysis can provide valuable insights into the effects of the pandemic on road safety and inform future strategies and interventions.

3. Materials and Methods

3.1. The HSM Crash Prediction Method for Divided Highway Segments

The required and desirable site characteristics for calibrating the SPFs for divided rural multilane roadways are described in

Table 2. Data needed to calibrate Part C predictive models by facility type for Rural Multilane Highway Segments [28].

Data Element	Data Need		Default Assumptions
	Required	Desirable
Segment length	X		Actual data required
Average annual daily traffic (AADT)	X		Actual data required
Lane width	X		Actual data required
Shoulder width	X		Actual data required
Presence of Lighting	X		Assume no lighting
Use of automated speed enforcement		X	Base default on current practice
Median width	X		Actual data required

.

The Nspf for rural multilane highways depends on the Annual Average Daily Traffic (AADT) for each year by segment and the segment length (L) in miles, as shown in Equation (4). The regression coefficients a and b are presented in the HSM (

Table 3. Regression coefficients for four-lane highways in HSM [28].

Facility Type/Severity	A	b	c
4-Lane Total	−9.025	1.049	1.549
4-Lane KABC	−8.837	0.958	1.687
4-Lane KAB	−8.505	0.874	1.740

).

$$Nspf=e^{\left(a+b\times \ln \left(AADT\right) +\ln \left(L\right)\right)}$$

(4)

The EB method should be applied to estimate better the expected number of crashes for a single site [54], as described in Equation (5).

$$k=\frac{1}{e^{\left(c+\ln \left(\mathrm{L}\right)\right)}}$$

(5)

Here, k represents the overdispersion parameter associated with the roadway segment, L is the length of the roadway segment (in miles), and c is a regression coefficient used to determine the overdispersion of this model (see Table 3). After determining the k value for each studied segment, the Site-Specific EB Method is applied to obtain the weighted adjustment (w) placed on the predictive model estimate in Equation (6).

$$w=\frac{1}{1+k\times \left(\sum _{all study years}{N}_{predicted}\right)}$$

(6)

The final step is to obtain the N_expected, as shown in Equation (7). This represents the final calibrated number of crashes for each segment.

$$N_{expected}=w\times N_{predicted}+\left(1-w\right)\times N_{observed}$$

(7)

3.2. Road Network Analysis

Five rural divided highways in São Paulo State managed by toll administration were analyzed. The selected segments are part of the highways SP-255, SP-318, SP-330, SP-334, and SP-345. The sections were chosen based on their geometric aspects and the availability of traffic volume information, as presented in

Table 4. Key aspects of the studied road segments.

Highway	ID	Start Point (km)	Endpoint (km)	Studied Length (km)	Total Number of Crashes (2009–2019)	Traffic VolumeData Available
						2016	2017	2018	2019	2020
SP 255	255_S01	2.8	48.1	45.3	1837	X	X	X	X	X
	255_S02	77.1	83.1	6.0	215	X	X	X	X	X
SP 318	318_S01	235.7	236.1	0.4	24				X	X
SP 330	330_S01	241.0	267.3	26.3	1865		X	X	X	X
	330_S02	267.3	304.0	36.7	3144	X	X	X	X	X
	330_S03	304.0	318.5	14.5	3175	X	X	X	X	X
SP 334	334_S01	319.3	349.5	30.2	2015	X	X	X	X	X
	334_S02	349.5	396.0	46.5	1856	X	X	X	X	X
	334_S03	396.0	406.0	10.0	1237	X	X	X	X	X
SP 345	345_S01	19.4	31.1	11.7	612	X	X	X	X	X
	345_S02	31.1	39.1	8.0	377				X	X

. The total length of the studied roads is 235.6 km. Traffic volume data was collected through sensors strategically placed along the highways.

3.2.1. Traffic Volume Data

The traffic volume data is detected by sensor devices called “SAT” or “TESC”. The available traffic volume data were verified to match the studied highways. The average annual daily traffic (AADT) data was collected for 2016, 2017, 2018, 2019, and 2020, as presented in

Table 5. AADT of the studied period.

Homogeneous Segments					AADT (veh/day)
Sensor	Highway ID	Direction	Start Point (km)	Endpoint (km)	2016	2017	2018	2019	2020
SAT10	255_S01	North	48.1	2.8	3173	5725	6049	5969	6374
SAT10	255_S01	South	2.8	48.1	3429	5889	6164	6394	6144
SAT11	255_S02	North	83.1	77.1	6600	5741	6813	6901	6855
SAT11	255_S02	South	77.1	83.1	7972	6602	8357	8461	8498
TESC2	318_S01	North	235.7	236.1	8195	8195	8195	8195	1683
TESC2	318_S01	South	236.1	235.7	8488	8488	8488	8488	1638
SAT01	330_S01	North	241	267.8	9565	9573	9581	9575	8913
SAT01	330_S01	South	267.8	241	8359	9322	9437	9400	8570
SAT04	330_S02	North	267.8	304	10,222	13,928	13,842	13,318	12,311
SAT04	330_S02	South	304	267.8	27,889	13,969	13,851	13,656	12,643
SAT05	330_S03	North	304	318.5	28,125	29,505	19,360	30,589	29,153
SAT05	330_S03	South	318.5	304	27,793	29,690	19,320	31,177	29,558
SAT06	334_S01	North	319.3	349.5	10,535	10,985	11,682	11,090	9764
SAT06	334_S01	South	349.5	319.3	9251	10,746	11,795	11,161	9766
SAT08	334_S02	North	349.5	396	4898	4281	4162	4402	3910
SAT08	334_S02	South	396	349.5	4257	4252	4461	4386	3887
SAT09	334_S03	North	396	406	9284	13,758	15,524	16,283	15,655
SAT09	334_S03	South	406	396	7937	14,159	14,801	15,031	13,896
SAT13	345_S01	East	31.1	19.4	6402	6750	6689	6692	6548
SAT13	345_S01	West	19.4	31	5469	5666	5541	5788	5748
SAT13	345_S02	East	36	31.1	6402	6750	6689	6692	6548
SAT13	345_S02	West	31.1	36	5469	5666	5541	5788	5748

. There are a few cases in which there was a lack of information. For SP 318, the available AADT data corresponds to 2019 and 2020 only. As recommended by HSM, the number has been repeated for previous years (2016, 2017, 2018). For SP330_S01, the AADT for 2016 was missing, completed by linear interpolating the existing data.

3.2.2. Crash Data

The crash data analysis for the study period is presented in

Table 6. Key information regarding the observed crash data throughout the study period.

Severity Type	Total				FI
Year of Study	2016	2017	2018	2019	2016	2017	2018	2019
∑	1653	1597	1398	1301	451	467	406	415
Mean	2.32	2.24	1.96	1.83	0.63	0.66	0.57	0.58
Standard deviation	3.47	3.08	2.94	3.08	1.18	1.13	1.13	1.17
Max	33	27	28	39	9	10	7	15
Min	0	0	0	0	0	0	0	0

, while Figure 1 depicts the severity-based distribution of crash data. The findings corroborate that the observed KABC data has exhibited lower variability than PDO data since 2015. As anticipated, the number of PDO crashes has been declining since 2015, which may be attributed to regulatory changes in Brazil’s crash reporting system, resulting in underreported crashes.

Table 6 provides key information regarding the observed crash data throughout the study period, encompassing severity types and corresponding totals for each year. The mean and standard deviation values highlight the decreasing trend in crash frequencies, particularly for fatal or injury (FI) crashes. The data ranges from a minimum of zero to a maximum of 39 crashes, with decreasing means and standard deviations over the years.

Figure 1 graphically illustrates the crash frequency distribution by severity type, further emphasizing the decreasing trend in crash occurrences over the study period. Furthermore,

Table 7. Proportion of crash data categorized by crash type.

Collision Type		FI	PDO	Total
Single vehicle		0.649	0.755	0.724
Multi-vehicle (total)		0.351	0.245	0.276
	Angle	0.037	0.016	0.022
	Head-on	0.011	0.001	0.004
	Rear-end	0.207	0.148	0.165
	Sideswipe	0.073	0.056	0.061
	Other multi-vehicle	0.022	0.025	0.024
Total Crashes		1.000	1.000	1.000

presents the proportion of crash data categorized by crash type, a crucial aspect for determining crash modification factors (CMFs). The table highlights the distribution of FI and PDO crashes among different collision types, including single-vehicle and multi-vehicle crashes. The proportions provide valuable insights for calculating CMFs, which serve as multiplicative factors in predicting the number of crashes based on specific road features.

3.3. Crash Modification Factor for Divided Roadway Segments (CMFs)

In Equation (2), the predicted number of crashes (N_predicted) is determined by multiplying the corresponding safety performance function (SPF) values (N_SPFx) with calibration factors (C_x) and the CMFs specific to each road characteristic (CMF_1x × CMF_2x × … × CMF_yx). The default base conditions for divided roadway segments on rural multilane highways include lane width of 12 feet, right-hand side shoulder width of 8 feet, median width of 30 feet, no lighting, and no automated speed enforcement. CMFs greater than 1.0 indicate an expected increase in crash frequencies due to specific road characteristics, while CMFs less than 1.0 signify a potential reduction in crash numbers.

4. Results and Discussion

The study applies the recommended methodology from Part C, Chapter 11 of the HSM 1st edition. The methodology includes calculating the network screening predicted frequency (N_predicted) using the crash modification factors (CMFs) and the observed crash data. A calibration factor (C_x) is determined by comparing the observed crashes (N_observed) to the predicted crashes (N_predicted).

4.1. The Local Calibration Factor (C_x)

The results of the calibration process are presented in

Table 8. Estimated N_predicted, N_expected, and C_x.

	Severity	2016	2017	2018	2019	Four-Years
Observed Crashes	Total	1653	1597	1398	1301	5949
	FI	451	467	406	415	1739
Predicted Crashes (Npredicted)	Total	565	570	545	587	2267
	FI	182	186	181	191	741
Expected Crashes (EB) (Nexpected)	Total	1622	1581	1402	1298	5892
	FI	457	472	420	422	1774
Cx	Total	2.92	2.80	2.57	2.22	2.62
	FI	2.47	2.50	2.24	2.17	2.35

, which shows the observed crashes (N_observed), predicted crashes (N_predicted), expected crashes (EB), and the corresponding calibration factors (C_x) for total and fatal injury (FI) crashes. The values of C_x indicate the similarity between the local road networks and the conditions for which the model was developed. Comparing the C_x values with previous studies, it is observed that the methodology performs closely to existing findings (C_x = 2.37 for the state of Minas Gerais) [37]. However, thoroughly examining the predicted points’ fit to the observed data is necessary to gain better insights.

4.2. The Goodness of Fit (GOF) Measures

The goodness-of-fit measures, including mean absolute deviation (MAD), mean absolute percentage error (MAPE), and root mean square error (RMSE), are presented in

Table 9. The goodness of Fit of the HSM predictive model by MAD, MAPE, and RMSE tests.

GOF	Calibrated Predicted Crashes			Expected Crashes
	MAD	MAPE	RMSE	MAD	MAPE	RMSE
Total	4.44	53%	8.59	0.80	10%	1.33
FI	1.92	78%	3.38	0.86	35%	1.35

. The smaller values of these measures indicate a better model fit. The results suggest that the total crash model performs better than the FI crash model, indicating the variability in calibrated predicted values. Moreover, the comparison with previous studies shows a good methodology performance regarding these measures.

As anticipated, applying the Empirical Bayes (EB) method yielded estimated values closely aligned with the observed values. The MAD, MAPE, and RMSE metrics indicate that the total crash model outperforms the FI model in the final step after implementing the EB method. Conversely, the MAD and RMSE values suggest that the calibrated predicted values exhibit more substantial variation when including PDO crashes, which is reasonable given that FI crashes represent only a smaller portion (approximately 30%) of the total crash data. The MAD for FI crashes accounted for 43% of the MAD for total crashes, while the RMSE was 39%. Moreover, a study by Waihrich and Andrade reported a MAD value of 5.54 for total crashes in the Minas Gerais state, which is 24% higher than the MAD value found for total crashes in the São Paulo state (MAD = 4.44) [37].

Figure 2 displays the results for the entire study period, comparing the calibrated N_predicted values with the observed total and FI crashes to the centerline. The proximity of the model’s output to the centerline signifies a closer alignment between the predicted and observed data. The graph underscores the dispersion of total crashes in comparison to FI crashes. Furthermore, points below the centerline indicate underestimation by the model, while points above the centerline indicate an overestimation of the observed data. Approximately 56% of the predicted values fall below the centerline trend, indicating that the model has predominantly underestimated the data. Figure 3 also compares the calibrated N_predicted values for total and FI crashes to the centerline for each study year.

To further analyze the difference between N_predicted and N_expected (results obtained after applying the EB method), Figure 4 and Figure 5 display the estimated data for total crashes, while Figure 6 and Figure 7 illustrate the estimated data for FI crashes. These graphs facilitate the estimation of the R² values for each severity type (total or FI) and year, as presented in

Table 10. Estimated R² for N_predicted and N_expected by year and by severity type.

Severity Type	Total					FI
Year of Study	2016	2017	2018	2019	2016–2019	2016	2017	2018	2019	2016–2019
Calibrated Npredicted	0.26	0.43	0.34	0.40	0.45	0.12	0.19	0.10	0.23	0.24
Nexpected	0.98	0.98	0.99	0.99	0.99	0.85	0.87	0.89	0.91	0.88

. Upon applying the EB method, the performance of N_expected aligns with previous literature studies, as indicated by the proximity of the points to the centerline. This close alignment suggests that N_expected closely resembles N_observed. In contrast, N_predicted exhibits a moderate dispersion. The graphs for each study year exhibit a similar pattern to the one depicted in Figure 4. Notably, 2019 displays a denser distribution of N_predicted values, indicating a closer prediction of the actual number of crashes. In contrast, the estimated values for 2016 appear more dispersed, suggesting a less accurate prediction for that particular year.

By adjusting the graph scale to accommodate the smaller sample represented by FI crashes (Figure 6), a clearer understanding of the performance of N_predicted can be achieved. Consistent with previous observations, the predicted values are denser above the centerline, indicating a tendency for underprediction by the model. However, in the case of N_expected for FI crashes, most values fall below the centerline, indicating that most expected values underestimated the observed FI crashes. This suggests that the EB method did not perform as effectively for all types of crashes. In this context, the expected crashes are more scattered, and the results indicate a general trend of underprediction.

Figure 7 demonstrates a similar performance of N_expected compared to Figure 6, indicating that the EB method consistently underpredicted the observed crash counts. The graphs reveal a pattern where the model underestimates FI crashes in segments where more than five crashes are observed annually. Lastly, Table 10 provides the R² estimates obtained from the developed graphs.

Table 10 presents the estimated R² values for N_predicted and N_expected, categorized by severity type and year. The R² value represents the goodness of fit between observed and estimated graphs, providing a correlation measure. A higher R² value indicates a better fit. As expected, the R² values for N_expected, which accounts for the observed crash counts, are significantly higher than those for N_predicted after applying the EB method. Notably, the FI crashes show lower R² values compared to all crashes.

Table 11. Result of all the GOF tests applied for calibrated predicted crashes.

GOF	MAD	MAPE	RMSE	R2
Total	4.44	53%	8.59	0.45
FI	1.92	78%	3.38	0.24

presents the results of various goodness of fit (GOF) tests for calibrated predicted crashes. These tests include MAD, MAPE, RMSE, and R². It is observed that using FI crashes yields lower MAD and RMSE values, indicating better accuracy, while MAPE and R² show better performance for all crashes.

On the other hand,

Table 12. Result of all the GOF tests applied for expected crashes.

GOF	MAD	MAPE	RMSE	R2
Total	0.80	10%	1.33	0.99
FI	0.86	35%	1.35	0.88

compares the GOF parameters for expected crashes. Here, improved results are observed for all crash types. This suggests that the prediction of crashes using the HSM model performed better across all crash types.

4.3. Crash Data Analysis for 2020

Due to the COVID-19 pandemic, 2020 witnessed significant disruptions in global traffic patterns. Ongoing studies are exploring the impact of the pandemic on various aspects of human health, including traffic-related fatalities and injuries [55]. In Figure 8, the number of Property Damage Only (PDO) crashes is depicted, while Figure 9 displays the reported traffic-related fatalities on state highways in the years 2019, 2020, and 2021, as documented by the São Paulo State government [56].

Notably, there is a substantial difference in PDO crashes between April 2019 and April 2020, with a 40% reduction in PDO crashes. Similarly, the highest reduction in fatal crashes was observed during the same period, with a decline of approximately 27%. Despite the lockdown measures commencing on 22 March 2020 in São Paulo State, the impact of these measures became more pronounced in April 2020. The graphs also present the moving average of crashes over a twelve-month interval, demonstrating the reduction in crashes during that period.

Table 13. Comparison of estimated variance in fatalities and AADT on state highways in recent years.

Year	Fatal Crash Counts	Mean	% Change Compared to		AADT	Mean	% Change Compared to
			Previous Year	Mean			Previous Year	Mean
2015	1872	1836	No data	1.94%	No data	--	-	-
2016	1853		−1.01%	0.90%	14,598	14,137	−	3.26%
2017	1911		3.13%	4.06%	14,149		−3.08%	0.08%
2018	1876		−1.83%	2.15%	14,183		0.24%	0.32%
2019	1929		2.83%	5.04%	15,128		6.66%	7.00%
2020	1651		−14.41%	−10.10%	12,092		−20.07%	−14.47%
2021	1763		6.78%	−4.00%	16,764		38.64%	18.58%

and Figure 10 illustrate the variation in fatal crashes and average AADT concerning the average and counts of the previous year. The data suggest that the decrease in fatal crashes is associated with the reduction in AADT, which can be attributed to the implementation of disease control measures during that period.

To assess the impact of COVID-19 on the analyzed segments,

Table 14. Key aspects related to crash data from 2016, 2017, 2018, 2019, and 2020.

Severity Type	Total					FI
Year of Study	2016	2017	2018	2019	2020	2016	2017	2018	2019	2020
∑	1653	1597	1398	1301	1182	451	467	406	415	389
Mean	2.32	2.24	1.96	1.83	1.66	0.63	0.66	0.57	0.58	0.55
Standard Deviation	3.47	3.08	2.94	3.08	2.73	1.18	1.13	1.13	1.17	1.13
Max	33	27	28	39	27	9	10	7	15	11
Min	0	0	0	0	0	0	0	0	0	0

presents the crash data specifically from 2020. A comparison is made between the crash counts in 2020 and the average counts from the previous four years. The data indicate a significant reduction in crashes during 2020, with a decrease of approximately 20% for all types of crashes and 11% for FI crashes compared to the average counts. This reduction in crash numbers suggests a notable influence of the COVID-19 pandemic on traffic safety, potentially due to factors such as reduced traffic volume, changes in driver behavior, and altered travel patterns resulting from pandemic-related restrictions and guidelines.

The 2020 AADT data played a crucial role in predicting the crash counts for that year using the HSM prediction model. This model relies on AADT values and segment length to estimate crash counts. In this study, the calibration factors obtained in Section 4.1 (C_x,_TOTAL = 2.62 and C_x,_FI = 2.35) were utilized to calculate the calibrated N predicted for 2020, as shown in

Table 15. HSM prediction model estimation compared to observed crashes.

	Severity	2016	2017	2018	2019	Four-Years Median	2020
Observed Crashes	Total	1653	1597	1398	1301	1487	1182
	FI	451	467	406	415	435	389
Predicted Crashes (Npredicted)	Total	565	570	545	587	567	498
	FI	182	186	181	191	185	164
Calibrated Predicted Crashes (Cal. Npredicted)	Total	-	-	-	-	-	1308
	FI	-	-	-	-	-	386
Expected Crashes (EB) (Nexpected)	Total	1622	1581	1402	1298	1476	1205
	FI	457	472	420	422	443	394

. These calibration factors were derived from the baseline data of the four previous years. The EB method was also applied using the obtained parameters to calculate the Nexpected, which represents the expected crash counts considering the observed crash data. By comparing the calibrated Npredicted and Nexpected, it becomes possible to evaluate the prediction model’s performance and assess its accuracy in estimating the crash counts for 2020.

The COVID-19 pandemic has affected crash counts and changed the Average Annual Daily Traffic (AADT) values. Consequently, the N_spf (predicted crash counts) reflects the impact of the pandemic on crash frequencies. Despite this, the N_observed (actual observed crash counts) remains approximately 10% lower than the calibrated N_predicted (predicted crash counts considering calibration factors) for all types of crashes. However, the calibrated prediction of FI crashes aligns closely with the observed counts, indicating high accuracy in predicting FI crashes.

Moreover, applying the EB method, which incorporates the observed number of crashes, brings N_expected (expected crash counts) closer to N_observed. This suggests that the prediction model performs well in estimating unseen data.

In

Table 16. Result of all the GOF tests applied for calibrated predicted crashes, including 2020.

GOF	MAD	MAPE	RMSE	R2
Total (4 years)	4.44	53%	8.59	0.45
FI (4 years)	1.92	78%	3.38	0.24
Total (2020)	1.33	80%	2.30	0.32
FI (2020)	0.62	114%	1.04	0.17

, the evaluation of model performance based on mean absolute deviation (MAD) and root mean square error (RMSE) reveals that FI crashes demonstrate a better model adjustment to the actual inputs compared to all types of crashes. Conversely, the mean absolute percentage error (MAPE) and R² (coefficient of determination) indicate that using all types of crashes yields successful model adjustment. The high MAPE value for 2020 can be attributed to the sudden reduction in crashes, which the model was not explicitly trained to anticipate as it was based on previous years’ data.

After applying the EB method and considering the 2020 data, the GOF (Goodness of Fit) tests demonstrate that the model performs better when using all types of crashes, as shown in

Table 17. Result of all the GOF tests applied for expected crashes, including 2020.

GOF	MAD	MAPE	RMSE	R2
Total (4 years)	0.80	10%	1.33	0.99
FI (4 years)	0.86	35%	1.35	0.88
Total (2020)	0.24	15%	0.37	0.98
FI (2020)	0.29	52%	0.44	0.90

. However, further investigation is required to fully understand the influence of infection prevention and control procedures on road safety in Brazil.

5. Conclusions

Injury crashes and fatalities have significant economic and social costs, impacting the development of a country by increasing medical expenses, insurance claims, and productivity losses. Additionally, these crashes contribute to a higher carbon footprint and infrastructure repair costs. For developing countries like Brazil, the impact is particularly significant. Predictive models, such as the HSM prediction model, can play a crucial role in identifying high-risk areas and situations, enabling targeted interventions to improve road safety, and contribute to a more sustainable transport system. This aligns with the objectives of the 2030 Agenda, which aims to ensure sustainable transport systems that promote economic growth, social inclusion, and environmental sustainability, including reducing the number of road traffic deaths and injuries by 50% by 2030.

The assessment of the HSM prediction model employment during the atypical year of 2020, marked by the COVID-19 pandemic and the resulting changes in traffic patterns, helped understand the temporal transferability of the model. The calibrated prediction model showed promising results, although there was a slight underestimation of crash counts for all types of crashes compared to the observed values. However, the calibrated prediction of fatal and injury crashes (FI crashes) closely matched the observed counts, demonstrating the model’s capability of capturing severe crash fluctuations.

Using all types of crashes in the model yielded better results in most goodness-of-fit tests, indicating that underreporting crashes did not significantly affect the model’s validity. However, it is essential to acknowledge that additional risk factors not accounted for by the Safety Performance Functions (SPFs) may influence road safety outcomes. The study also highlighted the importance of calibration to local conditions and the need to establish “good-enough” thresholds for other contexts, as Brazilian data and road characteristics may differ from those in the original development of the HSM SPFs.

This research provides valuable insights into applying the HSM prediction model for multilane rural highways in Brazil, serving as a reference for safety assessment and guidance for highway administration, municipalities, and toll agencies. It demonstrates the need for calibration and suggests further investigation into roadway characteristics, driver behavior, and crash patterns specific to the Brazilian context to enhance the accuracy of crash predictions. The findings also contribute to understanding SPF transferability and the model’s performance in atypical years.

Future studies should consider using different calibration methods and functions and explore additional goodness-of-fit tests such as cure plots, chi-square, and coefficient of variation. Developing jurisdiction-specific SPFs for local conditions and addressing questions related to SPF calibration frequency and temporal transferability acceptable thresholds would further enhance the knowledge in this research area.