Evaluating Matching Frontier Methods for Crash Modification Factor Estimation

Usama Elrawy Shahdah, Associate Professor, Public Works Engineering Department, Faculty of Engineering, Mansoura University, Mansoura, Egypt 35516Follow
Eman K . Ali, Assistant Professor, Public Works Engineering Department, Faculty of Engineering, Mansoura University, Mansoura 35516, Egypt.
Sania Reyad Elagamy, Assistant Professor, Public Works Engineering Department, Faculty of Engineering, Mansoura University, Mansoura 35516, Egypt.

Corresponding Author

Usama E. Shahdah

Subject Area

Civil and Environmental Engineering

Article Type

Original Study

Abstract

This study evaluates "matching frontier" methods against conventional approaches for estimating Crash Modification Factors (CMFs). Thirty-four method-model combinations were tested, including 16 frontier variants, 8 propensity-score-matching algorithms with negative binomial and mixed-effects models, and two benchmarks (empirical Bayes and cross-sectional), across multiple sample sizes and three true CMF values (0.80, 1.00, and 1.30). Of these, 30 converged successfully; 4 mixed-effects models with replacement-based matching failed due to duplicate control observations. Each scenario was evaluated across 10 Monte Carlo replications to quantify performance uncertainty.

Synthetic crash data were generated with known treatment effects, examining scenarios ranging from ideal (386 treated, 21,000 controls) to constrained (100 treated, 2,000 controls).

Results indicate that the empirical Bayes (EB) method performed competitively across scenarios, with mean relative errors of 1.5% (CMF=0.80), −0.4% (CMF=1.00), and 2.5% (CMF=1.30) under abundant data. Frontier methods demonstrated critical limitations, performing adequately only when control-to-treated ratios exceeded 20:1. Energy distance was the best frontier metric, though its marginal improvement did not justify substantial computational costs. Mixed-effects models on matched samples exhibited severe under-coverage, with empirical confidence interval coverage often below 50% despite a nominal 95% level.

These findings suggest EB is a robust choice for CMF estimation and reveal fundamental weaknesses in frontier methods under typical safety evaluation sample sizes. The inability of sophisticated optimization to surpass a simple heuristic indicates current balance metrics do not reliably translate into accurate treatment effect estimates. However, the data-generating process shares structural similarities with the EB framework, which may partially explain its favorable performance.

Keywords

crash modification factors; matching frontier; Empirical Bayes; Monte Carlo simulation; causal inference; road safety analysis

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 License.

Recommended Citation

Shahdah, Usama Elrawy; . Ali, Eman K; and Elagamy, Sania Reyad (2026) "Evaluating Matching Frontier Methods for Crash Modification Factor Estimation," Mansoura Engineering Journal: Vol. 51 : Iss. 6 , Article 4.
Available at: https://doi.org/10.58491/2735-4202.3495

Download