Improving Tree Structure with SR-CCP: A Regularized Pruning Strategy for Shallower and Simpler Decision Models

Torres-Chávez J.; Rivera-Olivera O.; Sucari-Ccansaya E.; Flores-Quispe R.; Velazco-Paredes Y. · International Journal of Computing and Digital Systems (2026) · DOI: 10.12785/ijcds/1571203472

Resumen

Decision trees are widely used for classification because of their interpretability and efficiency; however, they are prone to overfitting and excessive structural complexity when trained on noisy or high-dimensional data sets. Although Cost-Complexity Pruning (CCP) reduces tree size by penalizing terminal nodes, it does not explicitly control other structural characteristics such as tree depth and feature usage. This paper proposes Structural Regularized Cost-Complexity Pruning (SR-CCP), an extension of CCP that introduces structural regularization by incorporating penalties on tree depth and feature utilization into the pruning criterion. The proposed strategy combines the traditional complexity penalty with the generalization gap and structural constraints to promote shallower, simpler, and more interpretable decision trees while preserving predictive performance. SR-CCP was evaluated on seven public benchmark data sets using five-fold stratified cross-validation and compared with five pruning approaches (Baseline, CCP, REP, MEP, and DBP), with XGBoost included as an ensemble-based reference model. Performance was assessed using accuracy, F1 score, tree depth, number of nodes, leaves, features used, and inference time. Experimental results show that SR-CCP achieved the highest average accuracy (89.47%) and F1 score (87.72%) among the evaluated methods, performing within one percentage point of XGBoost (90.44% accuracy and 88.09% F1 score). At the same time, it dramatically reduced structural complexity by decreasing the average number of nodes from 142.3 to 9.1 and the average number of leaves from 71.7 to 5.0, a reduction confirmed to be statistically significant against every baseline pruning method (Friedman and post-hoc Wilcoxon/paired t-tests, Benjamini–Hochberg corrected, p < 0.05). These results demonstrate that SR-CCP produces substantially more compact and interpretable decision trees without sacrificing predictive performance.

Instituciones: Universidad Nacional de San Agustín de Arequipa