Analysis of Performance and Fairness of Classification Algorithms with Class and Protected Attribute Noise

Vargas A.; Delgadillo R. · Journal of the Brazilian Computer Society (2026) · DOI: 10.5753/jbcs.2026.5960

Resumen

Noise in data is an underexplored source of bias. This study investigates the impact of noise on both the performance and fairness of three classic classifiers: Random Forest (RF), Support Vector Machine (SVM), and Logistic Regression (RL), using four public datasets for binary classification. A comparative analysis is conducted to examine the effects of different types and levels of noise introduced in a predictor variable (protected attribute) and/or the response variable (class). Performance is measured using accuracy (ACC), while model fairness is evaluated using the Average Absolute Odds Difference (AAOD) metric, with a binary attribute as the protected. We results suggest that injecting noise into both the class label and the protected attribute (cn) yields results similar to injecting noise only into the labels (ln). The observed robustness of the Average Absolute Odds Difference (AAOD) under these conditions warrants careful interpretation. As AAOD values tend to decrease or remain stable despite increasing noise levels, there is a risk that the metric may be insensitive to noise. This suggests that AAOD might artificially report higher fairness in scenarios where significant underlying disparities actually persist.

Instituciones: Universidad Nacional Mayor de San Marcos