BERT-Based Approach for Quechua Dialect Identification

Solaligue C.A.; Santisteban J. · Communications in Computer and Information Science (2026) · DOI: 10.1007/978-3-032-20322-9_11

Resumen

The Identification of dialectal variants poses a significant challenge in indigenous languages due to their complex morphology and the limited availability of annotated data. In the case of Quechua, an agglutinative language with extensive dialectal diversity. This study proposes a Transformer-based approach, specifically using the BERT models, to classify Quechua dialects under low-resource conditions. DeBERTa was selected due to its disentangled attention mechanism and enhanced mask decoder, which enable more effective modeling of contextual relationships and positional information. The methodology involves collecting a multi-dialect Quechua corpus, performing data cleaning and normalization. A stratified cross-validation strategy and hyperparameter optimization using Optuna were implemented to improve model performance. Several architectures, including BERT, DeBERTa, RoBERTa, and ConvBERT, were evaluated under equivalent conditions. Results show that the models achieved improvement on the state of the art with a 0.983 precision and recall in Quechua del Cusco - QUZ with a much larger dataset, demonstrating their efficiency even in linguistically low-resource settings, while ConvBERT has the best performance. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2026.

Instituciones: Universidad Católica San Pablo