Benchmarking of BERT-based Prediction Models for Bibliographic References Classification

Medina-Mamani V.; Gutierrez-Choque A.-C.; Castro-Gutierrez E.; Núñez-Pacheco R.; Aguaded I. · RISTI - Revista Iberica de Sistemas e Tecnologias de Informacao (2024)

Resumen

Giving credit to ideas that are not ours through bibliographic references is an essential part of writing a scientific and/or academic work. This task is very tedious for the reviewer to perform, because there is a need to review the completeness of the bibliographic references. Using Artificial Intelligence techniques, this task should be done automatically. In addition, it has been detected that an incomplete citation makes access to the referenced work difficult or impossible. This article presents two binary classification prediction models based on Bidirectional Encoder Representation from Transformers (BERT) for the analysis and classification of bibliographic references in the APA format. The two proposed prediction models are assigned a unique dataset of 16002 bibliographic references. The first model obtained through the manual generation data augmentation technique achieved a score of 0.99 in the accuracy, F1 score and recall metrics, while it obtained 0.98 in the precision metric. Likewise, in the prediction model with the automated generation data augmentation technique, the score of 0.99 was obtained in accuracy, recall, F1 score and precision. The results indicate that the proposed models are achieving higher performance compared to the literature models.

Instituciones: Universidad Nacional de San Agustín de Arequipa