DataTransformer: Software Develpment for Named Entity Normalization Using Deep Learning: "EWUNN-NEN"

Chavez R.; Torres D.; Salas C. · Proceedings - 2025 IEEE International Conference on Future Machine Learning and Data Science, FMLDS 2025 (2025) · DOI: 10.1109/fmlds67896.2025.00070

Resumen

Data quality is a critical requirement in processes such as data analysis, auditing, and decision-making. One of the main challenges lies in named entity normalization (NEN) within high-cardinality datasets, where multiple mentions may refer to the same entity, or conversely, a single mention may correspond to different entities. In this context, we present DataTransformer, a web-based solution focused on named entity normalization, with a particular emphasis on company names in the Peruvian business sector. The application integrates Deep Learning techniques through fine-tuning of the BERT model, alongside graph-based methods, specifically leveraging the Edge Weight Updating Neural Network (EWUNN) model. This approach enables the alignment of variant mentions to their canonical identifiers by minimizing the divergence between similarity and ground-truth graphs. The system was implemented in Python using PyTorch for parameter manipulation and Hugging Face's Transformers library for BERT integration. During the evaluation phase, the system achieved an Accuracy of 91%, F1 Score of 89%, and Recall of 89% on unseen mentions.

Instituciones: Universidad Peruana de Ciencias Aplicadas