FormalStyler: GPT based Model for Formal Style Transfer based on Formality and Meaning Preservation

De Rivero M.; Tirado C.; Ugarte W. · International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, IC3K - Proceedings (2021) · DOI: 10.5220/0010674300003064

Resumen

Style transfer is a natural language processing generation task, it consists of substituting one given writing style for another one. In this work, we seek to perform informal-to-formal style transfers in the English language. This process is shown in our web interface where the user input a informal message by text or voice. This project's target audience are students and professionals in the need to improve the quality of their work by formalizing their texts. A style transfer is considered successful when the original semantic meaning of the message is preserved after the independent style has been replaced. This task is hindered by the scarcity of training and evaluation datasets alongside the lack of metrics. To accomplish this task we opted to utilize OpenAI's GPT-2 Transformer-based pre-trained model. To adapt the GPT-2 to our research, we finetuned the model with a parallel corpus containing informal text entries paired with the equivalent formal ones. We evaluate the fine-tuned model results with two specific metrics, formality and meaning preservation. To further fine-tune the model we integrate a human-based feedback system where the user selects the best formal sentence out of the ones generated by the model. The resulting evaluations of our solution exhibit similar to improved scores in formality and meaning preservation to state-of-the-art approaches. Copyright

Instituciones: Universidad Peruana de Ciencias Aplicadas