Development of a Virtual Assistant Based on LLMs for the Knowledge Domain in Biomedical Metrology

Carreon Y.; Chicchon M. · International Journal of Online and Biomedical Engineering (2025) · DOI: 10.3991/ijoe.v21i09.55653

Resumen

Accurate measurements are essential for effective diagnosis and treatment in the healthcare sector. However, there is limited training in biomedical metrology for Biomedical Engineers, which may hinder their performance. This study evaluated the knowledge of large language models (LLMs) in biomedical metrology to develop a specialized virtual assistant that sup-ports these professionals. The effectiveness of the LLMs was assessed based on the accuracy and coherence of their responses using the CBET Certification exam and metrics such as Rouge-L, F1 score, and cosine similarity. The Llama 3.2-3B Mini model, optimized with retriev-al-augmented generation (RAG), showed an increase in the F1 score from 0.402 to 0.526, a Rouge-L score of 0.497, and a cosine similarity of 0.657, demonstrating its ability to gener-ate relevant and accurate responses. The developed virtual assistant represents a promising tool for improving the training and performance of biomedical engineers, ensuring access to precise and reliable information, thereby strengthening safety in the healthcare sector. Our source code is publicly available at https://github.com/yamilet2662/assistant_biome.

Instituciones: Universidad Tecnológica del Perú