No-Reference Evaluation for Visual Question Answering using Large Language Models and Semi-supervised Learning

Lupo-Condori A.; Maldonado-Quispe P. · Proceedings - International Conference of the Chilean Computer Science Society, SCCC (2025) · DOI: 10.1109/sccc67219.2025.11420451

Resumen

Visual Question Answering (VQA) is a challenging multimodal task that requires the integration of visual and textual information to answer questions about images. While advances in VQA models have improved performance, evaluating their responses remains difficult, as traditional reference-based metrics often fail to capture semantic nuances and handle open-ended scenarios. In this work, we propose a reference-free automatic evaluation approach that leverages pretrained models and Chain-of-Thought (CoT) reasoning to assess the semantic coherence between the image, question, and answer. Experiments on the DAQUAR dataset demonstrate that the proposed approach achieves competitive alignment with human judgment, offering greater flexibility and interpretability than conventional metrics. These results highlight the potential of reference-free evaluation to enhance the robustness, transparency, and applicability of VQA systems in practical domains, including medicine and education.

Instituciones: Universidad Nacional de San Agustín de Arequipa