Mobile Application for Identification and Information of Medications for the Visually Impaired by OCR and Generative AI

Riega Ocharan W.M.; Palomino Santa Cruz B.M.; Egusquiza F.D.S. · Communications in Computer and Information Science (2026) · DOI: 10.1007/978-3-032-22641-9_12

Resumen

Identifying medications and accessing relevant information about them remains a significant challenge for people with visual impairments, who face barriers both in recognizing packaging and in obtaining accessible medical information. This paper presents VisionCare, a mobile application specifically designed for this population, which combines optical character recognition (OCR) and generative artificial intelligence to identify and query medication information in a fast and accessible way. The solution integrates the Azure Vision OCR service to extract printed text from drug packaging and the GPT-4 model to interpret and present critical information, such as medication name and expiration date, in clear text, along with extended details, including description, side effects, instructions, and warnings, which are also available through text-to-speech playback. The interface follows accessibility guidelines, enabling interaction through screen readers, voice commands, and adjustable text sizes. The system was evaluated through two experiments: a technical assessment of recognition accuracy using 29 medication images, and a usability study with visually impaired participants using the meCUE 2.0 questionnaire. The results showed 100% precision in the recognition of the name of the medication, 96.3% in the detection of the expiration date, and positive user perceptions of usefulness, usability and emotional impact. These findings suggest that VisionCare offers a practical and inclusive approach to improving autonomy and medication safety for visually impaired individuals. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2026.

Instituciones: Universidad Peruana de Ciencias Aplicadas