Cyber-Physical System Using Computer Vision and Gesture Recognition for Home Automation and Telecare

Yauri R.; Espino R. · Emerging Science Journal (2026) · DOI: 10.28991/esj-2026-010-04-01

Resumen

The integration of cyber-physical technologies with computer vision and gesture control is an important alternative for the creation of remote assistance home automation systems for people with reduced mobility. Previous research describes the use of contactless interaction technologies applied to health and home automation systems, but there are challenges in terms of integration, lack of compatibility with conventional systems, and multimodal solutions for users with reduced mobility. Therefore, this paper presents the development of a cyber-physical and computer vision system for hand gesture recognition in home automation applications. The objectives are to integrate sensor and actuator components with an ESP32 module, an algorithm for gesture processing and information visualization using augmented reality, and the development of the control stage with Node-Red and a mobile application. The results demonstrate the portability of the system and its adaptation to remote assistance and home automation scenarios. In addition, gesture recognition techniques were validated using MediaPipe, Python, and OpenCV, as well as user interfaces developed for mobile and desktop applications. As a novelty of the research, an accessible solution is provided that combines computer vision, gesture control, and IoT communication via MQTT, providing natural interaction with real-time remote monitoring to improve people's autonomy and safety.

Instituciones: Universidad Tecnológica del Perú; Universidad Nacional Tecnológica de Lima Sur