Deep Reinforcement Learning-Based Intelligent Water Level Control: From Simulation to Embedded Implementation

Cusihuallpa-Huamanttupa K.; Sacoto-Cabrera E.J.; Coaquira-Castillo R.J.; Utrilla Mego L.W.; Herrera-Levano J.C.; Concha-Ramos Y.; Moreno-Cardenas E. · Sensors (2026) · DOI: 10.3390/s26010245

Resumen

This article presents the design, simulation, and real-time implementation of an intelligent water level control system using Deep Reinforcement Learning (DRL) with the Deep Deterministic Policy Gradient (DDPG) algorithm. The control policy was initially trained in a MATLAB-based simulation environment, where actor–critic neural networks were trained and optimized to ensure accurate and robust performance under dynamic and nonlinear conditions. The trained policy was subsequently deployed on a low-cost embedded platform (Arduino Uno), demonstrating its feasibility for real-time embedded applications. Experimental results confirm the controller’s ability to adapt to external disturbances. Quantitatively, the proposed controller achieved a steady-state error of less than 0.05 cm and an overshoot of 16% in the physical implementation, outperforming conventional proportional–integral–derivative (PID) control by 22% in tracking accuracy. The combination of the DDPG algorithm and low-cost hardware implementation demonstrates the feasibility of real-time deep learning-based control for intelligent water management. Furthermore, the proposed architecture is directly applicable to low-cost Internet of Things (IoT)-based water management systems, enabling autonomous and adaptive control in real-world hydraulic infrastructures. This proposal demonstrates its potential for smart agriculture, distributed sensor networks, and scalable and resource-efficient water systems. Finally, the main novelty of this work is the deployment of a DRL-based controller on a resource-constrained microcontroller, validated under real-world perturbations and sensor noise.

Instituciones: Universidad Nacional de San Antonio Abad del Cusco; Universidad Continental