Flock Navigation by Coordinated Shepherds via Reinforcement Learning

Hasan Y.; Baxter J.E.G.; Salcedo C.A.; Delgado E.; Tapia L. · Springer Proceedings in Advanced Robotics (2023) · DOI: 10.1007/978-3-031-21090-7_27

Resumen

Shepherding is the problem of guiding a group of passive sheep agents (a flock) from some start position to a goal region by influencing the sheep motion with active guiding agents (the shepherds). Existing solutions are limited, as heuristic solutions are often designed and tuned for specific environments and flock dynamics, while learning solutions are often limited to a single shepherd and/or few sheep agents, fixed flock dynamics, discrete environments, or rely on tuning of heuristic solutions. In order to provide a more general shepherding solution, we contribute a mapping of this problem to a previously addressed planning problem: active agents protecting a moving payload from passive agents in a crowd. This inversion of an existing solution directly facilitates the creation of a deep reinforcement learning (deep RL) model that provides cooperation between multiple shepherd agents, shows robustness to changes in flock dynamics, and requires no predefined shepherding strategy. We experiment on the effect of varying the number of sheep agents and the number of shepherd agents to gauge the performance and scaling of each method. We also test our method’s robustness to positional observation noise and changed flock dynamics both with and without re-training. The experiments show that our deep RL solution shepherds as well as tuned heuristic methods, often with a reduced path length of the shepherds. Our solution also exhibited robustness to environmental situations that were unseen during training and high adaptability with simple re-training.

Instituciones: Universidad de Ingeniería y Tecnología