Kinematic Control for the Motion Generation of Robot Manipulators Using MoMask LLM

Obludzyner A.; Zaldivar F.; Ramos O.E. · Proceedings of the 2024 IEEE 31st International Conference on Electronics, Electrical Engineering and Computing, INTERCON 2024 (2024) · DOI: 10.1109/intercon63140.2024.10833232

Resumen

In recent years, numerous Large Language Models have been developed, offering a wide variety of applications. Among these, motion generation for 3D human representations has gained attention due to its potential in simulations, animations, and video games. However, the application of LLMs to robot motion is still in its early stages. This paper presents the use of an LLM to generate motion for a robot manipulator simulated in RViz within ROS. We obtain human animated motion from a pre-trained LLM and retarget it to a manipulator robot in the task space, using as example the UR5 and Sawyer robots. This reference is then used as input to a nulls pace hierarchical kinematic controller that follows the hand and elbow motion. The results show that the proposed approach generates motion that resembles the movements of a human character, achieving a maximum error of 0.04 m for the UR5 and 0.03 m for the Sawyer Robot.

Instituciones: Universidad de Ingeniería y Tecnología