Planning the Movement of Robots in a Social Environment Via Reinforcement Learning

Abstract

The work is devoted to the problem of controlling the movement of robots in a social environment in crowded places. An algorithm for planning the movement of mobile robots among stationary and moving obstacles using reinforcement learning has been developed and studied. The GA3C-CADRL algorithm was chosen as a prototype, in which the robot and obstacles are considered as interacting agents. The algorithm was modified and implemented using an LSTM recurrent neural network to approximate the value and policy functions simultaneously. The neural network was trained on a common data set obtained through actor-critic reinforcement learning. Additionally, the rl_ planner and social_msgs components have been developed to integrate a pre-trained planning algorithm into a robot control system on the Robot Operating System 2 software platform. The first component implements processing of input data, calculating the robot’s actions and generating the required speed of movement, and the second contains messages with information about neighboring agents. To test the algorithm, experiments were carried out with three different scenarios: – with static obstacles, – mixed, – with dynamic agents. The number of episodes for training the algorithm with 5 agents reached 1,500,000. Simulation of the movement of a robot on two tracks in the environment Gazebo showed that in conditions of static obstacles the robot reaches the goal in the shortest time. In the presence of dynamic obstacles, the time increased by 2 times due to collision avoidance. At the same time, the distance to the nearest agent remained safe (more than 2 meters).