Reducing Time Steps Needed of Multi-Agent Reinforcement Methods in a Dynamic Navigation Environment

Abstract

In a simpler version of the delivery variation of the multi-agent pathfinding problem, we use a reinforcement learning approach instead of other common traditional algorithmic heuristics to tackle this NP-hard problem. The paper proposes using either fully decentralized or decentralized data and a centralized model to alleviate the problem of requiring large training time steps in multi-agent reinforcement learning by reducing the number of time steps needed, especially in an environment with cumulative rewards. These approaches don’t require the concatenation of local data among agents, and, as such, are relieved of the computational burden of increased data complexity. The proposed approaches were tested in an environment where multiple agents try to work together and clean the maximum number of total nonregenerative unclean tiles in a grid filled with obstacles in a set amount of time. In a final test on a randomized dynamic environment with varying grid sizes, one of our approaches proved very promising in application and scalability. This paper's most commonly used algorithm was proximal policy optimization, with brief mentions of deep q-network. Different implementations of these algorithms had their training tracked and recorded, including Stablebaseline3 and self-implementation using TensorFlow.