Recent developments in Deep Reinforcement Learning (DRL) have demonstrated the potential of agents to perform in high-dimensional state spaces. In this paper, we investigate the impact of representation on agent performance in the MountainCar environment. We present a range of representation options with corresponding neural network architectures and resolutions. By comparing the performance of agents trained with varying representations using proximal policy optimization, we demonstrate the value of utilizing higher resolution representations. Finally, we demonstrate that a performant actor, receiving as input the high-dimensional rendering of the environment, can be trained by providing the critic with the Radial Basis Function (RBF) representation as input.
Article ID: 2023GL10
Publisher: Canadian Artificial Intelligence Association