Indexat a

Llicència i ús

Citacions

Cited 13 times in

Cited 1 times in

Altmetrics

Grant support

This work was supported in part by the Horizon Europe CODECO Project under Grant 101092696, in part by the Horizon Europe NEMO Project under Grant 101070118, and in part by the UNICO-5G I+D (B5GEMINI-AIUC) Project funded by the Ministry of Economic Affairs and Digital Transformation of the Spanish Government and the NextGeneration EU (Recovery, Transformation and Resilience Plan-PRTR) under Grant TSI063000-2021-79.

Anàlisi d'autories institucional

Compartir

Comparative Analysis of A3C and PPO Algorithms in Reinforcement Learning: A Survey on General Environments

Publicat a:Ieee Access. 12 146795-146806 - 2024-01-01 12(), DOI: 10.1109/ACCESS.2024.3472473

Autors: del Rio, A; Jimenez, D; Serrano, J

Afiliacions

Resum

This research article presents a comparison between two mainstream Deep Reinforcement Learning (DRL) algorithms, Asynchronous Advantage Actor-Critic (A3C) and Proximal Policy Optimization (PPO), in the context of two diverse environments: CartPole and Lunar Lander. DRL algorithms are widely known for their effectiveness in training agents to navigate complex environments and achieve optimal policies. Nevertheless, a methodical assessment of their effectiveness in various settings is crucial for comprehending their advantages and disadvantages. In this study, we conduct experiments on the CartPole and Lunar Lander environments using both A3C and PPO algorithms. We compare their performance in terms of convergence speed and stability. Our results indicate that A3C typically achieves quicker training times, but exhibits greater instability in reward values. Conversely, PPO demonstrates a more stable training process at the expense of longer execution times. An evaluation of the environment is needed in terms of algorithm selection, based on specific application needs, balancing between training time and stability. A3C is ideal for applications requiring rapid training, while PPO is better suited for those prioritizing training stability.

Paraules clau