MBRL-Lib: A Modular Library for Model-based Reinforcement Learning

ABSTRACT:

Model-based reinforcement learning is a compelling framework for data-efficient learning of agents that interact with the world. This family of algorithms has many subcomponents that need to be carefully selected and tuned. As a result the entry-bar for researchers to approach the field and to deploy it in real-world tasks can be daunting. In this paper, we present MBRL-Lib -- a machine learning library for model-based reinforcement learning in continuous state-action spaces based on PyTorch. MBRL-Lib is designed as a platform for both researchers, to easily develop, debug and compare new algorithms, and non-expert user, to lower the entry-bar of deploying state-of-the-art algorithms. MBRL-Lib is open-source at https://github.com/facebookresearch/mbrl-lib.

For up to date news on this repository, please navigate to the repository here: https://github.com/facebookresearch/mbrl-lib

We believe there is a lot of untapped potential in MBRL, and that a modular, easy-to-use and well-tested library will increase growth in MBRL research. MBRL-Lib is designed to facilitate development of new algorithms, and make it easy to mix & match models and controllers. We provide code to easily train & simulate dynamics models, including probabilistic ensembles. By following a minimal model API, you can swap between models without changing your training pipeline, reducing boilerplate and helping you focus on high-level research questions.

Same philosophy works for our Agent interface, letting you swap controllers & control optimizers, all via configuration files. We provide controller implementations, such as this CEM-based MPC agent which breaks HalfCheetah when controlling on true dynamics. We also think it's really important to have good easy-to-run visualization and debugging tools, and our released version already includes some that we have found useful (more to come!).

Citation

@Article{Pineda2021MBRL,
author = {Luis Pineda and Brandon Amos and Amy Zhang and Nathan O. Lambert and Roberto Calandra},
journal = {Arxiv},
title = {MBRL-Lib: A Modular Library for Model-based Reinforcement Learning},
year = {2021},
url = {https://arxiv.org/abs/2104.10159},
}

MBRL-Lib: A Modular Library for Model-based Reinforcement Learning

What you need to know:

Citation

Links

Video

Tags