Metadata-Version: 2.0
Name: Surreal
Version: 0.1.1
Summary: Stanford University Repository for Reinforcement Algorithms
Home-page: http://github.com/StanfordVL/Surreal
Author: Stanford Vision and Learning Lab
Author-email: UNKNOWN
License: GPLv3
Description-Content-Type: UNKNOWN
Keywords: Reinforcement Learning,Deep Learning,Distributed Computing
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Environment :: Console
Classifier: Programming Language :: Python :: 3
Requires-Dist: gym
Requires-Dist: robosuite
Requires-Dist: mujoco-py (<1.50.2,>=1.50.1)
Requires-Dist: tabulate
Requires-Dist: tensorflow
Requires-Dist: tensorboardX
Requires-Dist: imageio
Requires-Dist: pygame
Requires-Dist: benedict
Requires-Dist: nanolog
Requires-Dist: psutil
Requires-Dist: tabulate
Requires-Dist: imageio
Requires-Dist: caraml (>=0.10.0)
Requires-Dist: symphony (>=0.9.1)
Requires-Dist: torchx
Requires-Dist: tensorplex
Requires-Dist: cloudwise (>=0.1.1)
Requires-Dist: Cython (<0.29)

`SURREAL <https://surreal.stanford.edu>`__
==========================================

| `About <#open-source-distributed-reinforcement-learning-framework>`__
| `Installation <#installation>`__
| `Benchmarking <#benchmarking>`__
| `Citation <#citation>`__

Open-Source Distributed Reinforcement Learning Framework
--------------------------------------------------------

*Stanford Vision and Learning Lab*

`SURREAL <https://surreal.stanford.edu>`__ is a fully integrated
framework that runs state-of-the-art distributed reinforcement learning
(RL) algorithms.

.. container::

-  **Scalability**. RL algorithms are data hungry by nature. Even the
   simplest Atari games, like Breakout, typically requires up to a
   billion frames to learn a good solution. To accelerate training
   significantly, SURREAL parallelizes the environment simulation and
   learning. The system can easily scale to thousands of CPUs and
   hundreds of GPUs.

-  **Flexibility**. SURREAL unifies distributed on-policy and off-policy
   learning into a single algorithmic formulation. The key is to
   separate experience generation from learning. Parallel actors
   generate massive amount of experience data, while a *single,
   centralized* learner performs model updates. Each actor interacts
   with the environment independently, which allows them to diversify
   the exploration for hard long-horizon robotic tasks. They send the
   experiences to a centralized buffer, which can be instantiated as a
   FIFO queue for on-policy mode and replay memory for off-policy mode.

.. raw:: html

   <!--<img src=".README_images/distributed.png" alt="drawing" width="500" />-->

-  **Reproducibility**. RL algorithms are notoriously hard to reproduce
   [Henderson et al., 2017], due to multiple sources of variations like
   algorithm implementation details, library dependencies, and hardware
   types. We address this by providing an *end-to-end integrated
   pipeline* that replicates our full cluster hardware and software
   runtime setup.

.. raw:: html

   <!--<img src=".README_images/pipeline.png" alt="drawing" height="250" />-->

Installation
------------

| Surreal algorithms can be deployed at various scales. It can run on a
  single laptop and solve easier locomotion tasks, or run on hundreds of
  machines to solve complex manipulation tasks.
| \* `Surreal on your Laptop <docs/surreal_tmux.md>`__
| \* `Surreal on Google Cloud Kubenetes
  Engine <docs/surreal_kube_gke.md>`__
| \* `Customizing Surreal <docs/contributing.md>`__
| \* `Documentation Index <docs/index.md>`__

Benchmarking
------------

-  Scalability of Surreal-PPO with up to 1024 actors on Surreal Robotics
   Suite.

|image0|

-  Training curves of 16 actors on OpenAI Gym tasks for 3 hours,
   compared to other baselines.

Citation
--------

Please cite our CORL paper if you use this repository in your
publications:

::

   @inproceedings{corl2018surreal,
     title={SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark},
     author={Fan, Linxi and Zhu, Yuke and Zhu, Jiren and Liu, Zihua and Zeng, Orien and Gupta, Anchit and Creus-Costa, Joan and Savarese, Silvio and Fei-Fei, Li},
     booktitle={Conference on Robot Learning},
     year={2018}
   }

.. |image0| image:: .README_images/scalability-robotics.png