Metadata-Version: 1.1
Name: Surreal
Version: 0.1.1
Summary: Stanford University Repository for Reinforcement Algorithms
Home-page: http://github.com/StanfordVL/Surreal
Author: Stanford Vision and Learning Lab
Author-email: UNKNOWN
License: GPLv3
Description-Content-Type: UNKNOWN
Description: `SURREAL <https://surreal.stanford.edu>`__
        ==========================================
        
        | `About <#open-source-distributed-reinforcement-learning-framework>`__
        | `Installation <#installation>`__
        | `Benchmarking <#benchmarking>`__
        | `Citation <#citation>`__
        
        Open-Source Distributed Reinforcement Learning Framework
        --------------------------------------------------------
        
        *Stanford Vision and Learning Lab*
        
        `SURREAL <https://surreal.stanford.edu>`__ is a fully integrated
        framework that runs state-of-the-art distributed reinforcement learning
        (RL) algorithms.
        
        .. container::
        
        -  **Scalability**. RL algorithms are data hungry by nature. Even the
           simplest Atari games, like Breakout, typically requires up to a
           billion frames to learn a good solution. To accelerate training
           significantly, SURREAL parallelizes the environment simulation and
           learning. The system can easily scale to thousands of CPUs and
           hundreds of GPUs.
        
        -  **Flexibility**. SURREAL unifies distributed on-policy and off-policy
           learning into a single algorithmic formulation. The key is to
           separate experience generation from learning. Parallel actors
           generate massive amount of experience data, while a *single,
           centralized* learner performs model updates. Each actor interacts
           with the environment independently, which allows them to diversify
           the exploration for hard long-horizon robotic tasks. They send the
           experiences to a centralized buffer, which can be instantiated as a
           FIFO queue for on-policy mode and replay memory for off-policy mode.
        
        .. raw:: html
        
           <!--<img src=".README_images/distributed.png" alt="drawing" width="500" />-->
        
        -  **Reproducibility**. RL algorithms are notoriously hard to reproduce
           [Henderson et al., 2017], due to multiple sources of variations like
           algorithm implementation details, library dependencies, and hardware
           types. We address this by providing an *end-to-end integrated
           pipeline* that replicates our full cluster hardware and software
           runtime setup.
        
        .. raw:: html
        
           <!--<img src=".README_images/pipeline.png" alt="drawing" height="250" />-->
        
        Installation
        ------------
        
        | Surreal algorithms can be deployed at various scales. It can run on a
          single laptop and solve easier locomotion tasks, or run on hundreds of
          machines to solve complex manipulation tasks.
        | \* `Surreal on your Laptop <docs/surreal_tmux.md>`__
        | \* `Surreal on Google Cloud Kubenetes
          Engine <docs/surreal_kube_gke.md>`__
        | \* `Customizing Surreal <docs/contributing.md>`__
        | \* `Documentation Index <docs/index.md>`__
        
        Benchmarking
        ------------
        
        -  Scalability of Surreal-PPO with up to 1024 actors on Surreal Robotics
           Suite.
        
        |image0|
        
        -  Training curves of 16 actors on OpenAI Gym tasks for 3 hours,
           compared to other baselines.
        
        Citation
        --------
        
        Please cite our CORL paper if you use this repository in your
        publications:
        
        ::
        
           @inproceedings{corl2018surreal,
             title={SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark},
             author={Fan, Linxi and Zhu, Yuke and Zhu, Jiren and Liu, Zihua and Zeng, Orien and Gupta, Anchit and Creus-Costa, Joan and Savarese, Silvio and Fei-Fei, Li},
             booktitle={Conference on Robot Learning},
             year={2018}
           }
        
        .. |image0| image:: .README_images/scalability-robotics.png
Keywords: Reinforcement Learning,Deep Learning,Distributed Computing
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Environment :: Console
Classifier: Programming Language :: Python :: 3
