72a1d9d426
* reordering of the episode reset operation and allowing to store episodes only when they are terminated * reordering of the episode reset operation and allowing to store episodes only when they are terminated * revert tensorflow-gpu to 1.9.0 + bug fix in should_train() * tests readme file and refactoring of policy optimization agent train function * Update README.md * Update README.md * additional policy optimization train function simplifications * Updated the traces after the reordering of the environment reset * docker and jenkins files * updated the traces to the ones from within the docker container * updated traces and added control suite to the docker * updated jenkins file with the intel proxy + updated doom basic a3c test params * updated line breaks in jenkins file * added a missing line break in jenkins file * refining trace tests ignored presets + adding a configurable beta entropy value * switch the order of trace and golden tests in jenkins + fix golden tests processes not killed issue * updated benchmarks for dueling ddqn breakout and pong * allowing dynamic updates to the loss weights + bug fix in episode.update_returns * remove docker and jenkins file
Coach Benchmarks
The following table represents the current status of algorithms implemented in Coach relative to the results reported in the original papers. The detailed results for each algorithm can be seen by clicking on its name.
The X axis in all the figures is the total steps (for multi-threaded runs, this is the number of steps per worker). The Y axis in all the figures is the average episode reward with an averaging window of 100 timesteps.
For each algorithm, there is a command line for reproducing the results of each graph. These are the results you can expect to get when running the pre-defined presets in Coach.
The environments that were used for testing include:
- Atari - Breakout, Pong and Space Invaders
- Mujoco - Inverted Pendulum, Inverted Double Pendulum, Reacher, Hopper, Half Cheetah, Walker 2D, Ant, Swimmer and Humanoid.
- Doom - Basic, Health Gathering (D1: Basic), Health Gathering Supreme (D2: Navigation), Battle (D3: Battle)
- Fetch - Reach, Slide, Push, Pick-and-Place
Summary
Reproducing paper's results
Reproducing paper's results for some of the environments
Training but not reproducing paper's results
Not training
| Status | Environments | Comments | |
|---|---|---|---|
| DQN | Atari | ||
| Dueling DDQN | Atari | ||
| Dueling DDQN with PER | Atari | ||
| Bootstrapped DQN | Atari | ||
| QR-DQN | Atari | ||
| A3C | Atari, Mujoco | ||
| Clipped PPO | Mujoco | ||
| DDPG | Mujoco | ||
| NEC | Atari | ||
| HER | Fetch | ||
| HAC | Pendulum | ||
| DFP | Doom | Doom Battle was not verified |
Click on each algorithm to see detailed benchmarking results