74db141d5e
* SAC algorithm * SAC - updates to agent (learn_from_batch), sac_head and sac_q_head to fix problem in gradient calculation. Now SAC agents is able to train. gym_environment - fixing an error in access to gym.spaces * Soft Actor Critic - code cleanup * code cleanup * V-head initialization fix * SAC benchmarks * SAC Documentation * typo fix * documentation fixes * documentation and version update * README typo
Coach Benchmarks
The following table represents the current status of algorithms implemented in Coach relative to the results reported in the original papers. The detailed results for each algorithm can be seen by clicking on its name.
The X axis in all the figures is the total steps (for multi-threaded runs, this is the number of steps per worker). The Y axis in all the figures is the average episode reward with an averaging window of 100 timesteps.
For each algorithm, there is a command line for reproducing the results of each graph. These are the results you can expect to get when running the pre-defined presets in Coach.
The environments that were used for testing include:
- Atari - Breakout, Pong and Space Invaders
- Mujoco - Inverted Pendulum, Inverted Double Pendulum, Reacher, Hopper, Half Cheetah, Walker 2D, Ant, Swimmer and Humanoid.
- Doom - Basic, Health Gathering (D1: Basic), Health Gathering Supreme (D2: Navigation), Battle (D3: Battle)
- Fetch - Reach, Slide, Push, Pick-and-Place
Summary
Reproducing paper's results
Reproducing paper's results for some of the environments
Training but not reproducing paper's results
Not training
| Status | Environments | Comments | |
|---|---|---|---|
| DQN | Atari | ||
| Dueling DDQN | Atari | ||
| Dueling DDQN with PER | Atari | ||
| Bootstrapped DQN | Atari | ||
| QR-DQN | Atari | ||
| A3C | Atari, Mujoco | ||
| ACER | Atari | ||
| Clipped PPO | Mujoco | ||
| DDPG | Mujoco | ||
| SAC | Mujoco | ||
| NEC | Atari | ||
| HER | Fetch | ||
| DFP | Doom | Doom Battle was not verified |
Click on each algorithm to see detailed benchmarking results