ACER algorithm (#184)

* initial ACER commit

* Code cleanup + several fixes

* Q-retrace bug fix + small clean-ups

* added documentation for acer

* ACER benchmarks

* update benchmarks table

* Add nightly running of golden and trace tests. (#202)

Resolves #200

* comment out nightly trace tests until values reset.

* remove redundant observe ignore (#168)

* ensure nightly test env containers exist. (#205)

Also bump integration test timeout

* wxPython removal (#207)

Replacing wxPython with Python's Tkinter.
Also removing the option to choose multiple files as it is unused and causes errors, and fixing the load file/directory spinner.

* Create CONTRIBUTING.md (#210)

* Create CONTRIBUTING.md.  Resolves #188

* run nightly golden tests sequentially. (#217)

Should reduce resource requirements and potential CPU contention but increases
overall execution time.

* tests: added new setup configuration + test args (#211)

- added utils for future tests and conftest
- added test args

* new docs build

* golden test update

benchmarks/README.md

@@ -34,6 +34,7 @@ The environments that were used for testing include:
 |**[Bootstrapped DQN](bootstrapped_dqn)**| ![#2E8B57](https://placehold.it/15/2E8B57/000000?text=+) |Atari           | |
 |**[QR-DQN](qr_dqn)**            | ![#2E8B57](https://placehold.it/15/2E8B57/000000?text=+) |Atari           | |
 |**[A3C](a3c)**                  | ![#2E8B57](https://placehold.it/15/2E8B57/000000?text=+) |Atari, Mujoco   | |
+|**[ACER](acer)**                | ![#2E8B57](https://placehold.it/15/2E8B57/000000?text=+) |Atari           | |
 |**[Clipped PPO](clipped_ppo)**  | ![#2E8B57](https://placehold.it/15/2E8B57/000000?text=+) |Mujoco          | |
 |**[DDPG](ddpg)**                | ![#2E8B57](https://placehold.it/15/2E8B57/000000?text=+) |Mujoco          | |
 |**[NEC](nec)**                  | ![#2E8B57](https://placehold.it/15/2E8B57/000000?text=+) |Atari           | |

benchmarks/acer/README.md

@@ -0,0 +1,28 @@
+# ACER
+
+Each experiment uses 3 seeds.
+The parameters used for ACER are the same parameters as described in the [original paper](https://arxiv.org/abs/1611.01224), except for the optimizer (changed to ADAM) and learning rate (1e-4) used.
+
+### Breakout ACER - 16 workers
+
+```bash
+coach -p Atari_ACER -lvl breakout -n 16
+```
+
+<img src="breakout_acer_16_workers.png" alt="Breakout ACER" width="800"/>
+
+### Space Invaders ACER - 16 workers
+
+```bash
+coach -p Atari_ACER -lvl space_invaders -n 16
+```
+
+<img src="space_invaders_acer_16_workers.png" alt="Space Invaders ACER" width="800"/>
+
+### Pong ACER - 16 workers
+
+```bash
+coach -p Atari_ACER -lvl pong -n 16
+```
+
+<img src="pong_acer_16_workers.png" alt="Pong ACER" width="800"/>