ACER algorithm (#184)

* initial ACER commit

* Code cleanup + several fixes

* Q-retrace bug fix + small clean-ups

* added documentation for acer

* ACER benchmarks

* update benchmarks table

* Add nightly running of golden and trace tests. (#202)

Resolves #200

* comment out nightly trace tests until values reset.

* remove redundant observe ignore (#168)

* ensure nightly test env containers exist. (#205)

Also bump integration test timeout

* wxPython removal (#207)

Replacing wxPython with Python's Tkinter.
Also removing the option to choose multiple files as it is unused and causes errors, and fixing the load file/directory spinner.

* Create CONTRIBUTING.md (#210)

* Create CONTRIBUTING.md.  Resolves #188

* run nightly golden tests sequentially. (#217)

Should reduce resource requirements and potential CPU contention but increases
overall execution time.

* tests: added new setup configuration + test args (#211)

- added utils for future tests and conftest
- added test args

* new docs build

* golden test update

docs/_sources/components/agents/value_optimization/double_dqn.rst.txt

@@ -19,7 +19,7 @@ Training the network
 
 1. Sample a batch of transitions from the replay buffer.
 
-2. Using the next states from the sampled batch, run the online network in order to find the $Q$ maximizing
+2. Using the next states from the sampled batch, run the online network in order to find the :math:`Q` maximizing
    action :math:`argmax_a Q(s_{t+1},a)`. For these actions, use the corresponding next states and run the target
    network to calculate :math:`Q(s_{t+1},argmax_a Q(s_{t+1},a))`.
 

docs/_sources/components/agents/value_optimization/dqn.rst.txt

@@ -26,7 +26,7 @@ Training the network
    use the current states from the sampled batch, and run the online network to get the current Q values predictions.
    Set those values as the targets for the actions that were not actually played.
 
-4. For each action that was played, use the following equation for calculating the targets of the network:​                                                         $$ y_t=r(s_t,a_t)+γ\cdot max_a {Q(s_{t+1},a)} $$ 
+4. For each action that was played, use the following equation for calculating the targets of the network:​
    :math:`y_t=r(s_t,a_t )+\gamma \cdot max_a Q(s_{t+1})`
 
 5. Finally, train the online network using the current states as inputs, and with the aforementioned targets.