6d40ad1650
* updating the documentation website * adding the built docs * update of api docstrings across coach and tutorials 0-2 * added some missing api documentation * New Sphinx based documentation
1.2 KiB
Actions space: Discrete
References: Asynchronous Methods for Deep Reinforcement Learning
Network Structure
Algorithm Description
Training the network
The N-step Q learning algorithm works in similar manner to DQN except for the following changes:
No replay buffer is used. Instead of sampling random batches of transitions, the network is trained every N steps using the latest N steps played by the agent.
In order to stabilize the learning, multiple workers work together to update the network. This creates the same effect as uncorrelating the samples used for training.
Instead of using single-step Q targets for the network, the rewards from $N$ consequent steps are accumulated to form the N-step Q targets, according to the following equation: R(st, at) = ∑i = t + k − 1i = tγi − tri + γkV(st + k) where k is Tmax − State_Index for each state in the batch