6d40ad1650
* updating the documentation website * adding the built docs * update of api docstrings across coach and tutorials 0-2 * added some missing api documentation * New Sphinx based documentation
1.2 KiB
Actions space: Discrete
References: Count-Based Exploration with Neural Density Models
Network Structure
Algorithm Description
Training the network
In MMC, targets are calculated as a mixture between Double DQN targets and full Monte Carlo samples (total discounted returns).
The DDQN targets are calculated in the same manner as in the DDQN agent:
yDDQNt = r(st, at) + γQ(st + 1, argmaxaQ(st + 1, a))
The Monte Carlo targets are calculated by summing up the discounted rewards across the entire episode:
yMCt = ∑Tj = 0γjr(st + j, at + j)
A mixing ratio $alpha$ is then used to get the final targets:
yt = (1 − α)⋅yDDQNt + α⋅yMCt
Finally, the online network is trained using the current states as inputs, and the calculated targets. Once in every few thousand steps, copy the weights from the online network to the target network.