gryf/coach
1
0
Fork 0
mirror of https://github.com/gryf/coach.git synced 2025-12-17 11:10:20 +01:00
Code Issues Projects Releases Wiki Activity

Bug fix: when enabling 'heatup_using_network_decisions', we should add the configured noise (#162)

Browse Source 
During heatup we may want to add agent-generated-noise (i.e. not "simple" random noise).
This is enabled by setting 'heatup_using_network_decisions' to True.  For example:
	agent_params = DDPGAgentParameters()
	agent_params.algorithm.heatup_using_network_decisions = True

The fix ensures that the correct noise is added not just while in the TRAINING phase, but
also during the HEATUP phase.

No one has enabled 'heatup_using_network_decisions' yet, which explains why this problem
arose only now (in my configuration I do enable 'heatup_using_network_decisions').
This commit is contained in:
Neta Zmora
2018-12-17 10:08:54 +02:00
committed by Gal Leibovich
parent f9ee526536
commit b4bc8a476c
2 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
rl_coach/exploration_policies/additive_noise.py
View File

@@ -88,7 +88,7 @@ class AdditiveNoise(ExplorationPolicy):
action_values_mean = action_values.squeeze()
# step the noise schedule
if self.phase == RunPhase.TRAIN:
if self.phase is not RunPhase.TEST:
self.noise_percentage_schedule.step()
# the second element of the list is assumed to be the standard deviation
if isinstance(action_values, list) and len(action_values) > 1:

2
rl_coach/exploration_policies/truncated_normal.py
View File

@@ -92,7 +92,7 @@ class TruncatedNormal(ExplorationPolicy):
action_values_mean = action_values.squeeze()
# step the noise schedule
if self.phase == RunPhase.TRAIN:
if self.phase is not RunPhase.TEST:
self.noise_percentage_schedule.step()
# the second element of the list is assumed to be the standard deviation
if isinstance(action_values, list) and len(action_values) > 1: