1
0
mirror of https://github.com/gryf/coach.git synced 2026-01-04 12:54:17 +01:00
Files
coach/rl_coach/traces/Mujoco_A3C_hopper/trace.csv
Itai Caspi 72a1d9d426 Itaicaspi/episode reset refactoring (#105)
* reordering of the episode reset operation and allowing to store episodes only when they are terminated

* reordering of the episode reset operation and allowing to store episodes only when they are terminated

* revert tensorflow-gpu to 1.9.0 + bug fix in should_train()

* tests readme file and refactoring of policy optimization agent train function

* Update README.md

* Update README.md

* additional policy optimization train function simplifications

* Updated the traces after the reordering of the environment reset

* docker and jenkins files

* updated the traces to the ones from within the docker container

* updated traces and added control suite to the docker

* updated jenkins file with the intel proxy + updated doom basic a3c test params

* updated line breaks in jenkins file

* added a missing line break in jenkins file

* refining trace tests ignored presets + adding a configurable beta entropy value

* switch the order of trace and golden tests in jenkins + fix golden tests processes not killed issue

* updated benchmarks for dueling ddqn breakout and pong

* allowing dynamic updates to the loss weights + bug fix in episode.update_returns

* remove docker and jenkins file
2018-09-04 15:07:54 +03:00

31 KiB

1Episode #Training IterIn HeatupER #TransitionsER #EpisodesEpisode LengthTotal stepsEpsilonShaped Training RewardTraining RewardUpdate Target NetworkEvaluation RewardShaped Evaluation RewardSuccess RateLoss/MeanLoss/StdevLoss/MaxLoss/MinLearning Rate/MeanLearning Rate/StdevLearning Rate/MaxLearning Rate/MinGrads (unclipped)/MeanGrads (unclipped)/StdevGrads (unclipped)/MaxGrads (unclipped)/MinEntropy/MeanEntropy/StdevEntropy/MaxEntropy/MinAdvantages/MeanAdvantages/StdevAdvantages/MaxAdvantages/MinValues/MeanValues/StdevValues/MaxValues/MinValue Loss/MeanValue Loss/StdevValue Loss/MaxValue Loss/MinPolicy Loss/MeanPolicy Loss/StdevPolicy Loss/MaxPolicy Loss/Min
210.01.014.01.014.014.00.100000000000000020.0
320.01.020.01.020.034.00.100000000000000020.0
430.01.018.01.018.052.00.100000000000000020.0
540.01.024.01.024.076.00.100000000000000020.0
650.01.018.01.018.094.00.100000000000000020.0
760.01.033.01.033.0127.00.100000000000000020.0
870.01.016.01.016.0143.00.100000000000000020.0
980.01.020.01.020.0163.00.100000000000000020.0
1090.01.011.01.011.0174.00.100000000000000020.0
11100.01.020.01.020.0194.00.100000000000000020.0
12110.01.018.01.018.0212.00.100000000000000020.0
13120.01.013.01.013.0225.00.100000000000000020.0
14130.01.027.01.027.0252.00.100000000000000020.0
15140.01.021.01.021.0273.00.100000000000000020.0
16150.01.022.01.022.0295.00.100000000000000020.0
17160.01.018.01.018.0313.00.100000000000000020.0
18170.01.010.01.010.0323.00.100000000000000020.0
19180.01.015.01.015.0338.00.100000000000000020.0
20190.01.021.01.021.0359.00.100000000000000020.0
21200.01.040.01.040.0399.00.100000000000000020.0
22210.01.026.01.026.0425.00.100000000000000020.0
23220.01.025.01.025.0450.00.100000000000000020.0
24230.01.017.01.017.0467.00.100000000000000020.0
25240.01.036.01.036.0503.00.100000000000000020.0
26250.01.022.01.022.0525.00.100000000000000020.0
27260.01.018.01.018.0543.00.100000000000000020.0
28270.01.023.01.023.0566.00.100000000000000020.0
29280.01.025.01.025.0591.00.100000000000000020.0
30290.01.023.01.023.0614.00.100000000000000020.0
31300.01.016.01.016.0630.00.100000000000000020.0
32310.01.027.01.027.0657.00.100000000000000020.0
33320.01.029.01.029.0686.00.100000000000000020.0
34330.01.016.01.016.0702.00.100000000000000020.0
35340.01.011.01.011.0713.00.100000000000000020.0
36350.01.019.01.019.0732.00.100000000000000020.0
37360.01.018.01.018.0750.00.100000000000000020.0
38370.01.014.01.014.0764.00.100000000000000020.0
39380.01.024.01.024.0788.00.100000000000000020.0
40390.01.013.01.013.0801.00.100000000000000020.0
41400.01.028.01.028.0829.00.100000000000000020.0
42410.01.020.01.020.0849.00.100000000000000020.0
43420.01.037.01.037.0886.00.100000000000000020.0
44430.01.015.01.015.0901.00.100000000000000020.0
45440.01.012.01.012.0913.00.100000000000000020.0
46450.01.027.01.027.0940.00.100000000000000020.0
47460.01.011.01.011.0951.00.100000000000000020.0
48470.01.020.01.020.0971.00.100000000000000020.0
49480.01.013.01.013.0984.00.100000000000000020.0
50490.01.026.01.026.01010.00.100000000000000020.0
51500.01.015.01.015.01025.00.100000000000000020.0
52510.00.014.01.014.01039.00.100000000000000020.51393469190325910.278693838065180.0
53521.00.016.01.016.01055.00.100000000000000020.51760471723138510.3520943446276980.0
54532.00.032.01.032.01087.00.100000000000000021.247977085847960924.959541716959230.0
55543.00.030.01.030.01117.00.100000000000000020.527044608820304610.5408921764060860.0
56554.00.025.01.025.01142.00.100000000000000020.715167170608668614.3033434121733760.0
57565.00.015.01.015.01157.00.100000000000000020.637780574416920912.7556114883384130.0
58576.00.017.01.017.01174.00.100000000000000020.642938951210842312.8587790242168470.0
59587.00.031.01.031.01205.00.100000000000000021.575336284151849731.5067256830369860.0
60598.00.019.01.019.01224.00.100000000000000020.909581284410225918.1916256882045140.0
61609.00.013.01.013.01237.00.100000000000000020.37063929388836587.4127858777673170.0
626110.00.031.01.031.01268.00.100000000000000020.721215033118976814.4243006623795350.0
636211.00.034.01.034.01302.00.100000000000000021.817450279470100836.3490055894020260.0
646312.00.016.01.016.01318.00.100000000000000020.48751179441668319.750235888333660.0
656413.00.017.01.017.01335.00.100000000000000020.43192140148692518.63842802973850.0
666514.00.015.01.015.01350.00.100000000000000020.41994322843444898.3988645686889840.0
676615.00.017.01.017.01367.00.100000000000000020.639652725980973712.7930545196194740.0
686716.00.014.01.014.01381.00.100000000000000020.54651352285105610.9302704570211220.0
696817.00.021.01.021.01402.00.100000000000000020.614917021087196812.2983404217439390.0
706918.00.015.01.015.01417.00.100000000000000020.569954082376651811.3990816475330360.0
717019.00.09.01.09.01426.00.100000000000000020.38653724719543477.7307449439086940.0
727120.00.011.01.011.01437.00.100000000000000020.38238126915103787.64762538302075350.0
737221.00.018.01.018.01455.00.100000000000000020.688126325628118713.7625265125623670.0
747322.00.010.01.010.01465.00.100000000000000020.32644073093684986.5288146187369960.0
757423.00.014.01.014.01479.00.100000000000000020.592424358557102111.8484871711420360.0
767524.00.016.01.016.01495.00.100000000000000020.29310047042999555.8620094085999070.0
777625.00.09.01.09.01504.00.100000000000000020.36840230093211617.3680460186423220.0
787726.00.037.01.037.01541.00.100000000000000023.025147649104463560.502952982089260.0
797827.00.045.01.045.01586.00.100000000000000022.456862740629140349.137254812582780.0
807928.00.010.01.010.01596.00.100000000000000020.36134791676872357.226958335374470.0
818029.00.011.01.011.01607.00.100000000000000020.41298394367007998.2596788734015970.0
828130.00.020.01.020.01627.00.100000000000000020.811373673051290616.227473461025810.0
838231.00.016.01.016.01643.00.100000000000000020.728881312924915314.5776262584983060.0
848332.00.016.01.016.01659.00.100000000000000020.47296041701909049.4592083403818050.0
858433.00.013.01.013.01672.00.100000000000000020.41606133120599858.3212266241199730.0
868534.00.017.01.017.01689.00.100000000000000020.301309317955380636.0261863591076120.0
878635.00.027.01.027.01716.00.100000000000000021.533913168333604230.678263366672090.0
888736.00.014.01.014.01730.00.100000000000000020.608185048006066312.1637009601213290.0
898837.00.021.01.021.01751.00.100000000000000020.782481051979363115.6496210395872670.0
908938.00.019.01.019.01770.00.100000000000000020.60857280521183212.1714561042366420.0
919039.00.033.01.033.01803.00.100000000000000021.574380066867879131.487601337357590.0
929140.00.010.01.010.01813.00.100000000000000020.245912594833106964.9182518966621390.0
939241.00.012.01.012.01825.00.100000000000000020.49579875491571689.9159750983143380.0
949342.00.013.01.013.01838.00.100000000000000020.533018442231379810.6603688446275960.0
959443.00.016.01.016.01854.00.100000000000000020.711163034267333114.2232606853466610.0
969544.00.026.01.026.01880.00.100000000000000020.760916556437293415.2183311287458660.0
979645.00.053.01.053.01933.00.100000000000000024.185206179719501583.704123594390030.0
989746.00.017.01.017.01950.00.100000000000000020.826975318006485416.539506360129710.0
999847.00.041.01.041.01991.00.100000000000000023.254443076830743765.088861536614870.0
1009948.00.027.01.027.02018.00.100000000000000021.136020766134951322.7204153226990330.0
10110049.00.013.01.013.02031.00.100000000000000020.522647203861417410.4529440772283470.0
10210150.00.017.01.017.02048.00.100000000000000020.4412074458808918.8241489176178210.0
10310251.00.015.01.015.02063.00.100000000000000020.715499431357453914.3099886271490750.0
10410352.00.014.01.014.02077.00.100000000000000020.4592364481052099.1847289621041760.0
10510453.00.017.01.017.02094.00.100000000000000020.37657658411479487.5315316822958950.0
10610554.00.019.01.019.02113.00.100000000000000020.678632128238645813.5726425647729180.0
10710655.00.021.01.021.02134.00.100000000000000020.64451092987183812.8902185974367590.0
10810756.00.019.01.019.02153.00.100000000000000020.53923193152766110.7846386305532180.0
10910857.00.012.01.012.02165.00.100000000000000020.49515923979775219.9031847959550440.0
11010958.00.019.01.019.02184.00.100000000000000020.730683219128562814.6136643825712560.0
11111059.00.017.01.017.02201.00.100000000000000020.559086551848172911.181731036963460.0
11211160.00.016.01.016.02217.00.100000000000000020.46869038291647619.3738076583295180.0
11311261.00.010.01.010.02227.00.100000000000000020.42209393250984578.4418786501969140.0
11411362.00.017.01.017.02244.00.100000000000000020.43647949372817488.7295898745634940.0
11511463.00.016.01.016.02260.00.100000000000000020.32689143095662976.5378286191325940.0
11611564.00.026.01.026.02286.00.100000000000000020.799493970145070815.989879402901410.0
11711665.00.017.01.017.02303.00.100000000000000020.616208660015305912.3241732003061150.0
11811766.00.026.01.026.02329.00.100000000000000021.287644744092751925.752894881855040.0
11911867.00.023.01.023.02352.00.100000000000000021.18771593876279923.754318775255970.0
12011968.00.018.01.018.02370.00.100000000000000020.759461920825706215.1892384165141240.0
12112069.00.020.01.020.02390.00.100000000000000020.701349568318274314.0269913663654840.0
12212170.00.017.01.017.02407.00.100000000000000020.709678420134822714.1935684026964550.0
12312271.00.012.01.012.02419.00.100000000000000020.515286844883238610.3057368976647670.0
12412372.00.033.01.033.02452.00.100000000000000021.743547973806299134.870959476125980.0
12512473.00.035.01.035.02487.00.100000000000000021.198382836775352223.9676567355070450.0
12612574.00.013.01.013.02500.00.100000000000000020.49683421237798789.9366842475597520.0
12712675.00.013.01.013.02513.00.100000000000000020.40941160801670198.1882321603340390.0
12812776.00.039.01.039.02552.00.100000000000000021.368055225177083427.361104503541650.0
12912877.00.015.01.015.02567.00.100000000000000020.48537109901747129.7074219803494230.0
13012978.00.055.01.055.02622.00.100000000000000022.666661287778915353.333225755578310.0
13113079.00.028.01.028.02650.00.100000000000000020.600597630001309712.0119526000261950.0
13213180.00.015.01.015.02665.00.100000000000000020.535812946910116910.7162589382023390.0
13313281.00.042.01.042.02707.00.100000000000000021.22906578316462424.581315663292470.0
13413382.00.013.01.013.02720.00.100000000000000020.558763352525862511.1752670505172490.0
13513483.00.023.01.023.02743.00.100000000000000020.682477128361318513.6495425672263750.0
13613584.00.026.01.026.02769.00.100000000000000021.476838741909247729.5367748381849470.0
13713685.00.012.01.012.02781.00.100000000000000020.42596098904743418.5192197809486830.0
13813786.00.015.01.015.02796.00.100000000000000020.644512040019165112.8902408003833010.0
13913887.00.013.01.013.02809.00.100000000000000020.264389451412060055.28778902824120140.0
14013988.00.034.01.034.02843.00.100000000000000021.334550150081165826.6910030016233080.0
14114089.00.018.01.018.02861.00.100000000000000020.888944570243651817.778891404873030.0
14214190.00.013.01.013.02874.00.100000000000000020.599096771360438311.9819354272087640.0
14314291.00.022.01.022.02896.00.100000000000000020.591198220416238411.8239644083247680.0
14414392.00.014.01.014.02910.00.100000000000000020.49640713883463819.928142776692760.0
14514493.00.027.01.027.02937.00.100000000000000021.475276428944722929.5055285788944450.0
14614594.00.018.01.018.02955.00.100000000000000020.659281605045511613.1856321009102240.0
14714695.00.022.01.022.02977.00.100000000000000021.081907082585821721.6381416517164280.0
14814796.00.012.01.012.02989.00.100000000000000020.47953271460699929.5906542921399820.0
14914897.00.027.01.027.03016.00.100000000000000020.45675540380478299.135108076095660.0
15014998.00.015.01.015.03031.00.100000000000000020.703177680324539214.0635536064907850.0
15115099.00.012.01.012.03043.00.100000000000000020.45226766585490159.045353317098030.0
152151100.00.010.01.010.03053.00.100000000000000020.38999103741637757.799820748327550.0
153152101.00.024.01.024.03077.00.100000000000000020.881169659574024517.623393191480490.0
154153102.00.012.01.012.03089.00.100000000000000020.39595164316213337.91903286324266450.0
155154103.00.019.01.019.03108.00.100000000000000020.760686117260275515.2137223452055060.0
156155104.00.028.01.028.03136.00.100000000000000021.266514265396183425.3302853079236630.0
157156105.00.019.01.019.03155.00.100000000000000020.732215045006221414.6443009001244260.0
158157106.00.011.01.011.03166.00.100000000000000020.299604134027572635.9920826805514520.0
159158107.00.031.01.031.03197.00.100000000000000021.709198520589518434.1839704117903550.0
160159108.00.033.01.033.03230.00.100000000000000020.928890109366162518.577802187323250.0
161160109.00.018.01.018.03248.00.100000000000000020.385249806370338237.7049961274067640.0
162161110.00.048.01.048.03296.00.100000000000000023.323974392744708566.479487854894170.0
163162111.00.020.01.020.03316.00.100000000000000020.876672491358767517.533449827175350.0
164163112.00.021.01.021.03337.00.100000000000000020.552237507260096111.0447501452019240.0
165164113.00.032.01.032.03369.00.100000000000000021.956061833181884439.121236663637680.0
166165114.00.033.01.033.03402.00.100000000000000020.776826448856679715.5365289771335960.0
167166115.00.016.01.016.03418.00.100000000000000020.26141503537632355.22830070752646850.0
168167116.00.015.01.015.03433.00.100000000000000020.446639452480479768.9327890496095940.0
169168117.00.028.01.028.03461.00.100000000000000020.66409323283853113.2818646567706150.0
170169118.00.015.01.015.03476.00.100000000000000020.47477120970510329.495424194102060.0
171170119.00.020.01.020.03496.00.100000000000000020.748541356200675114.9708271240134930.0
172171120.00.031.01.031.03527.00.100000000000000021.266774286549609225.3354857309921750.0
173172121.00.033.01.033.03560.00.100000000000000020.551608012326396411.0321602465279260.0
174173122.00.020.01.020.03580.00.100000000000000020.37737280823749927.5474561647499860.0
175174123.00.034.01.034.03614.00.100000000000000021.732707621209164334.654152424183290.0
176175124.00.014.01.014.03628.00.100000000000000020.643552211604917512.8710442320983490.0
177176125.00.029.01.029.03657.00.100000000000000020.42973521033047738.5947042066095440.0
178177126.00.010.01.010.03667.00.100000000000000020.36203954342871697.2407908685743390.0
179178127.00.016.01.016.03683.00.100000000000000020.705063586124043314.1012717224808650.0
180179128.00.020.01.020.03703.00.100000000000000020.63550527626847412.7101055253694780.0
181180129.00.031.01.031.03734.00.100000000000000022.564702018697688351.294040373953760.0
182181130.00.011.01.011.03745.00.100000000000000020.41486476731551188.2972953463102370.0
183182131.00.054.01.054.03799.00.100000000000000022.734358255234264254.687165104685280.0
184183132.00.034.01.034.03833.00.100000000000000020.924286484263420418.485729685268410.0
185184133.00.020.01.020.03853.00.100000000000000020.153638484749813653.0727696949962740.0
186185134.00.092.01.092.03945.00.100000000000000025.701919928563215114.038398571264310.0
187186135.00.029.01.029.03974.00.100000000000000020.704698030812624514.0939606162524920.0
188187136.00.032.01.032.04006.00.100000000000000022.013169176497795540.2633835299559240.0
189188137.00.024.01.024.04030.00.100000000000000021.079197091981210821.5839418396242270.0
190189138.00.020.01.020.04050.00.100000000000000020.822741607796984516.4548321559396880.0
191190139.00.026.01.026.04076.00.100000000000000021.405821379596336628.1164275919267330.0
192191140.00.028.01.028.04104.00.100000000000000021.646239508200043232.924790164000860.0
193192141.00.023.01.023.04127.00.100000000000000020.726789022030692514.5357804406138480.0
194193142.00.037.01.037.04164.00.100000000000000020.44805933115581498.9611866231163050.0
195194143.00.013.01.013.04177.00.100000000000000020.54472772085039210.8945544170078360.0
196195144.00.016.01.016.04193.00.100000000000000020.633691554752870412.6738310950574090.0
197196145.00.015.01.015.04208.00.100000000000000020.33332648566510296.6665297133020590.0
198197146.00.028.01.028.04236.00.100000000000000021.130811837713271922.6162367542654440.0
199198147.00.075.01.075.04311.00.100000000000000026.394565139970903127.891302799417990.0
200199148.00.025.01.025.04336.00.100000000000000021.22879120088879924.5758240177759820.0
201200149.00.027.01.027.04363.00.100000000000000021.374038250373959227.4807650074791780.0
202201150.00.079.01.079.04442.00.100000000000000023.079545778348782361.590915566975640.0
203202151.00.023.01.023.04465.00.100000000000000021.09905326485084121.9810652970168260.0
204203152.00.017.01.017.04482.00.100000000000000020.544084014660442610.8816802932088470.0
205204153.00.032.01.032.04514.00.100000000000000021.705624773574553634.112495471491080.0
206205154.00.012.01.012.04526.00.100000000000000020.45581237227792089.1162474455584160.0
207206155.00.012.01.012.04538.00.100000000000000020.350049821660907767.0009964332181540.0
208207156.00.023.01.023.04561.00.100000000000000020.49323728867419619.864745773483920.0
209208157.00.023.01.023.04584.00.100000000000000020.725679463389378514.5135892677875730.0
210209158.00.052.01.052.04636.00.100000000000000024.70087155618122194.01743112362440.0
211210159.00.031.01.031.04667.00.100000000000000021.32673068251839926.5346136503679940.0
212211160.00.025.01.025.04692.00.100000000000000020.506411269934339510.1282253986867870.0
213212161.00.012.01.012.04704.00.100000000000000020.534925016410320810.6985003282064140.0
214213162.00.018.01.018.04722.00.100000000000000020.761506751246621615.230135024932430.0
215214163.00.038.01.038.04760.00.100000000000000021.94543158385809838.908631677161950.0
216215164.00.012.01.012.04772.00.100000000000000020.42067272723514128.4134545447028210.0
217216165.00.032.01.032.04804.00.100000000000000021.043671684424920820.8734336884984120.0
218217166.00.012.01.012.04816.00.100000000000000020.351845186848765767.0369037369753160.0
219218167.00.019.01.019.04835.00.100000000000000020.770967834181301815.4193566836260360.0
220219168.00.013.01.013.04848.00.100000000000000020.518904138550485610.378082771009710.0
221220169.00.015.01.015.04863.00.100000000000000020.665973760810803613.3194752162160710.0
222221170.00.026.01.026.04889.00.100000000000000021.453470520982511629.069410419650230.0
223222171.00.015.01.015.04904.00.100000000000000020.619504329027842912.3900865805568580.0
224223172.00.061.01.061.04965.00.100000000000000023.9156713062001278.313426124002460.0
225224173.00.019.01.019.04984.00.100000000000000020.656482608249913813.1296521649982760.0
226225174.00.069.01.069.05053.00.100000000000000024.52308545913327490.461709182665440.0
227226175.00.017.01.017.05070.00.100000000000000020.806609105422967616.1321821084593570.0
228227176.00.028.01.028.05098.00.100000000000000020.888214605468871917.764292109377440.0
229228177.00.033.01.033.05131.00.100000000000000021.625656681156948232.5131336231389550.0
230229178.00.026.01.026.05157.00.100000000000000020.895678621369559817.9135724273912020.0
231230179.00.075.01.075.05232.00.100000000000000024.511092995988903590.221859919778040.0
232231180.00.052.01.052.05284.00.100000000000000023.434718905090101768.694378101802030.0
233232181.00.023.01.023.05307.00.100000000000000021.034951963680454720.6990392736091060.0
234233182.00.017.01.017.05324.00.100000000000000020.533467059473986310.6693411894797240.0
235234183.00.09.01.09.05333.00.100000000000000020.35585579357820467.1171158715640930.0
236235184.00.069.01.069.05402.00.100000000000000024.74883710049449294.976742009889820.0
237236185.00.030.01.030.05432.00.100000000000000021.995252933414046739.905058668280940.0
238237186.00.015.01.015.05447.00.100000000000000020.749992086680424414.9998417336084910.0
239238187.00.015.01.015.05462.00.100000000000000020.47240368604732349.4480737209464680.0
240239188.00.024.01.024.05486.00.100000000000000021.041064551148992420.821291022979840.0
241240189.00.052.01.052.05538.00.100000000000000023.904802993240423378.096059864808450.0
242241190.00.019.01.019.05557.00.100000000000000020.519803741571171410.3960748314234320.0
243242191.00.051.01.051.05608.00.100000000000000024.579367747630391.587354952605950.0
244243192.00.016.01.016.05624.00.100000000000000020.516272363018554710.3254472603710940.0
245244193.00.022.01.022.05646.00.100000000000000020.828764879425846616.5752975885169360.0
246245194.00.023.01.023.05669.00.100000000000000020.727658627901942814.5531725580388520.0
247246195.00.021.01.021.05690.00.100000000000000021.05213402037572521.0426804075145030.0
248247196.00.019.01.019.05709.00.100000000000000020.715713642087403714.3142728417480730.0
249248197.00.028.01.028.05737.00.100000000000000020.977827936257585419.5565587251517030.0
250249198.00.021.01.021.05758.00.100000000000000020.36809597240784397.3619194481568780.0
251250199.00.024.01.024.05782.00.100000000000000021.26229395854816125.245879170963210.0
252251200.00.026.01.026.05808.00.100000000000000020.48499627352198269.6999254704396520.0
253252201.00.015.01.015.05823.00.100000000000000020.41004120478096768.200824095619350.0
254253202.00.022.01.022.05845.00.100000000000000021.277670865021991725.553417300439830.0
255254203.00.037.01.037.05882.00.100000000000000023.129075975725805362.58151951451610.0
256255204.00.025.01.025.05907.00.100000000000000021.256019996830677625.120399936613560.0