1
0
mirror of https://github.com/gryf/coach.git synced 2026-01-30 12:15:49 +01:00
Files
coach/rl_coach/traces/Fetch_DDPG_HER_baselines_reach/trace.csv
Itai Caspi 72a1d9d426 Itaicaspi/episode reset refactoring (#105)
* reordering of the episode reset operation and allowing to store episodes only when they are terminated

* reordering of the episode reset operation and allowing to store episodes only when they are terminated

* revert tensorflow-gpu to 1.9.0 + bug fix in should_train()

* tests readme file and refactoring of policy optimization agent train function

* Update README.md

* Update README.md

* additional policy optimization train function simplifications

* Updated the traces after the reordering of the environment reset

* docker and jenkins files

* updated the traces to the ones from within the docker container

* updated traces and added control suite to the docker

* updated jenkins file with the intel proxy + updated doom basic a3c test params

* updated line breaks in jenkins file

* added a missing line break in jenkins file

* refining trace tests ignored presets + adding a configurable beta entropy value

* switch the order of trace and golden tests in jenkins + fix golden tests processes not killed issue

* updated benchmarks for dueling ddqn breakout and pong

* allowing dynamic updates to the loss weights + bug fix in episode.update_returns

* remove docker and jenkins file
2018-09-04 15:07:54 +03:00

25 KiB

1Episode #Training IterIn HeatupER #TransitionsER #EpisodesEpisode LengthTotal stepsEpsilonShaped Training RewardTraining RewardUpdate Target NetworkEvaluation RewardShaped Evaluation RewardSuccess RateLoss/MeanLoss/StdevLoss/MaxLoss/MinLearning Rate/MeanLearning Rate/StdevLearning Rate/MaxLearning Rate/MinGrads (unclipped)/MeanGrads (unclipped)/StdevGrads (unclipped)/MaxGrads (unclipped)/MinEntropy/MeanEntropy/StdevEntropy/MaxEntropy/MinAdvantages/MeanAdvantages/StdevAdvantages/MaxAdvantages/MinValues/MeanValues/StdevValues/MaxValues/MinValue Loss/MeanValue Loss/StdevValue Loss/MaxValue Loss/MinPolicy Loss/MeanPolicy Loss/StdevPolicy Loss/MaxPolicy Loss/MinQ/MeanQ/StdevQ/MaxQ/MinTD targets/MeanTD targets/StdevTD targets/MaxTD targets/Minactions/Meanactions/Stdevactions/Maxactions/Min
210.01.0246.01.050.050.00.10.0
320.01.0492.02.050.0100.00.10.0
430.01.0738.03.050.0150.00.10.0
540.01.0984.04.050.0200.00.10.0
650.01.01230.05.050.0250.00.10.0
760.01.01476.06.050.0300.00.10.0
870.01.01722.07.050.0350.00.10.0
980.01.01968.08.050.0400.00.10.0
1090.01.02214.09.050.0450.00.10.0
11100.01.02460.010.050.0500.00.10.0
12110.01.02706.011.050.0550.00.10.0
13120.01.02952.012.050.0600.00.10.0
14130.01.03198.013.050.0650.00.10.0
15140.01.03444.014.050.0700.00.10.0
16150.01.03690.015.050.0750.00.10.0
17160.01.03936.016.050.0800.00.10.0
18170.01.04182.017.050.0850.00.10.0
19180.01.04428.018.050.0900.00.10.0
20190.01.04674.019.050.0950.00.10.0
21200.01.04920.020.050.01000.00.10.0
22210.01.05166.021.050.01050.00.11.0
232240.00.05412.022.050.01100.00.1-46.0-46.01.05.2728862565010792.20316384007571748.53934857249260.75417304039001460.0010.00.0010.0010.53170020.53742752.67826720.12463605-0.406944250.1169177140.022069091000000002-0.730958-0.69218839658167610.41428559144504380.0-1.04430308938026450.0219844926370731180.32013719128536290.9814501104952102-0.9372007403745176
242380.00.05658.023.050.01150.00.1-50.0-50.01.0-0.41702070.1371437-0.24129575-0.8606351000000001-0.0397861697443757860.38097994485430310.9568684479780788-0.94501333674486
2524120.00.05904.024.050.01200.00.1-46.0-46.01.0-0.560809730.11710674-0.27295074-0.86496603-0.155033116319803540.49299461917122150.9849893537921456-1.311290803570807
2625160.00.06150.025.050.01250.00.1-34.0-34.00.0-0.464642080.11209127-0.2652866-0.76236176-0.050512979587550740.56291375546952961.125240381996044-1.348682534013401
2726200.00.06396.026.050.01300.00.1-35.0-35.01.0-0.291022720.15320052-0.042197682-0.7351753000000001-0.031210929192613820.60620877156229361.2385231711559292-1.2429470911474896
2827240.00.06642.027.050.01350.00.1-6.0-6.01.0-0.269249980.233406020.029881956-1.02089760.194795401407471670.60753996361472151.362710617317857-1.1412981946019911
2928280.00.06888.028.050.01400.00.1-2.0-2.01.0-0.042807760.230064570.13516434-1.0150582000000001-0.011789727481644880.50141232872941921.2302989717776276-1.3886488908267705
3029320.00.07134.029.050.01450.00.1-12.0-12.01.0-0.152601180.35186770.17525306-1.19501050.012691828903625350.57304319359063141.249124258359712-1.2584634832377488
3130360.00.07380.030.050.01500.00.1-5.0-5.00.0-0.148052200000000020.277737950.10773307-1.26369180.0363483831066385150.59599988818427811.3286263387659598-1.1548506797671136
3231400.00.07626.031.050.01550.00.1-5.0-5.01.0-0.184660840.321963670.08791536-1.20640289999999980.048590189878530850.54281422579311641.2201469625997623-0.9913504469598621
3332440.00.07872.032.050.01600.00.1-9.0-9.01.0-0.211569880.344476499999999960.081073806-1.35571420.0566257718564368850.55190891197962981.2674420901630226-0.9983152529359643
3433480.00.08118.033.050.01650.00.1-9.0-9.01.0-0.371784060.4327107-0.08817409-1.5141906-0.17549221430483640.54476762181132441.0643172748924041-1.370976975153857
3534520.00.08364.034.050.01700.00.1-8.0-8.01.0-0.384971380.33896407-0.057881176-1.36638320.0561177522531417850.48829306790211541.2743613517964103-1.222909940488798
3635560.00.08610.035.050.01750.00.1-5.0-5.00.0-0.169596240.286673580.081009254-0.9562375-0.045270410319983320.53436119355532521.0625102558852273-1.1470136528691592
3736600.00.08856.036.050.01800.00.1-6.0-6.01.0-0.341756430.31717125-0.016218761-1.2953563999999997-0.0082776607080041610.56101071980440781.2756654929003477-1.2355330004029759
3837640.00.09102.037.050.01850.00.1-15.0-15.01.0-0.452571599999999960.5278461-0.01445999-1.486245-0.044034927006979190.53204769121018711.1980312358754541-1.2478175801963436
3938680.00.09348.038.050.01900.00.1-8.0-8.01.0-0.320793450.43125895-0.03217832-1.467122-0.0376459140581118860.50118401889592271.3869878210535596-1.2564681666707989
4039720.00.09594.039.050.01950.00.1-5.0-5.01.0-0.141167430.233803780.005386416-1.3339581-0.0109186509367555120.56364285628062391.269962142689417-1.2355026269097698
4140760.00.09840.040.050.02000.00.1-4.0-4.00.0-0.189825530.3344385-0.0033990983000000002-1.6050167-0.0215379252631886960.66787468811781821.3238071607008168-1.4277528884478166
4241800.00.010086.041.050.02050.00.1-6.0-6.01.0-0.448539560.41304794-0.17297234-1.7501773999999997-0.064027367164978850.48564147272923751.0255577416958612-1.1947008900638911
4342840.00.010332.042.050.02100.00.1-4.0-4.01.0-0.307965400000000060.37146740000000006-0.03877014-1.6268883-0.0136484656953681090.54456705411951991.193934647379189-1.3021191543972264
4443880.00.010578.043.050.02150.00.1-13.0-13.01.0-0.409472080.41609144-0.13703968-1.63460830.098881546496951840.69923816826258961.3025497731494657-1.5110693989813728
4544920.00.010824.044.050.02200.00.1-12.0-12.01.0-0.528726340.50485295-0.12703945-1.8293633000000002-0.11945556811533240.52640941608794281.165649092206002-1.35041474992645
4645960.00.011070.045.050.02250.00.1-4.0-4.00.0-0.297691880.32482535-0.08730304-1.65489750.00383123464715600560.48451394264135790.9980934098162764-1.02014820145686
47461000.00.011316.046.050.02300.00.1-4.0-4.01.0-0.212086000000000020.35386217-0.041998297000000004-1.69902629999999990.12400463817685960.56890297011678811.21785818606853-1.372869538366818
48471040.00.011562.047.050.02350.00.1-6.0-6.01.0-0.212026270.3844959-0.0005411245-1.4654028000000001-0.066701367016403970.48566173789386580.9663694286728484-1.1852976622145763
49481080.00.011808.048.050.02400.00.1-7.0-7.01.0-0.438471670.37343282-0.19374435-1.7563688000000002-0.012877383744792130.60856479711057721.2840393405934485-1.1723051332701249
50491120.00.012054.049.050.02450.00.1-6.0-6.01.0-0.26480140.46074203-0.015886346000000003-1.7840718-0.092877893086010730.57235104692179431.0611478482265122-1.1971651668093628
51501160.00.012300.050.050.02500.00.1-30.0-30.00.0-0.6590380.45721018-0.13472436-1.6326314-0.10057669870762870.64369636272478651.2637751754859214-1.3337451353362333
52511200.00.012546.051.050.02550.00.1-7.0-7.01.0-0.370501130.49309763-0.15040672-2.16283820.084572423551142290.66256709573771091.53445134159603-1.104432056329302
53521240.00.012792.052.050.02600.00.1-4.0-4.01.0-0.182269480.32638326-0.01282106-1.3475298999999998-0.098448930014743770.58658179152271111.1149270499531785-1.4250073462782114
54531280.00.013038.053.050.02650.00.1-4.0-4.01.0-0.33542020.5128558-0.14260967-2.2358072-0.018547743253001730.50547614602452431.2925570398383892-1.2713737583029847
55541320.00.013284.054.050.02700.00.1-7.0-7.01.0-0.282282470.37500617-0.08610266-1.6489343999999997-0.0092231140275748640.54533885901980461.22458317951629-1.2321072523498846
56551360.00.013530.055.050.02750.00.1-6.0-6.00.0-0.218221230.36336467-0.014703972-1.71146290000000010.094372764269542530.54469174494120511.331035892786309-0.9946180646183179
57561400.00.013776.056.050.02800.00.1-5.0-5.01.0-0.220506820.41666690.009601783000000001-1.7307801-0.012329091966749430.50036894270657611.4353393300183517-1.3342522711543707
58571440.00.014022.057.050.02850.00.1-8.0-8.01.0-0.32122620.5308435-0.030598007000000003-1.98283980.094539933778852880.55041842399292881.5666012311261561-1.1316535475938316
59581480.00.014268.058.050.02900.00.1-7.0-7.01.0-0.233160080.43947235-0.037777036-1.95365-0.171298047839841930.56383886594753520.9939428529090564-1.3540981612018674
60591520.00.014514.059.050.02950.00.1-7.0-7.01.0-0.28076990.485092-0.017208176000000002-2.0056467000000002-0.036867801153148320.66873526654619451.2771299437166523-1.4308311167636312
61601560.00.014760.060.050.03000.00.1-2.0-2.00.0-0.097911600000000020.206908990.0038049333-1.17755120.0050175428225569530.45021401163475280.9965365949028968-1.1649347180500662
62611600.00.015006.061.050.03050.00.1-6.0-6.01.0-0.245242160.47082582-0.017673476-2.0848897-0.0061757598912959450.52386795817620171.1629148220868293-1.3912813871782372
63621640.00.015252.062.050.03100.00.1-8.0-8.01.0-0.282336000000000030.44688663-0.025501877000000003-2.1076639999999998-0.0178245619468244550.56303201882135711.188843819753321-1.2891849324544902
64631680.00.015498.063.050.03150.00.1-4.0-4.01.0-0.206038860.37363875-0.06503886-2.0487978-0.0119103649457249580.48950330759618441.0423701939981196-1.1885090016238928
65641720.00.015744.064.050.03200.00.1-6.0-6.01.0-0.122978350.362100060.037675317-1.8098493000000002-0.26719689933922330.58553123565484131.3365902391101674-1.3248513864841
66651760.00.015990.065.050.03250.00.1-4.0-4.00.0-0.081184380.219609040.0143149495-1.09271240.0285208549264075950.47781646620933121.355392152010302-1.1081669136848125
67661800.00.016236.066.050.03300.00.1-16.0-16.01.0-0.361902799999999970.5305404-0.008638233-2.3107697999999997-0.0418455411961810.66755273717974751.4501628035761638-1.4544915576582993
68671840.00.016482.067.050.03350.00.1-8.0-8.01.0-0.329822780.43299633-0.089039-1.8652939-0.127672678718797920.58297123783281451.1270591809981678-1.2913839795826991
69681880.00.016728.068.050.03400.00.1-14.0-14.01.0-0.366166830.5278052-0.011882408999999998-2.38380980.0041913673182936790.78273733076459711.387449249078092-1.3611842457532866
70691920.00.016974.069.050.03450.00.1-4.0-4.01.0-0.191613880.38784048-0.010439545-2.164228-0.174126000045559750.63427502697934891.2450984506265532-1.3542347341431584
71701960.00.017220.070.050.03500.00.1-5.0-5.00.0-0.25016850.45993152-0.050869767-2.17498060.0216934694811392670.55646939706040431.191841553703726-1.1736667485848906
72712000.00.017466.071.050.03550.00.1-6.0-6.01.0-0.200326930.549336250.029257447000000002-2.6023392999999997-0.29697382173212280.54778843344247830.9840468016508408-1.3711175422168895
73722040.00.017712.072.050.03600.00.1-13.0-13.01.0-0.35356450.5873285-0.014556366999999999-2.384103-0.070947895784116250.63053768535269241.473316805213584-1.3116136712647788
74732080.00.017958.073.050.03650.00.1-8.0-8.01.0-0.163596820.30474820.07785887-1.0529268999999999-0.176867469920230670.57116567499503481.1584051943303506-1.465919312966704
75742120.00.018204.074.050.03700.00.1-7.0-7.01.0-0.180658090.380539099999999960.021932207000000002-1.3086073-0.101638413909103660.58286319157447261.2763375012648082-1.418514465334079
76752160.00.018450.075.050.03750.00.1-8.0-8.00.0-0.347538770.5725944000000001-0.06415391-2.58692460.138799461756922320.55150808059002041.3261100934434231-1.2847545188918992
77762200.00.018696.076.050.03800.00.1-16.0-16.01.0-0.712544260.80899066-0.15797406-3.17976740.21813076287873120.66305441743048551.3805395634733055-1.2925259864320122
78772240.00.018942.077.050.03850.00.1-5.0-5.01.0-0.0823412540.320653830.1116728-1.6559769-0.100748503414447060.6043148436861281.4041836091910649-1.2160530066512916
79782280.00.019188.078.050.03900.00.1-13.0-13.01.0-0.421297599999999940.781391560.028527372000000002-2.648484-0.114448908851558260.70721058584944611.4112573429168669-1.4483241427867437
80792320.00.019434.079.050.03950.00.1-10.0-10.01.0-0.256625180.42161843-0.010157827-1.7175516-0.046691167294307020.5880002492482761.3525262524759007-1.1142610128387525
81802360.00.019680.080.050.04000.00.1-14.0-14.00.0-0.375068499999999970.71783950000000010.037293755-2.3992615-0.044462487496858530.60704175633317771.3462864193538158-1.4103955919189517
82812400.00.019926.081.050.04050.00.1-8.0-8.01.0-0.252703700000000030.552807870.028826915-2.25901720.0060252031150218250.62424879462074721.2120547184306714-1.3632726814406682
83822440.00.020172.082.050.04100.00.1-6.0-6.01.0-0.208386820.313928070.018091843-1.2205618999999999-0.0145940477030065910.54844903402389111.2148665802435654-1.143510874813045
84832480.00.020418.083.050.04150.00.1-7.0-7.01.0-0.120556190.400998440.13174866-1.9004566999999999-0.19554013879629660.55600324931687661.0075502601067354-1.4040168782283564
85842520.00.020664.084.050.04200.00.1-10.0-10.01.0-0.213437780.322734860.012250107-1.3083220.0095919725811493950.47035395289406821.3039723382684552-1.1787996602560773
86852560.00.020910.085.050.04250.00.1-9.0-9.00.0-0.369415999999999970.66531396-0.053399727-2.8132071-0.203572932196783850.67096777909874771.049605155625067-1.4014223950847515
87862600.00.021156.086.050.04300.00.1-5.0-5.01.0-0.245523100000000020.5684642-0.008544434-2.997054-0.367185015043834160.59080258434311311.2710339879467538-1.5088983666882407
88872640.00.021402.087.050.04350.00.1-7.0-7.01.0-0.184235590.54926450.059881154000000006-2.471029-0.054054110737143080.62062242403675731.3041515884826391-1.4090556845178217
89882680.00.021648.088.050.04400.00.1-19.0-19.01.0-0.401557360.614109040.07774555-2.2672242999999996-0.226277635776010.64511425734560051.1819164653611656-1.4841059602521427
90892720.00.021894.089.050.04450.00.1-10.0-10.01.0-0.295283620.58127886-0.0007468014999999999-2.803614-0.09813116204224380.66785542189529351.247758372348676-1.297520825502246
91902760.00.022140.090.050.04500.00.1-10.0-10.00.0-0.379920930.77346640000000010.00026521087000000004-2.97408560.049347373266965770.54451007916196161.3153221107392314-1.1511792706753434
92912800.00.022386.091.050.04550.00.1-11.0-11.01.0-0.451999370.84681493-0.04392955-3.3464906-0.103013808644515370.57362768165745451.3268307752364354-1.162494002169657
93922840.00.022632.092.050.04600.00.1-10.0-10.01.0-0.408955700000000030.61014575-0.039490446-2.19346280.059891679203347890.63301452692416941.3066830491881858-1.5027102155196093
94932880.00.022878.093.050.04650.00.1-14.0-14.01.0-0.2914760.465084760.03947228-1.4904562-0.09808793444731460.61954095443338521.203949418503127-1.2411107867157043
95942920.00.023124.094.050.04700.00.1-5.0-5.01.0-0.169812130.32466394-0.026878792999999998-2.0009506-0.088636429467208660.61232930443523741.3107241115047228-1.1480845255486214
96952960.00.023370.095.050.04750.00.1-10.0-10.00.0-0.204322560.406888660.07209943-1.4465636999999998-0.048613312334165180.53677884838498141.1744927407667691-1.4847820134200247
97963000.00.023616.096.050.04800.00.1-9.0-9.01.0-0.242847460.478498850.0010736025999999999-2.4299834-0.107166712796797260.591926993530381.0070418878078162-1.3629839380264572
98973040.00.023862.097.050.04850.00.1-25.0-25.01.0-1.0302241.1442902-0.022970252000000004-3.42781780.069435866884844670.58791952074685521.359919392696689-1.2123885256102642
99983080.00.024108.098.050.04900.00.1-7.0-7.01.0-0.15982460.32014480.016046502-1.09841070.0095111425108868050.57325542293416191.270016855120134-1.2896197746563147
100993120.00.024354.099.050.04950.00.1-7.0-7.01.0-0.151220660.46696490.06971504-2.1169887000000003-0.10392818510552790.59258610071080691.1300944046906352-1.3487996221419198
1011003160.00.024600.0100.050.05000.00.1-6.0-6.00.0-0.277768099999999960.6187519-0.03443319-3.41275449999999970.13559312660509090.63225772586468941.4308110537839671-1.3546496704262838
1021013200.00.024846.0101.050.05050.00.1-6.0-6.01.0-0.142573650.295345400000000040.005846996-1.1098921-0.13269498554963290.53019138623159590.9161116119233748-1.3580411307586011
1031023240.00.025092.0102.050.05100.00.1-7.0-7.01.0-0.111091554000000010.32829150.052363068-1.4122856000000001-0.05608237454235640.53897769370766651.2834064202491409-1.4792622195542806
1041033280.00.025338.0103.050.05150.00.1-8.0-8.01.0-0.228147770.44967908-0.027521253-2.0313194-0.0237068078226290130.58647974776666471.3738713178402515-1.222729860461276
1051043320.00.025584.0104.050.05200.00.1-14.0-14.01.0-0.52936080.81966746-0.038332462000000005-2.92960330.087082339685732080.5702293104140271.280956222849594-1.1586762673972262
1061053360.00.025830.0105.050.05250.00.1-9.0-9.00.0-0.260191770.57444740.031422276-2.3183224-0.122863260530430480.62029304986971091.3138727499154292-1.3919010283825963
1071063400.00.026076.0106.050.05300.00.1-12.0-12.01.0-0.373011470.5400729999999999-0.018245034-2.1085150.033776906499401980.67899258578577651.1634539566903508-1.1932604950540129
1081073440.00.026322.0107.050.05350.00.1-5.0-5.01.0-0.151109310.29639940000000004-0.0041621514-1.0547383999999997-0.065764690828104280.53631616412215781.1427680282448462-1.1633418926524386
1091083480.00.026568.0108.050.05400.00.1-11.0-11.01.0-0.210477950.448143540.061123785-1.5154232-0.070466190227100060.56818617993978321.1336843388592863-1.5098454748215742
1101093520.00.026814.0109.050.05450.00.1-9.0-9.01.0-0.243854950.503787040.0011050701-2.5799710.082775298188151080.61773107833513461.3353604195223605-1.1411398654624316
1111103560.00.027060.0110.050.05500.00.1-15.0-15.00.0-0.468541230.80484974-0.0024912022-3.74598570000000030.026805809875721840.65698412188460311.2496405509627788-1.2932294863587304
1121113600.00.027306.0111.050.05550.00.1-10.0-10.01.0-0.351949450.848184050.0203727-3.3417394-0.072154635837997970.49989948208894241.4639599232329472-1.2822999718837775
1131123640.00.027552.0112.050.05600.00.1-12.0-12.01.0-0.206141860.430295200000000040.0536305-1.8091616999999998-0.142162872478582110.52754562381054741.0244909765609136-1.3665197370656752
1141133680.00.027798.0113.050.05650.00.1-14.0-14.01.0-0.344837550.5993560.0354063-2.31798220.094221970716014530.55904573390719171.2326232121569314-1.4188138791768692
1151143720.00.028044.0114.050.05700.00.1-9.0-9.01.0-0.170132830.374208060.0417809-1.0723688999999998-0.069003607848952890.54605227109455721.019577215316937-1.3968503986441518
1161153760.00.028290.0115.050.05750.00.1-4.0-4.00.0-0.065799960.290762070.058597103-1.4247187-0.0044935057955230750.61365541637028841.472823580373972-1.2416349537344498
1171163800.00.028536.0116.050.05800.00.1-8.0-8.01.0-0.161795840.352725630.040722996000000004-1.18670149999999990.0604016109431159960.48921858432144721.2113853893719546-1.1761648386468997
1181173840.00.028782.0117.050.05850.00.1-13.0-13.01.0-0.29597770.45060240.07142298-1.4123838-0.0150882153989219880.54033889318647651.1040019132381378-1.2430847464829855
1191183880.00.029028.0118.050.05900.00.1-8.0-8.01.0-0.152133480.33171450.020325545-1.0353725-0.051018564365703490.53870093795990471.3515704991835902-1.5471903533129372
1201193920.00.029274.0119.050.05950.00.1-11.0-11.01.0-0.283165570.631537850.034552652-2.49695230.156325403832018540.59984321574523961.2668380149528908-1.4226570735217936
1211203960.00.029520.0120.050.06000.00.1-14.0-14.00.0-0.668796061.26307690.057835735-4.03416870.0201197003105011430.58298658878172381.289173587063609-1.2461655323473746