1
0
mirror of https://github.com/gryf/coach.git synced 2026-01-03 04:14:16 +01:00
Files
coach/rl_coach/traces/Fetch_DDPG_HER_baselines_slide/trace.csv
Itai Caspi 72a1d9d426 Itaicaspi/episode reset refactoring (#105)
* reordering of the episode reset operation and allowing to store episodes only when they are terminated

* reordering of the episode reset operation and allowing to store episodes only when they are terminated

* revert tensorflow-gpu to 1.9.0 + bug fix in should_train()

* tests readme file and refactoring of policy optimization agent train function

* Update README.md

* Update README.md

* additional policy optimization train function simplifications

* Updated the traces after the reordering of the environment reset

* docker and jenkins files

* updated the traces to the ones from within the docker container

* updated traces and added control suite to the docker

* updated jenkins file with the intel proxy + updated doom basic a3c test params

* updated line breaks in jenkins file

* added a missing line break in jenkins file

* refining trace tests ignored presets + adding a configurable beta entropy value

* switch the order of trace and golden tests in jenkins + fix golden tests processes not killed issue

* updated benchmarks for dueling ddqn breakout and pong

* allowing dynamic updates to the loss weights + bug fix in episode.update_returns

* remove docker and jenkins file
2018-09-04 15:07:54 +03:00

25 KiB

1Episode #Training IterIn HeatupER #TransitionsER #EpisodesEpisode LengthTotal stepsEpsilonShaped Training RewardTraining RewardUpdate Target NetworkEvaluation RewardShaped Evaluation RewardSuccess RateLoss/MeanLoss/StdevLoss/MaxLoss/MinLearning Rate/MeanLearning Rate/StdevLearning Rate/MaxLearning Rate/MinGrads (unclipped)/MeanGrads (unclipped)/StdevGrads (unclipped)/MaxGrads (unclipped)/MinEntropy/MeanEntropy/StdevEntropy/MaxEntropy/MinAdvantages/MeanAdvantages/StdevAdvantages/MaxAdvantages/MinValues/MeanValues/StdevValues/MaxValues/MinValue Loss/MeanValue Loss/StdevValue Loss/MaxValue Loss/MinPolicy Loss/MeanPolicy Loss/StdevPolicy Loss/MaxPolicy Loss/MinQ/MeanQ/StdevQ/MaxQ/MinTD targets/MeanTD targets/StdevTD targets/MaxTD targets/Minactions/Meanactions/Stdevactions/Maxactions/Min
210.01.0246.01.050.050.00.10.0
320.01.0492.02.050.0100.00.10.0
430.01.0738.03.050.0150.00.10.0
540.01.0984.04.050.0200.00.10.0
650.01.01230.05.050.0250.00.10.0
760.01.01476.06.050.0300.00.10.0
870.01.01722.07.050.0350.00.10.0
980.01.01968.08.050.0400.00.10.0
1090.01.02214.09.050.0450.00.10.0
11100.01.02460.010.050.0500.00.10.0
12110.01.02706.011.050.0550.00.10.0
13120.01.02952.012.050.0600.00.10.0
14130.01.03198.013.050.0650.00.10.0
15140.01.03444.014.050.0700.00.10.0
16150.01.03690.015.050.0750.00.10.0
17160.01.03936.016.050.0800.00.10.0
18170.01.04182.017.050.0850.00.10.0
19180.01.04428.018.050.0900.00.10.0
20190.01.04674.019.050.0950.00.10.0
21200.01.04920.020.050.01000.00.10.0
22210.01.05166.021.050.01050.00.11.0
232240.00.05412.022.050.01100.00.1-50.0-50.01.01.71883171689696580.58401989982363162.4597169440239670.31352961063385010.0010.00.0010.0010.435808660.577739543.11232189999999950.062947065-0.24206230.063335570.013502683500000001-0.35456935-0.179711398938525220.32036782184730760.0-0.9691835492849350.054887021616675830.344466819931475930.9814501104952102-0.9372007403745176
242380.00.05658.023.050.01150.00.1-50.0-50.01.0-0.617402430.27791268-0.24723712-1.0403504000000001-0.112712034744373250.57426319948985891.0941433886250913-1.3697138444417851
2524120.00.05904.024.050.01200.00.1-50.0-50.01.0-0.576181050.04426721-0.4997121-0.69842064-0.0277370326361898420.53214323805015461.1100037668658276-1.2462838648428542
2625160.00.06150.025.050.01250.00.1-50.0-50.00.0-0.69272620.11330087-0.5510265000000001-1.02621630.062643364598740120.41391956432281330.919312415922513-0.9168376403568012
2726200.00.06396.026.050.01300.00.1-50.0-50.01.0-0.387571870.089267425-0.2293561-0.60237750.041798345861013980.46587376782710331.0224315869682925-0.98098311874754
2827240.00.06642.027.050.01350.00.1-50.0-50.01.0-1.13781050.031368796000000004-1.0906267-1.22275160.0346657606461745140.36939009016825770.9657766612792222-0.9263611181543424
2928280.00.06888.028.050.01400.00.1-50.0-50.01.0-0.95634560.13723812-0.7406679-1.21116880.119147665100773530.47273425878216761.1473878377720177-1.0633555916935966
3029320.00.07134.029.050.01450.00.1-50.0-50.01.0-0.78665330.04249498-0.7055869-0.86972390.214515923685393880.45828221369903081.1069574717361714-0.9945747161433748
3130360.00.07380.030.050.01500.00.1-50.0-50.00.0-1.24918190.08099759-1.1538235000000001-1.51981310000000010.18992638253036220.48567548238586531.1225672954415105-0.982814891819228
3231400.00.07626.031.050.01550.00.1-50.0-50.01.0-1.27586849999999980.09126878-1.0656763-1.4053245-0.17297747826966920.7159435607331211.3151942940852681-1.3435239665462697
3332440.00.07872.032.050.01600.00.1-50.0-50.01.0-1.30627199999999990.08536281-1.0545075000000002-1.42597750.0142436140331331520.72004280006573311.6567358048329008-1.0647613704372745
3433480.00.08118.033.050.01650.00.1-50.0-50.01.0-1.44152010000000020.032775674-1.3795292-1.5288534-0.143038137051485440.68928482212376621.5787098365265186-1.3623742856918404
3534520.00.08364.034.050.01700.00.1-50.0-50.01.0-0.95262250.074560665-0.8395200999999999-1.19053700000000020.223786695522087270.56892946295967921.2604852005171656-0.9357410694406424
3635560.00.08610.035.050.01750.00.1-50.0-50.00.0-1.22589730.09099273-1.0642208999999998-1.51390180.52143447398070530.6115065878571611.5950855195191391-1.0285206191816554
3736600.00.08856.036.050.01800.00.1-50.0-50.01.0-1.9533560.07440405-1.8437569-2.14552260.095156884456923680.71566577667302421.3925414990267515-1.2519778107822506
3837640.00.09102.037.050.01850.00.1-50.0-50.01.0-0.84542440000000010.32941390000000004-0.37694475-1.4998542-0.22141802713198650.71343695580875451.315860325484811-1.4363586739178655
3938680.00.09348.038.050.01900.00.1-50.0-50.01.0-1.78864030000000020.14224246-1.5692306-2.1063473-0.21512577269985740.72534912062100761.401767178450565-1.3549240396441895
4039720.00.09594.039.050.01950.00.1-50.0-50.01.0-0.675453660.07182985-0.5454933000000001-0.85040176-0.240562353450759440.60568714269816911.14125293498854-1.4873311782021892
4140760.00.09840.040.050.02000.00.1-50.0-50.00.0-1.19615639999999980.10231872-1.0433656999999998-1.3937783000000001-0.036321494525406190.51801771440906211.3512268220870058-1.1587167322890646
4241800.00.010086.041.050.02050.00.1-50.0-50.01.0-1.63718620.03962088-1.5843941000000001-1.7720933999999997-0.133196971019513060.50289020195856581.0600027941934318-1.0418515011207568
4342840.00.010332.042.050.02100.00.1-50.0-50.01.0-1.53769760.08765263-1.3890513-1.81459580.177918439328860530.71156438171035461.4655918300856945-1.2225499662739183
4443880.00.010578.043.050.02150.00.1-50.0-50.01.0-1.50105380.12090099-1.3382475-1.7832435000000002-0.083314684102704870.64047129396657821.2599290650143409-1.2854240356505318
4544920.00.010824.044.050.02200.00.1-50.0-50.01.0-0.791454850.18698223-0.5678517-1.25589-0.14174750613874130.53670561252520271.002319252499581-1.2170244628081603
4645960.00.011070.045.050.02250.00.1-50.0-50.00.0-0.196815060.047280043-0.13587418-0.38333774-0.134960153872883650.74952332871662511.2723735657738808-1.4984531053047674
47461000.00.011316.046.050.02300.00.1-50.0-50.01.0-1.16954580.19859816-0.9403771000000001-1.62238120.049565902923216360.66314787263449321.3486953817146463-1.3075225821137662
48471040.00.011562.047.050.02350.00.1-50.0-50.01.0-1.36285089999999980.24758416-1.0261942-1.82643450000000020.129785311270051580.50731351257089791.175029375256638-0.9629348236937084
49481080.00.011808.048.050.02400.00.1-50.0-50.01.0-1.35160390.115459956-1.1588036-1.5993525-0.0621673597740415240.51137298729571360.9809827813834358-1.1391775869597
50491120.00.012054.049.050.02450.00.1-50.0-50.01.0-1.61446860000000010.21469948-1.415216-2.22101620.27727099363557050.51258028252236751.17837052679628-0.9855610193337112
51501160.00.012300.050.050.02500.00.1-50.0-50.00.0-1.12962580.09693045-0.9719747999999999-1.45163250.195551584193280060.5996271490018241.366732715995532-1.342378806391384
52511200.00.012546.051.050.02550.00.1-50.0-50.01.0-1.4440530.13863231-1.2533459999999998-1.70970129999999990.183444428665241270.54036555204140251.4194536168972856-0.9847881151901791
53521240.00.012792.052.050.02600.00.1-50.0-50.01.0-2.08801410.1090059-1.9019004000000002-2.3887820.205199516801043120.65130716751688611.3795860599583245-1.0345755050291128
54531280.00.013038.053.050.02650.00.1-50.0-50.01.0-2.1801760.04374436-2.1205623-2.33598920.156699253635125130.39130986147515450.9709221833373916-0.9962228830131336
55541320.00.013284.054.050.02700.00.1-50.0-50.01.0-1.35938950.21013483-1.1201533000000001-1.8766505-0.139485477370492380.80932917570463881.6117412574911691-1.4496189843592957
56551360.00.013530.055.050.02750.00.1-50.0-50.00.0-2.18499780.06614068-2.0483747-2.3140001-0.0275170637356810.56508097090769251.1550543354033491-1.0845052085516422
57561400.00.013776.056.050.02800.00.1-50.0-50.01.0-2.16855810000000030.031280026-2.1135757-2.2656322-0.196520744205118450.42997280001618340.9954918918371648-1.004785164629367
58571440.00.014022.057.050.02850.00.1-50.0-50.01.0-2.56105070.26920515-2.2064169999999996-2.9158046000000004-0.02117491130223990.59508695756030691.405079827687446-1.3457864090625777
59581480.00.014268.058.050.02900.00.1-50.0-50.01.0-1.99704480.07583436-1.8006881000000001-2.12341710.142447940366659450.73682880112392361.3275323225514657-1.3471964215258283
60591520.00.014514.059.050.02950.00.1-50.0-50.01.0-1.59635300000000010.5743522-0.5994944000000001-2.4069877-0.098099328025291440.81800956592204071.378215237331767-1.3767520548235537
61601560.00.014760.060.050.03000.00.1-50.0-50.00.0-2.37801240.033048052-2.3072817000000003-2.4705890.0415965871345046060.385912533391748640.9965365949028968-0.9895655063359372
62611600.00.015006.061.050.03050.00.1-50.0-50.01.0-2.8008370.08441382-2.5906959-2.9377577-0.160653583673076860.49111294311274380.988001477065262-1.1097969691491265
63621640.00.015252.062.050.03100.00.1-50.0-50.01.0-2.06955800000000020.160446-1.8003315-2.38895820000000030.356028807639461650.56244188348843671.4402985481868258-0.9056072520528624
64631680.00.015498.063.050.03150.00.1-50.0-50.01.0-2.58970070000000030.06532196-2.4573104-2.7314050.078935800364342630.50908257396101291.172902664518461-1.0411446428165858
65641720.00.015744.064.050.03200.00.1-50.0-50.01.0-2.48493080.054665398-2.4130947999999997-2.624780.205665050783819050.37828732991601880.9950011449428064-0.8998928134546351
66651760.00.015990.065.050.03250.00.1-50.0-50.00.0-1.70587640.19166876-1.4136803-2.2017016-0.183922205591127260.56124413896700581.21483538898191-1.4198907054943042
67661800.00.016236.066.050.03300.00.1-50.0-50.01.0-2.44813849999999980.09029243-2.2610283-2.64846629999999950.24537160114128650.71588380176962511.4500535482622905-1.4609292169219101
68671840.00.016482.067.050.03350.00.1-50.0-50.01.0-2.20971820.06673066-2.0536735-2.33946079999999950.32835366647808070.84486514958039291.5660574524863924-1.360755298106077
69681880.00.016728.068.050.03400.00.1-50.0-50.01.0-2.27324650.08006355-2.1903055-2.49398100000000020.34823703540083830.7761389642110631.3706702370957684-1.5039174106796218
70691920.00.016974.069.050.03450.00.1-50.0-50.01.0-2.62640740.064830735-2.549128-2.84517650.309528621110023550.40480677117998221.3685533765846971-0.8440477935136792
71701960.00.017220.070.050.03500.00.1-50.0-50.00.0-2.85900950.06743985400000001-2.705806-2.9761326-0.11406472777638880.57946734509288471.1580088428125093-1.308084467261595
72712000.00.017466.071.050.03550.00.1-50.0-50.01.0-1.04436830000000010.45131195-0.5413604000000001-1.9176447-0.20451612098988880.62397852474622471.1236075853020535-1.6111703371597141
73722040.00.017712.072.050.03600.00.1-50.0-50.01.0-2.06781980.24895296-1.5700306-2.3396242000000003-0.0383175993691435850.58623478160092671.2819700706326604-1.4779521514829037
74732080.00.017958.073.050.03650.00.1-50.0-50.01.0-2.7572680.057650782000000005-2.6842852-2.955076-0.063976902153495390.49245420617843740.9706348051509576-1.2014591748955614
75742120.00.018204.074.050.03700.00.1-50.0-50.01.0-2.25405740.13137417-2.0654504-2.59384679999999970.200703253796810340.81456956183108251.5391162656590356-1.4185922493955112
76752160.00.018450.075.050.03750.00.1-50.0-50.00.0-2.4230680.04118608-2.3586411-2.5697980.30830233122955750.47545940533775121.3268536505318158-0.9895616689051594
77762200.00.018696.076.050.03800.00.1-50.0-50.01.0-2.53536940.12237016-2.320174-2.89671590.18633898525281840.68686091793161251.3792907269559715-1.162784438637446
78772240.00.018942.077.050.03850.00.1-50.0-50.01.0-3.07229949999999970.056910317-2.9488866000000002-3.167251-0.028165309193077150.66901114751625671.397227449122553-1.1754738495853323
79782280.00.019188.078.050.03900.00.1-50.0-50.01.0-3.579540.08289930000000001-3.392689-3.68270780.27743113803786050.4753533320112771.3172206545957164-0.926632937282216
80792320.00.019434.079.050.03950.00.1-50.0-50.01.0-2.79870629999999960.046286296-2.7161832-2.93671079999999970.284423153074198960.60178972727649391.3525040199433995-0.9745275302815136
81802360.00.019680.080.050.04000.00.1-50.0-50.00.0-1.32873120.20427454-1.0509951000000002-1.76318660000000030.176088319824120.59528346154711211.3463144442436947-1.1969953951174614
82812400.00.019926.081.050.04050.00.1-50.0-50.01.0-2.93764070.09256915-2.7281747000000003-3.1525323-0.187341061346052160.59495957803850711.18652439780973-1.4102067970898675
83822440.00.020172.082.050.04100.00.1-50.0-50.01.0-3.20799230.070021264-3.0861422999999997-3.32941560000000040.00322461576418085760.6234126947460261.1306997282032554-1.233139024922721
84832480.00.020418.083.050.04150.00.1-50.0-50.01.0-3.09748630.060003977-3.0229256-3.2685148999999996-0.0499514983246090240.43470059639430470.9689411154798222-0.9821274539735534
85842520.00.020664.084.050.04200.00.1-50.0-50.01.0-1.12218650.29446512-0.7096495-1.79376330000000020.029457689292068930.64461498675803851.3337265400447724-1.2534003341475497
86852560.00.020910.085.050.04250.00.1-50.0-50.00.0-2.10454399999999970.15051521-1.8818272-2.49893240.0545446273169917440.65364472013637631.4075452728763502-1.3051923441798232
87862600.00.021156.086.050.04300.00.1-50.0-50.01.0-2.77929260.08755199-2.6564338-3.10336040.46700572658399210.48154731887660651.3198472979888485-0.8496107650765157
88872640.00.021402.087.050.04350.00.1-50.0-50.01.0-2.85137180.13088353-2.6779985-3.29193160.0361952558257868650.60361677735121881.4080817143143587-1.1618184669228262
89882680.00.021648.088.050.04400.00.1-50.0-50.01.0-3.67441339999999970.055435885-3.569443-3.82437470000000030.205850153893310930.5076458973815211.170502257539154-0.9929165580456588
90892720.00.021894.089.050.04450.00.1-50.0-50.01.0-2.67024920.20479229999999998-2.3021027999999997-3.1633530.192004527696428450.82847753804584021.440473227442468-1.358935693640991
91902760.00.022140.090.050.04500.00.1-50.0-50.00.0-2.82880880.25550413-2.532669-3.3633180.24232266333422690.49041278652063721.3039552069573406-0.9529767081415732
92912800.00.022386.091.050.04550.00.1-50.0-50.01.0-1.84003510000000010.24452664-1.4642625-2.48901030.24014884101760210.74036997605642811.3844917479398429-1.3766769574712872
93922840.00.022632.092.050.04600.00.1-50.0-50.01.0-3.33848880.088029824-3.2452571-3.64418820000000030.26946973174365770.72616700961208411.5453778650708625-1.4175846270715562
94932880.00.022878.093.050.04650.00.1-50.0-50.01.0-3.60754079999999980.2172091-3.2896962-3.954105-0.088543087679020070.75466511501424041.4867712355191904-1.3439946568619674
95942920.00.023124.094.050.04700.00.1-50.0-50.01.0-2.28132270000000050.12847067-2.043287-2.5550199-0.176947357383526850.4922483744110541.3477518453705928-1.323845775142464
96952960.00.023370.095.050.04750.00.1-50.0-50.00.0-2.9460850.18847758-2.7402808999999997-3.490946-0.065364894647028520.53408487945621041.15493528444825-1.1227915793403196
97963000.00.023616.096.050.04800.00.1-50.0-50.01.0-3.43805170.040264957000000004-3.3771402999999998-3.5494611000000003-0.041841244143915570.43723700988456640.981070667017124-0.9673340280533124
98973040.00.023862.097.050.04850.00.1-50.0-50.01.0-3.40825080000000020.08805543-3.2786632000000004-3.61569879999999970.214719275166238070.69089562660564491.363433205715488-1.10005308211356
99983080.00.024108.098.050.04900.00.1-50.0-50.01.0-3.33463720000000040.17043224-2.898628-3.68016840.110145666657913030.71502927089703161.4883161567879957-1.394704008563589
100993120.00.024354.099.050.04950.00.1-50.0-50.01.0-3.11432240.26812983-2.7029066000000004-3.50362540000000020.06886925077511070.63604563489203391.2908667629138115-1.1661700689022887
1011003160.00.024600.0100.050.05000.00.1-50.0-50.00.0-3.45522170000000050.052327402-3.3602526-3.5982920.030098042331010910.44942025391715420.9973908415223632-1.0626481924322089
1021013200.00.024846.0101.050.05050.00.1-50.0-50.01.0-2.42470859999999980.16525234-2.142931-2.787823-0.052870827927572160.67531783013707181.3542644470683218-1.3783996459208494
1031023240.00.025092.0102.050.05100.00.1-50.0-50.01.0-2.6787470.08559595-2.4464822-2.870227-0.131734048474799780.57841920202457941.2169492347536337-1.174730418213738
1041033280.00.025338.0103.050.05150.00.1-50.0-50.01.0-3.84630270000000030.086537-3.6924984-4.0537557999999990.0188376934638366630.52928942733551291.1160684636035199-1.1952336367323015
1051043320.00.025584.0104.050.05200.00.1-50.0-50.01.0-4.4899490.07528181-4.3677335-4.61236140.205936685909838960.64171252257446131.4467558266129386-1.028509824118234
1061053360.00.025830.0105.050.05250.00.1-50.0-50.00.0-4.15153799999999950.064833164-4.0609226000000005-4.3274817-0.066684239866008660.66192484429539421.4754127007374265-1.2408326224280095
1071063400.00.026076.0106.050.05300.00.1-50.0-50.01.0-3.48107220000000030.06702451-3.3312163-3.61920550.215314192235087160.72336133483996131.4017970420110302-1.1412416033391242
1081073440.00.026322.0107.050.05350.00.1-50.0-50.01.0-2.77968740.24075618-2.3732238-3.35178610000000040.179381830074531260.48267346265634581.375500213010725-1.1902898082064852
1091083480.00.026568.0108.050.05400.00.1-50.0-50.01.0-3.90460870.06153155-3.8081887-4.083968-0.0208387540441727430.59225417978683191.2549717269742913-1.128198815531843
1101093520.00.026814.0109.050.05450.00.1-50.0-50.01.0-3.78199100000000050.07558259-3.622004-3.95667399999999960.0335714187437568260.54325342746247271.2469286439524447-1.0638029196453769
1111103560.00.027060.0110.050.05500.00.1-50.0-50.00.0-4.64567140.07783405-4.516008-4.83021900000000050.153163476765393080.47004702701112261.112775461830791-0.9698394164484369
1121113600.00.027306.0111.050.05550.00.1-50.0-50.01.0-4.1782120.20909244-3.8713586-4.673853-0.138714388716672740.83040893978999561.2713951386326898-1.7022056749134242
1131123640.00.027552.0112.050.05600.00.1-50.0-50.01.0-1.3717960.20281329999999997-1.11591-2.11633660000000030.23034437601942250.60895315604028411.2459408126552547-1.1911069905813476
1141133680.00.027798.0113.050.05650.00.1-50.0-50.01.0-3.92315150.15559714-3.6907074-4.244309400000001-0.033668244496377410.69319347092792051.2259610094750257-1.3658820467451251
1151143720.00.028044.0114.050.05700.00.1-50.0-50.01.0-4.1691885000000010.03364817-4.1142254000000005-4.23501630.094620046603432470.48676928832172811.046063145315637-0.9981789504134084
1161153760.00.028290.0115.050.05750.00.1-50.0-50.00.0-4.130070.14570194-3.9530773-4.5048237-0.089318874549623130.70514033608888721.399885818226195-1.4499302121880828
1171163800.00.028536.0116.050.05800.00.1-50.0-50.01.0-3.76096630.15003155-3.5411277-4.2424070.03663521294398210.60730786951176911.3067909684747725-1.4865657929189857
1181173840.00.028782.0117.050.05850.00.1-50.0-50.01.0-4.5718860.09401981-4.3963356-4.785101999999999-0.106316656467591780.51945878055806540.9389788315812376-1.2365603852539506
1191183880.00.029028.0118.050.05900.00.1-50.0-50.01.0-4.4746980.05127586-4.387929-4.608597799999999-0.109609633623521640.84466459205946881.3754326121386995-1.6541657760253172
1201193920.00.029274.0119.050.05950.00.1-50.0-50.01.0-3.87866190.72744113-3.1931589-5.3239837-0.428808900038126660.7186070785971571.398065421552768-1.4328618034255136
1211203960.00.029520.0120.050.06000.00.1-50.0-50.00.0-4.38188030.061143212-4.2775702-4.53516240.086221436697709930.71209965863713411.468810720034572-1.0683222411575068