1
0
mirror of https://github.com/gryf/coach.git synced 2025-12-17 19:20:19 +01:00
Files
coach/docs_raw/source/diagrams.xml
guyk1971 74db141d5e SAC algorithm (#282)
* SAC algorithm

* SAC - updates to agent (learn_from_batch), sac_head and sac_q_head to fix problem in gradient calculation. Now SAC agents is able to train.
gym_environment - fixing an error in access to gym.spaces

* Soft Actor Critic - code cleanup

* code cleanup

* V-head initialization fix

* SAC benchmarks

* SAC Documentation

* typo fix

* documentation fixes

* documentation and version update

* README typo
2019-05-01 18:37:49 +03:00

1 line
8.2 KiB
XML

<mxfile modified="2019-04-24T09:04:49.730Z" host="www.draw.io" agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" etag="Oj1WgTFAGXXvHH9O1PX0" version="10.6.3" type="device"><diagram id="33c2a640-8c1e-935c-0e0a-86b5dd5c932c" name="Page-1">7V1bd6M6sv41eUwWQlwfc+l073O6e7J3nzln5uksbBOH2Rg8GCed/etHwgiDSgZsSxjSyszqbcvmYr6qUtWnqtIVvl/9/JwF65dv6SKMr0xj8fMKP1yZJkK2Rf5DR953Iy4qB5ZZtCi/tB/4Ef0VloNGObqNFuGm8cU8TeM8WjcH52mShPO8MRZkWfrW/NpzGjevug6WIRj4MQ9iOPp/0SJ/2Y16trEf/xJGyxd2ZWSUn8yC+Z/LLN0m5fWuTPxc/O0+XgXsXOX3Ny/BIn2rDeFPV/g+S9N892r18z6M6bNlj2133OOBT6v7zsIk73UAxrtDXoN4G7J7Lu4sf2dPo/g9IT0CXeG7t5coD3+sgzn99I3gT8Ze8lVcfryMgw19+gZ5PU9X0bx8vcmz9M/wPo3TrDgrduZeOHuuPmHPGZOR5yiOa99cBKH3PKfjaZKXwmIa5fva94zij4wHcbRMyFgcPuf0bTYvj3LIO/iIyqf2GmZ5+LM2VD6yz2G6CvPsnXyFfWp5JX7vTG59bzfwthcXiwnFS01UsFM+36AU0WV19j1M5EWJlBg1x1MI2iLYvFTHtSL4yaH/64NgqQPkQlmwiMI9akmahDKAdeUAS/Bp4GoajgFxtQS4Osw2nIOrb2hcc0lIuk0NpTYaIIlFSCJLBpIIIBmQeWoYy+q7D4brSsYPmS34yQDMt25sDjLHv/Hw/s8GAFYaUwdQCn5mtyaGyeKW+hrk3ZxCQwGp41XN7fQJxcEsjO8q76D2EO+K/4nR6vn4yVPP3v9Br0QeYfn2nwfnXcN9vL1rAy0PsmWYNwU5XDQ8JghjDRVbAAoby8I4yKPXpp8lQqq8wlMakburpKSyvUxGKpeRnWOTbrN5WB5W93zAmWxe3rDBnWv3IMC5COrBe+1ra/qFzTE3jY3We7N998wDkN1+ALgl7gDyYvcr99pSYd1LgdweCqSnsp5TGeeU4L5TmckL9CmmEKl0NocHr9WQNgRLethgNnWuCidrMDqOKGbwZcxoGkUlKCK/J4oWP1OdpIzOLw+jpFAP8zOgQBsxFhlVGf4lnB23m5AMJGH+lmZ/FnwX/YVZuIho3GCQ6CFKE/JiEREAotl293as8cSAgQN2mkCKZkdlcQJkz/5e4Ji/0H/Dn2vyk4MSuXUaR/N3Bu0mWK1j+qXnLF1VR5SAEwg10t1IuwMibQGk51kY5BS0IKkraJQ8U3zT2b/CneYmFMydekd5BX+eZntBqQ7efTlIivBisYjocBDvzxrM0m0ODtOyAWTDwgPKBpyUK+0OaMxLUC10vIJ5m0TPabaK33n9r76xKQAz7+lYnCbL4peQH0I+IfdHrUP4HGzjxtRQykhxGxstFTVwL2EwYNi0NxgUbyIVm4jJwyYqIN4JAXn2OTMRi+IFszOL6jtJ+EYnmOSVyle4WacJNTBjhXwE9KLRJHt8a0BRYB5JBxtMfkrOUYiNZ8+eWu0hl0OMdJiTBxSS8Tv6YKJ5EN+WH6yixYJeRigEezEx+hItfdmQ4nvlLxQsTp6r0KbA1xMTIRJA1JRwY7YbCSOM7aZIIIfDujchbHAn4uN1SWwwf8Nd3C7yAE3Nie15TC3qkQHwS8i1+THlGvmc/PBs4oXk2jYUyzWM2LJtEYe/BEUwVoxRPyxKCu/LIT47kZVktllXIjP52blVmWTPzphfdDdcMD0jIacmY362e+H9Qnzw7VqjLcGlRj3QNlWh7QK05y9pummhRc6HtFzT+6jONc+hmIJwufKKpAMKF6jS2SbMXmFYO03dvGSc5AmQ9BQByc4xEX/yZ5TX3Eny7p/lPUj2NHfu2lU9gqo7nywPdyTeJx90WK59mvfJe4UOL2KSvE/+hm3U7n3y98V9/2zvs1JZrQWHtABNQA242Mni2cBT1cDFatSAD6ourgaQNOOWKz7E3D5slGX7XOaiI6BBmcGWPr1ruojBMCpTxWuyd2piLOZqW/yzs2IbonWk/YAkTsealzYgPQwIJyomNB+ipTAp5gOyNM1UiN3CJlvyjJLqg/k2K37z/sPXIIuCWazXPY9Y90SGAcFWtvBpQpYmWCwqSImByKKETg9Uod+CrPgorT6vr5FriPuyrvtq1DrEomVRKRD3SA//sP5APZxhlq2xpOSO2kdAxunxjHXj1P5cq+kzDLR2igyvo87Fbz/g7AAHf5A4X5Lwj1bQ3ZMFnTvTUMunnaLNL78cfwBoDtD5203JygNjSTLNUucb5EPyWbXNjLl0m6+LvFh2+Fi9hQt6fvaQGW8Yxm2/olkcGZ1pI27WbgqILBvpnE1unsEYYBhgcjaFJxA4WxIl0zAllw88eAvjeUNaGEECQI0hCNdXJXsQrqNNuigSQFjSx3JXmcvCJM0eHcce8YyC7QDYkSr6GcNws8kfNfLqa/TRKlyl5aW0OvfkEQTNb5TxCLhHgfKv4DF4o/IYLF4mzFPTUAcLpEDIf/Ewh6lHTbQXhXY/6vlHwvwjqtx2BSohY/qxLtYubzT194raKIgKNsV9FKRU88L4lIsS9n7EBoQI66xZq69DhaOXoWx8sd5eFowQiyKAD2KML5lniiyrC1ZVaadWj84oI/If1eWl2OaoPEjgXVmGUETOTVNBlt963iE5KEtQ+50mVKsOTC/kmkV3hyvaQTif0zJ/xnTrOaX/nCKWgCHmFKZ0ohYQcVSU86fPAPZG6wdNUBwNuOVeDnBIPw4E3WgCAUmNuPjMfeQIaCdhJy4ZbfFsSDvFYZAle+0sDbJ03/CDF5NV7HoFq2B1AJmKaGIHUi7b9WKXNlg6S7UATpvc3ibXvVxPZmdSqTbnloiVD2wsXrzlN+Xg9PRy/kw+LxtDeupsN4ZpyNRlihVLeEYdYxKj5DgeshzbwB7vUHhISsTJN4jZbw6hOhXMs9pXMGzn3AOsjn7hMIqX3C/cgaRcnKZ0Ef85paRssl1d1Zt1FEv8NIbKo5Wg2d00qbtB11GADDgCBl5VGYgjCp+cmGK2iF4bWDr/3tLdkO4opNclOrc0rKb+c/UpeVVmdezOslnTrpy7sTglKJTj5MbqH9WGi+uyUS1Ox4oT3zwCeaLG2IrEyYXuvt5upRUu03fB9heXc+1d6NqLl2Q0gm0IWublEFS5MP6L7/dhCyZmdVtXuXBpXNvSYzXR9i6nidCTDunrgvz6EH7NsDymANzO3AVVi9wuXOQu50nWOV63FT4NZC60x4KSeGWdC10XYEaJFfYDSzCaBrf+1GrI7hkit04RcSkHLuSIjuawgixnrFh5g8XYY0R/ZnlBAW9GBsuv0Nv9V5jn7yXgwTZPyVCa5S/pku568JXG/3WBPUCZtUrKmeQXm4rr5Jfbt6inN6vVu6U8DHOg5IyHrVSX48J8lJHwj5bFeWygAqr/roQeP9u4ijqmwZvu4AAdnv88+gBWYXb41/Oeb/OAs1lGb1rNiy5B97sidRvXYpRlc2JSbcd9tLrxZ3IUNS4Aks2qFHrrztEH4A7tBLeEJVP6HnRWo9U6S3VH16tTXFR+6wujk65TFYV4H6RUS6UVFfmN5risKC9Qzqn13y7f+R+cSZnL0mUUwW889gCzq8EMf0um5AYzvshlUbVSVjXHv+ha2UfPjBPsdD8k3eD36M0xodTVY9j5ffKq12bJj0heNfg6NgPDanhLtAuCxXqknIUlJHvJE1vTAoJVkARLQVXa+HytMSjlvhVaa1sD0XwsRScvtq+zM/fC2XMfnVwEofc8l6F8SNLCC8GIAw0JOiFjUdYB4hmak1CDdT5x+BrGSrVP7uw4Dt0DnScEK57qdK9HOKNG9zxzhp1e8+HCDr2FdbruyUGJy6tEgox+T5DQz1p7nLeFpij7W5UfTFSXOvkfyQ0eh6LzvTkQ4xIGSBIjU7xW9D4oIYjSkIoOXVqt6FNTdFA3NKyiX8ybnpaiY7DlrQtRUqfoQ+aAa0VXo+gYAUX3BlT0Hs3nddjcHTabopJrZWEzcQMBbDpulhE3m6Iaa1Vxc8W16Gm2CybORmIBvaFsmkU6cP4Ams6HZBgP6E8jUyv6SYHzsIo+5EqxVvSBAudhFV0zZL1QAoEzNgcMnJFmyKav6CBwxnjAwBldjCHzHRcHvRQ9RETV3QvP6Dafj48swRIzYnnE8nV9SJIsTF6jLE1WWuNVaDxyMZQlOG0gNpXIV3rNlp2Cm81P9qYFUROzZTI6FCKk2TIp+mfzqXqmI9A+ZWyZqdmynjDxjpHASCrzrU3Nln0ATedzObE5oG/N9ForejtKfK3FwIqu2bLpK7rD7wU0rKJrtqwXSnx9GFF0iJI6Rdds2fQV3XWAorsDKvrFAudpsWWOARgOLMxIUMWWmTBU1mzZNDXeQYB5xYIcUnVsGYbxOqBdjNcofNN49rMNnBNgCfrDIt9VhSaM60GhpEazP5eNEeCyh8UT+t6aFJVBiiKBb66OFMUi51yHUN2kKHIGDKGwaLVYh1AT03SebkPugCEUvth+adNSdECKDqvookBXK/q0FB2QosMquq6m74USIEWRA1FSpuiWaDFaK/q0FB2Qooj1Bx9C0S3RSvkgil490f5tpPo2C9ynDzlKwmZ+02rXgEEzFkZb/F5jp2FmAsw+1VnMQeAbfgNbxagCPXRZW64GFSKw5XLKKS24lH27FOGpW38dSMUGVLMrSPJ1BasWciwpJEGiZL2lXSYJFLmAypqsA1XJjxRr6nOo+aLUbNEWwzL2WUEWDGnTba5x69Y3H5DHAyPXYz130AbGrRgcalMM7ffR+yhUpmckPYkJ5F5DMOAeq32bEiOwp4vRs7f7CU16kQWj3nS2CbNX8jTShAU5s0wYWVHgrzeFJNDACrnrn7vIiIutsvAtyBaSTna0c0CE67H4a/HTBnYTkNuiNk2FM5Ek42ViXkIFLrxoczYproItitvHa7kOtl4/s716NfeOxW5ZtiXLbnWfSqLdsmEQH8xPNFnaxEgyMVVn6MuYGBtaFLm7kXXuvTDhzch6yMvpVs82RS5cX1MofTuy6n4agSvRSvK86Kl3HIQRp0vytIcJhc4rNGwFT1EI2/Qm9ssCdUKQ9V9pUEdylB0rUPaWnQe1rvf2cAQ7yFSDl9D1HsUOI/I89yJngqD5sKSeDZotAI1ZybH4qtg3wP4jp+5XKDqZwVsmmf5qD+aGSNmavoxWAX3glYP4lQrcU7qJCv8WP8zSPE9XrZJYeZ28k5lTC3EXbNbhnKLzHP2k5vGuuOQtGzXYCHn9kufrTeEsP5L/L6P8ZTu7IZaUvPlOaYIk+PG+ycPVhgzM02D+Qv6bBW/0gQSbgkp8jFZL8m8QL9OMHL7a3KyTpZxpyOP6CO43kW6sRQvoODl8nO0rnoU+lMeJuFmobeXrXIPGKuQbBs1Dsmeh4tBjdwMD7S9N13FarZXHLRjwB1ydu1kXYqG6luRpSLIxUUn2cfsBEiQZaUmekiT3dTJHJ8mecpssCFS0JI9XkqXHuANJMjJYkrs6UVZN12hRlirKfVfX5dM1jqUlZUqSYk3V6FW7AKgzeioWpLQoKxNlZ7KibCmfvx0tylMSZelLq4OJsuupFmVXi/KURNmfqiibHQ5GjyM8s+sIVhhz4AgJ6uJpdZmSuniTVRdLOZ8Gc53/t3hpGn9b59Eq+qtMeubkfYzFD62SJX8XJ4NHE8GCTSTK7JNSJ+bqxamR2iDXEtggZyRTtsVPp04nEcpbLUe2DXL16tSURNm9XI6qq1d/piQp/khW5I83ejbLYlJn9PTqz5RE2Z3s/G07xx7hsp0D1Qk/zD8WNznQYQa2ePjMQcMMvV4zVjPlimbckWRzAjPVSXVg3kxJpzpcvV4zKVGWnm8xlChbLKBVJ8qwW8lTGkfzd03bHT2fDkzb6aWDsRohNR4+NA+O7d34NtEaY/cvn6fg2TeeTXxwc/ev27zAgfqro80acPLdkuc7aNYcPibmjjjfrLEM/ppZe/j9+0BG7Mxq4yGNmG/wy0IetGGiqkMpJoxVDNVQ+v2Paw2UqBaPd2o9wb7grjKgTADUXZrm5LEF6zWBpdYVZNfjo8CQH9SoAvXDdjeqyBDBasmAFXaIfNDaJ1yiRRZfzzyspYQ017dv9xooCBSoGyJOEKxaVgcUbNr5/ZMGSgCU5XSbPnUwwa1Lvl//yMM1nLV+v/4aBlkSJUs9o/XAFRlmN66+CFdbBq4COuP2qwauD3A2tJwAOEvQ7AG3BK79gYMdPL7fPmrgegBX9XdrA06dJYVpf/dBHtJuJHPa70tHBieqo8+xOYPaUV/Anzw+aeB6AGfzbRpFtLAydfQhpVLx+Z/L7RI0jH2sqs+XyrAYrNGLXLRlsRQcTYCJZvdHwe77omZu/kiqG8GCYWeNC2aZkMpqXHxdcz4eUZ6egFqA+W4T0D5XRBx32r8HNyTh/eapJPY09CGTdDvPCdKmcZ9FOe2dq6fxzmncwpx5E03jbCld/iwOaaaHh6fPGrkeyGG+7J4EQEMyuT6kku7jaF0sdxlPT3/TIPYB0eVBdLEAROYiyAcR0koaur6WE65NDqh/Jlv11G7jCNzGegRkGoIIyDQuVstlGrrqb0qSwrJgLx6KoCbDg42ymORwrGw6rUecHSubbPFMi/IkRNkYC+1ztCibCKkWZV2WOClRHklrk+NF2e44QoIoawZzUqI8mgrbY0XZ4WIuBaIMWcXPJJ59+RYkwXIiWxMPzUPxKGLBQqCqMp/KxainXgdUWU3jj68aLpg34dhcpi5GBgRMmDkhBS9IG37RQIn0ii1xt8HENsmSDxMkBr9t4zy6DnT3ADEd6PIZ8ELAWAKmfMBgntmPMH6+XsfBu4YLZiFx25wPawQRTB/7lGzC1Yz8QI0VSDVidTitiiXKvJWCFcwY+3J7r5dI+kxhJvQ1WHu+IZZI2AawNeget7p8S+Rr8Bo2KEywzI62VkiTa5bIoQEDi48cYKYpmL5U+RosQajuHH798UXjBBXLdrpxYjttyccJkhnfbncJNhqpHv67wAiq0ynIY/z9/o4ORMEVrWCBdXfMZdxoZ6QbYN8y+Qplz4f4itxIGZv1mkjixg+/IhN+eGHg3IR11ryhuQg/9MplS1tom8uq9Q3uMn0TdH1+I1+PFRnLz881EWSQHrZhXNQJG7s2Ck6wopYnmW3WBVq8FXuLiJiYxtOnP7SF6zGFGSAjTWDgfGUGzpdn4PosAlp+w/hdGzcGfTof2ALSn/oUZhHBJswOCuyRxg+LjN9I1roNvoOcV3ZEOmwrmTE/cERXMYTh2jfINyzXKf9tns217Zvap+bJZrjtIp7VdhGJFroyMXz2vaEJrr7p21wRpM9O0sWbcKieZHFNUXuiAr9y4vyiJ85+MBoOHxuIgHREFLMUICW2ftaxgczYgBUoNabHsdQKgtJUH3U0lrQM3HpE1/SI+caUnntiKIKwZ96QoBwh17FcA5zXcm/2H3rMS1ExCUpsE6t1T73ujWSb++N1D2O/9YhO3aMKw2ufd3NyrS4CpgCz8ytQMwx9zbvwJXiN0ox2wTHu41S3EOvprjiIkyRRq0VlrTeqBjraYg6W09swgSy9Unn7bNi4mqcLe1sbx2nvxO2PtBO36/BdbmR34mbN0GuG8bdVlOtNBQ7YPqtH9yh12cZM9+pF77/p/ol9kHP5fgWiWYttKC0fOJh2/EcQJTMyF2jwusFDfLcv30CiPgWK+u2ZWG/mMR5vpLmA4ItcFOnF7qevnrr+jbP/422Q7dwg8OnR7g2yXcFpaqusLReRF2Td/fXwPX387d/P//PydPc12nxK//Ff1yYLO7W7LltBit//8LO6WfruvXx3UHk4leD75Llq9eQQGcDvPN3ddcy0Wo/o7jrGX/H0rmMWTx6y2WIoFbNgOv/tvV576eVZWLADkqDMSZE/fwBO3S9leFtKHs3elNI371d1u3qkle3sSNquyJdml4/vAmmDbQWOtMcef8WT7bGNwc4VyrpA/vz/u5d0nv8Zz/57M/+x9D7/+GN2XUUL9Vq4XyajAdhegYIcNsc2umhXMzGeTC2VGmRtdHmje2B26jbIwMftIYDjs6gg+ezsvrrcKnZ/iwpuRZqHS95maZrXv160u0gXIf3GfwA=</diagram></mxfile>