gryf/coach
1
0
Fork 0
mirror of https://github.com/gryf/coach.git synced 2025-12-18 03:30:19 +01:00
Code Issues Projects Releases Wiki Activity

Enabling Coach Documentation to be run even when environments are not installed (#326)

Browse Source
This commit is contained in:
anabwan
2019-05-27 10:46:07 +03:00
committed by Gal Leibovich
parent 2b7d536da4
commit 342b7184bc
157 changed files with 5167 additions and 7477 deletions
Download Patch File Download Diff File Expand all files Collapse all files

70
docs/design/horizontal_scaling.html
View File

@@ -8,7 +8,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Distributed Coach - Horizontal Scale-Out &mdash; Reinforcement Learning Coach 0.11.0 documentation</title>
<title>Distributed Coach - Horizontal Scale-Out &mdash; Reinforcement Learning Coach 0.12.1 documentation</title>
@@ -17,13 +17,21 @@
<script type="text/javascript" src="../_static/js/modernizr.min.js"></script>
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<script type="text/javascript" src="../_static/language_data.js"></script>
<script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../_static/js/theme.js"></script>
<link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../_static/css/custom.css" type="text/css" />
@@ -33,21 +41,16 @@
<link rel="prev" title="Network Design" href="network.html" />
<link href="../_static/css/custom.css" rel="stylesheet" type="text/css">
<script src="../_static/js/modernizr.min.js"></script>
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search">
<div class="wy-side-nav-search" >
@@ -190,14 +193,14 @@
three interfaces for horizontal scale-out, which allows for integration with different technologies and flexibility.
These three interfaces are orchestrator, memory backend and data store.</p>
<ul class="simple">
<li><strong>Orchestrator</strong> - The orchestrator interface provides basic interaction points for orchestration, scheduling and
<li><p><strong>Orchestrator</strong> - The orchestrator interface provides basic interaction points for orchestration, scheduling and
resource management of training and rollout workers in the distributed coach mode. The interactions points define
how Coach should deploy, undeploy and monitor the workers spawned by Coach.</li>
<li><strong>Memory Backend</strong> - This interface is used as the backing store or stream for the memory abstraction in
how Coach should deploy, undeploy and monitor the workers spawned by Coach.</p></li>
<li><p><strong>Memory Backend</strong> - This interface is used as the backing store or stream for the memory abstraction in
distributed Coach. The implementation of this module is mainly used for communicating experiences (transitions
and episodes) from the rollout to the training worker.</li>
<li><strong>Data Store</strong> - This interface is used as a backing store for the policy checkpoints. It is mainly used to
synchronizing policy checkpoints from the training to the rollout worker.</li>
and episodes) from the rollout to the training worker.</p></li>
<li><p><strong>Data Store</strong> - This interface is used as a backing store for the policy checkpoints. It is mainly used to
synchronizing policy checkpoints from the training to the rollout worker.</p></li>
</ul>
<a class="reference internal image-reference" href="../_images/horizontal-scale-out.png"><img alt="../_images/horizontal-scale-out.png" class="align-center" src="../_images/horizontal-scale-out.png" style="width: 800px;" /></a>
<div class="section" id="supported-synchronization-types">
@@ -207,12 +210,12 @@ rollout worker. For each algorithm, it is specified by using the <cite>Distribut
<cite>agent_params.algorithm.distributed_coach_synchronization_type</cite> in the preset. In distributed Coach, two types of
synchronization modes are supported: <cite>SYNC</cite> and <cite>ASYNC</cite>.</p>
<ul class="simple">
<li><strong>SYNC</strong> - In this type, the trainer waits for all the experiences to be gathered from distributed rollout workers
<li><p><strong>SYNC</strong> - In this type, the trainer waits for all the experiences to be gathered from distributed rollout workers
before training a new policy and the rollout workers wait for a new policy before gathering experiences. It is suitable
for ON policy algorithms.</li>
<li><strong>ASYNC</strong> - In this type, the trainer doesn’t wait for any set of experiences to be gathered from distributed
for ON policy algorithms.</p></li>
<li><p><strong>ASYNC</strong> - In this type, the trainer doesn’t wait for any set of experiences to be gathered from distributed
rollout workers and the rollout workers continously gather experiences loading new policies, whenever they become
available. It is suitable for OFF policy algorithms.</li>
available. It is suitable for OFF policy algorithms.</p></li>
</ul>
</div>
</div>
@@ -228,7 +231,7 @@ available. It is suitable for OFF policy algorithms.</li>
<a href="../contributing/add_agent.html" class="btn btn-neutral float-right" title="Adding a New Agent" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="network.html" class="btn btn-neutral" title="Network Design" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
<a href="network.html" class="btn btn-neutral float-left" title="Network Design" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
@@ -237,7 +240,7 @@ available. It is suitable for OFF policy algorithms.</li>
<div role="contentinfo">
<p>
&copy; Copyright 2018, Intel AI Lab
&copy; Copyright 2018-2019, Intel AI Lab
</p>
</div>
@@ -254,27 +257,16 @@ available. It is suitable for OFF policy algorithms.</li>
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<script type="text/javascript" src="../_static/language_data.js"></script>
<script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../_static/js/theme.js"></script>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</script>
</body>
</html>