update of api docstrings across coach and tutorials [WIP] (#91)

* updating the documentation website
* adding the built docs
* update of api docstrings across coach and tutorials 0-2
* added some missing api documentation
* New Sphinx based documentation

tutorials/0. Quick Start Guide.ipynb

@@ -11,7 +11,52 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Creating a very simple graph containing a single clipped ppo agent running with the CartPole-v0 Gym environment:"
+    "## Using Coach from the Command Line"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "When running Coach from the command line, we use a Preset module to define the experiment parameters.\n",
+    "As its name implies, a preset is a predefined set of parameters to run some agent on some environment.\n",
+    "Coach has many predefined presets that follow the algorithms definitions in the published papers, and allows training some of the existing algorithms with essentially no coding at all. This presets can easily be run from the command line. For example:\n",
+    "\n",
+    "`coach -p CartPole_DQN`\n",
+    "\n",
+    "You can find all the predefined presets under the `presets` directory, or by listing them using the following command:\n",
+    "\n",
+    "`coach -l`\n",
+    "\n",
+    "Coach can also be used with an externally defined preset by passing the absolute path to the module and the name of the graph manager object which is defined in the preset: \n",
+    "\n",
+    "`coach -p /home/my_user/my_agent_dir/my_preset.py:graph_manager`\n",
+    "\n",
+    "Some presets are generic for multiple environment levels, and therefore require defining the specific level through the command line:\n",
+    "\n",
+    "`coach -p Atari_DQN -lvl breakout`\n",
+    "\n",
+    "There are plenty of other command line arguments you can use in order to customize the experiment. A full documentation of the available arguments can be found using the following command:\n",
+    "\n",
+    "`coach -h`"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Using Coach as a Library"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Alternatively, Coach can be used a library directly from python. As described above, Coach uses the presets mechanism to define the experiments. A preset is essentially a python module which instantiates a `GraphManager` object. The graph manager is a container that holds the agents and the environments, and has some additional parameters for running the experiment, such as visualization parameters. The graph manager acts as the scheduler which orchestrates the experiment.\n",
+    "\n",
+    "Let's start with some examples.\n",
+    "\n",
+    "Creating a very simple graph containing a single Clipped PPO agent running with the CartPole-v0 Gym environment:"
    ]
   },
   {
@@ -52,7 +97,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Running each phase manually:"
+    "### Running each phase manually"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The graph manager simplifies the scheduling process by encapsulating the calls to each of the training phases. Sometimes, it can be beneficial to have a more fine grained control over the scheduling process. This can be easily done by calling the individual phase functions directly:"
    ]
   },
   {
@@ -77,7 +129,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Changing the default parameters"
+    "### Changing the default parameters\n",
+    "\n",
+    "Agents in Coach are defined along with some default parameters that follow the published paper definition. This may be sufficient when running the exact same experiments as in the paper, but otherwise, there would probably need to be some changes made to the algorithm parameters. Again, this is easily modifiable, and all the internal parameters are accessible from within the preset:"
    ]
   },
   {
@@ -117,11 +171,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Using a custom gym environment\n",
+    "### Using a custom gym environment\n",
     "\n",
     "We can use a custom gym environment without registering it. \n",
     "We just need the path to the environment module.\n",
-    "We can also pass custom parameters for the environment __init__"
+    "We can also pass custom parameters for the environment `__init__` function as `additional_simulator_parameters`."
    ]
   },
   {
@@ -164,12 +218,21 @@
     "graph_manager.improve()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The path to the environment can also be set as an absolute path, as follows: `<absolute python module path>:<environment class>`. For example:"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "env_params = GymVectorEnvironment(level='/home/user/my_environment_dir/my_environment_module.py:MyEnvironmentClass')"
+   ]
   }
  ],
  "metadata": {