This notebook is just an example to show how to run MuZero (https://github.com/werner-duvaud/muzero-general) in Google Colab or Jupyter Notebook. You can also launch MuZero directly by cloning the github repository and running the command python muzero.py
. See readme for detailed instructions.
# Set up the environment
!git clone https://github.com/werner-duvaud/muzero-general.git
%cd muzero-general/
!pip install -r requirements.txt
!pip uninstall -y pyarrow
%load_ext tensorboard
# If you have the error : "ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'", You did not import the muzero files correctly.
# If you have an import issue with ray or pyarrow in google colab, restart the environment (in the execution menu)
Cloning into 'muzero-general'... remote: Enumerating objects: 98, done. remote: Counting objects: 100% (98/98), done. remote: Compressing objects: 100% (60/60), done. remote: Total 429 (delta 60), reused 59 (delta 32), pack-reused 331 Receiving objects: 100% (429/429), 3.66 MiB | 3.20 MiB/s, done. Resolving deltas: 100% (237/237), done. /content/muzero-general Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 1)) (1.18.2) Requirement already satisfied: torch in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 2)) (1.4.0) Requirement already satisfied: tensorboard in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 3)) (2.1.1) Requirement already satisfied: gym in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 4)) (0.17.1) Collecting box2d-py Downloading https://files.pythonhosted.org/packages/06/bd/6cdc3fd994b0649dcf5d9bad85bd9e26172308bbe9a421bfc6fdbf5081a6/box2d_py-2.3.8-cp36-cp36m-manylinux1_x86_64.whl (448kB) |████████████████████████████████| 450kB 1.3MB/s Collecting ray Downloading https://files.pythonhosted.org/packages/5c/38/391eeaa496c4ead4ba47d98a6acb740d776fec9da37ec87ddf75f76bb14a/ray-0.8.3-cp36-cp36m-manylinux1_x86_64.whl (20.1MB) |████████████████████████████████| 20.1MB 231kB/s Requirement already satisfied: psutil in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 7)) (5.4.8) Collecting setproctitle Downloading https://files.pythonhosted.org/packages/5a/0d/dc0d2234aacba6cf1a729964383e3452c52096dc695581248b548786f2b3/setproctitle-1.1.10.tar.gz Collecting aiohttp Downloading https://files.pythonhosted.org/packages/7c/39/7eb5f98d24904e0f6d3edb505d4aa60e3ef83c0a58d6fe18244a51757247/aiohttp-3.6.2-cp36-cp36m-manylinux1_x86_64.whl (1.2MB) |████████████████████████████████| 1.2MB 2.9MB/s Requirement already satisfied: grpcio in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 10)) (1.27.2) Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard->-r requirements.txt (line 3)) (3.2.1) Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard->-r requirements.txt (line 3)) (46.0.0) Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.6/dist-packages (from tensorboard->-r requirements.txt (line 3)) (0.9.0) Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorboard->-r requirements.txt (line 3)) (1.7.2) Requirement already satisfied: wheel>=0.26; python_version >= "3" in /usr/local/lib/python3.6/dist-packages (from tensorboard->-r requirements.txt (line 3)) (0.34.2) Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard->-r requirements.txt (line 3)) (2.21.0) Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard->-r requirements.txt (line 3)) (3.10.0) Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard->-r requirements.txt (line 3)) (1.0.0) Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.6/dist-packages (from tensorboard->-r requirements.txt (line 3)) (0.4.1) Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard->-r requirements.txt (line 3)) (1.12.0) Requirement already satisfied: cloudpickle<1.4.0,>=1.2.0 in /usr/local/lib/python3.6/dist-packages (from gym->-r requirements.txt (line 4)) (1.3.0) Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from gym->-r requirements.txt (line 4)) (1.4.1) Requirement already satisfied: pyglet<=1.5.0,>=1.4.0 in /usr/local/lib/python3.6/dist-packages (from gym->-r requirements.txt (line 4)) (1.5.0) Requirement already satisfied: jsonschema in /usr/local/lib/python3.6/dist-packages (from ray->-r requirements.txt (line 6)) (2.6.0) Collecting colorama Downloading https://files.pythonhosted.org/packages/c9/dc/45cdef1b4d119eb96316b3117e6d5708a08029992b2fee2c143c7a0a5cc5/colorama-0.4.3-py2.py3-none-any.whl Requirement already satisfied: google in /usr/local/lib/python3.6/dist-packages (from ray->-r requirements.txt (line 6)) (2.0.3) Requirement already satisfied: pytest in /usr/local/lib/python3.6/dist-packages (from ray->-r requirements.txt (line 6)) (3.6.4) Collecting py-spy>=0.2.0 Downloading https://files.pythonhosted.org/packages/8e/a7/ab45c9ee3c4654edda3efbd6b8e2fa4962226718a7e3e3be6e3926bf3617/py_spy-0.3.3-py2.py3-none-manylinux1_x86_64.whl (2.9MB) |████████████████████████████████| 2.9MB 57.8MB/s Requirement already satisfied: click in /usr/local/lib/python3.6/dist-packages (from ray->-r requirements.txt (line 6)) (7.1.1) Requirement already satisfied: packaging in /usr/local/lib/python3.6/dist-packages (from ray->-r requirements.txt (line 6)) (20.3) Collecting redis>=3.3.2 Downloading https://files.pythonhosted.org/packages/f0/05/1fc7feedc19c123e7a95cfc9e7892eb6cdd2e5df4e9e8af6384349c1cc3d/redis-3.4.1-py2.py3-none-any.whl (71kB) |████████████████████████████████| 71kB 9.8MB/s Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from ray->-r requirements.txt (line 6)) (3.13) Requirement already satisfied: filelock in /usr/local/lib/python3.6/dist-packages (from ray->-r requirements.txt (line 6)) (3.0.12) Collecting yarl<2.0,>=1.0 Downloading https://files.pythonhosted.org/packages/95/8f/0209fc5d975f839344c33c822ff2f7ef80f6b1e984673a5a68f960bfa583/yarl-1.4.2-cp36-cp36m-manylinux1_x86_64.whl (252kB) |████████████████████████████████| 256kB 56.0MB/s Collecting idna-ssl>=1.0; python_version < "3.7" Downloading https://files.pythonhosted.org/packages/46/03/07c4894aae38b0de52b52586b24bf189bb83e4ddabfe2e2c8f2419eec6f4/idna-ssl-1.1.0.tar.gz Collecting async-timeout<4.0,>=3.0 Downloading https://files.pythonhosted.org/packages/e1/1e/5a4441be21b0726c4464f3f23c8b19628372f606755a9d2e46c187e65ec4/async_timeout-3.0.1-py3-none-any.whl Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.6/dist-packages (from aiohttp->-r requirements.txt (line 9)) (19.3.0) Requirement already satisfied: typing-extensions>=3.6.5; python_version < "3.7" in /usr/local/lib/python3.6/dist-packages (from aiohttp->-r requirements.txt (line 9)) (3.6.6) Requirement already satisfied: chardet<4.0,>=2.0 in /usr/local/lib/python3.6/dist-packages (from aiohttp->-r requirements.txt (line 9)) (3.0.4) Collecting multidict<5.0,>=4.5 Downloading https://files.pythonhosted.org/packages/30/2e/3ab2f1fb72571f75013db323a3799d505d99f3bc203513604f1ffb9b7858/multidict-4.7.5-cp36-cp36m-manylinux1_x86_64.whl (148kB) |████████████████████████████████| 153kB 55.7MB/s Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard->-r requirements.txt (line 3)) (0.2.8) Requirement already satisfied: rsa<4.1,>=3.1.4 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard->-r requirements.txt (line 3)) (4.0) Requirement already satisfied: cachetools<3.2,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard->-r requirements.txt (line 3)) (3.1.1) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard->-r requirements.txt (line 3)) (2019.11.28) Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard->-r requirements.txt (line 3)) (1.24.3) Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard->-r requirements.txt (line 3)) (2.8) Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->-r requirements.txt (line 3)) (1.3.0) Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from pyglet<=1.5.0,>=1.4.0->gym->-r requirements.txt (line 4)) (0.16.0) Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.6/dist-packages (from google->ray->-r requirements.txt (line 6)) (4.6.3) Requirement already satisfied: atomicwrites>=1.0 in /usr/local/lib/python3.6/dist-packages (from pytest->ray->-r requirements.txt (line 6)) (1.3.0) Requirement already satisfied: pluggy<0.8,>=0.5 in /usr/local/lib/python3.6/dist-packages (from pytest->ray->-r requirements.txt (line 6)) (0.7.1) Requirement already satisfied: py>=1.5.0 in /usr/local/lib/python3.6/dist-packages (from pytest->ray->-r requirements.txt (line 6)) (1.8.1) Requirement already satisfied: more-itertools>=4.0.0 in /usr/local/lib/python3.6/dist-packages (from pytest->ray->-r requirements.txt (line 6)) (8.2.0) Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from packaging->ray->-r requirements.txt (line 6)) (2.4.6) Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.6/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard->-r requirements.txt (line 3)) (0.4.8) Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->-r requirements.txt (line 3)) (3.1.0) Building wheels for collected packages: setproctitle, idna-ssl Building wheel for setproctitle (setup.py) ... done Created wheel for setproctitle: filename=setproctitle-1.1.10-cp36-cp36m-linux_x86_64.whl size=33912 sha256=43b31607d9ab48103fd3d18601341d72004db35eadaa9f5e761a97b7d5b913bb Stored in directory: /root/.cache/pip/wheels/e6/b1/a6/9719530228e258eba904501fef99d5d85c80d52bd8f14438a3 Building wheel for idna-ssl (setup.py) ... done Created wheel for idna-ssl: filename=idna_ssl-1.1.0-cp36-none-any.whl size=3162 sha256=ad3916cdd20a66f6a417eaf80dab39479bdb5a1e90b73e7f4eb582f6da38c17a Stored in directory: /root/.cache/pip/wheels/d3/00/b3/32d613e19e08a739751dd6bf998cfed277728f8b2127ad4eb7 Successfully built setproctitle idna-ssl Installing collected packages: box2d-py, multidict, yarl, idna-ssl, async-timeout, aiohttp, colorama, py-spy, redis, ray, setproctitle Successfully installed aiohttp-3.6.2 async-timeout-3.0.1 box2d-py-2.3.8 colorama-0.4.3 idna-ssl-1.1.0 multidict-4.7.5 py-spy-0.3.3 ray-0.8.3 redis-3.4.1 setproctitle-1.1.10 yarl-1.4.2 Uninstalling pyarrow-0.14.1: Successfully uninstalled pyarrow-0.14.1
%tensorboard --logdir ./results
!python muzero.py
# The error "OSError: [Errno 99] error while attempting to bind on address ('::1', 8265, 0, 0): cannot assign requested address" is not a problem. MuZero will keep running.
# The logs won't be printed below. Training results will be displayed in real time in the TensorBoard dashboard above.
Welcome to MuZero! Here's a list of games:
0. cartpole
1. connect4
2. gomoku
3. lunarlander
4. tictactoe
Enter a number to choose the game: 0
0. Train
1. Load pretrained model
2. Render some self play games
3. Play against MuZero
4. Exit
Enter a number to choose an action: 0
2020-03-28 15:36:44,740 INFO resource_spec.py:212 -- Starting Ray with 7.13 GiB memory available for workers and up to 3.57 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>).
2020-03-28 15:36:45,163 INFO services.py:1123 -- View the Ray dashboard at localhost:8265
2020-03-28 15:36:46.837861: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-03-28 15:36:49,049 WARNING worker.py:1072 -- The dashboard on node 04c508e9ced9 failed with the following error:
Traceback (most recent call last):
File "/usr/lib/python3.6/asyncio/base_events.py", line 1062, in create_server
sock.bind(sa)
OSError: [Errno 99] Cannot assign requested address
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/ray/dashboard/dashboard.py", line 920, in <module>
dashboard.run()
File "/usr/local/lib/python3.6/dist-packages/ray/dashboard/dashboard.py", line 368, in run
aiohttp.web.run_app(self.app, host=self.host, port=self.port)
File "/usr/local/lib/python3.6/dist-packages/aiohttp/web.py", line 433, in run_app
reuse_port=reuse_port))
File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
return future.result()
File "/usr/local/lib/python3.6/dist-packages/aiohttp/web.py", line 359, in _run_app
await site.start()
File "/usr/local/lib/python3.6/dist-packages/aiohttp/web_runner.py", line 104, in start
reuse_port=self._reuse_port)
File "/usr/lib/python3.6/asyncio/base_events.py", line 1066, in create_server
% (sa, err.strerror.lower()))
OSError: [Errno 99] error while attempting to bind on address ('::1', 8265, 0, 0): cannot assign requested address
Training...
Run tensorboard --logdir ./results and go to http://localhost:6006/ to see in real time the training performance.
2020-03-28 15:36:52,319 WARNING worker.py:1072 -- WARNING: 6 PYTHON workers have been started. This could be a result of using a large number of actors, or it could be a consequence of using nested tasks (see https://github.com/ray-project/ray/issues/3644) for some a discussion of workarounds.