#Imports for running this presentation live
from IPython.html.widgets import interact, interactive
from IPython.display import clear_output, display, HTML
import numpy as np
from scipy import integrate
from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib.colors import cnames
from matplotlib import animation
%matplotlib inline
Even though computers are often considered deterministic, computational software is a rapidly evolving and changing landscape. Libraries are constantly adding new features and fixing issues.
Image source: http://www.michaelogawa.com/research/storylines/
Even libraries with the strictest backwards-compatibility policies can change in significant ways.
Image source: http://www.bonkersworld.net/backwards-compatibility/
A reproducible computational environment has a sufficiently consistent state for the computational task at hand.
For example, this can consist of
Does not include:
Image source: https://www.youtube.com/watch?v=g1LgVfV5_ZQ
Image source: http://time-az.com/images/2014/02/20140203carjam.jpg
Linux container systems , like Docker, are new type of tool to easily build, ship, and run reproducible, binary applications.
It is "good enough" for a reproducible computational environment.
In this talk, we will introduce Docker from the perspective a scientific research software engineer. We will
Docker is an open-source engine that automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere.
!docker run --rm busybox sh -c 'echo "Hello Docker World!"'
Hello Docker World!
Docker works with images that consume minimal disk space, versioned, archiveable, and shareable. Executing applications in these images does not require dedicated resources and is high performance.
It works with containers as opposed to virtual machines (VM's).
%time !docker run --rm busybox sh -c 'echo "Hello Docker World!"'
Hello Docker World! CPU times: user 4 ms, sys: 4 ms, total: 8 ms Wall time: 1.23 s
A Docker container is similar to a running an application in a chroot, but it sandboxes processes and the network stack with Linux kernel:
Union file systems, or UnionFS, are file systems that operate by creating layers, making them very lightweight and fast while saving disk space.
Docker can make use of several union file system variants including:
!docker search itk
docker <subcommand>
docker push
, docker pull
, docker tag
docker export
will create a archiveable tarball of an image's filesystem.Here's what you need:
easy install of
Works, but adds layer of complexity
Mac native interface improving
Comes with busybox shell -> Write your Docker build.sh and run.sh in Bourne shell
!docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE busybox buildroot-2014.02 8c2e06607696 11 weeks ago 2.433 MB busybox latest 8c2e06607696 11 weeks ago 2.433 MB odise/busybox-python latest 649988b8bf0e 4 months ago 20.26 MB
!docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
!docker run -d busybox sh -c 'sleep 3'
3a6bf9d61548ae36bdc0bdb5a87aec17a8056517709c61e2df989aa0a37b7f32
!docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
!docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 3a6bf9d61548 busybox:buildroot-2014.02 "sh -c 'sleep 3'" 14 seconds ago Exited (0) 11 seconds ago goofy_almeida a3761241bf97 busybox:buildroot-2014.02 "sh -c 'sleep 3'" 53 minutes ago Exited (0) 53 minutes ago reverent_stallman
!ls $PWD/images/
BackwardsCompatibility.png DockerVM.jpg BuildInstructions1.png Eww.jpg BuildInstructions2.png FilesystemsGeneric.png BuildInstructions3.png itkka.png BuildInstructions4.png Jenkins.png CarJam.jpg Jupyter.png Chroot.png LayerCake.jpg ConfusedCat.jpg Liar.png Debian.png MakerwareScreenshot.png DockerDeploy.jpg MakerwareVTK.png DockerFilesystemsBusybox.png MakerwareWebsite.png DockerFilesystems.svg MasonJar.jpg DockerHub.png ModulesModulesModules.png DockerLogo.png PythonStoryline.svg
!docker run --rm --volume $PWD/images:/images busybox \
sh -c 'ls /images'
BackwardsCompatibility.png BuildInstructions1.png BuildInstructions2.png BuildInstructions3.png BuildInstructions4.png CarJam.jpg Chroot.png ConfusedCat.jpg Debian.png DockerDeploy.jpg DockerFilesystems.svg DockerFilesystemsBusybox.png DockerHub.png DockerLogo.png DockerVM.jpg Eww.jpg FilesystemsGeneric.png Jenkins.png Jupyter.png LayerCake.jpg Liar.png MakerwareScreenshot.png MakerwareVTK.png MakerwareWebsite.png MasonJar.jpg ModulesModulesModules.png PythonStoryline.svg itkka.png
%%writefile docker-ls-images/Dockerfile
# Best practice for Dockerfile's:
# specify exact versions whenever possible.
FROM busybox:4986bf8c1536
MAINTAINER Matt McCormick <matt.mccormick@kitware.com>
RUN mkdir -p /images
VOLUME /images
CMD ["/bin/sh", "-c", "ls /images"]
Overwriting docker-ls-images/Dockerfile
!docker build -t ls-images ./docker-ls-images
!docker run --rm -v $PWD/images:/images ls-images
YouTubeVideo('QqfjiuqVrV4')
A portable Docker image will only assume standard CPU/memory/disk/network resources are available. If local USB devices and video card devices are used the images will not be runnable anywhere.
Isn't there supposed to be no OpenGL?
It is possible to run accelerated X11 OpenGL 3D applications, but the Docker images that are built will only work on host systems with the same video driver and compatible video card.
See the docker-opengl-nvidia and the docker-opengl-mesa repositories.