JupyterLab

Sergey Zagoruyko @ WILLOW

Next iteration of Jupyter, ready to use!

My experience for keeping track of experiments:

  • Jupyter since 2015 (3 years+)
  • JupyterLab since 2018 (about 1 year)

The most powerful feature:

  • Extensions

Install

a bit messy but installs nodejs:

conda install -c conda-forge jupyterlab

otherwise:

pip install jupyterlab

Vim mode

Text editing (e.g. python code or yaml files) supports by default Sublime, Emacs and Vim.

To edit Jupyter cells there is Vim extension: https://github.com/jwkvam/jupyterlab-vim

Install with:

jupyter labextension install jupyterlab_vim

Kernels

  • IPyKernel
  • IJulia
  • IHaskell
  • Matlab Kernel
  • IOctave

and others https://github.com/jupyter/jupyter/wiki/Jupyter-kernels

UI improvements

  • Built-in terminals
  • Moving cells
  • Panes

Visual Studio Code integration

Microsoft Python extension allows importing and editing jupyter files:

Visualizations

Videos

In [10]:
from IPython.display import HTML

youtube_id = 'ctOM-Gza04Y'

HTML(f'<iframe width="560" height="315" src="https://www.youtube.com/embed/{youtube_id}?rel=0&amp;controls=0&amp;showinfo=0" frameborder="0" allowfullscreen></iframe>')
Out[10]:

Pandas

In [11]:
import pandas as pd
pd.read_json('/sequoia/data1/szagoruy/tmp/47179/exp_args.json')
Out[11]:
dataroot dataset depth nthread save seed sphere-alpha-min sphere-dim sphere-m width
0 /sequoia/data2/szagoruy/datasets/ CIFAR100 28 1 logs/resnet28-10_CIFAR100_resnet_sphereloss961... 1 0 8 4 10
1 /sequoia/data2/szagoruy/datasets/ CIFAR100 28 1 logs/resnet28-10_CIFAR100_resnet_sphereloss961... 1 0 16 4 10
2 /sequoia/data2/szagoruy/datasets/ CIFAR100 28 1 logs/resnet28-10_CIFAR100_resnet_sphereloss961... 1 0 32 4 10
3 /sequoia/data2/szagoruy/datasets/ CIFAR100 28 1 logs/resnet28-10_CIFAR100_resnet_sphereloss961... 1 0 64 4 10
4 /sequoia/data2/szagoruy/datasets/ CIFAR100 28 1 logs/resnet28-10_CIFAR100_resnet_sphereloss961... 1 0 128 4 10

Matplotlib

In [12]:
%pylab inline
%config InlineBackend.figure_format = 'retina'
import seaborn as sns
import torch
sns.set()
Populating the interactive namespace from numpy and matplotlib
In [13]:
a = torch.rand(64, 64, 3)
plt.imshow(a)
Out[13]:
<matplotlib.image.AxesImage at 0x7fde9c958e10>
In [14]:
pd.DataFrame({'x': torch.randn(128).cumsum(0),
              'y': torch.randn(128).cumsum(0),
             }).plot.hist(figsize=(16,8), alpha=0.8);

Graphs

In [15]:
import  torch
from torch import nn
from torchviz import make_dot
In [16]:
lstm_cell = nn.LSTMCell(128, 128)
x = torch.randn(1, 128)
make_dot(lstm_cell(x), params=dict(list(lstm_cell.named_parameters())))
Out[16]:
%3 140594086407248 ThMulBackward 140594086406352 SigmoidBackward 140594086406352->140594086407248 140594085971392 SplitBackward 140594085971392->140594086406352 140594086081088 SigmoidBackward 140594085971392->140594086081088 140594086082376 SigmoidBackward 140594085971392->140594086082376 140594086082264 TanhBackward 140594085971392->140594086082264 140594085973184 ThAddBackward 140594085973184->140594085971392 140590859554208 ThAddmmBackward 140590859554208->140594085973184 140590860327848 ExpandBackward 140590860327848->140590859554208 140594103675088 bias_ih (512) 140594103675088->140590860327848 140594086599088 TBackward 140594086599088->140590859554208 140594103673520 weight_ih (512, 128) 140594103673520->140594086599088 140590859552080 ThAddmmBackward 140590859552080->140594085973184 140594103675928 ExpandBackward 140594103675928->140590859552080 140594103673296 bias_hh (512) 140594103673296->140594103675928 140594103674808 TBackward 140594103674808->140590859552080 140594086081872 weight_hh (512, 128) 140594086081872->140594103674808 140594086405456 TanhBackward 140594086405456->140594086407248 140594086407136 ThAddBackward 140594086407136->140594086405456 140590859551800 ThMulBackward 140590859551800->140594086407136 140594086081088->140590859551800 140590860325216 ThMulBackward 140590860325216->140594086407136 140594086082376->140590860325216 140594086082264->140590860325216

LaTex rendering

Besides writing equations in jupyter cells like that: $$D_{KL} = \sum_i p(i) \log \frac{p(i)}{q(i)},$$ it is possible to write latex files that get compiled and rendered in Jupyter lab.

Debugging

%pdb on / %pdb off magick turns on debugging on errors

Downsides

Not a complete IDE!:

  • Git integration is pain (although there are attemps to improve it)
  • No linter
  • Execution order is mess
  • Code completion is not as good as VSCode or Atom (no completion outside of kernels)

My workflow

With JupyterLab:

  • Keep track of experiments (converge curves, notes)
  • Visualizations (debugging, share notebooks with other people)
  • Start new jobs on cluster

For writing code I use IDE (mostly vim, sometimes VSCode).

In [ ]: