This is one of the 100 recipes of the IPython Cookbook, the definitive guide to high-performance scientific computing and data science in Python.

1.6. Creating a simple kernel for IPython

This recipe has been tested on IPython 4. It should work on IPython 3 with minimal changes. We give all references about wrapper kernels and messaging protocols at the end of this recipe.

Besides, the code given here works with Python 3. It can be ported to Python 2 with minimal efforts.

In [ ]:
%%writefile plotkernel.py
# NOTE: We create the `plotkernel.py` file here so that 
# you don't have to do it...
from ipykernel.kernelbase import Kernel
#from IPython.kernel.zmq.kernelbase import Kernel # IPython < 4.x
import numpy as np
import matplotlib.pyplot as plt
from io import BytesIO
import urllib, base64

def _to_png(fig):
    """Return a base64-encoded PNG from a 
    matplotlib figure."""
    imgdata = BytesIO()
    fig.savefig(imgdata, format='png')
    imgdata.seek(0)
    return urllib.parse.quote(
        base64.b64encode(imgdata.getvalue()))

_numpy_namespace = {n: getattr(np, n) 
                    for n in dir(np)}
def _parse_function(code):
    """Return a NumPy function from a string 'y=f(x)'."""
    return lambda x: eval(code.split('=')[1].strip(),
                          _numpy_namespace, {'x': x})

class PlotKernel(Kernel):
    implementation = 'Plot'
    implementation_version = '1.0'
    banner = "Simple plotting"
    language_info = {
        'name': 'python', # will be used for syntax highlighting
        'version': '',
        'file_extension': '.plot',
        'mimetype': 'text/x-python'
    }
    # language and language_version needed only for protocol version < 5.0
    language = language_info['name']
    language_version = language_info['version']

    
    def do_execute(self, code, silent,
                   store_history=True,
                   user_expressions=None,
                   allow_stdin=False):

        # We create the plot with matplotlib.
        fig = plt.figure(figsize=(6,4), dpi=100)
        x = np.linspace(-5., 5., 200)
        functions = code.split('\n')
        for fun in functions:
            f = _parse_function(fun)
            y = f(x)
            plt.plot(x, y)
        plt.xlim(-5, 5)

        # We create a PNG out of this plot.
        png = _to_png(fig)

        if not silent:
            # We send the standard output to the client.
            self.send_response(self.iopub_socket,
                'stream', {
                    'name': 'stdout', 
                    'data': 'Plotting {n} function(s)'. \
                                format(n=len(functions))})

            # We prepare the response with our rich data
            # (the plot).
            content = {
                #'source': 'kernel',  # IPython < 4.x

                # This dictionary may contain different
                # MIME representations of the output.
                'data': {
                    'image/png': png
                },

                # We can specify the image size
                # in the metadata field.
                'metadata' : {
                      'image/png' : {
                        'width': 600,
                        'height': 400
                      }
                    }
            }        

            # We send the display_data message with the
            # contents.
            self.send_response(self.iopub_socket,
                'display_data', content)

        # We return the exection results.
        return {'status': 'ok',
                'execution_count': self.execution_count,
                'payload': [],
                'user_expressions': {},
               }

if __name__ == '__main__':
    from ipykernel.kernelapp import IPKernelApp
    #from IPython.kernel.zmq.kernelapp import IPKernelApp # IPython < 4.x
    IPKernelApp.launch_instance(kernel_class=PlotKernel)

1. First, we create a file plotkernel.py. This file will contain the implementation of our custom kernel. Let's import a few modules.

from ipykernel.kernelbase import Kernel
#from IPython.kernel.zmq.kernelbase import Kernel # IPython < 4.x
import numpy as np
import matplotlib.pyplot as plt
from io import BytesIO
import urllib, base64

2. We write a function that returns a PNG base64-encoded representation of a matplotlib figure.

def _to_png(fig):
    """Return a base64-encoded PNG from a 
    matplotlib figure."""
    imgdata = BytesIO()
    fig.savefig(imgdata, format='png')
    imgdata.seek(0)
    return urllib.parse.quote(
        base64.b64encode(imgdata.getvalue()))

3. Now, we write a function that parses a code string which has the form y = f(x), and returns a NumPy function. Here, f is an arbitrary Python expression that can use NumPy functions.

_numpy_namespace = {n: getattr(np, n) 
                    for n in dir(np)}
def _parse_function(code):
    """Return a NumPy function from a string 'y=f(x)'."""
    return lambda x: eval(code.split('=')[1].strip(),
                          _numpy_namespace, {'x': x})

4. For our new wrapper kernel, we create a class deriving from Kernel. There are a few metadata fields we need to provide.

class PlotKernel(Kernel):
    implementation = 'Plot'
    implementation_version = '1.0'
    banner = "Simple plotting"
    language_info = {
        'name': 'python', # will be used for syntax highlighting
        'version': '',
        'file_extension': '.plot',
        'mimetype': 'text/x-python'
    }
    # language and language_version needed only for protocol version < 5.0
    language = language_info['name']
    language_version = language_info['version']

5. In this class, we implement a do_execute() method that takes code as input, and sends responses to the client.

def do_execute(self, code, silent,
                   store_history=True,
                   user_expressions=None,
                   allow_stdin=False):

        # We create the plot with matplotlib.
        fig = plt.figure(figsize=(6,4), dpi=100)
        x = np.linspace(-5., 5., 200)
        functions = code.split('\n')
        for fun in functions:
            f = _parse_function(fun)
            y = f(x)
            plt.plot(x, y)
        plt.xlim(-5, 5)

        # We create a PNG out of this plot.
        png = _to_png(fig)

        if not silent:
            # We send the standard output to the client.
            self.send_response(self.iopub_socket,
                'stream', {
                    'name': 'stdout', 
                    'data': 'Plotting {n} function(s)'. \
                                format(n=len(functions))})

            # We prepare the response with our rich data
            # (the plot).
            content = {
                'source': 'kernel',

                # This dictionary may contain different
                # MIME representations of the output.
                'data': {
                    'image/png': png
                },

                # We can specify the image size
                # in the metadata field.
                'metadata' : {
                      'image/png' : {
                        'width': 600,
                        'height': 400
                      }
                    }
            }        

            # We send the display_data message with the
            # contents.
            self.send_response(self.iopub_socket,
                'display_data', content)

        # We return the exection results.
        return {'status': 'ok',
                'execution_count': self.execution_count,
                'payload': [],
                'user_expressions': {},
               }

6. Finally, we add the following lines at the end of the file.

if __name__ == '__main__':
    from ipykernel.kernelapp import IPKernelApp
    #from IPython.kernel.zmq.kernelapp import IPKernelApp # IPython < 4.x
    IPKernelApp.launch_instance(kernel_class=PlotKernel)

7. Our kernel is ready! The next step is to indicate to IPython that this new kernel is available. To do this, we need to create a kernel spec kernel.json file in a directory named after our kernel.

In [ ]:
!mkdir plot

The kernel.json file contains the following lines:

In [ ]:
%%writefile plot/kernel.json
{
 "argv": ["python", "-m",
          "plotkernel", "-f",
          "{connection_file}"],
 "display_name": "Plot",
 "language": "python"
}

If you are using IPython 3, simply move the directory plot (with kernel.json in it) to ~/.ipython/kernels/. This may still work (deprecated) for IPython 4, but since IPython 4 is Jupyter based, the proper way is to copy the whole directory to the appropriate location using the jupyter command:

In [ ]:
!jupyter kernelspec install plot --user --replace

The main argument to this command is the path to the directory containing the kernel spec. The option --user instructs Jupyter to install the kernel for the current user only. The option --replace overwrites any already installed kernel with the same name (if exists).

The installation can be verified by listing all kernels known to Jupyter and their locations:

In [ ]:
!jupyter kernelspec list

The plotkernel.py file needs to be importable by Python. For example, you could simply put it in the current directory.

8. In IPython 4, you can launch a notebook with this kernel from the IPython notebook dashboard, in a dropdown menu from the ‘New’ button.

In IPython 3, this feature may not be available in all versions. An alternative (deprecated) is to run the following command in a terminal:

ipython notebook --KernelManager.kernel_cmd="['python', '-m', 'plotkernel', '-f', '{connection_file}']"

9. Finally, in a new notebook backed by our custom plot kernel, we can simply write mathematical equations y=f(x). The corresponding graph appears in the output area.

You'll find all the explanations, figures, references, and much more in the book (to be released later this summer).

IPython Cookbook, by Cyrille Rossant, Packt Publishing, 2014 (500 pages).