#!/usr/bin/env python
# coding: utf-8

# # Custom D3.js Visualization in a Jupyter Notebook
# 
# by Stefaan Lippens ([@soxofaan](https://twitter.com/soxofaan)), May 2018

# TL;DR: If you want to skip the detailed mumbo jumbo, go straight to the code example in the [conclusion](#All-together-now).
# 
# 
# ## Introduction
# 
# When doing data science in a Jupyter notebook, there are plenty 
# of options for the standard data visualization needs:
# matplotlib, pandas, seaborn, bokeh, etc. 
# Occasionally you might be stuck in a situation where you can not 
# easily express the desired visualization with the 
# standard vocabulary provided by these tools.
# In these cases I like to leverage the flexibility
# of [D3.js](https://d3js.org/) to build a custom graph or diagram.
# 
# 
# In this article, I'll discuss an approach how to 
# implement a **custom do-it-yourself D3.js visualization** in a **Jupyter Notebook**.
# This topic is covered in some other places around the web,
# but I couldn't find a complete approach that
# connects all the dots and isn't too hackish. 
# 
# In particular, I'll try to keep these things in mind here:
# 
# - Leverage the [existing Jupyter functionality](http://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/JavaScript%20Notebook%20Extensions.html),
#   like a [RequireJS](http://requirejs.org/) environment 
#   to have clean dependency handling 
#   and to avoid `<script src="...">` loading/order headaches.
# - No additional packages or dependencies to install
# - The visualisation should work both in an interactive notebook context
#   and in the exported HTML version.
# - Avoid the typical D3.js boilerplate to insert the drawing at the desired
#   location (insert a `<div id="...">` in the markup and use the corresponding id in the javascript code).
#   Keeping these id's properly in sync is annoying to maintain, 
#   especially if you iterate a lot or want multiple drawings in the same notebook.
#   

# ## The basics with inline `%%javascript` snippets

# To cover the basics, let's start simple with just inline `%%javascript` snippet cells.
# 
# 
# First, we tell the RequireJS environment where to find the version of D3.js we want.
# Note that the `.js` extension is [omitted](http://requirejs.org/docs/api.html#config-paths) in the URL. 

# In[1]:


get_ipython().run_cell_magic('javascript', '', "require.config({\n    paths: {\n        d3: 'https://d3js.org/d3.v5.min'\n    }\n});\n")


# We can now create a D3.js powered SVG drawing, for example as follows:

# In[2]:


get_ipython().run_cell_magic('javascript', '', '(function(element) {\n    require([\'d3\'], function(d3) {   \n        var data = [1, 2, 4, 8, 16, 8, 4, 2, 1]\n\n        var svg = d3.select(element.get(0)).append(\'svg\')\n            .attr(\'width\', 400)\n            .attr(\'height\', 200);\n        svg.selectAll(\'circle\')\n            .data(data)\n            .enter()\n            .append(\'circle\')\n            .attr("cx", function(d, i) {return 40 * (i + 1);})\n            .attr("cy", function(d, i) {return 100 + 30 * (i % 3 - 1);})\n            .style("fill", "#1570a4")\n            .transition().duration(2000)\n            .attr("r", function(d) {return 2*d;})\n        ;\n    })\n})(element);\n')


# Note:
# - We leverage RequireJS to hand us a loaded `d3` library in a closure.
# - The SVG drawing is appended to the current output cell:
#     - `element` is the jQuery powered wrapper for this
#     - `element.get(0)` is the DOM node itself that can be handed to `d3.select()`
# - The `element` variable is a global variable and overwritten on each rendering of a Javascript cell, so to make sure we capture the correct `element` inside our D3.js code (which could be executed in a different context), we wrap the whole thing in a closure.

# ## And now, the real work

# Unless you're just toying around with simple visualizations,
# these D3.js scripts can get very extensive,
# which is not very ideal to work on directly in a interactive notebook.
# You probably want to develop the D3.js script in an editor or IDE 
# that gives you a bit more code intelligence.
# 
# 
# Let's cover how to get things working with an separate `.js` script
# and, additionally, with data that is initially defined or constructed in Python. 
# 
# 
# First, we'll need these imports:

# In[3]:


from IPython.display import display, Javascript, HTML
import json


# and we set up the RequireJS search path (repeated here for completeness)

# In[4]:


get_ipython().run_cell_magic('javascript', '', "require.config({\n    paths: { \n        d3: 'https://d3js.org/d3.v5.min'\n    }\n});\n")


# Let's say we have a script `circles.js` that implements a certain visualisation, rougly with this structure:
# 
#     define('circles', ['d3'], function (d3) {
#         function draw(container, data) {
#             var svg = d3.select(container).append("svg");
#             // D3.js drawing stuff here ...
#         }
#         return draw;
#     });
# 
# In addition to declaring a dependency on the `d3` library like before, we now `define` a "module" called `circles`. 
# This explicit naming is [not the standard RequireJS](http://requirejs.org/docs/api.html#modulename) way, 
# but we have to do it because we will embed the Javascript code in the HTML document directly, instead of loading the file with a separate request.
# For simplicity, the defined module is just a single function (internally called `draw`), 
# which expects a container to append the SVG element to and a data object.
# 
# Assuming that this script `circles.js` lives alongside the notebook file, we can inject the javascript code in the notebook like this:

# In[5]:


Javascript(filename='circles.js')


# Now, we define some data in Python space:

# In[6]:


data = [5, 10, 20, 40, 50, 30, 10, 20, 40, 10, 5]


# And we draw the circles like this:

# In[7]:


Javascript("""
(function(element){
    require(['circles'], function(circles) {
        circles(element.get(0), %s)
    });
})(element);
""" % json.dumps(data))


# Note:
# - We use RequireJS again to get our `circles` "module" which is just our drawing function
# - The container to add the drawing to is `element.get(0)` as discussed above
# - We convert the data to JSON and inject it in the `circles` function call in Javascript. For simplicity, I used basic Python string formatting with `%`, but other templating solutions are possible of course.
# - If you want multiple drawings for different data sets, you probably want to put the `Javascript` thing in a reusable function. Don't forget to return the `Javascript` object so it rendered properly.

# ## Some additional practical details

# ### Add CSS
# 
# Apart from your D3.js script, you usually also want to add some custom CSS to the mix, preferably in a separate file as well. Define the CSS in a HTML file, e.g. `circles.css.html` as follows
# 
#     <style>
#     svg circle {
#       stroke: #16527b;
#       stroke-width: 1px;
#     }
#     </style>
# 
# and load it like this:

# In[8]:


HTML(filename='circles.css.html')


# ### Easy reloading

# Reloading the code from the separate script while developing can be very cumbersome. RequireJS will not automatically "reload" a module that is already defined. For a full hard refresh you should: clear the output of the `Javascript(filename=..)` cell (or clear all outputs of the whole notebook), save the notebook and refresh the page in your browser.
# 
# Luckily there is a much easier way using [`require.undef`](http://requirejs.org/docs/api.html#undef). Put it at the top of the script file, before the `define` to "unload" the module before redefining it again. In our example:
# 
#     require.undef('circles');
# 
#     define('circles', ['d3'], function (d3) {
#         // ...
#     });
# 
# Now you just have the re-execute the `Javascript(filename=..)` cell and the code will be reloaded, which is a lot more intuitive during development.

# ## All together now

# To conclude, all the bits together, in a more compact way.

# In[9]:


from IPython.display import display, Javascript, HTML
import json


# In[10]:


display(Javascript("require.config({paths: {d3: 'https://d3js.org/d3.v5.min'}});"))
display(Javascript(filename="circles.js"))
display(HTML(filename="circles.css.html"))


# In[11]:


def draw_circles(data, width=600, height=400):
    display(Javascript("""
        (function(element){
            require(['circles'], function(circles) {
                circles(element.get(0), %s, %d, %d);
            });
        })(element);
    """ % (json.dumps(data), width, height)))


# In[12]:


draw_circles([10, 60, 40, 5, 30, 10], width=500, height=200)


# ## References
# 
# - [Source of this notebook on GitHub](https://github.com/soxofaan/jupyter-playground/tree/master/jupyter-custom-d3-visualization)
# - [This notebook in NBViewer](https://nbviewer.jupyter.org/github/soxofaan/jupyter-playground/blob/master/jupyter-custom-d3-visualization/jupyter-custom-d3-visualization.ipynb)

# In[ ]:





# In[ ]: