While Matplotlib is powerful, it requires a steep learning curve. Moreover, I found it often requires more lines of code for the type of visualisations I wish to produce than various *.js libriaries. Ever since I found out about IPython Notebook and d3js I wanted to combine the two and replace Matplotlib in my workflow. IPython offers a great interactive way of using Python. D3js coupled with developer tools in Chrome/Firefox does the same to graphics.
Due to security concerns there is currently no easy way to execute Javascript in a cell. With this script you can create the d3 code from within (I)Python but render them later on with PhantomJs. Becasue of that one can easily switch between producing interactive SVG visualisations by just copying the resulting HTML code from the rendered script file, or saving a static version in PNG, PDF, etc.
The main ingredients are coded as a class in Python
Dependencies:
PATH
)titlecase
from ipyD3 import *
from IPython.display import display
d3 = d3object()
All arguments are kwargs, and you can set:
height
of your objectwidth
of your objecttopHtml
)bottomHtml
)style
that already includes topHtml
, botttomHtml
, and csspublish
)precision
)phantomExec
)keepTempDir
)d3
objects if you wish to create a multi-page reportYou start by initializing the object.
import numpy as np
d3 = d3object(width=800,
height=200,
style='JFFigure',
number=1,
d3=None,
precision=100,
title='Example figure with d3js',
desc='Standrad normal distribution')
Variables are passed through addVar
function. It will convert the basic types to Javascript
.
d3.addVar(
interpolation1='basis',
interpolation2='step-before',
resolution=80,
domainRange=[-4,4],
imposeMax=0
)
You can add your own Javascript
and CSS
. Note, that all variables specified with addVar
are pasted at the beginning of the script, so if you need to change a variable's value within the script you need to use addJs
.
d3.addJs('''
var svg = d3.select("#"+d3ObjId)
.append("svg")
.attr("width", width)
.attr("height", height)
.append("g").attr("id", "#"+d3ObjId+'InnerG')
// A formatter for counts.
var formatCount = d3.format(",.0f");
var margin = {top: 10, right: 30, bottom: 30, left: 30},
width = width - margin.left - margin.right,
height = height - margin.top - margin.bottom;
var x = d3.scale.linear()
//.domain(domainRange)
.range([0, width]);
// Generate a histogram using twenty uniformly-spaced bins.
var xAxis = d3.svg.axis()
.ticks(4)
.scale(x)
.orient("bottom");
function appendHistogram(series, domainSeries, height, width, x_offset, y_offset, interpolation){
var svg2 = svg.append("g")
y_offset=Math.round(y_offset)
height=Math.round(height)
height+=y_offset
width+=x_offset
x.range([x_offset, width])
.domain(domainSeries)
var data = d3.layout.histogram()
.bins(x.ticks(resolution))
.frequency(0)
(series);
var seriesMax=d3.max([imposeMax, d3.max(data, function(d) { return d.y; })]);
var y = d3.scale.linear()
//.domain([0, d3.max(data, function(d) { return d.y; })])
.domain([0, seriesMax])
.range([height, y_offset]);
var area = d3.svg.area()
.interpolate(interpolation)
.x(function(d) {
if(interpolation=="step-before")
return x(d.x+d.dx/2)
return x(d.x);
})
.y0(height)
.y1(function(d) { return d3.max([y(d.y),y(seriesMax)]); });
svg2.append("path")
.datum(data)
.attr("class", "area")
.attr("d", area);
svg2.append("g")
.attr("class", "axis")
.attr("transform", "translate(0," + height + ")")
.call(xAxis);
}
appendHistogram(data, domainRange, height, width/2-15, 10, 0, interpolation1)
appendHistogram(data, domainRange, height, width/2-15, width/2+20, 0, interpolation2)
''')
d3.addCss('''
.polyline{
stroke: #000;
shape-rendering: crispEdges;
}
.area{
shape-rendering: geometricPrecision;
stroke: #000 !important;
stroke-width:1 !important;
fill: #ddd !important;
}
.axis path, .axis line {
fill: none;
stroke: #000;
stroke-width: 1px;
color-rendering: optimizeQuality !important;
shape-rendering: crispEdges !important;
text-rendering: geometricPrecision !important;
}
''')
E.g. dsitribution of standard normal. D3js allows for quick swtiching between histograms and approx. shape of the distribution.
With too few observations:
d3.addVar(data=np.random.randn(1000))
html=d3.render(mode=('show','html'))
display(html)
And with just enough:
d3.addVar(data=np.random.randn(300000))
html=d3.render(mode=('show','html'))
display(html)
While displaying the 'rendered' svg
(which is what ('show', 'html')
does) is handy. Some prefer to get a png
, although it tends to be of lower quality.
d3.addVar(data=np.random.randn(300000))
png=d3.render(mode=('show','png'))
display(png)
And then you can easily create other grpahics like Violin plots (see http://bl.ocks.org/z-m-k/5014368 or https://gist.github.com/z-m-k/5014368 for code)
html=d3.render(mode=('show','html'))
display(html)
You can also save the output to a file by adding file
to mode
and providing a path with fileName
in render
.
Furthermore, if the script requires more time to fully render you can change renderTime
(defaults to 1000ms).
d3.render(mode=['html', 'file'], renderTime=10000, fileName='PATH/TO/THE/FILE')
I often use this class to produce tables. Of course you could use ipy_table
too. The main difference between the two approaches is that ipy_table
produces tabls in HTML while ipyD3 outputs SVG. In the end, ipyD3 allows for more customization (see below).
d3 = d3object(width=800,
height=400,
style='JFTable',
number=1,
d3=None,
title='Example table with d3js',
desc='An example table created created with d3js with data generated with Python.')
data=[[1277.0, 654.0, 288.0, 1976.0, 3281.0, 3089.0, 10336.0, 4650.0, 4441.0, 4670.0, 944.0, 110.0],
[1318.0, 664.0, 418.0, 1952.0, 3581.0, 4574.0, 11457.0, 6139.0, 7078.0, 6561.0, 2354.0, 710.0],
[1783.0, 774.0, 564.0, 1470.0, 3571.0, 3103.0, 9392.0, 5532.0, 5661.0, 4991.0, 2032.0, 680.0],
[1301.0, 604.0, 286.0, 2152.0, 3282.0, 3369.0, 10490.0, 5406.0, 4727.0, 3428.0, 1559.0, 620.0],
[1537.0, 1714.0, 724.0, 4824.0, 5551.0, 8096.0, 16589.0, 13650.0, 9552.0, 13709.0, 2460.0, 720.0],
[5691.0, 2995.0, 1680.0, 11741.0, 16232.0, 14731.0, 43522.0, 32794.0, 26634.0, 31400.0, 7350.0, 3010.0],
[1650.0, 2096.0, 60.0, 50.0, 1180.0, 5602.0, 15728.0, 6874.0, 5115.0, 3510.0, 1390.0, 170.0],
[72.0, 60.0, 60.0, 10.0, 120.0, 172.0, 1092.0, 675.0, 408.0, 360.0, 156.0, 100.0]]
data=[list(i) for i in zip(*data)]
sRows=[['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'Deecember']]
sColumns=[['Prod {0}'.format(i) for i in xrange(1,9)],
[None, '', None, None, 'Group 1', None, None, 'Group 2']]
d3.addSimpleTable( data,
fontSizeCells=[12,],
sRows=sRows,
sColumns=sColumns,
sRowsMargins=[5,50,0],
sColsMargins=[5,20,10],
spacing=0,
addBorders=1,
addOutsideBorders=-1,
rectWidth=45,
rectHeight=0
)
html=d3.render(mode=['html', 'show'])
display(html)
For a slightly cleaner look you can remove the borders...
d3 = d3object(width=800,
height=400,
style='JFTable',
number=1,
d3=None,
title='Example table with d3js',
desc='An example table created created with d3js with data generated with Python.')
d3.addSimpleTable( data,
fontSizeCells=[12,],
sRows=sRows,
sColumns=sColumns,
sRowsMargins=[5,50,0],
sColsMargins=[5,20,10],
spacing=2,
addBorders=0,
addOutsideBorders=-1,
rectWidth=45,
rectHeight=0
)
html=d3.render(mode=['html', 'show'])
display(html)
But since it is HTML you can easily increase readability of the table!! With addTable instead of addSimpleTable. In fact addSimpleTable is just a reference to addTable
d3 = d3object(width=800,
height=400,
style='JFTable',
number=1,
d3=None,
title='Example table with d3js',
desc='An example table created created with d3js with data generated with Python.')
d3.addTable(data,
dataAdd=[],
pVals=-1,
fontSizeCells=[11,5],
fontSizeCellsLabels=[11,5],
sRows=sRows,
sColumns=sColumns,
sRowsMargins=[5,50,0],
sColsMargins=[5,20,10],
fontSizeHeaders=11,
shrinkHeadersBorders=1.5,
heatmapIgnoreText=1,
varLabels=[],
heatmap={
'draw':1,
'spacing':2,
'fillProportion':21,
'addText':1,
'addTextRows':1,
'addBorders':0,
'addOutsideBorders':-1,
'rectWidth':45,
'rectHeight':21,
},
smallHeatmap={
'draw':0,
'spacing':0,
'fillProportion':5,
'addText':0,
'addTextRows':0,
'addBorders':1,
'addOutsideBorders':-1,
'rectWidth':5,
'rectHeight':5,
},
heatmapLegendVert=1,
legend= {
'draw':0,
'width':2,
'height':15,
'rectWidth':60,
'rectHeight':60
},
rightPaneOffset=0,
colorDomainAuto=1,
#colorDomainAutoIgnoreRows=[5],
colorRange=['#ffffff', '#dddddd', '#cccccc', '#bbbbbb', '#aaaaaa', '#999999']
)
d3 = d3object(width=800,
height=400,
style='JFTable',
number=1,
d3=d3,
keepTempDir='c:\sandbox',
title='Example table with d3js with a small heatmap',
desc='An example table created created with d3js with data generated with Python.')
d3.addTable(data,
dataAdd=[],
pVals=-1,
fontSizeCells=[11,5],
fontSizeCellsLabels=[11,5],
sRows=sRows,
sColumns=sColumns,
sRowsMargins=[5,50,0],
sColsMargins=[5,20,10],
fontSizeHeaders=11,
shrinkHeadersBorders=1.5,
heatmapIgnoreText=1,
varLabels=['Sales'],
heatmap={
'draw':1,
'spacing':2,
'fillProportion':0,
'addText':1,
'addTextRows':1,
'addBorders':0,
'addOutsideBorders':-1,
'rectWidth':45,
'rectHeight':21,
},
smallHeatmap={
'draw':1,
'spacing':0,
'fillProportion':10,
'addText':0,
'addTextRows':0,
'addBorders':0,
'addOutsideBorders':0,
'rectWidth':10,
'rectHeight':10,
},
heatmapLegendVert=1,
legend= {
'draw':1,
'width':2,
'height':15,
'rectWidth':45,
'rectHeight':30
},
rightPaneOffset=100,
colorDomainAuto=1,
colorRange=['#ffffff', '#eeeeee', '#aaaaaa', '#111111', '#000000']
)
html=d3.render(mode=['html', 'show'])
display(html)
I often work with output from statistical packages and including more information in a cell also comes at little to no cost.
d3 = d3object(width=800,
height=400,
style='JFTable',
number=1,
d3=None,
title='Example table with d3js',
desc='An example table created created with d3js with data generated with Python.')
sRows=[['Long/Short Eq. Hedge',
'FoF',
'Not L/S nor FoF',
'All',
'Not FoFs',
]]
sColumns=[
['N',
'Initial',
'Average',
'Incentive',
'Management',
'Median',
'Mean [%]',
'Std. dev.',
'Skewness',
'Kurtosis'
],
[ '',
None,
'AUM [M, USD]',
None,
'Fees [%]',
None,
None,
None,
None,
'Returns'
]
]
data=[[3452, 3, 22, 20.0, 1.5, 0.666, 0.656, 3.538, -0.1, 1.0],
[5791, 7, 22, 10.0, 1.5, 0.502, 0.27, 1.17, -0.9, 1.8],
[6718, 4, 23, 20.0, 1.5, 0.7, 0.637, 1.448, -0.2, 1.2],
[15961,5,23, 20.0, 1.5, 0.6, 0.467, 1.467, -0.4, 1.3],
[10170,4, 23, 20.0, 1.5, 0.69, 0.646, 2.633, -0.1, 1.1],
]
dataAdd=[['', '', '', '', '', '', 0.015, '', '', 0.084],
['', '', '', '', '', '', 0.016, '', '', 0.015],
['', '', '', '', '', '', 0.024, '', '', 0.162],
['', '', '', '', '', '', 0.058, '', '', 0.126],
['', '', '', '', '', '', 0.093, '', '', 0.149],
]
dataAdd=[dataAdd,[[i for i in j] for j in dataAdd]]
d3.precision=2
data=d3.convertVar(data)
d3.precision=4
d3.addSimpleTable( data,
dataAdd=dataAdd,
fontSizeCells=[12,10,10],
pVals=0,
sRows=sRows,
sColumns=sColumns,
sRowsMargins=[5,100,0],
sColsMargins=[5,20,10],
spacing=2,
addBorders=0,
addOutsideBorders=-1,
rectWidth=45,
rectHeight=0
)
html=d3.render(mode=('show','html'))
display(html)