#!/usr/bin/env python
# coding: utf-8

# # Make Jupyter even more magical
# 
# ## (with cell magic extensions!)
# 
# ### by Nicolas Kruchten, from Datacratic

# <img src="http://nicolas.kruchten.com/datacratic.png" width="200" />
# 
# * Datacratic is Democratizing Machine Learning through the Machine Learning Database (http://mldb.ai/)
# * Based in Montreal, founded in 2010, venture-funded, 30 employees
# * Jupyter is shipped as part of MLDB as the main interface

# <br/><br/><br/><br/><br/>

# ## Intro to Jupyter
# 
# First things first, you can run Python code in Jupyter.

# In[29]:


print "Hello, world!"


# Objects and functions stick around from cell to cell: I can define a function here...

# In[30]:


def do_stuff():
    print "Hello again!"


# ...and call it there.

# In[31]:


do_stuff()


# Inspecting objects is easy too.

# In[4]:


import requests
get_ipython().run_line_magic('pinfo', 'requests')


# ## Markdown cells
# 
# This is a Markdown cell.
# 
# * this is a list item
# * and another

# <br/><br/><br/><br/><br/>

# ## Time for some Magic!
# 
# Lines and cells that start with `%` are not interpreted like normal Python code: they are magical. Let's try a simple one first which will show us what magic is available.

# In[32]:


get_ipython().run_line_magic('lsmagic', '')


# In[33]:


get_ipython().run_line_magic('ls', '')


# In[34]:


get_ipython().run_line_magic('time', 'print "Do you have the time?"')


# In[35]:


get_ipython().run_cell_magic('timeit', '', 'x = 0\nfor i in xrange(10): x += i\n')


# In[36]:


get_ipython().run_cell_magic('bash', '', '\ncurl http://localhost:5000/get\n')


# ## First big idea of this talk: Jupyter cells are not just for Python!

# <br/><br/><br/><br/><br/>

# ## Third-party Magic
# 
# Those were all built-in magics, but we can install and load third-party magic extensions too.

# In[37]:


get_ipython().run_line_magic('load_ext', 'sql')


# In[38]:


get_ipython().run_line_magic('sql', 'sqlite://')


# In[39]:


get_ipython().run_cell_magic('sql', '', '\nDROP TABLE IF EXISTS hockey;\nCREATE TABLE hockey ("Team", "Stanley Cups Won", "Country");\nINSERT INTO hockey VALUES ("Montreal Canadiens", 24, "Canada");\nINSERT INTO hockey VALUES ("Detroit Red Wings", 11, "USA");\nINSERT INTO hockey VALUES ("Boston Bruins", 6, "USA");\nINSERT INTO hockey VALUES ("Chicago Blackhawks", 11, "USA");\nINSERT INTO hockey VALUES ("Toronto Maple Leafs", 13, "Canada");\n')


# In[40]:


get_ipython().run_line_magic('sql', 'SELECT * FROM hockey ORDER BY "Stanley Cups Won" DESC LIMIT 3')


# Whoa, what happened there? That's not text, that's HTML!
# 
# The answer is that when displaying an object, Jupyter tries to get an HTML representation of that object from a method called `_repr_html_` if it exists. Any object with such a method gets displayed this way.

# In[41]:


class Thing():
    def _repr_html_(self):
        return """<h3 style="color: blue; text-align: center;">I am a blue thing</h3><br/>"""

thing = Thing()
thing


# ## Second big idea of this talk: `_repr_html_()` to control Notebook output

# <br/><br/><br/><br/><br/>

# ## Going visual with Matplotlib
# 
# The `matplotlib` cell magic lets us tell Jupyter we want to see charts inline with the notebook.

# In[42]:


get_ipython().run_line_magic('matplotlib', 'inline')


# We can also capture the output of a cell magic and assign it to a variable, like so.

# In[43]:


data = get_ipython().run_line_magic('sql', "SELECT * FROM hockey WHERE country='Canada'")

data.pie()
None


# In[44]:


import seaborn as sns
sns.set()

df = sns.load_dataset("iris")
sns.pairplot(df, hue="species")


# ## Interactive Data Exploration with PivotTable.js
# 
# (Shameless plug: I am the author of PivotTable.js and the `pivottablejs` Python module)

# In[45]:


import pandas as pd
df = pd.read_csv("mps.csv")
df.head()


# In[46]:


df.pivot_table(index="Party", columns="Province", aggfunc=len, values="Name")


# In[47]:


from pivottablejs import pivot_ui
pivot_ui(df)


# ## Making diagrams with Graphviz

# In[48]:


get_ipython().run_line_magic('load_ext', 'hierarchymagic')


# In[49]:


get_ipython().run_cell_magic('dot', '', '\ndigraph G { \n    rankdir=LR;\n    hello -> world;\n    goodbye -> cruel -> world;\n    world -> of -> warcraft;\n}\n')


# ## Interactive forms

# In[50]:


from ipywidgets import interact

@interact
def echo(input="Hello, world!", times=[1,5,1]):
    return (input+" ")*times


# In[51]:


import matplotlib.pyplot as plt

@interact
def polynomial(split=[0,100]):
    plt.pie([split/100.0, 1.0-split/100.0], labels=["Montreal Canadiens", "Toronto Maple Leafs"])
    plt.show()


# <br/><br/><br/><br/><br/>

# ## So let's make our own extension!
# 
# Here I'll switch to Jupyter's built-in text editor. Oh yeah, Jupyter has a built-in text editor! And a web-based terminal. (Yes, this is a giant security hole :)

# In[52]:


get_ipython().run_line_magic('load_ext', 'sample_ext')


# In[53]:


get_ipython().run_line_magic('http', 'blah blah')


# <br/><br/><br/><br/><br/>

# ## One more thing...
# 
# Any notebook can be served up as a slide deck!
# 
# `jupyter nbconvert jupyter_magic.ipynb --to slides --post serve`
# 
# Notebooks can also be shared really easily, for example this one is available at
# 
# ## http://github.com/nicolaskruchten/pyconca
# 
# <br/><br/><br/><br/><br/>
# 
# ![Contact](http://nicolas.kruchten.com/contact.png)

# In[ ]: