#!/usr/bin/env python
# coding: utf-8

# # Information Retrieval Lab: Jupyter Tutorial

# ### (Re)sources:
# - [Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/index.html) _by Jake VanderPlas (Code released under the MIT License)_
# - [A Whirlwind Tour of Python](https://jakevdp.github.io/WhirlwindTourOfPython/) _by Jake VanderPlas released under the "No Rights Reserved" CC0 license (O’Reilly). Copyright 2016 O’Reilly Media, Inc., 978-1-491-96465-1_
# - [Python for Data Analysis](https://github.com/wesm/pydata-book) _by Wes McKinney (Code released under the MIT License)_
# - [Markdown Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) by Adam Pritchard
# 
# __Obligatory Wikipedia excerpts:__
# 
# 
# >_Project Jupyter's name is a reference to the three core programming languages supported by Jupyter, which are Julia, Python and R, and also a homage to Galileo's notebooks recording the discovery of the moons of Jupiter_
# 
# >_Jupyter is language agnostic and it supports execution environments (aka kernels) in several dozen languages among which are Julia, R, Haskell, Ruby, and of course Python (via the IPython kernel)._
# 
# >_A notebook interface (also called a computational notebook) is a virtual notebook environment used for literate programming._
# 
# >_Literate programming is a programming paradigm introduced by Donald Knuth in which a computer program is given an explanation of its logic in a natural language, such as English, interspersed with snippets of macros and traditional source code, from which compilable source code can be generated._
# 
# >_The literate programming paradigm, as conceived by Knuth, represents a move away from writing computer programs in the manner and order imposed by the computer, and instead enables programmers to develop programs in the order demanded by the logic and flow of their thoughts._

# ### Navigating the Notebook:
# >_The Jupyter Notebook has two different keyboard input modes. Edit mode allows you to type code or text into a cell and is indicated by a green cell border. Command mode binds the keyboard to notebook level commands and is indicated by a grey cell border with a blue left margin._
# 
# <kbd>Esc</kbd>: Move from edit mode to command mode<br>
# 
# <kbd>Enter</kbd>: Move from command mode to edit mode<br>
# 
# ⚠️ Note that there are two kinds of notebook cells. What are you are reading right now are the contents of a __Markdown Cell__. Later, we will enter python code into a __Code Cell__.<br>
# 
# 
# ⚠️ <kbd>H</kbd>: List all keyboard shortcuts<br>
# ⚠️ <kbd>P</kbd>: Open the searchable Command Palette<br>
# 
# #### Useful keyboard shortcuts in command mode:
# <kbd>d</kbd><kbd>d</kbd>: Delete highlighted cell<br>
# <kbd>Space</kbd>: Go to bottom<br>
# <kbd>Shift</kbd>+<kbd>Space</kbd>: Go to top<br>
# (<kbd>Shift</kbd>+)<kbd>L</kbd>: Toggle (all) line numbers<br>
# <kbd>M</kbd>: Convert cell to Markdown Cell<br>
# <kbd>Y</kbd>: Convert cell to Code Cell<br>
# <kbd>C</kbd>: Copy Cell<br>
# <kbd>P</kbd>: Paste Cell<br>
# <kbd>X</kbd>: Cut Cell<br>
# 
# 
# #### Useful keyboard shortcuts in edit mode:
# <kbd>Shift</kbd>+<kbd>Enter</kbd>: Execute highlighted cell then highlight the cell below it<br>
# <kbd>Ctrl</kbd>+<kbd>Enter</kbd>: Execute highlighted cell and stay there<br>
# <kbd>Alt</kbd>+<kbd>Enter</kbd>: Execute highlighted, create cell below and insert cursor into it<br>
# <kbd>Ctrl</kbd>+<kbd>]</kbd>: Indent selection<br>
# <kbd>Ctrl</kbd>+<kbd>[</kbd>: Dedent selection<br>
# <kbd>Ctrl</kbd>+<kbd>/</kbd>: Toggle comment ON/OFF (Both Markdown and Code)<br>
# <kbd>Shift</kbd>+<kbd>Tab</kbd>: Display function doctstring<br>

# ### Running Bash Commands:
# 
# In a code cell, prefixing a command with <kbd>!</kbd> is equivalent to running it in a terminal. Try running the following commands:

# In[ ]:


get_ipython().system('whoami')


# In[ ]:


get_ipython().system('pwd')


# In[ ]:


get_ipython().system('ls -lah')


# You can use python variables and pass those to your bash commands by using <kbd>$</kbd> or <kbd>{.}</kbd>

# In[ ]:


a = "/"
get_ipython().system('ls $a')
#!ls {a}


# You can even assign the returned output of a bash command to a python variable

# In[ ]:


a = "/"
b = get_ipython().getoutput('ls $a')
print(b)


# People usually use this to install packages into their environments using the python package manager pip:

# In[ ]:


get_ipython().system('pip install numpy')


# ### Notebook Magics:
# 
# Commands specific to the IPython kernel and prefixed with a <kbd>%</kbd> (line magic) or a <kbd>%%</kbd> (cell magic)
# 
# #### Line Magics:
# 
# Prefix a line with <kbd>%</kbd> to enable behavior that affects that line.
# 
# #### Cell Magics:
# 
# Prefix  with <kbd>%%</kbd> to enable behavior that affects the entire cell. (Has to be the first thing in the cell)
# 
# ⚠️ You can use the `lsmagic` magic to list all available magics:

# In[ ]:


get_ipython().run_line_magic('lsmagic', '')


# ⚠️ You can prepend any magic command (or in fact any python object) with a <kbd>?</kbd> to bring up information in the pager.

# In[ ]:


get_ipython().run_line_magic('pinfo', '%time')


# In[ ]:


get_ipython().run_line_magic('pinfo', '%timeit')


# In[ ]:


get_ipython().run_line_magic('pinfo', '%prun')


# In[ ]:


get_ipython().run_line_magic('pinfo', '%load_ext')


# ⚠️ You can [define your own magic commands](https://ipython.readthedocs.io/en/stable/config/custommagics.html) or install and enable additional ones:

# In[ ]:


get_ipython().system('pip install -U memory_profiler')


# In[ ]:


get_ipython().run_line_magic('load_ext', 'memory_profiler')


# In[ ]:


get_ipython().run_line_magic('pinfo', '%memit')


# In[ ]:


get_ipython().run_line_magic('pinfo', '%mprun')


# In[ ]:


get_ipython().run_line_magic('time', 'a=[i**2 for i in range(100)];')


# In[ ]:


get_ipython().run_line_magic('timeit', 'a=[i**2 for i in range(100)];')


# In[ ]:


get_ipython().run_cell_magic('time', '', 'a=[i**2 for i in range(1000)]\nb=[i**i for i in range(1000)]\n')


# In[ ]:


get_ipython().run_cell_magic('timeit', '', 'a=[i**2 for i in range(1000)]\nb=[i**i for i in range(1000)]\n')


# In[ ]:


get_ipython().run_line_magic('prun', 'a=[i**2 for i in range(10000000)]')


# In[ ]:


get_ipython().run_line_magic('memit', 'a=[i**2 for i in range(10000000)]')


# In[ ]:


get_ipython().run_line_magic('pinfo', '%%writefile')


# In[ ]:


get_ipython().run_cell_magic('writefile', 'memory_profiler_demo.py', 'def list_comp(N=1000):\n    a = [i**2 for i in range(N)] # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81,...]\n    b = [i**3 for i in range(N)] # [0, 1, 8, 27, 64, 125, 216, 343, 512, 729,...]\n    c = [(i+j) for i,j in zip(a,b)] # [0, 2, 12, 36, 80, 150, 252, 392, 576, 810, ...]\n    return c\n')


# In[ ]:


list_comp(215)


# In[ ]:


from memory_profiler_demo import list_comp
get_ipython().run_line_magic('mprun', '-f list_comp list_comp(15000)')


# In[ ]:


get_ipython().run_cell_magic('perl', '', '\n@list = (1,2,3,4,5);\nforeach $a (@list) {\n    print "$a\\n";\n}\n')


# In[ ]:


get_ipython().run_cell_magic('bash', '', 'for i in 1,2,3,4,5; do\n    echo $i\ndone\n')


# In[ ]:


get_ipython().run_cell_magic('svg', '', '<!-- Source: https://www.w3schools.com/graphics/tryit.asp?filename=trysvg_path2 -->        \n<svg height="400" width="450">\n<path id="lineAB" d="M 100 350 l 150 -300" stroke="red" stroke-width="3" fill="none" />\n<path id="lineBC" d="M 250 50 l 150 300" stroke="red" stroke-width="3" fill="none" />\n<path d="M 175 200 l 150 0" stroke="green" stroke-width="3" fill="none" />\n<path d="M 100 350 q 150 -300 300 0" stroke="blue" stroke-width="5" fill="none" />\n<!-- Mark relevant points -->\n<g stroke="black" stroke-width="3" fill="black">\n<circle id="pointA" cx="100" cy="350" r="3" />\n<circle id="pointB" cx="250" cy="50" r="3" />\n<circle id="pointC" cx="400" cy="350" r="3" />\n</g>\n<!-- Label the points -->\n<g font-size="30" font-family="sans-serif" fill="black" stroke="none" text-anchor="middle">\n<text x="100" y="350" dx="-30">A</text>\n<text x="250" y="50" dy="-10">B</text>\n<text x="400" y="350" dx="30">C</text>\n</g>\n</svg>\n')


# ### Notebook Aesthetics and Rich Media:
# 
# There are 23 classes in the [display module](https://ipython.readthedocs.io/en/stable/api/generated/IPython.display.html)
# 
# The `display` function is implicitly called on the last expression of a cell

# In[12]:


1
2
3


# In[15]:


display(3)


# In[14]:


1
2
3; # Use a ; to suppress that functionality


# In[21]:


[1 for thing in range(100)]; #Sometimes display does weird things, try removing the semicolon


# In[22]:


print([1 for thing in range(100)]) #Print works better in this case


# In[23]:


get_ipython().run_line_magic('pinfo', 'display')


# In[ ]:


#MAKE PAGE WIDER
from IPython.display import HTML
display(HTML("<style>.container { width:80% !important; }</style>"))


# #### Displaying Tables:
# 
# | A | B | C |
# | --- | --- | --- |
# | 123 | 456 | 789 |
# | 123 | 456 | 789 |
# | 123 | 456 | 789 |
# | 123 | 456 | 789 |
# 
# <br>
# <br>
# 
# | Header left      | Header center | Header right     |
# | :---        |    :----:   |          ---: |
# | This is left aligned      | centered       | This text is right aligned   |
# | left   | also centered        | And more      |

# #### Displaying equations and Math notation with MathJax:

# __MathJax Code:__
# ```
# \begin{equation*}
# \mathbf{V}_1 \times \mathbf{V}_2 =  \begin{vmatrix}
# \mathbf{i} & \mathbf{j} & \mathbf{k} \\
# \frac{\partial X}{\partial u} &  \frac{\partial Y}{\partial u} & 0 \\
# \frac{\partial X}{\partial v} &  \frac{\partial Y}{\partial v} & 0
# \end{vmatrix}
# \end{equation*}
# ```
# __Rendered Display:__
# 
# \begin{equation*}
# \mathbf{V}_1 \times \mathbf{V}_2 =  \begin{vmatrix}
# \mathbf{i} & \mathbf{j} & \mathbf{k} \\
# \frac{\partial X}{\partial u} &  \frac{\partial Y}{\partial u} & 0 \\
# \frac{\partial X}{\partial v} &  \frac{\partial Y}{\partial v} & 0
# \end{vmatrix}
# \end{equation*}
# 
# 
# 
# You can also feature your equations, such as the Cauchy-Schwarz Inequality, $\left( \sum_{k=1}^n a_k b_k \right)^2 \leq \left( \sum_{k=1}^n a_k^2 \right) \left( \sum_{k=1}^n b_k^2 \right)$ inline as part of a sentence by enclosing the expression in <kbd>$</kbd>.

# #### Display an algorithm using Markdown and inline MathJax:
# source: https://ai.meta.stackexchange.com/questions/1679/writing-algorithm-formulas-using-mathjax
# 
# > Algorithm parameters: step size  $\alpha \in (0 , 1] , \epsilon > 0$   
# Initialize  $Q  ( s, a ), \  \forall s \in S^+ , a \in A ( s ),$ arbitrarily except that $Q ( terminal , \cdot ) = 0$    
# >
# > Loop for each episode:  
# $\quad$Initialize $S$   
# $\quad$Loop  for  each  step  of  episode:    
# $\qquad$Choose  $A$ from $S$ using some policy derived from $Q$ (eg $\epsilon$-greedy)   
# $\qquad$Take action $A$, observe $R, S'$   |
# $\qquad Q(S,A) \leftarrow Q(S,A) + \alpha[R+\gamma \max_a(S', a) - Q(S, A)]$   
# $\qquad S \leftarrow S'$    
# $\quad$ until $S$ is terminal

# #### Play Sound:

# In[ ]:


from IPython.display import Audio
import numpy as np
framerate = 44100
t = np.linspace(0,5,framerate*5)
dataleft = np.sin(2*np.pi*220*t)
dataright = np.sin(2*np.pi*224*t)
Audio([dataleft, dataright],rate=framerate)


# #### Embed a Youtube Video:

# In[ ]:


from IPython.display import YouTubeVideo
YouTubeVideo('2eCHD6f_phE')


# #### Embed a Photo:
# ![](https://scikit-learn.org/stable/_static/ml_map.png)

# #### Embed an arbitrary IFrame:

# In[ ]:


from IPython.display import IFrame
IFrame('https://pandas.pydata.org/Pandas_Cheat_Sheet.pdf', width=700, height=600)