#!/usr/bin/env python
# coding: utf-8

# # Probability Distributions

# ## Resources
# 
# - [Second Chapter of Yoshua Bengio's Deep Learning book](http://www.deeplearningbook.org/contents/prob.html)
# - [QA Site](http://stats.stackexchange.com/)

# ![boxes](images/boxes.png)

# *"Patterm Recognition and Machine Learning", Bishop, 2006*

# Random Variables: **_Boxes_**, and **_fruits_**.
# 
# As we draw random elements we can gain an insight on the population in general.

# Probability of choosing the red box.
# 
# $p(box = red) = \frac{\# Red boxes}{\# boxes} =  \frac{1}{2}$
# 
# $p(box = blue) = 1 - p(box = red)$

# Probability of choosing an apple in the red box
# 
# $p(fruit = apple | box = red) = \frac{2}{8}$
# 
# Probability of choosing an apple in the blue box
# 
# $p(fruit = apple | box = blue) = \frac{3}{4}$
# 
# Probability of choosing an orange in the red box
# 
# $p(fruit = orange | box = red) = \frac{6}{8}$
# 
# Probability of choosing an apple in the blue box
# 
# $p(fruit = orange | box = blue) = \frac{1}{4}$

# ## Some generalization
# 
# $p(fruit = apple, box = red) = p(fruit = apple | box = red)p(box = red)$ This is probability product rule.
# 
# $p(fruit = orange, box = red) = p(fruit = orange | box = red)p(box = red)$ 
# 
# $p(fruit = apple, box = blue) = p(fruit = apple | box = blue)p(box = blue)$ This is probability product rule.
# 
# $p(fruit = orange, box = blue) = p(fruit = orange | box = blue)p(box = blue)$ 
# 
# $p(fruit = orange) = p(fruit = orange, box = blue) + p(fruit = orange, box = red) $ This is probability sum rule
# 
# $p(fruit = apple) = p(fruit = apple, box = blue) + p(fruit = apple, box = red) $ This is probability sum rule
# 
# ## The golden rules of probability
# 
# $p(x) = \int p(x,y) dy$
# 
# $p(x,y) = p(x|y)p(y)$
# 
# $p(x,y) = p(x)p(y)$ If event x and y are unrelated (i.e. drawing a fruit is independent from the box)
# 
# 
# ## Bayes rule
# 
# $p(x|y) = \frac{p(y|x)p(x)}{p(y)}$
# 
# 

# For the moment this is it for probability distributions, we will revisit in further chapters.