#!/usr/bin/env python
# coding: utf-8

# # Exercise 3.3 - Solution
# ## Checkerboard
# 
# Open the Tensorflow Playground (www.playground.tensorflow.org) and select on the left the checkerboard pattern as the data basis.
# 
# The data is taken from a two-dimensional probability distribution and is represented by the value pairs $x_1$ and $x_2$. The regions $x1$, $x_2 > 0$ and $x_1$, $x_2 < 0$ are shown by one color. For value pairs with $x_1 > 0$, $x_2 < 0$ and $x_1 < 0$, $x_2 > 0$, the regions are indicated by a different color.  
# 
# In features, select the two independent variables $x_1$ and $x_2$ and start the network training. The network learns that $x_1$ and $x_2$ are for these data not independent variables, but are taken from the probability distribution of the checkerboard pattern.
# 
# [![Checkerboard](./images/checkerboard_tf_playground.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=4,2&seed=0.20784&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)
# 
# ## Tasks
# 1.  Try various settings for the number of layers and neurons using `ReLU` as activation function. What is the smallest network that gives a good fit result?
# 2.  What do you observe when training networks with the same settings multiple times? Explain your observations.
# 3.  Try additional input features: Which one is most helpful?
# 

# ## Solutions
# Hint: click on the images to open the correct playground settings needed to solve the task, respectively.

# ### Task 1
# Try various settings for the number of layers and neurons using ReLU as activation function. What is the smallest network that gives a good fit result?
# 

# [![Checkerboard](./images/checkerboard_tf_playground.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=3&seed=0.10528&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)
# 
# <em> A network with a single layer holding 3 nodes. However, this configuration is not stable.
# 
# A network with a single layer, holding 4 nodes, is way more stable. </em>

# ### Task 2
# What do you observe when training networks with the same settings multiple times? Explain your observations.

# <em> Due to the random initialization of weights, the network training always develops a little bit differently, leading to different results. </em>

# ### Task 3
# Try additional input features: Which one is most helpful?

# [![Checkerboard](./images/checkerboard_3_2_task_3.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=&seed=0.10528&showTestData=false&discretize=false&percTrainData=50&x=false&y=false&xTimesY=true&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)
# 
# <em> Obviously, the $x_1\cdot x_2$ feature is most helpful. </em>