The basic Neural Network has an INPUT, a HIDDEN, and an OUTPUT layer as shown in the figure above.
They are multiplied by *weights* and then a *bias* gets added to that calculation. The result of this calculation will then be your *OUTPUT. So for example, if you had a model that predicts home prices based on the size of your home. Your INPUT* will be the square footage of your house. Your model then will multiply this number by a Weight and then add it to a bias. The result of this calculation will be the home price predicted by your model. Pretty easy so far.
You need to train your model in order for it to learn. Below are the ingredients you need to have in place to teach.
Let's keep going with the home price example mentioned above. Suppose you gathered historical data from Zillow on home prices. You have two columns, one tells you the size of a house in square footage, and the other column is the price the home last sold for. This is what we call your *training data*. This is the data we are going to use to train our model with.
After the model makes a prediction as shown above, we need a way to determine how accurate the prediction was. This is where our *loss* function comes into play. It will tell the model that it was wrong and it needs to correct itself.
So how exactly are the Weights and Bias numbers calculated?
These numbers are initialized with some random values. After the first training iteration, the loss function will tell the model how far off it is from the correct output. The model will then adjust the values for *W* (weight) and *b* (bias) so that its prediction on iteration 2 is better than on iteration 1. This is also called *backpropagation*.
How does the model determine how much to adjust the W and b?
In the previous lessons, we have been using the Gradient Descent method. It is the Learning Rate that will determine how much of an adjustment to make on W and b. The larger the LR, the bigger the adjustment per iteration.
So after X number of iterations, your model has hopefully adjusted its Weights and bias numbers so that that the difference between its predicted values are very close to the actual values. After you have tuned your model, it is then ready to accept new data that is has *NOT* seeing before and give you an acurate prediction. Hope this makes sense.
I'm about to tie together the terms and ideas from above with the code you have already written. This will hopefully make something click in your head.
# let's generate 100 random samples
test_x = np.random.rand(100).astype(np.float32)
# let's compute y_data using 0.1 for m and 0.3 for b
test_y = 0.1 * data_x + 0.3
This is the "magic" that was being calculated in our output layer. Note that the model you created in lesson 4 did *NOT* have a hidden later.
# Model your graph
W = tf.Variable(np.random.rand())
b = tf.Variable(np.random.rand())
pred = tf.mul(W, x) + b
This is were all the backpropagation occured and were your model started to learn
# How good or bad is our model doing?
loss = tf.reduce_mean(tf.square(pred - y))
# Pick your Learning Rate
optimizer = tf.train.GradientDescentOptimizer(0.07)
# Pick the best W and b values so that we get the smallest prediction error
train = optimizer.minimize(loss)
This concludes the lesson for today. There is a whole lot more to learn but hopefully this lesson gave you a good head start.
This tutorial was created by HEDARO