Deadline: May 15, 9pm.
Late Penalty: There is a penalty-free grace period of one hour past the deadline. Any work that is submitted between 1 hour and 24 hours past the deadline will receive a 20% grade deduction. No other late work is accepted. Quercus submission time will be used, not your local computer time. You can submit your labs as many times as you want before the deadline, so please submit often and early.
TAs: Andrew Jung
This lab is partially based on an assignment developed by Prof. Jonathan Rose and Harris Chan.
Welcome to the first lab of APS360! This lab is a warm up to get you used to the programming environment used in the course, and also to help you review and renew your knowledge of Python and relevant Python libraries. The lab must be done individually. Please recall that the University of Toronto plagarism rules apply.
By the end of this lab, you should be able to:
numpy
.matplotlib
.You will need to use numpy and PyTorch documentations for this assignment:
You can also reference Python API documentations freely.
Submit a PDF file containing all your code, outputs, and write-up from parts 1-5. You can produce a PDF of your Google Colab file by going to File > Print and then save as PDF. The Colab instructions has more information.
Do not submit any other files produced by your code.
Include a link to your colab file in your submission.
Please use Google Colab to complete this assignment. If you want to use Jupyter Notebook, please complete the assignment and upload your Jupyter Notebook file to Google Colab for submission.
With Colab, you can export a PDF file using the menu option
File -> Print
and save as PDF file.
Please refer to Colab instructions https://colab.research.google.com/drive/1YKHHLSlG-B9Ez2-zf-YFxXTVgfC_Aqtt
If you want to use Jupyter Notebook locally, please refer to https://www.cs.toronto.edu/~lczhang/aps360_20191/files/install.pdf
The purpose of this section is to get you used to the basics of Python, including working with functions, numbers, lists, and strings.
Note that we will be checking your code for clarity and efficiency.
If you have trouble with this part of the assignment, please review http://cs231n.github.io/python-numpy-tutorial/
Write a function sum_of_cubes
that computes the sum of cubes up to n
. If the input to sum_of_cubes
invalid (e.g. negative or non-integer n
), the function should print out "Invalid input"
and return -1
.
def sum_of_cubes(n):
"""Return the sum (1^3 + 2^3 + 3^3 + ... + n^3)
Precondition: n > 0, type(n) == int
>>> sum_of_cubes(3)
36
>>> sum_of_cubes(1)
1
"""
Write a function word_lengths
that takes a sentence (string), computes the length of each word in that sentence, and returns the length of each word in a list. You can
assume that words are always separated by a space character " "
.
Hint: recall the str.split
function in Python.
If you arenot sure how this function works, try
typing help(str.split)
into a Python shell, or check out https://docs.python.org/3.6/library/stdtypes.html#str.split
help(str.split)
def word_lengths(sentence):
"""Return a list containing the length of each word in
sentence.
>>> word_lengths("welcome to APS360!")
[7, 2, 7]
>>> word_lengths("machine learning is so cool")
[7, 8, 2, 2, 4]
"""
Write a function all_same_length
that takes a sentence (string),
and checks whether every word in the string is the same length.
You should call the function word_lengths
in the body
of this new function.
def all_same_length(sentence):
"""Return True if every word in sentence has the same
length, and False otherwise.
>>> all_same_length("all same length")
False
>>> word_lengths("hello world")
True
"""
In this part of the assignment, you'll be manipulating arrays
usign NumPy. Normally, we use the shorter name np
to represent
the package numpy
.
import numpy as np
The below variables matrix
and vector
are numpy arrays. Explain what you think <NumpyArray>.size
and <NumpyArray>.shape
represent.
matrix = np.array([[1., 2., 3., 0.5],
[4., 5., 0., 0.],
[-1., -2., 1., 1.]])
vector = np.array([2., 0., 1., -2.])
matrix.size
matrix.shape
vector.size
vector.shape
Perform matrix multiplication output = matrix x vector
by using
for loops to iterate through the columns and rows.
Do not use any builtin NumPy functions.
Cast your output into a NumPy array, if it isn't one already.
Hint: be mindful of the dimension of output
output = None
output
Perform matrix multiplication output2 = matrix x vector
by using
the function numpy.dot
.
We will never actually write code as in
part(c), not only because numpy.dot
is more concise and easier to read/write, but also performance-wise numpy.dot
is much faster (it is written in C and highly optimized).
In general, we will avoid for loops in our code.
output2 = None
output2
As a way to test for consistency, show that the two outputs match.
Show that using np.dot
is faster than using your code from part (c).
You may find the below code snippit helpful:
import time
# record the time before running code
start_time = time.time()
# place code to run here
for i in range(10000):
99*99
# record the time after the code is run
end_time = time.time()
# compute the difference
diff = end_time - start_time
diff
A callable object is any object that can be called like a function.
In Python, any object whose class has a __call__
method will be callable.
For example, we can define an AddBias
class that is initialized with a value val
. When the object of the Adder class is called with input
, it will return the sum of val
and input
:
class AddBias(object): # this is a new class AddBias, which inherits from the class `object`
def __init__(self, val): # this is the object constructor
self.val = val
def __call__(self, input):
return self.val + input # `self` is like `this` in many languages
add4 = AddBias(4)
add4(3)
# AddBias works with numpy arrays as well
add1 = AddBias(1)
add1(np.array([3,4,5]))
Create a callable object class ElementwiseMultiply
that is initialized with weight
, which is a numpy array (with 1-dimension).
When called on input
of the same shape as weight
, the object will output an elementwise product of input
and weight
. For example, the 1st element in the output will be a product of the first element of input
and first element of weight
. If the input
and weight
have different shape, do not return anything.
Create a callable object class LeakyRelu
that is initialized
with alpha
, which is a scalar value.
When called on input x
, which may be a NumPy array,
the object will output:
For example,
>>> leaky_relu = LeakyRelu(0.1)
>>> leaky_relu(1)
1
>>> leaky_relu(-1)
-0.1
>>> x = np.array([1, -1])
>>> leaky_relu(x)
np.array([1, -0.1])
To obtain full marks, do not use any for-loops to implement this class.
Create a callable object class Compose
that is initialized with layers
, which is a list of callable objects each taking in one argument when called. For example, layers
can be something like [add1, add4]
that we created above. Each add1
and add4
take in one argument. When Compose
object is called on a single argument, the object will output a composition of object calls in layers
, in the order given in layers
(e.g. add1
will be called first and then add4
will be called after using the result from add1
call)
Run the below code and include the output in your report.
weight_1 = np.array([1, 2, 3, 4.])
weight_2 = np.array([-1, -2, -3, -4.])
bias_1 = 3.0
bias_2 = -2.0
alpha = 0.1
elem_mult_1 = ElementwiseMultiply(weight_1)
add_bias_1 = AddBias(bias_1)
leaky_relu = LeakyRelu(alpha)
elem_mult_2 = ElementwiseMultiply(weight_2)
add_bias_2 = AddBias(bias_2)
layers = Compose([elem_mult_1,
add_bias_1,
leaky_relu,
elem_mult_2,
add_bias_2,
leaky_relu])
input = np.array([10, 5, -5, -10.])
print("Input: ", input)
output = layers(input)
print("Output:", output)
A picture or image can be represented as a NumPy array of “pixels”, with dimensions H × W × C, where H is the height of the image, W is the width of the image, and C is the number of colour channels. Typically we will use an image with channels that give the the Red, Green, and Blue “level” of each pixel, which is referred to with the short form RGB.
You will write Python code to load an image, and perform several array manipulations to the image and visualize their effects.
import matplotlib.pyplot as plt
This is a photograph of a dog whose name is Mochi.
Load the image from its url (https://drive.google.com/uc?export=view&id=1oaLVR2hr1_qzpKQ47i9rVUIklwbDcews) into the variable img
using the plt.imread
function.
Hint: You can enter the URL directly into the plt.imread
function as a Python string.
img = None
Use the function plt.imshow
to visualize img
.
This function will also show the coordinate system used to identify pixels. The origin is at the top left corner, and the first dimension indicates the Y (row) direction, and the second dimension indicates the X (column) dimension.
Modify the image by adding a constant value of 0.25 to each pixel in the img
and
store the result in the variable img_add
. Note that, since the range for the pixels
needs to be between [0, 1], you will also need to clip img_add to be in the range [0, 1]
using numpy.clip
. Clipping sets any value that is outside of the desired range to the
closest endpoint. Display the image using plt.imshow
.
img_add = None
Crop the original image (img
variable) to a 130 x 150 image including Mochi's face. Discard the alpha colour channel (i.e. resulting img_cropped
should only have RGB channels)
Display the image.
img_cropped = None
PyTorch is a Python-based neural networks package. Along with tensorflow, PyTorch is currently one of the most popular machine learning libraries.
PyTorch, at its core, is similar to Numpy in a sense that they both try to make it easier to write codes for scientific computing achieve improved performance over vanilla Python by leveraging highly optimized C back-end. However, compare to Numpy, PyTorch offers much better GPU support and provides many high-level features for machine learning. Technically, Numpy can be used to perform almost every thing PyTorch does. However, Numpy would be a lot slower than PyTorch, especially with CUDA GPU, and it would take more effort to write machine learning related code compared to using PyTorch.
import torch
Use the function torch.from_numpy
to convert the numpy array img_cropped
into
a PyTorch tensor. Save the result in a variable called img_torch
.
img_torch = None
Use the method <Tensor>.shape
to find the shape (dimension and size) of img_torch
.
How many floating-point numbers are stored in the tensor img_torch
?
What does the code img_torch.transpose(0,2)
do? What does the expression return?
Is the original variable img_torch
updated? Explain.
What does the code img_torch.unsqueeze(0)
do? What does the expression return?
Is the original variable img_torch
updated? Explain.
Find the maximum value of img_torch
along each colour channel? Your output should be a one-dimensional
PyTorch tensor with exactly three values.
Hint: lookup the function torch.max
.