To start, we import the PyTorch library.
Note that the package name is torch
import torch
A tensor represents a (possibly multidimensional) array of numerical values
x = torch.arange(12, dtype=torch.float32)
x
tensor([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11.])
x.numel()
12
We can access a tensor's shape
x.shape
torch.Size([12])
Change the shape of a tensor without altering its size or values
X = x.reshape(3, 4)
X
tensor([[ 0., 1., 2., 3.], [ 4., 5., 6., 7.], [ 8., 9., 10., 11.]])
We can construct a tensor with all elements set to 0 or one
torch.zeros((2, 3, 4))
tensor([[[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]], [[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]]])
torch.ones((2, 3, 4))
tensor([[[1., 1., 1., 1.], [1., 1., 1., 1.], [1., 1., 1., 1.]], [[1., 1., 1., 1.], [1., 1., 1., 1.], [1., 1., 1., 1.]]])
Sample each element randomly (and independently)
torch.randn(3, 4)
tensor([[ 0.1351, -0.9099, -0.2028, 2.1937], [-0.3200, -0.7545, 0.8086, -1.8730], [ 0.3929, 0.4931, 0.9114, -0.7072]])
Supplying the exact values for each element
torch.tensor([[2, 1, 4, 3], [1, 2, 3, 4], [4, 3, 2, 1]])
tensor([[2, 1, 4, 3], [1, 2, 3, 4], [4, 3, 2, 1]])
[-1]
selects the last row and [1:3]
selects the second and third rows
X[-1], X[1:3]
(tensor([ 8., 9., 10., 11.]), tensor([[ 4., 5., 6., 7.], [ 8., 9., 10., 11.]]))
We can also write elements of a matrix by specifying indices
X[1, 2] = 17
X
tensor([[ 0., 1., 2., 3.], [ 4., 5., 17., 7.], [ 8., 9., 10., 11.]])
To assign multiple elements the same value, we apply the indexing on the left-hand side of the assignment operation
X[:2, :] = 12
X
tensor([[12., 12., 12., 12.], [12., 12., 12., 12.], [ 8., 9., 10., 11.]])
torch.exp(x)
tensor([162754.7969, 162754.7969, 162754.7969, 162754.7969, 162754.7969, 162754.7969, 162754.7969, 162754.7969, 2980.9580, 8103.0840, 22026.4648, 59874.1406])
x = torch.tensor([1.0, 2, 4, 8])
y = torch.tensor([2, 2, 2, 2])
x + y, x - y, x * y, x / y, x ** y
(tensor([ 3., 4., 6., 10.]), tensor([-1., 0., 2., 6.]), tensor([ 2., 4., 8., 16.]), tensor([0.5000, 1.0000, 2.0000, 4.0000]), tensor([ 1., 4., 16., 64.]))
concatenate multiple tensors
X = torch.arange(12, dtype=torch.float32).reshape((3,4))
Y = torch.tensor([[2.0, 1, 4, 3], [1, 2, 3, 4], [4, 3, 2, 1]])
torch.cat((X, Y), dim=0), torch.cat((X, Y), dim=1)
(tensor([[ 0., 1., 2., 3.], [ 4., 5., 6., 7.], [ 8., 9., 10., 11.], [ 2., 1., 4., 3.], [ 1., 2., 3., 4.], [ 4., 3., 2., 1.]]), tensor([[ 0., 1., 2., 3., 2., 1., 4., 3.], [ 4., 5., 6., 7., 1., 2., 3., 4.], [ 8., 9., 10., 11., 4., 3., 2., 1.]]))
Construct a binary tensor via logical statements
X == Y
tensor([[False, True, False, True], [False, False, False, False], [False, False, False, False]])
Summing all the elements in the tensor
X.sum()
tensor(66.)
Perform elementwise binary operations by invoking the broadcasting mechanism
a = torch.arange(3).reshape((3, 1))
b = torch.arange(2).reshape((1, 2))
a, b
(tensor([[0], [1], [2]]), tensor([[0, 1]]))
a + b
tensor([[0, 1], [1, 2], [2, 3]])
Running operations can cause new memory to be allocated to host results
before = id(Y)
Y = Y + X
id(Y) == before
False
Performing in-place operations
Z = torch.zeros_like(Y)
print('id(Z):', id(Z))
Z[:] = X + Y
print('id(Z):', id(Z))
id(Z): 140381179266448 id(Z): 140381179266448
If the value of X
is not reused in subsequent computations,
we can also use X[:] = X + Y
or X += Y
to reduce the memory overhead of the operation
before = id(X)
X += Y
id(X) == before
True
Converting to a NumPy tensor (ndarray
)
A = X.numpy()
B = torch.from_numpy(A)
type(A), type(B)
(numpy.ndarray, torch.Tensor)
Convert a size-1 tensor to a Python scalar
a = torch.tensor([3.5])
a, a.item(), float(a), int(a)
(tensor([3.5000]), 3.5, 3.5, 3)