PyTorch

在这节课中,我们将学习PyTorch,它是一个用于构建动态神经网络的学习库。我们将在本课程中了解其基础知识,以及如何创建和使用张量(Tensor),而我们将在下一课中使用它制作模型。

Tensor 基础知识

In [1]:
# 安装PyTorch库
!pip3 install torch
Collecting torch
  Downloading https://files.pythonhosted.org/packages/7e/60/66415660aa46b23b5e1b72bc762e816736ce8d7260213e22365af51e8f9c/torch-1.0.0-cp36-cp36m-manylinux1_x86_64.whl (591.8MB)
    100% |████████████████████████████████| 591.8MB 28kB/s 
tcmalloc: large alloc 1073750016 bytes == 0x61f04000 @  0x7f61a90372a4 0x591a07 0x5b5d56 0x502e9a 0x506859 0x502209 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x507641 0x502209 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x507641 0x504c28 0x502540 0x502f3d 0x507641
Installing collected packages: torch
Successfully installed torch-1.0.0

注意:上述安装PyTorch的方式与官网并不一致,因为这里是基于Colab云端安装的,而大家一般是使用pip或者conda来安装PyTorch。具体安装教程参考:https://pytorch.org/

In [0]:
import numpy as np
import torch
In [3]:
# 创建一个张量
x = torch.Tensor(3, 4)
print("Type: {}".format(x.type()))
print("Size: {}".format(x.shape))
print("Values: \n{}".format(x))
Type: torch.FloatTensor
Size: torch.Size([3, 4])
Values: 
tensor([[8.1343e-37, 0.0000e+00, 2.8026e-44, 0.0000e+00],
        [       nan, 0.0000e+00, 1.3733e-14, 4.7429e+30],
        [1.9431e-19, 4.7429e+30, 5.0938e-14, 0.0000e+00]])
In [4]:
# 创建一个随机张量
x = torch.randn(2, 3) # torch.randn对应于正态分布,而rand(2,3)对应于均匀分布
print (x)
tensor([[-0.2826, -2.0357, -1.6480],
        [ 0.7468, -1.1572,  0.4160]])
In [5]:
# 0和1张量
x = torch.zeros(2, 3)
print (x)
x = torch.ones(2, 3)
print (x)
tensor([[0., 0., 0.],
        [0., 0., 0.]])
tensor([[1., 1., 1.],
        [1., 1., 1.]])
In [6]:
# 列表(List) → 张量
x = torch.Tensor([[1, 2, 3],[4, 5, 6]])
print("Size: {}".format(x.shape)) 
print("Values: \n{}".format(x))
Size: torch.Size([2, 3])
Values: 
tensor([[1., 2., 3.],
        [4., 5., 6.]])
In [7]:
# NumPy 数组 → 张量
x = torch.from_numpy(np.random.rand(2, 3))
print("Size: {}".format(x.shape)) 
print("Values: \n{}".format(x))
Size: torch.Size([2, 3])
Values: 
tensor([[0.0632, 0.4443, 0.9658],
        [0.4391, 0.9068, 0.6760]], dtype=torch.float64)
In [8]:
# 改变张量类型(张量默认为float类型)
x = torch.Tensor(3, 4)
print("Type: {}".format(x.type()))
x = x.long()
print("Type: {}".format(x.type()))
Type: torch.FloatTensor
Type: torch.LongTensor

Tensor 运算

In [9]:
# 加法
x = torch.randn(2, 3)
y = torch.randn(2, 3)
z = x + y
print("Size: {}".format(z.shape)) 
print("Values: \n{}".format(z))
Size: torch.Size([2, 3])
Values: 
tensor([[ 0.7976,  0.3528,  2.3146],
        [-1.6895,  1.9033, -1.0576]])
In [10]:
# 向量点积
x = torch.randn(2, 3)
y = torch.randn(3, 2)
z = torch.mm(x, y)
print("Size: {}".format(z.shape)) 
print("Values: \n{}".format(z))
Size: torch.Size([2, 2])
Values: 
tensor([[ 0.3005, -2.8694],
        [ 1.5858, -3.1363]])
In [11]:
# 转置
x = torch.randn(2, 3)
print("Size: {}".format(x.shape)) 
print("Values: \n{}".format(x))
y = torch.t(x)
print("Size: {}".format(y.shape)) 
print("Values: \n{}".format(y))
Size: torch.Size([2, 3])
Values: 
tensor([[ 0.2164,  0.5022,  0.8684],
        [-0.7384,  0.9556,  1.0076]])
Size: torch.Size([3, 2])
Values: 
tensor([[ 0.2164, -0.7384],
        [ 0.5022,  0.9556],
        [ 0.8684,  1.0076]])
In [12]:
# Reshape
z = x.view(3, 2)
print("Size: {}".format(z.shape)) 
print("Values: \n{}".format(z))
Size: torch.Size([3, 2])
Values: 
tensor([[ 0.2164,  0.5022],
        [ 0.8684, -0.7384],
        [ 0.9556,  1.0076]])
In [14]:
# reshaping的危险(意外后果)
x = torch.tensor([
    [[1,1,1,1], [2,2,2,2], [3,3,3,3]],
    [[10,10,10,10], [20,20,20,20], [30,30,30,30]]
])
print("Size: {}".format(x.shape)) 
print("Values: \n{}\n".format(x))
a = x.view(x.size(1), -1)
print("Size: {}".format(a.shape)) 
print("Values: \n{}\n".format(a))
b = x.transpose(0,1).contiguous()
print("Size: {}".format(b.shape)) 
print("Values: \n{}\n".format(b))
c = b.view(b.size(0), -1)
print("Size: {}".format(c.shape)) 
print("Values: \n{}".format(c))
Size: torch.Size([2, 3, 4])
Values: 
tensor([[[ 1,  1,  1,  1],
         [ 2,  2,  2,  2],
         [ 3,  3,  3,  3]],

        [[10, 10, 10, 10],
         [20, 20, 20, 20],
         [30, 30, 30, 30]]])

Size: torch.Size([3, 8])
Values: 
tensor([[ 1,  1,  1,  1,  2,  2,  2,  2],
        [ 3,  3,  3,  3, 10, 10, 10, 10],
        [20, 20, 20, 20, 30, 30, 30, 30]])

Size: torch.Size([3, 2, 4])
Values: 
tensor([[[ 1,  1,  1,  1],
         [10, 10, 10, 10]],

        [[ 2,  2,  2,  2],
         [20, 20, 20, 20]],

        [[ 3,  3,  3,  3],
         [30, 30, 30, 30]]])

Size: torch.Size([3, 8])
Values: 
tensor([[ 1,  1,  1,  1, 10, 10, 10, 10],
        [ 2,  2,  2,  2, 20, 20, 20, 20],
        [ 3,  3,  3,  3, 30, 30, 30, 30]])
In [16]:
# 维度操作
x = torch.randn(2, 3)
print("Values: \n{}".format(x))
y = torch.sum(x, dim=0) # 为每列添加各行叠加的值
print("Values: \n{}".format(y))
z = torch.sum(x, dim=1) # 为每行添加各列叠加的值
print("Values: \n{}".format(z))
Values: 
tensor([[-2.2913,  0.1027, -0.3427],
        [ 0.7189,  0.4223, -0.0364]])
Values: 
tensor([-1.5724,  0.5250, -0.3791])
Values: 
tensor([-2.5313,  1.1048])

索引,切片和级联

In [17]:
x = torch.randn(3, 4)
print("x: \n{}".format(x))
print ("x[:1]: \n{}".format(x[:1]))
print ("x[:1, 1:3]: \n{}".format(x[:1, 1:3]))
x: 
tensor([[ 0.5660,  1.3207,  0.6355,  0.4770],
        [-1.5738, -1.6227,  0.1413, -1.8778],
        [-0.0666, -0.2320, -0.3943, -0.4921]])
x[:1]: 
tensor([[0.5660, 1.3207, 0.6355, 0.4770]])
x[:1, 1:3]: 
tensor([[1.3207, 0.6355]])
In [18]:
# 选择维度索引
x = torch.randn(2, 3)
print("Values: \n{}".format(x))
col_indices = torch.LongTensor([0, 2])
chosen = torch.index_select(x, dim=1, index=col_indices) # 第0和第2列的值
print("Values: \n{}".format(chosen)) 
row_indices = torch.LongTensor([0, 1])
chosen = x[row_indices, col_indices] # 来自(0,0)和(2,1)的值
print("Values: \n{}".format(chosen)) 
Values: 
tensor([[ 1.7797, -1.2580, -0.6378],
        [-1.3466, -0.2311,  0.8040]])
Values: 
tensor([[ 1.7797, -0.6378],
        [-1.3466,  0.8040]])
Values: 
tensor([1.7797, 0.8040])
In [20]:
# 级联
x = torch.randn(2, 3)
print("Values: \n{}".format(x))
y = torch.cat([x, x], dim=0) # 按行堆叠(dim = 1按列堆叠)
print("Values: \n{}".format(y))
Values: 
tensor([[-0.4772,  0.7716,  2.2474],
        [ 1.3227, -0.4346,  1.1521]])
Values: 
tensor([[-0.4772,  0.7716,  2.2474],
        [ 1.3227, -0.4346,  1.1521],
        [-0.4772,  0.7716,  2.2474],
        [ 1.3227, -0.4346,  1.1521]])

梯度

In [22]:
# 带有gradient bookkeeping的张量
x = torch.rand(3, 4, requires_grad=True)
y = 3*x + 2
z = y.mean()
z.backward() # z是标量
print("Values: \n{}".format(x))
print("x.grad: \n", x.grad)
Values: 
tensor([[0.9651, 0.9546, 0.2183, 0.5656],
        [0.2327, 0.4397, 0.7654, 0.8600],
        [0.5148, 0.8122, 0.1790, 0.7555]], requires_grad=True)
x.grad: 
 tensor([[0.2500, 0.2500, 0.2500, 0.2500],
        [0.2500, 0.2500, 0.2500, 0.2500],
        [0.2500, 0.2500, 0.2500, 0.2500]])
  • $ y = 3x + 2 $
  • $ z = \sum{y}/N $
  • $ \frac{\partial(z)}{\partial(x)} = \frac{\partial(z)}{\partial(y)} \frac{\partial(z)}{\partial(x)} = \frac{1}{N} * 3 = \frac{1}{12} * 3 = 0.25 $

CUDA 张量

In [23]:
# CUDA可用吗?
print (torch.cuda.is_available())
True

如果上面的代码返回False,那么请转到菜单栏上的RuntimeChange runtime type并在Hardware accelerator下选择GPU

In [25]:
# 创建一个CPU版的0张量
x = torch.Tensor(3, 4).to("cpu")
print("Type: {}".format(x.type()))
Type: torch.FloatTensor
In [27]:
# 创建一个CUDA版的0张量
x = torch.Tensor(3, 4).to("cuda")
print("Type: {}".format(x.type()))
Type: torch.cuda.FloatTensor
In [0]: