PyTorch-Introduction

发表于 更新于 分类于
本文字数： 5.1k 阅读时长 ≈ 5 分钟

Introduction

  • An open source machine learning framework
  • A Python package that provides two high-level features:
    • Tensor computation with strong GPU acceleration
    • Deep neural networks built on a tape-based autograd system
  • Use the FashionMNIST dataset to train a neural network that predicts if an input image belongs to one of the following classes: T-shirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag, or Ankle boot.

1

Overview of the DNN Training Procedure

2

Tensor

High-dimensional matrix(array)

Data Type

3

shape of Tensor

4

Constructor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
# From list / NumPy array
np_x = np.array([[1,-1],[-1,1]])
x_data = torch.from_numpy(np_x)

# From another tensor
x_ones = torch.ones_like(x_data) # retains the properties of x_data
print(f"Ones Tensor: \n {x_ones} \n")

>>> Ones Tensor: 
 tensor([[1, 1],
        [1, 1]]) 


x_rand = torch.rand_like(x_data, dtype=torch.float) # overrides the datatype of x_data
print(f"Random Tensor: \n {x_rand} \n")

>>> Random Tensor: 
 tensor([[0.1943, 0.8950],
        [0.5489, 0.7244]]) 
# With random or constant values

shape = (2,3)      # a tuple of tensor dimensions. it determines the dimensionality of the output tensor.
rand_tensor = torch.rand(shape)
ones_tensor = torch.ones(shape)
zeros_tensor = torch.zeros(shape)

>>> print(f"Random Tensor: \n {rand_tensor} \n")
>>> print(f"Ones Tensor: \n {ones_tensor} \n")
>>> print(f"Zeros Tensor: \n {zeros_tensor}")

Random Tensor: 
tensor([[0.4687, 0.9809, 0.0818],
        [0.8493, 0.7848, 0.3891]]) 

Ones Tensor: 
  tensor([[1., 1., 1.],
          [1., 1., 1.]]) 

Zeros Tensor: 
  tensor([[0., 0., 0.],
          [0., 0., 0.]])

Attributes

describe their shape, data type, and the device on which they are stored

1
2
3
4
5
6
7
8
9
tensor = torch.rand(3,4)

>>> print(f"Shape of tensor: {tensor.shape}")
>>> print(f"Datatype of tensor: {tensor.dtype}")
>>> print(f"Device tensor is stored on: {tensor.device}")

Shape of tensor: torch.Size([3, 4])
Datatype of tensor: torch.float32
Device tensor is stored on: cpu

Operators

Over 100 tensor operations, including arithmetic, linear algebra, matrix manipulation (transposing, indexing, slicing), sampling and more are comprehensively described

  • move tensor to the GPU if available
1
2
if torch.cuda.is_available():
  tensor = tensor.to('cuda')
  • Standard numpy-like indexing and slicing
1
2
3
4
5
6
7
8
9
10
11
12
13
14
>>> tensor = torch.ones(4, 4)
>>> print('First row: ',tensor[0])
>>> print('First column: ', tensor[:, 0])
>>> print('Last column:', tensor[..., -1])
>>> tensor[:,1] = 0
>>> print(tensor)

First row:  tensor([1., 1., 1., 1.])
First column:  tensor([1., 1., 1., 1.])
Last column: tensor([1., 1., 1., 1.])
tensor([[1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.]])
  • Joining tensors
1
2
3
4
5
6
7
8
# use torch.cat to concatenate a sequence of tensors along a given dimension
>>> t1 = torch.cat([tensor, tensor, tensor], dim=1)
>>> print(t1)

tensor([[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.]])
  • Arithmetic operations
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# This computes the matrix multiplication between two tensors. y1, y2, y3 will have the same value
>>> y1 = tensor @ tensor.T
>>> y2 = tensor.matmul(tensor.T)

>>> y3 = torch.rand_like(tensor)
>>> torch.matmul(tensor, tensor.T, out=y3)


# This computes the element-wise product. z1, z2, z3 will have the same value
>>> z1 = tensor * tensor
>>> z2 = tensor.mul(tensor)

>>> z3 = torch.rand_like(tensor)
>>> torch.mul(tensor, tensor, out=z3)

tensor([[1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.]])
  • Single-element tensors
1
2
3
4
5
6
7
# aggregating all values of a tensor into one value
>>> agg = tensor.sum()
>>> agg_item = agg.item()  
>>> print(agg, agg_item, type(agg_item))

tensor(12.) 12.0 <class 'float'>

  • In-place operations
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# In-place operations save some memory, but can be problematic when computing derivatives because of an immediate loss of history. 

>>> print(tensor, "\n")
>>> tensor.add_(5)
>>> print(tensor)

tensor([[1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.]]) 

tensor([[6., 5., 6., 6.],
        [6., 5., 6., 6.],
        [6., 5., 6., 6.],
        [6., 5., 6., 6.]])
  • Squeeze : remove the specified dimension with length = 1
1
2
3
4
5
6
>>> x = torch.zeros([1,2,3])
>>> x.shape
torch.Size([1,2,3])
>>> x = x.squeeze(0) # dim = 0
>>> x.shape
torch.Size([2,3])
  • Unsqueeze: expand a new dimension
1
2
3
4
5
6
>>> x = torch.zeros([2,3])
>>> x.shape
torch.Size([2,3])
>>> x = x.unsqueeze(1) # dim = 0
>>> x.shape
torch.Size([2,1,3])
  • Bridge with NumPy && Tensor to NumPy array

Tensors on the CPU and NumPy arrays can share their underlying memory locations, and changing one will change the other.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
>>> t = torch.ones(5)
>>> print(f"t: {t}")
>>> n = t.numpy()
>>> print(f"n: {n}")

t: tensor([1., 1., 1., 1., 1.])
n: [1. 1. 1. 1. 1.]

# A change in the tensor reflects in the NumPy array.
>>> t.add_(1)
>>> print(f"t: {t}")
>>> print(f"n: {n}")

t: tensor([2., 2., 2., 2., 2.])
n: [2. 2. 2. 2. 2.]