web123456

Getting started with pytorch Tensor (I)

Tensor

Tensor tensor, you can simply think that it is aArray, and supports efficient scientific computing. It can be a number (scalar),One-dimensional array(vector), two-dimensional array (matrix), and higher-dimensional arrays (higher-order data). Tensor andNumpyndarrays are similar, but PyTorch's tensor supportsGPUaccelerate.

# Let's begin
from __future__ import print_function
import torch  as t
t.__version__
  • 1
  • 2
  • 3
  • 4
'1.0.1'
  • 1

Basic Operation

The interface of tensor is similar to Numpy

From the perspective of interfaces, the operations of tensors can be divided into two categories:

  • ,likewait.
  • ,likewait.

like ((a, b))and ((b))Functional equivalent.

From the perspective of storage, the operations on tensor can be divided into two categories:

  • It will not modify its own data, such as(b), the result of addition will return a new tensor.
  • Will modify its own data, such asa.add_(b), The result of the addition is still stored in a, a is modified.

functionName_The ending is inplace, which means that the caller's own data will be modified and it needs to be distinguished in actual applications.

Create Tensor

existPyTorchThere are many methods to create new tensors, as shown in Table 3-1.

Common methods of creating new tensors

function Function
Tensor(*sizes) Basic constructor
tensor(data,) Similar constructors
ones(*sizes) All 1Tensor
zeros(*sizes) All 0Tensor
eye(*sizes) The diagonal is 1, the others are 0
arange(s,e,step From s to e, the step is step
linspace(s,e,steps) From s to e, split evenly into steps portions
rand/randn(*sizes) Uniform/standard distribution
normal(mean,std)/uniform(from,to) Normal distribution/uniform distribution
randperm(m) Random arrangement

These creation methods can specify the data type dtype and store device (cpu/gpu) when creating.

Which usesTensorThe most complex and variable way to create a new tensor function. It can not only receive a list, and create a new tensor based on the data of the list, but also create a new tensor based on the specified shape, and pass in other tensors.

# Specify the shape of the tensor
a = t.Tensor(2, 3)
a # The value depends on the state of the memory space. It may be overflow when printing
  • 1
  • 2
  • 3
tensor([[5.3285e+01, 4.5908e-41, 1.4013e-45],
        [0.0000e+00, 1.4013e-45, 0.0000e+00]])
  • 1
  • 2
# Create a tensor with list data
b = t.Tensor([[1,2,3],[4,5,6]])
b
  • 1
  • 2
  • 3
tensor([[1., 2., 3.],
        [4., 5., 6.]])
  • 1
  • 2
b.tolist() # Convert tensor to list
  • 1
[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]
  • 1

()returnObject, it is a subclass of tuple, but its usage is slightly different from tuple

b_size = b.size()
b_size
  • 1
  • 2
([2, 3])
  • 1
b.numel() # The total number of elements in b, 2*3, is equivalent to ()
  • 1
6
  • 1
# Create a tensor with the same shape as b
c = t.Tensor(b_size)
# Create a tensor with elements 2 and 3
d = t.Tensor((2, 3))
c, d
  • 1
  • 2
  • 3
  • 4
  • 5
(tensor([[5.3285e+01, 4.5908e-41, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00]]), tensor([2., 3.]))
  • 1
  • 2

Apart from(), can also be usedCheck the shape of the tensor directly.Equivalent to()

c.shape
  • 1
([2, 3])
  • 1

It should be noted that(*sizes)When creating a tensor, the system will not allocate space immediately, but will only calculate whether the remaining memory is sufficient to use it. It will only allocate when using the tensor. Other operations will allocate space immediately after creating the tensor. Other commonly used methods of creating tensors are given below.

t.ones(2, 3)
  • 1
tensor([[1., 1., 1.],
        [1., 1., 1.]])
  • 1
  • 2
t.zeros(2, 3)
  • 1
tensor([[0., 0., 0.],
        [0., 0., 0.]])
  • 1
  • 2
t.arange(1, 6, 2)
  • 1
tensor([1, 3, 5])
  • 1
# Cannot take 1 as step
t.linspace(1, 10, 3)
  • 1
  • 2
tensor([ 1.0000,  5.5000, 10.0000])
  • 1
t.randn(2, 3)
  • 1
tensor([[-1.4943, -0.7678, -1.3722],
        [-0.9745, -1.3788, -0.0625]])
  • 1
  • 2
t.randperm(5) # Random arrangement of length 5
  • 1
tensor([3, 2, 0, 1, 4])
  • 1
t.eye(2, 3, dtype=t.int) # The diagonal is 1, the number of rows and columns is not required to be consistent
  • 1
tensor([[1, 0, 0],
        [0, 1, 0]], dtype=torch.int32)
  • 1
  • 2
Common Tensor operations

passThe method can adjust the shape of the tensor, and it is necessary to ensure that the total number of elements before and after the adjustment is consistent. An application may often need to add or reduce a certain dimension, at this timesqueezeandunsqueezeTwo functions come in handy.

# is the reshape in numpy
a = t.Tensor([[1,2,3],[4,5,6]])
a.view(2,3)
  • 1
  • 2
  • 3
  • 4
tensor([[1., 2., 3.],
        [4., 5., 6.]])
  • 1
  • 2
b = a.view(-1, 3) # When a certain dimension is -1, its size will be calculated automatically
b.shape
b
  • 1
  • 2
  • 3
tensor([[1., 2., 3.],
        [4., 5., 6.]])
  • 1
  • 2
b.unsqueeze(1) # Pay attention to the shape and add "1" to the first dimension (subscript starts from 0).
#Equivalent to b[:,None]
b[:, None].shape # ([2, 1, 3])
print(b)
  • 1
  • 2
  • 3
  • 4
tensor([[1., 2., 3.],
        [4., 5., 6.]])
  • 1
  • 2
b.unsqueeze(-2) # -2 means the penultimate dimension
  • 1
tensor([[[1., 2., 3.]],

        [[4., 5., 6.]]])
  • 1
  • 2
  • 3
c = b.view(1, 1, 1, 2, 3)
c.squeeze(0) # Compress the 0th dimension "1"
  • 1
  • 2
tensor([[[[1., 2., 3.],
          [4., 5., 6.]]]])
  • 1
  • 2
c.squeeze() # Compression of all dimensions as "1"
  • 1
tensor([[1., 2., 3.],
        [4., 5., 6.]])
  • 1
  • 2
a[1] = 100
b # a modified, b after view will also be modified
  • 1
  • 2
tensor([[  1.,   2.,   3.],
        [100., 100., 100.]])
  • 1
  • 2

resizeIt's another one that can be used to adjustsizemethod, but withviewDifferently, it can modify the size of the tensor. If the new size exceeds the original size, the new memory space will be automatically allocated. If the new size is smaller than the original size, the previous data will still be saved. See an example.

b.resize_(1, 3)
b
  • 1
  • 2
tensor([[1., 2., 3.]])
  • 1
b.resize_(3, 3) # The old data is still stored, and the extra size will allocate new space
b
  • 1
  • 2
tensor([[1.0000e+00, 2.0000e+00, 3.0000e+00],
        [1.0000e+02, 1.0000e+02, 1.0000e+02],
        [1.6992e-07, 0.0000e+00, 0.0000e+00]])
  • 1
  • 2
  • 3
Indexing operations

Tensor supports similar index operations, and is also syntax similar. The following is a few examples to explain commonly used index operations. If there is no special description, the indexed results share memory with the original tensor, that is, one is modified, and the other will be modified accordingly.

a = t.randn(3, 4)
a
  • 1
  • 2
tensor([[-0.0766, -0.4981, -0.1263, -0.2356],
        [ 0.1508,  1.0202,  0.4058, -0.9278],
        [ 0.0286, -0.6448, -0.3914,  0.8246]])
  • 1
  • 2
  • 3
a[0] # Line 0 (subscript starts from 0)
  • 1
tensor([-0.0766, -0.4981, -0.1263, -0.2356])
  • 1
a[:, 0] # Column 0
  • 1
tensor([-0.0766,  0.1508,  0.0286])
  • 1
a[0][2] # The second element in line 0, equivalent to a[0, 2]
  • 1
tensor(-0.1263)
  • 1
a[0, -1] # The last element on line 0
  • 1
tensor(-0.2356)
  • 1
a[:2] # First two lines
  • 1
tensor([[-0.0766, -0.4981, -0.1263, -0.2356],
        [ 0.1508,  1.0202,  0.4058, -0.9278]])
  • 1
  • 2
a[:2, 0:2] # First two rows, column 0, 1
  • 1
tensor([[-0.0766, -0.4981],
        [ 0.1508,  1.0202]])
  • 1
  • 2
print(a[0:1, :2]) # Row 0, first two columns
print(a[0, :2]) # Note the difference between the two: different shapes
  • 1
  • 2
tensor([[-0.0766, -0.4981]])
tensor([-0.0766, -0.4981])
  • 1
  • 2
# None is similar to, adding an axis to a
# is equivalent to (1, [0], [1])
a[None].shape
  • 1
  • 2
  • 3
([1, 3, 4])
  • 1
a[None].shape # is equivalent to a[None,:,:]
  • 1
([1, 3, 4])
  • 1
a[:,None,:].shape
  • 1
([3, 1, 4])
  • 1
a[:,None,:,None,None].shape
  • 1
([3, 1, 4, 1, 1])
  • 1
a > 1 # Return a ByteTensor
  • 1
tensor([[0, 0, 0, 0],
        [0, 1, 0, 0],
        [0, 0, 0, 0]], dtype=torch.uint8)
  • 1
  • 2
  • 3
a[a>1] # is equivalent to a.masked_select(a>1)
# Select the result and the original tensor do not share memory space
  • 1
  • 2
tensor([1.0202])
  • 1
a[t.LongTensor([0,1])] # Line 0 and Line 1
  • 1
tensor([[-0.0766, -0.4981, -0.1263, -0.2356],
        [ 0.1508,  1.0202,  0.4058, -0.9278]])
  • 1
  • 2

Commonly used selection functions

function Function
index_select(input, dim, index) Select on the specified dimension dim, such as selecting certain rows and columns
masked_select(input, mask) Examples are as above, a[a>0], and use ByteTensor to select
non_zero(input) Subscript of non-0 elements
gather(input, dim, index) According to index, select data in dim dimension, and the output size is the same as index

gatherIt is a relatively complex operation. For a 2-dimensional tensor, each element output is as follows:

out[i][j] = input[index[i][j]][j]  # dim=0
out[i][j] = input[i][index[i][j]]  # dim=1
  • 1
  • 2

3D tensorgatherThe same goes for operation, let’s give a few examples below.

a = t.arange(0, 16).view(4, 4)
a
  • 1
  • 2
tensor([[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11],
        [12, 13, 14, 15]])
  • 1
  • 2
  • 3
  • 4
# Select the element of the diagonal
index = t.LongTensor([[0,1,2,3]])
a.gather(0, index)
  • 1
  • 2
  • 3
tensor([[ 0,  5, 10, 15]])
  • 1
# Select the element on the diagonal
index = t.LongTensor([[3,2,1,0]]).t()
a.gather(1, index)
  • 1
  • 2
  • 3
tensor([[ 3],
        [ 6],
        [ 9],
        [12]])
  • 1
  • 2
  • 3
  • 4
# Select the element on the opposition line and pay attention to the difference from the above
index = t.LongTensor([[3,2,1,0]])
a.gather(0, index)
  • 1
  • 2
  • 3
tensor([[12,  9,  6,  3]])
  • 1
# Select two elements on diagonal lines
index = t.LongTensor([[0,1,2,3],[3,2,1,0]]).t()
b = a.gather(1, index)
b
  • 1
  • 2
  • 3
  • 4
tensor([[ 0,  3],
        [ 5,  6],
        [10,  9],
        [15, 12]])
  • 1
  • 2
  • 3
  • 4