[in 2021, some people see dust and others see stars, but it doesn't matter. We'll turn the page soon. -- December 31, 2021]
In the article Pytorch tutorials [pytorch official tutorial in Chinese and English] - 1 Quickstart The quick introduction version is shown in. Next, let's look at the important concept in pytorch: Tensor.
Original link: Tensors — PyTorch Tutorials 1.10.1+cu102 documentation
Tensors are a specialized data structure that are very similar to arrays and matrices. In PyTorch, we use tensors to encode the inputs and outputs of a model, as well as the model's parameters.
[tensor is a special data structure, which is very similar to array and matrix. In PyTorch, we use tensor to encode the input and output of the model and the parameters of the model.]
Tensors are similar to NumPy's ndarrays, except that tensors can run on GPUs or other hardware accelerators. In fact, tensors and NumPy arrays can often share the same underlying memory, eliminating the need to copy data (see Bridge with NumPy). Tensors are also optimized for automatic differentiation (we'll see more about that later in the Autograd section). If you're familiar with ndarrays, you'll be right at home with the Tensor API. If not, follow along!
[tensor is similar to NumPy's ndarrays, except that tensor can run on gpu or other hardware accelerators. In fact, tensor arrays and NumPy arrays can usually share the same underlying memory, eliminating the need to copy data (see Bridge with NumPy). Tensor is also optimized for automatic difference (we will describe it in detail later in the Autograd section) . If you are familiar with ndarrays, you are familiar with the tensor API. If not, do it!]
import torch import numpy as np
1 Initializing a Tensor
Tensors can be initialized in various ways. Take a look at the following examples:
[tensors can be initialized in various ways. Take the following example:]
(a)Directly from data
Tensors can be created directly from data. The data type is automatically inferred.
[tensors can be created directly from data. Data types are automatically inferred.]
data = [[1, 2],[3, 4]] x_data = torch.tensor(data)
(b)From a NumPy array
Tensors can be created from NumPy arrays (and vice versa - see Bridge with NumPy).
[tensors can be created from the NumPy array (and vice versa - see Bridge with NumPy).]
np_array = np.array(data) x_np = torch.from_numpy(np_array)
(c)From another tensor:
The new tensor retains the properties (shape, datatype) of the argument tensor, unless explicitly overridden.
[the new tensor retains the properties (shape, data type) of the parameter tensor unless explicitly overridden.]
x_ones = torch.ones_like(x_data) # retains the properties of x_data
print(f"Ones Tensor: \n {x_ones} \n")
x_rand = torch.rand_like(x_data, dtype=torch.float) # overrides the datatype of x_data
print(f"Random Tensor: \n {x_rand} \n")Output results:
Ones Tensor:
tensor([[1, 1],
[1, 1]])
Random Tensor:
tensor([[0.4557, 0.7406],
[0.5935, 0.1859]])(d)With random or constant values:
shape is a tuple of tensor dimensions. In the functions below, it determines the dimensionality of the output tensor.
[shape is a tuple of tensor dimension. In the following function, it determines the dimension of output tensor.]
shape = (2,3,)
rand_tensor = torch.rand(shape)
ones_tensor = torch.ones(shape)
zeros_tensor = torch.zeros(shape)
print(f"Random Tensor: \n {rand_tensor} \n")
print(f"Ones Tensor: \n {ones_tensor} \n")
print(f"Zeros Tensor: \n {zeros_tensor}")Output results:
Random Tensor:
tensor([[0.8012, 0.4547, 0.4156],
[0.6645, 0.1763, 0.3860]])
Ones Tensor:
tensor([[1., 1., 1.],
[1., 1., 1.]])
Zeros Tensor:
tensor([[0., 0., 0.],
[0., 0., 0.]])2 Attributes of a Tensor
Tensor attributes describe their shape, datatype, and the device on which they are stored.
[tensor attributes describe their shape, data type and the device in which they are stored.]
tensor = torch.rand(3,4)
print(f"Shape of tensor: {tensor.shape}")
print(f"Datatype of tensor: {tensor.dtype}")
print(f"Device tensor is stored on: {tensor.device}")Output:
Shape of tensor: torch.Size([3, 4]) Datatype of tensor: torch.float32 Device tensor is stored on: cpu
3 Operations on Tensors
Over 100 tensor operations, including arithmetic, linear algebra, matrix manipulation (transposing, indexing, slicing), sampling and more are comprehensively described here.
More than 100 tensor operations, including arithmetic, linear algebra, matrix operations (transpose, index, slice), sampling and more are fully described here
Each of these operations can be run on the GPU (at typically higher speeds than on a CPU). If you're using Colab, allocate a GPU by going to Runtime > Change runtime type > GPU.
[these operations can be run on GPU (usually faster than on CPU). If you use Colab, allocate a GPU through runtime > change runtime type > GPU.]
By default, tensors are created on the CPU. We need to explicitly move tensors to the GPU using .to method (after checking for GPU availability). Keep in mind that copying large tensors across devices can be expensive in terms of time and memory!
[by default, the tensor is created on the CPU. We need to use the. To method to explicitly move the tensor to the GPU (after checking the GPU availability). Remember, it is very expensive to copy large tensors between devices in terms of time and memory!]
# We move our tensor to the GPU if available
if torch.cuda.is_available():
tensor = tensor.to('cuda')Try out some of the operations from the list. If you're familiar with the NumPy API, you'll find the Tensor API a breeze to use.
[try some operations in the list. If you are familiar with NumPy API, you will find it easy to use tensor API.]
(a)Standard numpy-like indexing and slicing:
[standard numpy class index and slice:]
tensor = torch.ones(4, 4)
print('First row: ', tensor[0])
print('First column: ', tensor[:, 0])
print('Last column:', tensor[..., -1])
tensor[:,1] = 0
print(tensor)Output results:
First row: tensor([1., 1., 1., 1.])
First column: tensor([1., 1., 1., 1.])
Last column: tensor([1., 1., 1., 1.])
tensor([[1., 0., 1., 1.],
[1., 0., 1., 1.],
[1., 0., 1., 1.],
[1., 0., 1., 1.]])Joining tensors You can use torch.cat to concatenate a sequence of tensors along a given dimension. See also torch.stack, another tensor joining op that is subtly different from torch.cat.
[you can use torch.cat to connect a series of tensors along a given dimension. See torch.stack, which is another tensor connection op, which is slightly different from torch.cat.]
t1 = torch.cat([tensor, tensor, tensor], dim=1) print(t1)
Output:
tensor([[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.]])(b)Arithmetic operations
[arithmetic operation]
# This computes the matrix multiplication between two tensors. y1, y2, y3 will have the same value y1 = tensor @ tensor.T y2 = tensor.matmul(tensor.T) y3 = torch.rand_like(tensor) torch.matmul(tensor, tensor.T, out=y3) # This computes the element-wise product. z1, z2, z3 will have the same value z1 = tensor * tensor z2 = tensor.mul(tensor) z3 = torch.rand_like(tensor) torch.mul(tensor, tensor, out=z3)
(c)Single-element tensors
[single element tensor]
If you have a one-element tensor, for example by aggregating all values of a tensor into one value, you can convert it to a Python numerical value using item():
[if you have a single element tensor, for example, aggregate all values of a tensor into one value, you can use item() to convert it to Python value:]
agg = tensor.sum() agg_item = agg.item() print(agg_item, type(agg_item))
Output:
12.0 <class 'float'>
(d)In-place operations
Operations that store the result into the operand are called in-place. They are denoted by a _ suffix. For example: x.copy_(y), x.t_(), will change x.
[operations that store results in operands (operands) are called in place operations. They are represented by the suffix. For example, x.copy_(y), x.t_ (), which will change X.]
print(tensor, "\n") tensor.add_(5) print(tensor)
result:
tensor([[1., 0., 1., 1.],
[1., 0., 1., 1.],
[1., 0., 1., 1.],
[1., 0., 1., 1.]])
tensor([[6., 5., 6., 6.],
[6., 5., 6., 6.],
[6., 5., 6., 6.],
[6., 5., 6., 6.]])NOTE
In-place operations save some memory, but can be problematic when computing derivatives because of an immediate loss of history. Hence, their use is discouraged.
[Note: local operation can save some memory, but there may be problems in calculating derivatives because history records will be lost immediately. Therefore, their use is not encouraged.]
4 Bridge with NumPy
Tensors on the CPU and NumPy arrays can share their underlying memory locations, and changing one will change the other.
[tensor and NumPy arrays on the CPU can share their underlying memory location. Changing one of them will change the other.]
Tensor to NumPy array
t = torch.ones(5)
print(f"t: {t}")
n = t.numpy()
print(f"n: {n}")result:
t: tensor([1., 1., 1., 1., 1.]) n: [1. 1. 1. 1. 1.]
A change in the tensor reflects in the NumPy array.
[the change of tensor is reflected in NumPy array.]
t.add_(1)
print(f"t: {t}")
print(f"n: {n}")Output:
t: tensor([2., 2., 2., 2., 2.]) n: [2. 2. 2. 2. 2.]
NumPy array to Tensor
n = np.ones(5) t = torch.from_numpy(n)
Changes in the NumPy array reflects in the tensor.
np.add(n, 1, out=n)
print(f"t: {t}")
print(f"n: {n}")result:
t: tensor([2., 2., 2., 2., 2.], dtype=torch.float64) n: [2. 2. 2. 2. 2.]
Note: take study notes. If you make mistakes, please correct them! It's not easy to write an article. Please contact me for reprint.