https://blog.csdn.net/Dear_learner/article/details/122920181

Two elements of building a model:
● build sub modules (such as convolution layer, pooling layer, activation layer and full connection layer in the network structure);
● splice sub modules (splice the sub modules in a certain order to finally get the desired network structure).

The LeNet class inherits NN Module, and in__ init__ The method realizes the construction of each sub module.

Forward method in LetNet class -- realize the splicing operation of sub modules by calling forward method.

NN is inherited in the construction model Module class. All network layers inherit from this class.

torch.nn: This is the neural network Module of pytoch. The Module here is one of its sub modules. In addition, there are several sub modules in parallel with the Module. For example:

nn.init: weight initialization {NN Functional: specific implementation of functions, such as convolution, pooling, activation, etc. NN Parameter: Zhang quantum class, which represents learnable parameters, such as weight and bias.

nn.Module: the base class of all networks, which manages the properties of the network.

In constructing the network structure, we need to inherit NN Module class and reconstruct it__ init__ And forward, but you should pay attention to:
● generally, the layers with learnable parameters in the network (such as convolution layer and full connection layer) are placed in the constructor__ init__ Of course, you can also put layers without parameters in it;
● generally, layers without learnable parameters (such as ReLU, dropout and batchspecification layers) can be placed in the constructor or not. If not, they can be placed in the constructor__ init__ Inside, NN can be used in the forward method Functional instead;
● the core function of rewriting is the connection between each model layer, which must be realized.
For example:
1. Put all the layers in the constructor__ init__ Then connect these layers in order in forward.

2. Put the layers without training parameters into forward, so these layers do not appear in the model, but the running relationship is through torch. In forward nn. Function implementation.

nn. Three Sequential implementation methods of module: packaging layer through Sequential
That is, several layers are packaged into a large layer (block), and then directly call the packaged layer.

The first: each layer of the model of Sequential output results has no name. It is named by 0, 1, 2 and 3 by default.

 1 import torch.nn as nn
 2 
 3 model = nn.Sequential(
 4                       nn.Conv2d(3, 24, 3),
 5                       nn.ReLU(),
 6                       nn.Conv2d(24, 5, 3),
 7                       nn.ReLU())
 8 print(model)
 9 
10 #output
11 Sequential(
12   (0): Conv2d(3, 24, kernel_size=(3, 3), stride=(1, 1))
13   (1): ReLU()
14   (2): Conv2d(24, 5, kernel_size=(3, 3), stride=(1, 1))
15   (3): ReLU()
16 )

The second method: you can add a name to each layer, but you can't get the layer by name. You still need to get it by index, that is, model[2] is correct and model ['conv2'] is wrong.

import torch.nn as nn
from collections import OrderedDict
model = nn.Sequential(OrderedDict([
                  ('conv1', nn.Conv2d(1,20,5)),
                  ('relu1', nn.ReLU()),
                  ('conv2', nn.Conv2d(20,64,5)),
                  ('relu2', nn.ReLU())
                ]))
 
print(model)
print(model[2]) # Which layer is obtained by index
"""
Output result:
Sequential(
  (conv1): Conv2d(1, 20, kernel_size=(5, 5), stride=(1, 1))
  (relu1): ReLU()
  (conv2): Conv2d(20, 64, kernel_size=(5, 5), stride=(1, 1))
  (relu2): ReLU()
)
Conv2d(20, 64, kernel_size=(5, 5), stride=(1, 1))
"""

Third: add_module method is a method that Sequential inherits the parent module.

import torch.nn as nn
from collections import OrderedDict

model = nn.Sequential()
model.add_module("conv1",nn.Conv2d(1,20,5))
model.add_module('relu1', nn.ReLU())
model.add_module('conv2', nn.Conv2d(20,64,5))
model.add_module('relu2', nn.ReLU())
 
print(model)
print(model[2]) # Which layer is obtained by index

"""
Output result:
Sequential(
  (conv1): Conv2d(1, 20, kernel_size=(5, 5), stride=(1, 1))
  (relu1): ReLU()
  (conv2): Conv2d(20, 64, kernel_size=(5, 5), stride=(1, 1))
  (relu2): ReLU()
)
Conv2d(20, 64, kernel_size=(5, 5), stride=(1, 1))
"""

nn. Several common methods of obtaining network information in module:

def children(self):
 
def named_children(self):
 
def modules(self):
 
def named_modules(self, memo=None, prefix=''):
 
'''
Note: these methods return a Iterator Iterator, so pass for Circular access, of course, can also be through next
'''

Summary:
(1)model.children() and model named_ The children () method returns the iterator iterator;

(2)model.children(): each element returned in each iteration is actually a Sequential type, and the Sequential type can use the subscript index index to obtain the specific layer in each Sequential, such as conv layer, deny layer, etc;

(3)model.named_children(): each element returned in each iteration is actually a tuple type. The first element of the tuple is the name, and the second element is the corresponding layer or Sequential.

Summary:

(1)model.modules() and model named_ The modules () method returns the iterator iterator;

(2) The modules() method of model and named_ The modules() method will traverse all the components of the whole model (including packaging layer, individual layer, user-defined layer, etc.) from shallow to deep, but each element returned by modules() is the directly returned layer object itself, not named_ Each element returned by modules() is a tuple. The first element is the name and the second element is the layer object itself.

(3) How to understand the difference between children and modules. Note that the model, layer, activation function and loss function in pytorch can be regarded as the expansion of module, so modules and named_modules will iterate layer by layer, from shallow to deep. Each custom block and each layer in the block will be iterated as modules. Children are more intuitive, which means the so-called "children", so there are no layers of iteration.