1. Start with the simplest neural network

In the process of learning machine learning, everyone should have a basic concept of neural network: neural network is like a function fitter, it can fit any function.

def NeuralNetwork(inputs, weight): # Define Neural Network
    prediction = inputs * weight   # Multiply data by weight
    return prediction

inputs = 2  # Input data can be changed at will
weight = -1 # weight
pred = NeuralNetwork(inputs, weight) # Forward propagation, that is, to get the predicted results
print(pred)

What we have just completed is the process of forward propagation, which is also a process of prediction.

Prediction is actually the result of the neural network "thinking" over the input data.

2. Overview of the working principle of neural networks

The above neural network "scales" the input data by multiplying the input by the weight.

Interaction of neural networks:

1. Accept the input variables as a source of information;

2. Have a weight variable and use it as knowledge;

3. Fuse information and knowledge to output prediction results

So far, all neural networks work in this way, using knowledge from weights to interpret information from input data.

Another way to understand the weights of a neural network is to use them as a measure of the sensitivity between the input of the network and the prediction: if the weights are very high, even the smallest input can have a very large impact on the prediction results; If the weight is small, it is a very large input and can only produce very small disturbances to the predicted results.

3. Predict using multiple inputs

In practical scenarios, if more information can be provided to the neural network, the neural network should be able to make more accurate predictions.

import numpy as np

weights = np.array([0.012, 0.3, 0.25])
inputs = np.array([26, 0.75, 1.2])

def NeuralNetwork(inputs, weights):
    pred=inputs.dot(weights)
      #The dot() function can be called either from the numpy library or from an array instance object. a.dot(b) and np.dot(a,b) works the same.
    return pred

pred = NeuralNetwork(inputs, weights)
print(pred)

4. Multiple outputs with only one input

Multiple outputs may be a simpler extension than multiple inputs. If there are three outputs, the prediction process is the same as three independent neural networks with a single weight. It should be noted that these three predictions are completely independent.

Suppose we are playing a ball game and want to predict a player's mood (happy or sad), the result of the next game, and the proportion of injuries to a team's winning or losing record. See how this example works:

import numpy as np

weights=np.arry([0.3,0.2,0.9])
inputs=0.65

def NeuralNetwork(inputs,weights):
    pred=inputs*weights
    return pred

pred = NeturalNetwork(inputs,weights)
#print(pred)
print("Prediction of injury rate:{}".format(pred[0]))
print("Competition Win/Lose Forecast:{}".format(pred[1]))
print("Sad degree prediction:{}".format(pred[2]))

5. Getting multiple outputs from multiple inputs

Given multiple inputs, the neural network predicts multiple outputs.

Suppose we want to predict a player's mood (happy or sad), the outcome of the next game, and the proportion of injuries to a player in a game by the winning or losing record of a team, the number of fans and the number of players. See how this example works:

import numpy as np

#From left to right are: [whether injured, won, sad]
weights = np.array([[0.02, 0.01, 0.05],    #Win or Lose Record
                    [0.001, 0.025, 0.04],  #Number of fans
                    [0.013, 0.03, 0.001]]) #Number of players
inputs = np.array([0.65, 12.0, 8.0])       #The winning or losing record, the number of fans, the number of players for this team

def NeuralNetwork(inputs, weights):
    pred = inputs.dot(weights)
    return pred

pred = NeuralNetwork(inputs, weights)
# print(pred)
print("Prediction of injury rate:{}".format(pred[0]))
print("Competition Win/Lose Forecast:{}".format(pred[1]))
print("Sad degree prediction:{}".format(pred[2]))

6. Predict further with the predicted results

Neural networks can be stacked! The output from one network can be supplied to another network as input. This is equivalent to two continuous vector matrix multiplications.

The reason for this is that when dealing with complex problems, the single weight matrix is too complex (i.e., the number of parameters is insufficient).

Establish a model of the smallest perfect square equal to n:

# Generate input data
import numpy

num_inputs=1
num_examples=10
features = numpy.random.randint(1, 10, (num_examples, num_inputs))

import math

def PerfectSquare(n): # The Method of Finding the Minimum Complete Square Number
    result = 0
    if n >= 0:
        result = n ** 0.5
        result = math.ceil(result)
        result = result ** 2
    return result

# Minimum Complete Square of Output 1~10
for item in range(1, 10):
    print(PerfectSquare(item), end=" ")

# Generate labels for input
labels = []
for item in features[:,0].tolist():
    labels.append(PerfectSquare(item))
labels = numpy.expand_dims(labels, axis=-1) #Note: You need to add a dimension at the end

# Convert data to tensor
import paddle

features = features.astype('float32')
labels = labels.astype('float32')
train_datas = paddle.to_tensor(features)
y_true = paddle.to_tensor(labels)

#Building a linear regression model
import paddle

model = paddle.nn.Linear(in_features=1, out_features=1) # Use the simplest linear transformation layer

# Define Momentum Optimizer
lr = paddle.optimizer.lr.CosineAnnealingDecay(learning_rate=0.25, T_max=100)
optimizer = paddle.optimizer.Adam(learning_rate=lr,
                                     parameters=model.parameters(),
                                     weight_decay=paddle.regularizer.L2Decay(0.0002))
mse_loss = paddle.nn.MSELoss() # Calculate mean square error of predicted and target values

#model training
for i in range(100):
    y_predict = model(train_datas)
    loss = mse_loss(y_predict, y_true)
    loss.backward()
    optimizer.step()
    optimizer.clear_grad()

print(loss.numpy())

#Model test
infer_features = numpy.random.randint(1, 10, (1, 1)).astype('float32')
infer_features = paddle.to_tensor(infer_features)

fetch_list = model(infer_features)