# Case: Implementing linear regression

Posted by Derleek on Sat, 20 Jun 2020 03:41:02 +0200

# Rimeng Society

AI: Keras PyTorch MXNet TensorFlow PaddlePaddle Deep Learning Actual Warfare (Update from time to time)

# 2.4 Cases: Realizing Linear Regression

## Learning Objectives

• target
• Applying op's name parameter to modify op's name
• Apply variable_scope implementation diagram program scope addition
• Tracking display of tensor values using scalar or histogram
• Apply merge_all implements the merging of tensor values
• Apply add_Summy implements writing tensor values to files
• applicationTf.app.flagsImplement command line parameter addition and use
• Apply reduce_mean, square to realize mean square error calculation
• Optimize loss with minimize function
• Know gradient explosions and common solutions
• application
• Implementing linear regression models
• Content Preview
• Review of 2.6.1 Linear Regression Principle
• 2.6.2 Cases: Training for Linear Regression
• 1 Increase variable display
• 2 Increase Namespace
• 3 Save and Load Model
• 4 Use of command line parameters

## Review of 2.4.1 Linear Regression Principle

Regression model based on data, w1x1+w2x2+.... +b = y, establishes the error between the true value and the predicted value, and uses gradient descent optimization to obtain the weight and offset corresponding to the minimum loss.Finally, the weights and offset parameters of the model are determined.Finally, these parameters can be used to make predictions.

## 2.4.2 Cases: Training for Linear Regression

### 1 Case Determination

• Assume that 100 points are randomly assigned with only one feature
• The distribution of the data itself is y = 0.8 * x + 0.7

The rule of data distribution is determined here to compare the training parameters with the actual parameters (i.e. 0.8 and 0.7) and whether they are accurate or not.

### 2-step analysis

• 1 Prepare the dataset: y = 0.8x + 0.7 100 samples
• 2 Establish a linear model
• Random Initialization W1 and b1
• y = W.X + b, objective: find the weight W and offset B
• 3 Determine loss function (error between predicted and true values) - mean square error
• 4 Gradient Down Optimized Loss: Need to specify learning rate (hyperparameter)

### 4 Implement full functionality

Related API s

operation

• Matrix operations
• tf.matmul(x, w)
• square
• tf.square(error)
• mean value
• tf.reduce_mean(error)
```error = tf.reduce_mean(tf.square(y_predict - y_true))
```

• learning_rate: learning rate, typically a small value between 0 and 1
• method:
• minimize(loss)
```# Tensorflow provides a variety of optimization methods, no need to implement gradient descent operation principle formulas by itself, custom use of optimization types
```

Complete Code Process Analysis:

```import tensorflow as tf
import os

def linear_regression():
"""
//Self-realizing linear regression
:return: None
"""
# 1) Prepare the dataset: y = 0.8x + 0.7 100 samples
# Eigenvalue X, Target Value y_true
X = tf.random_normal(shape=(100, 1), mean=2, stddev=2)
# y_true [100, 1]
# Matrix operation X(100,1)*(1,1)= y_true(100, 1)
y_true = tf.matmul(X, [[0.8]]) + 0.7
# 2) Establish a linear model:
# y = W.X + b, objective: find the weight W and offset B
# 3) Initialize W1 and b1 randomly
weights = tf.Variable(initial_value=tf.random_normal(shape=(1, 1)))
bias = tf.Variable(initial_value=tf.random_normal(shape=(1, 1)))
y_predict = tf.matmul(X, weights) + bias
# 4) Determine the loss function (the error between the predicted value and the true value) - the mean square error
error = tf.reduce_mean(tf.square(y_predict - y_true))
# 5) Optimized loss of gradient descent: Need to specify learning rate (hyperparameter)
# W2 = W1 - Learning rate* (Direction)
# b2 = b1 - learning rate* (direction)

# initialize variable
init = tf.global_variables_initializer()
# Open a session for training
with tf.Session() as sess:
# Run Initialization Variable Op
sess.run(init)
print("Randomly initialized weights are%f， Offset to%f" % (weights.eval(), bias.eval()))
# Training model
for i in range(100):
sess.run(optimizer)
print("No.%d The step error is%f，Weight is%f， Offset to%f" % (i, error.eval(), weights.eval(), bias.eval()))

return None
```

### Observation of trainable settings for 6 variables

Parameter role of trainable, specifying whether or not to train

```weight = tf.Variable(tf.random_normal([1, 1], mean=0.0, stddev=1.0), name="weights", trainable=False)
```

• Increase Namespace
• Command Line Parameter Settings

### 2 Increase Namespace

Is the code structure clearer, Tensorboard diagram structure clearer

```with tf.variable_scope("lr_model"):
```
```def linear_regression():
# 1) Prepare the dataset: y = 0.8x + 0.7 100 samples
# Eigenvalue X, Target Value y_true
with tf.variable_scope("original_data"):
X = tf.random_normal(shape=(100, 1), mean=2, stddev=2, name="original_data_x")
# y_true [100, 1]
# Matrix operation X(100,1)*(1,1)= y_true(100, 1)
y_true = tf.matmul(X, [[0.8]], name="original_matmul") + 0.7
# 2) Establish a linear model:
# y = W.X + b, objective: find the weight W and offset B
# 3) Initialize W1 and b1 randomly
with tf.variable_scope("linear_model"):
weights = tf.Variable(initial_value=tf.random_normal(shape=(1, 1)), name="weights")
bias = tf.Variable(initial_value=tf.random_normal(shape=(1, 1)), name="bias")
y_predict = tf.matmul(X, weights, name="model_matmul") + bias
# 4) Determine the loss function (the error between the predicted value and the true value) - the mean square error
with tf.variable_scope("loss"):
error = tf.reduce_mean(tf.square(y_predict - y_true), name="error_op")
# 5) Optimized loss of gradient descent: Need to specify learning rate (hyperparameter)
# W2 = W1 - Learning rate* (Direction)
# b2 = b1 - learning rate* (direction)
with tf.variable_scope("gd_optimizer"):

# 2) Collect variables
tf.summary.scalar("error", error)
tf.summary.histogram("weights", weights)
tf.summary.histogram("bias", bias)

# 3) Merge variables
merge = tf.summary.merge_all()

# initialize variable
init = tf.global_variables_initializer()
# Open a session for training
with tf.Session() as sess:
# Run Initialization Variable Op
sess.run(init)
print("Randomly initialized weights are%f， Offset to%f" % (weights.eval(), bias.eval()))
# 1) Create event file
file_writer = tf.summary.FileWriter(logdir="./summary", graph=sess.graph)
# Training model
for i in range(100):
sess.run(optimizer)
print("No.%d The step error is%f，Weight is%f， Offset to%f" % (i, error.eval(), weights.eval(), bias.eval()))
# 4) Run the merge variable op
summary = sess.run(merge)

return None
```

### 3 Save and Load Model

• tf.train.Saver(var_list=None,max_to_keep=5)
• Save and load the model (Save file format: checkpoint file)
• var_list: Specifies the variables to be saved and restored.It can be passed as a dict or as a list.
• max_to_keep: Indicates the maximum number of recent checkpoint files to keep.When you create a new file, older files are deleted.If none or 0, all checkpoint files are preserved.The default is 5 (that is, keep the latest five checkpoint files.)

Use

```For example:
Specify Catalog + Model Name
saver.save(sess, '/tmp/ckpt/test/myregression.ckpt')
saver.restore(sess, '/tmp/ckpt/test/myregression.ckpt')
```

To determine if a model exists, specify a directory directly

```checkpoint = tf.train.latest_checkpoint("./tmp/model/")

saver.restore(sess, checkpoint)
```

### 4 Use of command line parameters

• 1,tf.app.flags., there is a FLAGS flag in flags that can be called into our programs

Flag_as specifically defined aboveName

• 2. PassTf.app.run() Start main(argv) function
```# Define some common command line parameters
# Training steps
tf.app.flags.DEFINE_integer("max_step", 0, "Number of steps in training model")
# Define Path to Model
tf.app.flags.DEFINE_string("model_dir", " ", "Path to model save+Model name")

# Define Get Command Line Parameters
FLAGS = tf.app.flags.FLAGS

# Open Training
# Number of training steps (depending on model size)
for i in range(FLAGS.max_step):
sess.run(train_op)
```

### Complete Code

```import tensorflow as tf
import os

tf.app.flags.DEFINE_string("model_path", "./linear_regression/", "Path and file name to save the model")
FLAGS = tf.app.flags.FLAGS

def linear_regression():
# 1) Prepare the dataset: y = 0.8x + 0.7 100 samples
# Eigenvalue X, Target Value y_true
with tf.variable_scope("original_data"):
X = tf.random_normal(shape=(100, 1), mean=2, stddev=2, name="original_data_x")
# y_true [100, 1]
# Matrix operation X(100,1)*(1,1)= y_true(100, 1)
y_true = tf.matmul(X, [[0.8]], name="original_matmul") + 0.7
# 2) Establish a linear model:
# y = W.X + b, objective: find the weight W and offset B
# 3) Initialize W1 and b1 randomly
with tf.variable_scope("linear_model"):
weights = tf.Variable(initial_value=tf.random_normal(shape=(1, 1)), name="weights")
bias = tf.Variable(initial_value=tf.random_normal(shape=(1, 1)), name="bias")
y_predict = tf.matmul(X, weights, name="model_matmul") + bias
# 4) Determine the loss function (the error between the predicted value and the true value) - the mean square error
with tf.variable_scope("loss"):
error = tf.reduce_mean(tf.square(y_predict - y_true), name="error_op")
# 5) Optimized loss of gradient descent: Need to specify learning rate (hyperparameter)
# W2 = W1 - Learning rate* (Direction)
# b2 = b1 - learning rate* (direction)
with tf.variable_scope("gd_optimizer"):

# 2) Collect variables
tf.summary.scalar("error", error)
tf.summary.histogram("weights", weights)
tf.summary.histogram("bias", bias)

# 3) Merge variables
merge = tf.summary.merge_all()

# initialize variable
init = tf.global_variables_initializer()

# Open a session for training
with tf.Session() as sess:
# Run Initialization Variable Op
sess.run(init)
# Untrained weights and offsets
print("Randomly initialized weights are%f， Offset to%f" % (weights.eval(), bias.eval()))
# Load the model when a checkpoint file exists

# 1) Create event file
file_writer = tf.summary.FileWriter(logdir="./summary", graph=sess.graph)
# Training model
for i in range(100):
sess.run(optimizer)
print("No.%d The step error is%f，Weight is%f， Offset to%f" % (i, error.eval(), weights.eval(), bias.eval()))
# 4) Run the merge variable op
summary = sess.run(merge)

return None

def main(argv):
print("This is main function")
print(argv)
print(FLAGS.model_path)
linear_regression()

if __name__ == "__main__":
tf.app.run()
```

### Job: Change process-oriented to object-oriented

Reference Code

```# Self-implementation of a linear regression case using tensorflow

# Define some common command line parameters
# Training steps
tf.app.flags.DEFINE_integer("max_step", 0, "Number of steps in training model")
# Define Path to Model
tf.app.flags.DEFINE_string("model_dir", " ", "Path to model save+Model name")

FLAGS = tf.app.flags.FLAGS

class MyLinearRegression(object):
"""
//Self-realizing linear regression
"""
def __init__(self):
pass

def inputs(self):
"""
//Getting eigenvalue target data
:return:
"""
x_data = tf.random_normal([100, 1], mean=1.0, stddev=1.0, name="x_data")
y_true = tf.matmul(x_data, [[0.7]]) + 0.8

return x_data, y_true

def inference(self, feature):
"""
//Modeling from input data
:param feature:
:param label:
:return:
"""
with tf.variable_scope("linea_model"):
# 2. Establish a regression model to analyze the number of features of others'data - > the number of weights, offset b
# Since the gradient descent algorithm is optimized, random parameters, weights, and offsets are initially given
# Optimized parameters must be defined using variable op
# Variable Initialization Weights and Offsets
# weight 2-dimensional [1, 1] bias 
# There is a trainable parameter in the variable op that determines whether to train or not
self.weight = tf.Variable(tf.random_normal([1, 1], mean=0.0, stddev=1.0),
name="weights")

self.bias = tf.Variable(0.0, name='biases')

# Establish regression formulas to get prediction results
y_predict = tf.matmul(feature, self.weight) + self.bias

return y_predict

def loss(self, y_true, y_predict):
"""
//Target value and true value to calculate loss
:return: loss
"""
# 3. Find out the loss between our model and the real data
# Mean Square Error Formula
loss = tf.reduce_mean(tf.square(y_true - y_predict))

return loss

def merge_summary(self, loss):

# 1. Value of collection tensor
tf.summary.scalar("losses", loss)

tf.summary.histogram("w", self.weight)
tf.summary.histogram('b', self.bias)

# 2. Merge Variables
merged = tf.summary.merge_all()

return merged

def sgd_op(self, loss):
"""
//Get training OP
:return:
"""
# 4. Optimize using gradient descent optimizer
# Fill-in learning rate: 0-1 learning rate is very small,
# The learning rate determines how many steps you lose to reach
# Minimize loss

return train_op

def train(self):
"""
//Training model
:param loss:
:return:
"""

g = tf.get_default_graph()

with g.as_default():

x_data, y_true = self.inputs()

y_predict = self.inference(x_data)

loss = self.loss(y_true, y_predict)

train_op = self.sgd_op(loss)

# Collect observed result values
merged = self.merge_summary(loss)

saver = tf.train.Saver()

with tf.Session() as sess:

sess.run(tf.global_variables_initializer())

# Without training, model parameter values
print("Initialized weight:%f, Offset:%f" % (self.weight.eval(), self.bias.eval()))

# Open Training
# Number of training steps (depending on model size)
for i in range(FLAGS.max_step):

sess.run(train_op)

# Generate event file, view diagram structure
file_writer = tf.summary.FileWriter("./tmp/summary/", graph=sess.graph)

print("Training session%d Loss after step:%f, Weight:%f, Offset:%f" % (
i,
loss.eval(),
self.weight.eval(),
self.bias.eval()))

# Run the results of collecting variables
summary = sess.run(merged)