About Yolo 5's model training

Author: fat orange

preface

With the continuous development of artificial intelligence, the technology of model training is becoming more and more important. Many people have opened the chapter of learning model training. Now I will introduce the basic content of model training.

preparation

• data set
• labelImg
• Yolov5 source code

About datasets

discriminative power

What I want to say about data sets is that the higher the discrimination, the better. Because the greater the degree of distinction between each of our photos, the higher the efficiency of our computer learning. For example, if we learn a knowledge, we may not know a new type of question if we keep doing one type of example. Similarly, the higher the distinction between each of our photos, the better our computer will learn.

Training set and validation set

In supervised machine learning, we often talk about training set, validation set and test set. The distinction between these three sets may be confusing. In particular, some people will not know the difference between validation set and test set.

If we already have a large labeled data set and want to complete the test of a supervised model, we usually use the method of uniform random sampling to divide the data set into training set, verification set and test set. The three sets cannot have intersection. The common ratio is 8:1:1. Of course, the ratio is artificial. From this point of view, the three sets are identically distributed.

With the model, the training set is used to train parameters. To be precise, it is generally used for gradient descent. The validation set is basically used to test the accuracy of the current model after each epoch is completed. Because there is no intersection between the verification set and the training set, this accuracy is reliable.

How to distinguish between training set and verification set

To be exact, they were all ordinary photos before we classified them, It's just that we gave him other meanings (in other words, I say who is the verification set). So what proportion should we divide? We can divide according to the ratio of 7:3. However, we can't count too few pictures. Generally, we need at least 100 pictures to produce an effect. Then we put them in two folders respectively, and the folder name can be determined by ourselves . It can also be defined according to my.

|-train

| - images / / image storage of training set

| - labels / / txt file of training set

|-valid

| - images / / verify the image storage of the set

| - labels / / txt file of validation set

labelImg

Batch labeling

Use the Open Dir button to open the folder of the pictures to be marked.
Use the Change Save Dir button to open the folder where the annotation file is stored.
Use the w shortcut key or click create RectBox to start labeling. After labeling, you need to save it. After labeling all pictures, you can get the labeling file. The contents of the marked file represent the objects contained in the marked picture.

XML file to TXT file

When converting XML files, we can use written scripts to convert them.

The code is as follows:

import xml.etree.ElementTree as ET
import pickle
import os
from os import listdir,getcwd
from os.path import join
import glob

classes = ['Aubergine','Chili','Bone','Newspaper','Waste_Butteries','Medical_Cotton_Swabs','Pesticides','Leftover_Food','Radioactive_Material','Expired_Medicines']
#Write the name of the class we want to train
def convert(size,box):
    dw = 1./size[0]
    dh = 1./size[1]
    x = (box[0] + box[1])/2.0
    y = (box[2] + box[3])/2.0
    w = box[1] - box[0]
    h = box[3] - box[2]
    x = x * dw
    w = w * dw
    y = y * dh
    h = h * dh
    return (x,y,w,h)

def convert_annotation(image_name):
    in_file = open('./valid/labels/'+image_name[:-3]+'xml')
    out_file = open('./valid/labels/'+image_name[:-3]+'txt','w')
    tree = ET.parse(in_file)
    root = tree.getroot()
    size = root.find('size')
    w = int(size.find('width').text)
    h = int(size.find('height').text)

    for obj in root.iter('object'):
        cls = obj.find('name').text
        if cls not in classes:
            print(cls)
            continue
        cls_id = classes.index(cls)
        xmlbox = obj.find('bndbox')
        b = (float(xmlbox.find('xmin').text),float(xmlbox.find('xmax').text),float(xmlbox.find('ymin').text),float(xmlbox.find('ymax').text))
        bb = convert((w,h),b)
        out_file.write(str(cls_id)+" "+" ".join([str(a) for a in bb]) + '\n')

wd = getcwd()

if __name__ == "__main__":
    for image_path in glob.glob("./valid/images/*.jpg"):
        image_name = image_path.split('\\')[1]
        convert_annotation(image_name)

Then we put the converted txt file under our corresponding labels folder.

Compilation of yaml

YAML (YAML Ain't Markup Language) takes data as the center and is more suitable for configuration files than json and xml.

Our yaml will be placed in the folder on the data set layer.

|-train

|-valid

|-xxxx.yaml

yaml's writing style

train: ../Rubbish_Data/train/images
val: ../Rubbish_Data/valid/images

nc: 10
names: ['Aubergine','Chili','Bone','Newspaper','Waste_Butteries','Medical_Cotton_Swabs','Pesticides','Leftover_Food','Radioactive_Material','Expired_Medicines']

Start preparing for training

Add the Parameters file to the train file in pychart

You can refer to the following.

--data
../Rubbish_Data/Rubbish_data.yaml
--cfg
models/yolov5s.yaml
--batch-size
120

Finally, start running the training code.