Teach you how to turn yolov3/yolov4 into caffe model step by step

Posted by blawson7 on Sun, 23 Jan 2022 06:44:15 +0100

In practical work, target detection yolov3 or yolov4 model is transplanted to AI chip, which often needs to be converted into caffe1 X model, you may have more or less this demand. For example, Huawei Hisilicon NNIE only supports caffe1 X model, so if yolov3/yolov4 model wants to be deployed on Hisilicon chip, it should be converted to caffe1 The X model is required.

Today, I'd like to write the detailed steps of transforming yolov3/yolov4 model into caffe model.

0. System environment

Ubuntu 16.04
cuda 9.x

1. Deploy caffe environment

The conventional method is to download the source code of Caffe and compile and install it. However, this method is complex and not easy to succeed. A simpler way is to download caffe1.0 directly Docker image of X. This method requires you to install NVIDIA docker in Ubuntu in advance.

Open the docker hub website:

https://registry.hub.docker.com/

Search caffe and see the image:

Click inside to see the drop-down command of the image:

sudo docker pull bvlc/caffe

Open the Ubuntu terminal, enter the drop-down command above, and download the docker image of caffe.

After downloading, the terminal inputs the command:

sudo docker images

You can see the downloaded caffe image.

To create a container for a mirror:

docker run --runtime=nvidia --name caffe -i -t bvlc/caffe /bin/bash

Here we give the container the name caffe. You can set it freely and view each container according to the command 'sudo docker ps -a'.

In this way, we enter caffe1 X is in the docker container.

The container has already deployed caffe1 x，caffe1. The X path is:

2. Modification of Caffe source code

Because the official caffe1 The X framework does not support the upsample layer of yolo3/yolov4, so you need to manually add the upsample layer and modify the Caffe source code.

Clone the conversion tool project on GitHub:

git clone https://github.com/ChenYingpeng/darknet2caffe.git

Set darknet2cafe / caffe_ layers/mish_ Mish under layer_ layer. HPP files and darknet2cafe / tree / Master / caffe_ layers/upsample_ Upsample under layer_ layer. Copy HPP to container path: / opt / Cafe / include / Cafe / layers.

Set darknet2cafe / caffe_ layers/mish_ Mish under layer_ layer. cpp,mish_layer.cu file and darknet2cafe / tree / Master / Cafe_ layers/upsample_ Upsample under layer_ layer. cpp,upsample_layer.cu copy to container path: / opt / Cafe / SRC / Cafe / layers /.

Set darknet2cafe / caffe_ layers/pooling_ Pooling under layer_ layer. Copy CPP to the path of the container: / opt / Cafe / SRC / Cafe / layers /.

Then, open the cafe file in the container: / opt / Cafe / SRC / Cafe / proto / cafe proto. Follow the procedure below to modify the corresponding fields.

// LayerParameter next available layer-specific ID: 147 (last added: recurrent_param)
message LayerParameter {
  optional TileParameter tile_param = 138;
  optional VideoDataParameter video_data_param = 207;
  optional WindowDataParameter window_data_param = 129;
++optional UpsampleParameter upsample_param = 149; //added by chen for Yolov3, make sure this id 149 not the same as before.
++optional MishParameter mish_param = 150; //added by chen for yolov4,make sure this id 150 not the same as before.
}


// added by chen for YoloV3
++message UpsampleParameter{
++  optional int32 scale = 1 [default = 1];
++}


// Message that stores parameters used by MishLayer
++message MishParameter {
++  enum Engine {
++    DEFAULT = 0;
++    CAFFE = 1;
++    CUDNN = 2;
++  }
++  optional Engine engine = 2 [default = DEFAULT];
++}

Where, + + indicates that the line is added.

3. caffe recompile

After modifying some of the source code of caffe, you need to recompile caffe.

Enter the / opt/caffe/build directory and enter the following command:

make clean
make all -j8
make pycaffe -j8

After caffe recompilation, you can convert yolov3/yolov4 models to caffe.

4. Model transformation

Prepare the configuration file and weight file of our existing yolov3 model, for example: yolov3 CFG and yolov3 weights. In the darknet2caffe directory, enter the following command:

python darknet2caffe.py ./yolov3.cfg ./yolov3.weights ./yolov3.prototxt ./yolov3.caffemodel

If the output is similar to the following statement, the conversion is successful!

I0522 10:19:19.015708 25251 net.cpp:228] layer1-act does not need backward computation.
I0522 10:19:19.015712 25251 net.cpp:228] layer1-scale does not need backward computation.
I0522 10:19:19.015714 25251 net.cpp:228] layer1-bn does not need backward computation.
I0522 10:19:19.015718 25251 net.cpp:228] layer1-conv does not need backward computation.
I0522 10:19:19.015722 25251 net.cpp:228] input does not need backward computation.
I0522 10:19:19.015725 25251 net.cpp:270] This network produces output layer139-conv
I0522 10:19:19.015731 25251 net.cpp:270] This network produces output layer150-conv
I0522 10:19:19.015736 25251 net.cpp:270] This network produces output layer161-conv
I0522 10:19:19.015911 25251 net.cpp:283] Network initialization done.
unknow layer type yolo 
unknow layer type yolo 
save prototxt to ./yolov3.prototxt
save caffemodel to ./yolov3.caffemodel

Among them, yolov3 Prototext and yolov3 Caffemodel is the transformed caffe model.

At this point, yolov3/yolov4 is converted to caffe model!

Programmer Think