Mindspire tutorial - 9 reasoning

Posted by fwbbetsy on Thu, 24 Feb 2022 11:08:57 +0100

reasoning

This section is the last section of the initial tutorial. In order to better adapt to different reasoning devices, reasoning is divided into 1) shengteng AI processor reasoning and 2) mobile device reasoning.

Shengteng AI processor reasoning

1 Overview

Ascend AI processor is an energy-efficient and highly integrated AI processor for edge scenes. It can realize image, video and other data analysis and reasoning calculation, and can be widely used in intelligent monitoring, robot, UAV, video server and other scenes. In this section, we will learn how to use mindspire to perform reasoning on the Pentium AI processor.

2 Introduction to reasoning code

First, create a directory to place the reasoning code project, and the model directory is used to store the MindIR model file exported above, test_ The data directory is used to store the pictures to be classified. The reasoning code engineering directory structure is as follows:

└─home directory
    ├── CMakeLists.txt                    // Build script
    ├── README.md                         // instructions
    ├── main.cc                           // Main function
    ├── model
    │   └── resnet50_imagenet.mindir      // MindIR model file
    └── test_data
        ├── ILSVRC2012_val_00002138.JPEG  // Input sample picture 1
        ├── ILSVRC2012_val_00003014.JPEG  // Input sample picture 2
        ├── ...                           // Input sample picture n

Reference the namespaces of mindscore and mindscore:: dataset.

namespace ms = mindspore;
namespace ds = mindspore::dataset;

Initialize the environment, specify the hardware platform used for reasoning, and set the DeviceID.

  • Ascend 910 is kdevicetype ascend910
  • Ascend 310 is kdevicetype ascend310

Here, set the hardware to Ascend 910 and the DeviceID to 0. The example code is as follows:

ms::GlobalContext::SetGlobalDeviceTarget(ms::kDeviceTypeAscend910);
ms::GlobalContext::SetGlobalDeviceID(0);

Load model file:

// Loading MindIR model
auto graph =ms::Serialization::LoadModel(resnet_file, ms::ModelType::kMindIR);
// Building models with graphs
ms::Model resnet50((ms::GraphCell(graph)));
ms::Status ret = resnet50.Build({});

Obtain the input information required by the model:

std::vector<ms::MSTensor> model_inputs = resnet50.GetInputs();

Load picture file:

// Readfile is a function that reads an image
ms::MSTensor ReadFile(const std::string &file);
auto image = ReadFile(image_file);

Image preprocessing:

// Use the CPU operator provided by MindData for image preprocessing

// Encode the input into RGB format
std::shared_ptr<ds::TensorTransform> decode(new ds::vision::Decode());
// Zoom the picture to the specified size
std::shared_ptr<ds::TensorTransform> resize(new ds::vision::Resize({256}));
// Normalized input
std::shared_ptr<ds::TensorTransform> normalize(new ds::vision::Normalize(
    {0.485 * 255, 0.456 * 255, 0.406 * 255}, {0.229 * 255, 0.224 * 255, 0.225 * 255}));
// Crop input picture
std::shared_ptr<ds::TensorTransform> center_crop(new ds::vision::CenterCrop({224, 224}));
// Transform shape (H, W, C) into shape (C, H, W)
std::shared_ptr<ds::TensorTransform> hwc2chw(new ds::vision::HWC2CHW());

// Define a MindData data preprocessing function
ds::Execute preprocessor({decode, resize, normalize, center_crop, hwc2chw});

// Call the data preprocessing function to obtain the processed image
ret = preprocessor(image, &image);

Execute reasoning:

// Create output vector
std::vector<ms::MSTensor> outputs;
// Create input vector
std::vector<ms::MSTensor> inputs;
inputs.emplace_back(model_inputs[0].Name(), model_inputs[0].DataType(), model_inputs[0].Shape(),
                    image.Data().get(), image.DataSize());
// Call the Predict function of the Model for reasoning
ret = resnet50.Predict(inputs, &outputs);

Obtain reasoning results:

// Maximum output probability
std::cout << "Image: " << image_file << " infer result: " << GetMax(outputs[0]) << std::endl;

3 build script

Next, the build script is used to build the user program, because mindspire uses Old C++ ABI Therefore, the user program must be consistent with mindspire, otherwise the compilation of the link will fail.

add_compile_definitions(_GLIBCXX_USE_CXX11_ABI=0)
set(CMAKE_CXX_STANDARD 17)

Add header file search path for compiler:

option(MINDSPORE_PATH "mindspore install path" "")
include_directories(${MINDSPORE_PATH})
include_directories(${MINDSPORE_PATH}/include)

Find the required dynamic library in mindspire:

find_library(MS_LIB libmindspore.so ${MINDSPORE_PATH}/lib)
file(GLOB_RECURSE MD_LIB ${MINDSPORE_PATH}/_c_dataengine*)

Generate the target executable using the specified source file and link the mindspire library for the target file:

add_executable(resnet50_sample main.cc)
target_link_libraries(resnet50_sample ${MS_LIB} ${MD_LIB})
For detailed examples, please refer to: https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/ascend910_resnet50_preprocess_sample/CMakeLists.txt

4 compile inference code

Next, to compile the reasoning code, first enter the project directory ascend910_resnet50_preprocess_sample, set the following environment variables:

If it is Ascend 310 equipment, enter the project directory ascend310_resnet50_preprocess_sample, the following codes take Ascend 910 as an example. 
# Controls the print level of the log 0-DEBUG, 1-INFO, 2-WARNING, 3-ERROR, the default is WARNING level
export GLOG_v=2

# Select Conda environment
LOCAL_ASCEND=/usr/local/Ascend # Root directory of the running package

# lib library on which the running package depends
export LD_LIBRARY_PATH=${LOCAL_ASCEND}/ascend-toolkit/latest/fwkacllib/lib64:${LOCAL_ASCEND}/driver/lib64/common:${LOCAL_ASCEND}/driver/lib64/driver:${LOCAL_ASCEND}/opp/op_impl/built-in/ai_core/tbe/op_tiling:${LD_LIBRARY_PATH}

# lib library that mindspire depends on
export LD_LIBRARY_PATH=`pip3 show mindspore-ascend | grep Location | awk '{print $2"/mindspore/lib"}' | xargs realpath`:${LD_LIBRARY_PATH}

# Configure the necessary environment variables
export TBE_IMPL_PATH=${LOCAL_ASCEND}/ascend-toolkit/latest/opp/op_impl/built-in/ai_core/tbe            # Path of TBE operator
export ASCEND_OPP_PATH=${LOCAL_ASCEND}/ascend-toolkit/latest/opp                                       # OPP path
export PATH=${LOCAL_ASCEND}/ascend-toolkit/latest/fwkacllib/ccec_compiler/bin/:${PATH}                 # Path of TBE operator compilation tool
export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH}                                                       # Python libraries on which TBE depends

Execute the cmake command, where pip3 needs to be modified according to the actual situation:

cmake . -DMINDSPORE_PATH=`pip3 show mindspore-ascend | grep Location | awk '{print $2"/mindspore"}' | xargs realpath`

Then execute the make command to compile.

make

After compilation, in ascend910_ resnet50_ preprocess_ The executable main file will be generated under sample.

5 perform reasoning and view results

After the above operations are completed, we can begin to learn how to use executive reasoning.

First, log in to the Ascend 910 environment, create the model directory, and place the MindIR file resnet50_imagenet.mindir, for example, / home / hwhiaiuser / mindspot_ sample/ascend910_ resnet50_ preprocess_ sample/model. Create test_ Place pictures in the data directory, such as / home / hwhiaiuser / mindspot_ sample/ascend910_ resnet50_ preprocess_ sample/test_data. You can start reasoning:

./resnet50_sample

After execution, test_ All pictures placed in the data directory for reasoning, for example, 2 pictures are placed ImageNet2012 For the pictures with label 0 in the verification set, you can see the reasoning results as follows.

Image: ./test_data/ILSVRC2012_val_00002138.JPEG infer result: 0
Image: ./test_data/ILSVRC2012_val_00003014.JPEG infer result: 0

Mobile device reasoning

1 mindspire Lite overview

Mindspire Lite can complete the model reasoning process in end-side devices such as mobile phones. At present, it supports model reasoning under Android, Ubuntu-x64 and Windows-x64 operating systems, supports various hardware platforms such as end-side ARM CPU, ARM GPU and NPU, and supports two API s such as C + + and Java.

Next, the basic flow of end-to-end reasoning is demonstrated through C++ API. Demo takes the randomly generated data as the input data, executes the reasoning of MobileNetV2 model, and prints the output data.

2 model transformation

Before the model is used for end-to-side reasoning, the format needs to be transformed. Currently, mindspire Lite supports four types of AI frameworks: mindspire, TensorFlow Lite, cafe and onnx.

The following is the result of mindspire training mobilenetv2.mindir Take the model as an example to illustrate the mobilenetv2 used in the Demo How the MS model is generated.

This section expands and explains the operation process of conversion. You can skip this section if you only realize Demo operation.
This section is only for the models used in the Demo. Please refer to the official website for detailed instructions on the use of the conversion tool Reasoning model transformation Chapter.
  • Conversion tool download

Download the conversion tool according to the operating system used Compressed package Extract the tool to the local environment variable, and extract it to the local configuration variable.

  • Conversion tool usage
  • Linux instructions
    Enter converter_ The directory where the Lite executable file is located will download mobilenetv2 Put the mindir model into the same path and input the command in the computer terminal to complete the conversion:
    cpp ./converter_lite --fmk=MINDIR --modelFile=mobilenetv2.mindir --outputFile=mobilenetv2
  • Windows instructions
    Enter converter_ The directory where the Lite executable file is located will download mobilenetv2 Put the mindir model into the same path and input the command in the computer terminal to complete the conversion:
    cpp call converter_lite --fmk=MINDIR --modelFile=mobilenetv2.mindir --outputFile=mobilenetv2
  • Parameter description
    In the process of executing the command, three parameters are set, -fmk represents the original format of the input model, which is set here as MINDIR, that is, the export format of the mindspire framework training model-- modelFile refers to the path of the input model-- outputFile sets the output path of the model, where the converted model is automatically added ms suffix.

3 construction environment and operation

Construction and operation of Linux system

  • Compile build

In mindspot / LITE / examples / quick_ start_ Execute the build script under the CPP directory, and you will be able to automatically download relevant files and compile the Demo.

bash bash build.sh

  • Executive reasoning

After compiling and building, enter mindspot / LITE / examples / quick_ start_ CPP / build directory and execute the following commands to experience the mindspire Lite reasoning MobileNetV2 model.

bash ./mindspore_quick_start_cpp ../model/mobilenetv2.ms

After execution, the following results will be obtained: print out the name of Tensor, the size of Tensor, the number of tensors and the first 50 data:

shell tensor name is:Default/head-MobileNetV2Head/Softmax-op204 tensor size is:4000 tensor elements num is:1000 output data is:5.26823e-05 0.00049752 0.000296722 0.000377607 0.000177048 .......

Windows system construction and operation

  • Compile build
  • Library Download: please manually download the reasoning framework of mindspire Lite model whose hardware platform is CPU and operating system is Windows-x64 mindspore-lite-{version}-win-x64.zip , unzip the libmindspot lite.exe file in the information / lib directory A copy to mindspot / LITE / examples / quick_ start_ Copy CPP / lib directory and information / include directory to mindspot / LITE / examples / quick_ start_ CPP / include directory.
  • Model Download: please download relevant model files manually mobilenetv2.ms , and copy it to mindspot / LITE / examples / quick_ start_ CPP / model directory.
You can choose to use the mobilenetv2 obtained in the model transformation section MS model file.
  • Compile: in mindspot / LITE / examples / quick_ start_ Execute the build script under the CPP directory, and you will be able to automatically download relevant files and compile the Demo.

bash call build.bat

  • Executive reasoning

After compiling and building, enter mindspot / LITE / examples / quick_ start_ CPP / build directory and execute the following commands to experience the mindspire Lite reasoning MobileNetV2 model.

bash call ./mindspore_quick_start_cpp.exe ../model/mobilenetv2.ms

After execution, the following results will be obtained: print out the name of Tensor, the size of Tensor, the number of tensors and the first 50 data:

shell tensor name is:Default/head-MobileNetV2Head/Softmax-op204 tensor size is:4000 tensor elements num is:1000 output data is:5.26823e-05 0.00049752 0.000296722 0.000377607 0.000177048 .......

4 reasoning code analysis

The reasoning process in the Demo source code is analyzed below to show the specific usage of C++ API.

Model loading

First, read the mindspot Lite model from the file system, and import the model through the mindspot:: Lite:: model:: import function for parsing.

// Read model file
size_t size = 0;
char *model_buf = ReadFile(model_path, &size);
if (model_buf == nullptr) {
  std::cerr << "Read model file failed." << std::endl;
  return RET_ERROR;
}
// Loading model
auto model = mindspore::lite::Model::Import(model_buf, size);
delete[](model_buf);
if (model == nullptr) {
  std::cerr << "Import model file failed." << std::endl;
  return RET_ERROR;
}

Model compilation

Model compilation mainly includes the steps of creating configuration context, creating session, graph compilation and so on.

mindspore::session::LiteSession *Compile(mindspore::lite::Model *model) {
  // Initialize context
  auto context = std::make_shared<mindspore::lite::Context>();
  if (context == nullptr) {
    std::cerr << "New context failed while." << std::endl;
    return nullptr;
  }

  // Create session
  mindspore::session::LiteSession *session = mindspore::session::LiteSession::CreateSession(context.get());
  if (session == nullptr) {
    std::cerr << "CreateSession failed while running." << std::endl;
    return nullptr;
  }

  // Graph compilation
  auto ret = session->CompileGraph(model);
  if (ret != mindspore::lite::RET_OK) {
    delete session;
    std::cerr << "Compile failed while running." << std::endl;
    return nullptr;
  }

  // Note: if you use model - > free (), the model cannot be compiled again
  if (model != nullptr) {
    model->Free();
  }
  return session;
}

Model reasoning

Model reasoning mainly includes the steps of inputting data, executing reasoning and obtaining output. The input data in this example is generated through random data construction, and finally the output results after executing reasoning are printed out.

int Run(mindspore::session::LiteSession *session) {
  auto inputs = session->GetInputs();
  auto ret = GenerateInputDataWithRandom(inputs);
  if (ret != mindspore::lite::RET_OK) {
    std::cerr << "Generate Random Input Data failed." << std::endl;
    return ret;
  }

  ret = session->RunGraph();
  if (ret != mindspore::lite::RET_OK) {
    std::cerr << "Inference error " << ret << std::endl;
    return ret;
  }

  auto out_tensors = session->GetOutputs();
  for (auto tensor : out_tensors) {
    std::cout << "tensor name is:" << tensor.first << " tensor size is:" << tensor.second->Size()
              << " tensor elements num is:" << tensor.second->ElementsNum() << std::endl;
    auto out_data = reinterpret_cast<float *>(tensor.second->MutableData());
    std::cout << "output data is:";
    for (int i = 0; i < tensor.second->ElementsNum() && i <= 50; i++) {
      std::cout << out_data[i] << " ";
    }
    std::cout << std::endl;
  }
  return mindspore::lite::RET_OK;
}

Memory release

When you do not need to use the mindspire Lite reasoning framework, you need to release the created LiteSession and Model.

// Delete model cache
delete model;
// Delete session cache
delete session;

Topics: Machine Learning AI Deep Learning

Hot Topics

©2024 Programmer Think