OpenCV
4.10.0
Open Source Computer Vision
|
Prev Tutorial: Conversion of PyTorch Classification Models and Launch with OpenCV Python
Original author | Anastasia Murzova |
Compatibility | OpenCV >= 4.5 |
In this tutorial you will learn how to:
We will explore the above-listed points by the example of ResNet-50 architecture.
Let's briefly view the key concepts involved in the pipeline of PyTorch models transition with OpenCV API. The initial step in conversion of PyTorch models into cv::dnn::Net is model transferring into ONNX format. ONNX aims at the interchangeability of the neural networks between various frameworks. There is a built-in function in PyTorch for ONNX conversion: torch.onnx.export
. Further the obtained .onnx
model is passed into cv::dnn::readNetFromONNX or cv::dnn::readNet.
To be able to experiment with the below code you will need to install a set of libraries. We will use a virtual environment with python3.7+ for this:
For OpenCV-Python building from source, follow the corresponding instructions from the Introduction to OpenCV.
Before you start the installation of the libraries, you can customize the requirements.txt, excluding or including (for example, opencv-python
) some dependencies. The below line initiates requirements installation into the previously activated virtual environment:
In this part we are going to cover the following points:
The code in this subchapter is located in the samples/dnn/dnn_model_runner
module and can be executed with the line:
The following code contains the description of the below-listed steps:
.onnx
get_pytorch_onnx_model(original_model)
function is based on torch.onnx.export(...)
call:
After the successful execution of the above code we will get the following output:
The proposed in dnn/samples
module dnn_model_runner
allows us to reproduce the above conversion steps for the following PyTorch classification models:
To obtain the converted model, the following line should be executed:
For the ResNet-50 case the below line should be run:
The default root directory for the converted model storage is defined in module CommonConfig
:
Thus, the converted ResNet-50 will be saved in dnn_model_runner/dnn_conversion/models
.
Now we can use models/resnet50.onnx
for the inference pipeline using OpenCV C/C++ API. The implemented pipeline can be found in samples/dnn/classification.cpp. After the build of samples (BUILD_EXAMPLES
flag value should be ON
), the appropriate example_dnn_classification
executable file will be provided.
To provide model inference we will use the below squirrel photo (under CC0 license) corresponding to ImageNet class ID 335:
For the label decoding of the obtained prediction, we also need imagenet_classes.txt
file, which contains the full list of the ImageNet classes.
In this tutorial we will run the inference process for the converted PyTorch ResNet-50 model from the build (samples/build
) directory:
Let's explore classification.cpp
key points step by step:
The model
parameter value is taken from --model
key. In our case, it is resnet50.onnx
.
In this step we use cv::dnn::blobFromImage function to prepare model input. We set Size(rszWidth, rszHeight)
with --initial_width=256 --initial_height=256
for the initial image resize as it's described in PyTorch ResNet inference pipeline.
It should be noted that firstly in cv::dnn::blobFromImage mean value is subtracted and only then pixel values are multiplied by scale. Thus, we use --mean="123.675 116.28 103.53"
, which is equivalent to [0.485, 0.456, 0.406]
multiplied by 255.0
to reproduce the original image preprocessing order for PyTorch classification models:
Here we choose the most likely object class. The classId
result for our case is 335 - fox squirrel, eastern fox squirrel, Sciurus niger: