 |
OpenCV
4.10.0-dev
Open Source Computer Vision
|
Loading...
Searching...
No Matches
|
OpenCV
4.10.0-dev
Open Source Computer Vision
|
Prev Tutorial: OpenCV usage with OpenVINO
Next Tutorial: How to run deep networks in browser
| Original author | Alessandro de Oliveira Faria |
| Extended by | Abduragim Shtanchaev |
| Compatibility | OpenCV >= 4.9.0 |
Deploying pre-trained models is a common task in machine learning, particularly when working with hardware that does not support certain frameworks like PyTorch. This guide provides a comprehensive overview of exporting pre-trained YOLO family models from PyTorch and deploying them using OpenCV's DNN framework. For demonstration purposes, we will focus on the YOLOX model, but the methodology applies to other supported models.
This support includes pre and post-processing routines specific to these models. While other older version of YOLO are also supported by OpenCV in Darknet format, they are out of the scope of this tutorial.
Assuming that we have successfully trained YOLOX model, the subsequent step involves exporting and running this model with OpenCV. There are several critical considerations to address before proceeding with this process. Let's delve into these aspects.
Understanding the nature of inputs and outputs associated with YOLO family detectors is pivotal. These detectors, akin to most Deep Neural Networks (DNN), typically exhibit variation in input sizes contingent upon the model's scale.
| Model Scale | Input Size |
|---|---|
| Small Models 1 | 416x416 |
| Midsize Models 2 | 640x640 |
| Large Models 3 | 1280x1280 |
This table provides a quick reference to understand the different input dimensions commonly used in various YOLO models inputs. These are standard input shapes. Make sure you use input size that you trained model with, if it is differed from from the size mentioned in the table.
The next critical element in the process involves understanding the specifics of image pre-processing for YOLO detectors. While the fundamental pre-processing approach remains consistent across the YOLO family, there are subtle yet crucial differences that must be accounted for to avoid any degradation in performance. Key among these are the resize type and the padding value applied post-resize. For instance, the YOLOX model utilizes a LetterBox resize method and a padding value of 114.0. It is imperative to ensure that these parameters, along with the normalization constants, are appropriately matched to the model being exported.
Regarding the model's output, it typically takes the form of a tensor with dimensions [BxNxC+5] or [BxNxC+4], where 'B' represents the batch size, 'N' denotes the number of anchors, and 'C' signifies the number of classes (for instance, 80 classes if the model is trained on the COCO dataset). The additional 5 in the former tensor structure corresponds to the objectness score (obj), confidence score (conf), and the bounding box coordinates (cx, cy, w, h). Notably, the YOLOv8 model's output is shaped as [BxNxC+4], where there is no explicit objectness score, and the object score is directly inferred from the class score. For the YOLOX model, specifically, it is also necessary to incorporate anchor points to rescale predictions back to the image domain. This step will be integrated into the ONNX graph, a process that we will detail further in the subsequent sections.
Now that we know know the parameters of the pre-precessing we can go on and export the model from Pytorch to ONNX graph. Since in this tutorial we are using YOLOX as our sample model, lets use its export for demonstration purposes (the process is identical for the rest of the YOLO detectors except YOLOv10 model, see details on how to export it later in the post). To exporting YOLOX we can just use export script. Particularly we need following commands:
NOTE: Here --decode_in_inference is to include anchor box creation in the ONNX graph itself. It sets this value to True, which subsequently includes anchor generation function.
Below we demonstrated the minimal version of the export script (which could be used for models other than YOLOX) in case it is needed. However, usually each YOLO repository has predefined export script.
In oder to run YOLOv10 one needs to cut off postporcessing with dynamic shapes from torch and then convert it to ONNX. If someone is looking for on how to cut off the postprocessing, there is this forked branch from official YOLOv10. The forked branch cuts of the postprocessing by returning output of the model before postprocessing procedure itself. To convert torch model to ONNX follow this proceduce.
By default --model="yolov10s" and --imgsz=(480,640). This will generate file yolov10s.onnx, which can be use for inference in OpenCV
Once we have our ONNX graph of the model, we just simply can run with OpenCV's sample. To that we need to make sure:
build directory--input is not provided camera with index 0 will used by default.Here mean, scale, padvalue, paddingmode should exactly match those that we discussed in pre-processing section in order for the model to match result in PyTorch
To demonstrate how to run OpenCV YOLO samples without your own pretrained model, follow these instructions:
-DBUILD_EXAMPLES=ON flag.Run the YOLOX detector(with default values):
This will execute the YOLOX detector with your camera. For YOLOv8 (for instance), follow these additional steps:
For YOLOv10, follow these steps:
This will run YOLOv10 detector on first camera found on your system. If you want to run it on a image/video file, you can use --input option to specify the path to the file.
VIDEO DEMO:
Sometimes there is a need to make some custom adjustments in the inference pipeline. With OpenCV DNN module this is also quite easy to achieve. Below we will outline the sample implementation details:
All post-processing steps are implemented in function yoloPostProcess. Please pay attention, that NMS step is not included into onnx graph. Sample uses OpenCV function for it.
1.9.8