In this tutorial you will learn how to use the 'dnn_superres' interface to upscale an image via pre-trained neural networks. It works in C++ and Python.
Building
When building OpenCV, run the following command to build all the contrib module:
cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/
Or only build the dnn_superres module:
cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/dnn_superres
Or make sure you check the dnn_superres module in the GUI version of CMake: cmake-gui.
Source Code of the sample
You can run the sample code by doing
<path_of_your_opencv_build_directory>/bin/example_dnn_superres_dnn_superres <path_to_image.png> <algo_string> <upscale_int> <model_path.pb>
Example:
/home/opencv/build/bin/example_dnn_superres_dnn_superres /home/image.png edsr 2 /home/EDSR_x2.pb
15 using namespace dnn_superres;
17 int main(
int argc,
char *argv[])
22 cout <<
"usage: Arg 1: image | Path to image" << endl;
23 cout <<
"\t Arg 2: algorithm | bilinear, bicubic, edsr, espcn, fsrcnn or lapsrn" << endl;
24 cout <<
"\t Arg 3: scale | 2, 3 or 4 \n";
25 cout <<
"\t Arg 4: path to model file \n";
29 string img_path = string(argv[1]);
30 string algorithm = string(argv[2]);
31 int scale = atoi(argv[3]);
35 path = string(argv[4]);
39 Mat original_img(img);
42 std::cerr <<
"Couldn't load image: " << img <<
"\n";
51 if( algorithm ==
"bilinear" ){
54 else if( algorithm ==
"bicubic" )
58 else if( algorithm ==
"edsr" || algorithm ==
"espcn" || algorithm ==
"fsrcnn" || algorithm ==
"lapsrn" )
61 sr.setModel(algorithm, scale);
62 sr.upsample(img, img_new);
65 std::cerr <<
"Algorithm not recognized. \n";
68 if ( img_new.
empty() )
70 std::cerr <<
"Upsampling failed. \n";
73 cout <<
"Upsampling succeeded. \n";
Mat imread(const String &filename, int flags=IMREAD_COLOR)
Loads an image from a file.
void imshow(const String &winname, InputArray mat)
Displays an image in the specified window.
"black box" representation of the file storage associated with a file on disk.
Definition: affine.hpp:51
void namedWindow(const String &winname, int flags=WINDOW_AUTOSIZE)
Creates a window.
Size2i Size
Definition: types.hpp:370
void resize(int src_type, const uchar *src_data, size_t src_step, int src_width, int src_height, uchar *dst_data, size_t dst_step, int dst_width, int dst_height, double inv_scale_x, double inv_scale_y, int interpolation)
the user cannot resize the window, the size is constrainted by the image displayed.
Definition: highgui.hpp:188
n-dimensional dense array class
Definition: mat.hpp:810
bool empty() const
Returns true if the array has no elements.
int waitKey(int delay=0)
Waits for a pressed key.
Explanation
Set header and namespaces
using namespace std;
using namespace dnn;
using namespace dnn_superres;
If you want you can set the namespace like the code above.
Create the Dnn Superres object
This is just to create the object, register the custom dnn layers and get access to the class functions.
Read the model
path = "models/FSRCNN_x2.pb"
sr.readModel(path);
This reads the TensorFlow model from the .pb file. Here 'path' is one of the pre-trained Tensorflow models' path file. You can download the models from OpenCV's GitHub, in the 'dnn_superres' module.
Set the model
sr.setModel("fsrcnn", 2);
Depending on the model you want to run, you have to set the algorithm and upscale factor. This is to know the desired algorithm and scale, even if you change the .pb file's name. For example: if you chose FSRCNN_x2.pb, your algorithm and scale will be 'fsrcnn' and 2, respectively. (Other algorithm options include "edsr", "espcn" and "lapsrn".)
Upscale an image
Mat img_new;
sr.upsample(img, img_new);
Now we can upscale any image. Load an image via the standard 'imread' function and create a new Mat for the destination image. Then simple upscale. Your upscaled image is located in 'img_new'.
An example in python
import cv2
from cv2 import dnn_superres
sr = dnn_superres.DnnSuperResImpl_create()
image = cv2.imread('./image.png')
path = "EDSR_x4.pb"
sr.readModel(path)
sr.setModel("edsr", 4)
result = sr.upsample(image)
cv2.imwrite("./upscaled.png", result)
Original:
Upscaled Image via FSRCNN:
Upscaled Image via Bicubic Interpolation: