OpenCV  4.4.0
Open Source Computer Vision
Upscaling video

In this tutorial you will learn how to use the 'dnn_superres' interface to upscale video via pre-trained neural networks.

Building

When building OpenCV, run the following command to build all the contrib module:

cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/

Or only build the dnn_superres module:

cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/dnn_superres

Or make sure you check the dnn_superres module in the GUI version of CMake: cmake-gui.

Source Code of the sample

1 // This file is part of OpenCV project.
2 // It is subject to the license terms in the LICENSE file found in the top-level directory
3 // of this distribution and at http://opencv.org/license.html.
4 
5 #include <iostream>
6 
8 
9 #include <opencv2/imgproc.hpp>
10 #include <opencv2/highgui.hpp>
11 
12 using namespace std;
13 using namespace cv;
14 using namespace dnn_superres;
15 
16 int main(int argc, char *argv[])
17 {
18  // Check for valid command line arguments, print usage
19  // if insufficient arguments were given.
20  if (argc < 4) {
21  cout << "usage: Arg 1: input video path" << endl;
22  cout << "\t Arg 2: output video path" << endl;
23  cout << "\t Arg 3: algorithm | edsr, espcn, fsrcnn or lapsrn" << endl;
24  cout << "\t Arg 4: scale | 2, 3, 4 or 8 \n";
25  cout << "\t Arg 5: path to model file \n";
26  return -1;
27  }
28 
29  string input_path = string(argv[1]);
30  string output_path = string(argv[2]);
31  string algorithm = string(argv[3]);
32  int scale = atoi(argv[4]);
33  string path = string(argv[5]);
34 
35  VideoCapture input_video(input_path);
36  int ex = static_cast<int>(input_video.get(CAP_PROP_FOURCC));
37  Size S = Size((int) input_video.get(CAP_PROP_FRAME_WIDTH) * scale,
38  (int) input_video.get(CAP_PROP_FRAME_HEIGHT) * scale);
39 
40  VideoWriter output_video;
41  output_video.open(output_path, ex, input_video.get(CAP_PROP_FPS), S, true);
42 
43  if (!input_video.isOpened())
44  {
45  std::cerr << "Could not open the video." << std::endl;
46  return -1;
47  }
48 
49  DnnSuperResImpl sr;
50  sr.readModel(path);
51  sr.setModel(algorithm, scale);
52 
53  for(;;)
54  {
55  Mat frame, output_frame;
56  input_video >> frame;
57 
58  if ( frame.empty() )
59  break;
60 
61  sr.upsample(frame, output_frame);
62  output_video << output_frame;
63 
64  namedWindow("Upsampled video", WINDOW_AUTOSIZE);
65  imshow("Upsampled video", output_frame);
66 
67  namedWindow("Original video", WINDOW_AUTOSIZE);
68  imshow("Original video", frame);
69 
70  char c=(char)waitKey(25);
71  if(c==27)
72  break;
73  }
74 
75  input_video.release();
76  output_video.release();
77 
78  return 0;
79 }
the user cannot resize the window, the size is constrainted by the image displayed.
Definition: highgui.hpp:184
Video writer class.
Definition: videoio.hpp:858
STL namespace.
void imshow(const String &winname, InputArray mat)
Displays an image in the specified window.
"black box" representation of the file storage associated with a file on disk.
Definition: affine.hpp:51
virtual bool open(const String &filename, int fourcc, double fps, Size frameSize, bool isColor=true)
Initializes or reinitializes video writer.
Class for video capturing from video files, image sequences or cameras.
Definition: videoio.hpp:622
Template class for specifying the size of an image or rectangle.
Definition: types.hpp:315
4-character code of codec. see VideoWriter::fourcc .
Definition: videoio.hpp:138
Width of the frames in the video stream.
Definition: videoio.hpp:135
Height of the frames in the video stream.
Definition: videoio.hpp:136
void scale(cv::Mat &mat, const cv::Mat &range, const T min, const T max)
Definition: quality_utils.hpp:90
void namedWindow(const String &winname, int flags=WINDOW_AUTOSIZE)
Creates a window.
Size2i Size
Definition: types.hpp:347
Frame rate.
Definition: videoio.hpp:137
n-dimensional dense array class
Definition: mat.hpp:791
int waitKey(int delay=0)
Waits for a pressed key.

Explanation

  1. Set header and namespaces
    using namespace std;
    using namespace cv;
    using namespace dnn_superres;
  2. Create the Dnn Superres object
    DnnSuperResImpl sr;
    Instantiate a dnn super-resolution object.
  3. Read the model
    path = "models/ESPCN_x2.pb"
    sr.readModel(path);
    sr.setModel("espcn", 2);
    Read the model from the given path and sets the algorithm and scaling factor.
  4. Upscale a video
    for(;;)
    {
    Mat frame, output_frame;
    input_video >> frame;
    if ( frame.empty() )
    break;
    sr.upsample(frame, output_frame);
    ...
    }
    Process and upsample video frame by frame.