Optical Flow

Note

  • A general optical flow example can be found at opencv_source_code/samples/gpu/optical_flow.cpp
  • A general optical flow example using the Nvidia API can be found at opencv_source_code/samples/gpu/opticalflow_nvidia_api.cpp

cuda::BroxOpticalFlow

class cuda::BroxOpticalFlow

Class computing the optical flow for two images using Brox et al Optical Flow algorithm ([Brox2004]).

class BroxOpticalFlow
{
public:
    BroxOpticalFlow(float alpha_, float gamma_, float scale_factor_, int inner_iterations_, int outer_iterations_, int solver_iterations_);

    //! Compute optical flow
    //! frame0 - source frame (supports only CV_32FC1 type)
    //! frame1 - frame to track (with the same size and type as frame0)
    //! u      - flow horizontal component (along x axis)
    //! v      - flow vertical component (along y axis)
    void operator ()(const GpuMat& frame0, const GpuMat& frame1, GpuMat& u, GpuMat& v, Stream& stream = Stream::Null());

    //! flow smoothness
    float alpha;

    //! gradient constancy importance
    float gamma;

    //! pyramid scale factor
    float scale_factor;

    //! number of lagged non-linearity iterations (inner loop)
    int inner_iterations;

    //! number of warping iterations (number of pyramid levels)
    int outer_iterations;

    //! number of linear system solver iterations
    int solver_iterations;

    GpuMat buf;
};

Note

  • An example illustrating the Brox et al optical flow algorithm can be found at opencv_source_code/samples/gpu/brox_optical_flow.cpp

cuda::FarnebackOpticalFlow

class cuda::FarnebackOpticalFlow

Class computing a dense optical flow using the Gunnar Farneback’s algorithm.

class CV_EXPORTS FarnebackOpticalFlow
{
public:
    FarnebackOpticalFlow()
    {
        numLevels = 5;
        pyrScale = 0.5;
        fastPyramids = false;
        winSize = 13;
        numIters = 10;
        polyN = 5;
        polySigma = 1.1;
        flags = 0;
    }

    int numLevels;
    double pyrScale;
    bool fastPyramids;
    int winSize;
    int numIters;
    int polyN;
    double polySigma;
    int flags;

    void operator ()(const GpuMat &frame0, const GpuMat &frame1, GpuMat &flowx, GpuMat &flowy, Stream &s = Stream::Null());

    void releaseMemory();

private:
    /* hidden */
};

cuda::FarnebackOpticalFlow::operator ()

Computes a dense optical flow using the Gunnar Farneback’s algorithm.

C++: void cuda::FarnebackOpticalFlow::operator()(const GpuMat& frame0, const GpuMat& frame1, GpuMat& flowx, GpuMat& flowy, Stream& s=Stream::Null())
Parameters:
  • frame0 – First 8-bit gray-scale input image
  • frame1 – Second 8-bit gray-scale input image
  • flowx – Flow horizontal component
  • flowy – Flow vertical component
  • s – Stream

cuda::FarnebackOpticalFlow::releaseMemory

Releases unused auxiliary memory buffers.

C++: void cuda::FarnebackOpticalFlow::releaseMemory()

cuda::PyrLKOpticalFlow

class cuda::PyrLKOpticalFlow

Class used for calculating an optical flow.

class PyrLKOpticalFlow
{
public:
    PyrLKOpticalFlow();

    void sparse(const GpuMat& prevImg, const GpuMat& nextImg, const GpuMat& prevPts, GpuMat& nextPts,
        GpuMat& status, GpuMat* err = 0);

    void dense(const GpuMat& prevImg, const GpuMat& nextImg, GpuMat& u, GpuMat& v, GpuMat* err = 0);

    Size winSize;
    int maxLevel;
    int iters;
    bool useInitialFlow;

    void releaseMemory();
};

The class can calculate an optical flow for a sparse feature set or dense optical flow using the iterative Lucas-Kanade method with pyramids.

Note

  • An example of the Lucas Kanade optical flow algorithm can be found at opencv_source_code/samples/gpu/pyrlk_optical_flow.cpp

cuda::PyrLKOpticalFlow::sparse

Calculate an optical flow for a sparse feature set.

C++: void cuda::PyrLKOpticalFlow::sparse(const GpuMat& prevImg, const GpuMat& nextImg, const GpuMat& prevPts, GpuMat& nextPts, GpuMat& status, GpuMat* err=0)
Parameters:
  • prevImg – First 8-bit input image (supports both grayscale and color images).
  • nextImg – Second input image of the same size and the same type as prevImg .
  • prevPts – Vector of 2D points for which the flow needs to be found. It must be one row matrix with CV_32FC2 type.
  • nextPts – Output vector of 2D points (with single-precision floating-point coordinates) containing the calculated new positions of input features in the second image. When useInitialFlow is true, the vector must have the same size as in the input.
  • status – Output status vector (CV_8UC1 type). Each element of the vector is set to 1 if the flow for the corresponding features has been found. Otherwise, it is set to 0.
  • err – Output vector (CV_32FC1 type) that contains the difference between patches around the original and moved points or min eigen value if getMinEigenVals is checked. It can be NULL, if not needed.

cuda::PyrLKOpticalFlow::dense

Calculate dense optical flow.

C++: void cuda::PyrLKOpticalFlow::dense(const GpuMat& prevImg, const GpuMat& nextImg, GpuMat& u, GpuMat& v, GpuMat* err=0)
Parameters:
  • prevImg – First 8-bit grayscale input image.
  • nextImg – Second input image of the same size and the same type as prevImg .
  • u – Horizontal component of the optical flow of the same size as input images, 32-bit floating-point, single-channel
  • v – Vertical component of the optical flow of the same size as input images, 32-bit floating-point, single-channel
  • err – Output vector (CV_32FC1 type) that contains the difference between patches around the original and moved points or min eigen value if getMinEigenVals is checked. It can be NULL, if not needed.

cuda::PyrLKOpticalFlow::releaseMemory

Releases inner buffers memory.

C++: void cuda::PyrLKOpticalFlow::releaseMemory()

cuda::interpolateFrames

Interpolates frames (images) using provided optical flow (displacement field).

C++: void cuda::interpolateFrames(const GpuMat& frame0, const GpuMat& frame1, const GpuMat& fu, const GpuMat& fv, const GpuMat& bu, const GpuMat& bv, float pos, GpuMat& newFrame, GpuMat& buf, Stream& stream=Stream::Null())
Parameters:
  • frame0 – First frame (32-bit floating point images, single channel).
  • frame1 – Second frame. Must have the same type and size as frame0 .
  • fu – Forward horizontal displacement.
  • fv – Forward vertical displacement.
  • bu – Backward horizontal displacement.
  • bv – Backward vertical displacement.
  • pos – New frame position.
  • newFrame – Output image.
  • buf – Temporary buffer, will have width x 6*height size, CV_32FC1 type and contain 6 GpuMat: occlusion masks for first frame, occlusion masks for second, interpolated forward horizontal flow, interpolated forward vertical flow, interpolated backward horizontal flow, interpolated backward vertical flow.
  • stream – Stream for the asynchronous version.
[Brox2004]
  1. Brox, A. Bruhn, N. Papenberg, J. Weickert. High accuracy optical flow estimation based on a theory for warping. ECCV 2004.