OpenCV  3.2.0
Open Source Computer Vision
Template Matching

Goal

In this tutorial you will learn how to:

Theory

What is template matching?

Template matching is a technique for finding areas of an image that match (are similar) to a template image (patch).

While the patch must be a rectangle it may be that not all of the rectangle is relevant. In such a case, a mask can be used to isolate the portion of the patch that should be used to find the match.

How does it work?

How does the mask work?

Which are the matching methods available in OpenCV?

Good question. OpenCV implements Template matching in the function cv::matchTemplate . The available methods are 6:

  1. method=CV_TM_SQDIFF

    \[R(x,y)= \sum _{x',y'} (T(x',y')-I(x+x',y+y'))^2\]

  2. method=CV_TM_SQDIFF_NORMED

    \[R(x,y)= \frac{\sum_{x',y'} (T(x',y')-I(x+x',y+y'))^2}{\sqrt{\sum_{x',y'}T(x',y')^2 \cdot \sum_{x',y'} I(x+x',y+y')^2}}\]

  3. method=CV_TM_CCORR

    \[R(x,y)= \sum _{x',y'} (T(x',y') \cdot I(x+x',y+y'))\]

  4. method=CV_TM_CCORR_NORMED

    \[R(x,y)= \frac{\sum_{x',y'} (T(x',y') \cdot I(x+x',y+y'))}{\sqrt{\sum_{x',y'}T(x',y')^2 \cdot \sum_{x',y'} I(x+x',y+y')^2}}\]

  5. method=CV_TM_CCOEFF

    \[R(x,y)= \sum _{x',y'} (T'(x',y') \cdot I'(x+x',y+y'))\]

    where

    \[\begin{array}{l} T'(x',y')=T(x',y') - 1/(w \cdot h) \cdot \sum _{x'',y''} T(x'',y'') \\ I'(x+x',y+y')=I(x+x',y+y') - 1/(w \cdot h) \cdot \sum _{x'',y''} I(x+x'',y+y'') \end{array}\]

  6. method=CV_TM_CCOEFF_NORMED

    \[R(x,y)= \frac{ \sum_{x',y'} (T'(x',y') \cdot I'(x+x',y+y')) }{ \sqrt{\sum_{x',y'}T'(x',y')^2 \cdot \sum_{x',y'} I'(x+x',y+y')^2} }\]

Code

Explanation

  1. Declare some global variables, such as the image, template and result matrices, as well as the match method and the window names:
    Mat img; Mat templ; Mat result;
    char* image_window = "Source Image";
    char* result_window = "Result window";
    int match_method;
    int max_Trackbar = 5;
  2. Load the source image, template, and optionally, if supported for the matching method, a mask:
    bool method_accepts_mask = (CV_TM_SQDIFF == match_method || match_method == CV_TM_CCORR_NORMED);
    if (use_mask && method_accepts_mask)
    { matchTemplate( img, templ, result, match_method, mask); }
    else
    { matchTemplate( img, templ, result, match_method); }
  3. Create the windows to show the results:
    namedWindow( image_window, WINDOW_AUTOSIZE );
    namedWindow( result_window, WINDOW_AUTOSIZE );
  4. Create the Trackbar to enter the kind of matching method to be used. When a change is detected the callback function MatchingMethod is called.
    char* trackbar_label = "Method: \n 0: SQDIFF \n 1: SQDIFF NORMED \n 2: TM CCORR \n 3: TM CCORR NORMED \n 4: TM COEFF \n 5: TM COEFF NORMED";
    createTrackbar( trackbar_label, image_window, &match_method, max_Trackbar, MatchingMethod );
  5. Wait until user exits the program.
    return 0;
  6. Let's check out the callback function. First, it makes a copy of the source image:
    Mat img_display;
    img.copyTo( img_display );
  7. Next, it creates the result matrix that will store the matching results for each template location. Observe in detail the size of the result matrix (which matches all possible locations for it)
    int result_cols = img.cols - templ.cols + 1;
    int result_rows = img.rows - templ.rows + 1;
    result.create( result_rows, result_cols, CV_32FC1 );
  8. Perform the template matching operation:
    bool method_accepts_mask = (CV_TM_SQDIFF == match_method || match_method == CV_TM_CCORR_NORMED);
    if (use_mask && method_accepts_mask)
    { matchTemplate( img, templ, result, match_method, mask); }
    else
    { matchTemplate( img, templ, result, match_method); }
    the arguments are naturally the input image I, the template T, the result R, the match_method (given by the Trackbar), and optionally the mask image M
  9. We normalize the results:
    normalize( result, result, 0, 1, NORM_MINMAX, -1, Mat() );
  10. We localize the minimum and maximum values in the result matrix R by using cv::minMaxLoc .
    double minVal; double maxVal; Point minLoc; Point maxLoc;
    Point matchLoc;
    minMaxLoc( result, &minVal, &maxVal, &minLoc, &maxLoc, Mat() );
    the function calls as arguments:
    • result: The source array
    • **&minVal** and **&maxVal:** Variables to save the minimum and maximum values in result
    • **&minLoc** and **&maxLoc:** The Point locations of the minimum and maximum values in the array.
    • Mat(): Optional mask
  11. For the first two methods ( TM_SQDIFF and MT_SQDIFF_NORMED ) the best match are the lowest values. For all the others, higher values represent better matches. So, we save the corresponding value in the matchLoc variable:
    if( match_method == TM_SQDIFF || match_method == TM_SQDIFF_NORMED )
    { matchLoc = minLoc; }
    else
    { matchLoc = maxLoc; }
  12. Display the source image and the result matrix. Draw a rectangle around the highest possible matching area:
    rectangle( img_display, matchLoc, Point( matchLoc.x + templ.cols , matchLoc.y + templ.rows ), Scalar::all(0), 2, 8, 0 );
    rectangle( result, matchLoc, Point( matchLoc.x + templ.cols , matchLoc.y + templ.rows ), Scalar::all(0), 2, 8, 0 );
    imshow( image_window, img_display );
    imshow( result_window, result );

Results

  1. Testing our program with an input image such as:

    Template_Matching_Original_Image.jpg

    and a template image:

    Template_Matching_Template_Image.jpg
  2. Generate the following result matrices (first row are the standard methods SQDIFF, CCORR and CCOEFF, second row are the same methods in its normalized version). In the first column, the darkest is the better match, for the other two columns, the brighter a location, the higher the match.
    Template_Matching_Correl_Result_0.jpg
    Result_0
    Template_Matching_Correl_Result_1.jpg
    Result_1
    Template_Matching_Correl_Result_2.jpg
    Result_2
    Template_Matching_Correl_Result_3.jpg
    Result_3
    Template_Matching_Correl_Result_4.jpg
    Result_4
    Template_Matching_Correl_Result_5.jpg
    Result_5
  3. The right match is shown below (black rectangle around the face of the guy at the right). Notice that CCORR and CCDEFF gave erroneous best matches, however their normalized version did it right, this may be due to the fact that we are only considering the "highest match" and not the other possible high matches.

    Template_Matching_Image_Result.jpg