OpenCV  3.0.0-rc1
Open Source Computer Vision
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
Extract horizontal and vertical lines by using morphological operations

Goal

In this tutorial you will learn how to:

Theory

Morphology Operations

Morphology is a set of image processing operations that process images based on predefined structuring elements known also as kernels. The value of each pixel in the output image is based on a comparison of the corresponding pixel in the input image with its neighbors. By choosing the size and shape of the kernel, you can construct a morphological operation that is sensitive to specific shapes regarding the input image.

Two of the most basic morphological operations are dilation and erosion. Dilation adds pixels to the boundaries of the object in an image, while erosion does exactly the opposite. The amount of pixels added or removed, respectively depends on the size and shape of the structuring element used to process the image. In general the rules followed from these two operations have as follows:

As it can be seen above and in general in any morphological operation the structuring element used to probe the input image, is the most important part.

A structuring element is a matrix consisting of only 0's and 1's that can have any arbitrary shape and size. Typically are much smaller than the image being processed, while the pixels with values of 1 define the neighborhood. The center pixel of the structuring element, called the origin, identifies the pixel of interest – the pixel being processed.

For example, the following illustrates a diamond-shaped structuring element of 7x7 size.

morph12.gif
A Diamond-Shaped Structuring Element and its Origin

A structuring element can have many common shapes, such as lines, diamonds, disks, periodic lines, and circles and sizes. You typically choose a structuring element the same size and shape as the objects you want to process/extract in the input image. For example, to find lines in an image, create a linear structuring element as you will see later.

Code

This tutorial code's is shown lines below. You can also download it from here.

1 
7 #include <iostream>
8 #include <opencv2/opencv.hpp>
9 
10 using namespace std;
11 using namespace cv;
12 
13 int main(int, char** argv)
14 {
16  // Load the image
17  Mat src = imread(argv[1]);
18 
19  // Check if image is loaded fine
20  if(!src.data)
21  cerr << "Problem loading image!!!" << endl;
22 
23  // Show source image
24  imshow("src", src);
26 
28  // Transform source image to gray if it is not
29  Mat gray;
30 
31  if (src.channels() == 3)
32  {
33  cvtColor(src, gray, CV_BGR2GRAY);
34  }
35  else
36  {
37  gray = src;
38  }
39 
40  // Show gray image
41  imshow("gray", gray);
43 
45  // Apply adaptiveThreshold at the bitwise_not of gray, notice the ~ symbol
46  Mat bw;
48 
49  // Show binary image
50  imshow("binary", bw);
52 
54  // Create the images that will use to extract the horizontal and vertical lines
55  Mat horizontal = bw.clone();
56  Mat vertical = bw.clone();
58 
60  // Specify size on horizontal axis
61  int horizontalsize = horizontal.cols / 30;
62 
63  // Create structure element for extracting horizontal lines through morphology operations
64  Mat horizontalStructure = getStructuringElement(MORPH_RECT, Size(horizontalsize,1));
65 
66  // Apply morphology operations
67  erode(horizontal, horizontal, horizontalStructure, Point(-1, -1));
68  dilate(horizontal, horizontal, horizontalStructure, Point(-1, -1));
69 
70  // Show extracted horizontal lines
71  imshow("horizontal", horizontal);
73 
75  // Specify size on vertical axis
76  int verticalsize = vertical.rows / 30;
77 
78  // Create structure element for extracting vertical lines through morphology operations
79  Mat verticalStructure = getStructuringElement(MORPH_RECT, Size( 1,verticalsize));
80 
81  // Apply morphology operations
82  erode(vertical, vertical, verticalStructure, Point(-1, -1));
83  dilate(vertical, vertical, verticalStructure, Point(-1, -1));
84 
85  // Show extracted vertical lines
86  imshow("vertical", vertical);
88 
90  // Inverse vertical image
91  bitwise_not(vertical, vertical);
92  imshow("vertical_bit", vertical);
93 
94  // Extract edges and smooth image according to the logic
95  // 1. extract edges
96  // 2. dilate(edges)
97  // 3. src.copyTo(smooth)
98  // 4. blur smooth img
99  // 5. smooth.copyTo(src, edges)
100 
101  // Step 1
102  Mat edges;
103  adaptiveThreshold(vertical, edges, 255, CV_ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 3, -2);
104  imshow("edges", edges);
105 
106  // Step 2
107  Mat kernel = Mat::ones(2, 2, CV_8UC1);
108  dilate(edges, edges, kernel);
109  imshow("dilate", edges);
110 
111  // Step 3
112  Mat smooth;
113  vertical.copyTo(smooth);
114 
115  // Step 4
116  blur(smooth, smooth, Size(2, 2));
117 
118  // Step 5
119  smooth.copyTo(vertical, edges);
120 
121  // Show final result
122  imshow("smooth", vertical);
124 
125  waitKey(0);
126  return 0;
127 }
void adaptiveThreshold(InputArray src, OutputArray dst, double maxValue, int adaptiveMethod, int thresholdType, int blockSize, double C)
Applies an adaptive threshold to an array.
Definition: types_c.h:587
void blur(InputArray src, OutputArray dst, Size ksize, Point anchor=Point(-1,-1), int borderType=BORDER_DEFAULT)
Blurs an image using the normalized box filter.
void cvtColor(InputArray src, OutputArray dst, int code, int dstCn=0)
Converts an image from one color space to another.
Mat getStructuringElement(int shape, Size ksize, Point anchor=Point(-1,-1))
Returns a structuring element of the specified size and shape for morphological operations.
int channels() const
Returns the number of matrix channels.
int rows
the number of rows and columns or (-1, -1) when the matrix has more than 2 dimensions ...
Definition: mat.hpp:1865
Mat imread(const String &filename, int flags=IMREAD_COLOR)
Loads an image from a file.
Definition: types_c.h:121
uchar * data
pointer to the data
Definition: mat.hpp:1867
void copyTo(OutputArray m) const
Copies the matrix to another one.
void erode(InputArray src, OutputArray dst, InputArray kernel, Point anchor=Point(-1,-1), int iterations=1, int borderType=BORDER_CONSTANT, const Scalar &borderValue=morphologyDefaultBorderValue())
Erodes an image by using a specific structuring element.
void imshow(const String &winname, InputArray mat)
Displays an image in the specified window.
#define CV_8UC1
Definition: cvdef.h:116
void dilate(InputArray src, OutputArray dst, InputArray kernel, Point anchor=Point(-1,-1), int iterations=1, int borderType=BORDER_CONSTANT, const Scalar &borderValue=morphologyDefaultBorderValue())
Dilates an image by using a specific structuring element.
int cols
Definition: mat.hpp:1865
Mat clone() const
Creates a full copy of the array and the underlying data.
Definition: imgproc.hpp:309
Size2i Size
Definition: types.hpp:308
void bitwise_not(InputArray src, OutputArray dst, InputArray mask=noArray())
Inverts every bit of an array.
a rectangular structuring element:
Definition: imgproc.hpp:234
int main(int argc, const char *argv[])
Definition: facerec_demo.cpp:67
n-dimensional dense array class
Definition: mat.hpp:726
Point2i Point
Definition: types.hpp:181
int waitKey(int delay=0)
Waits for a pressed key.

Explanation / Result

  1. Load the source image and check if it is loaded without any problem, then show it:
    // Load the image
    Mat src = imread(argv[1]);
    // Check if image is loaded fine
    if(!src.data)
    cerr << "Problem loading image!!!" << endl;
    // Show source image
    imshow("src", src);
    src.png
  2. Then transform image to grayscale if it not already:
    // Transform source image to gray if it is not
    Mat gray;
    if (src.channels() == 3)
    {
    cvtColor(src, gray, CV_BGR2GRAY);
    }
    else
    {
    gray = src;
    }
    // Show gray image
    imshow("gray", gray);
    gray.png
  3. Afterwards transform grayscale image to binary. Notice the ~ symbol which indicates that we use the inverse (i.e. bitwise_not) version of it:
    // Apply adaptiveThreshold at the bitwise_not of gray, notice the ~ symbol
    Mat bw;
    // Show binary image
    imshow("binary", bw);
    binary.png
  4. Now we are ready to apply morphological operations in order to extract the horizontal and vertical lines and as a consequence to separate the the music notes from the music sheet, but first let's initialize the output images that we will use for that reason:
    // Create the images that will use to extract the horizontal and vertical lines
    Mat horizontal = bw.clone();
    Mat vertical = bw.clone();
  5. As we specified in the theory in order to extract the object that we desire, we need to create the corresponding structure element. Since here we want to extract the horizontal lines, a corresponding structure element for that purpose will have the following shape:
    linear_horiz.png
    and in the source code this is represented by the following code snippet:
    // Specify size on horizontal axis
    int horizontalsize = horizontal.cols / 30;
    // Create structure element for extracting horizontal lines through morphology operations
    Mat horizontalStructure = getStructuringElement(MORPH_RECT, Size(horizontalsize,1));
    // Apply morphology operations
    erode(horizontal, horizontal, horizontalStructure, Point(-1, -1));
    dilate(horizontal, horizontal, horizontalStructure, Point(-1, -1));
    // Show extracted horizontal lines
    imshow("horizontal", horizontal);
    horiz.png
  6. The same applies for the vertical lines, with the corresponding structure element:
    linear_vert.png
    and again this is represented as follows:
    // Specify size on vertical axis
    int verticalsize = vertical.rows / 30;
    // Create structure element for extracting vertical lines through morphology operations
    Mat verticalStructure = getStructuringElement(MORPH_RECT, Size( 1,verticalsize));
    // Apply morphology operations
    erode(vertical, vertical, verticalStructure, Point(-1, -1));
    dilate(vertical, vertical, verticalStructure, Point(-1, -1));
    // Show extracted vertical lines
    imshow("vertical", vertical);
    vert.png
  7. As you can see we are almost there. However, at that point you will notice that the edges of the notes are a bit rough. For that reason we need to refine the edges in order to obtain a smoother result:
    // Inverse vertical image
    bitwise_not(vertical, vertical);
    imshow("vertical_bit", vertical);
    // Extract edges and smooth image according to the logic
    // 1. extract edges
    // 2. dilate(edges)
    // 3. src.copyTo(smooth)
    // 4. blur smooth img
    // 5. smooth.copyTo(src, edges)
    // Step 1
    Mat edges;
    imshow("edges", edges);
    // Step 2
    Mat kernel = Mat::ones(2, 2, CV_8UC1);
    dilate(edges, edges, kernel);
    imshow("dilate", edges);
    // Step 3
    Mat smooth;
    vertical.copyTo(smooth);
    // Step 4
    blur(smooth, smooth, Size(2, 2));
    // Step 5
    smooth.copyTo(vertical, edges);
    // Show final result
    imshow("smooth", vertical);
    smooth.png