How to write a tutorial for OpenCV?¶

Okay, so assume you have just finished a project of yours implementing something based on OpenCV and you want to present/share it with the community. Luckily, OpenCV is an open source project. This means that in theory anyone has access to the full source code and may extend it. While making a robust and practical library (like OpenCV) is great, the success of a library also depends on how user friendly it is. To improve on this aspect, the OpenCV team has already been listening to user feedback from its Yahoo user group and by making samples you can find in the source directories sample folder. The addition of the tutorials (in both online and PDF format) is an extension of these efforts. Goal ==== .. _reST: http://docutils.sourceforge.net/rst.html .. |reST| replace:: reStructuredText .. |Sphinx| replace:: Sphinx .. _Sphinx: http://sphinx.pocoo.org/ The tutorials are just as an important part of the library as the implementation of those crafty data structures and algorithms you can find in OpenCV. Therefore, the source codes for the tutorials are part of the library. And yes, I meant source codes. The reason for this formulation is that the tutorials are written by using the |Sphinx|_ documentation generation system. This is based on the popular python documentation system called |reST|_ (reST). ReStructuredText is a really neat language that by using a few simple conventions (indentation, directives) and emulating old school e-mail writing techniques (text only) tries to offer a simple way to create and edit documents. Sphinx extends this with some new features and creates the resulting document in both HTML (for web) and PDF (for offline usage) format. Usually, an OpenCV tutorial has the following parts: 1. A source code demonstration of an OpenCV feature:

1. One or more CPP, Python, Java or other type of files depending for what OpenCV offers support and for what language you make the tutorial.
2. Occasionaly, input resource files required for running your tutorials application.
1. A table of content entry (so people may easily find the tutorial): a. Adding your stuff to the tutorials table of content (reST file). #. Add an image file near the TOC entry.
2. The content of the tutorial itself: a. The reST text of the tutorial #. Images following the idea that “A picture is worth a thousand words”. #. For more complex demonstrations you may create a video.

• Add a standard output with the description of what your program does. Keep it short and yet, descriptive. This output is at the start of the program. In my example files this usually takes the form of a help function containing the output. This way both the source file viewer and application runner can see what all is about in your sample. Here’s an instance of this: .. code-block:: cpp

void help() { cout << “————————————————————————–” << endl << “This program shows how to write video files. You can extract the R or G or B color channel ” << ” of the input video. You can choose to use the source codec (Y) or select a custom one. (N)”<< endl << “Usage:” << endl << ”./video-write inputvideoName [ R | G | B] [Y | N]” << endl << “————————————————————————–” << endl << endl; } // ... int main(int argc, char *argv[], char *window_name) { help(); // here comes the actual source code }

Additionally, finalize the description with a short usage guide. This way the user will know how to call your programs, what leads us to the next point.

• Prefer command line argument controlling instead of hard coded one. If your program has some variables that may be changed use command line arguments for this. The tutorials, can be a simple try-out ground for the user. If you offer command line controlling for the input image (for example), then you offer the possibility for the user to try it out with his/her own images, without the need to mess in the source code. In the upper example you can see that the input image, channel and codec selection may all be changed from the command line. Just compile the program and run it with your own input arguments.

• Be as verbose as possible. There is no shame in filling the source code with comments. This way the more advanced user may figure out what’s happening right from the sample code. This advice goes for the output console too. Specify to the user what’s happening. Never leave the user hanging there and thinking on: “Is this program now crashing or just doing some computationally intensive task?.” So, if you do a training task that may take some time, make sure you print out a message about this before starting and after finishing it.

• Throw out unnecessary stuff from your source code. This is a warning to not take the previous point too seriously. Balance is the key. If it’s something that can be done in a fewer lines or simpler than that’s the way you should do it. Nevertheless, if for some reason you have such sections notify the user why you have chosen to do so. Keep the amount of information as low as possible, while still getting the job done in an elegant way.

• Put your sample file into the opencv/samples/cpp/tutorial_code/sectionName folder. If you write a tutorial for other languages than cpp, then change that part of the path. Before completing this you need to decide that to what section (module) does your tutorial goes. Think about on what module relies most heavily your code and that is the one to use. If the answer to this question is more than one modules then the general section is the one to use. For finding the opencv directory open up your file system and navigate where you downloaded our repository.

• If the input resources are hard to acquire for the end user consider adding a few of them to the opencv/samples/cpp/tutorial_code/images. Make sure that who reads your code can try it out!

Add the TOC entry¶

The first line is a reference to the section title in the reST system. The section title will be a link and you may refer to it via the :ref: directive. The include directive imports the template text from the definitions directories noContent.rst file. Sphinx does not creates the PDF from scratch. It does this by first creating a latex file. Then creates the PDF from the latex file. With the raw directive you can directly add to this output commands. Its unique argument is for what kind of output to add the content of the directive. For the PDFs it may happen that multiple sections will overlap on a single page. To avoid this at the end of the TOC we add a pagebreak latex command, that hints to the LATEX system that the next line should be on a new page. If you have one of this, try to transform it to the following form: .. include:: ../../definitions/tocDefinitions.rst .. code-block:: rst

If this is already present just add a new section of the content between the include and the raw directives (excluding those lines). Here you’ll see a new include directive. This should be present only once in a TOC tree and the reST file contains the definitions of all the authors contributing to the OpenCV tutorials. We are a multicultural community and some of our name may contain some funky characters. However, reST only supports ANSI characters. Luckily we can specify Unicode characters with the unicode directive. Doing this for all of your tutorials is a troublesome procedure. Therefore, the tocDefinitions file contains the definition of your author name. Add it here once and afterwards just use the replace construction. For example here’s the definition for my name: .. code-block:: rst

The |Author_BernatG| is the text definitions alias. I can use later this to add the definition, like I’ve done in the TOCs Author part. After the :: and a space you start the definition. If you want to add an UNICODE character (non-ASCI) leave an empty space and specify it in the format U+(UNICODE code). To find the UNICODE code of a character I recommend using the FileFormat websites service. Spaces are trimmed from the definition, therefore we add a space by its UNICODE character (U+0020). Until the raw directive what you can see is a TOC tree entry. Here’s how a TOC entry will look like: +

As you can see we have an image to the left and a description box to the right. To create two boxes we use a table with two columns and a single row. In the left column is the image and in the right one the description. However, the image directive is way too long to fit in a column. Therefore, we need to use the substitution definition system. We add this definition after the TOC tree. All images for the TOC tree are to be put in the images folder near its |reST|_ file. We use the point measurement system because we are also creating PDFs. PDFs are printable documents, where there is no such thing that pixels (px), just points (pt). And while generally space is no problem for web pages (we have monitors with huge resolutions) the size of the paper (A4 or letter) is constant and will be for a long time in the future. Therefore, size constrains come in play more like for the PDF, than the generated HTML code. Now your images should be as small as possible, while still offering the intended information for the user. Remember that the tutorial will become part of the OpenCV source code. If you add large images (that manifest in form of large image size) it will just increase the size of the repository pointlessly. If someone wants to download it later, its download time will be that much longer. Not to mention the larger PDF size for the tutorials and the longer load time for the web pages. In terms of pixels a TOC image should not be larger than 120 X 120 pixels. Resize your images if they are larger! .. note:

If you add a larger image and specify a smaller image size, *Sphinx* will not resize that. At build time will add the full size image and the resize will be done by your browser after the image is loaded. A 120 X 120 image is somewhere below 10KB. If you add a 110KB image, you have just pointlessly added a 100KB extra data to transfer over the internet for every user!

Generally speaking you shouldn’t need to specify your images size (excluding the TOC entries). If no such is found Sphinx will use the size of the image itself (so no resize occurs). Then again if for some reason you decide to specify a size that should be the width of the image rather than its height. The reason for this again goes back to the PDFs. On a PDF page the height is larger than the width. In the PDF the images will not be resized. If you specify a size that does not fit in the page, then what does not fits in will be cut off. When creating your images for your tutorial you should try to keep the image widths below 500 pixels, and calculate with around 400 point page width when specifying image widths. The image format depends on the content of the image. If you have some complex scene (many random like colors) then use jpg. Otherwise, prefer using png. They are even some tools out there that optimize the size of PNG images, such as PNGGauntlet. Use them to make your images as small as possible in size. Now on the right side column of the table we add the information about the tutorial: .. container:: enumeratevisibleitemswithsquare

• In the first line it is the title of the tutorial. However, there is no need to specify it explicitly. We use the reference system. We’ll start up our tutorial with a reference specification, just like in case of this TOC entry with its  .. _Table-Of-Content-Section: . If after this you have a title (pointed out by the following line of -), then Sphinx will replace the :ref:Table-Of-Content-Section directive with the tile of the section in reference form (creates a link in web page). Here’s how the definition looks in my case: .. code-block:: rst

Note, that according to the |reST|_ rules the * should be as long as your title.

• Compatibility. What version of OpenCV is required to run your sample code.

• Author. Use the substitution markup of |reST|_.

• A short sentence describing the essence of your tutorial.

Now before each TOC entry you need to add the three lines of: .. code-block:: cpp

The plus sign (+) is to enumerate tutorials by using bullet points. So for every TOC entry we have a corresponding bullet point represented by the +. Sphinx is highly indenting sensitive. Indentation is used to express from which point until to which point does a construction last. Un-indentation means end of that construction. So to keep all the bullet points to the same group the following TOC entries (until the next +) should be indented by two spaces. Here, I should also mention that always prefer using spaces instead of tabs. Working with only spaces makes possible that if we both use monotype fonts we will see the same thing. Tab size is text editor dependent and as should be avoided. Sphinx translates all tabs into 8 spaces before interpreting it. It turns out that the automatic formatting of both the HTML and PDF(LATEX) system messes up our tables. Therefore, we need to help them out a little. For the PDF generation we add the .. tabularcolumns:: m{100pt} m{300pt} directive. This means that the first column should be 100 points wide and middle aligned. For the HTML look we simply name the following table of a toctableopencv class type. Then, we can modify the look of the table by modifying the CSS of our web page. The CSS definitions go into the opencv/doc/_themes/blue/static/default.css_t file. .. code-block:: css

.toctableopencv {

width: 100% ; table-layout: fixed;

} .toctableopencv colgroup col:first-child {

width: 100pt !important; max-width: 100pt !important; min-width: 100pt !important;

} .toctableopencv colgroup col:nth-child(2) {

width: 100% !important;

}

However, you should not need to modify this. Just add these three lines (plus keep the two space indentation) for all TOC entries you add. At the end of the TOC file you’ll find: .. code-block:: rst

• Create the reference point and the title. .. code-block:: rst

You start the tutorial by specifying a reference point by the .. _matTheBasicImageContainer: and then its title. The name of the reference point should be a unique one over the whole documentation. Therefore, do not use general names like tutorial1. Use the * character to underline the title for its full width. The subtitles of the tutorial should be underlined with = charachter.

• Goals. You start your tutorial by specifying what you will present. You can also enumerate the sub jobs to be done. For this you can use a bullet point construction. There is a single configuration file for both the reference manual and the tutorial documentation. In the reference manuals at the argument enumeration we do not want any kind of bullet point style enumeration. Therefore, by default all the bullet points at this level are set to do not show the dot before the entries in the HTML. You can override this by putting the bullet point in a container. I’ve defined a square type bullet point view under the name enumeratevisibleitemswithsquare. The CSS style definition for this is again in the opencvdoc_themesbluestaticdefault.css_t file. Here’s a quick example of using it: .. code-block:: rst

Note that you need the keep the indentation of the container directive. Directive indentations are always three (3) spaces. Here you may even give usage tips for your sample code.

• Source code. Present your samples code to the user. It’s a good idea to offer a quick download link for the HTML page by using the download directive and pointing out where the user may find your source code in the file system by using the file directive: .. code-block:: rst

Text samples/cpp/tutorial_code/highgui/video-write/ folder of the OpenCV source library or text to appear in the webpage.

For the download link the path is a relative one, hence the multiple back stepping operations (..). Then you can add the source code either by using the code block directive or the literal include one. In case of the code block you will need to actually add all the source code text into your |reST|_ text and also apply the required indentation: .. code-block:: rst

The only argument of the directive is the language used (here CPP). Then you add the source code into its content (meaning one empty line after the directive) by keeping the indentation of the directive (3 spaces). With the literal include directive you do not need to add the source code of the sample. You just specify the sample and Sphinx will load it for you, during build time. Here’s an example usage: .. code-block:: rst

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 #include // for standard I/O #include // for strings #include // Basic OpenCV structures (cv::Mat) #include // Video write using namespace std; using namespace cv; int main(int argc, char *argv[], char *window_name) { if (argc != 4) { cout << "Not enough parameters" << endl; return -1; } const string source = argv[1]; // the source file name const bool askOutputType = argv[3][0] =='Y'; // If false it will use the inputs codec type VideoCapture inputVideo(source); // Open input if ( !inputVideo.isOpened()) { cout << "Could not open the input video." << source << endl; return -1; } string::size_type pAt = source.find_last_of('.'); // Find extension point const string NAME = source.substr(0, pAt) + argv[2][0] + ".avi"; // Form the new name with container int ex = static_cast(inputVideo.get(CV_CAP_PROP_FOURCC)); // Get Codec Type- Int form // Transform from int to char via Bitwise operators char EXT[] = {ex & 0XFF , (ex & 0XFF00) >> 8,(ex & 0XFF0000) >> 16,(ex & 0XFF000000) >> 24, 0}; Size S = Size((int) inputVideo.get(CV_CAP_PROP_FRAME_WIDTH), //Acquire input size (int) inputVideo.get(CV_CAP_PROP_FRAME_HEIGHT)); VideoWriter outputVideo; // Open the output if (askOutputType) outputVideo.open(NAME , ex=-1, inputVideo.get(CV_CAP_PROP_FPS),S, true); else outputVideo.open(NAME , ex, inputVideo.get(CV_CAP_PROP_FPS),S, true); if (!outputVideo.isOpened()) { cout << "Could not open the output video for write: " << source << endl; return -1; } union { int v; char c[5];} uEx ; uEx.v = ex; // From Int to char via union uEx.c[4]='\0'; cout << "Input frame resolution: Width=" << S.width << " Height=" << S.height << " of nr#: " << inputVideo.get(CV_CAP_PROP_FRAME_COUNT) << endl; cout << "Input codec type: " << EXT << endl; int channel = 2; // Select the channel to save switch(argv[2][0]) { case 'R' : {channel = 2; break;} case 'G' : {channel = 1; break;} case 'B' : {channel = 0; break;} } Mat src,res; vector spl; while( true) //Show the image captured in the window and repeat { inputVideo >> src; // read if( src.empty()) break; // check if at end split(src, spl); // process - extract only the correct channel for( int i =0; i < 3; ++i) if (i != channel) spl[i] = Mat::zeros(S, spl[0].type()); merge(spl, res); //outputVideo.write(res); //save or outputVideo << res; } cout << "Finished writing" << endl; return 0; } 

After the directive you specify a relative path to the file from what to import. It has four options: the language to use, if you add the :linenos: the line numbers will be shown, you can specify the tab size with the :tab-width: and you do not need to load the whole file, you can show just the important lines. Use the lines option to do not show redundant information (such as the help function). Here basically you specify ranges, if the second range line number is missing than that means that until the end of the file. The ranges specified here do no need to be in an ascending order, you may even reorganize the structure of how you want to show your sample inside the tutorial.

• The tutorial. Well here goes the explanation for why and what have you used. Try to be short, clear, concise and yet a thorough one. There’s no magic formula. Look into a few already made tutorials and start out from there. Try to mix sample OpenCV code with your explanations. If with words is hard to describe something do not hesitate to add in a reasonable size image, to overcome this issue. When you present OpenCV functionality it’s a good idea to give a link to the used OpenCV data structure or function. Because the OpenCV tutorials and reference manual are in separate PDF files it is not possible to make this link work for the PDF format. Therefore, we use here only web page links to the opencv.itseez.com website. The OpenCV functions and data structures may be used for multiple tasks. Nevertheless, we want to avoid that every users creates its own reference to a commonly used function. So for this we use the global link collection of Sphinx. This is defined in the file:opencv/doc/conf.py configuration file. Open it and go all the way down to the last entry: .. code-block:: py

# —- External links for tutorials —————– extlinks = {

In short here we defined a new huivideo directive that refers to an external webpage link. Its usage is: .. code-block:: rst

A sample function of the highgui modules image write and read page is the imread() function.

Which turns to: A sample function of the highgui modules image write and read page is the imread() function. The argument you give between the <> will be put in place of the %s in the upper definition, and as the link will anchor to the correct function. To find out the anchor of a given function just open up a web page, search for the function and click on it. In the address bar it should appear like: http://opencv.itseez.com/modules/highgui/doc/reading_and_writing_images_and_video.html#imread . Look here for the name of the directives for each page of the OpenCV reference manual. If none present for one of them feel free to add one for it. For formulas you can add LATEX code that will translate in the web pages into images. You do this by using the math directive. A usage tip: .. code-block:: latex

That after build turns into: .. math:

MSE = \frac{1}{c*i*j} \sum{(I_1-I_2)^2}

You can even use it inline as :math: MSE = \frac{1}{c*i*j} \sum{(I_1-I_2)^2} that turns into . If you use some crazy LATEX library extension you need to add those to the ones to use at build time. Look into the file:opencv/doc/conf.py configuration file for more information on this.

• Results. Well, here depending on your program show one of more of the following: - Console outputs by using the code block directive. - Output images. - Runtime videos, visualization. For this use your favorite screens capture software. Camtasia Studio certainly is one of the better choices, however their prices are out of this world. CamStudio is a free alternative, but less powerful. If you do a video you can upload it to YouTube and then use the raw directive with HTML option to embed it into the generated web page:

This results in the text and video: You may observe a runtime instance of this on the YouTube here. .. raw:: html

<div align=”center”> <iframe title=”Creating a video with OpenCV” width=”560” height=”349” src=”http://www.youtube.com/embed/jpBwHxsl1_0?rel=0&loop=1” frameborder=”0” allowfullscreen align=”middle”></iframe> </div>

When these aren’t self-explanatory make sure to throw in a few guiding lines about what and why we can see.

• Build the documentation and check for errors or warnings. In the CMake make sure you check or pass the option for building documentation. Then simply build the docs project for the PDF file and the docs_html project for the web page. Read the output of the build and check for errors/warnings for what you have added. This is also the time to observe and correct any kind of not so good looking parts. Remember to keep clean our build logs.

• Read again your tutorial and check for both programming and spelling errors. If found any, please correct them.

Take home the pride and joy of a job well done!¶

Once you are done contact me or dr. Gary Bradski with the tutorial. We may submit the tutorial ourselves to the trunk branch of our repository or ask you to do so. Now, to see your work live you may need to wait some time. The PDFs are updated usually at the launch of a new OpenCV version. The web pages are a little more diverse. They are automatically rebuilt in each evening. However, the opencv.itseez.com website contains only the most recent stable branch of OpenCV. Currently this is 2.3. When we add something new (like a tutorial) that first goes to the trunk branch of our repository. A build of this you may find on the opencv.itseez.com/trunk website. Although, we try to make a build every night occasionally we might freeze any of the branches to fix upcoming issues. During this it may take a little longer to see your work live, however if you submited it, be sure that eventually it will show up. If you have any questions or advices relating to this tutorial you can contact me at -delete-bernat@-delete-primeranks.net. Of course, delete the -delete- parts of that e-mail address.