ArUco markers and boards detection for robust camera pose estimation#

Detailed Description#

ArUco Marker Detection Square fiducial markers (also known as Augmented Reality Markers) are useful for easy, fast and robust camera pose estimation.

The main functionality of ArucoDetector class is detection of markers in an image. If the markers are grouped as a board, then you can try to recover the missing markers with ArucoDetector::refineDetectedMarkers(). ArUco markers can also be used for advanced chessboard corner finding. To do this, group the markers in the CharucoBoard and find the corners of the chessboard with the CharucoDetector::detectBoard().

The implementation is based on the ArUco Library by R. Muñoz-Salinas and S. Garrido-Jurado [111].

Markers can also be detected based on the AprilTag 2 [328] fiducial marker detection method.

See also

[111] This code has been originally developed by Sergio Garrido-Jurado as a project for Google Summer of Code 2015 (GSoC 15).

Warning

In OpenCV, the order of the returned corners locations for the AprilTag family is not aligned with the ArUco one.

Note that this order is also different from the convention adopted by the official AprilTag library.

An overview of the supported ArUco markers family is visible in the following image:

An overview of the supported AprilTag markers family is visible in the following image:

Note

The generated images (in the above picture) using aruco::generateImageMarker for the AprilTag markers have been rotated by 180 degree in order to match the official AprilTag images. When using the aruco::generateImageMarker function, it will output by default a different image from the official AprilTag convention, see the AprilRobotics/apriltag-imgs repository. This is the reason why you see a different corners order between ArUco and AprilTag in the above image.

For the ArUco marker family, the recommended family is the DICT_ARUCO_MIP_36h12 one, see. In general, a smaller marker family (e.g. 4x4 vs 6x6) should give you a better detection rate with respect to the camera distance, at the expense of having more probability to have issues with false detection or marker id decoding error. The number of marker ids in a family is also something to take into account with respect to the application use case and the ability to correct wrong bits during the marker id decoding process.

You can download some pregenerated MIP_36h12 ArUco marker images from:

For the AprilTag family, you can find some pregenerated marker images in the AprilRobotics/apriltag-imgs repository.

Note

For accurate corners location extraction, a white border (to have a strong gradient between white and black transition) around the marker is important. This is necessary to precisely extract the marker contour in difficult conditions such as bad illumination, confusing color background, etc.

There are multiple parameters which can be tweaked to improve the marker detection rate or to be adapted to your use case (e.g. image resolution). Please refer to the:

The corner refinement method can be changed according to the aruco::CornerRefineMethod to improve the corners location accuracy at the expense of more computation time.

To estimate the marker pose with respect to the camera frame, we recommend you to look at the following sources of information:

  • tutorial_aruco_detection for a tutorial about ArUco markers detection

  • Point-cloud sampling and segmentation for some theoretical background about the pinhole camera model and the calib3d_solvePnP page

  • solvePnP, solvePnPGeneric, solveP3P for the relevant pose estimation methods

Classes#

Name

Description

class cv::aruco::ArucoDetector

The main functionality of ArucoDetector class is detection of markers in an image with detectMarkers() method. View details

class cv::aruco::Board

Board of ArUco markers. View details

class cv::aruco::CharucoBoard

ChArUco board is a planar chessboard where the markers are placed inside the white squares of a chessboard. View details

class cv::aruco::CharucoDetector

View details

struct cv::aruco::CharucoParameters

View details

struct cv::aruco::DetectorParameters

struct DetectorParameters is used by ArucoDetector View details

class cv::aruco::Dictionary

Dictionary is a set of unique ArUco markers of the same size. View details

class cv::aruco::GridBoard

Planar board with grid arrangement of markers. View details

struct cv::aruco::RefineParameters

struct RefineParameters is used by ArucoDetector View details

Enumerations#

enum cv::aruco::CornerRefineMethod {
    cv::aruco::CORNER_REFINE_NONE,
    cv::aruco::CORNER_REFINE_SUBPIX,
    cv::aruco::CORNER_REFINE_CONTOUR,
    cv::aruco::CORNER_REFINE_APRILTAG
}

View details

enum cv::aruco::PredefinedDictionaryType {
    cv::aruco::DICT_4X4_50 = 0,
    cv::aruco::DICT_4X4_100,
    cv::aruco::DICT_4X4_250,
    cv::aruco::DICT_4X4_1000,
    cv::aruco::DICT_5X5_50,
    cv::aruco::DICT_5X5_100,
    cv::aruco::DICT_5X5_250,
    cv::aruco::DICT_5X5_1000,
    cv::aruco::DICT_6X6_50,
    cv::aruco::DICT_6X6_100,
    cv::aruco::DICT_6X6_250,
    cv::aruco::DICT_6X6_1000,
    cv::aruco::DICT_7X7_50,
    cv::aruco::DICT_7X7_100,
    cv::aruco::DICT_7X7_250,
    cv::aruco::DICT_7X7_1000,
    cv::aruco::DICT_ARUCO_ORIGINAL,
    cv::aruco::DICT_APRILTAG_16h5,
    cv::aruco::DICT_APRILTAG_25h9,
    cv::aruco::DICT_APRILTAG_36h10,
    cv::aruco::DICT_APRILTAG_36h11,
    cv::aruco::DICT_ARUCO_MIP_36h12
}

Predefined markers dictionaries/sets. View details

Enumeration Type Documentation#

CornerRefineMethod#

enum cv::aruco::CornerRefineMethod

#include <opencv2/objdetect/aruco_detector.hpp>

Enumerator:

CORNER_REFINE_NONE
Python: cv.aruco.CORNER_REFINE_NONE

Tag and corners detection based on the ArUco approach.

CORNER_REFINE_SUBPIX
Python: cv.aruco.CORNER_REFINE_SUBPIX

ArUco approach and refine the corners locations using corner subpixel accuracy.

CORNER_REFINE_CONTOUR
Python: cv.aruco.CORNER_REFINE_CONTOUR

ArUco approach and refine the corners locations using the contour-points line fitting.

CORNER_REFINE_APRILTAG
Python: cv.aruco.CORNER_REFINE_APRILTAG

Tag and corners detection based on the AprilTag 2 approach [328].

PredefinedDictionaryType#

enum cv::aruco::PredefinedDictionaryType

#include <opencv2/objdetect/aruco_dictionary.hpp>

Predefined markers dictionaries/sets.

Each dictionary indicates the number of bits and the number of markers contained

  • DICT_ARUCO_ORIGINAL: standard ArUco Library Markers. 1024 markers, 5x5 bits, 0 minimum distance

Enumerator:

DICT_4X4_50
Python: cv.aruco.DICT_4X4_50

4x4 bits, minimum hamming distance between any two codes = 4, 50 codes

DICT_4X4_100
Python: cv.aruco.DICT_4X4_100

4x4 bits, minimum hamming distance between any two codes = 3, 100 codes

DICT_4X4_250
Python: cv.aruco.DICT_4X4_250

4x4 bits, minimum hamming distance between any two codes = 3, 250 codes

DICT_4X4_1000
Python: cv.aruco.DICT_4X4_1000

4x4 bits, minimum hamming distance between any two codes = 2, 1000 codes

DICT_5X5_50
Python: cv.aruco.DICT_5X5_50

5x5 bits, minimum hamming distance between any two codes = 8, 50 codes

DICT_5X5_100
Python: cv.aruco.DICT_5X5_100

5x5 bits, minimum hamming distance between any two codes = 7, 100 codes

DICT_5X5_250
Python: cv.aruco.DICT_5X5_250

5x5 bits, minimum hamming distance between any two codes = 6, 250 codes

DICT_5X5_1000
Python: cv.aruco.DICT_5X5_1000

5x5 bits, minimum hamming distance between any two codes = 5, 1000 codes

DICT_6X6_50
Python: cv.aruco.DICT_6X6_50

6x6 bits, minimum hamming distance between any two codes = 13, 50 codes

DICT_6X6_100
Python: cv.aruco.DICT_6X6_100

6x6 bits, minimum hamming distance between any two codes = 12, 100 codes

DICT_6X6_250
Python: cv.aruco.DICT_6X6_250

6x6 bits, minimum hamming distance between any two codes = 11, 250 codes

DICT_6X6_1000
Python: cv.aruco.DICT_6X6_1000

6x6 bits, minimum hamming distance between any two codes = 9, 1000 codes

DICT_7X7_50
Python: cv.aruco.DICT_7X7_50

7x7 bits, minimum hamming distance between any two codes = 19, 50 codes

DICT_7X7_100
Python: cv.aruco.DICT_7X7_100

7x7 bits, minimum hamming distance between any two codes = 18, 100 codes

DICT_7X7_250
Python: cv.aruco.DICT_7X7_250

7x7 bits, minimum hamming distance between any two codes = 17, 250 codes

DICT_7X7_1000
Python: cv.aruco.DICT_7X7_1000

7x7 bits, minimum hamming distance between any two codes = 14, 1000 codes

DICT_ARUCO_ORIGINAL
Python: cv.aruco.DICT_ARUCO_ORIGINAL

6x6 bits, minimum hamming distance between any two codes = 3, 1024 codes

DICT_APRILTAG_16h5
Python: cv.aruco.DICT_APRILTAG_16h5

4x4 bits, minimum hamming distance between any two codes = 5, 30 codes

DICT_APRILTAG_25h9
Python: cv.aruco.DICT_APRILTAG_25h9

5x5 bits, minimum hamming distance between any two codes = 9, 35 codes

DICT_APRILTAG_36h10
Python: cv.aruco.DICT_APRILTAG_36h10

6x6 bits, minimum hamming distance between any two codes = 10, 2320 codes

DICT_APRILTAG_36h11
Python: cv.aruco.DICT_APRILTAG_36h11

6x6 bits, minimum hamming distance between any two codes = 11, 587 codes

DICT_ARUCO_MIP_36h12
Python: cv.aruco.DICT_ARUCO_MIP_36h12

6x6 bits, minimum hamming distance between any two codes = 12, 250 codes

Function Documentation#

drawDetectedCornersCharuco()#

void cv::aruco::drawDetectedCornersCharuco(
InputOutputArray image,
InputArray charucoCorners,
InputArray charucoIds = noArray(),
Scalar cornerColor = Scalar(255, 0, 0) )

#include <opencv2/objdetect/charuco_detector.hpp>

Python:

cv.aruco.drawDetectedCornersCharuco(image, charucoCorners[, charucoIds[, cornerColor]]) -> image

Draws a set of Charuco corners.

This function draws a set of detected Charuco corners. If identifiers vector is provided, it also draws the id of each corner.

Parameters

  • image — input/output image. It must have 1 or 3 channels. The number of channels is not altered.

  • charucoCorners — vector of detected charuco corners

  • charucoIds — list of identifiers for each corner in charucoCorners

  • cornerColor — color of the square surrounding each corner

drawDetectedDiamonds()#

void cv::aruco::drawDetectedDiamonds(
InputOutputArray image,
InputArrayOfArrays diamondCorners,
InputArray diamondIds = noArray(),
Scalar borderColor = Scalar(0, 0, 255) )

#include <opencv2/objdetect/charuco_detector.hpp>

Python:

cv.aruco.drawDetectedDiamonds(image, diamondCorners[, diamondIds[, borderColor]]) -> image

Draw a set of detected ChArUco Diamond markers.

Given an array of detected diamonds, this functions draws them in the image. The marker borders are painted and the markers identifiers if provided. Useful for debugging purposes.

Parameters

  • image — input/output image. It must have 1 or 3 channels. The number of channels is not altered.

  • diamondCorners — positions of diamond corners in the same format returned by detectCharucoDiamond(). (e.g std::vector<std::vector<cv::Point2f> > ). For N detected markers, the dimensions of this array should be Nx4. The order of the corners should be clockwise.

  • diamondIds — vector of identifiers for diamonds in diamondCorners, in the same format returned by detectCharucoDiamond() (e.g. std::vector). Optional, if not provided, ids are not painted.

  • borderColor — color of marker borders. Rest of colors (text color and first corner color) are calculated based on this one.

drawDetectedMarkers()#

void cv::aruco::drawDetectedMarkers(
InputOutputArray image,
InputArrayOfArrays corners,
InputArray ids = noArray(),
Scalar borderColor = Scalar(0, 255, 0) )

#include <opencv2/objdetect/aruco_detector.hpp>

Python:

cv.aruco.drawDetectedMarkers(image, corners[, ids[, borderColor]]) -> image

Draw detected markers in image.

Given an array of detected marker corners and its corresponding ids, this functions draws the markers in the image. The marker borders are painted and the markers identifiers if provided. Useful for debugging purposes.

Parameters

  • image — input/output image. It must have 1 or 3 channels. The number of channels is not altered.

  • corners — positions of marker corners on input image. (e.g std::vector<std::vector<cv::Point2f> > ). For N detected markers, the dimensions of this array should be Nx4. The order of the corners should be clockwise.

  • ids — vector of identifiers for markers in markersCorners . Optional, if not provided, ids are not painted.

  • borderColor — color of marker borders. Rest of colors (text color and first corner color) are calculated based on this one to improve visualization.

extendDictionary()#

Dictionary cv::aruco::extendDictionary(
int nMarkers,
int markerSize,
const Dictionary & baseDictionary = Dictionary(),
int randomSeed = 0 )

#include <opencv2/objdetect/aruco_dictionary.hpp>

Python:

cv.aruco.extendDictionary(nMarkers, markerSize[, baseDictionary[, randomSeed]]) -> retval

Extend base dictionary by new nMarkers.

This function creates a new dictionary composed by nMarkers markers and each markers composed by markerSize x markerSize bits. If baseDictionary is provided, its markers are directly included and the rest are generated based on them. If the size of baseDictionary is higher than nMarkers, only the first nMarkers in baseDictionary are taken and no new marker is added.

Parameters

  • nMarkers — number of markers in the dictionary

  • markerSize — number of bits per dimension of each markers

  • baseDictionary — Include the markers in this dictionary at the beginning (optional)

  • randomSeed — a user supplied seed for theRNG()

generateImageMarker()#

void cv::aruco::generateImageMarker(
const Dictionary & dictionary,
int id,
int sidePixels,
OutputArray img,
int borderBits = 1 )

#include <opencv2/objdetect/aruco_detector.hpp>

Python:

cv.aruco.generateImageMarker(dictionary, id, sidePixels[, img[, borderBits]]) -> img

Generate a canonical marker image.

This function returns a marker image in its canonical form (i.e. ready to be printed)

Parameters

  • dictionary — dictionary of markers indicating the type of markers

  • id — identifier of the marker that will be returned. It has to be a valid id in the specified dictionary.

  • sidePixels — size of the image in pixels

  • img — output image with the marker

  • borderBits — width of the marker border.

getPredefinedDictionary()#

Dictionary cv::aruco::getPredefinedDictionary(int dict)

#include <opencv2/objdetect/aruco_dictionary.hpp>

Python:

cv.aruco.getPredefinedDictionary(dict) -> retval

Returns one of the predefined dictionaries referenced by DICT_*.

getPredefinedDictionary()#

Dictionary cv::aruco::getPredefinedDictionary(PredefinedDictionaryType name)

#include <opencv2/objdetect/aruco_dictionary.hpp>

Python:

cv.aruco.getPredefinedDictionary(dict) -> retval

Returns one of the predefined dictionaries defined in PredefinedDictionaryType.

Variable Documentation#

DEFAULT_VALID_BIT_ID_THRESHOLD#

static float cv::aruco::DEFAULT_VALID_BIT_ID_THRESHOLD {0.49f}

#include <opencv2/objdetect/aruco_detector.hpp>