Fisheye camera model

Detailed Description

Definitions: Let P be a point in 3D of coordinates X in the world reference frame (stored in the matrix X) The coordinate vector of P in the camera reference frame is:

\[Xc = R X + T\]

where R is the rotation matrix corresponding to the rotation vector om: R = rodrigues(om); call x, y and z the 3 coordinates of Xc:

\[\begin{array}{l} x = Xc_1 \\ y = Xc_2 \\ z = Xc_3 \end{array} \]

The pinhole projection coordinates of P is [a; b] where

\[\begin{array}{l} a = x / z \ and \ b = y / z \\ r^2 = a^2 + b^2 \\ \theta = atan(r) \end{array} \]

Fisheye distortion:

\[\theta_d = \theta (1 + k_1 \theta^2 + k_2 \theta^4 + k_3 \theta^6 + k_4 \theta^8)\]

The distorted point coordinates are [x'; y'] where

\[\begin{array}{l} x' = (\theta_d / r) a \\ y' = (\theta_d / r) b \end{array} \]

Finally, conversion into pixel coordinates: The final pixel coordinates vector [u; v] where:

\[\begin{array}{l} u = f_x (x' + \alpha y') + c_x \\ v = f_y y' + c_y \end{array} \]

Summary: Generic camera model [143] with perspective projection and without distortion correction

Namespaces
namespace	cv::fisheye
	The methods in this namespace use a so-called fisheye camera model.

Functions
double	cv::fisheye::calibrate (InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints, const Size &image_size, InputOutputArray K, InputOutputArray D, OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs, int flags=0, TermCriteria criteria=TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON))
	Performs camera calibration.
double	cv::fisheye::stereoCalibrate (InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints1, InputArrayOfArrays imagePoints2, InputOutputArray K1, InputOutputArray D1, InputOutputArray K2, InputOutputArray D2, Size imageSize, OutputArray R, OutputArray T, int flags=CALIB_FIX_INTRINSIC, TermCriteria criteria=TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON))
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.
double	cv::fisheye::stereoCalibrate (InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints1, InputArrayOfArrays imagePoints2, InputOutputArray K1, InputOutputArray D1, InputOutputArray K2, InputOutputArray D2, Size imageSize, OutputArray R, OutputArray T, OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs, int flags=CALIB_FIX_INTRINSIC, TermCriteria criteria=TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON))
	Performs stereo calibration.

Function Documentation

calibrate()

double cv::fisheye::calibrate	(	InputArrayOfArrays	objectPoints,
		InputArrayOfArrays	imagePoints,
		const Size &	image_size,
		InputOutputArray	K,
		InputOutputArray	D,
		OutputArrayOfArrays	rvecs,
		OutputArrayOfArrays	tvecs,
		int	flags = 0,
		TermCriteria	criteria = TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON) )

Python:
	cv.fisheye.calibrate(	objectPoints, imagePoints, image_size, K, D[, rvecs[, tvecs[, flags[, criteria]]]]	) ->	retval, K, D, rvecs, tvecs

#include <opencv2/calib.hpp>

Performs camera calibration.

Parameters

objectPoints	vector of vectors of calibration pattern points in the calibration pattern coordinate space.
imagePoints	vector of vectors of the projections of calibration pattern points. imagePoints.size() and objectPoints.size() and imagePoints[i].size() must be equal to objectPoints[i].size() for each i.
image_size	Size of the image used only to initialize the camera intrinsic matrix.
K	Output 3x3 floating-point camera intrinsic matrix \(\cameramatrix{A}\) . If cv::CALIB_USE_INTRINSIC_GUESS is specified, some or all of fx, fy, cx, cy must be initialized before calling the function.
D	Output vector of distortion coefficients \(\distcoeffsfisheye\).
rvecs	Output vector of rotation vectors (see Rodrigues ) estimated for each pattern view. That is, each k-th rotation vector together with the corresponding k-th translation vector (see the next output parameter description) brings the calibration pattern from the model coordinate space (in which object points are specified) to the world coordinate space, that is, a real position of the calibration pattern in the k-th pattern view (k=0.. M -1).
tvecs	Output vector of translation vectors estimated for each pattern view.
flags	Different flags that may be zero or a combination of the following values: cv::CALIB_USE_INTRINSIC_GUESS cameraMatrix contains valid initial values of fx, fy, cx, cy that are optimized further. Otherwise, (cx, cy) is initially set to the image center ( imageSize is used), and focal distances are computed in a least-squares fashion. cv::CALIB_RECOMPUTE_EXTRINSIC Extrinsic will be recomputed after each iteration of intrinsic optimization. cv::CALIB_CHECK_COND The functions will check validity of condition number. cv::CALIB_FIX_SKEW Skew coefficient (alpha) is set to zero and stay zero. cv::CALIB_FIX_K1,..., cv::CALIB_FIX_K4 Selected distortion coefficients are set to zeros and stay zero. cv::CALIB_FIX_PRINCIPAL_POINT The principal point is not changed during the global optimization. It stays at the center or at a different location specified when cv::CALIB_USE_INTRINSIC_GUESS is set too. cv::CALIB_FIX_FOCAL_LENGTH The focal length is not changed during the global optimization. It is the \(max(width,height)/\pi\) or the provided \(f_x\), \(f_y\) when cv::CALIB_USE_INTRINSIC_GUESS is set too.
criteria	Termination criteria for the iterative optimization algorithm.

stereoCalibrate() [1/2]

double cv::fisheye::stereoCalibrate	(	InputArrayOfArrays	objectPoints,
		InputArrayOfArrays	imagePoints1,
		InputArrayOfArrays	imagePoints2,
		InputOutputArray	K1,
		InputOutputArray	D1,
		InputOutputArray	K2,
		InputOutputArray	D2,
		Size	imageSize,
		OutputArray	R,
		OutputArray	T,
		int	flags = CALIB_FIX_INTRINSIC,
		TermCriteria	criteria = TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON) )

Python:
	cv.fisheye.stereoCalibrate(	objectPoints, imagePoints1, imagePoints2, K1, D1, K2, D2, imageSize[, R[, T[, rvecs[, tvecs[, flags[, criteria]]]]]]	) ->	retval, K1, D1, K2, D2, R, T, rvecs, tvecs
	cv.fisheye.stereoCalibrate(	objectPoints, imagePoints1, imagePoints2, K1, D1, K2, D2, imageSize[, R[, T[, flags[, criteria]]]]	) ->	retval, K1, D1, K2, D2, R, T

#include <opencv2/calib.hpp>

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

stereoCalibrate() [2/2]

double cv::fisheye::stereoCalibrate	(	InputArrayOfArrays	objectPoints,
		InputArrayOfArrays	imagePoints1,
		InputArrayOfArrays	imagePoints2,
		InputOutputArray	K1,
		InputOutputArray	D1,
		InputOutputArray	K2,
		InputOutputArray	D2,
		Size	imageSize,
		OutputArray	R,
		OutputArray	T,
		OutputArrayOfArrays	rvecs,
		OutputArrayOfArrays	tvecs,
		int	flags = CALIB_FIX_INTRINSIC,
		TermCriteria	criteria = TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON) )

Python:
	cv.fisheye.stereoCalibrate(	objectPoints, imagePoints1, imagePoints2, K1, D1, K2, D2, imageSize[, R[, T[, rvecs[, tvecs[, flags[, criteria]]]]]]	) ->	retval, K1, D1, K2, D2, R, T, rvecs, tvecs
	cv.fisheye.stereoCalibrate(	objectPoints, imagePoints1, imagePoints2, K1, D1, K2, D2, imageSize[, R[, T[, flags[, criteria]]]]	) ->	retval, K1, D1, K2, D2, R, T

#include <opencv2/calib.hpp>

Performs stereo calibration.

Parameters

objectPoints	Vector of vectors of the calibration pattern points.
imagePoints1	Vector of vectors of the projections of the calibration pattern points, observed by the first camera.
imagePoints2	Vector of vectors of the projections of the calibration pattern points, observed by the second camera.
K1	Input/output first camera intrinsic matrix: \(\vecthreethree{f_x^{(j)}}{0}{c_x^{(j)}}{0}{f_y^{(j)}}{c_y^{(j)}}{0}{0}{1}\) , \(j = 0,\, 1\) . If any of cv::CALIB_USE_INTRINSIC_GUESS , cv::CALIB_FIX_INTRINSIC are specified, some or all of the matrix components must be initialized.
D1	Input/output vector of distortion coefficients \(\distcoeffsfisheye\) of 4 elements.
K2	Input/output second camera intrinsic matrix. The parameter is similar to K1 .
D2	Input/output lens distortion coefficients for the second camera. The parameter is similar to D1 .
imageSize	Size of the image used only to initialize camera intrinsic matrix.
R	Output rotation matrix between the 1st and the 2nd camera coordinate systems.
T	Output translation vector between the coordinate systems of the cameras.
rvecs	Output vector of rotation vectors ( Rodrigues ) estimated for each pattern view in the coordinate system of the first camera of the stereo pair (e.g. std::vector<cv::Mat>). More in detail, each i-th rotation vector together with the corresponding i-th translation vector (see the next output parameter description) brings the calibration pattern from the object coordinate space (in which object points are specified) to the camera coordinate space of the first camera of the stereo pair. In more technical terms, the tuple of the i-th rotation and translation vector performs a change of basis from object coordinate space to camera coordinate space of the first camera of the stereo pair.
tvecs	Output vector of translation vectors estimated for each pattern view, see parameter description of previous output parameter ( rvecs ).
flags	Different flags that may be zero or a combination of the following values: cv::CALIB_FIX_INTRINSIC Fix K1, K2? and D1, D2? so that only R, T matrices are estimated. cv::CALIB_USE_INTRINSIC_GUESS K1, K2 contains valid initial values of fx, fy, cx, cy that are optimized further. Otherwise, (cx, cy) is initially set to the image center (imageSize is used), and focal distances are computed in a least-squares fashion. cv::CALIB_RECOMPUTE_EXTRINSIC Extrinsic will be recomputed after each iteration of intrinsic optimization. cv::CALIB_CHECK_COND The functions will check validity of condition number. cv::CALIB_FIX_SKEW Skew coefficient (alpha) is set to zero and stay zero. cv::CALIB_FIX_K1,..., cv::CALIB_FIX_K4 Selected distortion coefficients are set to zeros and stay zero.
criteria	Termination criteria for the iterative optimization algorithm.