Fisheye camera model

Detailed Description

Definitions: Let P be a point in 3D of coordinates X in the world reference frame (stored in the matrix X) The coordinate vector of P in the camera reference frame is:

\[Xc = R X + T\]

where R is the rotation matrix corresponding to the rotation vector om: R = rodrigues(om); call x, y and z the 3 coordinates of Xc:

\[\begin{array}{l} x = Xc_1 \\ y = Xc_2 \\ z = Xc_3 \end{array} \]

The pinhole projection coordinates of P is [a; b] where

\[\begin{array}{l} a = x / z \ and \ b = y / z \\ r^2 = a^2 + b^2 \\ \theta = atan(r) \end{array} \]

Fisheye distortion:

\[\theta_d = \theta (1 + k_1 \theta^2 + k_2 \theta^4 + k_3 \theta^6 + k_4 \theta^8)\]

The distorted point coordinates are [x'; y'] where

\[\begin{array}{l} x' = (\theta_d / r) a \\ y' = (\theta_d / r) b \end{array} \]

Finally, conversion into pixel coordinates: The final pixel coordinates vector [u; v] where:

\[\begin{array}{l} u = f_x (x' + \alpha y') + c_x \\ v = f_y y' + c_y \end{array} \]

Summary: Generic camera model [143] with perspective projection and without distortion correction

Namespaces
namespace	cv::fisheye
	The methods in this namespace use a so-called fisheye camera model.

Enumerations
enum	{   cv::fisheye::CALIB_USE_INTRINSIC_GUESS = 1 << 0 ,   cv::fisheye::CALIB_RECOMPUTE_EXTRINSIC = 1 << 1 ,   cv::fisheye::CALIB_CHECK_COND = 1 << 2 ,   cv::fisheye::CALIB_FIX_SKEW = 1 << 3 ,   cv::fisheye::CALIB_FIX_K1 = 1 << 4 ,   cv::fisheye::CALIB_FIX_K2 = 1 << 5 ,   cv::fisheye::CALIB_FIX_K3 = 1 << 6 ,   cv::fisheye::CALIB_FIX_K4 = 1 << 7 ,   cv::fisheye::CALIB_FIX_INTRINSIC = 1 << 8 ,   cv::fisheye::CALIB_FIX_PRINCIPAL_POINT = 1 << 9 ,   cv::fisheye::CALIB_ZERO_DISPARITY = 1 << 10 ,   cv::fisheye::CALIB_FIX_FOCAL_LENGTH = 1 << 11 }

Functions
double	cv::fisheye::calibrate (InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints, const Size &image_size, InputOutputArray K, InputOutputArray D, OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs, int flags=0, TermCriteria criteria=TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON))
	Performs camera calibration.
void	cv::fisheye::distortPoints (InputArray undistorted, OutputArray distorted, InputArray K, InputArray D, double alpha=0)
	Distorts 2D points using fisheye model.
void	cv::fisheye::distortPoints (InputArray undistorted, OutputArray distorted, InputArray Kundistorted, InputArray K, InputArray D, double alpha=0)
void	cv::fisheye::estimateNewCameraMatrixForUndistortRectify (InputArray K, InputArray D, const Size &image_size, InputArray R, OutputArray P, double balance=0.0, const Size &new_size=Size(), double fov_scale=1.0)
	Estimates new camera intrinsic matrix for undistortion or rectification.
void	cv::fisheye::initUndistortRectifyMap (InputArray K, InputArray D, InputArray R, InputArray P, const cv::Size &size, int m1type, OutputArray map1, OutputArray map2)
	Computes undistortion and rectification maps for image transform by remap. If D is empty zero distortion is used, if R or P is empty identity matrixes are used.
void	cv::fisheye::projectPoints (InputArray objectPoints, OutputArray imagePoints, const Affine3d &affine, InputArray K, InputArray D, double alpha=0, OutputArray jacobian=noArray())
	Projects points using fisheye model.
void	cv::fisheye::projectPoints (InputArray objectPoints, OutputArray imagePoints, InputArray rvec, InputArray tvec, InputArray K, InputArray D, double alpha=0, OutputArray jacobian=noArray())
bool	cv::fisheye::solvePnP (InputArray objectPoints, InputArray imagePoints, InputArray cameraMatrix, InputArray distCoeffs, OutputArray rvec, OutputArray tvec, bool useExtrinsicGuess=false, int flags=SOLVEPNP_ITERATIVE, TermCriteria criteria=TermCriteria(TermCriteria::MAX_ITER+TermCriteria::EPS, 10, 1e-8))
	Finds an object pose from 3D-2D point correspondences for fisheye camera moodel.
double	cv::fisheye::stereoCalibrate (InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints1, InputArrayOfArrays imagePoints2, InputOutputArray K1, InputOutputArray D1, InputOutputArray K2, InputOutputArray D2, Size imageSize, OutputArray R, OutputArray T, int flags=fisheye::CALIB_FIX_INTRINSIC, TermCriteria criteria=TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON))
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.
double	cv::fisheye::stereoCalibrate (InputArrayOfArrays objectPoints, InputArrayOfArrays imagePoints1, InputArrayOfArrays imagePoints2, InputOutputArray K1, InputOutputArray D1, InputOutputArray K2, InputOutputArray D2, Size imageSize, OutputArray R, OutputArray T, OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs, int flags=fisheye::CALIB_FIX_INTRINSIC, TermCriteria criteria=TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON))
	Performs stereo calibration.
void	cv::fisheye::stereoRectify (InputArray K1, InputArray D1, InputArray K2, InputArray D2, const Size &imageSize, InputArray R, InputArray tvec, OutputArray R1, OutputArray R2, OutputArray P1, OutputArray P2, OutputArray Q, int flags, const Size &newImageSize=Size(), double balance=0.0, double fov_scale=1.0)
	Stereo rectification for fisheye camera model.
void	cv::fisheye::undistortImage (InputArray distorted, OutputArray undistorted, InputArray K, InputArray D, InputArray Knew=cv::noArray(), const Size &new_size=Size())
	Transforms an image to compensate for fisheye lens distortion.
void	cv::fisheye::undistortPoints (InputArray distorted, OutputArray undistorted, InputArray K, InputArray D, InputArray R=noArray(), InputArray P=noArray(), TermCriteria criteria=TermCriteria(TermCriteria::MAX_ITER+TermCriteria::EPS, 10, 1e-8))
	Undistorts 2D points using fisheye model.

Enumeration Type Documentation

anonymous enum

anonymous enum

#include <opencv2/calib3d.hpp>

Enumerator
CALIB_USE_INTRINSIC_GUESS  Python: cv.fisheye.CALIB_USE_INTRINSIC_GUESS
CALIB_RECOMPUTE_EXTRINSIC  Python: cv.fisheye.CALIB_RECOMPUTE_EXTRINSIC
CALIB_CHECK_COND  Python: cv.fisheye.CALIB_CHECK_COND
CALIB_FIX_SKEW  Python: cv.fisheye.CALIB_FIX_SKEW
CALIB_FIX_K1  Python: cv.fisheye.CALIB_FIX_K1
CALIB_FIX_K2  Python: cv.fisheye.CALIB_FIX_K2
CALIB_FIX_K3  Python: cv.fisheye.CALIB_FIX_K3
CALIB_FIX_K4  Python: cv.fisheye.CALIB_FIX_K4
CALIB_FIX_INTRINSIC  Python: cv.fisheye.CALIB_FIX_INTRINSIC
CALIB_FIX_PRINCIPAL_POINT  Python: cv.fisheye.CALIB_FIX_PRINCIPAL_POINT
CALIB_ZERO_DISPARITY  Python: cv.fisheye.CALIB_ZERO_DISPARITY
CALIB_FIX_FOCAL_LENGTH  Python: cv.fisheye.CALIB_FIX_FOCAL_LENGTH

Function Documentation

calibrate()

double cv::fisheye::calibrate	(	InputArrayOfArrays	objectPoints,
		InputArrayOfArrays	imagePoints,
		const Size &	image_size,
		InputOutputArray	K,
		InputOutputArray	D,
		OutputArrayOfArrays	rvecs,
		OutputArrayOfArrays	tvecs,
		int	flags = 0,
		TermCriteria	criteria = TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON) )

Python:
	cv.fisheye.calibrate(	objectPoints, imagePoints, image_size, K, D[, rvecs[, tvecs[, flags[, criteria]]]]	) ->	retval, K, D, rvecs, tvecs

#include <opencv2/calib3d.hpp>

Performs camera calibration.

Parameters

objectPoints	vector of vectors of calibration pattern points in the calibration pattern coordinate space.
imagePoints	vector of vectors of the projections of calibration pattern points. imagePoints.size() and objectPoints.size() and imagePoints[i].size() must be equal to objectPoints[i].size() for each i.
image_size	Size of the image used only to initialize the camera intrinsic matrix.
K	Output 3x3 floating-point camera intrinsic matrix \(\cameramatrix{A}\) . If fisheye::CALIB_USE_INTRINSIC_GUESS is specified, some or all of fx, fy, cx, cy must be initialized before calling the function.
D	Output vector of distortion coefficients \(\distcoeffsfisheye\).
rvecs	Output vector of rotation vectors (see Rodrigues ) estimated for each pattern view. That is, each k-th rotation vector together with the corresponding k-th translation vector (see the next output parameter description) brings the calibration pattern from the model coordinate space (in which object points are specified) to the world coordinate space, that is, a real position of the calibration pattern in the k-th pattern view (k=0.. M -1).
tvecs	Output vector of translation vectors estimated for each pattern view.
flags	Different flags that may be zero or a combination of the following values: fisheye::CALIB_USE_INTRINSIC_GUESS cameraMatrix contains valid initial values of fx, fy, cx, cy that are optimized further. Otherwise, (cx, cy) is initially set to the image center ( imageSize is used), and focal distances are computed in a least-squares fashion. fisheye::CALIB_RECOMPUTE_EXTRINSIC Extrinsic will be recomputed after each iteration of intrinsic optimization. fisheye::CALIB_CHECK_COND The functions will check validity of condition number. fisheye::CALIB_FIX_SKEW Skew coefficient (alpha) is set to zero and stay zero. fisheye::CALIB_FIX_K1,..., fisheye::CALIB_FIX_K4 Selected distortion coefficients are set to zeros and stay zero. fisheye::CALIB_FIX_PRINCIPAL_POINT The principal point is not changed during the global optimization. It stays at the center or at a different location specified when fisheye::CALIB_USE_INTRINSIC_GUESS is set too. fisheye::CALIB_FIX_FOCAL_LENGTH The focal length is not changed during the global optimization. It is the \(max(width,height)/\pi\) or the provided \(f_x\), \(f_y\) when fisheye::CALIB_USE_INTRINSIC_GUESS is set too.
criteria	Termination criteria for the iterative optimization algorithm.

Here is the call graph for this function:

distortPoints() [1/2]

void cv::fisheye::distortPoints	(	InputArray	undistorted,
		OutputArray	distorted,
		InputArray	K,
		InputArray	D,
		double	alpha = 0 )

Python:
	cv.fisheye.distortPoints(	undistorted, K, D[, distorted[, alpha]]	) ->	distorted
	cv.fisheye.distortPoints(	undistorted, Kundistorted, K, D[, distorted[, alpha]]	) ->	distorted

#include <opencv2/calib3d.hpp>

Distorts 2D points using fisheye model.

Parameters

undistorted	Array of object points, 1xN/Nx1 2-channel (or vector<Point2f> ), where N is the number of points in the view.
K	Camera intrinsic matrix \(\cameramatrix{K}\).
D	Input vector of distortion coefficients \(\distcoeffsfisheye\).
alpha	The skew coefficient.
distorted	Output array of image points, 1xN/Nx1 2-channel, or vector<Point2f> .

Note that the function assumes the camera intrinsic matrix of the undistorted points to be identity. This means if you want to distort image points you have to multiply them with \(K^{-1}\) or use another function overload.

Here is the call graph for this function:

distortPoints() [2/2]

void cv::fisheye::distortPoints	(	InputArray	undistorted,
		OutputArray	distorted,
		InputArray	Kundistorted,
		InputArray	K,
		InputArray	D,
		double	alpha = 0 )

Python:
	cv.fisheye.distortPoints(	undistorted, K, D[, distorted[, alpha]]	) ->	distorted
	cv.fisheye.distortPoints(	undistorted, Kundistorted, K, D[, distorted[, alpha]]	) ->	distorted

#include <opencv2/calib3d.hpp>

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts. Overload of distortPoints function to handle cases when undistorted points are obtained with non-identity camera matrix, e.g. output of estimateNewCameraMatrixForUndistortRectify.

Parameters

undistorted	Array of object points, 1xN/Nx1 2-channel (or vector<Point2f> ), where N is the number of points in the view.
Kundistorted	Camera intrinsic matrix used as new camera matrix for undistortion.
K	Camera intrinsic matrix \(\cameramatrix{K}\).
D	Input vector of distortion coefficients \(\distcoeffsfisheye\).
alpha	The skew coefficient.
distorted	Output array of image points, 1xN/Nx1 2-channel, or vector<Point2f> .

See also

estimateNewCameraMatrixForUndistortRectify

Here is the call graph for this function:

estimateNewCameraMatrixForUndistortRectify()

void cv::fisheye::estimateNewCameraMatrixForUndistortRectify	(	InputArray	K,
		InputArray	D,
		const Size &	image_size,
		InputArray	R,
		OutputArray	P,
		double	balance = 0.0,
		const Size &	new_size = Size(),
		double	fov_scale = 1.0 )

Python:
	cv.fisheye.estimateNewCameraMatrixForUndistortRectify(	K, D, image_size, R[, P[, balance[, new_size[, fov_scale]]]]	) ->	P

#include <opencv2/calib3d.hpp>

Estimates new camera intrinsic matrix for undistortion or rectification.

Parameters

K	Camera intrinsic matrix \(\cameramatrix{K}\).
image_size	Size of the image
D	Input vector of distortion coefficients \(\distcoeffsfisheye\).
R	Rectification transformation in the object space: 3x3 1-channel, or vector: 3x1/1x3 1-channel or 1x1 3-channel
P	New camera intrinsic matrix (3x3) or new projection matrix (3x4)
balance	Sets the new focal length in range between the min focal length and the max focal length. Balance is in range of [0, 1].
new_size	the new size
fov_scale	Divisor for new focal length.

Here is the call graph for this function:

initUndistortRectifyMap()

void cv::fisheye::initUndistortRectifyMap	(	InputArray	K,
		InputArray	D,
		InputArray	R,
		InputArray	P,
		const cv::Size &	size,
		int	m1type,
		OutputArray	map1,
		OutputArray	map2 )

Python:
	cv.fisheye.initUndistortRectifyMap(	K, D, R, P, size, m1type[, map1[, map2]]	) ->	map1, map2

#include <opencv2/calib3d.hpp>

Computes undistortion and rectification maps for image transform by remap. If D is empty zero distortion is used, if R or P is empty identity matrixes are used.

Parameters

K	Camera intrinsic matrix \(\cameramatrix{K}\).
D	Input vector of distortion coefficients \(\distcoeffsfisheye\).
R	Rectification transformation in the object space: 3x3 1-channel, or vector: 3x1/1x3 1-channel or 1x1 3-channel
P	New camera intrinsic matrix (3x3) or new projection matrix (3x4)
size	Undistorted image size.
m1type	Type of the first output map that can be CV_32FC1 or CV_16SC2 . See convertMaps for details.
map1	The first output map.
map2	The second output map.

projectPoints() [1/2]

void cv::fisheye::projectPoints	(	InputArray	objectPoints,
		OutputArray	imagePoints,
		const Affine3d &	affine,
		InputArray	K,
		InputArray	D,
		double	alpha = 0,
		OutputArray	jacobian = noArray() )

Python:
	cv.fisheye.projectPoints(	objectPoints, rvec, tvec, K, D[, imagePoints[, alpha[, jacobian]]]	) ->	imagePoints, jacobian

#include <opencv2/calib3d.hpp>

Projects points using fisheye model.

Parameters

objectPoints	Array of object points, 1xN/Nx1 3-channel (or vector<Point3f> ), where N is the number of points in the view.
imagePoints	Output array of image points, 2xN/Nx2 1-channel or 1xN/Nx1 2-channel, or vector<Point2f>.
affine
K	Camera intrinsic matrix \(\cameramatrix{K}\).
D	Input vector of distortion coefficients \(\distcoeffsfisheye\).
alpha	The skew coefficient.
jacobian	Optional output 2Nx15 jacobian matrix of derivatives of image points with respect to components of the focal lengths, coordinates of the principal point, distortion coefficients, rotation vector, translation vector, and the skew. In the old interface different components of the jacobian are returned via different output parameters.

The function computes projections of 3D points to the image plane given intrinsic and extrinsic camera parameters. Optionally, the function computes Jacobians - matrices of partial derivatives of image points coordinates (as functions of all the input parameters) with respect to the particular parameters, intrinsic and/or extrinsic.

projectPoints() [2/2]

void cv::fisheye::projectPoints	(	InputArray	objectPoints,
		OutputArray	imagePoints,
		InputArray	rvec,
		InputArray	tvec,
		InputArray	K,
		InputArray	D,
		double	alpha = 0,
		OutputArray	jacobian = noArray() )

Python:
	cv.fisheye.projectPoints(	objectPoints, rvec, tvec, K, D[, imagePoints[, alpha[, jacobian]]]	) ->	imagePoints, jacobian

#include <opencv2/calib3d.hpp>

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

solvePnP()

bool cv::fisheye::solvePnP	(	InputArray	objectPoints,
		InputArray	imagePoints,
		InputArray	cameraMatrix,
		InputArray	distCoeffs,
		OutputArray	rvec,
		OutputArray	tvec,
		bool	useExtrinsicGuess = false,
		int	flags = SOLVEPNP_ITERATIVE,
		TermCriteria	criteria = TermCriteria(TermCriteria::MAX_ITER+TermCriteria::EPS, 10, 1e-8) )

Python:
	cv.fisheye.solvePnP(	objectPoints, imagePoints, cameraMatrix, distCoeffs[, rvec[, tvec[, useExtrinsicGuess[, flags[, criteria]]]]]	) ->	retval, rvec, tvec

#include <opencv2/calib3d.hpp>

Finds an object pose from 3D-2D point correspondences for fisheye camera moodel.

Parameters

objectPoints	Array of object points in the object coordinate space, Nx3 1-channel or 1xN/Nx1 3-channel, where N is the number of points. vector<Point3d> can be also passed here.
imagePoints	Array of corresponding image points, Nx2 1-channel or 1xN/Nx1 2-channel, where N is the number of points. vector<Point2d> can be also passed here.
cameraMatrix	Input camera intrinsic matrix \(\cameramatrix{A}\) .
distCoeffs	Input vector of distortion coefficients (4x1/1x4).
rvec	Output rotation vector (see Rodrigues ) that, together with tvec, brings points from the model coordinate system to the camera coordinate system.
tvec	Output translation vector.
useExtrinsicGuess	Parameter used for SOLVEPNP_ITERATIVE. If true (1), the function uses the provided rvec and tvec values as initial approximations of the rotation and translation vectors, respectively, and further optimizes them.
flags	Method for solving a PnP problem: see calib3d_solvePnP_flags This function returns the rotation and the translation vectors that transform a 3D point expressed in the object coordinate frame to the camera coordinate frame, using different methods: P3P methods (SOLVEPNP_P3P, SOLVEPNP_AP3P): need 4 input points to return a unique solution. SOLVEPNP_IPPE Input points must be >= 4 and object points must be coplanar. SOLVEPNP_IPPE_SQUARE Special case suitable for marker pose estimation. Number of input points must be 4. Object points must be defined in the following order: point 0: [-squareLength / 2, squareLength / 2, 0] point 1: [ squareLength / 2, squareLength / 2, 0] point 2: [ squareLength / 2, -squareLength / 2, 0] point 3: [-squareLength / 2, -squareLength / 2, 0] for all the other flags, number of input points must be >= 4 and object points can be in any configuration.
criteria	Termination criteria for internal undistortPoints call. The function interally undistorts points with undistortPoints and call cv::solvePnP, thus the input are very similar. More information about Perspective-n-Points is described in Perspective-n-Point (PnP) pose computation for more information.

stereoCalibrate() [1/2]

double cv::fisheye::stereoCalibrate	(	InputArrayOfArrays	objectPoints,
		InputArrayOfArrays	imagePoints1,
		InputArrayOfArrays	imagePoints2,
		InputOutputArray	K1,
		InputOutputArray	D1,
		InputOutputArray	K2,
		InputOutputArray	D2,
		Size	imageSize,
		OutputArray	R,
		OutputArray	T,
		int	flags = fisheye::CALIB_FIX_INTRINSIC,
		TermCriteria	criteria = TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON) )

Python:
	cv.fisheye.stereoCalibrate(	objectPoints, imagePoints1, imagePoints2, K1, D1, K2, D2, imageSize[, R[, T[, rvecs[, tvecs[, flags[, criteria]]]]]]	) ->	retval, K1, D1, K2, D2, R, T, rvecs, tvecs
	cv.fisheye.stereoCalibrate(	objectPoints, imagePoints1, imagePoints2, K1, D1, K2, D2, imageSize[, R[, T[, flags[, criteria]]]]	) ->	retval, K1, D1, K2, D2, R, T

#include <opencv2/calib3d.hpp>

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

stereoCalibrate() [2/2]

double cv::fisheye::stereoCalibrate	(	InputArrayOfArrays	objectPoints,
		InputArrayOfArrays	imagePoints1,
		InputArrayOfArrays	imagePoints2,
		InputOutputArray	K1,
		InputOutputArray	D1,
		InputOutputArray	K2,
		InputOutputArray	D2,
		Size	imageSize,
		OutputArray	R,
		OutputArray	T,
		OutputArrayOfArrays	rvecs,
		OutputArrayOfArrays	tvecs,
		int	flags = fisheye::CALIB_FIX_INTRINSIC,
		TermCriteria	criteria = TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 100, DBL_EPSILON) )

Python:
	cv.fisheye.stereoCalibrate(	objectPoints, imagePoints1, imagePoints2, K1, D1, K2, D2, imageSize[, R[, T[, rvecs[, tvecs[, flags[, criteria]]]]]]	) ->	retval, K1, D1, K2, D2, R, T, rvecs, tvecs
	cv.fisheye.stereoCalibrate(	objectPoints, imagePoints1, imagePoints2, K1, D1, K2, D2, imageSize[, R[, T[, flags[, criteria]]]]	) ->	retval, K1, D1, K2, D2, R, T

#include <opencv2/calib3d.hpp>

Performs stereo calibration.

Parameters

objectPoints	Vector of vectors of the calibration pattern points.
imagePoints1	Vector of vectors of the projections of the calibration pattern points, observed by the first camera.
imagePoints2	Vector of vectors of the projections of the calibration pattern points, observed by the second camera.
K1	Input/output first camera intrinsic matrix: \(\vecthreethree{f_x^{(j)}}{0}{c_x^{(j)}}{0}{f_y^{(j)}}{c_y^{(j)}}{0}{0}{1}\) , \(j = 0,\, 1\) . If any of fisheye::CALIB_USE_INTRINSIC_GUESS , fisheye::CALIB_FIX_INTRINSIC are specified, some or all of the matrix components must be initialized.
D1	Input/output vector of distortion coefficients \(\distcoeffsfisheye\) of 4 elements.
K2	Input/output second camera intrinsic matrix. The parameter is similar to K1 .
D2	Input/output lens distortion coefficients for the second camera. The parameter is similar to D1 .
imageSize	Size of the image used only to initialize camera intrinsic matrix.
R	Output rotation matrix between the 1st and the 2nd camera coordinate systems.
T	Output translation vector between the coordinate systems of the cameras.
rvecs	Output vector of rotation vectors ( Rodrigues ) estimated for each pattern view in the coordinate system of the first camera of the stereo pair (e.g. std::vector<cv::Mat>). More in detail, each i-th rotation vector together with the corresponding i-th translation vector (see the next output parameter description) brings the calibration pattern from the object coordinate space (in which object points are specified) to the camera coordinate space of the first camera of the stereo pair. In more technical terms, the tuple of the i-th rotation and translation vector performs a change of basis from object coordinate space to camera coordinate space of the first camera of the stereo pair.
tvecs	Output vector of translation vectors estimated for each pattern view, see parameter description of previous output parameter ( rvecs ).
flags	Different flags that may be zero or a combination of the following values: fisheye::CALIB_FIX_INTRINSIC Fix K1, K2? and D1, D2? so that only R, T matrices are estimated. fisheye::CALIB_USE_INTRINSIC_GUESS K1, K2 contains valid initial values of fx, fy, cx, cy that are optimized further. Otherwise, (cx, cy) is initially set to the image center (imageSize is used), and focal distances are computed in a least-squares fashion. fisheye::CALIB_RECOMPUTE_EXTRINSIC Extrinsic will be recomputed after each iteration of intrinsic optimization. fisheye::CALIB_CHECK_COND The functions will check validity of condition number. fisheye::CALIB_FIX_SKEW Skew coefficient (alpha) is set to zero and stay zero. fisheye::CALIB_FIX_K1,..., fisheye::CALIB_FIX_K4 Selected distortion coefficients are set to zeros and stay zero.
criteria	Termination criteria for the iterative optimization algorithm.

stereoRectify()

void cv::fisheye::stereoRectify	(	InputArray	K1,
		InputArray	D1,
		InputArray	K2,
		InputArray	D2,
		const Size &	imageSize,
		InputArray	R,
		InputArray	tvec,
		OutputArray	R1,
		OutputArray	R2,
		OutputArray	P1,
		OutputArray	P2,
		OutputArray	Q,
		int	flags,
		const Size &	newImageSize = Size(),
		double	balance = 0.0,
		double	fov_scale = 1.0 )

Python:
	cv.fisheye.stereoRectify(	K1, D1, K2, D2, imageSize, R, tvec, flags[, R1[, R2[, P1[, P2[, Q[, newImageSize[, balance[, fov_scale]]]]]]]]	) ->	R1, R2, P1, P2, Q

#include <opencv2/calib3d.hpp>

Stereo rectification for fisheye camera model.

Parameters

K1	First camera intrinsic matrix.
D1	First camera distortion parameters.
K2	Second camera intrinsic matrix.
D2	Second camera distortion parameters.
imageSize	Size of the image used for stereo calibration.
R	Rotation matrix between the coordinate systems of the first and the second cameras.
tvec	Translation vector between coordinate systems of the cameras.
R1	Output 3x3 rectification transform (rotation matrix) for the first camera.
R2	Output 3x3 rectification transform (rotation matrix) for the second camera.
P1	Output 3x4 projection matrix in the new (rectified) coordinate systems for the first camera.
P2	Output 3x4 projection matrix in the new (rectified) coordinate systems for the second camera.
Q	Output \(4 \times 4\) disparity-to-depth mapping matrix (see reprojectImageTo3D ).
flags	Operation flags that may be zero or fisheye::CALIB_ZERO_DISPARITY . If the flag is set, the function makes the principal points of each camera have the same pixel coordinates in the rectified views. And if the flag is not set, the function may still shift the images in the horizontal or vertical direction (depending on the orientation of epipolar lines) to maximize the useful image area.
newImageSize	New image resolution after rectification. The same size should be passed to initUndistortRectifyMap (see the stereo_calib.cpp sample in OpenCV samples directory). When (0,0) is passed (default), it is set to the original imageSize . Setting it to larger value can help you preserve details in the original image, especially when there is a big radial distortion.
balance	Sets the new focal length in range between the min focal length and the max focal length. Balance is in range of [0, 1].
fov_scale	Divisor for new focal length.

undistortImage()

void cv::fisheye::undistortImage	(	InputArray	distorted,
		OutputArray	undistorted,
		InputArray	K,
		InputArray	D,
		InputArray	Knew = cv::noArray(),
		const Size &	new_size = Size() )

Python:
	cv.fisheye.undistortImage(	distorted, K, D[, undistorted[, Knew[, new_size]]]	) ->	undistorted

#include <opencv2/calib3d.hpp>

Transforms an image to compensate for fisheye lens distortion.

Parameters

distorted	image with fisheye lens distortion.
undistorted	Output image with compensated fisheye lens distortion.
K	Camera intrinsic matrix \(\cameramatrix{K}\).
D	Input vector of distortion coefficients \(\distcoeffsfisheye\).
Knew	Camera intrinsic matrix of the distorted image. By default, it is the identity matrix but you may additionally scale and shift the result by using a different matrix.
new_size	the new size

The function transforms an image to compensate radial and tangential lens distortion.

The function is simply a combination of fisheye::initUndistortRectifyMap (with unity R ) and remap (with bilinear interpolation). See the former function for details of the transformation being performed.

See below the results of undistortImage.

• a) result of undistort of perspective camera model (all possible coefficients (k_1, k_2, k_3, k_4, k_5, k_6) of distortion were optimized under calibration)
  • b) result of fisheye::undistortImage of fisheye camera model (all possible coefficients (k_1, k_2, k_3, k_4) of fisheye distortion were optimized under calibration)
  • c) original image was captured with fisheye lens

Pictures a) and b) almost the same. But if we consider points of image located far from the center of image, we can notice that on image a) these points are distorted.

image

Here is the call graph for this function:

undistortPoints()

void cv::fisheye::undistortPoints	(	InputArray	distorted,
		OutputArray	undistorted,
		InputArray	K,
		InputArray	D,
		InputArray	R = noArray(),
		InputArray	P = noArray(),
		TermCriteria	criteria = TermCriteria(TermCriteria::MAX_ITER+TermCriteria::EPS, 10, 1e-8) )

Python:
	cv.fisheye.undistortPoints(	distorted, K, D[, undistorted[, R[, P[, criteria]]]]	) ->	undistorted

#include <opencv2/calib3d.hpp>

Undistorts 2D points using fisheye model.

Parameters

distorted	Array of object points, 1xN/Nx1 2-channel (or vector<Point2f> ), where N is the number of points in the view.
K	Camera intrinsic matrix \(\cameramatrix{K}\).
D	Input vector of distortion coefficients \(\distcoeffsfisheye\).
R	Rectification transformation in the object space: 3x3 1-channel, or vector: 3x1/1x3 1-channel or 1x1 3-channel
P	New camera intrinsic matrix (3x3) or new projection matrix (3x4)
criteria	Termination criteria
undistorted	Output array of image points, 1xN/Nx1 2-channel, or vector<Point2f> .