OpenCV
4.10.0-dev
Open Source Computer Vision
|
Prev Tutorial: Using Orbbec Astra 3D cameras
Next Tutorial: Using Creative Senz3D and other Intel RealSense SDK compatible depth sensors
Original author | Jinyue Chen |
Compatibility | OpenCV >= 4.10 |
This tutorial is devoted to the Orbbec 3D cameras based on UVC protocol. For the use of the older Orbbec 3D cameras which depends on OpenNI, please refer to the previous tutorial.
Unlike working with the OpenNI based Astra 3D cameras which requires OpenCV built with OpenNI2 SDK, Orbbec SDK is not required to be installed for accessing Orbbec UVC 3D cameras via OpenCV. By using cv::VideoCapture
class, users get the stream data from 3D cameras, similar to working with USB cameras. The calibration and alignment of the depth map and color image are done internally.
In order to use the 3D cameras with OpenCV. You can refer to Get Started to install OpenCV.
Note since 4.11 on, Mac OS users need to compile OpenCV from source with flag -DOBSENSOR_USE_ORBBEC_SDK=ON
in order to use the cameras:
cv.VideoCapture(0, cv.CAP_OBSENSOR)
to attempt to open the first Orbbec depth sensor device. If the camera fails to open, the program will exit and display an error message.orbbec_cap.grab()
method attempts to grab a frame.orbbec_cap.retrieve(None, cv.CAP_OBSENSOR_BGR_IMAGE)
to retrieve the BGR image data. If successfully retrieved, the BGR image is displayed in a window using cv.imshow("BGR", bgr_image)
.orbbec_cap.retrieve(None, cv.CAP_OBSENSOR_DEPTH_MAP)
to retrieve the depth image data. If successfully retrieved, the depth image is first normalized to a range of 0 to 255, then a false color image is applied, and the result is displayed in a window using cv.imshow("DEPTH", color_depth_map)
.cv.pollKey()
to detect keyboard events. If a key is pressed, the loop breaks and the program ends.orbbec_cap.release()
.VideoCapture obsensorCapture(0, CAP_OBSENSOR)
to attempt to open the first Orbbec depth sensor device. If the camera fails to open, an error message is displayed, and the program exits.obsensorCapture.get()
to retrieve the intrinsic parameters of the camera, including focal lengths (fx
, fy
) and principal points (cx
, cy
).obsensorCapture.grab()
method attempts to grab a frame.obsensorCapture.retrieve(image, CAP_OBSENSOR_BGR_IMAGE)
to retrieve the BGR image data. If successfully retrieved, the BGR image is displayed in a window using imshow("BGR", image)
.obsensorCapture.retrieve(depthMap, CAP_OBSENSOR_DEPTH_MAP)
to retrieve the depth image data. If successfully retrieved, the depth image is normalized and a false color image is applied, then the result is displayed in a window using imshow("DEPTH", adjDepthMap)
. The retrieved depth values are in millimeters and are truncated to a range between 300 and 5000 (millimeter). This fixed range can be interpreted as a truncation based on the depth camera's depth range, removing invalid pixels on the depth map.alpha
). The overlaid image is displayed in a window using imshow("DepthToColor", image)
.pollKey()
to detect keyboard events. If a key is pressed, the loop breaks and the program ends.Mac users need sudo privileges to execute the code.