
Compatibility	OpenCV >= 4.11.0

Enable OpenCV with FastCV for Qualcomm Chipsets

This document scope is to guide the Developers to enable OpenCV Acceleration with FastCV for the Qualcomm chipsets with ARM64 architecture. Entablement of OpenCV with FastCV back-end on non-Qualcomm chipsets or Linux platforms other than Qualcomm Linux is currently out of scope.

About FastCV

FastCV provides two main features to computer vision application developers:

A library of frequently used computer vision (CV) functions, optimized to run efficiently on a wide variety of Qualcomm’s Snapdragon devices.
A clean processor-agnostic hardware acceleration API, under which chipset vendors can hardware accelerate FastCV functions on Qualcomm’s Snapdragon hardware.

FastCV is released as a unified binary, a single binary containing two implementations of the algorithms:

Generic implementation runs on Arm® architecture and is referred to as FastCV for Arm architecture.
Implementation runs only on Qualcomm® Snapdragon™ chipsets and is called FastCV for Snapdragon.

FastCV library is Qualcomm proprietary and provides faster implementation of CV algorithms on various hardware as compared to other CV libraries.

OpenCV Acceleration with FastCV HAL and Extensions

OpenCV and FastCV integration is implemented in two ways:

FastCV-based HAL for basic computer vision and arithmetic algorithms acceleration.
FastCV module in opencv_contrib with custom algorithms and FastCV function wrappers that do not fit generic OpenCV interface or behaviour.

Supported Platforms

Android : Qualcomm Chipsets with the Android from Snapdragon 8 Gen 1 onwards(https://www.qualcomm.com/products/mobile/snapdragon/smartphones#product-list)
Linux : Qualcomm Linux Program related boards mentioned in Hardware

Compiling OpenCV with FastCV for Android

Follow Wiki page for OpenCV Compilation : https://github.com/opencv/opencv/wiki/Custom-OpenCV-Android-SDK-and-AAR-package-build

Once the OpenCV repository code is cloned into the workspace, please add -DWITH_FASTCV=ON flag to cmake vars as below to arm64 entry in opencv/platforms/android/default.config.py or create new one with the option to enable FastCV HAL and/or extenstions compilation:

ABI("3", "arm64-v8a", None, 24, cmake_vars=dict(WITH_FASTCV='ON')),
Remaining steps can be followed as mentioned in the wiki page

Compiling OpenCV with FastCV for Qualcomm Linux

Note: : Only Ubuntu 22.04 is supported as host platform for eSDK deployment.

Install eSDK by following Qualcomm® Linux Documentation
After installing the eSDK, set the ESDK_ROOT:

export ESDK_ROOT=<eSDK install location>
Add SDK tools and libraries to your environment:

source environment-setup-armv8-2a-qcom-linux

If you encounter the following message:
Your environment is misconfigured, you probably need to 'unset LD_LIBRARY_PATH'

but please check why this was set in the first place and that it's safe to unset.

The SDK will not operate correctly in most cases when LD_LIBRARY_PATH is set.

just unset your host LD_LIBRARY_PATH environment variable: unset LD_LIBRARY_PATH.
Clone OpenCV Repositories:

Clone the OpenCV main and optionally opencv_contrib repositories into any directory (it does not need to be inside the SDK directory).

git clone https://github.com/opencv/opencv.git

git clone https://github.com/opencv/opencv_contrib.git
Build OpenCV

Create a build directory, navigate into it and build the project with CMake there:

mkdir build

cd build

cmake -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=aarch64 -DWITH_FASTCV=ON -DBUILD_SHARED_LIBS=ON -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules/fastcv/ ../opencv

make -j$(nproc)

If the FastCV library is updated, please replace the old FastCV libraries located at:
<ESDK_PATH>\qcom-wayland_sdk\tmp\sysroots\qcs6490-rb3gen2-vision-kit\usr\lib

with the latest FastCV libraries downloaded in:
build\3rdparty\fastcv\libs
Validate

Push the OpenCV libraries, test binaries and test data on to the target. Execute the OpenCV conformance or performance tests. During runtime, If libwebp.so.7 lib is missing, find the lib in the below Path and push it on the target
<ESDK_PATH>\qcom-wayland_sdk\tmp\sysroots\qcs6490-rb3gen2-vision-kit\usr\lib\libwebp.so.7

HAL and Extension list of APIs

FastCV based OpenCV HAL APIs list :

OpenCV module	OpenCV API	Underlying FastCV API for OpenCV acceleration
IMGPROC	medianBlur	fcvFilterMedian3x3u8_v3
	sobel	fcvFilterSobel3x3u8s16
		fcvFilterSobel5x5u8s16
		fcvFilterSobel7x7u8s16
	boxFilter	fcvBoxFilter3x3u8_v3
		fcvBoxFilter5x5u8_v2
	adaptiveThreshold	fcvAdaptiveThresholdGaussian3x3u8_v2
		fcvAdaptiveThresholdGaussian5x5u8_v2
		fcvAdaptiveThresholdMean3x3u8_v2
		fcvAdaptiveThresholdMean5x5u8_v2
	pyrUp & pyrDown	fcvPyramidCreateu8_v4
	cvtColor	fcvColorRGB888toYCrCbu8_v3
		fcvColorRGB888ToHSV888u8
	GaussianBlur	fcvFilterGaussian5x5u8_v3
		fcvFilterGaussian3x3u8_v4
	cvWarpPerspective	fcvWarpPerspectiveu8_v5
	Canny	fcvFilterCannyu8

CORE	lut	fcvTableLookupu8
	norm	fcvHammingDistanceu8
	multiply	fcvElementMultiplyu8u16_v2
	transpose	fcvTransposeu8_v2
		fcvTransposeu16_v2
		fcvTransposef32_v2
	meanStdDev	fcvImageIntensityStats_v2
	flip	fcvFlipu8
		fcvFlipu16
		fcvFlipRGB888u8
	rotate	fcvRotateImageu8
		fcvRotateImageInterleavedu8
	multiply	fcvElementMultiplyu8
		fcvElementMultiplys16
		fcvElementMultiplyf32
	addWeighted	fcvAddWeightedu8_v2
	subtract	fcvImageDiffu8f32_v2

FastCV based OpenCV Extensions APIs list :

OpenCV Extension APIs	Underlying FastCV API for OpenCV acceleration
matmuls8s32	fcvMatrixMultiplys8s32
clusterEuclidean	fcvClusterEuclideanu8
FAST10	fcvCornerFast10InMaskScoreu8
	fcvCornerFast10InMasku8
	fcvCornerFast10Scoreu8
	fcvCornerFast10u8
FFT	fcvFFTu8
IFFT	fcvIFFTf32
fillConvexPoly	fcvFillConvexPolyu8
houghLines	fcvHoughLineu8
moments	fcvImageMomentsu8
	fcvImageMomentss32
	fcvImageMomentsf32
runMSER	fcvMserInit
	fcvMserNN8Init
	fcvMserExtu8_v3
	fcvMserExtNN8u8
	fcvMserNN8u8
	fcvMserRelease
remap	fcvRemapu8_v2
remapRGBA	fcvRemapRGBA8888BLu8
	fcvRemapRGBA8888NNu8
resizeDownBy2	fcvScaleDownBy2u8_v2
resizeDownBy4	fcvScaleDownBy4u8_v2
meanShift	fcvMeanShiftu8
	fcvMeanShifts32
	fcvMeanShiftf32
bilateralRecursive	fcvBilateralFilterRecursiveu8
thresholdRange	fcvFilterThresholdRangeu8_v2
bilateralFilter	fcvBilateralFilter5x5u8_v3
	fcvBilateralFilter7x7u8_v3
	fcvBilateralFilter9x9u8_v3
calcHist	fcvImageIntensityHistogram
gaussianBlur	fcvFilterGaussian3x3u8_v4
	fcvFilterGaussian5x5u8_v3
	fcvFilterGaussian5x5s16_v3
	fcvFilterGaussian5x5s32_v3
	fcvFilterGaussian11x11u8_v2
filter2D	fcvFilterCorrNxNu8
	fcvFilterCorrNxNu8s16
	fcvFilterCorrNxNu8f32
sepFilter2D	fcvFilterCorrSepMxNu8
	fcvFilterCorrSep9x9s16_v2
	fcvFilterCorrSep11x11s16_v2
	fcvFilterCorrSep13x13s16_v2
	fcvFilterCorrSep15x15s16_v2
	fcvFilterCorrSep17x17s16_v2
	fcvFilterCorrSepNxNs16
sobel3x3u8	fcvImageGradientSobelPlanars8_v2
sobel3x3u9	fcvImageGradientSobelPlanars16_v2
sobel3x3u10	fcvImageGradientSobelPlanars16_v3
sobel3x3u11	fcvImageGradientSobelPlanarf32_v2
sobel3x3u12	fcvImageGradientSobelPlanarf32_v3
sobel	fcvFilterSobel3x3u8_v2
	fcvFilterSobel3x3u8s16
	fcvFilterSobel5x5u8s16
	fcvFilterSobel7x7u8s16
DCT	fcvDCTu8
iDCT	fcvIDCTs16
sobelPyramid	fcvPyramidAllocate
	fcvPyramidAllocate_v2
	fcvPyramidAllocate_v3
	fcvPyramidSobelGradientCreatei8
	fcvPyramidSobelGradientCreatei16
	fcvPyramidSobelGradientCreatef32
	fcvPyramidDelete
	fcvPyramidDelete_v2
	fcvPyramidCreatef32_v2
	fcvPyramidCreateu8_v4
trackOpticalFlowLK	fcvTrackLKOpticalFlowu8_v3
	fcvTrackLKOpticalFlowu8
warpPerspective2Plane	fcv2PlaneWarpPerspectiveu8