Introduction
This tutorial guidelines how to run your models in OpenCV deep learning module using Halide language backend. Halide is an open-source project that let us write image processing algorithms in well-readable format, schedule computations according to specific device and evaluate it with a quite good efficiency.
An official website of the Halide project: http://halide-lang.org/.
An up to date efficiency comparison: https://github.com/opencv/opencv/wiki/DNN-Efficiency
Requirements
LLVM compiler
- Note
- LLVM compilation might take a long time.
- Download LLVM source code from http://releases.llvm.org/4.0.0/llvm-4.0.0.src.tar.xz. Unpack it. Let llvm_root is a root directory of source code.
- Create directory llvm_root/tools/clang
- Download Clang with the same version as LLVM. In our case it will be from http://releases.llvm.org/4.0.0/cfe-4.0.0.src.tar.xz. Unpack it into llvm_root/tools/clang. Note that it should be a root for Clang source code.
- Build LLVM on Linux
cd llvm_root
mkdir build && cd build
cmake -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_TARGETS_TO_BUILD="X86" -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release ..
make -j4
- Build LLVM on Windows (Developer Command Prompt)
mkdir \\path-to-llvm-build\\ && cd \\path-to-llvm-build\\
cmake.exe -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=\\path-to-llvm-install\\ -G "Visual Studio 14 Win64" \\path-to-llvm-src\\
MSBuild.exe /m:4 /t:Build /p:Configuration=Release .\\INSTALL.vcxproj
- Note
\\path-to-llvm-build\\
and \\path-to-llvm-install\\
are different directories.
Halide language.
- Download source code from GitHub repository, https://github.com/halide/Halide or using git. The root directory will be a halide_root.
git clone https://github.com/halide/Halide.git
- Build Halide on Linux
cd halide_root
mkdir build && cd build
cmake -DLLVM_DIR=llvm_root/build/lib/cmake/llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_VERSION=40 -DWITH_TESTS=OFF -DWITH_APPS=OFF -DWITH_TUTORIALS=OFF ..
make -j4
- Build Halide on Windows (Developer Command Prompt)
cd halide_root
mkdir build && cd build
cmake.exe -DLLVM_DIR=\\path-to-llvm-install\\lib\\cmake\\llvm -DLLVM_VERSION=40 -DWITH_TESTS=OFF -DWITH_APPS=OFF -DWITH_TUTORIALS=OFF -DCMAKE_BUILD_TYPE=Release -G "Visual Studio 14 Win64" ..
MSBuild.exe /m:4 /t:Build /p:Configuration=Release .\\ALL_BUILD.vcxproj
Build OpenCV with Halide backend
When you build OpenCV add the following configuration flags:
ENABLE_CXX11
- enable C++11 standard
WITH_HALIDE
- enable Halide linkage
HALIDE_ROOT_DIR
- path to Halide build directory
Sample
#include <fstream>
#include <iostream>
#include <cstdlib>
static void getMaxClass(
const Mat &probBlob,
int *classId,
double *classProb)
{
minMaxLoc(probMat, NULL, classProb, NULL, &classNumber);
*classId = classNumber.
x;
}
static std::vector<std::string> readClassNames(const char *filename = "synset_words.txt")
{
std::vector<std::string> classNames;
std::ifstream fp(filename);
if (!fp.is_open())
{
std::cerr << "File with classes labels not found: " << filename << std::endl;
exit(-1);
}
std::string name;
while (!fp.eof())
{
std::getline(fp, name);
if (name.length())
classNames.push_back( name.substr(name.find(' ')+1) );
}
fp.close();
return classNames;
}
int main(int argc, char **argv)
{
std::string modelTxt = "train_val.prototxt";
std::string modelBin = "squeezenet_v1.1.caffemodel";
std::string imageFile = (argc > 1) ? argv[1] : "space_shuttle.jpg";
{
std::cerr << "Can't load network by using the following files: " << std::endl;
std::cerr << "prototxt: " << modelTxt << std::endl;
std::cerr << "caffemodel: " << modelBin << std::endl;
std::cerr << "SqueezeNet v1.1 can be downloaded from:" << std::endl;
std::cerr << "https://github.com/DeepScale/SqueezeNet/tree/master/SqueezeNet_v1.1" << std::endl;
exit(-1);
}
{
std::cerr << "Can't read image from the file: " << imageFile << std::endl;
exit(-1);
}
{
std::cerr << "Image " << imageFile << " isn't 3-channel" << std::endl;
exit(-1);
}
int classId;
double classProb;
getMaxClass(prob, &classId, &classProb);
std::vector<std::string> classNames = readClassNames();
std::cout << "Best class: #" << classId << " '" << classNames.at(classId) << "'" << std::endl;
std::cout << "Probability: " << classProb * 100 << "%" << std::endl;
return 0;
}
Explanation
Download Caffe model from SqueezeNet repository: train_val.prototxt and squeezenet_v1.1.caffemodel.
Also you need file with names of ILSVRC2012 classes: synset_words.txt.
Put these files into working dir of this program example.
- Read and initialize network using path to .prototxt and .caffemodel files
- Check that network was read successfully
if (net.empty())
{
std::cerr << "Can't load network by using the following files: " << std::endl;
std::cerr << "prototxt: " << modelTxt << std::endl;
std::cerr << "caffemodel: " << modelBin << std::endl;
std::cerr << "SqueezeNet v1.1 can be downloaded from:" << std::endl;
std::cerr << "https://github.com/DeepScale/SqueezeNet/tree/master/SqueezeNet_v1.1" << std::endl;
exit(-1);
}
- Read input image and convert to the 4-dimensional blob, acceptable by SqueezeNet v1.1
if (img.empty())
{
std::cerr << "Can't read image from the file: " << imageFile << std::endl;
exit(-1);
}
if (img.channels() != 3)
{
std::cerr << "Image " << imageFile << " isn't 3-channel" << std::endl;
exit(-1);
}
- Pass the blob to the network
- Enable Halide backend for layers where it is implemented
- Make forward pass
Mat prob = net.forward("prob");
Remember that the first forward pass after initialization require quite more time that the next ones. It's because of runtime compilation of Halide pipelines at the first invocation.
- Determine the best class
int classId;
double classProb;
getMaxClass(prob, &classId, &classProb);
- Print results
std::vector<std::string> classNames = readClassNames();
std::cout << "Best class: #" << classId << " '" << classNames.at(classId) << "'" << std::endl;
std::cout << "Probability: " << classProb * 100 << "%" << std::endl;
For our image we get:
Best class: #812 'space shuttle'
Probability: 97.9812%