Random trees have been introduced by Leo Breiman and Adele Cutler: http://www.stat.berkeley.edu/users/breiman/RandomForests/ . The algorithm can deal with both classification and regression problems. Random trees is a collection (ensemble) of tree predictors that is called forest further in this section (the term has been also introduced by L. Breiman). The classification works as follows: the random trees classifier takes the input feature vector, classifies it with every tree in the forest, and outputs the class label that received the majority of “votes”. In case of a regression, the classifier response is the average of the responses over all the trees in the forest.
All the trees are trained with the same parameters but on different training sets. These sets are generated from the original training set using the bootstrap procedure: for each training set, you randomly select the same number of vectors as in the original set ( =N
). The vectors are chosen with replacement. That is, some vectors will occur more than once and some will be absent. At each node of each trained tree, not all the variables are used to find the best split, but a random subset of them. With each node a new subset is generated. However, its size is fixed for all the nodes and all the trees. It is a training parameter set to
by default. None of the built trees are pruned.
In random trees there is no need for any accuracy estimation procedures, such as crossvalidation or bootstrap, or a separate test set to get an estimate of the training error. The error is estimated internally during the training. When the training set for the current tree is drawn by sampling with replacement, some vectors are left out (socalled
oob (outofbag) data
). The size of oob data is about N/3
. The classification error is estimated by using this oobdata as follows:
For the random trees usage example, please, see letter_recog.cpp sample in OpenCV distribution.
References:
 Machine Learning, Wald I, July 2002. http://statwww.berkeley.edu/users/breiman/wald20021.pdf
 Looking Inside the Black Box, Wald II, July 2002. http://statwww.berkeley.edu/users/breiman/wald20022.pdf
 Software for the Masses, Wald III, July 2002. http://statwww.berkeley.edu/users/breiman/wald20023.pdf
 And other articles from the web site http://www.stat.berkeley.edu/users/breiman/RandomForests/cc_home.htm
CvRTParams
: public CvDTreeParams
¶Training parameters of random trees.
The set of training parameters for the forest is a superset of the training parameters for a single tree. However, random trees do not need all the functionality/features of decision trees. Most noticeably, the trees are not pruned, so the crossvalidation parameters are not used.
The constructors.
CvRTParams::
CvRTParams
()¶
CvRTParams::
CvRTParams
(int max_depth, int min_sample_count, float regression_accuracy, bool use_surrogates, int max_categories, const float* priors, bool calc_var_importance, int nactive_vars, int max_num_of_trees_in_the_forest, float forest_accuracy, int termcrit_type)¶Parameters: 


For meaning of other parameters see CvDTreeParams::CvDTreeParams()
.
The default constructor sets all parameters to default values which are different from default values of CvDTreeParams
:
CvRTParams::CvRTParams() : CvDTreeParams( 5, 10, 0, false, 10, 0, false, false, 0 ),
calc_var_importance(false), nactive_vars(0)
{
term_crit = cvTermCriteria( CV_TERMCRIT_ITER+CV_TERMCRIT_EPS, 50, 0.1 );
}
CvRTrees
: public CvStatModel
¶The class implements the random forest predictor as described in the beginning of this section.
Trains the Random Trees model.
bool CvRTrees::
train
(const Mat& trainData, int tflag, const Mat& responses, const Mat& varIdx=Mat(), const Mat& sampleIdx=Mat(), const Mat& varType=Mat(), const Mat& missingDataMask=Mat(), CvRTParams params=CvRTParams() )¶
bool CvRTrees::
train
(const CvMat* trainData, int tflag, const CvMat* responses, const CvMat* varIdx=0, const CvMat* sampleIdx=0, const CvMat* varType=0, const CvMat* missingDataMask=0, CvRTParams params=CvRTParams() )¶
bool CvRTrees::
train
(CvMLData* data, CvRTParams params=CvRTParams() )¶
cv2.RTrees.
train
(trainData, tflag, responses[, varIdx[, sampleIdx[, varType[, missingDataMask[, params]]]]]) → retval¶The method CvRTrees::train()
is very similar to the method CvDTree::train()
and follows the generic method CvStatModel::train()
conventions. All the parameters specific to the algorithm training are passed as a CvRTParams
instance. The estimate of the training error (ooberror
) is stored in the protected class member oob_error
.
The function is parallelized with the TBB library.
Predicts the output for an input sample.
float CvRTrees::
predict
(const Mat& sample, const Mat& missing=Mat() ) const
¶
float CvRTrees::
predict
(const CvMat* sample, const CvMat* missing=0 ) const
¶
cv2.RTrees.
predict
(sample[, missing]) → retval¶Parameters: 


The input parameters of the prediction method are the same as in CvDTree::predict()
but the return value type is different. This method returns the cumulative result from all the trees in the forest (the class that receives the majority of voices, or the mean of the regression function estimates).
Returns a fuzzypredicted class label.
float CvRTrees::
predict_prob
(const cv::Mat& sample, const cv::Mat& missing=cv::Mat() ) const
¶
float CvRTrees::
predict_prob
(const CvMat* sample, const CvMat* missing=0 ) const
¶
cv2.RTrees.
predict_prob
(sample[, missing]) → retval¶Parameters: 


The function works for binary classification problems only. It returns the number between 0 and 1. This number represents probability or confidence of the sample belonging to the second class. It is calculated as the proportion of decision trees that classified the sample to the second class.
Returns the variable importance array.
Mat CvRTrees::
getVarImportance
()¶
const CvMat* CvRTrees::
get_var_importance
()¶
cv2.RTrees.
getVarImportance
() → retval¶The method returns the variable importance vector, computed at the training stage when CvRTParams::calc_var_importance
is set to true. If this flag was set to false, the NULL
pointer is returned. This differs from the decision trees where variable importance can be computed anytime after the training.
Retrieves the proximity measure between two training samples.
float CvRTrees::
get_proximity
(const CvMat* sample1, const CvMat* sample2, const CvMat* missing1=0, const CvMat* missing2=0 ) const
¶Parameters: 


The method returns proximity measure between any two samples. This is a ratio of those trees in the ensemble, in which the samples fall into the same leaf node, to the total number of the trees.
Returns error of the random forest.
float CvRTrees::
calc_error
(CvMLData* data, int type, std::vector<float>* resp=0 )¶The method is identical to CvDTree::calc_error()
but uses the random forest as predictor.
Returns the train error.
float CvRTrees::
get_train_error
()¶The method works for classification problems only. It returns the proportion of incorrectly classified train samples.
Returns the state of the used random number generator.
CvRNG* CvRTrees::
get_rng
()¶