The Expectation Maximization(EM) algorithm estimates the parameters of the multivariate probability density function in the form of a Gaussian mixture distribution with a specified number of mixtures.
Consider the set of the N feature vectors { } from a d-dimensional Euclidean space drawn from a Gaussian mixture:
where is the number of mixtures, is the normal distribution density with the mean and covariance matrix , is the weight of the k-th mixture. Given the number of mixtures and the samples , the algorithm finds the maximum-likelihood estimates (MLE) of all the mixture parameters, that is, , and :
The EM algorithm is an iterative procedure. Each iteration includes
two steps. At the first step (Expectation step or E-step), you find a
probability
(denoted
in the formula below) of
sample i
to belong to mixture k
using the currently
available mixture parameter estimates:
At the second step (Maximization step or M-step), the mixture parameter estimates are refined using the computed probabilities:
Alternatively, the algorithm may start with the M-step when the initial values for
can be provided. Another alternative when
are unknown is to use a simpler clustering algorithm to pre-cluster the input samples and thus obtain initial
. Often (including machine learning) the
kmeans()
algorithm is used for that purpose.
One of the main problems of the EM algorithm is a large number of parameters to estimate. The majority of the parameters reside in covariance matrices, which are elements each where is the feature space dimensionality. However, in many practical problems, the covariance matrices are close to diagonal or even to , where is an identity matrix and is a mixture-dependent “scale” parameter. So, a robust computation scheme could start with harder constraints on the covariance matrices and then use the estimated parameters as an input for a less constrained optimization problem (often a diagonal covariance matrix is already a good enough approximation).
References:
EM
: public Algorithm
¶The class implements the EM algorithm as described in the beginning of this section. It is inherited from Algorithm
.
The constructor of the class
EM::
EM
(int nclusters=EM::DEFAULT_NCLUSTERS, int covMatType=EM::COV_MAT_DIAGONAL, const TermCriteria& termCrit=TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, EM::DEFAULT_MAX_ITERS, FLT_EPSILON) )¶
cv2.
EM
([nclusters[, covMatType[, termCrit]]]) → <EM object>¶Parameters: |
|
---|
Estimates the Gaussian mixture parameters from a samples set.
bool EM::
train
(InputArray samples, OutputArray logLikelihoods=noArray(), OutputArray labels=noArray(), OutputArray probs=noArray())¶
bool EM::
trainE
(InputArray samples, InputArray means0, InputArray covs0=noArray(), InputArray weights0=noArray(), OutputArray logLikelihoods=noArray(), OutputArray labels=noArray(), OutputArray probs=noArray())¶
bool EM::
trainM
(InputArray samples, InputArray probs0, OutputArray logLikelihoods=noArray(), OutputArray labels=noArray(), OutputArray probs=noArray())¶
cv2.EM.
train
(samples[, logLikelihoods[, labels[, probs]]]) → retval, logLikelihoods, labels, probs¶
cv2.EM.
trainE
(samples, means0[, covs0[, weights0[, logLikelihoods[, labels[, probs]]]]]) → retval, logLikelihoods, labels, probs¶
cv2.EM.
trainM
(samples, probs0[, logLikelihoods[, labels[, probs]]]) → retval, logLikelihoods, labels, probs¶Parameters: |
|
---|
Three versions of training method differ in the initialization of Gaussian mixture model parameters and start step:
The methods return true
if the Gaussian mixture model was trained successfully, otherwise it returns false
.
Unlike many of the ML models, EM is an unsupervised learning algorithm and it does not take responses (class labels or function values) as input. Instead, it computes the
Maximum Likelihood Estimate of the Gaussian mixture parameters from an input sample set, stores all the parameters inside the structure:
in probs
,
in means
,
in covs[k]
,
in weights
, and optionally computes the output “class label” for each sample:
(indices of the most probable mixture component for each sample).
The trained model can be used further for prediction, just like any other classifier. The trained model is similar to the
CvNormalBayesClassifier
.
Returns a likelihood logarithm value and an index of the most probable mixture component for the given sample.
Vec2d EM::
predict
(InputArray sample, OutputArray probs=noArray()) const
¶
cv2.EM.
predict
(sample[, probs]) → retval, probs¶Parameters: |
|
---|
The method returns a two-element double
vector. Zero element is a likelihood logarithm value for the sample. First element is an index of the most probable mixture component for the given sample.
Returns true
if the Gaussian mixture model was trained.
bool EM::
isTrained
() const
¶
cv2.EM.
isTrained
() → retval¶See Algorithm::read()
and Algorithm::write()
.
See Algorithm::get()
and Algorithm::set()
. The following parameters are available:
"nclusters"
"covMatType"
"maxIters"
"epsilon"
"weights"
(read-only)"means"
(read-only)"covs"
(read-only)