Bounds for the loss in probability of correct classification under model based approximation
2006 (English)In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 7, 2449-2480 p.Article in journal (Refereed) Published
In many pattern recognition/classification problem the true class conditional model and class probabilities are approximated for reasons of reducing complexity and/or of statistical estimation. The approximated classifier is expected to have worse performance, here measured by the probability of correct classification. We present an analysis valid in general, and easily computable formulas for estimating the degradation in probability of correct classification when compared to the optimal classifier. An example of an approximation is the Naive Bayes classifier. We show that the performance of the Naive Bayes depends on the degree of functional dependence between the features and labels. We provide a sufficient condition for zero loss of performance, too.
Place, publisher, year, edition, pages
2006. Vol. 7, 2449-2480 p.
Bayesian networks, naive Bayes, plug-in classifier, Kolmogorov distance, of variation, variational learning, bayesian belief networks, discrete probability, distributions, discrimination, divergence, dependence, variables
IdentifiersURN: urn:nbn:se:kth:diva-16270ISI: 000245390700007OAI: oai:DiVA.org:kth-16270DiVA: diva2:334312
QC 201005252010-08-052010-08-05Bibliographically approved