Bayesian Theorem - Advantages

by Dax Bradley

Creating complex models from complex data can prove to be an arduous task, even in a system of computers.  Using even existing data, a computing system must perform many calculations, often at the expense of vast resources.  It can be difficult to express a problem mathematically, particularly using highly details models available through direct or indirect complex mechanisms.  It would be no small challenge for a computing tool to interpolate shapes and behavior from raw video footage, providing a prohibitively large amount of variability and noise (Hertzmann, 2004, p. 5).

Bayesian reasoning provides a methodology for many data-modeling problems.  By searching for logic in uncertainty, Bayesian methods provide a unified approach that on the surface is like human logic.  Bayesian logic provides three main areas of interest:

1. Principled modeling of uncertainty

2. General purpose models for unstructured data

3. Effective algorithms for data fitting and analysis under uncertainty

Courtesy: CBS

There are advantages to Bayes learning method.  One area is in interpolation.  When faced with a challenge in engineering, there is always a balancing act to determine whether the time and resources a human would consume vs. a computing system.  With the formulation of an engineering system, a world model must be built, along with a “controller” for that environment.  Bayesian methods interpolate this to the extreme because the Bayesian prior can be a delta function on one model of the world (Barak, 2005).

The Bayesian method also obeys the likelihood principle.  In a case where two proportional likelihood functions come from two distinct samples for , then all inferences about both groups will be similar (Jones & Huddleson, 2015).  It also provides interpretable answers in a convenient setting for a wide range of models.  This includes hierarchical models as well as missing-data problems.  Less training data is also required; in fact, lower training data is preferred with the Bayes naïve approach (Chen,  2011, para. 3).  A naïve Bayes classifier will converge more efficiently than discriminative models such as logistic regression.

References

Barak, B. (2005). Advantages and disadvantages of Bayesian learning. Retrieved January 17, 2017, from http://hunch.net/?p=65

Chen, E. ( 2011, April 27). Choosing a machine learning classifier [Blog post]. Retrieved from http://blog.echen.me/2011/04/27/choosing-a-machine-learning-classifier

Hertzmann, A. (2004). Introduction to Bayesian learning [Lecture notes]. Retrieved from https://www.dgp.toronto.edu/~hertzman/ibl2004/notes.pdf

Jones, A., & Huddleson, E. (2015). Bayesian analysis: Advantages and disadvantages. Retrieved January 17, 2017, from https://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_introbayes_sect006.htm