EZ Study

Actuarial Biology Chemistry Economics Calculators Confucius Engineer

Introduction to Bayesian Analysis

0. Intro. to Bayesian   1. Bayesian VS. Frequentist   2. Metropolis Algorithm Example   3.Gibbs Sampler Example
4. Bayesian Proc in SAS     5. Intuitive example for Beta distri.     6. Trick to compare two baseball players

A Bayesian is one who, vaguely expecting a horse, and catching a glimpse of a donkey, strongly believes he has seen a mule.   Bayesian Fun

A Bayesian and a Frequentist were to be executed. The judge asked them what were their last wishes. The Bayesian replied that he would like to give the Frequentist one more lecture.

The judge granted the Bayesian's wish and then turned to the Frequentist for his last wish. The Frequentist quickly responded that he wished to hear the lecture again and again and again and again........

The term Bayesian derives from Thomas Bayes (1702-1761), who was a British mathematician and Presbyterian minister. Bayes introduced Bayes' theorem, which is the fundamental to our Bayesian study.
Fundamental Theorem to Bayesian Analysis

Example: Why we need to use Bayes Themorem

Some woman uses those costco pregancy test pack, and it shows positive! If you read those test description carefully, you can see that there are numbers like: the test is 95% accurate given you are pregnant. What she knows is that the test shows positive, but the test might also show positive if she is not pregnant, is she really pregnant? What is the probability that she is indeed pregnant?

Similarly, some person tests positive for AIDs. What is the probability that the person actually has AIDs?

Here are some good lecture slides from Prof. Bowman, and solutions to following classic book: Bayesian data analysis.
Read the following tutorial: Intuitive example to understand the fundamental difference between Bayesian Stats and Frequentist Stats.

Lecture Notes
Thanks to the valuable work by Professor Draper, here are some lecture notes from his previous graduate class. If you think those notes are very useful, you can support professor Draper by buying his book on the above link. It's highly recommended by many Bayesian experts. Also a book by Professor Lynch: Introduction to Applied Bayesian Statistics and Estimation for Social Scientists is very useful for Social-Area experts.

Lecture01: Background and Basics   Lecture02: Frequentist Modeling
Lecture03: Exchangeability and Conjugate Modeling
Lecture04: DeFinetti theorem for 1s and 0s
Lecture05: Conjugate Analysis
Lecture06: The exponential family
Lecture07: Diffuse priors
Lecture08: Gaussian Analysis
Lecture10: Rejection sampling
Lecture11: Metropolis-Hastings
Lecture12: Proof sketch
Lecture13: MCMC Sampling diagnostics
Lecture14: WinBUGS implementation
Lecture15: Hierarchical Models for Combining Information

Related links:
Continue to Intuitive example to understand Bayesian Analysis   Statistics Home
Back to Linear Regression tutorial home     Time Series Modeling     Homepage