EZ Study

Actuarial Biology Chemistry Economics Calculators Confucius Engineer
Physics
C.S.

1. Bayesian Stats. VS. Frequentist Stats.

0. Intro. to Bayesian   1. Bayesian VS. Frequentist   2. Metropolis Algorithm Example   3.Gibbs Sampler Example
4. Bayesian Proc in SAS     5. Intuitive example for Beta distri.     6. Trick to compare two baseball players

Did you hear about the statistician who had his head in an oven and his feet in a bucket of ice? When asked how he felt, he replied, "On the average I feel just fine."
"When she told me I was average, she was just being mean!!"   Statistics Fun

Frequently Asked Question: People with some background in Statistics might got confused for the difference between Bayesian Stats and Frequentist Stats.
What is Frequentist statistics? To put it simple, it's just the traditional classic Statistics we learnt at School. Then what is Bayesian Statistics? As we mentioned previously, Bayesian Statistics comes with its root in Bayes' theorem
(alternatively Bayes' law or Bayes' rule).

Some related terms like Markov chain Monte Carlo (MCMC) Simulation, Gibbs Sampling seems to be very popular in this area. How do we get some taste of this fashion? Nobody wants to be too much outdated, Let us try to explain the difference of those two in some layman's language, by some intuitive example.

Intuitive Example: We are using the example most used in statistics: flipping a coin, say, it comes out with 10 heads, and 4 tails. Can we make a conclusion the coin is even in two sides? We will illustrate how the two techniques(Bayesian Stats and Frequentist Stats) lead to different conclusions.

Frequentist approach: assume the probability of getting head is p, by the statistics in school, we know the best(ML: maximum likelihood) estimator for p is:

Bayesian approach: p is not a value, it's a distribution
Instead of considering only the ML estimate for p, it would treat p as a random variable with its own distribution of possible values. The distribution can be defined by the existing evidence. The logic goes as follows. What is the probability of a given value of p, given the data? We actually answer the question in a different way, what is the probability of p>0.5? Credits from Dr. Ipeirotis.

Now we know the distribution of the parameter p: it's actually Beta(p;a+10,b+4). By assuming initial values for a,b(prior information), then we can calculate the expectation of p, the standard error of this parameter, the confidence interval about p by using the classic theory, also we can calculate prob(p>0.5). In other words, we can do a lot more stuff in this Bayesian approach.

The key issue for Bayesian approach is how to choose those values a,b, or even how to choose the best distribution for p(we did assume the beta distribution, we can also choose other distribution). This invovles a lot more deep techniques, where Bayesian updating, Gibbs sampling, Rejection Sampling, Markov chain Monte Carlo (MCMC) Simulation comes into play. Here are highly-recommended books for further reading.

Related links:
Back to Bayesian Statistics Tutorial home     Math Humor Home    Statistics Home
Continue to Time Series Modeling tutorial home     Homepage