0. Intro. to Bayesian 1. Bayesian VS. Frequentist 2. Metropolis Algorithm Example 3.Gibbs Sampler Example

4. Bayesian Proc in SAS 5. Intuitive example for Beta distri. 6. Trick to compare two baseball players

What is Frequentist statistics? To put it simple, it's just the traditional classic Statistics we learnt at School. Then what is Bayesian Statistics? As we mentioned previously, Bayesian Statistics comes with its root in Bayes' theorem

Some related terms like

Instead of considering only the ML estimate for p, it would treat p as a random variable with its own distribution of possible values. The distribution can be defined by the existing evidence. The logic goes as follows. What is the probability of a given value of p, given the data? We actually answer the question in a different way, what is the probability of p>0.5?

Credits from Dr. Ipeirotis.

Now we know the distribution of the parameter p: it's actually Beta(p;a+10,b+4). By assuming initial values for a,b(prior information), then we can calculate the expectation of p, the standard error of this parameter, the confidence interval about p by using the classic theory, also we can calculate prob(p>0.5). In other words, we can do a lot more stuff in this Bayesian approach.

The key issue for Bayesian approach is how to choose those values a,b, or even how to choose the best distribution for p(we did assume the beta distribution, we can also choose other distribution). This invovles a lot more deep techniques, where Bayesian updating, Gibbs sampling, Rejection Sampling, Markov chain Monte Carlo (MCMC) Simulation comes into play. Here are highly-recommended books for further reading.