Bayesian Inference

Meaning of Probability in Bayesian Inference

In the frequentist approach, probability is a limiting frequency. Thus, probabilities are always associated with random events. In Bayesian inference, on the other hand, probability is used to quantify your belief that an unobservable variable has a particular value. For example a Bayesian can ask questions such as:

  • What is the probability that heritability for milk yield is larger than 0.5?

  • What is the probability that variability in milk yield is due to more than 100 loci?

These Bayesian probabilities are not necessarily associated with a random experiment that assigns values to the variables in question.

Essential Elements of Bayesian Inference

  • Bayesian inference starts by specifying what you believe about the parameters or unknowns through prior probabilities. In whole-genome analyses, we will use a prior probability density to quantify our belief that the effect of most marker covariates is zero or close to zero and only a few covariates have effects that deviate from zero.

  • These parameters are related to the data through the model or “likelihood”, which are conditional probabilities for the data given the parameters. In whole-genome analyses, this is usually a multiple regression model with normally distributed residuals.

  • The prior and the likelihood are combined using Bayes theorem to obtain posterior probabilities, which are conditional probabilities for the parameters given the data.

  • Inferences about the parameters are based on the posterior.

Use of Bayes Theorem

  • Let $f(\boldsymbol{\theta)}$ denote the prior probability density for $\boldsymbol{\theta}$.

  • Let $f(\boldsymbol{y}|{\boldsymbol{\theta}})$ denote the likelihood

  • Then, the posterior probability of $\boldsymbol{\theta}$ is:

$$\begin{eqnarray} f({\boldsymbol \theta}|{\boldsymbol y}) & =\frac{f({\boldsymbol y}|{\boldsymbol \theta})f({\boldsymbol \theta})}{f({\boldsymbol y})}\\ & \propto f({\boldsymbol y}|{\boldsymbol \theta})f({\boldsymbol \theta}) \end{eqnarray}$$