How the BUGS software works
The BUGS software is built round two ideas
1) Graphical models
2) Monte Carlo simulation
Bayesian models
Bayesian models are specified by giving a joint probability
distribution for the data and the parameters in the model.
This joint probability distribution is a product of simple
factors.
Each factor can be represented by a node in a directed
acyclic graph (DAG)
Each factor is a function of the variable at the node representing
that factor and the node representing its parents.
Graphical models as picture
Graphical models as language
model
{
for( i in 1 : N ) {
r[i] ~ dbin(p[i], n[i])
b[i] ~ dnorm(0.0,tau)
logit(p[i]) <- alpha0 + alpha1 * x1[i] + alpha2 * x2[i] +
alpha12 * x1[i] * x2[i] + b[i]
}
alpha0 ~ dnorm(0.0,1.0E-6)
alpha1 ~ dnorm(0.0,1.0E-6)
alpha2 ~ dnorm(0.0,1.0E-6)
alpha12 ~ dnorm(0.0,1.0E-6)
tau ~ dgamma(0.001,0.001)
sigma <- 1 / sqrt(tau)
}
The joint probability distribution for
this model
P(r, b, a 0 , a 1 , a 12 , a 2 , t ) =
N( a 0 |0, 1.0E-6) N( a 1 |0, 1.0E-6)
N( a 12 |0, 1.0E-6) N( a 2 |0, 1.0E-6) Gamma( t |0.001, 0.001)
P Bin(r i |p i , n i ) P N(b i |0, t )
where logit(p i ) <- a 0 + a 1 x 1 + a 2 x 2 + a 12 x 1 x 2 + b i
Conditional distributions
The Monte Carlo simulation needs to be able to calculate
the conditional distribution of each parameter in the model
given the data and all the other parameters.
This is just all the factors in the joint probability distribution
that depend on the parameter of interest.
For example the condition distribution of a 0 is
N( a 0 |0, 1.0E-6) P Bin(r i |p i , n i )
The condition distribution of b i is
Bin(r i |p i , n i )N(b i |0, t )
The condition distribution of t is
Gamma( t |0.001, 0.001) P N(b i |0, t )
Conditional distributions cont.
Conditional distributions have a very simple interpretation
in terms of the graphical model.
Just multiply the factor associated with the node by the
factor associated with each of its children.
Building the graphical model
When we specify a Bayesian model in the BUGS language
we actually just specify what the parents of each node are
and the functional form of the factor at each node.
We do not know who the children of each node are.
But this is easy to work out.
We work out the children of each node and store them in a list
associated with the node. We call this list the "likelihood" list.
Using these "likelihood" list we can easily calculate the
conditional distribution needed for the Monte Carlo sampling.
Monte Carlo sampling
There are many possible Monte Carlo sampling algorithms.
Not all sampling algorithms are suitable for a particular
problem.
Some algorithms might work very badly or even be invalid.
Given the expression for a conditional distribution BUGS works
out which sampling algorithms are valid for it and which
sampling algorithm should behave well.