Bayesian Workflow

Prof. Sam Berchuck

Feb 04, 2025

Review of last lecture

On Thursday, we learned about various ways compare models.

AIC, DIC, WAIC
LOO-CV/LOO-IC

Today, we will put these concepts within the larger framework of the Bayesian workflow.

Bayes theorem

\[f(\boldsymbol{\theta} | \mathbf{Y}) = \frac{f(\mathbf{Y} | \boldsymbol{\theta})f(\boldsymbol{\theta})}{f(\mathbf{Y})}\]

Rethinking Bayes theorem:

\[f(\boldsymbol{\theta} | \mathbf{Y}) \propto f(\mathbf{Y}, \boldsymbol{\theta}) = f(\mathbf{Y} | \boldsymbol{\theta})f(\boldsymbol{\theta}) \]

In Stan:

\[\log f(\mathbf{Y} | \boldsymbol{\theta}) + \log f(\boldsymbol{\theta})\]

Bayesian workflow

Gelman A., Vehtari A., Simpson D., Margossian, C., Carpenter, B. and Yao, Y., Kennedy, L., Gabry, J., Bürkner P. C., & Modrák M. (2020). Bayesian Workflow.

Bayesian workflow

workflow0 — Taken from Bayesian workflow by Francesca Capel

Today we will talk about a general strategy for taking a question and data to a robust conclusion.

A simplified workflow

Setting up a full probability model: a joint probability distribution for all observable and unobservable quantities in a problem. The model should be consistent with knowledge about the underlying scientific problem and the data collection process.
Conditioning on observed data: calculating and interpreting the appropriate posterior distribution — the conditional probability distribution of the unobserved quantities of ultimate interest, given the observed data.
Evaluating the fit of the model and the implications of the resulting posterior distribution: how well does the model fit the data, are the substantive conclusions reasonable, and how sensitive are the results to the modeling assumptions in step 1? In response, one can alter or expand the model and repeat the three steps.

From BDA3.

Bayesian workflow

Research question: What are your dependent and indepednent variables? What associations are you interested in? EDA.

Specify likelihood & priors: Use knowledge of the problem to construct a generative model.

Check the model with simulated data: Generate data from the model and evaluate fit as a sanity check (prior predictive checks).

Fit the model to real data: Estimate parameters using MCMC.

Bayesian workflow

Check diagnostics: Use MCMC diagnostics to guarentee that the algorithm converged.

Examine posterior fit: Create posterior summaries that are relevant to the research question.

Check predictions: Examing posterior predictive checks.

Compare models: Iterate on model design and choose a model.

Motivating example: predicting weight from height

Research question: We would like to understand the relationship between a person’s height and weight. A few particular questions we have are:

How much does a person’s weight increase when their height increases?
How certain we can be about the magnitude of the increase?
Can we predict a person’s weight based on their height?

Data: We will use the bdims dataset from the openintro package. This dataset contains body girth measurements and skeletal diameter measurements, as well as age, weight, height and gender.

Prepare data

library(openintro)
dat <- data.frame(weight = bdims$wgt * 2.20462, # convert weight to lbs
                  height = bdims$hgt * 0.393701, # convert height to inches
                  sex = ifelse(bdims$sex == 1, "Male", "Female"))
head(dat)

    weight   height  sex
1 144.6231 68.50397 Male
2 158.2917 69.01579 Male
3 177.9128 76.18114 Male
4 160.0554 73.42524 Male
5 173.7241 73.70083 Male
6 164.9056 71.45673 Male

1. Research question:

1. Research question:

1. Research question:

2. Specify likelihood & priors:

Construct a data generating process.
We would like to model weight as a function of height using a linear regression model.
Define, \(Y_i\) as the weight of observation \(i\) and \(\mathbf{x}_i\) as a vector of covariates (here only height).

\[Y_i = \alpha + \mathbf{x}_i\boldsymbol{\beta} + \epsilon_i,\quad \epsilon_i \sim N(0,\sigma^2)\]

data {
  int<lower = 1> n; // number of observations
  int<lower = 1> p; // number of covariates (excluding intercept)
  vector[n] Y;      // outcome variable
  matrix[n, p] X;   // covariate matrix
}

2. Specify likelihood & priors:

Construct a data generating process.
We would like to model weight as a function of height using a linear regression model.

\[Y_i = \alpha + \mathbf{x}_i\boldsymbol{\beta} + \epsilon_i,\quad \epsilon_i \sim N(0,\sigma^2)\]

parameter {
  real alpha;            // intercept on the original scale
  vector[p] beta;             // regression parameters
  real<lower = 0> sigma; // measurement error
}

2. Specify likelihood & priors:

Construct a data generating process.
We would like to model weight as a function of height using a linear regression model.

\[Y_i = \alpha + \mathbf{x}_i\boldsymbol{\beta} + \epsilon_i,\quad \epsilon_i \sim N(0,\sigma^2)\]

model {
  target += normal_lpdf(Y | alpha + X * beta, sigma);
}