Probability II: Bayesian Updating

Instructions

Answer every part below. Show your work, and include the R code and output you used to obtain any numerical answers or plots.

The Setup

A professor wants to know whether students like his class, but he does not have a good sense of the answer beforehand (a weak prior). To learn more, he surveys students who have taken his class before.

Suppose 50 individuals are randomly chosen from the large population of students who have taken the class. Each is asked a single question with two possible responses:

• “Yes, I liked it.”
• “No, I hated it.”

Let \(\theta\) denote the proportion of the population who would answer “Yes.” The professor’s prior distribution for \(\theta\) is

$$\theta \sim \text{Beta}(1.5,\ 1.5),$$

with known parameters. In the sample, 37 students responded “Yes” and 13 responded “No.”

Big question: What can we infer about \(\theta\) after observing the data?

1. Prior mean and standard deviation

Find the prior mean and prior standard deviation of \(\theta\).

Hint: If \(X \sim \text{Beta}(a, b)\), then

$$E(X) = \frac{a}{a+b} \qquad\text{and}\qquad \text{Var}(X) = \frac{ab}{(a+b)^2\,(a+b+1)}.$$

2. A prior probability

Find the prior probability that \(\theta < 0.6\).

Hint: If \(X \sim \text{Beta}(a, b)\), you can find \(\Pr(X < c)\) in R with pbeta(c, a, b).

3. The likelihood

Find the likelihood \(f(x \mid \theta)\).

Hint: With binary (Yes/No) responses, the likelihood is proportional to

$$f(x \mid \theta) \propto \theta^{\,n_{\text{yes}}}\,(1-\theta)^{\,n_{\text{no}}}.$$

4. The posterior distribution

Find the posterior distribution \(\xi(\theta \mid x)\).

Hints:

• Bayes’ rule in proportional form: $$\underbrace{\xi(\theta \mid x)}_{\text{posterior}} \;\propto\; \underbrace{f(x \mid \theta)}_{\text{likelihood}} \;\times\; \underbrace{\xi(\theta)}_{\text{prior}}$$
• The density of a \(\text{Beta}(a, b)\) distribution is proportional to \(\theta^{a-1}(1-\theta)^{b-1}\).
• So the prior density here is \(\xi(\theta) \propto \theta^{1.5-1}(1-\theta)^{1.5-1}\).

5. Plot the prior and posterior

Plot the prior density \(\xi(\theta)\) and the posterior density \(\xi(\theta \mid x)\) on the same axes.

Hint: You may use R commands such as the following, replacing a, b, c, and d with the appropriate parameter values:

theta     <- seq(0.01, 0.99, 0.01)
prior     <- dbeta(theta, a, b)
posterior <- dbeta(theta, c, d)

plot(theta, posterior, type = "l",
     xlab = expression(theta), ylab = "Density")
lines(theta, prior, lty = 2)
legend("topleft", legend = c("Posterior", "Prior"), lty = c(1, 2))

6. Interpretation

Explain what you have learned by generating the plot. How did the data change the professor’s beliefs about \(\theta\)?