## Exercise 1 (Normal-Normal Model)
Assume:
- Likelihood: $X_1, \ldots, X_n \sim N(\theta, \sigma^2)$
- Prior: $\theta \sim N(a, b^2)$
- $\sigma^2$ is a fixed and known quantity
Find the posterior distribution of $\theta \mid X_1, \ldots, X_n$.
## Exercise 2 (Gamma-Poisson Model)
Assume:
- Likelihood: $X_1, \ldots, X_n \sim \text{Poisson}(\lambda)$
- Prior: $\lambda \sim \text{Gamma}(\alpha, \beta)$
For this and other problems on this homework, use the following parameterization for the Gamma distribution:
$$
f(x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x ^ {\alpha - 1} e^{-\beta x}
$$
Find the posterior distribution of $\lambda \mid X_1, \ldots, X_n$.
## Exercise 3 (Using the Beta-Bernoulli Model)
Assume:
- Likelihood: $X_1, \ldots, X_n \sim \text{Bernoulli}(p)$
- Prior: $p \sim \text{Beta}(\alpha = 5, \beta = 5)$
Use the following data and the posterior mean to arrive at a Bayesian estimate of $p$. Compare this value of the prior mean.
```{r}
some_data = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1)
```
## Exercise 4 (Using the Gamma-Poisson Model)
Given:
- Likelihood: $X_1, \ldots, X_n \sim \text{Poisson}(\lambda)$
- Prior: $\lambda \sim \text{Gamma}(\alpha = 4, \beta = 2)$
Use the following data and the posterior interval to arrive at a Bayesian interval estimate of $\lambda$. Compare this interval to an interval based on the prior distribution.
```{r}
some_data = c(3,3,2,9,1,4,5,4,2,6,7,5,4,4,2,3,6,3,5,5,4,3,5,5,5)
```
## Exercise 5 (Using the Gamma-Poisson Model, Again)
Given:
- Likelihood: $X_1, \ldots, X_n \sim \text{Poisson}(\lambda)$
- Prior: $\lambda \sim \text{Gamma}(\alpha = 7.5, \beta = 1)$
Use the following data and the posterior distribution to calculate the posterior probabilities of the following hypotheses. Compare these probabilities to probabilities based only on the prior distribution.
$$
H_0: \lambda \leq 4 \quad \text{versus} \quad H_1: \lambda > 4.
$$
```{r}
some_data = c(3, 1, 1, 2, 2, 2, 1, 3, 3, 3, 3, 3, 1, 2, 3)
```
## Exercise 6 (Prior vs Data: Effect of Data)
Given:
- Likelihood: $X_1, \ldots, X_n \sim \text{Bernoulli}(p)$
- Prior: $p \sim \text{Beta}(\alpha = 2, \beta = 2)$
- Data: `data_1`, `data_2`, `data_3`
```{r}
data_1 = c(0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1)
data_2 = c(0,1,0,1,0,0,0,1,0,0,1,0,0,0,1,1,0,1,0,0,1,0,0,0,0)
data_3 = c(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,1,1,1,0,1)
```
Create graphics that show:
- The prior distribution and an estimate of $p$ based on this distribution
- The likelihood and the MLE for each dataset
- The posterior and an estimate of $p$ based on each of the datasets
## Exercise 7 (Prior vs Data: Effect of Prior)
Given:
- Likelihood: $X_1, \ldots, X_n \sim \text{Bernoulli}(p)$
- Prior 1: $p \sim \text{Beta}(\alpha = 2, \beta = 5)$
- Prior 2: $p \sim \text{Beta}(\alpha = 2, \beta = 2)$
- Prior 3: $p \sim \text{Beta}(\alpha = 5, \beta = 2)$
- Data: `some_data`
```{r}
some_data = c(0,1,0,1,0,0,0,1,0,0,1,0,0,0,1,1,0,1,0,0,1,0,0,0,0)
```
Create graphics that show:
- The prior distribution and an estimate of $p$ based on each prior
- The likelihood and the MLE given the data
- The posterior and an estimate of $p$ based on each of the priors
## Exercise 8 (Prior vs Data: Strength of Prior)
Given:
- Likelihood: $X_1, \ldots, X_n \sim \text{Bernoulli}(p)$
- Prior 1: $p \sim \text{Beta}(\alpha = 2, \beta = 2)$
- Prior 2: $p \sim \text{Beta}(\alpha = 5, \beta = 5)$
- Prior 3: $p \sim \text{Beta}(\alpha = 10, \beta = 10)$
- Data: `some_data`
```{r}
some_data = c(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,0,1,1,1,1,1)
```
Create graphics that show:
- The prior distribution and an estimate of $p$ based on each prior
- The likelihood and the MLE given the data
- The posterior and an estimate of $p$ based on each of the priors
## Exercise 9 (Bayes Risk in the Beta-Bernoulli Model)
Suppose $X_1, \ldots, X_n \sim \text{Bernoulli}(p)$ and $p \sim \text{Beta}(\alpha, \beta)$. Using squared error loss, find the Bayes estimator and the Bayes risk.
## Exercise 10 (The James-Stein Estimator)
Consider $X_1, \ldots, X_k \sim N(\theta_i, 1)$. Define $\theta = (\theta_1, \ldots, \theta_k)$. Consider the loss
$$
L\left(\theta, \hat{\theta}\right) = \sum_{j = 1}^{k}(\theta_j - \hat{\theta}_j) ^ 2.
$$
where $\hat{\theta}$ is some estimator of $\theta$.
Use simulation to compare the risk of the MLE to the James-Stein estimator. Consider at least three simulation setups:
- $k = 2$
- a relative "large" $k$ and a dense $\theta$ vector
- a relative "large" $k$ and a sparse $\theta$ vector
You are free to further specify $k$ and $\theta$ as you wish. You are also free to add additional setups. Summarize your findings.
## Exercise 11 (Free Points)
The previous two problems were pretty difficult. Draw a smiley face for a free point!