# tSNE(t-distributed stochastic neighbor embedding)

## 符号定义
|符号|含义|
|:--:|:--:|
|$\pmb{x}$|数据点|
|X|数据点集合|
|N|数据点总数|
|$\pmb{y}$|降维后数据点|
|$p_{ij}$|原空间中数据点$\pmb{x_j}$与数据点$\pmb{x_i}$的联合概率|
|$q_{ij}$|低维空间中数据点$\pmb{y_j}$与数据点$\pmb{y_i}$的联合概率|
|$\mathcal{L}$|损失函数|
|$d$|原空间维度|
|$m$|降维后维度|
|$W$|权重矩阵|
|$D$|度矩阵|

## 概念

对于可视化来说，SNE有一个很大的问题(Crowding Problem)：不同类的数据聚集在一起，边界不清晰。若不手动标注类别信息很难区分不同的类。对于降维来说，这个问题始终存在，相较于低维空间，在高维空间中有更大的空间可供距离相同的点去分布。降维到低维后，可容纳这些点的空间不够，并且对于不同的距离，空间减小的程度不同，最终导致Crowding Problem。相较于距离较近的点，可容纳中等距离的点的空间减少的更多。反映到降维上就是：非常相近的数据聚集在一起，这没有问题；但是相距更远一些的数据会有相互远离的趋势。在SNE的优化过程中则会设法对这些相互远离的数据点施加“吸引力”使得这些点聚集到一起，最终导致不同类数据的分界线不明显。

上述问题的一个改进方式是人为添加“斥力”，这正是UNI-SNE的做法，但是并没有根本上解决Crowding Problem。

t-SNE在SNE的基础上有如下改进：
1. 使用对称SNE代替原有的SNE
2. 在低维空间使用t-分布代替高斯分布计算概率

其中第一点并不能解决上述的Crowding Problem，主要是使用联合概率$p_{ij}$以及$q_{ij}$代替了原有的条件概率$p_{j|i}$以及$q_{j|i}$。这样操作后，一方面优化更为简洁，另一方面相较于SNE也有一定的提升。

第二点才是t-SNE的主要工作。t-分布相较于高斯分布“更平”，即数据点落在离期望更远的位置的概率更大。

对于SNE系列方法，在高维空间计算联合概率实际上是在将距离转化为概率，在低维空间则是将概率转化为距离。当高维空间和低维空间使用相同的方法计算概率时，相当于距离保持，即保持高维空间内数据点之间的距离分布和低维空间内数据点之间的距离分布一致。t-SNE在低维空间中用t-分布代替了高斯分布，显然，对于高维空间中距离较远的两个点，其对应的利用高斯函数计算得到的联合概率也比较小，反映到t-分布中，同一概率则会对应到距离相对更远的两个点，从而实现高维空间中相距较远的两个点在降维后不至于相距过近。

## 推导

* **对称SNE**

SNE中使用的是条件概率，对于条件概率，$p_{i|j}$和$p_{j|i}$不相等，t-SNE中使用的是联合概率，其计算方式如下

对于高维空间有
$$
\begin{equation}
    p_{ij} = \frac{\exp(-||\pmb{x_i}-\pmb{x_j}||^2/2\sigma^2)}{\sum_{k\neq l}\exp(-||\pmb{x_k}-\pmb{x_l}||^2/2\sigma^2)}
\end{equation}
$$

对于低维空间有
$$
\begin{equation}
    q_{ij} = \frac{\exp(-||\pmb{y_i}-\pmb{y_j}||^2)}{\sum_{k\neq l}\exp(-||\pmb{y_k}-\pmb{y_l}||^2)}
\end{equation}
$$

对于高维空间中的点来说，式-1并不是一个非常合适的选择。原始数据分布可能相当分散，对于离群点，其与其他点的距离均较大，使得与该点相关的概率非常小，不利于对该点的降维结果进行监督。在t-SNE中，对于高维空间使用下式作为概率计算的方式
$$
\begin{equation}
    p_{ij} = \frac{p_{i|j} + p_{j|i}}{2N}
\end{equation}
$$

当将概率计算方式替换为式-2和式-3后，会得到一个更为简洁的梯度计算表达。
$$
\begin{equation}
    \frac{\partial{\mathcal{L}}}{\partial{\pmb{y_i}}} = 4\sum_{j=1}^N(p_{ij}-q_{ij})(\pmb{y_i}-\pmb{y_j})
\end{equation}
$$

* **t-分布计算联合概率**

式-2仍是使用高斯函数计算低维空间内的联合概率，t-SNE中将这一计算方式改为

$$
\begin{equation}
    q_{ij} = \frac{(1+||\pmb{y_i}-\pmb{y_j}||^2)^{-1}}{\sum_{k\neq l}(1+||\pmb{y_k}-\pmb{y_l}||^2)^{-1}}
\end{equation}
$$

* **损失函数与优化**

在式-3以及式-5的基础上可以得到对应的梯度计算方式

$$
\begin{equation}
    \frac{\partial{\mathcal{L}}}{\partial{\pmb{y_i}}} = 4\sum_{j=1}^N(p_{ij}-q_{ij})(\pmb{y_i}-\pmb{y_j})(1+||\pmb{y_i}-\pmb{y_j}||^2)^{-1}
\end{equation}
$$

t-SNE原论文中对SNE、UNI-SNE以及t-SNE不同距离下梯度计算结果的对比能很好的说明这三个算法的区别

图中正值表示两个点相互吸引（降维后两点距离有减小的趋势），负值表示两个点相互排斥（降维后两点距离有增大的趋势）

主要是分析两种极端情况：
1. 原空间中两点相距较远，而降维后，尚未优化前两点相距较近
2. 原空间中两点相距较近，而降维后，尚未优化前两点相距较远

首先分析SNE

从子图a的左侧区域可以判断SNE能很好的处理第二种情况，当发生第二种情况时，SNE会迅速减小两点的间的距离以匹配原空间中的距离；但是SNE无法处理第一种情况，当原空间中两点相距较远，而降维后两点距离较近时，SNE没有足够的“修正能力”（梯度太小）来修正这种错误

然后分析UNI-SNE

不同于SNE，UNI-SNE在整个范围内添加了一个基础“排斥力”。同样的，从子图b的左侧区域可以非常清晰的判断SNE能很好的处理第二种情况。但是对于第一种情况，UNI-SNE同样没能有效解决。并且可以注意到子图b的右上角区域，即原空间和降维空间距离均较大的区域梯度为负，这会导致该区域$q_{ij}$总是要大于$p_{ij}$

最后分析t-SNE

从子图c就能看出t-SNE对于上述两个问题有更好的表现。对于原空间相距较远，而降维后相距较近的两个点，t-SNE会促使这两个点相互远离（子图c底部靠近横轴区域）；对于原空间相距较近，而降维后相距较远的两个点，t-SNE会促使这两个点相互靠近（子图c左侧靠近纵轴区域）；而对于原空间以及降维空间距离相近的区域（子图c左下角以及右上角区域），t-SNE会尽量保持不变（梯度为0）

* **训练优化技巧**

t-SNE原论文给出了一些训练优化技巧，可以总结为如下几点

1. 带动量的梯度下降法
$$
\begin{equation}
    \pmb{y_i}(t+1) = \pmb{y_i}(t) - \eta\frac{\partial{\mathcal{L}}}{\partial{\pmb{y_i}}} + \alpha(t)(\pmb{y_i}(t) - \pmb{y_i}(t-1))
\end{equation}
$$
2. 学习率衰减。t-SNE中借鉴的Increased rates of convergence through learning rate adaptation
3. early exaggeration。在训练早期，将所有$p_{ij}$扩大一定倍数，这样操作后，会促使$q_{ij}$尽可能大，即使得训练早期所有相同类别的样本点尽可能聚集，不同类别的样本点则相互远离，以便于样本点以簇的形式迭代更新，并有利于形成比较好的全局结构

t-SNE还给出了一个训练流程示例：
1. 总迭代次数设定为1000
2. early exaggeration：设定early exaggeration=4训练50轮
3. 动量设置：前250轮设定动量为0.5，后续训练设定动量为0.8
4. 学习率设置：初始学习率为100，然后依照上述的学习率衰减策略进行学习率衰减。

## 算法步骤
1. 定义数据集$X$，降维后维度m
2. 确定参数$\sigma$
3. 在低维空间中随机生成降维结果
4. 利用下式计算降维前的联合概率
$$
p_{j|i} = \frac{\exp(-||\pmb{x_i}-\pmb{x_j}||^2/2\sigma^2)}{\sum_{k\neq i}\exp(-||\pmb{x_i}-\pmb{x_k}||^2/2\sigma^2)}
$$
$$
p_{ij} = \frac{p_{j|i} + p_{i|j}}{2N}
$$
5. 利用下式计算降维后的联合概率
$$
q_{ij} = \frac{(1+||\pmb{y_i}-\pmb{y_j}||^2)^{-1}}{\sum_{k\neq l}(1+||\pmb{y_k}-\pmb{y_l}||^2)^{-1}}
$$
6. 利用下式计算梯度
$$
\frac{\partial{\mathcal{L}}}{\partial{\pmb{y_i}}} = 4\sum_{j=1}^N(p_{ij}-q_{ij})(\pmb{y_i}-\pmb{y_j})(1+||\pmb{y_i}-\pmb{y_j}||^2)^{-1}
$$
7. 利用带动量的梯度下降法更新降维结果
$$
\pmb{y_i}(t+1) = \pmb{y_i}(t) - \eta\frac{\partial{\mathcal{L}}}{\partial{\pmb{y_i}}} + \alpha(t)(\pmb{y_i}(t) - \pmb{y_i}(t-1))
$$
8. 重复6、7步直至达到迭代停止条件

## 参考资料

https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html

L. Van der Maaten, G. Hinton. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(11).

```python
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.ticker import NullFormatter

import torch
from torchvision import transforms, datasets

from sklearn.manifold import _utils, TSNE
from scipy.stats import entropy
from sklearn.metrics.pairwise import pairwise_distances
```

```python
class MytSNE(object):

    def __init__(self, n_components, perplexity, random_state, learning_rate, n_iter):

        self.n_components = n_components
        self.perplexity = perplexity
        self.random_state = random_state
        self.learning_rate = learning_rate
        self.n_iter = n_iter

        self.condition_p = None
        self.condition_q = None

    def fit_transform(self, input_data, reduction_mat_init=None):
        self.input_data = np.array(input_data)
        n_samples, sample_dims = self.input_data.shape

        # compute condition p
        self._compute_condition_p(self.input_data)

        # create reduction result
        if reduction_mat_init is not None:
            reduction_mat = reduction_mat_init.copy()
        else:
            np.random.seed(self.random_state)
            reduction_mat = 1e-4 * np.random.randn(n_samples, self.n_components).astype(np.float32)

        # part 1
        # momentum: 0.5
        # early exaggeration：4
        # iter：250
        print("learning schedule part 1 begin...")
        self.condition_p *= 12.
        reduction_mat = self._optimize(reduction_mat, 0.5, 250, self.learning_rate, n_samples=n_samples)
        print("learning schedule part 1 done...")

        # part 2
        # momentum: 0.8
        # early exaggeration：1
        # iter：max_iter - 250
        print("learning schedule part 2 begin...")
        self.condition_p /= 12.
        reduction_mat = self._optimize(reduction_mat, 0.8, self.n_iter, self.learning_rate, n_samples=n_samples)
        print("learning schedule part 2 done...")

        return reduction_mat

    def _compute_condition_p(self, input_data):
        distance_vector = pairwise_distances(input_data, squared=True).astype(np.float32, copy=False)
        self.condition_p = _utils._binary_search_perplexity(distance_vector, self.perplexity, False)
        self.condition_p = (self.condition_p + self.condition_p.T)/(2 * np.sum(self.condition_p))

    def _optimize(self, params, momentum, max_iter, learning_rate, n_samples):

        temp_params = params.copy()
        temp_update_mat = np.zeros_like(params)
        gains = np.ones_like(params)

        for i in range(max_iter):

            train_loss, grad_mat = self.kl_loss(temp_params, n_samples)

            inc = temp_update_mat * grad_mat
            gains[np.argwhere(inc < 0)] += 0.2
            gains[np.argwhere(inc >= 0)] *= 0.8
            np.clip(gains, 0.01, np.inf, out=gains)
            grad_mat *= gains

            temp_update_mat = - learning_rate * grad_mat + momentum * temp_update_mat

            temp_params += temp_update_mat

        return temp_params

    def kl_loss(self, input_data, n_samples):
        distance_mat = pairwise_distances(input_data, squared=True).astype(np.float32, copy=False)
        distance_mat += 1.
        distance_mat = np.power(distance_mat, -1)

        self.condition_q = distance_mat / (np.sum(distance_mat) - np.sum(np.diag(distance_mat)))

        _loss = np.sum(entropy(self.condition_p, self.condition_q))

        grad_mat = np.zeros((n_samples, self.n_components), dtype=input_data.dtype)

        PQd = (self.condition_p - self.condition_q) * distance_mat
        for i in range(n_samples):
            grad_mat[i] = np.matmul(PQd[i].reshape(1, -1), input_data[i] - input_data).reshape(-1)

        grad_mat *= 4.
        return _loss, grad_mat
```

```python
# ------------------------------- data -------------------------------------------
transform_ = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5), (0.5))])
data_total = datasets.MNIST('../data/mnist', train=True, transform=transform_, download=True)

# using 0~4
data_index = torch.where(data_total.targets < 5)
data_total.targets = data_total.targets[data_index][:1000]
data_total.data = data_total.data[data_index][:1000]

# init
np.random.seed(0)
reduction_init = 1e-4 * np.random.randn(data_total.data.numpy().shape[0], 2)

# ---------------------------- sklearn TSNE ---------------------------
sklearn_tsne = TSNE(n_components=2, random_state=0, perplexity=50, learning_rate=100.0, n_iter=1000, method="exact", init=reduction_init)
sklearn_tsne_result = sklearn_tsne.fit_transform(data_total.data.numpy().reshape(-1, 28*28))

# ---------------------------- My TSNE ---------------------------
my_tsne = MytSNE(n_components=2, random_state=0, perplexity=50, learning_rate=100.0, n_iter=1000)
my_tsne_result = my_tsne.fit_transform(data_total.data.numpy().reshape(-1, 28*28), reduction_mat_init=reduction_init)

# ---------------------- draw --------------------------
fig = plt.figure(figsize=(20, 10))

ax = fig.add_subplot(121)
plt.title("Projection of MNIST using Sklearn t-SNE", fontsize=15)
for i in np.unique(data_total.targets.numpy()):
    point_index_list = np.argwhere(data_total.targets == i)
    ax.scatter(sklearn_tsne_result[point_index_list, 0], sklearn_tsne_result[point_index_list, 1], cmap=plt.cm.Spectral, label=i)
ax.xaxis.set_major_formatter(NullFormatter())
ax.yaxis.set_major_formatter(NullFormatter())
ax.axis("tight")
plt.legend()

ax = fig.add_subplot(122)
plt.title("Projection of MNIST using My t-SNE", fontsize=15)
for i in np.unique(data_total.targets.numpy()):
    point_index_list = np.argwhere(data_total.targets == i)
    ax.scatter(my_tsne_result[point_index_list, 0], my_tsne_result[point_index_list, 1], cmap=plt.cm.Spectral, label=i)
ax.xaxis.set_major_formatter(NullFormatter())
ax.yaxis.set_major_formatter(NullFormatter())
ax.axis("tight")
plt.legend()
plt.show()
```


```python
from flip_lizards import BinomialModel
from scipy.stats import beta, binom
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = [9.5, 6]
```

This notebook is meant to complement Section 2.4 on Bayesian statistics from Luke Harmon's
[book on comparative methods](https://lukejharmon.github.io/pcm/chapters/).
My goal is to more clearly delineate Bayesian parameter estimation and model choice.
I assume you've read Chapter 2 from the book, so I gloss over some things.

We will use the same example dataset from Chapter 2 of the book, where we have flipped a lizard $N = 100$ times, and it landed heads-up $H = 63$ of those times.
As in the book, we are interested in 2 models:

$M_1$
:   The lizard is fair; i.e., the probability of the lizard landing heads-up is 0.5 ($p_H = 0.5$).

$M_2$
:   The probability of the lizard landing heads-up is an unkown parameter that can be any value between 0 and 1.

# Bayes' theorem

Bayes' theorem is a basic rule about conditional probabilities.
Briefly, if we are interested in two events, $A$ and $B$ (they can be anything), the probability that both $A$ and $B$ are true is

\begin{equation}
\begin{split}
    P(A \cap B) & = P(A | B) P(B) \\
    & = P(B | A) P(A)
\end{split}
\end{equation}

This is true regardless of whether or not $A$ and $B$ are independent.
If we want to be able to learn about $A$, conditional on $B$ being true, all we need to do is a little rearranging of these terms to get Bayes' theorem:

\begin{equation}
\begin{split}
    & P(A | B) P(B) = P(B | A) P(A) \\
    & P(A | B) = \frac{P(B | A) P(A)}{P(B)}
\end{split}
\end{equation}

$P(A | B)$ is called the posterior probability of $A$ given $B$.
It's called "posterior," because it's the probability *after* we have observed $B$.
The probability of $A$ *before* we observed $B$ is the prior probability of $A$, and is written as $P(A)$ in Bayes' theorem above.
The basic idea of is that we have prior beliefs about $A$, and we want to use our observation about $B$ (i.e., data!) to update our beliefs.

Below, we will use Bayes' theorem to formulate the posterior distribution of $p_H$ under each model.
This allows us to estimate things about $p_H$ and make probability statements about certain values of $p_H$.
We will also use Bayes' rule to formulate the posterior probabilities of both models.

Because we can use Bayes' rule for both parameter estimatoin and model comparison,
it is easy to conflate the two, and think of them as one and the same.
However, there are important differences between Bayesian parameter estimation
and model comparison.
My goal is for you to appreciate those differences.

# Bayesian statistics

## Bayesian parameter estimation

Let's use Bayes' theorem to look at the probability distribution for $p_H$ under our second model ($M_2$) that the probability of our lizard landing heads up can range from 0 to 1.

\begin{equation}
    P(p_H | D) = \frac{P(D | p_H) P(p_H)}{P(D)}
\end{equation}

This tells us that the posterior probability density of any value of $p_H$, given the data $D$, is equal to the likelihood of that value---$P(D | p_H) = \mathcal{L}(p_H | D)$---times the prior probability density of that value---$P(p_H)$---divided by the marginal (average) probability of our data under the model (also called the marginal likelihood of the model)---$p(D)$.
More conceptually, we have prior beliefs about $p_H$, and we want to use the results of our lizard flipping to update those beliefs.

The marginal probability of the data (the denominator of Bayes' rule) is equal the likelihood averaged over all the parameters in the model; in the case of $M_2$ we just have a single parameter ($p_H$) to average over:

\begin{equation}
    P(D) = \int_{0}^{1} P(D | p_H) P(p_H) dp_H
\end{equation}

Note that this is a weighted average, weighted by the prior probability density of $p_H$.

Let's assume a uniform distribution for the prior on $p_H$; i.e., we assume all values of $p_H$ have equal probability density before we see our data.
Now, we can calculate and plot the prior and posterior distribution of $p_H$, along with the (scaled) likelihood of $p_H$:

```python
# Our lizard-flipping data
n = 100
h = 63
# Get equally spaced x values for plotting
x = np.linspace(0, 1, 100)
# We are assuming a uniform pror on pH, which is equivalent to a beta(a=1, b=1)
a = 1
b = 1
prior_density = beta.pdf(x = x, a = a, b = b)
# The posterior distribution and (scaled) likelihood function are both beta
# distributions (don't worry about why this is true)
likelihood_density = beta.pdf(x = x, a = h + 1, b = n - h + 1)
posterior_density = beta.pdf(x = x, a = a + h, b = b + n - h)
# Plot the prior, likelihood, and posterior
fig, ax = plt.subplots()
ax.plot(x, prior_density, label = "prior")
ax.plot(x, posterior_density, label = "posterior")
ax.plot(x, likelihood_density, label = "likelihood", linestyle = "--")
plt.xlabel("Probability of heads")
plt.ylabel("Probability density")
plt.legend()
```

*A quick aside: Don't worry too much about how the code above works.*
*In this simple example of lizard flipping, the posterior and (scaled) likelihood can be solved with a bit of math and turn out to be beta distributions.*
*I take advantage of that above so we can visualize the prior, likelihood, and posterior densities.*
*However, with most real-world models, we cannot solve for the posterior so easily, so don't worry about **why** the posterior is a beta distribution.*

So, Bayes' rule gives us a way to make probability statements about certain values of $p_H$, conditional on our data.
Let's try a different prior distribution to see how sensitive our posterior is to our *a priori* assumptions about the value of $p_H$.
Let's try a beta(1, 5) distribution (the uniform distribution we assumed above is equivalent to a beta(1, 1) distribution):

```python
b = 5
prior_density = beta.pdf(x = x, a = a, b = b)
posterior_density = beta.pdf(x = x, a = a + h, b = b + n - h)
plt.plot(x, prior_density, label = "prior")
plt.plot(x, posterior_density, label = "posterior")
plt.plot(x, likelihood_density, label = "likelihood", linestyle = "--")
plt.xlabel("Probability of heads")
plt.ylabel("Probability density")
plt.legend()
```

As we can see, the posterior probability distribution for $p_H$ is quite robust to this large change in our prior assumption about $p_H$. It shifts a bit, but still prefers values in the neighborhood of 0.63.
This isn't too surprising.
Our beta(1, 5) distribution puts a lot of prior density toward $p_H = 0$.
Our 63 heads in 100 flips clearly demonstrates that is not true,
so our posterior is dominated by the data (likelihood).

Before we get into $M_1$, let's add some notation to our equations above, so that we know they apply to $M_2$:

\begin{equation}
    P(p_H | D, M_2) = \frac{P(D | p_H, M_2) P(p_H | M_2)}{P(D | M_2)}
\end{equation}

\begin{equation}
    P(D | M_2) = \int_{0}^{1} P(D | p_H, M_2) P(p_H | M_2) dp_H
\end{equation}

These equations are the same as the ones above, we have just made them more explicit to make it clear that they apply to $M_2$.

Now, let's turn our attention to Model 1.
Under $M_1$, the posterior probabiltiy density of $p_H$ given our data is

\begin{equation}
    P(p_H | D, M_1) = \frac{P(D | p_H, M_1) P(p_H | M_1)}{P(D | M_1)}
\end{equation}

Our prior assumption for $M_1$ is that $P(p_H | M_1) = 1$ when $p_H = 0.5$, and $0$ otherwise.
So, our posterior is only defined when $p_H = 0.5$, in which case the posterior is 1 ($P(p_H | D, M_1) = 1$).
Weird, right? The posterior is always one, no matter what the data are?
Yes, $M_1$ doesn't have any free parameters to allow it to learn from the data.

Because there are no free parameters to average over, the marginal probability of the data under $M_1$ is simply equal to the probability of the data when $p_H = 0.5$:

\begin{equation}
\begin{split}
    P(D | M_1) & = P(D | p_H, M_1) P(p_H | M_1) \\
    & = P(D | p_H = 0.5) P(p_H = 0.5) \\
    & = P(D | p_H = 0.5)
\end{split}
\end{equation}

In our case, we have $H = 63$ heads in $N = 100$ flips, and the likelihood is a binomial probability:

\begin{equation}
\begin{split}
    P(D | p_H = 0.5) & = P(H=63, N=100 | p_H = 0.5) \\
    & = \textrm{binomial}(k = 63, n = 100, p = 0.5)
\end{split}
\end{equation}

## Bayesian model comparison

Above, we used the conditional probability gymnastics of Bayes' rule to formulate the posterior probability distribution of $p_H$ under each model.
But, we want to know which model is "better."
Well, we can use Bayes' rule to get the posterior probability of each model:

\begin{equation}
    P(M_i | D) = \frac{P(D | M_i) P(M_i)}{P(D)}
\end{equation}

The probability of the data, is averaged over all possible models, weighted by their prior probability:

\begin{equation}
    P(D) = P(D | M_1)P(M_1) + P(D | M_2)P(M_2)
\end{equation}

This is very similar to what we did above (it is Bayes' rule, after all), but one difference is that we have replaced a likelihood density function (e.g., $P(D|p_H, M_2)$) with a marginal likelihood of a model (e.g., $P(D|M_2)$).
So, the mechanism by which our data change our prior beliefs is now a *marginal* (average) likelihood.
As we will see in a bit, that is an important difference.

If we assume *a priori* that the probability of the two models is equal, we get:

\begin{equation}
\begin{split}
    P(M_i | D) & = \frac{P(D | M_i) \frac{1}{2}}{P(D | M_1)\frac{1}{2} + P(D | M_2)\frac{1}{2}} \\
    & = \frac{P(D | M_i)}{P(D | M_1) + P(D | M_2)}
\end{split}
\end{equation}

So, to get the posterior probablity of each model, we need to calculate the marginal likelihood for our two models. We formulated these above, but didn't calculate them.

\begin{equation}
    P(D | M_2) = \int_{0}^{1} P(D | p_H, M_2) P(p_H | M_2) dp_H
\end{equation}

\begin{equation}
    P(D | M_1) = \textrm{binomial}(k = 63, n = 100, p = 0.5)
\end{equation}

Let's (again) assume a uniform prior distribution on $p_H$ and use a little Python code to calculate these marginal likelihoods, and then get the posterior probability of each model:

```python
p_data_given_m1 = binom.pmf(k = 63, n = 100, p = 0.5)
m2 = BinomialModel(number_of_flips = 100,
                   number_of_heads = 63,
                   prior_beta_a = 1.0,
                   prior_beta_b = 1.0)
p_data_given_m2 = m2.get_marginal_likelihood()

p_data = p_data_given_m1 + p_data_given_m2

p_m1_given_data = p_data_given_m1 / p_data
p_m2_given_data = p_data_given_m2 / p_data

print("P(D | M_1) =", p_data_given_m1)
print("P(D | M_2) =", p_data_given_m2)
print("P(D) =", p_data)
print("P(M_1 | D) =", p_m1_given_data)
print("P(M_2 | D) =", p_m2_given_data)
```

    P(D | M_1) = 0.002697927604718661
    P(D | M_2) = 0.009900990099009696
    P(D) = 0.012598917703728358
    P(M_1 | D) = 0.2141396323209788
    P(M_2 | D) = 0.7858603676790211

So, the posterior probability of Model 1 and 2 is about 0.214 and 0.786, respectively.
Did you expect larger differences in the posterior probabilities between the models, given the tiny P-values for $M_1$ and the large AIC weight for $M_2$ we saw in the book?
Why isn't there more support for Model 2?

Well, the frequentist approaches in the book are based on the maximum likelihoods of each model, whereas the Bayesian posterior probabilities are based on the *marginal* (or average) likelihoods of each model.
To get a sense of why that is an important difference, let's look at the plot of the prior, posterior, and likelihood under Model 2 again:

```python
fig
```

When we marginalize the likelihood of Model 2, we are taking the average of the likelihood (the green dotted) line across the x-axis, and weighting that average by the prior (the blue line).
Looking at the plot above, we have quite a bit of prior weight on values of $p_H$ with very small likelihood.
That is going to "hurt" the average likelihood of Model 2.

Above, we saw that the posterior probability density of $p_H$ under Model 2 (i.e., Bayesian parameter estimation) seemed quite robust to different prior assumptions about $p_H$.
Will that still be true for Bayesian model choice, which is working with *marginal* likelihoods?
Let's test this and see!

Above we tried a beta(1, 5) prior on $p_H$ and saw only a small shift in the posterior of $p_H$ under $M_2$.
Let's see how a beta(1, 5) prior affects the posterior probabilities of the models:

```python
# Changing the prior to beta(1,5) for Model 2
m2 = BinomialModel(number_of_flips = 100,
                   number_of_heads = 63,
                   prior_beta_a = 1.0,
                   prior_beta_b = 5.0)
p_data_given_m2 = m2.get_marginal_likelihood()

p_data = p_data_given_m1 + p_data_given_m2

p_m1_given_data = p_data_given_m1 / p_data
p_m2_given_data = p_data_given_m2 / p_data

print("P(D | M_1) =", p_data_given_m1)
print("P(D | M_2) =", p_data_given_m2)
print("P(D) =", p_data)
print("P(M_1 | D) =", p_m1_given_data)
print("P(M_2 | D) =", p_m2_given_data)
```

    P(D | M_1) = 0.002697927604718661
    P(D | M_2) = 0.0010487709454636315
    P(D) = 0.0037466985501822926
    P(M_1 | D) = 0.7200813112086094
    P(M_2 | D) = 0.2799186887913906

With this change in prior assumption, the posterior probability of Model 1 and 2 are now 0.72 and 0.28, respectively.
Wow! The posterior probabilities have flipped, and now we prefer $M_1$! Whaaat?!

In general, Bayesian model choice tends to be much more sensitive to differences in the priors placed on the models' parameters than Bayesian parameter estimation.
This makes sense, because the posterior probability of a model is based on its marginal likelihood, which is averaged with respect to (and weighted by) the priors on the model's parameters ($p_H$ in our example).
This prior sensitivity may seem alarming, but can actually be viewed as a positive feature of Bayesian model choice; the priors serve as a "natural" penalty for adding parameters to a model.
But, we have to be careful not to choose priors that put lots of weight in regions of parameter space that are biologically unreasonable, because this can "sink" a good model!

Just to reiterate, Bayesian model choice can be sensitive to the priors chosen for the *parameters* of the models.
When doing model comparison, it's easy to worry about the prior probabilities we use for the models themselves (we assumed they are both equally probable above), and forget about the priors on the models' parameters.
But, as we just demonstrated, the priors on the parameters can be *very* important!

### Bayes factors

Bayes factors are another common way of comparing models in Bayesian statistics.
A Bayes factor is the factor by which the posterior odds of a model has changed from its prior odds.
This is easy to see if we take the ratio of the posterior probabilities of two models; the denominator ($P(D)$) cancels, and we are left with:

\begin{equation}
\frac{P(M_1 | D)}{P(M_2 | D)} = \frac{P(D | M_1)}{P(D | M_2)} \times \frac{P(M_1)}{P(M_2)}
\end{equation}

The posterior odds is simply the prior odds multiplied by the ratio of the models' marginal likelihoods, which is the Bayes factor.
With a little rearranging, we can see that the ratio of the posterior odds to the prior odds is also equal to the Bayes factor

\begin{equation}
\textrm{Bayes factor} = \frac{P(D | M_1)}{P(D | M_2)} = \frac{\frac{P(M_1 | D)}{P(M_2 | D)}}{\frac{P(M_1)}{P(M_2)}}
\end{equation}

This gives us two ways of calculating a Bayes factor to compare 2 models: By getting their marginal likelihoods and taking the ratio, *or* getting the posterior probabilities and taking their ratio and dividing by the ratio of the prior probabilities.
They are equivalent (barring approximation error), but depending on the models, one whay might be much easier than the other.

```python

```