# 简单了解 SISSO 特征筛选方法在回归问题下应用

> 创建时间：2019-09-09

这份文档我们会用非常小的演示模型，简单地讨论 SISSO 的使用与输出。我们会尽可能地理解所有程序的输出文件，但对于 (我所认为的) SISSO 的最关键的两个技术问题，即特征自动构造以及高性能筛选 (以及 $L_0$ 惩罚的算子选择算法) 问题，我们不做讨论。

SISSO (Sure Independence Screening and Sparsifying Operator) 是欧阳润海提出的算法。其原始文献为 Ouyang, Ghiringhelli et al. [^Ouyang-Ghiringhelli.PRM.2018] [^Ouyang-Ghiringhelli.JPM.2019]。

针对回归问题，通过用户给定一系列数据的特征 (Feature) 和目标值 (Target or Property)，以及一些 SISSO 特化的设定，可以导出哪些特征、以及特征间的运算所导出的描述子 (Descriptor) (运算包括但不限于四则运算、指数与对数、三角函数等) 会对目标值有最大程度的贡献；从而达到筛选出能最佳地描述目标的描述子的目的。

该算法尽管是出于解决材料设计问题而产生的，但它本身完全可以看作一种纯应用数学的方法。同时，该算法的根基应当被认为是传统机器学习 (非深度学习) 的方法。

由于 SISSO 并不是 Python 程序，因此这份文档中，我们会在 Jupyter 中使用 bash 命令。为了执行这份文档，读者需要事先编译好 SISSO 程序到当前目录的 `sisso`。

```python
%matplotlib notebook

import numpy as np
from numpy import exp, sqrt, log, sin, cos
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, max_error, r2_score
from matplotlib import pyplot as plt

np.set_printoptions(precision=5, linewidth=150, suppress=True)
```

## 运行前的准备

### 数据集 (Dataset)

数据集的构造如下。该数据集的样本数量为 5；目标值在第二列，三个初步的特征列在后三列。该数据集除了名称作简化外，其余与 SISSO Github 上的 [train.dat_regression](https://github.com/rouyang2017/SISSO/blob/master/input_template/train.dat_regression) 一致。

```python
! cat train.dat
```

    materials  property  F1      F2      F3
    sample1    3.0862    0.8626  0.7043  0.6312
    sample2    2.8854    0.7260  0.7818  0.6119
    sample3    0.6907    0.4943  0.0044  0.4420
    sample4    0.9902    0.0106  0.0399  0.9877
    sample5    0.7242    0.0970  0.3199  0.5504

我们之后会经常地用到这些数据，因此会使用变量与数学记号存储与表示。其中，

- `P` $\boldsymbol{P}$ (vector length: $n_\mathrm{Sample}$) 目标值

- `F1`, `F2`, `F3` $\boldsymbol{f}_1$, $\boldsymbol{f}_2$, $\boldsymbol{f}_3$ (vector length: $n_\mathrm{Sample}$) 初始特征

这里指出，`train.dat` 中的 `F1`, `F2`, `F3` 可以是其它的字符串。同时，这些特征字符串可以通过 Python 的 `exec` statement 直接执行而不一定需要 Hard Coding；但为了程序可读且易执行，在这篇文档中我们就手动地声明这些变量了。

```python
with open("train.dat", "r") as f:
    dataset = np.array([line.split() for line in f.readlines()])
    dataset = np.array(dataset[1:, 1:], dtype=float)
P, F1, F2, F3 = dataset.T
P
```

    array([3.0862, 2.8854, 0.6907, 0.9902, 0.7242])

### SISSO 参数

SISSO 的参数也参考 Github 上的 [SISSO.in_regression](https://github.com/rouyang2017/SISSO/blob/master/input_template/SISSO.in_regression)；但我们对其中一些设置做了更改。一些重要但也许需要额外说明的参数列举如下：

`desc_dim` 作为最终筛选结果的描述子的维度限制 $n_\mathrm{Descriptor}$。在这篇文档中，我们将“特征”与“描述子”分开；特征是用户输入的样本描述性信息，而描述子是导出的描述性信息。$n_\mathrm{Descriptor}$ 会在后文使用。

```python
! grep "desc_dim=" SISSO.in
```

    desc_dim=3           ! dimension of the descriptor

`rung` 描述子所需要的构造次数。譬如，描述子

- $f_2$ (恰好也是输入特征) 是 Rung 0 级别的；
- $f_2 - f_1$, $\exp(f_1)$ 等是 Rung 1 级别的 (分别使用了减号、指数)；
- $\log(f_3) \mathrm{abs} (f_2 - f_1)$ 是 Rung 2 级别的 (其中，$\log(f_3)$ 与 $\mathrm{abs} (f_2 - f_1)$ 分别是 Rung 1 级别的，它们之间的运算多了一个乘法，因此为 Rung 2 级别的)；
- $\sqrt{\log(f_3)}$ 也是 Rung 2 级别的。

```python
! grep "rung=" SISSO.in
```

    rung=2               ! rung of the feature space to be constructed

`maxcomplexity` 从特征构造描述子时所使用的最多的符号数量。尽管我们设定了最大符号数量为 10，事实上 10 个符号是不可能达到的；因为 Rung 1 最多使用到 1 个符号，Rung 2 最多是两个 Rung 1 描述子的运算即 3 个符号；可以推知 Rung 3 最多使用 7 个符号，Rung $n$ 最多使用 $2^n - 1$ 个符号。

```python
! grep "maxcomplexity=" SISSO.in
```

    maxcomplexity=10     ! max feature complexity (number of operators in a feature)

`dimclass` 第 1, 2 号特征 $f_1$, $f_2$ 共享相同的物理单位，第 3 号特征 $f_3$ 有独立的物理单位。这会避免构造类似于 $f_1 - f_3$ 等不符合量纲的描述子。

```python
! grep "dimclass=" SISSO.in
```

    dimclass=(1:2)(3:3)  ! group features according to their dimension/unit; those not in any () are dimensionless

`subs_sis` 对于每个描述子维度，选取最好的 7 个描述子作为备选 $n_\mathrm{sub}$。后文会对其作补充说明。

```python
! grep "subs_sis=" SISSO.in
```

    subs_sis=7           ! SAME one size for every SIS-selected subspace. Otherwise, input a size for each dimension seperated by comma

## SISSO 运行与结果

当设置完毕数据集 {download}`train.dat` 与 SISSO 设定 {download}`SISSO.in` 后，就可以执行 SISSO 程序 `sisso` 了。SISSO 可以单核计算，也可以 MPI 多核计算。我们将屏幕输出定向到文件 `SISSO.log`，该文件没有实质性内容。

```python
! mpirun -np 4 ./sisso > SISSO.log
```

    [DESKTOP-8BRL880:01147] 3 more processes have sent help message help-btl-vader.txt / cma-permission-denied
    [DESKTOP-8BRL880:01147] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

但程序会输出大量其他信息。最重要的信息放在 {download}`SISSO.out` 文件中：

```python
! grep -A 10000 "Final model" SISSO.out | grep -B 10000 "===="
```

    Final model/descriptor to report
    ================================================================================
      3D descriptor (model):
    Total RMSE,MaxAE:   0.000083  0.000137
    @@@descriptor:
                          5:[((F2)^6/log(F1))]
                         12:[(exp(-F3)/abs(F1-F2))]
                         16:[((F1)^3/(F1-F2))]
           coefficients_001:    -0.2813852886E+01    0.2638598223E-01    0.4721473413E-02
              Intercept_001:     0.6547943803E+00
             RMSE,MaxAE_001:     0.8272942564E-04    0.1372716760E-03
    ================================================================================

这就意味着，从 SISSO 中导出的最佳描述子是 `@@@descriptor:` 下的三个描述子；这三行是所有输出中最为关键的信息：

$$
\begin{align}
D_1 &= f_2^6 \log(f_1)^{-1} \\
D_2 &= \exp(- f_3) |f_1 - f_2|^{-1} \\
D_3 &= f_1^3 (f_1 - f_2)^{-1} \\
\end{align}
$$

这从这三个描述子出发作 (包含截距 Interception，或者称偏置 Bias 的) 线性拟合：

$$
\tilde P = C_1 D_1 + C_2 D_2 + C_3 D_3 + b
$$

可以得到参数 $C_1, C_2, C_3$、偏置 $b$ 与误差 $\boldsymbol{\tilde{P}} - \boldsymbol{P}$ 向量的 RMSE 值与 MaxAE 值。

我们可以用 scikit-learn 来验证上述结果。

```python
D1 = ((F2)**6/log(F1))
D2 = (exp(-F3)/abs(F1-F2))
D3 = ((F1)**3/(F1-F2))
```

```python
reg = LinearRegression()
reg.fit(np.array([D1, D2, D3]).T, P)
```

    LinearRegression()

线性拟合参数 $C_1, C_2, C_3$ 与偏置 $b$ 列举为

```python
reg.coef_, reg.intercept_
```

    (array([-2.81385,  0.02639,  0.00472]), 0.6547943803493903)

RMSE 误差与 MaxAE 误差分别可以表达为

```python
P_pred = reg.predict(np.array([D1, D2, D3]).T)
sqrt(mean_squared_error(P, P_pred)), max_error(P, P_pred)
```

    (8.272942030223837e-05, 0.0001372716671202978)

如果用户不关心细节与中间输出，我们就只需要了解到这里即可。

## SISSO 第一轮描述子筛选

### 描述子表达式及其在数据点上的值

SISSO 会经历多次描述子筛选。SISSO 每次都会构建数以万计，甚至亿记的描述子空间，随后依据线性拟合判标筛选出最好的 $n_\mathrm{sub}$ 个描述子。我们不讨论 SISSO 是如何构建庞大的描述子空间的过程。**一般来说**，`desc_dim` 的数量决定了特征筛选次数 (如果从特征构造出的描述子的空间不够大，筛选次数会变少；SISSO 会对这种情形作警告)。因此，当前的 SISSO 会有三次特征筛选。筛选的结果会储存在 `feature_space` 目录下。

我们先拿第一次筛选结果讨论。第一次所选出的描述子储存在下述文件中：

```python
! cat feature_space/space_001d.name
```

    ((F1*F2)/log(F3))  corr=      0.9953
    cos((F1*F2))  corr=      0.9949
    ((F1*F2))^2  corr=      0.9949
    ((F1*F2)*(F1+F2))  corr=      0.9948
    ((F2)^6/log(F1))  corr=      0.9947
    ((F1)^2*(F2)^3)  corr=      0.9946
    ((F1*F2)*sqrt(F1))  corr=      0.9938

Python 提供了 `eval` 函数，可以直接将特征的算式读入并作运算。我们使用该功能给出特征运算后得到的描述子矩阵 `d_1D` $\mathbf{d}_\mathrm{1D}$ (matrix dim: $(n_\mathrm{sub}, n_\mathrm{sample})$)。

```python
with open("feature_space/space_001d.name", "r") as f:
    tokens_1D = [line.split()[0].replace("^", "**") for line in f.readlines()]
d_1D = np.array([eval(token) for token in tokens_1D])
```

```python
d_1D
```

    array([[-1.32034, -1.15554, -0.00266, -0.03417, -0.05197],
           [ 0.82106,  0.8432 ,  1.     ,  1.     ,  0.99952],
           [ 0.36909,  0.32215,  0.     ,  0.     ,  0.00096],
           [ 0.95194,  0.85581,  0.00108,  0.00002,  0.01294],
           [-0.82577, -0.71309, -0.     , -0.     , -0.00046],
           [ 0.25995,  0.25186,  0.     ,  0.     ,  0.00031],
           [ 0.56425,  0.48362,  0.00153,  0.00004,  0.00966]])

譬如第一行代表在所有 $n_\mathrm{sample} = 5$ 个数据点下，运算 $f_1 f_2 \log (f_3)^{-1}$ 的结果：

```python
((F1*F2)/log(F3))
```

    array([-1.32034, -1.15554, -0.00266, -0.03417, -0.05197])

在 SISSO 原文中，也会用 $\boldsymbol{S}_1$ 代表第一轮筛选得到的描述子空间。它只是一种符号表示，而不是可以用程序执行的记号。

### 相关性

我们注意到文件 {download}`feature_space/space_001d.name` 除了包含描述子的具体构造之外，还包括相关性说明。这种相关性实际上是筛选出的描述子与目标值之间的线性相关程度。譬如，对第 0 个描述子，$\boldsymbol{d}_\mathrm{1D}^0$ 即所给出的每个数据点对应的值为

```python
d_1D[0]
```

    array([-1.32034, -1.15554, -0.00266, -0.03417, -0.05197])

这也与文件 {download}`feature_space/space_001d_p001.dat` 的输出有关。该文件的首列是目标值，随后是描述子在给定数据点上的值：

```python
! cat feature_space/space_001d_p001.dat
```

        0.3086200000E+01   -0.1320335268E+01    0.8210609668E+00    0.3690917046E+00    0.9519374721E+00   -0.8257705126E+00    0.2599512875E+00    0.5642503915E+00
        0.2885400000E+01   -0.1155542560E+01    0.8432007625E+00    0.3221547755E+00    0.8558073770E+00   -0.7130919424E+00    0.2518606035E+00    0.4836159293E+00
        0.6907000000E+00   -0.2663889108E-02    0.9999976349E+00    0.4730277006E-05    0.1084632604E-02   -0.1029830186E-13    0.2081321883E-07    0.1529109520E-02
        0.9902000000E+00   -0.3417345965E-01    0.9999999106E+00    0.1788782436E-06    0.2135847000E-04   -0.8874049547E-09    0.7137241920E-08    0.4354433812E-04
        0.7242000000E+00   -0.5196747734E-01    0.9995185989E+00    0.9628795181E-03    0.1293653207E-01   -0.4593698157E-03    0.3080251578E-03    0.9664332013E-02

它实际上与目标值有比较好的线性关系：

```python
P
```

    array([3.0862, 2.8854, 0.6907, 0.9902, 0.7242])

可能光是看这两组向量并不能清楚地把握情况。它们的线性关系可能通过两种方式呈现。一种是 Perason R2 的开方，即线性拟合的拟合优度 R 量标：

```python
reg = LinearRegression()
reg.fit(d_1D[0].reshape(-1, 1), P)
np.sqrt(r2_score(P, reg.predict(d_1D[0].reshape(-1, 1))))
```

    0.9952505043650735

上述值与 {download}`feature_space/space_001d.name` 的结果相同。

另一种则是绘图呈现。我们不妨将所有 7 个描述子都用绘图呈现：

```python
fig, ax = plt.subplots(figsize=(5, 3))
for i, d in zip(range(7), d_1D):
    ax.scatter(P, d, label="Desp {:}".format(i))
    reg = LinearRegression()
    reg.fit(d.reshape(-1, 1), P)
    ax.plot(reg.predict([[-1.5], [1.5]]), [-1.5, 1.5], linestyle=":")
ax.set_xlim(0, 4)
ax.set_ylim(-1.5, 1.2)
ax.set_xlabel("Target $P$")
ax.set_ylabel("Descriptor Value $d_\mathrm{1D}$")
ax.legend(loc='lower right', bbox_to_anchor=(1, 0., 0.5, 0.5))
fig.tight_layout()
```

    <IPython.core.display.Javascript object>

### 残差与后一轮描述子筛选准备

第一轮描述子筛选已经结束了；我们需要对第二轮描述子筛选作准备。

第一轮筛选的依据是描述子与目标值 $\boldsymbol{P}$ 相关性。但我们知道，多个描述子之间一般不应当具有较强的相关性，否则这些描述子会被认为是接近线性相关的。

为了尽可能保证第二轮所选取的描述子与第一轮线性无关，第二轮筛选的依据是第一轮描述子中，对目标 $\boldsymbol{P}$ 作线性拟合得到的最低残差 `Delta_1D` $\boldsymbol{\Delta}_\mathrm{1D}$ (vector len: $n_\mathrm{sample}$) (或者说第一轮第 0 个描述子的拟合残差)：

$$
\boldsymbol{\Delta}_\mathrm{1D} = \boldsymbol{P} - \boldsymbol{\tilde{P}} (\boldsymbol{d}_\mathrm{1D}^0)
$$

```python
reg = LinearRegression()
reg.fit(d_1D[0].reshape(-1, 1), P)
```

    LinearRegression()

```python
Delta_1D = P - reg.predict(d_1D[0].reshape(-1, 1)).reshape(-1)
Delta_1D
```

    array([-0.04482,  0.05149, -0.06466,  0.17803, -0.12005])

这与文件 {download}`residual/res_001d_p001.dat` 的结果一致：

```python
! cat residual/res_001d_p001.dat
```

       -0.4481707500E-01
        0.5149135000E-01
       -0.6465814650E-01
        0.1780325505E+00
       -0.1200486783E+00

由于拟合过程中使用到了偏置 (Bias)，因此原文中出现的类似于 $\boldsymbol{\Delta}_\mathrm{1D} = \boldsymbol{P} - \boldsymbol{c}_\mathrm{1D}^{0, \mathrm{T}} \boldsymbol{d}_\mathrm{1D}^0$ 的公式表述会在实现中会有些不同。

## SISSO 第二轮描述子筛选

### 描述子的表达与相关性

第二轮描述子筛选与第一轮过程几乎是相同的；但唯一不同的是筛选描述子的判标从 $\boldsymbol{P}$ 更换到 $\boldsymbol{\Delta}_\mathrm{1D}$。因此，第二轮所选出的描述子几乎都与第一轮的第 0 号描述子某种程度上线性无关；或者说，这些描述子并不是希望能与最终目标值 $\boldsymbol{P}$ 有很好的契合，但其实是契合最终目标与第一轮最好的描述子之间的误差。但第二轮描述子之间又比较线性相关，因为它们都需要较好地线性拟合到相同的 $\boldsymbol{\Delta}_\mathrm{1D}$。

第二轮筛选得到的描述子和相关性列举如下：

```python
! cat feature_space/space_002d.name
```

    (abs(F1-F2))^-1  corr=      0.9693
    (sqrt(F3)/abs(F1-F2))  corr=      0.9612
    (sin(F3)/abs(F1-F2))  corr=      0.9540
    (exp(F3)/abs(F1-F2))  corr=      0.9534
    (exp(-F3)/abs(F1-F2))  corr=      0.9486
    (F3/abs(F1-F2))  corr=      0.9474
    (cos(F2)/abs(F1-F2))  corr=      0.9468

可以将所有第二轮描述子在所有数据点上的值储存到矩阵 `d_2D` $\mathbf{d}_\mathrm{2D}$：

```python
with open("feature_space/space_002d.name", "r") as f:
    tokens_2D = [line.split()[0].replace("^", "**") for line in f.readlines()]
d_2D = np.array([eval(token) for token in tokens_2D])
```

与第一轮描述子相同地，第二轮描述子空间会记为 $\boldsymbol{S}_2$。

对于第二轮第 0 个描述子，其相对于 $\boldsymbol{\Delta}_\mathrm{1D}$ 所拟合的相关性可以用下述程序给出：

```python
reg = LinearRegression()
reg.fit(d_2D[0].reshape(-1, 1), Delta_1D)
np.sqrt(r2_score(Delta_1D, reg.predict(d_2D[0].reshape(-1, 1))))
```

    0.9692849978551799

这与上述文件的首行信息相同。

### 残差与后一轮描述子筛选准备

从第二轮描述子筛选开始，残差的表达式会较为不同。

我们现在已经筛选出 $2 n_\mathrm{sub} = 2 \times 7 = 14$ 个描述子，我们会记两轮描述子的并空间为 $\boldsymbol{S}_1 \cup \boldsymbol{S}_2$。

如果描述子空间是 $\boldsymbol{S}_1$ 且我们只需要挑出一个描述子来线性拟合最终目标 $\boldsymbol{P}$，那么这个被挑出的最好的描述子就是第一轮中第 0 个描述子。但如果描述子空间是 $\boldsymbol{S}_1 \cup \boldsymbol{S}_2$，并且要挑出两个描述子来线性拟合最终目标 $\boldsymbol{P}$，那就不一定是第一轮第 0 个与第二轮第 0 个描述子了。这种情况下，我们有必要将所有描述子成对地考察一遍。

现在该空间有 14 个描述子；如果允许一对相同的描述子，那么总共有 $14 \times 14 = 196$ 对可能的描述子。我们需要对这 196 种所有可能的情况都作考察；最终判断哪一对描述子是最优秀的。

我们开一个 $(14, 14)$ 大小的矩阵 `rmse_2D`，以 RMSE 为判标，判断一对描述子能否比较好地拟合最终目标 $\boldsymbol{P}$。`dunion_2D` 表示两轮描述子在数据点上数值的矩阵并 $[\mathbf{d}_\mathrm{1D}, \mathbf{d}_\mathrm{2D}]$：

```python
rmse_2D = np.zeros((14, 14))
dunion_2D = np.concatenate([d_1D, d_2D])
for i in range(14):
    for j in range(14):
        reg = LinearRegression()
        reg.fit(np.array([dunion_2D[i], dunion_2D[j]]).T, P)
        P_pred = reg.predict(np.array([dunion_2D[i], dunion_2D[j]]).T)
        rmse_2D[i, j] = np.sqrt(mean_squared_error(P, P_pred))
```

我们对上述 `rmse_2D` 作热区图 (heatmap)：

```python
fig, ax = plt.subplots(figsize=(4, 3))
img = ax.imshow(np.log(rmse_2D), cmap="PuOr")
cbar = ax.figure.colorbar(img, ax=ax)
cbar.ax.set_ylabel("Logscale of RMSE", rotation=-90, va="bottom")
fig.tight_layout()
```

    <IPython.core.display.Javascript object>

从上图中，我们可以看出，大多数情况下，当一个描述子处在 $\boldsymbol{S}_1$ 空间，另一个描述子处在 $\boldsymbol{S}_2$ 时 (即上述矩阵的非对角块，棕黄色部分)，拟合的结果通常会比较好。这与描述子的筛选过程有着直接的关系。尽管绝大多数情况下，描述子要处于不同的空间中才能得到好的结果；但还可能有极少量例外。

经过简单的查找后，应当能知道当选取 $\boldsymbol{S}_1$ 的第 2 个描述子，$\boldsymbol{S}_2$ 的第 0 个描述子时，给出的线性拟合结果是最佳的。其残差 `Delta_2D` $\boldsymbol{\Delta}_\mathrm{2D}$ 写为

```python
reg = LinearRegression()
reg.fit(np.array([d_1D[2], d_2D[0]]).T, P)
Delta_2D = P - reg.predict(np.array([d_1D[2], d_2D[0]]).T)
Delta_2D
```

    array([ 0.00406, -0.00466, -0.00297,  0.00161,  0.00196])

这与文件 {download}`residual/res_002d_p001.dat` 的结果一致：

```python
! cat residual/res_002d_p001.dat
```

        0.4058658000E-02
       -0.4655793000E-02
       -0.2972292700E-02
        0.1614245300E-02
        0.1955182100E-02

残差 `Delta_2D` $\boldsymbol{\Delta}_\mathrm{2D}$ 就将会作为第三轮描述子筛选过程的指标了。

同时，上述拟合过程给出的 RMSE 值也与 {download}`models/top0091_002d` 文件一致。其中，SISSO 输出的 `Feature_ID` 一栏给出的是 1-index 的 $\boldsymbol{S}_1 \cup \boldsymbol{S}_2$ 的序号。

```python
! cat models/top0091_002d | head -n 5
```

            Rank        RMSE       MaxAE  Feature_ID
               1    0.003268    0.004656  (       3       8)
               2    0.004235    0.005166  (       2       9)
               3    0.005273    0.006898  (       5       8)
               4    0.005682    0.008797  (       2       8)

事实上，第三轮描述子筛选的过程与第二轮过程是非常类似的。我们之后就不再对第三轮进行说明。

## 最终模型与备选模型

作为用户，尽管中间过程与中间输出非常重要，也确实地能加深对 SISSO 算法的了解；但我们所最关心的是最终的结果。我们前面已经提及了从输出文件 {download}`SISSO.out` 查看最佳模型；但 SISSO 的输出中，还包含众多拟合效果也尚可的各种备选模型。

由于我们在设置中选择了 `desc_dim` 为 3，即最终描述子数量为 3；因此在查看模型时，我们也要找到 `model` 文件夹下末尾为 `003d` 为名的文件。我们打出前 6 行：

```python
! cat models/top0100_003d | head -n 6
```

            Rank        RMSE       MaxAE  Feature_ID
               1    0.000083    0.000137  (       5      12      16)
               2    0.000097    0.000147  (       4       5      11)
               3    0.000100    0.000165  (       5      12      20)
               4    0.000107    0.000178  (       6      12      17)
               5    0.000109    0.000182  (       6      12      18)

上述的排名是通过 RMSE 为量标给出的。我们会发现，排名第一的模型就是 {download}`SISSO.out` 所给出的模型 (回见 [上文](#SISSO-运行与结果))。线性拟合的参数在下述文件：

```python
! cat models/top0100_003d_coeff | head -n 6
```

    Model_ID, [(c_i,i=0,n)_j,j=1,ntask]
               1    0.6547943803E+00   -0.2813852886E+01    0.2638598223E-01    0.4721473413E-02
               2    0.6781322860E+00    0.1483762489E+01   -0.1156767698E+01    0.3405072802E-02
               3    0.6563705862E+00   -0.2822110944E+01    0.2627937583E-01    0.3542080508E-02
               4    0.6562623115E+00    0.8694024465E+01    0.2629715431E-01    0.2124739537E-01
               5    0.6560493168E+00    0.8311600413E+01    0.2628677203E-01    0.9378780197E+01

描述子的编号 (`Feature_ID`) 储存在下述文件中。但需要注意，该文件中的 `corr=` 以后所表现的相关性并不是统一的，因此不具有意义；有意义的部分只有描述子表达式。

```python
! cat -n feature_space/Uspace.name
```

         1	((F1*F2)/log(F3))  corr=      0.9953
         2	cos((F1*F2))  corr=      0.9949
         3	((F1*F2))^2  corr=      0.9949
         4	((F1*F2)*(F1+F2))  corr=      0.9948
         5	((F2)^6/log(F1))  corr=      0.9947
         6	((F1)^2*(F2)^3)  corr=      0.9946
         7	((F1*F2)*sqrt(F1))  corr=      0.9938
         8	(abs(F1-F2))^-1  corr=      0.9693
         9	(sqrt(F3)/abs(F1-F2))  corr=      0.9612
        10	(sin(F3)/abs(F1-F2))  corr=      0.9540
        11	(exp(F3)/abs(F1-F2))  corr=      0.9534
        12	(exp(-F3)/abs(F1-F2))  corr=      0.9486
        13	(F3/abs(F1-F2))  corr=      0.9474
        14	(cos(F2)/abs(F1-F2))  corr=      0.9468
        15	((F1)^6/(F1-F2))  corr=      0.8401
        16	((F1)^3/(F1-F2))  corr=      0.8265
        17	((F1*F2)/(F1-F2))  corr=      0.8177
        18	((F2)^6*(F1-F2))  corr=      0.8177
        19	((F1)^2/(F1-F2))  corr=      0.8100
        20	((F2)^2/(F1-F2))  corr=      0.7965
        21	((F2)^3/(F1-F2))  corr=      0.7951

其中一些简单的结论会是：

- 在较少描述子下结果良好的模型，并不意味着更多描述子下也会有更好的表现。譬如，1 号描述子在单描述子下表现最好，3, 8 号描述子在双描述子下表现最好；但这些描述子都没有在最终的三描述子模型的前五名中出现。

- 一般来说，比较好的模型中，三个描述子分别会出现在三个描述子集 $\boldsymbol{S}_1, \boldsymbol{S}_2, \boldsymbol{S}_3$ 中。但排名第 2 的模型是例外：两个描述子出现在 $\boldsymbol{S}_1$，而一个描述子出现在 $\boldsymbol{S}_2$ 中。

- 最好的若干个模型的线性拟合 RMSE 误差并不能说相差很大。

- 抛开偏置项 (Bias，即文件 {download}`models/top0100_003d_coeff` 的首列) 看，越靠前的描述子对预测值的贡献越大；但三个描述子分配不均的排名第 2 模型的情况会是例外，并且也存在排名第 5 的例外情况。

实际上，SISSO 给出了许多可供替代的模型。最佳和稍次的模型的表现未必相差很大；若表现稍次的模型所使用的描述子可能更符合物理直觉，那么这些描述子也不应被忽视。所以，从这个角度上讲，SISSO 不只是推荐一个具体的模型，而可以是推荐一系列表现相近的模型与描述子。譬如就上面的例子而言，用户一定会发现到描述子子结构 $f_1 f_2, \mathrm{abs} (f_1 - f_2)^{-1}, (f_1 - f_2)^{-1}$ 的重要性。因此，SISSO 可以启发用户使用这些描述子描述物理或材料问题的目的，或者从这些描述子出发进行下一步的机器学习过程。

[^Ouyang-Ghiringhelli.PRM.2018]: Ouyang, R.; Curtarolo, S.; Ahmetcik, E.; Scheffler, M.; Ghiringhelli, L. M. SISSO: A Compressed-Sensing Method for Identifying the Best Low-Dimensional Descriptor in an Immensity of Offered Candidates. *Phys. Rev. Materials* **2018**, *2* (8), 83802. doi: [10.1103/physrevmaterials.2.083802](https://doi.org/10.1103/physrevmaterials.2.083802).

[^Ouyang-Ghiringhelli.JPM.2019]: Ouyang, R.; Ahmetcik, E.; Carbogno, C.; Scheffler, M.; Ghiringhelli, L. M. Simultaneous Learning of Several Materials Properties from Incomplete Databases with Multi-Task SISSO. *J. Phys. Mater.* **2019**, *2* (2), 24002. doi: [10.1088/2515-7639/ab077b](https://doi.org/10.1088/2515-7639/ab077b).


# Homework (Chapter 4) - 201601639 홍승현

- 연습문제 5, 7, 8, 9, 10

## 연습문제 5

1. 가중치 행렬 U1, U2, U3, U4를 식(4.1) 처럼 쓰시오.
2. x = (1,0)T가 입력되었을 때 출력 o를 구하시오. 활성함수로 로지스틱 시그모이드를 사용하시오.
3. x = (1,0)T가 입력되었을 때 출력 o를 구하시오. 활성함수로 ReLU를 사용하시오.
4. (1,0)T의 기대 출력이 o = (0,1)T일 때, 현재 1.0인 u3(12) 가중치를 0.9로 줄이면 오류에 어떤 영향을 미치는지 설명하시오.

### 1. 가중치 행렬 U1, U2, U3, U4를 식(4.1) 처럼 쓰시오.

행렬 연산을 수월하게 하기 위해 `numpy`를 사용하였다.

```python
import numpy as np
```

```python
# DMLP의 가중치 초기화
weights = np.array([[[-0.3, 1.0, 1.2], [1.6, -1.0, -1.1]], [[1.0, 1.0, -1.0], [0.7, 0.5, 1.0]], [[0.5, -0.8, 1.0], [-0.1, 0.3, 0.4]], [[1.0, 0.1, -0.2], [-0.2, 1.3, -0.4]]])

u1 = weights[0].T
u2 = weights[1].T
u3 = weights[2].T
u4 = weights[3].T
```

```python
from sympy import * # 수식 표현을 위해 임포트
```

```python
for i in range(len(weights)):
    u = Symbol(f"U^{i+1}")
    mat = Matrix(weights[i], evaluate=False)
    equation = Eq(u, mat, evaluate=False)
    pprint(equation)
```

         ⎡-0.3  1.0   1.2 ⎤
    U¹ = ⎢                ⎥
         ⎣1.6   -1.0  -1.1⎦
         ⎡1.0  1.0  -1.0⎤
    U² = ⎢              ⎥
         ⎣0.7  0.5  1.0 ⎦
         ⎡0.5   -0.8  1.0⎤
    U³ = ⎢               ⎥
         ⎣-0.1  0.3   0.4⎦
         ⎡1.0   0.1  -0.2⎤
    U⁴ = ⎢               ⎥
         ⎣-0.2  1.3  -0.4⎦

### 2. x = (1,0)^T가 입력되었을 때 출력 o를 구하시오. 활성함수로 로지스틱 시그모이드를 사용하시오.

```python
def sigmoid(x):
    return 1 / (1 + np.exp(-x))
```

```python
x = np.array([1, 1, 0])
# forward
z1 = np.dot(x, u1)
o1 = sigmoid(z1)
o1 = np.append(1, o1)
o1
z2 = np.dot(o1, u2)
o2 = sigmoid(z2)
o2 = np.append(1, o2)

z3 = np.dot(o2, u3)
o3 = sigmoid(z3)
o3 = np.append(1, o3)

z4 = np.dot(o3, u4)
o4 = sigmoid(z4)

sigmoid_result1 = Eq(Function("Sigmoid")(Symbol("o_1")), Matrix(o4), evaluate=False)
sigmoid_result1

```

$\displaystyle \operatorname{Sigmoid}{\left(o_{1} \right)} = \left[\begin{matrix}0.720212909981978\\0.608070768763911\end{matrix}\right]$

### 3. x = (1,0)T가 입력되었을 때 출력 o를 구하시오. 활성함수로 ReLU를 사용하시오.

```python
def ReLU(x):
    return np.maximum(0.0, x)
```

```python
x = np.array([1, 1, 0]).T
# forward
z1 = np.dot(x, u1)
o1 = ReLU(z1)
o1 = np.append(1, o1)

z2 = np.dot(o1, u2)
o2 = ReLU(z2)
o2 = np.append(1, o2)

z3 = np.dot(o2, u3)
o3 = ReLU(z3)
o3 = np.append(1, o3)

z4 = np.dot(o3, u4)
o4 = ReLU(z4)

relu_result1 = Eq(Function("ReLU")(Symbol("o_1")), Matrix(o4), evaluate=False)
relu_result1

```

$\displaystyle \operatorname{ReLU}{\left(o_{1} \right)} = \left[\begin{matrix}0.949\\1.095\end{matrix}\right]$

### 4. (1,0)T의 기대 출력이 o = (0,1)T일 때, 현재 1.0인 u3(12) 가중치를 0.9로 줄이면 오류에 어떤 영향을 미치는지 설명하시오.

```python
# DMLP의 가중치 초기화
weights = np.array([[[-0.3, 1.6], [1.0, -1.0], [1.2, -1.1]],
                    [[1.0, 0.7], [1.0, 0.5], [-1.0, 1.0]],
                    [[0.5, -0.1], [-0.8, 0.3], [0.9, 0.4]], # [1.0, 0.4]를 [0.9, 0.4]로 수정
                    [[1.0, -0.2], [0.1, 1.3], [-0.2, -0.4]]])
u1 = weights[0]
u2 = weights[1]
u3 = weights[2]
u4 = weights[3]
```

```python
x = np.array([1, 1, 0]).T
# forward
z1 = np.dot(x, u1)
o1 = sigmoid(z1)
o1 = np.append(1, o1)

z2 = np.dot(o1, u2)
o2 = sigmoid(z2)
o2 = np.append(1, o2)

z3 = np.dot(o2, u3)
o3 = sigmoid(z3)
o3 = np.append(1, o3)

z4 = np.dot(o3, u4)
o4 = sigmoid(z4)

sigmoid_result2 = Eq(Function("Sigmoid")(Symbol("o_2")), Matrix(o4), evaluate=False)
sigmoid_result2
```

$\displaystyle \operatorname{Sigmoid}{\left(o_{2} \right)} = \left[\begin{matrix}0.71983780114165\\0.602290985036324\end{matrix}\right]$

```python
x = np.array([1, 1, 0]).T
# forward
z1 = np.dot(x, u1)
o1 = ReLU(z1)
o1 = np.append(1, o1)

z2 = np.dot(o1, u2)
o2 = ReLU(z2)
o2 = np.append(1, o2)

z3 = np.dot(o2, u3)
o3 = ReLU(z3)
o3 = np.append(1, o3)

z4 = np.dot(o3, u4)
o4 = ReLU(z4)

relu_result2 = Eq(Function("ReLU")(Symbol("o_2")), Matrix(o4), evaluate=False)
relu_result2

```

$\displaystyle \operatorname{ReLU}{\left(o_{2} \right)} = \left[\begin{matrix}0.9325\\0.8805\end{matrix}\right]$

#### 가중치 갱신 후 적용한 output과 이전의 output을 비교하면 다음과 같다.

```python
pprint(sigmoid_result1)
pprint(sigmoid_result2)
pprint(relu_result1)
pprint(relu_result2)
```

                  ⎡0.720212909981978⎤
    Sigmoid(o₁) = ⎢                 ⎥
                  ⎣0.608070768763911⎦
                  ⎡0.71983780114165 ⎤
    Sigmoid(o₂) = ⎢                 ⎥
                  ⎣0.602290985036324⎦
               ⎡0.949⎤
    ReLU(o₁) = ⎢     ⎥
               ⎣1.095⎦
               ⎡0.9325⎤
    ReLU(o₂) = ⎢      ⎥
               ⎣0.8805⎦

#### 결론

- 기존 o1과 가중치를 갱신한 후의 값인 o2를 비교하자면 기존 값에 비해 소량 감소했음을 알 수 있다.
- 이는 기대출력인 `(0, 1)^T`를 보았을 때, 첫 번째 값은 아주 미세하게 가까워지고 두 번째 값은 미세하게 멀어지고 있다고 볼 수 있다.

## 연습문제 7

- `그림 4-14`나머지 8개 화소의 값을 계산하시오.

```python
R = np.array([[1, 1, 1], [2, 1, 3], [0, 1, 0]])
G = np.array([[2, 2, 2], [1, 0, 1], [0, 0, 1]])
B = np.array([[0, 3, 0], [1, 0, 1], [1, 0, 0]])

pad_R = np.pad(R, (1, 1))
pad_G = np.pad(G, (1, 1))
pad_B = np.pad(B, (1, 1))

# pad_R, pad_G, pad_B를 합쳐 3차원으로 구성
pad_mat = np.array([pad_R, pad_G, pad_B])

pprint(pad_mat)
```

     [[[0 0 0 0 0]
       [0 1 1 1 0]
       [0 2 1 3 0]
       [0 0 1 0 0]
       [0 0 0 0 0]]

      [[0 0 0 0 0]
       [0 2 2 2 0]
       [0 1 0 1 0]
       [0 0 0 1 0]
       [0 0 0 0 0]]

      [[0 0 0 0 0]
       [0 0 3 0 0]
       [0 1 0 1 0]
       [0 1 0 0 0]
      [0 0 0 0 0]]]

```python
kernel = np.array([[[0, 0, 0],
                    [0, 0, 1],
                    [0, 1, 0]],
                   [[0, 2, 0],
                    [0, 2, 0],
                    [0, 2, 0]],
                   [[1, 0, 0],
                    [0, 2, 0],
                    [0, 0, 1]]])
```

### 변수 설정

- 보폭: s
- feature-map 크기: x

```python
# 보폭 s = 1
# feature map 크기 = x
s = 1
x = (pad_R.shape[0]-kernel.shape[1])/s+1
assert x.is_integer()   # x가 무조건 정수여야함
x = int(x)
feature_map = np.zeros((x, x))
for i in range(x):
    for j in range(x):
        sum = 0
        for rgb, ker in zip(pad_mat, kernel):
            sum += np.sum(rgb[i:i+3, j:j+3]*ker)
        feature_map[i][j] = sum
```

```python
pprint(Eq(Symbol("FeatureMap1"), Matrix(feature_map), evaluate=False))
```

                  ⎡9.0  13.0  9.0 ⎤
                  ⎢               ⎥
    FeatureMap₁ = ⎢9.0  8.0   13.0⎥
                  ⎢               ⎥
                  ⎣5.0  1.0   4.0 ⎦

## 연습문제 8

- `그림 4-8(b)`에서 커널$ \begin{bmatrix} -1 & -1 & -1 \\\\ 0 & 0 & 0 \\\\ 1 & 1 & 1 \end{bmatrix}$을 적용한 결과를 쓰시오. 이 때 0 덧대기를 하고 바이어스로 0.5를 사용하시오.

```python
conv_2d = np.array([[2, 2, 2, 2, 2, 1, 1, 1],
                     [2, 2, 2, 2, 2, 1, 1, 1],
                     [2, 2, 2, 2, 2, 1, 1, 1],
                     [2, 2, 2, 2, 2, 1, 1, 1],
                     [2, 2, 2, 9, 9, 9, 9, 9],
                     [2, 2, 2, 9, 9, 9, 9, 9],
                     [2, 2, 2, 9, 9, 9, 9, 9],
                     [2, 2, 2, 9, 9, 9, 9, 9]])

# 0 - padding
pad_conv_2d = np.pad(conv_2d, (1, 1))
# conv_2d.shape, pad_conv_2d.shape
pprint(Eq(Symbol("Pad-conv2D"), Matrix(pad_conv_2d), evaluate=False))
```

                 ⎡0  0  0  0  0  0  0  0  0  0⎤
                 ⎢                            ⎥
                 ⎢0  2  2  2  2  2  1  1  1  0⎥
                 ⎢                            ⎥
                 ⎢0  2  2  2  2  2  1  1  1  0⎥
                 ⎢                            ⎥
                 ⎢0  2  2  2  2  2  1  1  1  0⎥
                 ⎢                            ⎥
                 ⎢0  2  2  2  2  2  1  1  1  0⎥
    Pad-conv2D = ⎢                            ⎥
                 ⎢0  2  2  2  9  9  9  9  9  0⎥
                 ⎢                            ⎥
                 ⎢0  2  2  2  9  9  9  9  9  0⎥
                 ⎢                            ⎥
                 ⎢0  2  2  2  9  9  9  9  9  0⎥
                 ⎢                            ⎥
                 ⎢0  2  2  2  9  9  9  9  9  0⎥
                 ⎢                            ⎥
                 ⎣0  0  0  0  0  0  0  0  0  0⎦

```python
# 커널 설정
kernel = np.array([[-1, -1, -1], [0, 0, 0], [1, 1, 1]])

pprint(Eq(Symbol("kernel"), Matrix(kernel), evaluate=False))
```

             ⎡-1  -1  -1⎤
             ⎢          ⎥
    kernel = ⎢0   0   0 ⎥
             ⎢          ⎥
             ⎣1   1   1 ⎦

### 변수 설정

- 보폭: s
- 바이어스: bias
- feature map의 크기: x

```python
s = 1
bias = 0.5
x = (pad_conv_2d.shape[0]-kernel.shape[0])/s+1
assert x.is_integer()
x = int(x)
feature_map2 = np.zeros((x, x))
for i in range(x):
    for j in range(x):
        feature_map2[i][j] = np.sum(pad_conv_2d[i:i+3, j:j+3]*kernel) + bias
```

```python
pprint(Eq(Symbol("FeatureMap2"), Matrix(feature_map2), evaluate=False))
```

                  ⎡4.5   6.5    6.5    6.5    5.5    4.5    3.5    2.5 ⎤
                  ⎢                                                    ⎥
                  ⎢0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5 ⎥
                  ⎢                                                    ⎥
                  ⎢0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5 ⎥
                  ⎢                                                    ⎥
                  ⎢0.5   0.5    7.5   14.5   22.5   23.5   24.5   16.5 ⎥
    FeatureMap₂ = ⎢                                                    ⎥
                  ⎢0.5   0.5    7.5   14.5   22.5   23.5   24.5   16.5 ⎥
                  ⎢                                                    ⎥
                  ⎢0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5 ⎥
                  ⎢                                                    ⎥
                  ⎢0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5 ⎥
                  ⎢                                                    ⎥
                  ⎣-3.5  -5.5  -12.5  -19.5  -26.5  -26.5  -26.5  -17.5⎦

## 연습문제 9

- [문제8](#연습문제-8)에서 보폭을 `s = 2`로 설정했을 때 컨볼루션 결과를 쓰시오.

```python
s = 2
bias = 0.5
x = (pad_conv_2d.shape[0]-kernel.shape[0])/s+1
assert x.is_integer()
x = int(x)
feature_map2 = np.zeros((x, x))
for i in range(x):
    for j in range(x):
        feature_map2[i][j] = np.sum(pad_conv_2d[i:i+3, j:j+3]*kernel) + bias
```

### 결론

- 컽볼루션 연산을 하기 위해서는 `(size-kernel_size)/stride`가 정수여야 하지만 실수가 나오게 된다면 크기가 맞지 않으므로 컨볼루션 연산을 할 수 없게 된다.

## 연습문제 10

- [문제8](#연습문제-8)의 결과에 최대 풀링과 평균 풀링을 적용한 결과를 각각 쓰시오. 보폭으로 1을 사용하시오.

문제 8에 대한 결과는 `feature_map2` 변수에 저장해놓았었기에 출력으로 확인해본다.

```python
# 문제 8의 결과
pprint(Eq(Symbol("FeatureMap2"), Matrix(feature_map2), evaluate=False))
```

                  ⎡4.5   6.5    6.5    6.5    5.5    4.5    3.5    2.5 ⎤
                  ⎢                                                    ⎥
                  ⎢0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5 ⎥
                  ⎢                                                    ⎥
                  ⎢0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5 ⎥
                  ⎢                                                    ⎥
                  ⎢0.5   0.5    7.5   14.5   22.5   23.5   24.5   16.5 ⎥
    FeatureMap₂ = ⎢                                                    ⎥
                  ⎢0.5   0.5    7.5   14.5   22.5   23.5   24.5   16.5 ⎥
                  ⎢                                                    ⎥
                  ⎢0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5 ⎥
                  ⎢                                                    ⎥
                  ⎢0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5 ⎥
                  ⎢                                                    ⎥
                  ⎣-3.5  -5.5  -12.5  -19.5  -26.5  -26.5  -26.5  -17.5⎦

### 0-padding

- `8x8`의 크기를 유지하기 위해 0 덧대기를 진행한다.

```python
pad_feature_map2 = np.pad(feature_map2, (1, 1))
pprint(Eq(Symbol("Pad-FeatureMap2"), Matrix(pad_feature_map2), evaluate=False))

```

                      ⎡0.0  0.0   0.0    0.0    0.0    0.0    0.0    0.0    0.0
                      ⎢
                      ⎢0.0  4.5   6.5    6.5    6.5    5.5    4.5    3.5    2.5
                      ⎢
                      ⎢0.0  0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5
                      ⎢
                      ⎢0.0  0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5
                      ⎢
                      ⎢0.0  0.5   0.5    7.5   14.5   22.5   23.5   24.5   16.5
    Pad-FeatureMap2 = ⎢
                      ⎢0.0  0.5   0.5    7.5   14.5   22.5   23.5   24.5   16.5
                      ⎢
                      ⎢0.0  0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5
                      ⎢
                      ⎢0.0  0.5   0.5    0.5    0.5    0.5    0.5    0.5    0.5
                      ⎢
                      ⎢0.0  -3.5  -5.5  -12.5  -19.5  -26.5  -26.5  -26.5  -17.5
                      ⎢
                      ⎣0.0  0.0   0.0    0.0    0.0    0.0    0.0    0.0    0.0

    0.0⎤
       ⎥
    0.0⎥
       ⎥
    0.0⎥
       ⎥
    0.0⎥
       ⎥
    0.0⎥
       ⎥
    0.0⎥
       ⎥
    0.0⎥
       ⎥
    0.0⎥
       ⎥
    0.0⎥
       ⎥
    0.0⎦

### 변수 설정(이전과 동일)

- 보폭: s
- feature map의 크기: x

```python
s = 1
kernel_shape = 3
x = (pad_feature_map2.shape[0]-kernel_shape)/s+1
assert x.is_integer()
x = int(x)
max_pool_map = np.zeros((x, x))
avg_pool_map = np.zeros((x, x))
for i in range(x):
    for j in range(x):
        max_pool_map[i][j] = np.max(pad_feature_map2[i:i+3, j:j+3])
        avg_pool_map[i][j] = np.average(pad_feature_map2[i:i+3, j:j+3])
```

```python
pprint(Eq(Symbol("Max-Pool"), Matrix(max_pool_map), evaluate=False))
```

               ⎡6.5  6.5  6.5   6.5   6.5   5.5   4.5   3.5 ⎤
               ⎢                                            ⎥
               ⎢6.5  6.5  6.5   6.5   6.5   5.5   4.5   3.5 ⎥
               ⎢                                            ⎥
               ⎢0.5  7.5  14.5  22.5  23.5  24.5  24.5  24.5⎥
               ⎢                                            ⎥
               ⎢0.5  7.5  14.5  22.5  23.5  24.5  24.5  24.5⎥
    Max-Pool = ⎢                                            ⎥
               ⎢0.5  7.5  14.5  22.5  23.5  24.5  24.5  24.5⎥
               ⎢                                            ⎥
               ⎢0.5  7.5  14.5  22.5  23.5  24.5  24.5  24.5⎥
               ⎢                                            ⎥
               ⎢0.5  0.5  0.5   0.5   0.5   0.5   0.5   0.5 ⎥
               ⎢                                            ⎥
               ⎣0.5  0.5  0.5   0.5   0.5   0.5   0.5   0.5 ⎦

```python
pprint(Eq(Symbol("Average-Pool"), Matrix(avg_pool_map), evaluate=False))
```

                   ⎡ 1.33333333333333   2.11111111111111   2.33333333333333   2.22
                   ⎢
                   ⎢ 1.44444444444444   2.27777777777778          2.5         2.38
                   ⎢
                   ⎢0.333333333333333   1.27777777777778   2.83333333333333   5.27
                   ⎢
                   ⎢0.333333333333333   2.05555555555556   5.16666666666667   10.0
    Average-Pool = ⎢
                   ⎢0.333333333333333   2.05555555555556   5.16666666666667   10.0
                   ⎢
                   ⎢0.333333333333333   1.27777777777778   2.83333333333333   5.27
                   ⎢
                   ⎢-0.777777777777778  -2.05555555555556  -3.83333333333333  -6.1
                   ⎢
                   ⎣-0.888888888888889  -2.22222222222222        -4.0         -6.3

    222222222222          2.0         1.66666666666667   1.33333333333333   0.7777

    888888888889   2.16666666666667   1.83333333333333          1.5         0.8888

    777777777778   7.05555555555556   8.16666666666667          7.5         4.7777

    555555555556   13.6111111111111   15.8333333333333         14.5         9.2222

    555555555556   13.6111111111111   15.8333333333333         14.5         9.2222

    777777777778   7.05555555555556   8.16666666666667          7.5         4.7777

    6666666666667  -7.72222222222222        -8.5               -7.5         -4.666

    3333333333333  -7.88888888888889  -8.66666666666667  -7.66666666666667  -4.777

    77777777778⎤
               ⎥
    88888888889⎥
               ⎥
    7777777778 ⎥
               ⎥
    2222222222 ⎥
               ⎥
    2222222222 ⎥
               ⎥
    7777777778 ⎥
               ⎥
    66666666667⎥
               ⎥
    77777777778⎦

pprint로 하였을 때 2줄 이상으로 나오므로 `평균풀링`을 다시 출력해보면 다음과 같다.

```python
pprint(avg_pool_map)
```

    [[ 1.33333333  2.11111111  2.33333333  2.22222222  2.          1.66666667
                               1.33333333  0.77777778]
     [ 1.44444444  2.27777778  2.5         2.38888889  2.16666667  1.83333333
                               1.5         0.88888889]
     [ 0.33333333  1.27777778  2.83333333  5.27777778  7.05555556  8.16666667
                               7.5         4.77777778]
     [ 0.33333333  2.05555556  5.16666667 10.05555556 13.61111111 15.83333333
                              14.5         9.22222222]
     [ 0.33333333  2.05555556  5.16666667 10.05555556 13.61111111 15.83333333
                              14.5         9.22222222]
     [ 0.33333333  1.27777778  2.83333333  5.27777778  7.05555556  8.16666667
                               7.5         4.77777778]
         [-0.77777778 -2.05555556 -3.83333333 -6.16666667 -7.72222222 -8.5
                              -7.5        -4.66666667]
     [-0.88888889 -2.22222222 -4.         -6.33333333 -7.88888889 -8.66666667
                             -7.66666667 -4.77777778]]

```python

```


# PerfForesightConsumerType: Perfect foresight consumption-saving

```python
# Initial imports and notebook setup, click arrow to show

from copy import copy

import matplotlib.pyplot as plt
import numpy as np

from HARK.ConsumptionSaving.ConsIndShockModel import PerfForesightConsumerType
from HARK.utilities import plot_funcs

mystr = lambda number: "{:.4f}".format(number)
```

The module `HARK.ConsumptionSaving.ConsIndShockModel` concerns consumption-saving models with idiosyncratic shocks to (non-capital) income.  All of the models assume CRRA utility with geometric discounting, no bequest motive, and income shocks are fully transitory or fully permanent.

`ConsIndShockModel` currently includes three models:
1. A very basic "perfect foresight" model with no uncertainty.
2. A model with risk over transitory and permanent income shocks.
3. The model described in (2), with an interest rate for debt that differs from the interest rate for savings.

This notebook provides documentation for the first of these three models.
$\newcommand{\CRRA}{\rho}$
$\newcommand{\DiePrb}{\mathsf{D}}$
$\newcommand{\PermGroFac}{\Gamma}$
$\newcommand{\Rfree}{\mathsf{R}}$
$\newcommand{\DiscFac}{\beta}$

## Statement of perfect foresight consumption-saving model

The `PerfForesightConsumerType` class the problem of a consumer with Constant Relative Risk Aversion utility
${\CRRA}$
\begin{equation}
U(C) = \frac{C^{1-\CRRA}}{1-\rho},
\end{equation}
has perfect foresight about everything except whether he will die between the end of period $t$ and the beginning of period $t+1$, which occurs with probability $\DiePrb_{t+1}$.  Permanent labor income $P_t$ grows from period $t$ to period $t+1$ by factor $\PermGroFac_{t+1}$.

At the beginning of period $t$, the consumer has an amount of market resources $M_t$ (which includes both market wealth and currrent income) and must choose how much of those resources to consume $C_t$ and how much to retain in a riskless asset $A_t$, which will earn return factor $\Rfree$.  The consumer cannot necessarily borrow arbitarily; instead, he might be constrained to have a wealth-to-income ratio at least as great as some "artificial borrowing constraint" $\underline{a} \leq 0$.

The agent's flow of future utility $U(C_{t+n})$ from consumption is geometrically discounted by factor $\DiscFac$ per period. If the consumer dies, he receives zero utility flow for the rest of time.

The agent's problem can be written in Bellman form as:

\begin{eqnarray*}
V_t(M_t,P_t) &=& \max_{C_t}~U(C_t) ~+ \DiscFac (1 - \DiePrb_{t+1}) V_{t+1}(M_{t+1},P_{t+1}), \\
& s.t. & \\
A_t &=& M_t - C_t, \\
A_t/P_t &\geq& \underline{a}, \\
M_{t+1} &=& \Rfree A_t + Y_{t+1}, \\
Y_{t+1} &=& P_{t+1}, \\
P_{t+1} &=& \PermGroFac_{t+1} P_t.
\end{eqnarray*}

The consumer's problem is characterized by a coefficient of relative risk aversion $\CRRA$, an intertemporal discount factor $\DiscFac$, an interest factor $\Rfree$, and age-varying sequences of the permanent income growth factor $\PermGroFac_t$ and survival probability $(1 - \DiePrb_t)$.

While it does not reduce the computational complexity of the problem (as permanent income is deterministic, given its initial condition $P_0$), HARK represents this problem with *normalized* variables (represented in lower case), dividing all real variables by permanent income $P_t$ and utility levels by $P_t^{1-\CRRA}$.  The Bellman form of the model thus reduces to:

\begin{eqnarray*}
v_t(m_t) &=& \max_{c_t}~U(c_t) ~+ \DiscFac (1 - \DiePrb_{t+1}) \PermGroFac_{t+1}^{1-\CRRA} v_{t+1}(m_{t+1}), \\
& s.t. & \\
a_t &=& m_t - c_t, \\
a_t &\geq& \underline{a}, \\
m_{t+1} &=& \Rfree/\PermGroFac_{t+1} a_t + 1.
\end{eqnarray*}

## Solution method for PerfForesightConsumerType

Because of the assumptions of CRRA utility, no risk other than mortality, and no artificial borrowing constraint, the problem has a closed form solution.  In fact, the consumption function is perfectly linear, and the value function composed with the inverse utility function is also linear.  The mathematical solution of this model is described in detail in the lecture notes [PerfForesightCRRA](https://www.econ2.jhu.edu/people/ccarroll/public/lecturenotes/consumption/PerfForesightCRRA).

The one period problem for this model is solved by the function `solveConsPerfForesight`, which creates an instance of the class `ConsPerfForesightSolver`. To construct an instance of the class `PerfForesightConsumerType`, several parameters must be passed to its constructor as shown in the table below.

## Example parameter values to construct an instance of PerfForesightConsumerType

| Parameter | Description | Code | Example value | Time-varying? |
| :---: | --- | --- | --- | --- |
| $\DiscFac$ |Intertemporal discount factor  | $\texttt{DiscFac}$ | $0.96$ |  |
| $\CRRA $ |Coefficient of relative risk aversion | $\texttt{CRRA}$ | $2.0$ | |
| $\Rfree$ | Risk free interest factor | $\texttt{Rfree}$ | $1.03$ | |
| $1 - \DiePrb_{t+1}$ |Survival probability | $\texttt{LivPrb}$ | $[0.98]$ | $\surd$ |
|$\PermGroFac_{t+1}$|Permanent income growth factor|$\texttt{PermGroFac}$| $[1.01]$ | $\surd$ |
|$\underline{a}$|Artificial borrowing constraint|$\texttt{BoroCnstArt}$| $None$ |  |
|$(none)$|Maximum number of gridpoints in consumption function |$\texttt{aXtraCount}$| $200$ |  |
|$T$| Number of periods in this type's "cycle" |$\texttt{T_cycle}$| $1$ | |
|(none)| Number of times the "cycle" occurs |$\texttt{cycles}$| $0$ | |

Note that the survival probability and income growth factor have time subscripts; likewise, the example values for these parameters are *lists* rather than simply single floats.  This is because those parameters are *time-varying*: their values can depend on which period of the problem the agent is in.  All time-varying parameters *must* be specified as lists, even if the same value occurs in each period for this type.

The artificial borrowing constraint can be any non-positive `float`, or it can be `None` to indicate no artificial borrowing constraint.  The maximum number of gridpoints in the consumption function is only relevant if the borrowing constraint is not `None`; without an upper bound on the number of gridpoints, kinks in the consumption function will propagate indefinitely in an infinite horizon model if there is a borrowing constraint, eventually resulting in an overflow error.  If there is no artificial borrowing constraint, then the number of gridpoints used to represent the consumption function is always exactly two.

The last two parameters in the table specify the "nature of time" for this type: the number of (non-terminal) periods in this type's "cycle", and the number of times that the "cycle" occurs.  *Every* subclass of `AgentType` uses these two code parameters to define the nature of time.  Here, `T_cycle` has the value $1$, indicating that there is exactly one period in the cycle, while `cycles` is $0$, indicating that the cycle is repeated in *infinite* number of times-- it is an infinite horizon model, with the same "kind" of period repeated over and over.

In contrast, we could instead specify a life-cycle model by setting `T_cycle` to $1$, and specifying age-varying sequences of income growth and survival probability.  In all cases, the number of elements in each time-varying parameter should exactly equal $\texttt{T_cycle}$.

The parameter $\texttt{AgentCount}$ specifies how many consumers there are of this *type*-- how many individuals have these exact parameter values and are *ex ante* homogeneous.  This information is not relevant for solving the model, but is needed in order to simulate a population of agents, introducing *ex post* heterogeneity through idiosyncratic shocks.  Of course, simulating a perfect foresight model is quite boring, as there are *no* idiosyncratic shocks other than death!

The cell below defines a dictionary that can be passed to the constructor method for `PerfForesightConsumerType`, with the values from the table here.

```python
PerfForesightDict = {
    # Parameters actually used in the solution method
    "CRRA": 2.0,  # Coefficient of relative risk aversion
    "Rfree": 1.03,  # Interest factor on assets
    "DiscFac": 0.96,  # Default intertemporal discount factor
    "LivPrb": [0.98],  # Survival probability
    "PermGroFac": [1.01],  # Permanent income growth factor
    "BoroCnstArt": None,  # Artificial borrowing constraint
    "aXtraCount": 200,  # Maximum number of gridpoints in consumption function
    # Parameters that characterize the nature of time
    "T_cycle": 1,  # Number of periods in the cycle for this agent type
    "cycles": 0,  # Number of times the cycle occurs (0 --> infinitely repeated)
}
```

## Solving and examining the solution of the perfect foresight model

With the dictionary we have just defined, we can create an instance of `PerfForesightConsumerType` by passing the dictionary to the class (as if the class were a function).  This instance can then be solved by invoking its `solve` method.

```python
PFexample = PerfForesightConsumerType(**PerfForesightDict)
PFexample.cycles = 0
PFexample.solve()
```

The $\texttt{solve}$ method fills in the instance's attribute `solution` as a time-varying list of solutions to each period of the consumer's problem.  In this case, `solution` will be a list with exactly one instance of the class `ConsumerSolution`, representing the solution to the infinite horizon model we specified.

```python
print(PFexample.solution)
```

    [<HARK.ConsumptionSaving.ConsIndShockModel.ConsumerSolution object at 0x7f7ab2b7e370>]

Each element of `solution` has a few attributes. To see all of them, we can use the \texttt{vars} built in function:

the consumption functions reside in the attribute $\texttt{cFunc}$ of each element of `ConsumerType.solution`.  This method creates a (time varying) attribute $\texttt{cFunc}$ that contains a list of consumption functions.

```python
print(vars(PFexample.solution[0]))
```

    {'cFunc': <HARK.interpolation.LinearInterp object at 0x7f7ab2b7e4f0>, 'vFunc': <HARK.interpolation.ValueFuncCRRA object at 0x7f7ab2b7c7f0>, 'vPfunc': <HARK.interpolation.MargValueFuncCRRA object at 0x7f7ab2b7cb20>, 'vPPfunc': <HARK.utilities.NullFunc object at 0x7f7ab2b7e3a0>, 'mNrmMin': -50.49994992551661, 'hNrm': 50.49994992551661, 'MPCmin': 0.04428139169919579, 'MPCmax': 0.04428139169919579}

The two most important attributes of a single period solution of this model are the (normalized) consumption function $\texttt{cFunc}$ and the (normalized) value function $\texttt{vFunc}$.  Let's plot those functions near the lower bound of the permissible state space (the attribute $\texttt{mNrmMin}$ tells us the lower bound of $m_t$ where the consumption function is defined).

```python
print("Linear perfect foresight consumption function:")
mMin = PFexample.solution[0].mNrmMin
plot_funcs(PFexample.solution[0].cFunc, mMin, mMin + 10.0)
```

```python
print("Perfect foresight value function:")
plot_funcs(PFexample.solution[0].vFunc, mMin + 0.1, mMin + 10.1)
```

An element of `solution` also includes the (normalized) marginal value function $\texttt{vPfunc}$, and the lower and upper bounds of the marginal propensity to consume (MPC) $\texttt{MPCmin}$ and $\texttt{MPCmax}$.  Note that with a linear consumption function, the MPC is constant, so its lower and upper bound are identical.

### Liquidity constrained perfect foresight example

Without an artificial borrowing constraint, a perfect foresight consumer is free to borrow against the PDV of his entire future stream of labor income-- his "human wealth" $\texttt{hNrm}$-- and he will consume a constant proportion of his total wealth (market resources plus human wealth).  If we introduce an artificial borrowing constraint, both of these features vanish.  In the cell below, we define a parameter dictionary that prevents the consumer from borrowing *at all*, create and solve a new instance of `PerfForesightConsumerType` with it, and then plot its consumption function.

```python
LiqConstrDict = copy(PerfForesightDict)
LiqConstrDict["BoroCnstArt"] = 0.0  # Set the artificial borrowing constraint to zero

LiqConstrExample = PerfForesightConsumerType(**LiqConstrDict)
LiqConstrExample.cycles = 0  # Make this type be infinite horizon
LiqConstrExample.solve()

print("Liquidity constrained perfect foresight consumption function:")
plot_funcs(LiqConstrExample.solution[0].cFunc, 0.0, 10.0)
```

```python
# At this time, the value function for a perfect foresight consumer with an artificial borrowing constraint is not computed nor included as part of its $\texttt{solution}$.
```

## Simulating the perfect foresight consumer model

Suppose we wanted to simulate many consumers who share the parameter values that we passed to `PerfForesightConsumerType`-- an *ex ante* homogeneous *type* of consumers.  To do this, our instance would have to know *how many* agents there are of this type, as well as their initial levels of assets $a_t$ and permanent income $P_t$.

### Setting simulation parameters

Let's fill in this information by passing another dictionary to `PFexample` with simulation parameters.  The table below lists the parameters that an instance of `PerfForesightConsumerType` needs in order to successfully simulate its model using the `simulate` method.

| Description | Code | Example value |
| :---: | --- | --- |
| Number of consumers of this type | $\texttt{AgentCount}$ | $10000$ |
| Number of periods to simulate | $\texttt{T_sim}$ | $120$ |
| Mean of initial log (normalized) assets | $\texttt{aNrmInitMean}$ | $-6.0$ |
| Stdev of initial log  (normalized) assets | $\texttt{aNrmInitStd}$ | $1.0$ |
| Mean of initial log permanent income | $\texttt{pLvlInitMean}$ | $0.0$ |
| Stdev of initial log permanent income | $\texttt{pLvlInitStd}$ | $0.0$ |
| Aggregrate productivity growth factor | $\texttt{PermGroFacAgg}$ | $1.0$ |
| Age after which consumers are automatically killed | $\texttt{T_age}$ | $None$ |

We have specified the model so that initial assets and permanent income are both distributed lognormally, with mean and standard deviation of the underlying normal distributions provided by the user.

The parameter $\texttt{PermGroFacAgg}$ exists for compatibility with more advanced models that employ aggregate productivity shocks; it can simply be set to 1.

In infinite horizon models, it might be useful to prevent agents from living extraordinarily long lives through a fortuitous sequence of mortality shocks.  We have thus provided the option of setting $\texttt{T_age}$ to specify the maximum number of periods that a consumer can live before they are automatically killed (and replaced with a new consumer with initial state drawn from the specified distributions).  This can be turned off by setting it to `None`.

The cell below puts these parameters into a dictionary, then gives them to `PFexample`.  Note that all of these parameters *could* have been passed as part of the original dictionary; we omitted them above for simplicity.

```python
SimulationParams = {
    "AgentCount": 10000,  # Number of agents of this type
    "T_sim": 120,  # Number of periods to simulate
    "aNrmInitMean": -6.0,  # Mean of log initial assets
    "aNrmInitStd": 1.0,  # Standard deviation of log initial assets
    "pLvlInitMean": 0.0,  # Mean of log initial permanent income
    "pLvlInitStd": 0.0,  # Standard deviation of log initial permanent income
    "PermGroFacAgg": 1.0,  # Aggregate permanent income growth factor
    "T_age": None,  # Age after which simulated agents are automatically killed
}

PFexample.assign_parameters(**SimulationParams)
```

To generate simulated data, we need to specify which variables we want to track the "history" of for this instance.  To do so, we set the `track_vars` attribute of our `PerfForesightConsumerType` instance to be a list of strings with the simulation variables we want to track.

In this model, valid arguments to `track_vars` include $\texttt{mNrm}$, $\texttt{cNrm}$, $\texttt{aNrm}$, and $\texttt{pLvl}$.  Because this model has no idiosyncratic shocks, our simulated data will be quite boring.

### Generating simulated data

Before simulating, the `initialize_sim` method must be invoked.  This resets our instance back to its initial state, drawing a set of initial $\texttt{aNrm}$ and $\texttt{pLvl}$ values from the specified distributions and storing them in the attributes $\texttt{aNrmNow_init}$ and $\texttt{pLvlNow_init}$.  It also resets this instance's internal random number generator, so that the same initial states will be set every time `initialize_sim` is called.  In models with non-trivial shocks, this also ensures that the same sequence of shocks will be generated on every simulation run.

Finally, the `simulate` method can be called.

```python
PFexample.track_vars = ['mNrm']
PFexample.initialize_sim()
PFexample.simulate()
```

    {'mNrm': array([[  1.01834458,   1.00217811,   1.00344698, ...,   1.00498677,
               1.00181962,   1.00082289],
            [ -0.28796625,  -0.3037228 ,  -0.30248611, ...,  -0.30098536,
               1.00624616,  -0.30504366],
            [ -1.56115391,  -1.57651094,  -1.5753056 , ...,  -1.57384291,
              -0.2997579 ,  -1.5777983 ],
            ...,
            [-38.27661177, -47.94841819, -47.94835533, ..., -44.8431327 ,
             -14.5538836 , -18.88031749],
            [-38.58654905, -48.01311453, -48.01305326, ..., -44.9865675 ,
             -15.4653408 , -19.68207246],
            [-38.88862747, -48.07617041, -48.0761107 , ..., -45.12636532,
             -16.35368684,   1.00096976]])}

```python
# Each simulation variable $\texttt{X}$ named in $\texttt{track_vars}$ will have the *history* of that variable for each agent stored in the attribute $\texttt{X_hist}$ as an array of shape $(\texttt{T_sim},\texttt{AgentCount})$.  To see that the simulation worked as intended, we can plot the mean of $m_t$ in each simulated period:
```

```python
plt.plot(np.mean(PFexample.history['mNrm'], axis=1))
plt.xlabel("Time")
plt.ylabel("Mean normalized market resources")
plt.show()
```

A perfect foresight consumer can borrow against the PDV of his future income-- his human wealth-- and thus as time goes on, our simulated agents approach the (very negative) steady state level of $m_t$ while being steadily replaced with consumers with roughly $m_t=1$.

The slight wiggles in the plotted curve are due to consumers randomly dying and being replaced; their replacement will have an initial state drawn from the distributions specified by the user.  To see the current distribution of ages, we can look at the attribute $\texttt{t_age}$.

```python
N = PFexample.AgentCount
F = np.linspace(0.0, 1.0, N)
plt.plot(np.sort(PFexample.t_age), F)
plt.xlabel("Current age of consumers")
plt.ylabel("Cumulative distribution")
plt.show()
```

The distribution is (discretely) exponential, with a point mass at 120 with consumers who have survived since the beginning of the simulation.

One might wonder why HARK requires users to call `initialize_sim` before calling `simulate`: Why doesn't `simulate` just call `initialize_sim` as its first step?  We have broken up these two steps so that users can simulate some number of periods, change something in the environment, and then resume the simulation.

When called with no argument, `simulate` will simulate the model for $\texttt{T_sim}$ periods.  The user can optionally pass an integer specifying the number of periods to simulate (which should not exceed $\texttt{T_sim}$).

In the cell below, we simulate our perfect foresight consumers for 80 periods, then seize a bunch of their assets (dragging their wealth even more negative), then simulate for the remaining 40 periods.

The `state_prev` attribute of an AgenType stores the values of the model's state variables in the _previous_ period of the simulation.

```python
PFexample.initialize_sim()
PFexample.simulate(80)
PFexample.state_prev['aNrm'] += -5.0  # Adjust all simulated consumers' assets downward by 5
PFexample.simulate(40)

plt.plot(np.mean(PFexample.history['mNrm'], axis=1))
plt.xlabel("Time")
plt.ylabel("Mean normalized market resources")
plt.show()
```

```python

```


# Constant selection

```python
import numpy as np
import matplotlib.pyplot as plt
from tqdm.notebook import tqdm
import ipywidgets as widgets
%matplotlib inline
%matplotlib widget
```

## The Moran model + constant selection

The difference, the _only_ difference, between this and the original Moran model is that we now consider fitness in who we select when drawing an individual to reproduce. Before we considered each individual equally but now we weight individuals of each species using their fitness.

we represent this weighting by the `fit_ratio` parameter which described the ratio between species 1 fitness($k_1$) and species 2 fitness ($k_2$). Inorder to do this weighting without violating the rule that the probabilities of each species must add up to 1, we do the weighting like this:
$$
P_{birth}(S=1) = \frac{(k_1/k_2) f_1}{(k_1/k_2) f_1 + f_2}
$$

This is the only change we have to make to the basic Moran model to introduce constant selection, and we will never have to consider the individual species' fitnesses on their own.

## Temporally varying selection

Vellend also introduces a 'temporally varying' selection. He introduces this after frequency dependence but his model is more like this constant selection model so we will consider that here. In Vellend's description, the change in time comes as a change in the fitness ratio that depends on the simulated year.

First let's say that we've arrived at our fitness ratio, we'll call this $\gamma$
\begin{align}
\frac{k_1}{k_2} = \gamma
\end{align}

We do the temporal variation, at least the simple example vellend shows, by flipping the ratio every 10 years. In general we flip the ratio every $y_{flip}$ years. When we've gone through $y_{flip}$ years we simple redefine the fitness ratio as:
\begin{align}
\frac{k_1}{k_2} = \gamma^{-1}
\end{align}

I've represented this in the code with a variable called `time_dep` that can either be `True` or `False`. If its true, we do the 10 year flip as shown in the chapter.

```python
def moran_constant_selection(n_indiv, n_years, init_f1, fit_ratio, time_dep):
    """ Implements the Moran model including constant selection and simulates it in time.

        Parameters
        ----------
        n_indiv: int
            number of individuals in the community
        n_years: int
            number of potential 'turnovers' of the community
        init_f1: float
            initial frequency of species 1
        fit_ratio: float
            ratio of fitness of species 1 to fitness of species 2

        Return
        ------
        moran: np.array (n_years, 2)
            contains the species frequencies for each year of the simulation """

    # set up an empty array for the simulated frequencies
    # initialize with the given frequency. python counts from 0, sorrryyyy
    moran = np.zeros((n_years, 2))
    moran[0] = np.array([init_f1, 1 - init_f1])

    # get a vector representing the community. vellend calls this COM
    # this just makes a random vector drawn from the initial frequency
    comm = np.random.choice([0, 1], size=n_indiv, replace=True, p=moran[0])

    # now we can do the loop as in vellend. I'm going to write it a bit differently
    # but the idea is basically the same. (yes its slower)
    for year in range(1, n_years):

        # if time dependence is true we flip the fitness ratio every ten years
        if time_dep and year % 10 == 0:
            fit_ratio = fit_ratio**-1

        # for each year we potentially replace each individual so we can loop
        # over that as well
        for indiv in range(n_indiv):
            # get probability of species 1
            f1 = np.sum(comm == 0) / n_indiv
            pr1 = fit_ratio * f1 / (fit_ratio * f1 + (1 - f1))

            # replace one individual with another
            death = np.random.randint(n_indiv)
            birth = np.random.choice([0, 1], p=[pr1, 1 - pr1])
            comm[death] = birth

        # when we're done looping over all of the individuals we can update
        # frequencies for the year (the timescale we care about)
        f1 = np.sum(comm == 0) / n_indiv
        moran[year] = np.array([f1, 1 - f1])

    return moran
```

## Running the simulations and visualization
As before we have a bunch of messy code to draw the plots. Once again we can change parameter values in the first block and not worry about the rest (unless you want to!)

```python
def draw_simulation(iterations=10, individuals=250, years=50, initial_freq=0.5, fit_ratio=1, time_dep=False):
    # the plot bit, this just makes a blank plot
    fig, ax = plt.subplots(ncols=2, figsize=(10,4), sharey=True, gridspec_kw = {'width_ratios':[3, 1]})
    # trajectory labels
    ax[0].set_xlabel('Years')
    ax[0].set_ylabel('Species 1 frequency')
    ax[0].set_ylim([0, 1])
    # distribution labels and bins
    hist_bins = np.arange(0, 1.1, 0.1)
    ax[1].set_xlabel('Count')
    ax[1].set_xlim([0, iterations])

    # we're going to plot a distribution too need an array for it
    dist = np.zeros(iterations)

    # we run the simulation and draw a trajectory
    for i in tqdm(range(iterations)):
        simulation = moran_constant_selection(individuals, years, initial_freq, fit_ratio, time_dep)
        ax[0].plot(range(years), simulation[:, 0], c='C0')
        # add final freq to our dist array
        dist[i] = simulation[-1, 0]

    # plot the distribution too
    ax[1].hist(dist, bins=hist_bins, orientation='horizontal')
    ax[1].axhline(np.mean(dist), linestyle='--')

    plt.tight_layout()
    plt.show()

# set up the interface
widgets.interact_manual(draw_simulation, iterations=(5, 50, 5), individuals=(10, 1000, 10), years=(5, 200, 5),
                  initial_freq=(0.0, 1.0, 0.01), fit_ratio=(0.67, 1.5, 0.01), time_dep=(False))
```

    interactive(children=(IntSlider(value=10, description='iterations', max=50, min=5, step=5), IntSlider(value=25…

    <function __main__.draw_simulation(iterations=10, individuals=250, years=50, initial_freq=0.5, fit_ratio=1, time_dep=False)>

```python

```