# 第八讲：核方法、序列最小优化算法

先回顾上一讲的内容：

我们写出凸优化问题（假设数据都是线性可分的）：

$$\begin{align}\displaystyle\operatorname*{min}_{w,b}&\quad\frac{1}{2}\lVert w\rVert^2\\\mathrm{s.t.}&\quad y^{(i)}\left(w^Tx^{(i)}+b\right)\geq 1,\quad i=1,\cdots,m\end{align}$$

对于给定的训练集，算法会找到该数据集的最优间隔分类器，可以使训练样本的几何间隔最大化，进而推导出了这个问题的对偶问题：

$$\begin{align}\displaystyle\operatorname*{max}_{\alpha}&\quad W(\alpha)=\sum_{i=1}^m\alpha_i-\frac{1}{2}\sum_{i,j=1}^m\alpha_i\alpha_jy^{(i)}y^{(j)}\left\langle x^{(i)},x^{(j)}\right\rangle\\\mathrm{s.t.}&\quad \alpha\geq 0,\quad i=1,\cdots,m\\&\quad\sum_{i=1}^m\alpha_iy^{(i)}=0\end{align}$$

在推导过程中，我们也得到$w$：

$$w=\sum_{i=1}^m\alpha_iy^{(i)}x^{(i)}$$

于是，当我们需要对样本进行预测时，我们需要计算输入向量的假设值：

$$\begin{align}h_{w,b}(x)&=g\left(w^Tx+b\right)\\&=g\left(\left(\sum_{i=1}^m\alpha_iy^{(i)}x^{(i)}\right)^Tx+b\right)\\&=g\left(\sum_{i=1}^m\alpha_iy^{(i)}\left\langle x^{(i)},x\right\rangle+b\right)\end{align}$$

接下来，我们介绍核方法。

## 7. 核方法（Kernels）

回忆关于线性回归的讨论，我们用输入$x$表示“出租房屋的实用面积”，然后考虑使用诸如$x,x^2,x^3$的特征值来拟合一个三次函数。为了区分这两组变量，我们称“最初的”输入值（即本例中的“实用面积”$x$）为问题的**输入属性（input attributes）**；而当“输入属性”映射在一组新的量中，进而转给学习算法时，我们称这组新的量为**输入特征（input features）**（不过，不同的作者会使用不同的名称来指代这两个概念，我们将在后面的学习中继续保持这种称谓）。我们使用$\phi$表示**特征映射（feature mapping）**，即属性到特征的映射，比如在本例中有：

$$\phi(x)=\begin{bmatrix}x\\x^2\\x^3\end{bmatrix}$$

相比于将支持向量机应用在输入属性$x$上，我们更需要知道如何将其使用在输入特征$\phi(x)$上。其实方法很简单，只需要回顾前面的算法，将式子中的$x$变为$\phi(x)$即可。

前面我们知道了支持向量机算法完全可以使用向量内积$\langle x,z\rangle$表示，所以现在我们可以将这些内积替换为$\left\langle\phi(x),\phi(z)\right\rangle$。特别的，对于给定的特征映射$\phi$，我们定义相应的**核（kernel）**为：

$$K(x,z)=\phi(x)^T\phi(z)$$

如此，我们就可以将前面出现的$\langle x,z\rangle$统统替换为$K(x,z)$，而此时的算法将使用$\phi$作为学习的特征。

对于给定的$\phi$，我们可以很容易的通过求$\phi(x)$与$\phi(z)$的内积得到$K(x,z)$。更有意思的是，即使$\phi$本身很难计算（比如它可能是一个维数极高的向量甚至是无限维向量，这种情况下计算$\phi(x)$的代价将会非常大甚至是不可能的，但是我们可以用很小的代价计算两个向量之间的核），$K(x,z)$也会很容易得到。实际上，通过我们的算法高效解出$K(x,z)$后，就可以使用支持向量机在维数极高的特征空间（特征空间由$\phi$给出）中学习，而且甚至不需要将向量$\phi(x)$计算或表达出来。

我们来看一个例子：设$x,z\in\mathbb{R}^n$，考虑：

$$K(x,z)=\left(x^Tz\right)^2$$

我们可以将其展开为：

$$\begin{align}K(x,z)&=\left(\sum_{i=1}^nx_iz_i\right)\left(\sum_{j=1}^nx_jz_j\right)\\&=\sum_{i=1}^n\sum_{j=1}^nx_ix_jz_iz_j\\&=\sum_{i,j=1}^n\left(x_ix_j\right)\left(z_iz_j\right)\end{align}$$

于是我们可以得知$K(x,z)=\phi(x)^T\phi(z)$中的$\phi$应为（这里我们先假设$n=3$）：

$$\phi(x)=\begin{bmatrix}x_1x_1\\x_1x_2\\x_1x_3\\x_2x_1\\x_2x_2\\x_2x_3\\x_3x_1\\x_3x_2\\x_3x_3\end{bmatrix}$$

我们发现，求出$\phi(x)$需要$O(n^2)$的复杂度，而求出$K(x,z)$仅仅需要$O(n)$的复杂度，也就是与输入属性的维数呈线性关系。

再看另一个核：

$$\begin{align}K(x,z)&=\left(x^Tz+c\right)^2\\&=\sum_{i,j=1}^n\left(x_ix_j\right)\left(z_iz_j\right)+\sum_{i=1}^n\left(\sqrt{2c}x_i\right)\left(\sqrt{2c}z_i\right)+c^2\end{align}$$

这个核相应的特征映射为（仍然假设$n=3$）：

$$\phi(x)=\begin{bmatrix}x_1x_1\\x_1x_2\\x_1x_3\\x_2x_1\\x_2x_2\\x_2x_3\\x_3x_1\\x_3x_2\\x_3x_3\\\sqrt{2c}x_1\\\sqrt{2c}x_2\\\sqrt{2c}x_3\\c\end{bmatrix}$$

式中的参数$c$控制$x_i$（一次项）与$x_ix_j$（二次项）之间的权重。

推广到高维，$K(x,z)=\left(x^Tz+c\right)^d$相应的特征映射在$\binom{n+d}{d}$特征空间中，也就是一共有$\binom{n+d}{d}$个项特征单项式，其中每一项型为$x_{i1}x_{i2}\cdots x_{ik}$一直到$d$阶。尽管在复杂度为$O(n^d)$的高维空间中，对$K(x,z)$的计算仍然仅有$O(n)$的线性复杂度，因为我们并不需要在这种维度极高的空间中直接计算出特征值向量。

我们再来换一个角度观察核，能够直观（这种直观印象并非严格成立）的看出，如果向量$\phi(x)$与$\phi(z)$方向靠的比较近，那么根据$K(x,z)=\phi(x)^T\phi(z)$可知这个值会比较大；反正，如果两个向量近乎正交（方向离的比较远），则$K(x,z)=\phi(x)^T\phi(z)$将会很小。如此，我们就可以用$K(x,z)$度量$\phi(x)$与$\phi(z)$的相似度，或者$x$与$z$的相似度。

基于这种直观印象，如果我们在处理某个学习问题，在遇到$K(x,z)$时，会联想到这个量可以合理的计算$x,z$的相似度。举个例子，介绍我们选择如下的$K$：

$$K(x,z)=\exp\left(-\frac{\left\lVert x-z\right\rVert^2}{2\sigma^2}\right)$$

在这个例子中，$K$可以得到一个合理的关于$x$与$z$的相似度，当$x$和$z$靠近时$K$接近$1$，当$x$和$z$分开时$K$接近$0$。那么，我们可以使用这个$K$作为支持向量机的核吗？在本例中，答案是肯定的。（这个核也叫**高斯核（Gaussian kernel：[中文](https://zh.wikipedia.org/wiki/%E5%BE%84%E5%90%91%E5%9F%BA%E5%87%BD%E6%95%B0%E6%A0%B8)，[英文](https://en.wikipedia.org/wiki/Radial_basis_function_kernel)）**，相应的$\phi$是一个无限维的特征映射）。但是更一般的，对于一个给定的$K$，我们怎么才能知道它是否是一个有效的核？也就是说，对于一个$K$，怎样判断是否存在一个特征映射$\phi$对于所有的$x,z$，能够使得$K(x,z)=\phi(x)^T\phi(z)$？（即判断$\exists\phi\ \mathrm{s.t.}\ K(x,z)=\left\langle\phi(x),\phi(z)\right\rangle$是否成立。）

假设对于某个$\phi$，我们已经有了一个有效核$K$，现在我们有一个有限集合$\left\{x^{(1)},\cdots,x^{(m)}\right\}$（不一定是训练集，可以是任意一个包含$m$个点的集合），含有$m$个点，然后在构造一个$m$阶方阵$K$，它的第$(i,j)$个元素定义为$K_{ij}=K(x^{(i)},x^{(j)})$，我们把这个矩阵成为**核矩阵（kernel matrix）**。注意到我们重复使用了记号$K$，既用来表示核函数$K(x,z)$，又用来表示核矩阵$K$，因为这两个概念很接近。

那么，如果$K$是一个有效的核，则$K_{ij}=K\left(x^{(i)},x^{(j)}\right)=\phi\left(x^{(i)}\right)^T\phi\left(x^{(j)}\right)=\phi\left(x^{(j)}\right)^T\phi\left(x^{(i)}\right)=K\left(x^{(j)},x^{(i)}\right)=K_{ji}$，即$K$为对称矩阵。还有，如果我们使用$\phi_k(x)$表示向量$\phi(x)$的第$k$个元素，则对于任意向量$z$都有：

$$\begin{align}z^TKz&=\sum_{i}\sum_{j}z_iK_{ij}z_j\\&=\sum_i\sum_jz_i\phi\left(x^{(i)}\right)^T\phi\left(x^{(j)}\right)z_j\\&=\sum_i\sum_jz_i\left(\sum_k\phi_k\left(x^{(i)}\right)\phi_k\left(x^{(j)}\right)\right)z_j\\&=\sum_k\sum_i\sum_jz_i\phi_k\left(x^{(i)}\right)\phi_k\left(x^{(j)}\right)z_j\\&=\sum_k\underbrace{\left(\sum_iz_i\phi_k\left(x^{(i)}\right)\right)}_{z^T\phi_k(x)}\underbrace{\left(\sum_jz_j\phi_k\left(x^{(j)}\right)\right)}_{z^T\phi_k(x)}\\&=\sum_k\left(\sum_iz_i\phi_k\left(x^{(i)}\right)\right)^2\\&\geq0\end{align}$$

由于$z$是任意向量，所以$K$是一个半正定矩阵（$K\geq0$）。（倒数第三步的证明可以在[问题集2](http://cs229.stanford.edu/materials/ps2.pdf)问题1中找到思路。）

于是，我们证明了如果$K$是有效的核（即存在相应的特征映射$\phi$），则相应的核矩阵$K\in\mathbb{R}^{m\times m}$是半正定对称矩阵。一般的，可以证明这个条件也是“$K$是一个有效核”的充要条件。接下来的结论基于**Mercer定理（Mercer’s theorem）**。（很多教材通过引入$L^2$函数介绍Mercer定理，略显复杂，因为如果输入属性仅在$\mathbb{R}^n$上取值，与我们接下来给出的结论是等价的。）

**定理（Mercer）**：定义$K:\mathbb{R}^n\times\mathbb{R}^n\to\mathbb{R}^n$，则$K$是有效的（Mercer）核的充要条件是：对于任意$\{x^{(1)},\cdots,x^{(m)}\},\ (m\lt\infty)$，$K$相对应的核矩阵为半正定对称矩阵。

对于给定的函数$K$，除了寻找相应的特征映射$\phi$外，Mercer定理提供了另一种测试核有效性的方法。（在[问题集2](http://cs229.stanford.edu/materials/ps2.pdf)中可以深入理解这个概念。）

为什么我们要将输入属性转变为输入特征，将其从低维转换成高维呢？回忆一下我们为什么要引出支持向量机——我们引出支持向量机是为了解决非线性学习问题，来看下图（来自一篇[介绍核方法的博客](http://blog.csdn.net/xianlingmao/article/details/7719122)）：

比如说我们的原始数据如左图所示，一个一维的输入属性，而核的作用是将原始输入属性映射到高维特征空间，即右图所示，然后我们在高维特征空间运行支持向量机，就可以找到最优间隔分类器（即分类器将高维空间的的数据分隔开来并使几何间隔最大化）。在本例中，原始数据不是线性可分隔的，当时将它映射在高维空间中，数据就变成线性可分隔的了，于是我们就可以使用线性分类器对原始空间中并非线性可分隔的数据进行分类了。这就是支持向量机输出非线性决策边界的整个过程。（[维基百科关于核方法的解释](https://en.wikipedia.org/wiki/Kernel_method)。）

我们在讲解中还会简要的介绍一些关于其他核的例子。比如，考虑数字识别问题，输入是一张手写的数字（数字$0-9$）图片（$16\times16$像素），我们需要预测图片里的数字到底是几。我们可以使用一个简单多项核$K(x,z)=\left(x^Tz\right)^d$，也可以使用高斯核，而支持向量机将在这种问题上大显身手。令人惊奇的是，因为输入是一个关于图像像素亮度的$256$维向量，而我们的模型对视觉并没有先验知识，它甚至不知道哪些像素是相邻的。

再举一个例子，我们试图分类的对象$x$是一个字符串（假设，它是一个氨基酸列表，连在一起能够组成蛋白质），看上去很难构造一个对于其他学习算法大小较为合理（其他算法通常要求特征值向量不宜过大）的特征值集合，尤其是当不同的字符串长短不一时。令$\phi(x)$为统计$x$中长度为$k$的子字符串出现次数的特征值向量，如果考虑英文字母，则共有$26^k$种子字符串。$\phi(x)$是一个$26^k$维向量，即使再有节制的取$k$，也很难保证我们高效的处理如此高维度的向量。（比如只取$4$位就有$26^4\approx460,000$。）然而使用（动态规划的）字符匹配算法，就可以高效的计算$K(x,z)=\phi(x)^T\phi(x)$，毫无疑问的，现在已经可以处理$26^k$维度的问题了，而且并不需要直接计算出该空间中的特征值向量。

我们已经讨论了将核方法应用于支持向量机的过程，一定要了解的是，核方法的适用性比支持向量机更广泛。特别是当遇到只用输入属性向量的内积$\left\langle x,z\right\rangle$就可以表示的学习算法时，如果我们使用核方法$K(x,z)$代替内积，就会“神奇”的发现现在的算法可以有效的处理$K$相关的维度极高的特征空间了。比如，将核方法用在感知算法上，就可以得到核感知算法。我们在后面将要介绍的很多算法都可以使用核方法处理，这个过程也叫作“核技巧（kernel trick）”。

## 8. 正则化及不可分隔情形

到目前为止，我们都是在数据线性可分隔的假设下讨论支持向量机算法的。通过$\phi$将数据映射到更高纬度的特征空间通常能够增加数据可分隔的概率，但是我们并不能保证总是如此。受可能存在的离群值的影响，在某些情形下，我们并不能直接用算法确定分类超平面。比如下图的情形：

左图为最优间隔分类器，当我们在左上角加入了一个离群值后，如右图所示，它引起了判别边界剧的烈变动，而新的判别边界的间隔变小了不少。

为了使算法在面对线性不可分隔的数据集时有效，同时也为了降低算法对离群值的敏感度，我们重新定义了如下的优化目标（使用$\mathscr{l}_1$**正则化（regularization）**，也叫$\mathscr{l}_1$ norm soft margin SVM）：

$$\begin{align}\operatorname*{min}_{w,b}&\quad\frac{1}{2}\lVert w\rVert^2+C\sum_{i=1}^m\xi_i\\\mathrm{s.t.}&\quad y^{(i)}\left(w^Tx^{(i)}+b\right)\geq1-\xi_i,\quad i=1,\cdots,m\\&\quad\xi_i\geq0,\quad i=1,\cdots,m\end{align}$$

可以看到，现在我们允许样本的（函数）间隔小于$1$（甚至小于零，函数间隔小于零也就意味着产生了分类错误，我们允许这种情况的发生），而每当样本的函数间隔为$1-\xi_i,\ \xi\gt0$时，我们就给目标函数（总体）加上大小为$C\xi_i$的惩罚开销（也就是说，我们并不鼓励减小间隔，更不鼓励分类错误，但是这是被允许的）。参数$C$用来控制我们的两个目标之间的权重，一是减小$\lVert w\rVert^2$（我们之前看到这样可以增大间隔），二是保证大部分样本的函数间隔至少为$1$。

和以前一样，我们需要重新构造拉格朗日算子：

$$\mathcal{L}(w,b,\xi,\alpha,r)=\frac{1}{2}w^Tw+C\sum_{i=1}^m\xi_i-\sum_{i=1}^m\alpha_i\left[y^{(i)}\left(x^Tw+b\right)-1+\xi_i\right]-\sum_{i=1}^mr_i\xi_i$$

其中$\alpha_i,r_i$是拉格朗日乘数（均为非负实数）。这里我们就不再详细写出对偶问题的推导过程了，在分别计算其关于$w$与$b$的偏导数后将偏导数置为零，再将结果代回拉格朗日算子并化简，得到如下对偶问题：

$$\begin{align}\displaystyle\operatorname*{max}_{\alpha}&\quad W(\alpha)=\sum_{i=1}^m\alpha_i-\frac{1}{2}\sum_{i,j=1}^my^{(i)}y^{(j)}\alpha_i\alpha_j\left\langle x^{(i)},x^{(j)}\right\rangle\\\mathrm{s.t.}&\quad0\leq\alpha_i\leq C,\quad i=1,\cdots,m\\&\quad\sum_{i=1}^m\alpha_iy^{(i)}=0\end{align}$$

参见上一讲的$(9)$式，我们可以将$w$写为关于$\alpha$的函数，在计算出对偶问题的解之后，可以使用上一讲的$(13)$式对新样本进行预测。有趣的是，加入了$\mathscr{l}_1$正则化后，唯一的改变就是约束条件从原来的$0\leq\alpha_i$变为现在的$0\leq\alpha_i\leq C$。$b^*$的计算也发生了改变（上一讲的$(11)$式不再有效，参考下一节的讲解及Platt的文章）。

由原来的KKT对偶互补条件推导出的新的KKT对偶互补条件（在下一节中也会用于测试序列最小优化算法的收敛情况）：

$$\begin{align}\alpha_i=0&\implies y^{(i)}\left(w^Tx^{(i)}+b\right)\geq1\tag{14}\\\alpha_i=C&\implies y^{(i)}\left(w^Tx^{(i)}+b\right)\leq1\tag{15}\\0\lt\alpha_i\lt C&\implies y^{(i)}\left(w^Tx^{(i)}+b\right)=1\tag{16}\end{align}$$

万事俱备，接下来我们就缺一个能够直接求解对偶问题的算法了，下一节就来介绍这个算法。

## 9. 序列最小优化算法（SMO:  sequential minimal optimization）

序列最小优化算法来源于John Platt，他找到了一种高效解决“由支持向量机引出的对偶问题”的算法。在继续介绍该算法之前，我们先来看一下坐标下降/上升算法。

### 9.1 坐标下降/上升法

考虑如何解决一个不加约束条件的优化问题：

$$\operatorname*{max}_\alpha W\left(\alpha_1,\alpha_2,\cdots,\alpha_m\right)$$

现在，我们只将$W$看做关于$\alpha_i$的函数，与支持向量机无关。到目前为止我们已经了解了两种优化算法：梯度下降/上升法，牛顿法，这里我们再来介绍一种新的优化算法——**坐标下降/上升法（coordinate descent/ascent algorithm，[中文](https://zh.wikipedia.org/wiki/%E5%9D%90%E6%A0%87%E4%B8%8B%E9%99%8D%E6%B3%95)，[英文](https://en.wikipedia.org/wiki/Coordinate_descent)）**：

```
Loop until convergence {
    for i=1 to m, {
```
$\quad\qquad\quad\quad\displaystyle\alpha_i:=\mathrm{arg}\operatorname*{max}_{\hat\alpha_i}W\left(\alpha_1,\cdots,\alpha_{i-1},\hat\alpha_i,\alpha_{i+1},\cdots,\alpha_m\right)$
```
    }
}
```

在算法里面一层循环中，我们会固定除$\alpha_i$以外的参数，然后重新优化这个关于$\alpha_i$的函数$W$。对上面的这个例子，里面的循环会按照$\alpha_1,\alpha_2,\cdots,\alpha_m,\alpha_1,\alpha_2,\cdots$的顺序进行优化。（对一些复杂的问题可能会使用别的顺序，比如，我们可以看哪个参数会带来$W(\alpha)$最大幅度的增长，就选择哪个参数作为下一个优化的对象。）

而当函数$W$恰好符合里面一层循环可以高效运行的状态时（事实上有很多优化问题很容易固定其他参数只对一个参数求最优值，在这种情况下本算法的内层循环将会执行的非常快。而支持向量机通常也属于这种情况），坐标下降/上升法确实是一个效率很高的算法，如图所示：

这是一个需要优化的二次函数的等高线图，坐标上升算法初始值取在$(2,-2)$，图中的线就是坐标上升算法从初始值到全局最大值的优化路径。注意到算法的每一步都是沿着坐标轴方向优化，因为算法每次只优化一个参数。

### 9.2 序列最小优化

我们通过简要的介绍序列最小优化来完结支持向量机算法，推导过程中将有一些细节被省略。

我们再一次列出（对偶）优化问题：

$$\begin{align}\displaystyle\operatorname*{max}_{\alpha}&\quad W(\alpha)=\sum_{i=1}^m\alpha_i-\frac{1}{2}\sum_{i,j=1}^my^{(i)}y^{(j)}\alpha_i\alpha_j\left\langle x^{(i)},x^{(j)}\right\rangle\tag{17}\\\mathrm{s.t.}&\quad0\leq\alpha_i\leq C,\quad i=1,\cdots,m\tag{18}\\&\quad\sum_{i=1}^m\alpha_iy^{(i)}=0\tag{19}\end{align}$$

假设我们已经有一组满足约束条件$(18-19)$式的$\alpha_i$，现在，我们固定$\alpha_2,\cdots,\alpha_m$，然后进行坐标上升优化步骤，重新优化关于$\alpha_1$的目标函数，那么我们能够得到更加优化的目标函数吗？答案是否定的，因为$(19)$式：

$$\alpha_1y^{(1)}=-\sum_{i=2}^m\alpha_iy^{(i)}$$

或者上式两边同时乘以$y^{(1)}$有：

$$\alpha_1=-y^{(1)}\sum_{i=2}^m\alpha_iy^{(i)}$$

（因为$y^{(1)}\in\{-1,1\}$，所以$\left(y^{(1)}\right)^2=1$。）我们发现，$\alpha_1$直接可以由其余的$\alpha_i$确定，所以如果我们固定$\alpha_2,\cdots,\alpha_m$，在不违反约束条件$(19)$的前提下，我们根本不能改变$\alpha_1$。

如果我们想要更新某个$\alpha_i$，我们就至少得同时更新两个参数以保证算法仍在约束条件内。这就是序列最小优化算法（最小就是指一次改变最少个$\alpha_i$，在这里即两个）的思路：

Repeat till convergence {
1. 选择一对参数$\alpha_i,\alpha_j$作为将要优化的对象（启发式（也就是经验法则）的找到两个可以使优化目标朝全局最大值最快收敛的参数）。
2. 优化关于$\alpha_i,\alpha_j$的函数$W(\alpha)$，也就是将其余的$\alpha_k,k\neq i,j$。

}

为了测试算法的收敛性，我们可以检测算法是否满足带有某$tol$的KKT条件（即$(14-16)$式）。其中$tol$是收敛容差参数（convergence tolerance parameter），通常设为$0.01$到$0.001$。

序列最小优化是一个高效算法的关键在于对$\alpha_i,\alpha_i$的计算效率非常高（它可能比牛顿法迭代的步骤更多，但是它每次迭代的代价非常小）。我们来简要推导一下这种高效的参数更新。

假设我们有一组满足约束条件$(18-19)$的$\alpha_i$，现在将参数$\alpha_3,\cdots,\alpha_m$固定，然后优化关于$\alpha_1,\alpha_2$的函数$W(\alpha_1,\alpha_2,\cdots,\alpha_m)$（在约束条件下）。根据$(19)$式得到：

$$\alpha_1y^{(1)}+\alpha^{(2)}=-\sum_{i=3}^m\alpha_iy^{(i)}$$

等式右边是固定的（因为我们固定了参数$\alpha_3,\cdots,\alpha_m$），我们把右边记作常量$\zeta$：

$$\alpha_1y^{(1)}+\alpha^{(2)}=\zeta\tag{20}$$

我们可以用下图表示$\alpha_1,\alpha_2$上的约束条件：

从$(18)$得知$\alpha_1,\alpha_2$必须落在图中$[0,C]\times[0,C]$的区域内，图中的斜线为$\alpha_1y^{(1)}+\alpha^{(2)}=\zeta$，即$\alpha_1,\alpha_2$所在直线。从约束条件我们还能得到$L\leq\alpha_2\leq H$；否则$(\alpha_1,\alpha_2)$就不能同时满足图中的区域约束和直线约束了。在本例中$L=0$。但并不总是如此，这需要参考直线$\alpha_1y^{(1)}+\alpha^{(2)}=\zeta$。通常，$\alpha_2$可能会有更低的下限$L$和更高的上限$H$，但总会保证$\alpha_1,\alpha_2$在$[0,C]\times[0,C]$的区域内。

通过$(20)$式，我们可以将$\alpha_1$表示成关于$\alpha_2$的函数：

$$\alpha_1=\left(\zeta-\alpha_2y^{(i)}\right)y^{(1)}$$

（我们又一次使用了由$y^{(1)}\in\{-1,1\}$推出$\left(y^{(1)}\right)^2=1$的技巧。）因此$W(\alpha)$可以写为：

$$W(\alpha_1,\alpha_2,\cdots,\alpha_m)=W\left(\left(\zeta-\alpha^{(2)}y^{(2)}\right)y^{(1)},\alpha_2,\cdots,\alpha_m\right)$$

将$\alpha_3,\cdots,\alpha_m$当做常数，易知这就是一个关于$\alpha_2$的二次函数。也就是说可以将$W$看做$a\alpha_2^2+b\alpha_2+c$（选用合适的$a,b,c$作为系数）。如果忽略$(18)$式的区域约束条件（或等价的$L\leq\alpha_2\leq H$），则我们可以通过将这个二次函数的导数置为零的方法，求出最大值。我们使用$\alpha_2^{new,unclipped}$标记解出的$\alpha_2$。如果我们服从$(18)$式的约束条件，想要最大化$W$关于$\alpha_2$的函数时，只需要简单的将$\alpha_2^{new,unclipped}$“剪入”$[L,H]$的区间即可：

$$\alpha_2^{new}=\begin{cases}H&if\ \alpha_2^{new,unclipped}\gt H\\\alpha_2^{new,unclipped}&if\ L\leq\alpha_2^{new,unclipped}\leq H\\L&if\ \alpha_2^{new,unclipped}\lt L\end{cases}$$

当我们求出$\alpha_2^{new}$之后，代回$(20)$式即可得到$\alpha_1^{new}$的优化值。

这个算法还有一些别的细节，可以参考Platt的文章：“ the choice of the heuristics used to select the next $\alpha_i$, $\alpha_j$ to update”和“how to update $b$ as the SMO algorithm is run”。


# Lecture 11: Introduction to Multiobjective Optimization

## What means multiobjective?

* Consider several criteria simultaneously

* Criteria are conflicting (e.g. usually good quality is not cheap) $\Rightarrow$ all the criteria can not be optimized simultaneously

* Need for considering compromises between the criteria

* Compromise can be better than optimal solution in practice (e.g., optimize only costs/profit)

* http://en.wikipedia.org/wiki/Multiobjective_optimization

# Multiobjective optimization process

# Example 1: Continuous casting of steel

* Optimal control of the secondary cooling of continuous casting of steel
* Long history of research in the Dept. of Mathematical Information Technology, Univ. of Jyväskylä
  * modelling (1988)
  * single objective optimization (1988-1994)
  * multiobjective optimization (1994-1998)

* Liquid steel enters (tundish)
* Initial cooling by a water cooled mold $\Rightarrow$ thin crust
* Movement supported by rollers
* Secondary cooling by water sprays
* Rest of the cooling by radiation

## Modelling
* Measuring temperature in casting difficult $\Rightarrow$ numerical temperature distribution
* Process modelled as a multiphase heat equation (PDEs, solid & liquid phase) $\Rightarrow$  temperature distribution
* Numerical model by using the finite element method (FEM)
* Dynamic process

## Single-objective optimization

* Secondary cooling significant: intensity of sprays (easy to control) affects significantly to the solidification of steel
* Goal: minimize the amount of defects in steel
* Quality depends on e.g. the temperature distribution at the surface of steel
  * too slow cooling → too long liquid part
  * too fast cooling → defects appear
* Objective function: keep the surface temperature as close to a given profile as possible
* Constraints e.g. for the change of temperature and for the temperature in critical spots

## Single-objective optimization results

* Analysis of single objective optimization problem:
  * constraints tight (based on metallurgy)
  * no feasible solutions
* which constraints to relax?
* $\Rightarrow$ Convert constraints into objective functions      (5 in total)
  * Enables simultaneous relaxation of different constraints
  * Gives information on satisfaction of different constraints and their interdependences

## Multiobjective optimization results

** Found a solution, which only violated one of the constraints**

$\Rightarrow$ satisfactory to the application expert

## Example 2: Data-Based Forest Management with Uncertainties and Multiple Objectives

* A recent study by Profs Annika Kangas (Forest research institute, Joensuu) and Kaisa Miettinen, PhD Kyle Eyvindson (University of Helsinki) and myself
* Main research question: How to use multiobjective optimization to handle in forest management simultaneously
  * conflicting objectives income and biodiversity, and
  * uncertainties.

### Main problems

* **Harvesting implies incomes from forests**, but on the other hand, it **diminishes the recreational and esthetical values of the forest**, and it may have **adverse effects on the natural values of the area**, for instance, the biodiversity within the area and the viability of wildlife populations living in the area.

* On the other hand, forest management planning **involves lots of uncertainty**. It is **not possible to measure all trees** within a forest area, so there is **uncertainty concerning the current state** of the forests. **Measuring biodiversity is prohibitively expensive**, so using **proxy variables** (biodiversity indices) is the only possibility.

* All forest decisions **concern the future** (typically the next 5-20 years), so that the **state of the forest stands and biodiversity and the consequences of the treatment options needs to be predicted** using statistical models. As we do not know the exact consequences of the decisions, the **decisions involve uncertainty which the decision makers may wish to manage.**

## Our approach

* Model the decision problem as a six-objective optimization problem
* Model uncertainty using scenario based-approach
  * Different scenarios represent possible future states
* Handle the uncertainty using a well known risk measure called "value-at-risk" with different risk levels
* Objectives to be maximized
  1. Expected minimum (over all time periods) income in all the scenarios.
  2. Expected minimum (over all time periods) biodiversity in all the scenarios
  3. Value of the biodiversity at the given risk
  4. Value of the income at the given risk
  5. Probability of the income being greater than the value-at-risk.  The risk level for income is, thus, one minus this probability.
  6. Probability of the biodiversity being greater than the value-at-risk.  The risk level for biodiversity is, thus, one minus this probability.

## Multiobjective optimization results

* The decision maker was able to study interdependencies between income and biodiversity, gains and losses in income and biodiversity at different risk levels and risk levels associated to these
* The results will be submitted to KDD2016 conference for review the end of this week
* I will *hopefully* present the results at the KDD2016 conference in August in San Francisco, USA

## Mathematical formulation of multiobjective optimization problems

Multiobjective optimization problems are often formulated as
$$
\begin{align} \
\min \quad &\{f_1(x),\ldots,f_k(x)\}\\
\text{s.t.} \quad & g_j(x) \geq 0\text{ for all }j=1,\ldots,J\\
& h_k(x) = 0\text{ for all }k=1,\ldots,K\\
&a_i\leq x_i\leq b_i\text{ for all } i=1,\ldots,n\\
&x\in \mathbb R^n,
\end{align}
$$
where $$f_1,\ldots,f_k:\{x\in\mathbb R^n: g_j(x) \geq 0 \text{ for all }j=1,\ldots,J \text{ and } h_k(x) = 0\text{ for all }k=1,\ldots,K\}\mapsto\mathbb R$$ are the objective functions.

## Basic concepts

Basic concepts of solution, feasible solution, equality and inequality constraints are the same.

However, the concept of optimality gets replaced by the concept of **Pareto optimality**. A feasible solution $x^1$ is Pareto optimal to the multiobjective optimization problem, if there does not exist a feasible solution $x^2$ such that
$$
\left\{
\begin{align}
&f_i(x^2)\leq f_i(x^1)\text{ for all }i\in \{1,\ldots,k\}\\
&f_j(x^2)<f_j(x^1)\text{ for some }j\in \{1,\ldots,k\}.\\
\end{align}
\right.
$$

There are now two spaces connected to the problem: the space $\mathbb R^n$ is called the decision space and $\mathbb R^k$ is called the objective space.

** Usually there exist multiple Pareto optimal solutions to a problem. However, the most preferred one needs to be selected for implementation. Thus, a decision maker is needed.**

In addition to Pareto optimality, two more concepts are important, which are called the ideal and the nadir vector. Mathematically the ideal vector $z^{ideal}$ can be defined as having
$$
z^{ideal}_i = \begin{align} \
\min \quad &f_i(x)\\
\text{s.t.} \quad &x\text{ is feasible}
\end{align}
$$
for all $i=1,\ldots,k$. The nadir vector $z^{nadir}$ on the other hand has
$$
z^{nadir}_i =
\begin{align}
\max \quad &f_i(x)\\
\text{s.t.} \quad &x\text{ is Pareto optimal},
\end{align}
$$
for all $i=1,\ldots,k$.

## Example

Consider multiobjective optimization problem
$$
\min \{x^2+y,1-x\}\\
\text{s.t. }x\in[0,1], y\geq0.
$$
#### Pareto optimal solutions
Now, the set of Pareto optimal solutions is
$$
\{(x,0):x\in[0,1]\}.
$$
Let's show that $(x',0)$ is Pareto optimal for all $x'\in[0,1]$. Let's assume $(x,y)$ with $x\in[0,1]$ and $y\geq0$ such that
$$
\left\{
\begin{align}
x^2+y\leq x'^2,\text{ and}\\
1-x\leq 1-x'.
\end{align}
\right.
$$
and
$$
\left\{
\begin{align}
x^2+y< x'^2 \text{ or}\\
1-x< 1-x'.
\end{align}
\right.
$$
Second inequality in the first system of inequalities gives $x\geq x'$. This yields from the first inequality in that same system of inequalities
$$
y\leq x'^2-x^2\leq 0.
$$
Thus, $y=0$. This means that $x=x'$ using again the first inequality.

This means that the solution cannot satisfy the second system of strict inequalities.

Let's assume a solution $(x,y)$, where $x\in[0,1]$ and $y>0$ and show that this is not Pareto optimal:

By choosign solution $(x,0)$, we have
$$
\left\{
\begin{align}
x^2<x^2+y ,\text{ and}\\
1-x\leq 1-x.
\end{align}
\right.
$$
Thus, the solution cannot be Pareto optimal.

#### Ideal
Now

$$
\begin{align}
\min x^2+y\\
\text{s.t. }x\in[0,1],\ y\geq0
\end{align}
= 0
$$
and
$$
\begin{align}
\min 1-x\\
\text{s.t. }x\in[0,1],\ y\geq0
\end{align}
= 0.
$$

Thus, the ideal is
$$
z^{ideal} = (0,0)^T
$$
#### Nadir
Now,
$$
\begin{align}
\max x^2+y\\
\text{s.t. }x\in[0,1],\ y=0
\end{align}
= 1
$$
and
$$
\begin{align}
\max 1-x\\
\text{s.t. }x\in[0,1],\ y=0
\end{align}
= 1.
$$

Thus,
$$
z^{nadir}=(1,1)^T.
$$

Lecture 11, Introduction to multiobjective optimization.ipynb


```julia
using QuantumSpin
versioninfo()
```

    Julia Version 1.5.0
    Commit 96786e22cc (2020-08-01 23:44 UTC)
    Platform Info:
      OS: Windows (x86_64-w64-mingw32)
      CPU: Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
      WORD_SIZE: 64
      LIBM: libopenlibm
      LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
    Environment:
      JULIA_DEPOT_PATH = C:\Users\shopp\.julia;C:\Users\shopp\AppData\Local\JuliaPro-1.5.0-1\Julia-1.5.0\local\share\julia;C:\Users\shopp\AppData\Local\JuliaPro-1.5.0-1\Julia-1.5.0\share\julia
      JULIA_PKG_SERVER = pkg.juliahub.com

# Tutorial

We consider a quantum spin system whose hamiltonian is given by

\begin{align}
H = \sum_{i>j} \left(
    J^x_{ij} \sigma_i^x \sigma_j^x
    + J^y_{ij} \sigma_i^y \sigma_j^y
    + J^z_{ij} \sigma_i^z \sigma_j^z
    \right)
    +
    \sum_{i} \left(
    Γ^x_i \sigma_i^x
    + Γ^y_i \sigma_i^y
    + Γ^z_i \sigma_i^z
    \right).
\end{align}

Note that the coefficients in spin-spin interaction term spedify not only the strength of the interaction but also the dimension and toplogy of the lattice.
In the language of the graph theory, $J_{ij}^\alpha$ is called [adjacency matrix](https://en.wikipedia.org/wiki/Adjacency_matrix).
For instance, the adjacency matrix of the 1D chain with periodic boundary consition is given by

\begin{align}
\begin{bmatrix}
0 &1  &  &  &  1\\
1& 0 &1  &  &  \\
&1  & 0 &1  &  \\
&  & {\ddots} & {\ddots} &1  \\
1&  &  & 1 & 0
\end{bmatrix}.
\end{align}

In `QuantumSpin` package, several adjacency matrices are availabe:

```julia
Achain = lattice(:chain, 4)
```

    4×4 Array{Int64,2}:
     0  1  0  1
     1  0  1  0
     0  1  0  1
     1  0  1  0

```julia
A□ = lattice(:square, 9)
```

    9×9 Array{Int64,2}:
     0  1  1  1  0  0  1  0  0
     1  0  1  0  1  0  0  1  0
     1  1  0  0  0  1  0  0  1
     1  0  0  0  1  1  1  0  0
     0  1  0  1  0  1  0  1  0
     0  0  1  1  1  0  0  0  1
     1  0  0  1  0  0  0  1  1
     0  1  0  0  1  0  1  0  1
     0  0  1  0  0  1  1  1  0

 Corresonding Ising Hamiltonian

\begin{align}
H = \sum_{<i,j>} \sigma_i^z \sigma_j^z.
\end{align}

is obtained as

```julia
O = zeros(4,4)
Ov = zeros(4)
hamiltonian(O, O, Achain, Ov, Ov, Ov)
```

    16×16 SparseArrays.SparseMatrixCSC{Complex{Float64},Int64} with 4 stored entries:
      [1 ,  1]  =  4.0+0.0im
      [6 ,  6]  =  -4.0+0.0im
      [11, 11]  =  -4.0+0.0im
      [16, 16]  =  4.0+0.0im

```julia
O = zeros(9,9)
Ov = zeros(9)
hamiltonian(O, O, A□, Ov, Ov, Ov)
```

    512×512 SparseArrays.SparseMatrixCSC{Complex{Float64},Int64} with 512 stored entries:
      [1  ,   1]  =  18.0+0.0im
      [2  ,   2]  =  10.0+0.0im
      [3  ,   3]  =  10.0+0.0im
      [4  ,   4]  =  6.0+0.0im
      [5  ,   5]  =  10.0+0.0im
      [6  ,   6]  =  6.0+0.0im
      [7  ,   7]  =  6.0+0.0im
      [8  ,   8]  =  6.0+0.0im
      [9  ,   9]  =  10.0+0.0im
      [10 ,  10]  =  6.0+0.0im
      [11 ,  11]  =  2.0+0.0im
      [12 ,  12]  =  2.0+0.0im
      ⋮
      [500, 500]  =  2.0+0.0im
      [501, 501]  =  2.0+0.0im
      [502, 502]  =  2.0+0.0im
      [503, 503]  =  6.0+0.0im
      [504, 504]  =  10.0+0.0im
      [505, 505]  =  6.0+0.0im
      [506, 506]  =  6.0+0.0im
      [507, 507]  =  6.0+0.0im
      [508, 508]  =  10.0+0.0im
      [509, 509]  =  6.0+0.0im
      [510, 510]  =  10.0+0.0im
      [511, 511]  =  10.0+0.0im
      [512, 512]  =  18.0+0.0im

Heisenberg chain under an longitudinal magnetic field

\begin{align}
H = \sum_{<i,j>} \left(
    J^x \sigma_i^x \sigma_j^x
    + J^y \sigma_i^y \sigma_j^y
    + J^z \sigma_i^z \sigma_j^z
    \right)
    +
    \sum_{i} Γ^z \sigma_i^z
\end{align}

is given by

```julia
Jˣ, Jʸ, Jᶻ, Γᶻ = 1.0, 1.0, 0.5, 1.0
ov = zeros(4)
hamiltonian(Jˣ*Achain, Jʸ*Achain, Jᶻ*Achain, ov, ov, Γᶻ*ones(4))
```

    16×16 SparseArrays.SparseMatrixCSC{Complex{Float64},Int64} with 44 stored entries:
      [1 ,  1]  =  6.0+0.0im
      [2 ,  2]  =  2.0+0.0im
      [3 ,  2]  =  2.0+0.0im
      [9 ,  2]  =  2.0+0.0im
      [2 ,  3]  =  2.0+0.0im
      [3 ,  3]  =  2.0+0.0im
      [5 ,  3]  =  2.0+0.0im
      [6 ,  4]  =  2.0+0.0im
      [11,  4]  =  2.0+0.0im
      [3 ,  5]  =  2.0+0.0im
      [5 ,  5]  =  2.0+0.0im
      [9 ,  5]  =  2.0+0.0im
      ⋮
      [13, 11]  =  2.0+0.0im
      [8 , 12]  =  2.0+0.0im
      [12, 12]  =  -2.0+0.0im
      [14, 12]  =  2.0+0.0im
      [6 , 13]  =  2.0+0.0im
      [11, 13]  =  2.0+0.0im
      [12, 14]  =  2.0+0.0im
      [14, 14]  =  -2.0+0.0im
      [15, 14]  =  2.0+0.0im
      [8 , 15]  =  2.0+0.0im
      [14, 15]  =  2.0+0.0im
      [15, 15]  =  -2.0+0.0im
      [16, 16]  =  -2.0+0.0im

We note that the adjacency matrix must be symmetric, otherwise you will get an error:

```julia
Acustom = [
    0 1 1 1
    1 0 0 0
    0 0 0 0
    0 1 0 0
]
O = zeros(4,4)
Ov = zeros(4)
hamiltonian(O, O, Acustom, Ov, Ov, Ov)
```

```julia

```


```python
import ipywidgets as widgets
import matplotlib.pyplot as plt
import numpy as np
from scipy import integrate
import sympy as sym

import models
import selection_functions
import symbolics
```

```python
%matplotlib inline
plt.style.use("ggplot")
```

# Monomorphic $\gamma$ model

## Only G allele present

Set $x_2=1-x_1$, $x_3=x4=0$ so that only the G allele of the $\gamma$ gene is present

```python
x1, x2, x3 = sym.symbols("x1, x2, x3", real=True, nonnegative=True)
x4 = 1 - x1 - x2 - x3
T, R, P, S = sym.symbols('T, R, P, S', real=True, positive=True)
M, m = sym.symbols("M, m", real=True, nonnegative=True)
epsilon = sym.symbols("epsilon", real=True, positive=True)

UGA = symbolics.UGA
UgA = symbolics.UgA
```

```python
x = np.array([[x1], [x2], [x3], [x4]])
payoff_kernel = np.array([[R, S], [T, P]])
W = models.generalized_sexual_selection(x, UGA, UgA, payoff_kernel, M, m, epsilon)
```

```python
male_genotype_fitness = W.sum(axis=0).sum(axis=0)
```

```python
male_GA_fitness = sym.together(sym.expand(male_genotype_fitness[0]))
```

```python
male_GA_fitness
```

    4.0*x1*(-M*UGA(x1 + x3) + R*UGA(x1 + x3)**2 + R*UgA(x1 + x3)**2 - S*UGA(x1 + x3)**2 + S*UGA(x1 + x3) - S*UgA(x1 + x3)**2 + S*UgA(x1 + x3) - m*UgA(x1 + x3))/(x1 + x3)

Fitness of males carrying the $GA$ genotype.

$$ 4\big((R - S)U_{GA}(x_1)^2 + (S - M)U_{GA}(x_1)\big) + 4\big((R - S)U_{gA}(x_1)^2 + (S - m)U_{gA}(x_1)\big) $$

```python
male_Ga_fitness = sym.together(sym.expand(male_genotype_fitness[1]))
```

```python
male_Ga_fitness
```

    x2*(4.0*M*UGA(x1 + x3) - 4.0*M + 4.0*P*UGA(x1 + x3)**2 - 8.0*P*UGA(x1 + x3) + 4.0*P*UgA(x1 + x3)**2 - 8.0*P*UgA(x1 + x3) + 8.0*P - 4.0*T*UGA(x1 + x3)**2 + 4.0*T*UGA(x1 + x3) - 4.0*T*UgA(x1 + x3)**2 + 4.0*T*UgA(x1 + x3) + 4.0*m*UgA(x1 + x3) - 4.0*m)/(-x1 - x3 + 1)

Fitness of males carrying the $Ga$ genotype.

$$ 4\big(2P - (M + m)\big) + 4\big((T + M - 2P)U_{GA}(x_1) - (T - P)U_{GA}(x_1)^2\big) + 4\big((T + m - 2P)U_{gA}(x_1) - (T - P)U_{gA}(x_1)^2\big) $$

```python
sym.collect(male_GA_fitness - male_Ga_fitness, (UGA(x1), UgA(x1)))
```

    4.0*x1*(-M*UGA(x1 + x3) + R*UGA(x1 + x3)**2 + R*UgA(x1 + x3)**2 - S*UGA(x1 + x3)**2 + S*UGA(x1 + x3) - S*UgA(x1 + x3)**2 + S*UgA(x1 + x3) - m*UgA(x1 + x3))/(x1 + x3) - x2*(4.0*M*UGA(x1 + x3) - 4.0*M + 4.0*P*UGA(x1 + x3)**2 - 8.0*P*UGA(x1 + x3) + 4.0*P*UgA(x1 + x3)**2 - 8.0*P*UgA(x1 + x3) + 8.0*P - 4.0*T*UGA(x1 + x3)**2 + 4.0*T*UGA(x1 + x3) - 4.0*T*UgA(x1 + x3)**2 + 4.0*T*UgA(x1 + x3) + 4.0*m*UgA(x1 + x3) - 4.0*m)/(-x1 - x3 + 1)

Fitness differential is

$$ 4\big((T + R - P - S)U_{GA}(x_1)^2 - (T + 2M - 2P - S)U_{GA}(x_1)\big) + 4\big((T + R - P - S)U_{gA}(x_1)^2 - (T + 2m - 2P - S)U_{gA}(x_1)\big) - 4\big(2P - (M + m)\big) $$

```python

```

```python
def plot_male_genotype_fitness(x1, x2, x3, selection_function, d1, d3,
                               T, R, P, S, M, m, epsilon, max_time):

    fig, axes = plt.subplots(1,2, figsize=(15,8), sharex=True)

    # prepare the axes
    axes[0].set_ylim((0, 1.05))
    axes[0].set_xlabel(r"Time, $t$", fontsize=15)
    axes[0].set_ylabel(r"Offspring genotype shares, $x_i$", fontsize=15)

    axes[1].set_xlabel(r"Time, $t$", fontsize=15)
    axes[1].set_ylabel(r"Male genotype fitness", fontsize=15)

    # create the initial condition
    x4 = 1 - x1 - x2 - x3
    y0 = np.array([x1, x2, x3, x4])
    assert y0.sum() <= 1

    # create the payoff kernel
    assert (T > R) and (R > P) and (R > S), "Payoffs must satisfy either Prisoner's Dilemma or Stag Hunt constraints! T={}, R={}, P={}, S={}".format(T, R, P, S)
    payoff_kernel = np.array([[R, S], [T, P]])

    # create the selection functions
    if selection_function == "kirkpatrick":
        UGA = lambda x_A: selection_functions.kirkpatrick_selection(x_A, d1)
        UgA = lambda x_A: selection_functions.kirkpatrick_selection(x_A, d3)
    elif selection_function == "seger":
        UGA = lambda x_A: selection_functions.seger_selection(x_A, d1)
        UgA = lambda x_A: selection_functions.seger_selection(x_A, d3)
    elif selection_function == "wright":
        UGA = lambda x_A: selection_functions.wright_selection(x_A, d1)
        UgA = lambda x_A: selection_functions.wright_selection(x_A, d3)
    else:
        valid_funcs = ("kirkpatrick", "seger", "wright")
        msg = "Selection_function must be one of {}, {}, or {}.".format(*valid_funcs)
        raise ValueError(msg)

    # simulate the model starting from a random initial condition
    def f(t, y):
        W = models.generalized_sexual_selection(y, UGA, UgA, payoff_kernel, M, m, epsilon)
        y_dot = models.offspring_genotypes_evolution(W, y)
        return y_dot

    solution = integrate.solve_ivp(f, t_span=(0, max_time), y0=y0, method="RK45",
                                   rtol=1e-9, atol=1e-12, dense_output=True, vectorized=True)

    axes[0].plot(solution.t, solution.y[0], label="GA")
    axes[0].plot(solution.t, solution.y[1], label="Ga")
    axes[0].plot(solution.t, solution.y[2], label="gA")
    axes[0].plot(solution.t, solution.y[3], label="ga")
    axes[0].legend()

    def _male_genotype_fitness(yt):
        W = models.generalized_sexual_selection(yt, UGA, UgA, payoff_kernel, M, m, epsilon)
        male_genotype_fitness = W.sum(axis=0).sum(axis=0).reshape(-1, 1)
        return male_genotype_fitness

    def male_genotype_fitness(y):
        _, T = y.shape
        ws = []
        for t in range(T):
            yt = y[:,[t]]
            w = _male_genotype_fitness(yt)
            ws.append(w)
        return np.hstack(ws)

    fitness = male_genotype_fitness(solution.y)
    axes[1].plot(solution.t, fitness[0], label="GA")
    axes[1].plot(solution.t, fitness[1], label="Ga")
    axes[1].plot(solution.t, fitness[2], label="gA")
    axes[1].plot(solution.t, fitness[3], label="ga")
    axes[1].legend()

    plt.show()

    return (solution, fitness)
```

```python
# sliders used to control the initial condition
x1_slider = widgets.FloatSlider(value=0.25, min=0.0, max=1.0, step=1e-3, description=r"$x_1$", readout_format=".3f")
x2_slider = widgets.FloatSlider(value=0.25, min=0.0, max=1.0, step=1e-3, description=r"$x_2$", readout_format=".3f")
x3_slider = widgets.FloatSlider(value=0.25, min=0.0, max=1.0, step=1e-3, description=r"$x_3$", readout_format=".3f")

# sliders used to control the Prisoner's Dilemma Payoffs
T_slider = widgets.FloatSlider(value=10, min=0, max=100, step=0.1, description=r"$T$")
R_slider = widgets.FloatSlider(value=8, min=0, max=100, step=0.1, description=r"$R$")
P_slider = widgets.FloatSlider(value=6, min=0, max=100, step=0.1, description=r"$P$")
S_slider = widgets.FloatSlider(value=4, min=0, max=100, step=0.1, description=r"$S$")

# sliders used to control the metabolic costs
M_slider = widgets.FloatSlider(value=1, min=0, max=100, step=0.1, description=r"$M_G$")
m_slider = widgets.FloatSlider(value=0, min=0, max=100, step=0.1, description=r"$m_g$")

# slider used to control which selection function is being used
U_slider = widgets.Dropdown(options=["kirkpatrick", "seger", "wright"], index=0, description=r"$U_{\gamma(j)A}$")

# slider that controls the parameters of the selection function
d1_slider = widgets.FloatSlider(value=1, min=0.0, max=10, step=0.05, description=r"$d_1$")
d3_slider = widgets.FloatSlider(value=1, min=0.0, max=10, step=0.05, description=r"$d_3$")

# slider used to control the mutation rate
e_slider = widgets.FloatSlider(value=0.0, min=0.0, max=1.0, step=1e-3, description=r"$\epsilon$", readout_format=".3f")

# slider that controls max simulation time
max_time_slider = widgets.IntSlider(value=25, min=1, max=100000, description=r"$\max t$")

w = widgets.interactive(plot_male_genotype_fitness, x1=x1_slider, x2=x2_slider, x3=x3_slider,
                        selection_function=U_slider, d1=d1_slider, d3=d3_slider,
                        T=T_slider, R=R_slider, P=P_slider, S=S_slider,
                        M=M_slider, m=m_slider, epsilon=e_slider,
                        max_time=max_time_slider)
display(w)
```

    A Jupyter Widget

```python

```


# Asymptotic solutions in short-times

Projectile motion in a linear potential field with images is described by the equation

$$y_{\tau \tau} + \alpha \frac{1}{(1 + \epsilon y)^2} + 1= 0,$$

with $y(0) = \epsilon$ and $y_{\tau}(0)=1$, and where $\epsilon \ll 1$ is expected.

```python
import sympy as sym
from sympy import init_printing
init_printing(order='rev-lex')
```

```python
y, eps, a, b, t, alpha = sym.symbols('y, epsilon, a, b, t, alpha')
y0 = sym.Function('y0')(t)
y1 = sym.Function('y1')(t)
y2 = sym.Function('y2')(t)
y3 = sym.Function('y3')(t)
y4 = sym.Function('y4')(t)
```

```python
y = sym.Eq(y0 + eps*y1 + eps**2*y2 + eps**3*y3 + eps**4*y4) # naive expansion

class f(sym.Function):

    @classmethod
    def eval(cls, y):
        return y.lhs.diff(t,t) + alpha*1/(1 + eps*y.lhs)**2 + 1
        #return y.lhs.diff(tau, tau) + eps/y.lhs**2

```

```python
the_series = sym.series(f(y), eps, x0=0, n=5)
by_order = sym.collect(the_series, eps, evaluate=False)
the_series
```

### $\mathcal{O} \left( 1 \right) \mbox{Solution}$

```python
sym.Eq(by_order[1].removeO())
```

```python
eqn = sym.Eq(by_order[1].removeO())    #1 + y0(tau).diff(tau, tau))
soln0 = sym.dsolve(eqn, y0)
constants = sym.solve([soln0.rhs.subs(t,0) - 0, \
                      soln0.rhs.diff(t).subs(t,0) - 1])
C1, C2 = sym.symbols('C1 C2')
soln0 = soln0.subs(constants)
print(sym.latex(soln0))
soln0
```

### $\mathcal{O} \left( \epsilon \right) \mbox{Solution}$

```python
by_order[eps]
```

```python
try:
    eqn = sym.Eq(by_order[eps].replace(y0, soln0.rhs))
except NameError:
    eqn = sym.Eq(by_order[eps])

soln1 = sym.dsolve(eqn, y1)
constants = sym.solve([soln1.rhs.subs(t,0) - 0, \
                      soln1.rhs.diff(t,1).subs(t,0) - 0])
C1, C2 = sym.symbols('C1 C2')
soln1 = soln1.subs(constants)
soln1

```

### $\mathcal{O} \left( \epsilon^2 \right) \mbox{Solution}$

```python
by_order[eps**2]
```

```python
try:
    eqn = sym.Eq(by_order[eps**2].replace(y1, soln1.rhs).replace(y0, soln0.rhs))
except NameError:
    eqn = sym.Eq(by_order[eps**2].replace(y1, soln1.rhs))

soln2 = sym.dsolve(eqn, y2)
constants = sym.solve([soln2.rhs.subs(t,0) - 0, \
                      soln2.rhs.diff(t,1).subs(t,0) - 0])
C1, C2 = sym.symbols('C1 C2')
soln2 = soln2.subs(constants)
sym.factor(soln2)
```

### $\mathcal{O} \left( \epsilon^3 \right) \mbox{Solution}$

```python
by_order[eps**3]
```

```python
try:
    eqn = sym.Eq(by_order[eps**3].replace(y2, soln2.rhs).replace(y1, soln1.rhs).replace(y0, soln0.rhs))
except NameError:
    eqn = sym.Eq(by_order[eps**3].replace(y2, soln2.rhs))

soln3 = sym.dsolve(eqn, y3)
constants = sym.solve([soln3.rhs.subs(t,0) - 0, \
                      soln3.rhs.diff(t,1).subs(t,0) - 0])
C1, C2 = sym.symbols('C1 C2')
soln3 = soln3.subs(constants)
sym.factor(soln3)
```

### $\mathcal{O} \left( \epsilon^4 \right) \mbox{Solution}$

```python
by_order[eps**4]
```

```python
try:
    eqn = sym.Eq(by_order[eps**4].replace(y3, soln3.rhs).replace(
        y2, soln2.rhs).replace(y1, soln1.rhs).replace(y0, soln0.rhs))
except NameError:
    eqn = sym.Eq(by_order[eps**4].replace(y3, soln3.rhs))

soln4 = sym.dsolve(eqn, y4)
constants = sym.solve([soln4.rhs.subs(t,0) - 0, \
                      soln4.rhs.diff(t,1).subs(t,0) - 0])
C1, C2 = sym.symbols('C1 C2')
soln4 = soln4.subs(constants)
sym.factor(soln4)
```

### $\mbox{Composite Solution}$

```python
y_comp = sym.symbols('y_{comp}', cls=sym.Function)
try:
    y_comp = sym.Eq(y_comp, soln0.rhs + eps*soln1.rhs + eps**2*soln2.rhs + eps**3*soln3.rhs + eps**4*soln4.rhs) # + eps**2*soln2.rhs)
except NameError:
    y_comp = sym.Eq(y_comp, eps*soln1.rhs + eps**2*soln2.rhs + eps**3*soln3.rhs + eps**4*soln4.rhs) # + eps**2*soln2.rhs)

#print(sym.latex(y_comp))
y_comp
print(str(y_comp.rhs.subs(t, 1)))
```

    -alpha/2 + epsilon**4*(alpha*(159*alpha + 100)/420 + alpha*(-305*alpha**2 - 441*alpha - 150)/1120 + alpha*(3548*alpha**3 + 8424*alpha**2 + 6453*alpha + 1575)/45360 + alpha*(-3548*alpha**4 - 11972*alpha**3 - 14877*alpha**2 - 8028*alpha - 1575)/453600 - alpha/6) + epsilon**3*(alpha*(-17*alpha - 12)/60 + alpha*(73*alpha**2 + 117*alpha + 45)/630 + alpha*(-73*alpha**3 - 190*alpha**2 - 162*alpha - 45)/5040 + alpha/5) + epsilon**2*(alpha*(11*alpha + 9)/60 + alpha*(-11*alpha**2 - 20*alpha - 9)/360 - alpha/4) + epsilon*(alpha*(-alpha - 1)/12 + alpha/3) + 1/2

### $\mbox{The Trajectory}$

```python
def savefig(filename, pics):
    if pics == True:
        plt.savefig('../doc/figures/{}.pgf'.format(filename), bbox_inches='tight', dpi=400)
    else:
        pass
pics = True
```

```python
import matplotlib.pyplot as plt
import matplotlib
import numpy as np
import scipy as sp

%config InlineBackend.figure_format = 'retina'
#plt.rc('text', usetex=True)
#plt.rc('font', family='serif')
#plt.rcParams['figure.dpi'] = 300
%matplotlib inline
matplotlib.rcParams.update(
{   'text.color': 'k',
    'xtick.color': 'k',
    'ytick.color': 'k',
    'axes.labelcolor': 'k'
})
plt.rc('font', size=14)

eps_val = [.1, .5, 1.][::-1]
linestyle = ['rs--', 'bo-', 'cv-.', 'k+:', 'm']
tt = sp.arange(0,1.2,0.001)
al = [2, 1., .5, .01]

fig, axs = plt.subplots(2,2, figsize=(10, 8), sharex='col', sharey='row')
fig.subplots_adjust(hspace = .2, wspace=.2)
axs = axs.ravel()
i = 0
for aas in al:
    yc = y_comp.rhs.subs(alpha, aas)
    #plt.figure(figsize=(6, 4), dpi=100)
    for keys, vals in enumerate(eps_val):
        y_compP = sym.lambdify(t, yc.subs(eps, vals), 'numpy')
        if aas == 2:
            label='$\mathbf{E}\mbox{u}=$'+ ' {}'.format(vals).rstrip('0').rstrip('.')
        else:
            label=None
        axs[i].plot(tt, y_compP(tt), linestyle[keys],label=label,
                 markevery=100)
    axs[i].set_ylim(ymin=0., ymax=0.5)
    axs[i].set_xlim(xmax=1.05)
    axs[i].tick_params(axis='both', which='major', labelsize=16)
    leg = axs[i].legend(title = r'$\mathbf{I}\mbox{g} = $' + ' {:1.2f}'.format(aas).rstrip('0').rstrip('.'), loc=2)
    leg.get_frame().set_linewidth(0.0)
    i += 1

fig.text(0.5, -0.01, r'$t^*$', ha='center', fontsize=20)
fig.text(-0.03, 0.5, r'$y^*$', va='center', rotation='vertical', fontsize=20)
fig.tight_layout()
savefig('short_times', pics)
plt.show()
```

```python
eps_val = [.01, .1, 1.][::-1]
linestyle = ['rs--', 'bo-', 'cv-.', 'k+:', 'm']
tt = sp.arange(0,2.5,0.001)
yc = y_comp.rhs.subs(alpha, eps*0.0121 + 0.2121)

plt.figure(figsize=(6, 4))#, dpi=100)
for keys, vals in enumerate(eps_val):
    y_compP = sym.lambdify(t, yc.subs(eps, vals), 'numpy')
    plt.plot(tt, y_compP(tt), linestyle[keys], label='$\mathbf{E}\mbox{u} =$'+ ' {}'.format(vals).rstrip('0').rstrip('.'),
             markevery=100)
plt.ylim(ymin=0., ymax=0.5)
plt.xlim(xmax=2.05)
plt.legend()

plt.xlabel(r'$t^*$')
plt.ylabel(r'$y^*$')
#savefig('short_times_better', pics)
plt.show()
```

## Time aloft

```python
y2 = sym.symbols('y2', cls=sym.Function)
y2 = sym.Function('y2')(t)
try:
    y2 = sym.Eq(y2, soln0.rhs + eps*soln1.rhs + eps**2*soln2.rhs) # + eps**2*soln2.rhs)
except NameError:
    y2 = sym.Eq(y2, eps*soln1.rhs + eps**2*soln2.rhs)
y2.rhs
#y2.diff(t)
```

```python
tau0, tau1, tau2 = sym.symbols('tau0 tau1 tau2')
tau = sym.Eq(tau0 + eps*tau1 + eps**2*tau2)
y3 = y2.rhs.subs(t, tau.lhs).series(eps)
col = sym.collect(y3, eps, evaluate=False)
```

### $\mathcal{O} \left( 1 \right) \mbox{Solution}$

```python
#tau0 = 2
sym.Eq(col[1].removeO())
```

### $\mathcal{O} \left( \epsilon \right) \mbox{Solution}$

```python
order_eps = col[eps].subs(tau0, 2)
order_eps
soln_eps = sym.solve(order_eps, tau1)
```

### $\mathcal{O} \left( \epsilon^2 \right) \mbox{Solution}$

```python
order_eps2 = col[eps**2].subs(tau0, 2).subs(tau1, soln_eps[0])
order_eps2
soln_eps2 = sym.solve(order_eps2, tau2)
```

### Composite Solution
Using the linear regression for Im.

```python
tau0, tau1, tau2 = sym.symbols('tau0 tau1 tau2')
tau = sym.Eq(tau0 + eps*tau1 + eps**2*tau2)
tau = tau.subs(tau0, 2).subs(tau1, soln_eps[0]).subs(tau2, soln_eps2[0])
print(str(tau.subs(alpha, eps*0.0121 + 0.2121)))
tau.subs(alpha, eps*0.0121 + 0.2121)
```

```python
ttt = np.arange(0.01, 2.,0.001)
#betas = [bet]
linestyle = ['k','rs--', 'bo-', 'cv-.', 'k+:', 'm']
plt.figure(figsize=(6, 4), dpi=100)

#taun = tau.subs(beta, vals)
tau_soln = sym.lambdify(eps, tau.lhs.subs(alpha, eps*0.0121 + 0.2121), 'numpy')
plt.semilogx(ttt, tau_soln(ttt), 'k', markevery=100)
plt.xlabel(r'$\mathbf{E}\mbox{u}$')
plt.ylabel(r'$t_f$')
#plt.legend()
#savefig('drag', pics)
plt.show();
```

```python

```


```python
#this piece of code calculates the Coastline length for a specific elevation on earth
```

```python
#Libraries
from scipy.ndimage.filters import gaussian_filter
import matplotlib.pyplot as plt
from netCDF4 import Dataset
import numpy as np
import os
from tqdm import tqdm
from scipy.ndimage import measurements
import time
import sympy
import concurrent.futures
```

```python
#we calculate the coastline length for 5 specific elevation of the orginal data. these elevations are the peaks of the graph in the paper
from skimage import measure

unique_hieght = np.unique(hieght)
total_lat = np.sum(np.abs(np.cos(lat_topo)))

new = np.array([6056, 9576, 10655, 10656, 10657])

dphi = 0.0167
dtheta = 0.0167
squares1 = []

for i in range(5):
    print(i)
    sphere = 0
    hieght_array = (topo[:,:] <= unique_hieght[new[i]])
    contours = measure.find_contours(hieght_array, 0.7)
    if np.shape(contours)[0]>0:

        coordinate = []
        for k in range(len(contours)):
            lat = np.array(lat_topo)[contours[k][:,0].astype(int), contours[k][:,1].astype(int)]
            long = np.array(lon_topo)[contours[k][:,0].astype(int), contours[k][:,1].astype(int)]
            coordinate.append(np.column_stack((lat, long)))
        for k in range(len(coordinate)):
            for j in range(len(coordinate[k])-1):

                if(coordinate[k][j,0] == coordinate[k][j+1,0] and coordinate[k][j,1] != coordinate[k][j+1,1]):
                    sphere += np.cos(coordinate[k][j,0])*dphi
                if(coordinate[k][j,0] != coordinate[k][j+1,0] and coordinate[k][j,1] == coordinate[k][j+1,1]):
                    sphere += dtheta
                if(coordinate[k][j,0] != coordinate[k][j+1,0] and coordinate[k][j,1] != coordinate[k][j+1,1]):
                    sphere += dtheta + np.cos(coordinate[k][j,0])*dphi

    squares1.append(sphere)

np.save("/home/complex/c++/Earth/total_iso_hight_1.npy", squares1)
```


## Subsurface Data Analytics

## Artificial Neural Networks for Prediction in Python

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1) | [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy)

#### Honggeun Jo, Graduate Student, The University of Texas at Austin

##### [LinkedIn](https://www.linkedin.com/in/honggeun-jo/?originalSubdomain=kr) | [Twitter](https://twitter.com/HonggeunJ)

### PGE 383 Exercise: Artificial Neural Networks for Subsurface Modeling in Python

Here's a simple workflow, demonstration of artificial neural networks for subsurface modeling workflows. This should help you get started with building subsurface models that use data analytics and machine learning. Here's some basic details about neural networks.

#### Neural Networks

Machine learning method for supervised learning for classification and regression analysis.  Here are some key aspects of support vector machines.

**Basic Design** *"...a computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs."* Caudill (1989).

**Nature-inspire Computing** based on the neuronal structure in the brain, including many interconnected simple, processing units, known as nodes that are capable of complicated emergent pattern detection due to a large number of nodes and interconnectivity.

**Training and Testing** just like and other predictive model (e.g. linear regression, decision trees and support vector machines) we perform training to fit parameters and testing to tune hyperparameters.  Here we observe the error with training and testing datasets, but do not demonstrate tuning of the hyperparameters.

**Parameters** are the weights applied to each connection and a bias term applied to each node.  For a single node in an artificial neural network, this includes the slope terms, $\beta_i$, and the bias term, $\beta_{0}$.

\begin{equation}
Y = \sum_{i=1}^m \beta_i X + \beta_0
\end{equation}

it can be seen that the number of parameters increases rapidly as we increase the number of nodes and the connectivity between the nodes.

**Layers** the typical artificial neural net is structured with an **input layer**, with one node for each $m$ predictor feature, $X_1,\ldots,X_m$. There is an **ouput layer**, with one node for each $r$ response feature, $Y_1,\ldots,Y_r$.  There may be one or more layers of nodes between the input and output layers, known as **hidden layer(s)**.

**Connections** are the linkages between the nodes in adjacent layers.  For example, in a fully connected artificial neural network, all the input nodes are connected to all of the nodes in the first layer, then all of the nodes in the first layer are connected to the next layer and so forth. Each connection includes a weight parameter as indicated above.

**Nodes** receive the weighted signal from the connected previous layer nodes, sum and then apply this result to the **activation** function in the node. Some example activation functions include:

* **Binary** the node fires or not.  This is represented by a Heaviside step function.

* **Identify** the input is passed to the output $f(x) = x$

* **Linear** the node passes a signal that increases linearly with the weighted input.

* **Logistic** also known as sigmoid or soft step $f(x) = \frac{1}{1+e^{-x}}$

the node output is the nonlinear activiation function applied to the linearly weighted inputs. This is fed to all nodes in the next layer.

**Training Cycles** - the presentation of a batch of data, forward application of the current prediction model to make estimates, calculation of error and then backpropagation of error to correct the artificial neural network parameters to reduce the error over all of the batches.

**Batch** is the set of training data for each training cycle of forward prediction and back propagation of error, drawn to train for each iteration. There is a trade-off, a larger batch results in more computational time per iteration, but a more accurate estimate of the error to adjust the weights.  Smaller batches result in a nosier estimate of the error, but faster epochs, this results in faster learning and even possibly more robust models.

**Epochs** - is a set of training cycles, batches covering all available training data.

**Local Minimums** - if one calculated the error hypersurface over the range of model parameters it would be hyparabolic, there is a global minimium error solution.  But this error hyper surface is rough and it is possible to be stuck in a local minimum. **Learning Rate** and **Mommentum** coefficients are introduced to avoid getting stuck in local minimums.

* **Mommentum** is a hyperparameter to control the use of information from the weight update over the last epoch for consideration in the current epoch.  This can be accomplished with an update vector, $v_i$, a mommentum parameter, $\alpha$, to calculate the current weight update, $v_{i+1}$ given the new update $\theta_{i+1}$.

\begin{equation}
v_{i+1} = \alpha v_i + \theta_{i+1}
\end{equation}

* **Learning Rate** is a hyperparameter that controls the adjustment of the weights in response to the gradient indicated by backpropagation of error

##### Applications to subsurface modeling

We demonstrate the estimation of normal score transformed porosity from depth.  This would be useful for building a vertical trend model.

* modeling the complicated relationship between porosity and depth.

#### Limitations of Neural Network Estimation

Since we demonstrate the use of an artificial neural network to estimate porosity from sparsely sampled data over depth, we should comment on limitations of our artificial neural networks for this estimation problem:

* does not honor the well data

* does not honor the histogram of the data

* does not honor spatial correlation

* does not honor the multivariate relationship

* generally low interpretability models

* requires a large number of data for effective training

* high model complexity with high model variance

#### Workflow Goals

Learn the basics of machine learning in python to predict subsurface features. This includes:

* Loading and visualizing sample data
* Trying out neural nets

#### Objective

In the PGE 383: Subsurface Machine Learning Class, I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows.

The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - 12_sample_data.csv found [here](https://github.com/GeostatsGuy/GeoDataSets/blob/master/12_sample_data.csv).

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

#### Import Required Packages

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
import geostatspy.GSLIB as GSLIB          # GSLIB utilies, visualization and wrapper
import geostatspy.geostats as geostats    # GSLIB methods convert to Python
```

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
import numpy as np                        # ndarrys for gridded data
import pandas as pd                       # DataFrames for tabular data
import os                                 # set working directory, run executables
import matplotlib.pyplot as plt           # for plotting
import seaborn as sns                     # for plotting
import warnings                           # supress warnings from seaborn pairplot
from sklearn.model_selection import train_test_split # train / test DatFrame split
```

We will also need the following packages to train and test our artificial neural nets:

* Tensorflow - open source machine learning

* Keras - high level application programing interface (API) to build and train models

More information is available at [tensorflow install](https://www.tensorflow.org/install).

* This workflow was designed with tensorflow version 1.13.1 and does not work with version 2.0.0 alpha.

To check your current version of tensorflow you could run the next block of code.

```python
import tensorflow as tf
tf.__version__                            # check the installed version of tensorflow
```

    '2.2.0'

Let's import all of the tensorflow and keras methods that we will need in our workflow.

```python
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Activation, Dropout
#from keras.models import Sequential, load_model
#from keras.layers.core import Dense, Dropout, Activation
from tensorflow.keras.optimizers import Adam
from tensorflow.python.keras import backend as k
```

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).

```python
os.chdir("c:/PGE383")
```

#### Loading Data
Let's load the provided multivariate, spatial dataset '12_sample_data.csv'.  It is a comma delimited file with:

* Depth ($m$)
* Normal Score Porosity

It is common to transform properties to standard normal for geostatistical workflows.

We load it with the pandas 'read_csv' function into a data frame we called 'df' and then preview it to make sure it loaded correctly.

**Python Tip: using functions from a package** just type the label for the package that we declared at the beginning:

```python
import pandas as pd
```

so we can access the pandas function 'read_csv' with the command:

```python
pd.read_csv()
```

but read csv has required input parameters. The essential one is the name of the file. For our circumstance all the other default parameters are fine. If you want to see all the possible parameters for this function, just go to the docs [here](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html).

* The docs are always helpful
* There is often a lot of flexibility for Python functions, possible through using various inputs parameters

also, the program has an output, a pandas DataFrame loaded from the data.  So we have to specficy the name / variable representing that new object.

```python
df = pd.read_csv("1D_Porosity.csv")
```

Let's run this command to load the data and then look at the resulting DataFrame to ensure that we loaded it.

```python
df2 = pd.read_csv("1D_Porosity.csv")                    # read a .csv file in as a DataFrame                                     # display first 4 samples in the table as a preview
df2.head()                                                   # we could also use this command for a table preview
```

      Depth
      Nporosity

      0
      0.25
      -1.37

      1
      0.50
      -2.08

      2
      0.75
      -1.67

      3
      1.00
      -1.16

      4
      1.25
      -0.24

#### Data Normalization

We must normalize the features before we apply them in an artificial neural network model. These are the motivations for this normalization:

* remove impact of scale of different types of data (i.e., depth varies between $[0,10]$, but porosity only varies between $[-3,3.0]$).

* activation functions in artificial neural networks are designed to be more sensive to values of nodes closer to 0.0 (i.e., results in higher gradient and improves backpropagation in training)

Let's normalize each feature.

* We apply the min max normalization by-hand to force both the predictor and response features to be bound $[-1,1]$.

* It is easy to backtransform given we keep track of the original min and max values

```python
depth_min = df2['Depth'].values.min(); depth_max = df2['Depth'].values.max()
Npor_min = df2['Nporosity'].values.min(); Npor_max = df2['Nporosity'].values.max()
df2['norm_Depth'] = (df2['Depth'] - depth_min)/(depth_max - depth_min) * 2 - 1
df2['norm_Nporosity'] = (df2['Nporosity'] - Npor_min)/(Npor_max - Npor_min) * 2 - 1
df2.head()
```

      Depth
      Nporosity
      norm_Depth
      norm_Nporosity

      0
      0.25
      -1.37
      -1.000000
      -0.679458

      1
      0.50
      -2.08
      -0.948718
      -1.000000

      2
      0.75
      -1.67
      -0.897436
      -0.814898

      3
      1.00
      -1.16
      -0.846154
      -0.584650

      4
      1.25
      -0.24
      -0.794872
      -0.169300

It is also a good idea to check the summary statistics.

* All normalized features should now range from -1.0 to 1.0

```python
df2.describe().transpose()
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Depth
      40.0
      5.125000e+00
      2.922613
      0.25
      2.687500
      5.125000e+00
      7.562500
      10.00

      Nporosity
      40.0
      2.225000e-02
      0.992111
      -2.08
      -0.757500
      1.400000e-01
      0.742500
      2.35

      norm_Depth
      40.0
      -5.551115e-18
      0.599510
      -1.00
      -0.500000
      -5.551115e-17
      0.500000
      1.00

      norm_Nporosity
      40.0
      -5.090293e-02
      0.447906
      -1.00
      -0.402935
      2.257336e-03
      0.274266
      1.00

#### Separation of Training and Testing Data

We also need to split our data into training / testing datasets so that we:

* can train our artificial neural networks using the training data

* while testing their performance with the withheld testing (validation) data.

```python
X2 = df2.iloc[:,[0,2]]                        # extract the predictor feature - acoustic impedance
y2 = df2.iloc[:,[1,3]]                        # extract the response feature - porosity
X2_train, X2_test, y2_train, y2_test = train_test_split(X2, y2, test_size=0.2, random_state=73073)
```

#### Visualize the Dataset

Let's visualize the train and test data split on a single scatter plot.

* we want to make sure it is fair

* ensure that the test samples are not clustered or too far away from the training data.

We will look at the original data and normalized data, the input to the neural network.

```python
plt.subplot(211)
plt.plot(X2_train['Depth'].values,y2_train['Nporosity'].values, 'o', markerfacecolor='red', markeredgecolor='black', alpha=0.4, label = "Train")
plt.plot(X2_test['Depth'].values,y2_test['Nporosity'].values, 'o', markerfacecolor='blue', markeredgecolor='black', alpha=0.4, label = "Test")
plt.title('Standard Normal Porosity vs. Depth')
plt.xlabel('Z (m)')
plt.ylabel('NPorosity')
plt.legend()
plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.0, wspace=0.2, hspace=0.2)

plt.subplot(212)
plt.plot(X2_train['norm_Depth'].values,y2_train['norm_Nporosity'].values, 'o', markerfacecolor='red', markeredgecolor='black', alpha=0.4, label = "Train")
plt.plot(X2_test['norm_Depth'].values,y2_test['norm_Nporosity'].values, 'o', markerfacecolor='blue', markeredgecolor='black', alpha=0.4, label = "Test")
plt.title('Normalized Standard Normal Porosity vs. Normalized Depth')
plt.xlabel('Normalized Z')
plt.ylabel('Normalized NPorosity')
plt.legend()

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.5, wspace=0.2, hspace=0.3)
plt.show()
```

#### Specify the Prediction Locations

Given this training and testing data, let's specify the prediction locations over the range of the observed depths at regularly spaced $nbins$ locations.

```python
# Specify the prediction locations
nbins = 1000
depth_bins = np.linspace(depth_min, depth_max, nbins) # set the bins for prediction
norm_depth_bins = (depth_bins-depth_min)/(depth_max-depth_min)*2-1 # use normalized bins
```

#### Build and Train a Simple Neural Network

For our first model we will build a simple model with:

* 1 predictor feature - depth ($d$)

* 1 responce feature - normal score porosity ($N\{\phi\}$)

we will build a model to predict normal score porosity from depth over all locations in our model $\bf{u} \in AOI$.

\begin{equation}
N\{\phi(\bf{u})\} = \hat{f} (d(\bf{u}))
\end{equation}

and use this model to support the prediction of porosity between the wells.

#### Design, Train and Test a Neural Network

In the following code we use keras / tensorflow to:

1. **Design the Network** - we use a fully connected, feed forward neural network with one node in the input and output layers, to receive the normalized depth and output the normalized (normal score) porosity.  We found by trial and error, given the complexity of the dataset, that we required a significant network width (about 500 nodes) and a network depth of atleast 4 hidden layers.

2. **Select the Optimizer** - we selected the adam optimizer (Kingma and Ba, 2015).  This optimizer is computationally efficient and is well suited to problems with noisy data and sparse gradients. It is an extension of stochastic gradient descent with the addition of adaptive gradient algorithm that calculates per-parameter learning rates to improve learning with sparse gradients, and root mean square propopagation that sets the learning rate based on the recent magnitudes of the gradients for each parameter to improve performance with noisy data.  We include stochastic gradient descent for experimentation.

    * we found a learning rate of 0.01 to 0.001 works well
    * we found the rate of decay parameters of $\beta_1=0.9$ and $\beta_2=0.999$ performed well

3. **Compile the Machine** - specify the optimizer, loss function and the metric for model training.

4. **Train the Network** - fit / train the model parameters over a specified number of batch size within a specified number of epochs.  We specify the train and test normalized predictor and response features.

Then we visualize the model in the original units.

```python
# Design the neural network
model_2 = Sequential([
    Dense(1, activation='linear', input_shape=(1,)),                  # input layer
    Dense(50, activation='relu'),
    Dense(50, activation='relu'),
    Dense(50, activation='relu'),
    Dense(20, activation='relu'),                                    # uncomment these to add hidden layers
#    Dense(20, activation='relu'),
#    Dense(20, activation='relu'),
#    Dense(20, activation='relu'),
    Dense(1, activation='linear'),                                     # output layer
])

# Select the Optimizer
adam = Adam(lr=0.003, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False) # adam optimizer
#sgd = keras.optimizers.SGD(lr=0.001, momentum=0.0, decay = 0.0, nesterov=False)  # stochastic gradient descent

# Compile the Machine
model_2.compile(optimizer=adam,loss='mse',metrics=['accuracy'])

# Train the Network
hist_2 = model_2.fit(X2_train['norm_Depth'], y2_train['norm_Nporosity'],
    batch_size=5, epochs=200,
    validation_data=(X2_test['norm_Depth'], y2_test['norm_Nporosity']),verbose = 0)

# Predict with the Network
pred_norm_Nporsity = model_2.predict(np.array(norm_depth_bins)) # predict with our ANN
pred_Nporosity = ((pred_norm_Nporsity + 1)/2*(Npor_max - Npor_min)+Npor_min)

# Plot the Model Predictions
plt.subplot(1,1,1)
plt.plot(depth_bins,pred_Nporosity,'black',linewidth=2)
plt.plot(X2_train['Depth'].values,y2_train['Nporosity'].values, 'o', markerfacecolor='red', markeredgecolor='black', alpha=1.0, label = "Train")
plt.plot(X2_test['Depth'].values,y2_test['Nporosity'].values, 'o', markerfacecolor='blue', markeredgecolor='black', alpha=1.0, label = "Test")
plt.xlabel('Depth (m)')
plt.ylabel('Porosity (faction)')
plt.legend()
plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=0.8, wspace=0.2, hspace=0.2)
plt.show()
```

#### Evaluation of the Model

For my specified artificial neural network design and optimization parameters I have a very flexible model to fit the data.

* artificial neural networks live up to their designation as **Universal Function Approximators**

Let's check the training curve, loss functions for our model over training and testing datasets.

* **square loss** ($L_2$ loss) is the:

\begin{equation}
L_2 = \sum_{\bf{u}_{\alpha} \in AOI} \left(y(\bf{u}_{\alpha}) - \hat{f}(x_1(\bf{u}_{\alpha}),\ldots,x_m(\bf{u}_{\alpha})\right)
\end{equation}

* this is a measure of the inaccuracy over the available data

We can see the progress of the model over epochs in reduction of training and testing error.

* we can observe that the model matches the training data after about 200 epochs, but continues to improve up the 1,000 epochs

```python
# Plot the Loss vs. Training Epoch
plt.subplot(1,1,1)
plt.plot(hist_2.history['loss'])
plt.plot(hist_2.history['val_loss'])
plt.title('Loss function of Artificial Neural Net Example #2')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper right')
plt.ylim(0,0.4)
plt.grid()
plt.tight_layout()
plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=0.8, wspace=0.2, hspace=0.2)
plt.show()
```

#### Some Observations

We performed a set of experiments and made the following observations that may help you experiment with the design and training of this artificial neural network. Each machine was trained over 1000 epochs with a batch size of 5.

For a 1 $\times$ 100 $\times$ 100 $\times$ 1 neural network:
* Learning Rate of 0.001 still converging at 1000 epochs, missing some features
* Learning Rate of 0.1 - stuck in a local minum as a step function

For a 1 $\times$ 500 $\times$ 500 $\times$ 1 neural network:
* Learning Rate of 0.01 - 0.001 for a close fit to training data
* Learning Rate of 0.1 - stuck in a local minimum as a line
* Learning Rate of $\le$ 0.0001 still converging at 1000 epochs, missing some features

For a 1 $\times$ 500 $\times$ 500 $\times$ 500 $\times$ 1 neural network:
* Learning Rate of 0.01 - 0.001 for a close fit to training data
* Learning Rate of 0.1 - stuck as a line
* Learning Rate of $\le$ 0.0001 still converging at 1000 epochs, missing some features

#### Visualizing the Neural Net

There are some methods available to interogate the artificial neural net.

* neural net summary

* weights

Here's the summary from our neural net.  It lists by layers the number of nodes and number of parameters.

```python
model_2.summary()                                      # artificial neural network design and number of parameters
```

    Model: "sequential_4"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    dense_12 (Dense)             (None, 1)                 2
    _________________________________________________________________
    dense_13 (Dense)             (None, 50)                100
    _________________________________________________________________
    dense_14 (Dense)             (None, 50)                2550
    _________________________________________________________________
    dense_15 (Dense)             (None, 1)                 51
    =================================================================
    Total params: 2,703
    Trainable params: 2,703
    Non-trainable params: 0
    _________________________________________________________________

We can also see the actual trained weights for each node in each layer.

```python
for layer in model_2.layers:                            # weights for the trained artificial neural network
    g = layer.get_config()
    h = layer.get_weights()
    print(g)
    print(h)
    print('\n')
```

    {'name': 'dense', 'trainable': True, 'batch_input_shape': (None, 1), 'dtype': 'float32', 'units': 1, 'activation': 'linear', 'use_bias': True, 'kernel_initializer': {'class_name': 'GlorotUniform', 'config': {'seed': None}}, 'bias_initializer': {'class_name': 'Zeros', 'config': {}}, 'kernel_regularizer': None, 'bias_regularizer': None, 'activity_regularizer': None, 'kernel_constraint': None, 'bias_constraint': None}
    [array([[-0.6884202]], dtype=float32), array([0.1548591], dtype=float32)]

    {'name': 'dense_1', 'trainable': True, 'dtype': 'float32', 'units': 1, 'activation': 'relu', 'use_bias': True, 'kernel_initializer': {'class_name': 'GlorotUniform', 'config': {'seed': None}}, 'bias_initializer': {'class_name': 'Zeros', 'config': {}}, 'kernel_regularizer': None, 'bias_regularizer': None, 'activity_regularizer': None, 'kernel_constraint': None, 'bias_constraint': None}
    [array([[-1.1786897]], dtype=float32), array([-0.15738262], dtype=float32)]

    {'name': 'dense_2', 'trainable': True, 'dtype': 'float32', 'units': 1, 'activation': 'linear', 'use_bias': True, 'kernel_initializer': {'class_name': 'GlorotUniform', 'config': {'seed': None}}, 'bias_initializer': {'class_name': 'Zeros', 'config': {}}, 'kernel_regularizer': None, 'bias_regularizer': None, 'activity_regularizer': None, 'kernel_constraint': None, 'bias_constraint': None}
    [array([[1.4482979]], dtype=float32), array([-0.16542506], dtype=float32)]

We can observe the weights and bias terms associate with our neural network.

#### Comments

This was a basic demonstration of artificial neural networks.

The Texas Center for Geostatistics has many other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling and many other workflows available [here](https://github.com/GeostatsGuy/PythonNumericalDemos), along with a package for geostatistics in Python called [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy).

We hope this was helpful,

*Michael* and *Honggeun*

***

#### More on Michael Pyrcz and the Texas Center for Data Analytics and Geostatistics:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```

```python

```

```python

```


```python
%matplotlib notebook

from matplotlib import pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import animation
import time

fig = plt.figure()
ax = fig.add_subplot(projection='3d')

def gen(n):
    phi = 0
    while phi < 2*np.pi:
        yield np.array([np.cos(phi), np.sin(phi), phi])
        phi += 2*np.pi/n

def update(num, data, line):
    line.set_data(data[:2, :num])
    line.set_3d_properties(data[2, :num])

N = 100
data = np.array(list(gen(N))).T
line, = ax.plot(data[0, 0:1], data[1, 0:1], data[2, 0:1])

# Setting the axes properties
ax.set_xlim3d([-1.0, 1.0])
ax.set_xlabel('X')

ax.set_ylim3d([-1.0, 1.0])
ax.set_ylabel('Y')

ax.set_zlim3d([0.0, 10.0])
ax.set_zlabel('Z')

ani = animation.FuncAnimation(fig, update, N, fargs=(data, line), interval=10000/N, blit=False)
#ani.save('matplot003.gif', writer='imagemagick')
plt.show()
```

```python
# Prueba visualizacion de marcos de referencia y puntos en 3D

from sympy.physics.mechanics import ReferenceFrame,Point,Vector
from sympy import symbols
from mpl_toolkits import mplot3d
from matplotlib import pyplot
import sys
sys.path.append("../tools")
from vis import Visualizer
%matplotlib notebook

q1,q2=symbols('q1,q2')
N=ReferenceFrame('N')
o=Point('O')
p=o.locatenew('P',q1*N.x)
r=o.locatenew('r',q1*N.x)

#O.locatenew('P',Vector([1,2,3]))
#Crea la visualizacion con el marco de referencia N y punto de origen en o
vis=Visualizer(N,o)
#Agrega la visualizacion del marco N en el punto p con escala x10
vis.add(N,p,frame_scale=10)
#Grafica el modelo para un valor determinado de la variable de movimiento q1
vis.plot({q1:5})
```

    <IPython.core.display.Javascript object>

    10

    c:\opt\ros\noetic\x64\lib\site-packages\numpy\core\fromnumeric.py:87: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      return ufunc.reduce(obj, axis, dtype, out, **passkwargs)

```python
#Modifique el valor de q1
vis.plot({q1:-0})
```

    10


# Import Problem Instance
We start by importing a simple problem instance to demonstrate the tsplib reader.

```python
from tsplib95 import tsplib95
import itertools
import networkx as nx

instance = tsplib95.load_problem('./tsplib/ulysses16.tsp')

instance.comment
```

    'Odyssey of Ulysses (Groetschel/Padberg)'

Remember, this repository contains a small selection of TSP instances that you can use to test your algorithms.

| name | nodes | description |
|------|-------|-------------|
| ulysses16.tsp | 16 | Odyssey of Ulysses |
| ulysses7.tsp | 7 | subset of ulysses16 for testing purposes |
| bayg29.tsp | 29 | 29 Cities in Bavaria |
| bier127.tsp | 127 | 127 Biergaerten in Augsburg |
| bier20.tsp | 20 | subset of bier127 |
| brazil58.tsp | 58 | 58 cities in Brazil |
| ali535.tsp | 535 | 535 Airports around the globe |
| d18512.tsp | 18512 | 18512 places in Germany |

The following calls show the dimension = number of nodes of the problem, its node set and the edge weights. The functions `instance.get_nodes()` and `instance.get_edges()` are implemented as iterators, so you can loop over the nodes or edges. To get a list of nodes or edges, you have to explicitly construct one using `list(instance.get_nodes())`. Note that node counting may start at 1 for some instances while others use 0 as starting point. For convenience, we store the index of the first node as `first_node`.

```python
instance.dimension

instance.get_nodes()
print("List of nodes: ", list(instance.get_nodes()))

first_node = min(instance.get_nodes())
first_node

for i,j in instance.get_edges():
    if i >= j:
        continue
    print(f"edge {{ {i:2},{j:2} }} has weight {instance.wfunc(i,j):3}.")

```

    List of nodes:  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
    edge {  1, 2 } has weight   6.
    edge {  1, 3 } has weight   5.
    edge {  1, 4 } has weight   3.
    edge {  1, 5 } has weight  11.
    edge {  1, 6 } has weight   8.
    edge {  1, 7 } has weight   7.
    edge {  1, 8 } has weight   1.
    edge {  1, 9 } has weight  12.
    edge {  1,10 } has weight   8.
    edge {  1,11 } has weight  26.
    edge {  1,12 } has weight   5.
    edge {  1,13 } has weight   5.
    edge {  1,14 } has weight   5.
    edge {  1,15 } has weight   7.
    edge {  1,16 } has weight   1.
    edge {  2, 3 } has weight   1.
    edge {  2, 4 } has weight   4.
    edge {  2, 5 } has weight  17.
    edge {  2, 6 } has weight  14.
    edge {  2, 7 } has weight  13.
    edge {  2, 8 } has weight   6.
    edge {  2, 9 } has weight  17.
    edge {  2,10 } has weight  13.
    edge {  2,11 } has weight  32.
    edge {  2,12 } has weight  11.
    edge {  2,13 } has weight  11.
    edge {  2,14 } has weight  11.
    edge {  2,15 } has weight  13.
    edge {  2,16 } has weight   7.
    edge {  3, 4 } has weight   5.
    edge {  3, 5 } has weight  16.
    edge {  3, 6 } has weight  13.
    edge {  3, 7 } has weight  12.
    edge {  3, 8 } has weight   6.
    edge {  3, 9 } has weight  16.
    edge {  3,10 } has weight  12.
    edge {  3,11 } has weight  31.
    edge {  3,12 } has weight  10.
    edge {  3,13 } has weight  10.
    edge {  3,14 } has weight  11.
    edge {  3,15 } has weight  12.
    edge {  3,16 } has weight   6.
    edge {  4, 5 } has weight  13.
    edge {  4, 6 } has weight  11.
    edge {  4, 7 } has weight  10.
    edge {  4, 8 } has weight   3.
    edge {  4, 9 } has weight  15.
    edge {  4,10 } has weight  11.
    edge {  4,11 } has weight  28.
    edge {  4,12 } has weight   8.
    edge {  4,13 } has weight   8.
    edge {  4,14 } has weight   8.
    edge {  4,15 } has weight   9.
    edge {  4,16 } has weight   5.
    edge {  5, 6 } has weight   4.
    edge {  5, 7 } has weight   6.
    edge {  5, 8 } has weight  11.
    edge {  5, 9 } has weight   8.
    edge {  5,10 } has weight   8.
    edge {  5,11 } has weight  16.
    edge {  5,12 } has weight   7.
    edge {  5,13 } has weight   7.
    edge {  5,14 } has weight   6.
    edge {  5,15 } has weight   4.
    edge {  5,16 } has weight  11.
    edge {  6, 7 } has weight   1.
    edge {  6, 8 } has weight   8.
    edge {  6, 9 } has weight   5.
    edge {  6,10 } has weight   4.
    edge {  6,11 } has weight  17.
    edge {  6,12 } has weight   3.
    edge {  6,13 } has weight   3.
    edge {  6,14 } has weight   3.
    edge {  6,15 } has weight   3.
    edge {  6,16 } has weight   8.
    edge {  7, 8 } has weight   7.
    edge {  7, 9 } has weight   5.
    edge {  7,10 } has weight   3.
    edge {  7,11 } has weight  18.
    edge {  7,12 } has weight   2.
    edge {  7,13 } has weight   2.
    edge {  7,14 } has weight   2.
    edge {  7,15 } has weight   3.
    edge {  7,16 } has weight   7.
    edge {  8, 9 } has weight  12.
    edge {  8,10 } has weight   8.
    edge {  8,11 } has weight  26.
    edge {  8,12 } has weight   5.
    edge {  8,13 } has weight   5.
    edge {  8,14 } has weight   5.
    edge {  8,15 } has weight   6.
    edge {  8,16 } has weight   2.
    edge {  9,10 } has weight   4.
    edge {  9,11 } has weight  15.
    edge {  9,12 } has weight   7.
    edge {  9,13 } has weight   7.
    edge {  9,14 } has weight   7.
    edge {  9,15 } has weight   8.
    edge {  9,16 } has weight  11.
    edge { 10,11 } has weight  19.
    edge { 10,12 } has weight   3.
    edge { 10,13 } has weight   4.
    edge { 10,14 } has weight   4.
    edge { 10,15 } has weight   6.
    edge { 10,16 } has weight   7.
    edge { 11,12 } has weight  20.
    edge { 11,13 } has weight  21.
    edge { 11,14 } has weight  20.
    edge { 11,15 } has weight  20.
    edge { 11,16 } has weight  25.
    edge { 12,13 } has weight   0.
    edge { 12,14 } has weight   1.
    edge { 12,15 } has weight   3.
    edge { 12,16 } has weight   5.
    edge { 13,14 } has weight   1.
    edge { 13,15 } has weight   3.
    edge { 13,16 } has weight   4.
    edge { 14,15 } has weight   2.
    edge { 14,16 } has weight   5.
    edge { 15,16 } has weight   7.

You have already seen how to draw a graph, here is the relevant code again.

```python
G = instance.get_graph()
if instance.is_depictable():
    pos = {i: instance.get_display(i) for i in instance.get_nodes()}
else:
    pos = nx.drawing.layout.spring_layout(G)
nx.draw_networkx_nodes(G, pos, node_color='#66a3ff', node_size=200)
nx.draw_networkx_labels(G, pos, font_weight='bold' )
nx.draw_networkx_edges(G, pos,  edge_color='#e6e6e6')
```

# Implementing the standard model with subtour elimination in Gurobi
We will implement the standard model with subtour elimination callback using binary variables $x_{ij} \in \{0,1\}$ to indicate whether edge $\{i,j\}$ is being used in the tour. To avoid double counting edges, we employ the convention that $i < j$ for an edge $\{i,j\}$ and we will denote the resulting edge set by $E$. The formulation looks like this:

\begin{align}
\min\;&\sum_{\{i,j\} \in E} c_{i,j} \cdot x_{i,j}\\
&\sum_{j: \{i,j\} \in E} x_{i,j} = 2 \quad \text{for all nodes $i$}\\
&\sum_{\{i,j\} \in \delta(S)} x_{ij} \ge 2 \quad \text{for all $S \subsetneq V$, $S \ne \emptyset$}\\
&x_{i,j} \in \{0,1\}
\end{align}

## Creating the variables
We start by creating the model and the variables. Notice that we already define the objective function by using the `obj` parameter upon variable creation.

```python
import gurobipy as grb

model = grb.Model(name="Subtour TSP formulation")

x = grb.tupledict()
for i,j in instance.get_edges():
    if i < j:
        x[i,j] = model.addVar(obj=instance.wfunc(i,j), vtype=grb.GRB.BINARY, name=f"x[{i},{j}]")
```

    Academic license - for non-commercial use only

## Adding the degree constraints
Next, we add the constraints for our model with the exception of the subtour elimination constraints. We use the sum method of our variables to express the summation in an elegant way.

```python
for i in instance.get_nodes():
    model.addConstr(x.sum(i,'*') + x.sum('*',i) == 2, name=f"degree_ctr[{i}]")
```

## Starting the Optimization Process
Finally, we set the objective to minimization and call the optimizer.

```python
model.ModelSense = grb.GRB.MINIMIZE
model.reset()
model.optimize()
```

    Optimize a model with 16 rows, 120 columns and 240 nonzeros
    Variable types: 0 continuous, 120 integer (120 binary)
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      Objective range  [1e+00, 3e+01]
      Bounds range     [1e+00, 1e+00]
      RHS range        [2e+00, 2e+00]
    Found heuristic solution: objective 148.0000000
    Presolve time: 0.00s
    Presolved: 16 rows, 120 columns, 240 nonzeros
    Variable types: 0 continuous, 120 integer (120 binary)

    Root relaxation: objective 6.200000e+01, 20 iterations, 0.00 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

    *    0     0               0      62.0000000   62.00000  0.00%     -    0s

    Explored 0 nodes (20 simplex iterations) in 0.01 seconds
    Thread count was 8 (of 8 available processors)

    Solution count 2: 62 148

    Optimal solution found (tolerance 1.00e-04)
    Best objective 6.200000000000e+01, best bound 6.200000000000e+01, gap 0.0000%

## Querying and Visualizing the Solution
Before we visualize our result, let us look at a few key figures of our solution.

```python
model.ObjVal
```

```python
solution_edges = [(i,j) for i,j in x.keys() if x[i,j].x > 0.9]
solution_edges
```

    [(1, 8),
     (1, 16),
     (2, 3),
     (2, 4),
     (3, 4),
     (5, 11),
     (5, 15),
     (6, 7),
     (6, 15),
     (7, 10),
     (8, 16),
     (9, 10),
     (9, 11),
     (12, 13),
     (12, 14),
     (13, 14)]

For debugging purposes, it might be helpful to export the model held by Gurobi into a human-readable format:

```python
model.write('test.lp')
```

 Finally, let us visualize the solution using NetworkX. In this case, we need to prescribe positions and draw the nodes and two layers of edges separately.

```python
if instance.is_depictable():
    pos = {i: instance.get_display(i) for i in instance.get_nodes()}
else:
    pos = nx.drawing.layout.spring_layout(G)
nx.draw_networkx_nodes(G, pos, node_color='#66a3ff', node_size=500)
nx.draw_networkx_labels(G, pos, font_weight='bold' )
nx.draw_networkx_edges(G, pos,  edge_color='#e6e6e6')
nx.draw_networkx_edges(G, pos,  edgelist=solution_edges, edge_color='#ffa31a', width=4)
```

## Subtour elimination
As you can hopefully see (depending on the instance you selected), the solution may contain a subtour. Let us add a callback to detect and eliminate subtours in an integer solution.

We start by defining a function that, given a list of edges, finds a subtour if there is one. For a more concise implementation, we use a function of networkx to find such a subtour: We construct an auxiliary graph $G_{\text{aux}}$, find a cycle in this graph and return just the nodes contained in this cycle.

```python
def find_subtour_set(nodes, edges):
    G_aux = nx.Graph()
    G_aux.add_nodes_from(nodes)
    G_aux.add_edges_from(edges)
    return set(itertools.chain(*nx.find_cycle(G_aux)))

find_subtour_set(instance.get_nodes(), [(i,j) for i,j in x.keys() if x[i,j].x > 0.9])
```

    {1, 8, 16}

### Define a callback
Next, we need to define a callback function that adds a violated subtour inequality if one exists. In Gurobi, there is just one "global" callback function that two parameters: The `model` and a constant called `where` (a "Callback Code") that indicates at which position in the optimization process the callback has been invoked. The documentation contains a list of the available codes at http://www.gurobi.com/documentation/8.0/refman/callback_codes.html#sec:CallbackCodes. We want our callback to spring into action whenever a new integer solution has been found, so the relevant callback code is `GRB.Callback.MIPSOL`.

Notice that we can only access parameters and current values of our model through the model object, not through the variables we have defined above. The model supplies a number of `cbGet...` methods for this purpose. To access our variables, we need to define a _user variable_ in the model that stores this information and makes it accessible in the callback. User variables can be any member of the model objects that starts with an underscore. We will add a parameter `_vars` to our model that simply stores the x variables and then use this parameter in the callback to access the current solution.

To add a new constraint, we use the method `cbLazy` of our model that adds a new lazy constraint. The node will then be re-evaluated automatically.

### Subtour Elimination Callback
 **Task 1:** In the following function, complete the definition of the `cut_edges` list to make the subtour elimination work properly.

```python
def subtour_callback(model, where):
    if where == grb.GRB.Callback.MIPSOL:
        sol = model.cbGetSolution(model._vars)
        S = find_subtour_set(model._instance.get_nodes(), [(i,j) for i,j in model._vars.keys() if sol[i,j] > 0.9])
        if len(S) < model._instance.dimension:
            # TODO: cut_edges
            cut_edges = [(i,j) for i,j in model._vars.keys() if ((i in S and j not in S) or (i not in S and j in S))]
            model.cbLazy(sum(model._vars[i,j] for i,j in cut_edges) >= 2)
```

### Add Callback to the Model
Let us now add the variables to the model and resolve, this time using the callback. Also, we need to switch on lazy constraints by setting the appropropriate parameter. The callback function is simply passed as a parameter to the optimizer.

```python
model._vars = x # for use in the callback
model._instance = instance # for use in the callback
model.reset()
model.Params.lazyConstraints = 1 # use lazy constraints
model.optimize(subtour_callback) # use callback to add lazy constraints
```

    Parameter lazyConstraints unchanged
       Value: 1  Min: 0  Max: 1  Default: 0
    Optimize a model with 16 rows, 120 columns and 240 nonzeros
    Variable types: 0 continuous, 120 integer (120 binary)
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      Objective range  [1e+00, 3e+01]
      Bounds range     [1e+00, 1e+00]
      RHS range        [2e+00, 2e+00]
    Presolve time: 0.00s
    Presolved: 16 rows, 120 columns, 240 nonzeros
    Variable types: 0 continuous, 120 integer (120 binary)

    Root relaxation: objective 6.800000e+01, 25 iterations, 0.00 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0   68.00000    0    6          -   68.00000      -     -    0s
         0     0   70.50000    0   10          -   70.50000      -     -    0s
    H    0     0                      72.0000000   70.50000  2.08%     -    0s
    H    0     0                      71.0000000   70.50000  0.70%     -    0s

    Cutting planes:
      Gomory: 1
      Zero half: 1
      Lazy constraints: 10

    Explored 1 nodes (42 simplex iterations) in 0.03 seconds
    Thread count was 8 (of 8 available processors)

    Solution count 2: 71 72

    Optimal solution found (tolerance 1.00e-04)
    Best objective 7.100000000000e+01, best bound 7.100000000000e+01, gap 0.0000%

### Results
Let's have a look at the results.

```python
model.ObjVal
```

    71.0

```python
solution_edges = [(i,j) for i,j in x.keys() if x[i,j].x > 0.9]
solution_edges
```

    [(1, 3),
     (1, 16),
     (2, 3),
     (2, 4),
     (4, 8),
     (5, 6),
     (5, 11),
     (6, 7),
     (7, 14),
     (8, 15),
     (9, 10),
     (9, 11),
     (10, 12),
     (12, 13),
     (13, 16),
     (14, 15)]

```python
if instance.is_depictable():
    pos = {i: instance.get_display(i) for i in instance.get_nodes()}
else:
    pos = nx.drawing.layout.spring_layout(G)
nx.draw_networkx_nodes(G, pos, node_color='#66a3ff', node_size=500)
nx.draw_networkx_labels(G, pos, font_weight='bold' )
nx.draw_networkx_edges(G, pos,  edge_color='#e6e6e6')
nx.draw_networkx_edges(G, pos,  edgelist=solution_edges, edge_color='#ffa31a', width=4)
```

## Fractional Subtour Elimination
Finally, let us try to implement a separation procedure for subtour elimination constraints that works on the relaxation and does not need an integer solution. This can be done by solving a minimum cut problem on an auxiliary graph where we fix an arbitrary source node (we will use node $1$) and iterate through all possible target nodes until we find a minimum cut that has a value of less than $2$. For this cut, the corresponding subtour elimination constraint is inserted as a lazy cut. We modify our callback to use the callback code `MIPNODE` instead of `MIPSOL` and query `GRB.Callback.MIPNODE_STATUS` to see whether the node has already been solved to (fractional) optimality. For computing the minimum cut we again use an algorithm provided by the `networkx` package.
Note that we have substituted `cbGetSolution` by `cbGetNodeRel`, as a solution is not generally available at any node. Also, we have to include the `MIPSOL` callback branch, because `MIPNODE` might not be called for nodes that yield an integer optimum right away.

### Define Separation Method

```python
import itertools
def find_minimum_cut_partition(instance, sol):
    G_flow = instance.get_graph()
    for i,j in G_flow.edges():
            if (i,j) in sol:
                G_flow[i][j]['capacity'] = sol[i,j]
            else:
                G_flow[i][j]['capacity'] = 0
    for t in G_flow.nodes() - {1}:
        cut_value, S = nx.minimum_cut(G_flow, 1, t, capacity='capacity')
        if cut_value < 2:
            return S[0]
    return set() #no cut with value < 2 has been found
```

### Subtour Elimination Callback
**Task 2:** In the following callback, fill in the code for adding the correct subtour elimination inequalities.

```python
def subtour_elimination_callback(model, where):
    if where == grb.GRB.Callback.MIPSOL:
        sol = model.cbGetSolution(model._vars)
        S = find_subtour_set(model._instance.get_nodes(), [(i,j) for i,j in model._vars.keys() if sol[i,j] > 0.9])
        if len(S) < model._instance.dimension:
            cut_edges = [(i,j) for i,j in model._vars.keys() if ((i in S and j not in S) or (i not in S and j in S))]
            model.cbLazy(sum(model._vars[i,j] for i,j in cut_edges) >= 2)
    elif where == grb.GRB.Callback.MIPNODE and model.cbGet(grb.GRB.Callback.MIPNODE_STATUS) == grb.GRB.Status.OPTIMAL:
        sol = model.cbGetNodeRel(model._vars)
        S = find_minimum_cut_partition(model._instance, sol)
        if 0 < len(S) < model._instance.dimension:
            cut_edges = [(i,j) for i,j in model._vars.keys() if ((i in S and j not in S) or (i not in S and j in S))]
            model.cbLazy(sum(model._vars[i,j] for i,j in cut_edges) >= 2)
```

### Optimize

```python
model._vars = x
model._instance = instance
model.reset()
model.Params.lazyConstraints = 1
model.optimize(subtour_elimination_callback)
```

    Parameter lazyConstraints unchanged
       Value: 1  Min: 0  Max: 1  Default: 0
    Optimize a model with 16 rows, 120 columns and 240 nonzeros
    Variable types: 0 continuous, 120 integer (120 binary)
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      Objective range  [1e+00, 3e+01]
      Bounds range     [1e+00, 1e+00]
      RHS range        [2e+00, 2e+00]
    Presolve time: 0.00s
    Presolved: 16 rows, 120 columns, 240 nonzeros
    Variable types: 0 continuous, 120 integer (120 binary)

    Root relaxation: objective 6.800000e+01, 25 iterations, 0.00 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0   68.00000    0    6          -   68.00000      -     -    0s
         0     0   70.50000    0   10          -   70.50000      -     -    0s
    H    0     0                      72.0000000   70.50000  2.08%     -    0s
    H    0     0                      71.0000000   70.50000  0.70%     -    0s

    Cutting planes:
      Gomory: 1
      Zero half: 1
      Lazy constraints: 10

    Explored 1 nodes (42 simplex iterations) in 0.03 seconds
    Thread count was 8 (of 8 available processors)

    Solution count 2: 71 72

    Optimal solution found (tolerance 1.00e-04)
    Best objective 7.100000000000e+01, best bound 7.100000000000e+01, gap 0.0000%

### Results

```python
model.ObjVal
```

    71.0

```python
solution_edges = [(i,j) for i,j in x.keys() if x[i,j].x > 0.9]
solution_edges
```

    [(1, 3),
     (1, 16),
     (2, 3),
     (2, 4),
     (4, 8),
     (5, 6),
     (5, 11),
     (6, 7),
     (7, 14),
     (8, 15),
     (9, 10),
     (9, 11),
     (10, 12),
     (12, 13),
     (13, 16),
     (14, 15)]

```python
if instance.is_depictable():
    pos = {i: instance.get_display(i) for i in instance.get_nodes()}
else:
    pos = nx.drawing.layout.spring_layout(G)
nx.draw_networkx_nodes(G, pos, node_color='#66a3ff', node_size=500)
nx.draw_networkx_labels(G, pos, font_weight='bold' )
nx.draw_networkx_edges(G, pos,  edge_color='#e6e6e6')
nx.draw_networkx_edges(G, pos,  edgelist=solution_edges, edge_color='#ffa31a', width=4)
```

### Comparison
**Task 3:** Compare the two different callbacks with respect to running time and number of cuts added. Try different TSPLIB instances. Compare this formulation to MTZ.

```python

```