# 一、什么是EM算法
EM（Expectation-Maximum）算法也称期望最大化算法，曾入选“数据挖掘十大算法”中，可见EM算法在机器学习、数据挖掘中的影响力。EM算法是最常见的隐变量估计方法，在机器学习中有极为广泛的用途，例如常被用来学习高斯混合模型（Gaussian mixture model，简称GMM）的参数；隐式马尔科夫算法（HMM）、LDA主题模型的变分推断等等。

# 二、EM算法原理

EM算法是一种迭代优化策略，由于它的计算方法中每一次迭代都分两步，其中一个为期望步（E步），另一个为极大步（M步），所以算法被称为EM算法（Expectation-Maximization Algorithm）。EM算法受到缺失思想影响，最初是为了解决数据缺失情况下的参数估计问题，其算法基础和收敛有效性等问题在Dempster、Laird和Rubin三人于1977年所做的文章《Maximum likelihood from incomplete data via the EM algorithm》中给出了详细的阐述。其基本思想是：首先根据己经给出的观测数据，估计出模型参数的值；然后再依据上一步估计出的参数值估计缺失数据的值，再根据估计出的缺失数据加上之前己经观测到的数据重新再对参数值进行估计，然后反复迭代，直至最后收敛，迭代结束。

# 三、EM算法公式推导

对于n个样本观察数据$x=\left(x_{1}, x_{2}, \ldots x_{n}\right)$，找出样本的模型参数θ, 极大化模型分布的对数似然函数如下：
$$\hat{\theta}=\operatorname{argmax} \sum_{i=1}^{n} \log p\left(x_{i} ; \theta\right)$$

> **极大似然估计** \
> 假设有一枚不均匀的硬币，抛10次，8次正面，估计抛硬币出现正面的概率。\
> 凭感觉就可以知道这个概率是0.8，其实这就是极大似然的思想。下面我们来细化我们的思路。\
> 假定抛出正面的概率为$\theta$，那么抛出如题所述的这10次结果的概率为：\
> $$L(\theta)=P(8正\mid \theta)=\theta^{8} \cdot(1-\theta)^{2}$$ \
> 这个L就是似然函数。为了方便求导，我们对L取对数，得到对数似然函数LL，然后令LL的导数等于0，解得最优化的$\theta$。\
> $$L L(\theta)=\log L(\theta)=8 \log \theta+2 \log (1-\theta)$$ \
> $$\frac{d L L}{d \theta}=\frac{8}{\theta}-\frac{2}{1-\theta}=0 \Longrightarrow \theta=0.8$$ \
> 简单来说，每一组参数，都会对应一个观测结果出现的概率。极大似然估计，就是找出使得观测结果出现的概率最大的参数。

现实中$\theta$可能是多个分布的参数，也就是说这个$x$的分布并不是只属于一个分布的，而是多个分布的组合，例如混合高斯模型。如果是多个分布的参数，那么利用分别求偏导再令偏导等于0，去求参数值的方法就不好求了。我们需要引入隐含变量来简化问题，下面就是为了简化问题，而使用的方法，是别人反复研究的成果。

如果我们得到的观察数据有未观察到的隐含数据$z=\left(z_{1}, z_{2}, \ldots, z_{n}\right)$，即上文中每个样本属于哪个分布是未知的，此时我们极大化模型分布的对数似然函数如下：
$$\hat{\theta}=\operatorname{argmax} \sum_{i=1}^{n} \log p\left(x_{i} ; \theta\right)=\operatorname{argmax} \sum_{i=1}^{n} \log \sum_{z_{i}} p\left(x_{i}, z_{i} ; \theta\right)$$

上面这个式子是根据$x_i$的边缘概率计算得来，引入$z$不影响结果，但仍然没有办法直接求出θ，再进一步进行变化，
$$\sum_{i=1}^{n} \log \sum_{z_{i}} p\left(x_{i}, z_{i} ; \theta\right)=\sum_{i=1}^{n} \log \sum_{z_{i}} Q_{i}\left(z_{i}\right) \frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}$$

$Q_{i}$是$z_{i}$的概率分布函数，所以$\sum_z Q_{i} (z_{i}) = 1$。

此时我们发现，有些规律可循了，$\sum_{z_{i}} Q_{i}\left(z_{i}\right) \frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}$是个期望，因为对于某离散随机变量X来说，其数学期望$\mathrm{E}[X]=\sum_{i} p_{i} x_{i}$。竟然是个期望我们就可以利用Jensen不等式进一步往下推导。

> **Jensen不等式**\
>（1）定义 \
> 设f是定义域为实数的函数，如果对所有的实数x，f(x)的二阶导数都大于0，那么f是凸函数。\
> Jensen不等式定义如下：\
> 如果f是凸函数，X是随机变量，那么：$E[f(X)] \geq f(E[X])$。当且仅当X是常量时，该式取等号。其中，E(X)表示X的数学期望。\
> 注：Jensen不等式应用于凹函数时，不等号方向反向。当且仅当x是常量时，该不等式取等号。\
>（2）举例\
> \
>图中，实线f表示凸函数，X是随机变量，有0.5的概率是a，有0.5的概率是b。X的期望值就是a和b的中值，从图中可以看到$E[f(X)] \geq f(E[X])$成立。

那么
$$
\begin{align}
X&=\frac{p\left(x_{i}, z_{i};\theta\right)}{Q_{i}\left(z_{i}\right)} \\
f(E[X])&=\sum_{i=1}^{n} \log \sum_{z_{i}} Q_{i}\left(z_{i}\right)\frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}\\
E[f(X)]&=\sum p(x)*f(x)=\sum_{i=1}^{n} \sum_{z_{i}} Q_{i}\left(z_{i}\right)\log\frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}\\
\end{align}
$$

因为f(x)在这里是log函数，以10为底的函数，它是凹函数，当jensen不等式引用于凹函数时，需要取反。

>

所以
$$
\begin{align}
f(E[x])&>=E[f(x)]\\
即\sum_{i=1}^{n} \log \sum_{z_{i}} Q_{i}\left(z_{i}\right)\frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}&>=\sum_{i=1}^{n} \sum_{z_{i}} Q_{i}\left(z_{i}\right)\log\frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}\\
\end{align}
$$

到了这一步，我们发现仍然求不出最大值，这里只有个最小值。我们设前一个式子为$L(\theta)$，后一个式子为$J(z,Q)$，那么$L(\theta)>=J(z,Q)$。
那么我们可以通过不断的最大化这个下界J，来使得L(θ)不断提高，最终达到它的最大值。

上图首先固定θ，调整$Q(z)$使下界$J(z,Q)$上升至与$L(θ)$在此点θ处相等（绿色曲线到蓝色曲线），然后固定$Q(z)$，调整θ使下界$J(z,Q)$达到最大值（$θ_t$到$θ_{t+1}$），然后再固定θ，调整$Q(z)$……直到收敛到似然函数$L(θ)$的最大值处的θ*。这里有两个问题：什么时候下界$J(z,Q)$与$L(θ)$在此点θ处相等？为什么一定会收敛？

* **什么时候下界$𝐽(𝑧,𝑄)$与$𝐿(θ)$在此点θ处相等？**

在Jensen不等式中，当自变量X是常数的时候，等式成立。即：
$$\frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}=c, \quad c为常数 \quad \tag{1}$$

再推导下，由于$\sum_z Q_{i} z_{i} = 1$，则可以得到：分子的和等于c（分子分母都对所有$z_{i}$求和：多个等式分子分母相加不变，这个认为每个样例的两个概率比值都是c），则：
$$\sum_{z} p\left(x_{i}, z_{i} ; \theta\right)=c$$

再代入到(1)式子中，得到

$$Q_{i}\left(z_{i}\right)=\frac{p\left(x_{i}, z_{i} ; \theta\right)}{\sum_{z} p\left(x_{i}, z_{i} ; \theta\right)}=\frac{p\left(x_{i}, z_{i} ; \theta\right)}{p\left(x_{i} ; \theta\right)}=p\left(z_{i} \mid x_{i} ; \theta\right)$$

所以我们固定了$\theta$后，$Q_{i}\left(z_{i}\right)=p\left(z_{i} \mid x_{i} ; \theta\right)$时，能取到等号。也就是这个时候下界 $𝐽(𝑧,𝑄)$ 与 $𝐿(θ)$ 在此点θ处相等。

* **为什么一定会收敛？**

首先我们上面已经推出了：

$$\sum_{i=1}^{n} \log p\left(x_{i} ; \theta\right) = \sum_{i=1}^{n} \log \sum_{z_{i}} Q_{i}\left(z_{i}\right)\frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}>=\sum_{i=1}^{n} \sum_{z_{i}} Q_{i}\left(z_{i}\right)\log\frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}$$

要证明收敛性，我们只要证明：
$$\sum_{i=1}^{n} \log p\left(x_{i} ; \theta^{t+1}\right) >= \sum_{i=1}^{n} \log p\left(x_{i} ; \theta^t\right) 即可$$

令
$$L(\theta, \theta^{t}) = \sum\limits_{i=1}^n\sum\limits_{z^{(i)}}p( z^{(i)}|x^{(i)};\theta^{t})\log \frac{p(x^{(i)}， z^{(i)};\theta)}{p( z^{(i)}|x^{(i)};\theta^{t})} \tag{2}$$

$$H(\theta, \theta^{t}) =  \sum\limits_{i=1}^n\sum\limits_{z^{(i)}}p( z^{(i)}|x^{(i)};\theta^{t})\log \frac{p( z^{(i)}|x^{(i)};\theta)}{p( z^{(i)}|x^{(i)};\theta^{t})} \tag{3}$$

(2)-(3)得到:

$$\sum\limits_{i=1}^n \log p(x^{(i)};\theta) = L(\theta, \theta^{j}) - H(\theta, \theta^{j})$$

在上式中分别取𝜃为$𝜃^t$和$𝜃^{t+1}$，并相减得到：

$$\sum\limits_{i=1}^n \log p(x^{(i)};\theta^{t+1})  - \sum\limits_{i=1}^n \log p(x^{(i)};\theta^{t}) = [L(\theta^{t+1}, \theta^{t}) - L(\theta^{t}, \theta^{t}) ] -[H(\theta^{t+1}, \theta^{t}) - H(\theta^{t}, \theta^{t}) ]$$

要证明EM算法的收敛性，我们只需要证明上式的右边是非负的即可。

由于$𝜃^{t+1}$使得$𝐿(𝜃,𝜃^t)$极大，因此有:

$$L(\theta^{t+1}, \theta^{t}) - L(\theta^{t}, \theta^{t})  \geq 0$$

而对于第二部分：
$$
\begin{align}
H(\theta^{t+1}, \theta^{t}) - H(\theta^{t}, \theta^{t}) & = \sum\limits_{i=1}^n\sum\limits_{z^{(i)}}P( z^{(i)}|x^{(i)};\theta^{t})log\frac{P( z^{(i)}|x^{(i)};\theta^{t+1})}{P( z^{(i)}|x^{(i)};\theta^t)} \\
& \leq  \sum\limits_{i=1}^nlog(\sum\limits_{z^{(i)}}P( z^{(i)}|x^{(i)};\theta^{t})\frac{P( z^{(i)}|x^{(i)};\theta^{t+1})}{P( z^{(i)}|x^{(i)};\theta^t)}) 【依据jensen不等式】\\
& = \sum\limits_{i=1}^nlog(\sum\limits_{z^{(i)}}P( z^{(i)}|x^{(i)};\theta^{t+1})) = 0
\end{align}
$$

至此收敛性证明完成。

从上面的推导可以看出，EM算法可以保证收敛到一个稳定点，但是却不能保证收敛到全局的极大值点，因此它是局部最优的算法，当然，如果我们的优化目标$𝐿(𝜃,𝜃^t)$是凸的，则EM算法可以保证收敛到全局最大值，这点和梯度下降法这样的迭代算法相同。

# 四、算法流程总结

**输入：**观察到的数据$x=\left(x_{1}, x_{2}, \ldots x_{n}\right)$，联合分布 $p(x, z ; \theta)$，条件分布$p(z|x ; \theta)$，最大迭代次数T。

**算法步骤：**

（1）随机初始化模型参数θ的初值 $\theta_0$ 。

（2）t=1,2,...,T 开始EM算法迭代：

E步：计算联合分布的条件概率期望：

$$Q_{i}\left(z_{i}\right)=p\left(z_{i} \mid x_{i}, \theta_{t}\right)$$

$$l\left(\theta, \theta_{t}\right)=\sum_{i=1}^{n} \sum_{z_{i}} Q_{i}\left(z_{i}\right) \log \frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}$$

M步：极大化$l\left(\theta, \theta_{t}\right)$,得到$\theta_{t+1}$:

$$\theta_{t+1}=\operatorname{argmax}l\left(\theta, \theta_{t}\right)$$

如果$\theta_{t+1}$已经收敛，则算法结束。否则继续进行E步和M步进行迭代。

注意：其实$\theta_{t+1}=\operatorname{argmax}l\left(\theta, \theta_{t}\right)$还可以再进行简化

简化为：
$$
\begin{align}
l\left(\theta, \theta_{t}\right)&=\operatorname{argmax}\sum_{i=1}^{n} \sum_{z_{i}} Q_{i}\left(z_{i}\right) \log \frac{p\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)} \\
&=\operatorname{argmax}\sum_{i=1}^{n}\sum_{z_{i}} \left[ Q_{i}\left(z_{i}\right) \log p\left(x_{i}, z_{i} ; \theta \right) - Q_{i}\left(z_{i}\right) \log Q_{i}\left(z_{i}\right)\right] \\
&=\operatorname{argmax}\sum_{i=1}^{n}\sum_{z_{i}}Q_{i}\left(z_{i}\right) \log p\left(x_{i}, z_{i} ; \theta \right)\\
\end{align}
$$

因为$Q_{i}\left(z_{i}\right) \log Q_{i}\left(z_{i}\right)$是常数项，去掉对于求函数最大化时的$\theta$没有影响，也可以理解为常数项求导时的导数为0。

**输出：**模型参数θ。

# 五、EM实例

**三硬币问题(李航)**

假设有3枚硬币A、B、C，硬币正面出现的概率分别为$π、p、q$。进行如下抛币试验：先掷硬币A，根据其结果选择硬币B（当A正面）或硬币C（当A反面）；然后投掷选出的硬币，掷硬币的结果，出现正面记为1，出现反面记为0；独立重复进行N次试验。若N次(N=10)试验的观测结果为：

>1,1,0,1,0,0,1,0,1,1

假设只能观测到掷硬币的结果，无法观测到掷硬币的过程，问如何估计三枚硬币正面朝上的概率。

我们先来定义变量$\theta={\pi,p,q}$，套用EM算法公式：

$$Q_{i}\left(z_{i}\right)=P\left(z_{i} \mid x_{i}, \theta_{t}\right)$$

$$l\left(\theta, \theta_{t}\right)=\sum_{i=1}^{n} \sum_{z_{i}} Q_{i}\left(z_{i}\right) \log \frac{P\left(x_{i}, z_{i} ; \theta\right)}{Q_{i}\left(z_{i}\right)}$$

---
$$
\begin{align}
Q_{i}\left(z_{i}\right)&=P\left(z_{i} \mid x_{i}, \theta_{t}\right)\\
&=\frac{P\left(x_{i}, z_{i} ; \theta_t\right)}{\sum_z P\left(x_{i}, z_{i} ; \theta_t\right)}【这个计算关系上文中有】\\
&=\frac{P(x_i|z_i;\theta_t)P(z_i;\theta_t)}{\sum_z P(x_i|z_i;\theta_t)P(z_i;\theta_t)}
\end{align}
$$

因为$z_i=\{0,1\}$，所以$\sum_{z_i}Q(z_i)=Q(0)+Q(1)$，所以我们要计算出$Q(0)和Q(1)$。

$$Q_{i}\left(1\right)=\frac{P(x_i|1;\theta_t)P(1;\theta_t)}{\sum_z P(x_i|z_i;\theta_t)P(z_i;\theta_t)}$$

$$
\begin{align}
因为：P(1;\theta_t)&=\pi_t \\
P(x_i|1;\theta_t)&=p_t^{x_i} (1-p_t)^{1-x_i} \\
P(0;\theta_t)&=1-\pi_t \\
P(x_i|0;\theta_t)&=q_t^{x_i} (1-q_t)^{1-x_i} \\
所以：Q_{i}\left(1\right)&=\frac{\pi_t(p_t^{x_i} (1-p_t)^{1-x_i})}{\pi_t(p_t^{x_i} (1-p_t)^{1-x_i})+(1-\pi_t)(q_t^{x_i} (1-q_t)^{1-x_i})} \\
Q_{i}\left(0\right)&=1-Q_{i}\left(1\right) \\
\end{align}
$$

所以：
$$l(\theta,\theta_t)=\sum_{i=1}^n\left[Q_i(1)\log \frac{\pi(p^{x_i}(1-p)^{1-x_i})}{Q_i(1)} + Q_i(0)\log \frac{(1-\pi)(q^{x_i}(1-q)^{1-x_i})}{Q_i(0)}\right]$$

求每个变量的偏导：
$$
\begin{align}
\frac{\partial f}{\partial \pi}&=\sum_{i=1}^{n}[\frac{Q_i(1)}{\pi}+\frac{Q_i(1)(-1)}{1-\pi}]=\sum_{i=1}^{n} \frac{Q_i(1)-\pi}{\pi(1-\pi)} =\frac{Q_1(1)-\pi}{\pi(1-\pi)}+\frac{Q_2(1)-\pi}{\pi(1-\pi)}+...+\frac{Q_n(1)-\pi}{\pi(1-\pi)}=\frac{\sum_{i=1}^{n}Q_i-n\pi}{\pi(1-\pi)}= 0 \\
\frac{\partial f}{\partial p}&=\sum_{i=1}^{n} \frac{x_i-p}{p(1-p)} Q_i(1) = 0 \\
\frac{\partial f}{\partial q}&=\sum_{i=1}^{n} \frac{x_i-q}{q(1-q)}\left(1-Q_i(1)\right) = 0
\end{align}
$$

令偏导数为0，求得下一次变量的值：
$$
\begin{align}
\pi &=\frac{1}{n} \sum_{i=1}^{n} Q_i(1) \\
p &=\frac{\sum_{i=1}^{n} Q_i(1) x_i}{\sum_{i=1}^{n} Q_i(1)} \\
q &=\frac{\sum_{i=1}^{n}\left(1-Q_i(1)\right) x_i}{\sum_{i=1}^{n}\left(1-Q_i(1)\right)}\\
\end
{align}$$

**代码**

```python
import numpy as np

y = np.array([1,1,0,1,0,0,1,0,1,1])
N = len(y)
pi_n = 0.4
p_n = 0.6
q_n = 0.7
flag = 1
iter = 0
while flag:
    pi = pi_n
    p = p_n
    q = q_n
    Q = np.zeros(N)
    i = 0
    for n in y:
        t1 = pi*np.power(p,n)*np.power(1-p,1-n)
        t2 = (1-pi)*np.power(q,n)*np.power(1-q,1-n)
        Q[i] = t1/(t1+t2)
        i = i + 1

    pi_n = np.sum(Q)/N
    p_n  = np.sum(y*Q)/np.sum(Q)
    q_n  = np.sum(y*(1-Q))/np.sum(1-Q)
    print(('%1.4f %5.4f %5.4f') % (pi_n,p_n,q_n))
    iter = iter + 1
    if iter==2:
        flag = 0
```

    0.4064 0.5368 0.6432
    0.4064 0.5368 0.6432

**三硬币问题**

假设有3枚硬币A、B、C，硬币正面出现的概率分别为$π、p、q$。进行如下抛币试验：先掷硬币A，根据其结果选择硬币B（当A正面）或硬币C（当A反面）；然后投掷选出的硬币抛掷N次(N=10)，掷硬币的结果，出现正面记为1，出现反面记为0，，独立重复进行M次(M=5)试验；试验的观测结果为：

>1,0,0,0,1,1,0,1,0,1 \
>1,1,1,1,0,1,1,1,1,1 \
>1,0,1,1,1,1,1,0,1,1 \
>1,0,1,0,0,0,1,1,0,0 \
>0,1,1,1,0,1,1,1,0,1

假设只能观测到掷硬币的结果，无法观测到掷硬币的过程，问如何估计三枚硬币正面朝上的概率。

实质：相比于第一个三硬币问题(李航)，这个三硬币问题就相当于抛掷完A后，再抛掷B或C时，多抛掷几次，概率多乘几次而已。

我们先来定义变量$\theta={\pi,p,q}$，套用EM算法公式：

$$Q_{j}\left(z_{j}\right)=p\left(z_{j} \mid x_{ji}, \theta_{t}\right)$$

$$l\left(\theta, \theta_{t}\right)=\sum_{j=1}^m\sum_{i=1}^{n} \sum_{z_{j}} Q_{j}\left(z_{j}\right) \log \frac{p\left(x_{ji}, z_{j} ; \theta\right)}{Q_{j}\left(z_{j}\right)}$$

---
$$
\begin{align}
Q_{j}\left(z_{j}\right)&=P\left(z_{j} \mid x_{ji}, \theta_{t}\right)\\
&=\frac{P\left(x_{ji}, z_{j} ; \theta_t\right)}{\sum_z P\left(x_{ji}, z_{j} ; \theta_t\right)}【这个计算关系上文中有】\\
&=\frac{P(x_{ji}|z_j;\theta_t)P(z_j;\theta_t)}{\sum_z P(x_{ji}|z_j;\theta_t)P(z_j;\theta_t)}
\end{align}
$$

因为$z_j=\{0,1\}$，所以$\sum_{z_j}Q(z_j)=Q(0)+Q(1)$，所以我们要计算出$Q(0)和Q(1)$。

$$Q_{j}\left(1\right)=\frac{P(x_{ji}|1;\theta_t)P(1;\theta_t)}{\sum_z P(x_{ji}|z_j;\theta_t)P(z_j;\theta_t)}$$

$$
\begin{align}
因为：P(1;\theta_t)&=\pi_t \\
P(x_{ji}|1;\theta_t)&=\prod_{i=1}^n p_t^{x_{ji}} (1-p_t)^{1-x_{ji}} \\
P(0;\theta_t)&=1-\pi_t \\
P(x_{ji}|0;\theta_t)&=\prod_{i=1}^n q_t^{x_{ji}} (1-q_t)^{1-x_{ji}} \\
所以：Q_{j}\left(1\right)&=\frac{\pi_t\prod_{i=1}^n(p_t^{x_{ji}} (1-p_t)^{1-x_{ji}})}{\pi_t\prod_{i=1}^n(p_t^{x_{ji}} (1-p_t)^{1-x_{ji}})+(1-\pi_t)\prod_{i=1}^n(q_t^{x_{ji}} (1-q_t)^{1-x_{ji}})} \\
Q_{j}\left(0\right)&=1-Q_{j}\left(1\right) \\
\end{align}
$$

所以：
$$l(\theta,\theta_t)=\sum_{j=1}^m\sum_{i=1}^n\left[Q_j(1)\log \frac{\pi\prod_{i=1}^n (p^{x_{ji}}(1-p)^{1-x_{ji}})}{Q_j(1)} + Q_j(0)\log \frac{(1-\pi)\prod_{i=1}^n (q^{x_{ji}}(1-q)^{1-x_{ji}})}{Q_j(0)}\right]$$

求每个变量的偏导：
$$
\begin{align}
\frac{\partial f}{\partial \pi}&=\sum_{j=1}^m\sum_{i=1}^{n} \frac{Q_j(1)-\pi}{\pi(1-\pi)} =n(\frac{Q_1(1)-\pi}{\pi(1-\pi)})+n(\frac{Q_2(1)-\pi}{\pi(1-\pi)})+...+n(\frac{Q_m(1)-\pi}{\pi(1-\pi)})=n(\frac{\sum_{j=1}^m Q_j(1)-m\pi}{\pi(1-\pi)})= 0 \\
\frac{\partial f}{\partial p}&=\sum_{j=1}^m\sum_{i=1}^{n} \frac{x_{ji}-p}{p(1-p)} Q_j(1) = 0 \\
\frac{\partial f}{\partial q}&=\sum_{j=1}^m\sum_{i=1}^{n} \frac{x_{ji}-q}{q(1-q)}\left(1-Q_j(1)\right) = 0
\end{align}
$$

令偏导数为0，求得下一次变量的值：
$$
\begin{align}
\pi &=\frac{1}{m} \sum_{j=1}^m Q_j(1) \\
p &=\frac{\sum_{j=1}^m\sum_{i=1}^{n} Q_j(1) x_{ji}}{\sum_{j=1}^m\sum_{i=1}^{n} Q_j(1)} \\
q &=\frac{\sum_{j=1}^m \sum_{i=1}^{n}\left(1-Q_j(1)\right) x_{ji}}{\sum_{j=1}^m \sum_{i=1}^{n}\left(1-Q_j(1)\right)}\\
\end
{align}$$

```python
# -*- coding: utf-8 -*-
# @author : wanglei
# @date : 2021/4/16 3:58 PM
# @description :

import numpy as np

y = np.array([[1, 0, 0, 0, 1, 1, 0, 1, 0, 1],
              [1, 1, 1, 1, 0, 1, 1, 1, 1, 1],
              [1, 0, 1, 1, 1, 1, 1, 0, 1, 1],
              [1, 0, 1, 0, 0, 0, 1, 1, 0, 0],
              [0, 1, 1, 1, 0, 1, 1, 1, 0, 1]])
M, N = y.shape
print(M, N)  # 5，10
pi_n = 0.5
p_n = 0.5
q_n = 0.6
flag = 1
iter = 0
while flag:
    pi = pi_n
    p = p_n
    q = q_n
    Q = np.zeros(M)
    for j in range(M):
        s1 = 1
        s2 = 1
        for i in range(N):
            s1 *= np.power(p, y[j][i]) * np.power(1 - p, 1 - y[j][i])
            s2 *= np.power(q, y[j][i]) * np.power(1 - q, 1 - y[j][i])
        t1 = pi * s1
        t2 = (1 - pi) * s2
        Q[j] = t1 / (t1 + t2)

    pi_n = 0.5 #np.sum(Q)/M
    p_n = np.sum([np.sum(y[j] * Q[j]) for j in range(M)]) / np.sum(N*Q)
    q_n = np.sum([np.sum(y[j] * (1 - Q[j])) for j in range(M)]) / np.sum(N*(1-Q))
    print(('%1.4f %5.4f %5.4f') % (pi_n, p_n, q_n))
    iter = iter + 1
    if iter == 10:
        flag = 0
```

    5 10
    0.5000 0.5813 0.7130
    0.5000 0.5693 0.7453
    0.5000 0.5495 0.7681
    0.5000 0.5346 0.7832
    0.5000 0.5263 0.7911
    0.5000 0.5224 0.7945
    0.5000 0.5207 0.7959
    0.5000 0.5200 0.7965
    0.5000 0.5198 0.7967
    0.5000 0.5197 0.7967

```python
from sympy import *

x = symbols("x")# 符号x，自变量
y = ln(1-x) # 公式

dify = diff(y,x) #求导
print(dify)
```

    -1/(1 - x)


# Exotic Options

- ***Plain vanilla options***: standard call and put options.
- ***Exotic options***: options different from the standard call and put options.
    - Exotic options are *less liquid* than standard options, and often have discontinuous payoffs.
- ***Path-dependent options***: payoff depends on what the path of the stock price has been up to maturity (rather than only on the price at maturity).

And following the previous chapter, we assume the stock price following the Merton-Black-Scholes model, and the stock does not pay dividends.

$Remark\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\plim}{plim}
\newcommand{\ffrac}{\displaystyle \frac}
\newcommand{\d}[1]{\displaystyle{#1}}
\newcommand{\space}{\text{ }}
\newcommand{\bspace}{\;\;\;\;}
\newcommand{\bbspace}{\;\;\;\;\;\;\;\;}
\newcommand{\QQQ}{\boxed{?\:}}
\newcommand{\void}{\left.\right.}
\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}
\newcommand{\CB}[1]{\left\{ #1 \right\}}
\newcommand{\SB}[1]{\left[ #1 \right]}
\newcommand{\P}[1]{\left( #1 \right)}
\newcommand{\abs}[1]{\left| #1 \right|}
\newcommand{\norm}[1]{\left\| #1 \right\|}
\newcommand{\given}[1]{\left. #1 \right|}
\newcommand{\using}[1]{\stackrel{\mathrm{#1}}{=}}
\newcommand{\asim}{\overset{\text{a}}{\sim}}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\EE}{\mathbb{E}}
\newcommand{\II}{\mathbb{I}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\ZZ}{\mathbb{Z}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\PP}{\mathbb{P}}
\newcommand{\AcA}{\mathcal{A}}
\newcommand{\FcF}{\mathcal{F}}
\newcommand{\AsA}{\mathscr{A}}
\newcommand{\FsF}{\mathscr{F}}
\newcommand{\dd}{\mathrm{d}}
\newcommand{\I}[1]{\mathrm{I}\left( #1 \right)}
\newcommand{\N}[1]{\mathcal{N}\left( #1 \right)}
\newcommand{\Exp}[1]{\mathrm{E}\left[ #1 \right]}
\newcommand{\Var}[1]{\mathrm{Var}\left[ #1 \right]}
\newcommand{\Avar}[1]{\mathrm{Avar}\left[ #1 \right]}
\newcommand{\Cov}[1]{\mathrm{Cov}\left( #1 \right)}
\newcommand{\Corr}[1]{\mathrm{Corr}\left( #1 \right)}
\newcommand{\ExpH}{\mathrm{E}}
\newcommand{\VarH}{\mathrm{Var}}
\newcommand{\AVarH}{\mathrm{Avar}}
\newcommand{\CovH}{\mathrm{Cov}}
\newcommand{\CorrH}{\mathrm{Corr}}
\newcommand{\ow}{\text{otherwise}}
\newcommand{\wp}{\text{with probability }}
\newcommand{\FSD}{\text{FSD}}
\newcommand{\SSD}{\text{SSD}}$

>For the parameters in pricing an option, say, a European call, we'll write $c\P{t.S_t,T,K}$. And an intuitive proposition is that
>
>$$c\P{t,k\cdot S_t, T,k\cdot K} = k\cdot c\P{t,S_t,T,K}$$

## Forward Start Option
To buy a ***Forward start options*** requires the payment for today, but it starts to count at some future time. We'll foucus on one special case: A European call option with maturity $T$ and strike price $S_{t_{\void_1}}$, $t_1 < T$. The current time is zero.

The price of a forward start options is

$$\begin{align}
&\ExpH^\QQ\SB{e^{-r\P{T-0}}\P{S_T - S_{t_{\void_1}}}^+ \mid \FcF_0}\\
=\;& \ExpH^\QQ\SB{e^{-rt_{\void_1}} \ExpH^\QQ \SB{e^{-r\P{T-t_{\void_1}}}\P{S_T - S_{t_{\void_1}}}^+ \mid \FcF_{t_{\void_1}}} \mid \FcF_0}\\
=\;& \ExpH^\QQ_0\SB{c\P{t_1,S_{t_{\void_1}},T,S_{t_{\void_1}}} e^{-rt_{\void_1}}}\\
=\;& \ExpH^\QQ_0\SB{ \ffrac{S_{t_{\void_1}}}{e^{rt_{\void_1}}} c\P{t_1,\color{brown}{1},T,\color{brown}{1}}},\bspace \text{martingale}\\
=\;& S_{t_{\void_0}} \cdot c\P{t_1,1,T,1} = S_0c\P{t_1,1,T,1}\\
=\;& c\P{t_1,S_0,T,S_0}\\
=\;& S_0 \N{d_1'} - S_0 e^{-r\P{T-t_{\void_1}}}\N{d_2'}\\
d'_{1,2} = \;& \ffrac{1}{\sigma\sqrt{T-t_1}} \P{\log \ffrac{S_0}{S_0} + \P{r\pm\ffrac{\sigma^2}{2}}\P{T-t_1}} = \ffrac{\sqrt{T-t_1}}{\sigma} \P{r\pm \ffrac{\sigma^2}{2}}
\end{align}$$

$Remark$

>Must be at the money state. And this price, the price of the forward start at-the-money call option starting at the future date $t_1$, s the same as the price of the regular at-the-money call option with time to maturity $T-t_1$.

## Binary or Digital Options

A binary option (or digital option) pays $1$ if the asset price ends up above a given strike price $K$, and $0$ dollars otherwise. It's payoff is $\begin{cases}
1,&\text{if } S_T>K\\
0,&\ow
\end{cases}$

Now if the stock price follows the BS model, then the price of this binary options is

$$\ExpH^\QQ_t\SB{e^{-r\P{T-t}} \mathbf 1_{\CB{S_T>K}}} = e^{-r\P{T-t}}\N{d_2}$$

where $d_2$ has the same definition with that in BS model.

## Asian Options
The payoff of Asian options depends on the *average* of the asset price over the *lifetime* of the contract. The payoff has two definitions, that is

$$\P{S_{\text{avg}} - K}^+,\bspace S_{\text{avg}} = \begin{cases}
\ffrac{1}{T} \d{\int_0^T S_T \;\dd t},&\text{arithmetic Asian option}\\
\exp\CB{\ffrac{1}{T} \d{\int_0^T \log S_T \;\dd t}},&\text{geometric Asian option}
\end{cases}$$

Asian options can reduce the risk. And it's normally cheaper than European or American options. And about the two different calculation, for the geometric one, since the *geometric average* of a set of *lognormally distributed* $r.v.$ is also *lognormal*, we are able to find the analytical formulas for valuing geometric Asian Options.

## Barrier Options
A barrier option is one that is *activated* or *deactivated* if the asset price crosses a preset barrier. And the payoff is same with the vanillas'.

- **knock-in**s
    - The barrier is ***up-and-in*** if the option is only *activate* if the barrier is hit from below (the rising price)
    - The barrier is ***down-and-in*** if the option is only *activate* if the barrier is hit from above (the falling price)
- **knock-out**s
    - The barrier is ***up-and-out*** if the option will be *deactivated*  (become *worthless*) if the barrier is hit from below (the rising price)
    - The barrier is ***down-and-out*** if the option will be *deactivated* (become *worthless*) if the barrier is hit from above (the falling price)

**e.g.** **down-and-in call** option.

It has the same payoff with the vanilla call option, but only if the option is *activated*, when *asset price goes below a certain level* $L$ at some time during the life of the contract. Otherwise the payoff would be $0$. Namely,

$$\begin{cases}
\P{S_T - K}^+, &\text{if }\d{\min_{0\leq t\leq T} S_t \leq L}\\
0,&\ow
\end{cases}$$

And its price can be valued as

$$\ExpH^\QQ_t\SB{e^{-r\P{T-t}} \P{S_T- K}^+ \mathbf 1_{\CB{\min\limits_{0\leq t\leq T}\;\:S_t \leq L}}}$$

A useful relation is that $c_{\text{d&i}} = c - c_{\text{d&o}}$, since

$$\ExpH^\QQ_t\SB{e^{-r\P{T-t}} \P{S_T- K}^+ \mathbf 1_{\CB{\min\limits_{0\leq t\leq T}\;\:S_t \leq L}}} + \ExpH^\QQ_t\SB{e^{-r\P{T-t}} \P{S_T- K}^+ \mathbf 1_{\CB{\min\limits_{0\leq t\leq T}\;\:S_t > L}}} = \ExpH^\QQ_t\SB{e^{-r\P{T-t}} \P{S_T- K}^+}$$

Besides, they cost less than their plain vanilla options.

## Lookback Options
They are those whose payoff is depended on the maximum or minimum asset price reached during the life of the contract.

- ***floating strike***: the *optimal value* of the underlying asset’s price during the option life.
    - ***floating lookback call option***: with payoff $S_T - \d{\min_{0 \leq t \leq T}}S_t$
    - ***floating lookback put option***: with payoff $\d{\max_{0 \leq t \leq T}}S_t - S_T$
- ***fixed strike***: $K$, while the price will be considered NOT floating
    - ***fixed lookback call option***: with payoff $\big(\d{\max_{0 \leq t \leq T}}S_t - K\big)^+$
    - ***fixed lookback put option***: with payoff $\big(K - \d{\min_{0 \leq t \leq T}}S_t\big)^+$

## Exchange options
The holder has the right to exchange asset $S_t^\P{1}$ for $S_t^\P{2}$ and thus its payoff is $\P{S_T^\P{2} - S_T^\P 1}^+$. There're two method to price this, and the first is baesd on the **Two Risky Assets model**

### Two Risky Assets model
Now $\dd S_t^\P 1 = S_t^\P 1 \SB{\mu_1 \dd t + \sigma_1 \dd W_t^\P 1}$ and $\dd S_t^\P 2 = S_t^\P 2 \SB{\mu_2 \dd t + \sigma_2 \dd W_t^\P 2}$ and suppose that $W_t^\P 1$ and $W_t^\P 2$ are Brownian motions with instantaneous correlation $\rho$ such that $\Exp{W_t^\P 1W_t^\P 2} = \rho t$, $\forall\: t> 0$.

### Two-Dimensional Itô's rule
The $\CB{X_t}$ and $\CB{Y_t}$ are two Itô processes:

$$\begin{align}
\dd X_t &= \mu_X \dd t + \sigma_{X,1}\dd W_t^\P 1 + \sigma_{X,2} \dd W_t^\P 2\\
\dd Y_t &= \mu_Y \dd t + \sigma_{Y,1}\dd W_t^\P 1 + \sigma_{Y,2} \dd W_t^\P 2
\end{align}$$

Then combine the two we have the Itô's formula like

$$\dd f\P{t,X_t,Y_t} = \ffrac{\partial f}{\partial t}\dd t +\ffrac{\partial f}{\partial X}\dd X_t +\ffrac{\partial f}{\partial Y}\dd Y_t + \ffrac{1}{2}\ffrac{\partial^2 f}{\partial X^2}\P{\dd X_t}^2 + \ffrac{1}{2}\ffrac{\partial^2 f}{\partial Y^2}\P{\dd Y_t}^2 + \color{brown}{\ffrac{1}{2}\ffrac{\partial^2 f}{\partial X\partial Y}\P{\dd X_t}\P{\dd Y_t}}$$

### Product Rule
$$\dd \P{X_t Y_t} = X_t \dd Y_t + Y_t \dd X_t + \color{brown}{\dd X_t \dd Y_t}$$

And here $W_t^\P 1$ and $W_t^\P 2$ are Brownian motions with correlation $\rho$, then

$$\dd W_t^\P 1 \dd W_t^\P 2 = \rho \dd t$$

### Dynamic Replication (No Arbitrage)
We now use dynamic replication to price the option. The portfolio process is now two-dimensional $\pi_t = \P{\pi_t^\P 1,\pi_t^\P2}$.

And the wealth process satisfies

$$\dd V_t = \ffrac{V_t - \P{\pi_t^\P 1 + \pi_t^\P2}}{B_t} \dd B_t + \ffrac{\pi_t^\P1}{S_t^\P1}\dd S_t^\P1 + \ffrac{\pi_t^\P2}{S_t^\P2}\dd S_t^\P2$$

$$\dd V_t = \P{rV_t + \pi_t^\P1\P{\mu_1 - r} + \pi_t^\P2\P{\mu_2 - r}}\dd t + \pi_t^\P1\sigma_1 \dd W_t^\P1 + \pi_t^\P2\sigma_2 \dd W_t^\P2$$

Let $c\P{t, S_t^\P1,S_t^\P2}$ denote the price of the exchange option at time $t$. Using **two-dimensional Itô's rule** yields

$$\begin{align}\dd c &= \ffrac{\partial c}{\partial t}\dd t + \ffrac{\partial c}{\partial S_t^\P 1} \dd S_t^\P 1 + \ffrac{\partial c}{\partial S_t^\P 2} \dd S_t^\P 2 + \ffrac{1}{2}\ffrac{\partial^2 c}{\partial S_t^\P1\partial S_t^\P1} \P{\dd S_t^\P 1}^2 + \ffrac{1}{2}\ffrac{\partial^2 c}{\partial S_t^\P2\partial S_t^\P2} \P{\dd S_t^\P 2}^2 + \color{brown}{\ffrac{1}{2}\ffrac{\partial^2 c}{\partial S_t^\P1\partial S_t^\P2}\dd S_t^\P 1\dd S_t^\P 2}\\
&= \ffrac{\partial c}{\partial t}\dd t + \ffrac{\partial c}{\partial S_t^\P 1} S_t^\P 1 \SB{\mu_1 \dd t + \sigma_1 \dd W_t^\P 1} + \ffrac{\partial c}{\partial S_t^\P 2} S_t^\P 2 \SB{\mu_2 \dd t + \sigma_2 \dd W_t^\P 2}\\
&\bspace + \ffrac{1}{2}\ffrac{\partial^2 c}{\partial S_t^\P1\partial S_t^\P1} \P{S_t^\P 1}^2 \SB{\mu_1 \dd t + \sigma_1 \dd W_t^\P 1}^2 + \ffrac{1}{2}\ffrac{\partial^2 c}{\partial S_t^\P2\partial S_t^\P2} \P{S_t^\P 2}^2 \SB{\mu_2 \dd t + \sigma_2 \dd W_t^\P 2}^2\\
&\bspace + \color{brown}{\ffrac{1}{2}\ffrac{\partial^2 c}{\partial S_t^\P1\partial S_t^\P2}S_t^\P 1 \SB{\mu_1 \dd t + \sigma_1 \dd W_t^\P 1}S_t^\P 2 \SB{\mu_2 \dd t + \sigma_2 \dd W_t^\P 2}}\\
&\bbspace\text{since } \dd t\dd t = 0, \dd W_t^\P 1 \dd W_t^\P 1 = \dd t, \dd W_t^\P 1 \dd t = 0, \text{ and } \color{brown}{\dd W_t^\P 1 \dd W_t^\P 2 = \rho \dd t}\\[0.7em]
&= \Big[ \ffrac{\partial c}{\partial t} + \mu_1 S_t^\P 1 \ffrac{\partial c}{\partial S_t^\P 1} + \mu_2 S_t^\P 2 \ffrac{\partial c}{\partial S_t^\P 2} + \ffrac{1}{2}\P{\sigma_1S_t^\P1}^2 \ffrac{\partial^2 c}{\partial S_t^\P1\partial S_t^\P1} + \ffrac{1}{2}\P{\sigma_2S_t^\P2}^2 \ffrac{\partial^2 c}{\partial S_t^\P2\partial S_t^\P2}\\
&\bspace+ \color{brown}{\rho \sigma_1 \sigma_2 S_t^\P1 S_t^\P2 \ffrac{\partial^2 c}{\partial S_t^\P1\partial S_t^\P2}}\Big]\dd t + \sigma_1 S_t^\P1 \ffrac{\partial c}{\partial S_t^\P1} \dd W_t^\P 1 + \sigma_2 S_t^\P2 \ffrac{\partial c}{\partial S_t^\P2} \dd W_t^\P 2
\end{align}$$

Thus to replicate, $c\P{t,S_t^\P1,S_t^\P2} = V_t$. And similar to the one-asset case, comparing the $\dd t$ term and the $\dd W_t^\P 1$, $\dd W_t^\P 2$ terms, we have the ***Black-Scholes equation for two assets***:

$$\begin{align}
\ffrac{\partial c}{\partial t} + \ffrac{1}{2}\P{\sigma_1S_t^\P1}^2 \ffrac{\partial^2 c}{\partial S_t^\P1\partial S_t^\P1} + \ffrac{1}{2}\P{\sigma_2S_t^\P2}^2 \ffrac{\partial^2 c}{\partial S_t^\P2\partial S_t^\P2} + \rho \sigma_1 \sigma_2 S_t^\P1 S_t^\P2 \ffrac{\partial^2 c}{\partial S_t^\P1\partial S_t^\P2}&\\
+ rS_t^\P1 \ffrac{\partial c}{\partial S_t^\P1} + rS_t^\P2 \ffrac{\partial c}{\partial S_t^\P2} - rc &= 0
\end{align}$$

with the **replicating portfolio** $\ffrac{\pi_t^\P1}{S_t^\P 1} = \ffrac{\partial c}{\partial S_t^\P1}$ and $\ffrac{\pi_t^\P2}{S_t^\P 2} = \ffrac{\partial c}{\partial S_t^\P2}$ and **boundary condition**: $c\P{T,S_T^\P 1,S_T^\P2} = \P{S_T^\P 2 - S_T^\P1}^+ = S_T^\P1\P{\ffrac{S_T^\P 2}{S_T^\P1} - 1}^+$.

And to solve this, a little conjecture would be that the price may be of the form $c\P{t,S_t^\P 1,S_t^\P2} = S_t^\P 1\cdot h\P{t,Z_t}$ for some function $h$ and $Z_t = \ffrac{S_t^\P 2}{S_t^\P 1}$. Thus

$$\begin{align}
0=\;&S_t^\P 1 \ffrac{\partial h}{\partial t} + \ffrac{1}{2} \P{\sigma_1 S_t^\P1}^2 \ffrac{\partial^2 c}{\partial Z_t^2}\P{\ffrac{\partial Z_t}{\partial S_t^\P1}}^2 + \ffrac{1}{2} \P{\sigma_2 S_t^\P2}^2 \ffrac{\partial^2 c}{\partial Z_t^2}\P{\ffrac{\partial Z_t}{\partial S_t^\P2}}^2\\
&\bspace+\rho \sigma_1 \sigma_2 S_t^\P1 S_t^\P2 \ffrac{\partial^2 c}{\partial Z_t^2}\ffrac{\partial Z_t^2}{\partial S_t^\P1\partial S_t^\P2} + rS_t^\P1 \ffrac{\partial c}{\partial Z_t}\ffrac{\partial Z_t}{\partial S_t^\P1} + rS_t^\P2 \ffrac{\partial c}{\partial Z_t}\ffrac{\partial Z_t}{\partial S_t^\P2} - rS_t^\P1 h\P{t,Z_t}\\
\Rightarrow 0=\;& h_t + \ffrac{1}{2} \P{\sigma_1^2 + \sigma_2^2 - 2 \rho\sigma_1\sigma_2}Z_t^2\ffrac{\partial^2 h}{\partial Z_t^2}
\end{align}$$

That's just the form of **Feynman–Kac Formula**, with its **boundary condition**: $h\P{T,Z_T} = \P{Z_T-1}^+$ (with "strike price" $K=1$). Comparing with the standard **Feynman–Kac Formula** we see that $r = 0$, $\mu=0$, and $\sigma = \sqrt{\sigma_1^2 + \sigma_2^2-2\rho\sigma_1\sigma_2}$. So the preceding *PDE and its boundary condition* is equivalent to the following *SDE* (stochastic differential equation)

$$\dd Z_t = \sqrt{\sigma_1^2 + \sigma_2^2-2\rho\sigma_1\sigma_2} \cdot Z_t \dd \hat{W}_t$$

where $\hat{W}_t$ is the standard brownian motion in the measure where $Z_t$ has no drift (which is hard to find and so omitted here). Then we can use the regular BS formula for $h$ and then multiply $S_t^\P1$:

$$c\P{t,S_t^{\P1},S_t^{\P2}} = S_t^{\P1}\P{Z_t\N{\hat d_1} - 1\cot e^0 N\P{\hat d_2}} = S_t^{\P2} \N{\hat d_1} - S_t^{\P1} \N{\hat d_2}$$

$$\begin{align}
\hat d_1 &= \ffrac{1}{\sigma\sqrt{T-t}} \SB{\log\P{Z_t}+ \ffrac{1}{2}\sigma^2\P{T-t}} = \ffrac{1}{\sigma\sqrt{T-t}} \SB{\log\P{\ffrac{S_t^{\P2}}{S_t^{\P1}}}+ \ffrac{1}{2}\sigma^2\P{T-t}}\\
\hat d_2 &= \ffrac{1}{\sigma\sqrt{T-t}} \SB{\log\P{\ffrac{S_t^{\P2}}{S_t^{\P1}}} - \ffrac{1}{2}\sigma^2\P{T-t}}
\end{align}$$

$Remark$

>Obtaining the formula is not the end. Here's a little extension. To price options on a *maximum* or a *minimum* of two assets, remember that
>
>$$\min\P{S_T^\P1,S_T^\P2} = S_T^\P2 - \P{S_T^\P2 - S_T^\P1}^+,\max\P{S_T^\P1,S_T^\P2} = S_T^\P2 + \P{S_T^\P1 - S_T^\P2}^+$$

And I really want to introduce another method to price the exchange option whose payoff is $\P{S_T^\P{2} - S_T^\P 1}^+$. First we write

$$\ExpH^\QQ_t\SB{e^{-r\P{T-t}}\P{S_T^\P2 - S_T^\P1}^+} = \ExpH^\QQ_t\SB{e^{-r\P{T-t}}S_T^\P1\P{\ffrac{S_T^\P2}{S_T^\P1} -1}^+}$$

Out purpose is to find a new measure that could bring $S_T^\P1$ out from the expectation. How? See a remark first

$Remark$

>How should we understand the conclusion from Girsanov's Theorem, that if $\left.\ffrac{\dd \tilde P}{\dd \PP}\right|_{\FcF_t} = L_t$, then $\ExpH^{\tilde P}\SB{\Phi\P t\mid \FcF_s} = \ExpH_s^{\tilde P}\SB{\Phi\P t} = \ExpH_s^{\PP}\SB{\Phi\P t \ffrac{L_t}{L_s}}$? Actually the $L_s$ in the *denominator* implies the "measure transform" of the given information, $\FcF_s$; and the $L_t$ in the *numerator* implies the "measure transform" of the calculated events, $\Phi\P t$. And after this, here's a little generalization
>
>$$\ExpH^\QQ_t\SB{g\P{S_T}} = \ExpH^{\QQ^*}_t\SB{g\P{S_T}\cdot \ffrac{\left.\ffrac{\dd \QQ}{\dd\QQ^*} \right|_{\FcF_T}}{\left.\ffrac{\dd \QQ}{\dd\QQ^*} \right|_{\FcF_t}}}$$
>
>Especially, when $t=0$, $\left.\ffrac{\dd \QQ}{\dd\QQ^*} \right|_{\FcF_0} = 1$ and thus
>
>$$\ExpH^\QQ_0\SB{g\P{S_T}} = \ExpH^{\QQ^*}_0\SB{g\P{S_T}\cdot \left.\ffrac{\dd \QQ}{\dd\QQ^*} \right|_{\FcF_T}}$$

Thinking inversely, we now define $L_t = \left.\ffrac{\dd Q^*}{\dd Q}\right|_{\FcF_t} = \ffrac{e^{-rt}S_t^\P 1}{S_0^\P1}$. We divide $S_0^\P1$ here to gurantee that $L_0 = 1$. Then

$$\begin{align}
\ExpH^\QQ_t\SB{e^{-r\P{T-t}}\P{S_T^\P2 - S_T^\P1}^+} &= \ExpH^\QQ_t\SB{e^{-r\P{T-t}}S_T^\P1\P{\ffrac{S_T^\P2}{S_T^\P1} -1}^+}\\
&= \ExpH^{\QQ^*}_t\SB{e^{-r\P{T-t}}S_T^\P1\P{\ffrac{S_T^\P2}{S_T^\P1} -1}^+ \cdot \ffrac{L_t}{L_T}}\\
&= \ExpH^{\QQ^*}_t\SB{e^{-r\P{T-t}}S_T^\P1\P{\ffrac{S_T^\P2}{S_T^\P1} -1}^+ \cdot e^{-r\P{T-t}}\ffrac{S_t^\P1}{S_T^\P1}}\\
&= S_t^\P1\ExpH^{\QQ^*}_t\SB{\P{\ffrac{S_T^\P2}{S_T^\P1} -1}^+}\\
\end{align}$$

## Alteration to the BS Model
### Merton-Black-Scholes Model with Deterministic Parameters
If the *volatility* is a deterministic function of time only, $\sigma = \sigma_t$; and the interest rate is also a  deterministic function of time, $r = r_t$. Then the BS formula at time $t$ is then modifies by

$$\sigma^2\P{T-t} \longrightarrow \int_t^T \sigma^2\P u\;\dd u,\bspace r\P{T-t}\longrightarrow \int_t^T r\P u\;\dd u\\
\Rightarrow S_T = S_0 \exp\CB{\int_t^T \P{r_s - \ffrac{1}{2}\sigma_s^2}\;\dd s + \int_t^T \sigma_s\;\dd \widetilde W_s}$$

### Stochastic Volatility Model
No enough time
### Merton’s Jump-Diffusion Model
Same

***

FinMath/Models and Pricing of Financial Derivatives/Chap_05_Exotic_Options.ipynb


# Start-to-Finish Example: Evolving Maxwell's Equations with Toriodal Dipole Field Initial Data, in Flat Spacetime and Curvilinear Coordinates

## Following the work of [Knapp, Walker & Baumgarte (2002)](https://arxiv.org/abs/gr-qc/0201051), we numerically implement the second version of Maxwell’s equations - System II (BSSN - like) in curvilinear coordinates.

## Author: Terrence Pierre Jacques and Zachariah Etienne
### Formatting improvements courtesy Brandon Clark

**Notebook Status:**  Validated

**Validation Notes:** This module has been validated to exhibit convergence to the exact solution for the electric field $E^i$ and vector potential $A^i$ at the expected order, *after a short numerical evolution of the initial data* (see [plots at bottom](#convergence)).

### NRPy+ Source Code for this module:
* [Maxwell/InitialData.py](../edit/Maxwell/InitialData.py); [\[**tutorial**\]](Tutorial-VacuumMaxwell_InitialData.ipynb): Purely toriodal dipole field initial data; sets all electromagnetic variables in the Cartesian basis
* [Maxwell/VacuumMaxwell_Flat_Evol_Curvilinear.py](../edit/Maxwell/VacuumMaxwell_Flat_Evol_Cartesian.py); [\[**tutorial**\]](Tutorial-Tutorial-VacuumMaxwell_Curvilinear_RHS-Rescaling.ipynb): Generates the right-hand sides for Maxwell's equations in curvilinear coordinates

## Introduction:
Here we use NRPy+ to generate the C source code necessary to set up initial data for a purely toriodal dipole field, as defined in [Knapp, Walker & Baumgarte (2002)](https://arxiv.org/abs/gr-qc/0201051). We then evolve the RHSs of Maxwell's equations using the [Method of Lines](https://reference.wolfram.com/language/tutorial/NDSolveMethodOfLines.html) time integration based on an [explicit Runge-Kutta fourth-order scheme](https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods) (RK4 is chosen below, but multiple options exist).

The entire algorithm is outlined as follows, with links to the relevant NRPy+ tutorial notebooks listed at each step:

1. Allocate memory for gridfunctions, including temporary storage for the Method of Lines time integration
    * [**NRPy+ tutorial on Method of Lines algorithm**](Tutorial-Method_of_Lines-C_Code_Generation.ipynb).
1. Set gridfunction values to initial data
    * [**NRPy+ tutorial on purely toriodal dipole field initial data**](Tutorial-VacuumMaxwell_InitialData.ipynb)
1. Next, integrate the initial data forward in time using the Method of Lines coupled to a Runge-Kutta explicit timestepping algorithm:
    1. At the start of each iteration in time, output the divergence constraint violation
        * [**NRPy+ tutorial on Maxwell's equations constraints**](Tutorial-VacuumMaxwell_Curvilinear_RHSs.ipynb)
    1. At each RK time substep, do the following:
        1. Evaluate RHS expressions
            * [**NRPy+ tutorial on Maxwell's equations right-hand sides**](Tutorial-VacuumMaxwell_Curvilinear_RHSs.ipynb)
        1. Apply Sommerfeld boundary conditions in curvilinear coordinates
            * [**NRPy+ tutorial on setting up Sommerfeld boundary conditions**](Tutorial-SommerfeldBoundaryCondition.ipynb)
1. Repeat above steps at two numerical resolutions to confirm convergence to zero.

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#initializenrpy): Set core NRPy+ parameters for numerical grids and reference metric
    1. [Step 1.a](#cfl) Output needed C code for finding the minimum proper distance between grid points, needed for [CFL](https://en.wikipedia.org/w/index.php?title=Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&oldid=806430673)-limited timestep
1. [Step 2](#mw): Generate symbolic expressions and output C code for evolving Maxwell's equations
    1. [Step 2.a](#mwid): Generate symbolic expressions for toroidal dipole field initial data
    1. [Step 2.b](#mwevol): Generate symbolic expressions for evolution equations
    1. [Step 2.c](#mwcon): Generate symbolic expressions for constraint equations
    1. [Step 2.d](#mwcart): Generate symbolic expressions for converting $A^i$ and $E^i$ to the Cartesian basis
    1. [Step 2.e](#mwccode): Output C codes for initial data and evolution equations
    1. [Step 2.f](#mwccode_con): Output C code for constraint equations
    1. [Step 2.g](#mwccode_cart): Output C code for converting $A^i$ and $E^i$ to Cartesian coordinates
    1. [Step 2.h](#mwccode_xzloop): Output C code for printing 2D data
    1. [Step 2.i](#cparams_rfm_and_domainsize): Output C codes needed for declaring and setting Cparameters; also set `free_parameters.h`
1. [Step 3](#bc_functs): Set up Sommerfeld boundary condition functions
1. [Step 4](#mainc): `Maxwell_Playground.c`: The Main C Code
1. [Step 5](#compileexec): Compile generated C codes & perform simulation of the propagating toriodal electromagnetic field
1. [Step 6](#visualize): Visualize the output!
    1. [Step 6.a](#installdownload): Install `scipy` and download `ffmpeg` if they are not yet installed/downloaded
    1. [Step 6.b](#genimages): Generate images for visualization animation
    1. [Step 6.c](#genvideo): Generate visualization animation
1. [Step 7](#convergence): Plot the numerical error, and confirm that it converges to zero with increasing numerical resolution (sampling)
1. [Step 8](#div_e): Comparison of Divergence Constrain Violation
1. [Step 9](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Set core NRPy+ parameters for numerical grids and reference metric \[Back to [top](#toc)\]
$$\label{initializenrpy}$$

```python
# Step P1: Import needed NRPy+ core modules:
from outputC import outputC,lhrh,outCfunction # NRPy+: Core C code output module
import finite_difference as fin  # NRPy+: Finite difference C code generation module
import NRPy_param_funcs as par   # NRPy+: Parameter interface
import grid as gri               # NRPy+: Functions having to do with numerical grids
import indexedexp as ixp         # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support
import reference_metric as rfm   # NRPy+: Reference metric support
import cmdline_helper as cmd     # NRPy+: Multi-platform Python command-line interface
import shutil, os, sys           # Standard Python modules for multiplatform OS-level functions, benchmarking

# Step P2: Create C code output directory:
Ccodesdir = os.path.join("MaxwellEvolCurvi_Playground_Ccodes/")
# First remove C code output directory if it exists
# Courtesy https://stackoverflow.com/questions/303200/how-do-i-remove-delete-a-folder-that-is-not-empty
shutil.rmtree(Ccodesdir, ignore_errors=True)
# Then create a fresh directory
cmd.mkdir(Ccodesdir)

# Step P3: Create executable output directory:
outdir = os.path.join(Ccodesdir,"output/")
cmd.mkdir(outdir)

# Step 1: Set the spatial dimension parameter
#         to three (BSSN is a 3+1 decomposition
#         of Einstein's equations), and then read
#         the parameter as DIM.
par.set_parval_from_str("grid::DIM",3)
DIM = par.parval_from_str("grid::DIM")

# Step 2: Set some core parameters, including CoordSystem MoL BoundaryCondition timestepping algorithm,
#                                 FD order, floating point precision, and CFL factor:
# Choices are: Spherical, SinhSpherical, SinhSphericalv2, Cylindrical, SinhCylindrical,
#              SymTP, SinhSymTP
CoordSystem     = "SinhCylindrical"

# Step  2.a: Set boundary conditions
# Current choices are QuadraticExtrapolation (quadratic polynomial extrapolation) or Sommerfeld
BoundaryCondition = "Sommerfeld"

# Step 2.b: Set defaults for Coordinate system parameters.
#           These are perhaps the most commonly adjusted parameters,
#           so we enable modifications at this high level.

# domain_size sets the default value for:
#   * Spherical's params.RMAX
#   * SinhSpherical*'s params.AMAX
#   * Cartesians*'s -params.{x,y,z}min & .{x,y,z}max
#   * Cylindrical's -params.ZMIN & .{Z,RHO}MAX
#   * SinhCylindrical's params.AMPL{RHO,Z}
#   * *SymTP's params.AMAX
domain_size     = 10.0 # Needed for all coordinate systems.

# sinh_width sets the default value for:
#   * SinhSpherical's params.SINHW
#   * SinhCylindrical's params.SINHW{RHO,Z}
#   * SinhSymTP's params.SINHWAA
sinh_width      = 0.4 # If Sinh* coordinates chosen

# sinhv2_const_dr sets the default value for:
#   * SinhSphericalv2's params.const_dr
#   * SinhCylindricalv2's params.const_d{rho,z}
sinhv2_const_dr = 0.05 # If Sinh*v2 coordinates chosen

# SymTP_bScale sets the default value for:
#   * SinhSymTP's params.bScale
SymTP_bScale    = 0.5 # If SymTP chosen

# Step 2.c: Set the order of spatial and temporal derivatives;
#           the core data type, and the CFL factor.
# RK_method choices include: Euler, "RK2 Heun", "RK2 MP", "RK2 Ralston", RK3, "RK3 Heun", "RK3 Ralston",
#              SSPRK3, RK4, DP5, DP5alt, CK5, DP6, L6, DP8
RK_method = "RK4"
FD_order  = 4        # Finite difference order: even numbers only, starting with 2. 12 is generally unstable
REAL      = "double" # Best to use double here.
default_CFL_FACTOR= 0.5      # (GETS OVERWRITTEN WHEN EXECUTED.) In pure axisymmetry (symmetry_axes = 2 below) 1.0 works fine. Otherwise 0.5 or lower.

# Step 3: Generate Runge-Kutta-based (RK-based) timestepping code.
#       3.A: Evaluate RHSs (RHS_string)
#       3.B: Apply boundary conditions (RHS_string)
import MoLtimestepping.C_Code_Generation as MoL
from MoLtimestepping.RK_Butcher_Table_Dictionary import Butcher_dict
RK_order  = Butcher_dict[RK_method][1]
cmd.mkdir(os.path.join(Ccodesdir,"MoLtimestepping/"))

RHS_string = "rhs_eval(&rfmstruct, &params, RK_INPUT_GFS, RK_OUTPUT_GFS);"

if BoundaryCondition == "QuadraticExtrapolation":
    # Extrapolation BCs are applied to the evolved gridfunctions themselves after the MoL update
    post_RHS_string = "apply_bcs_curvilinear(&params, &bcstruct, NUM_EVOL_GFS, evol_gf_parity, RK_OUTPUT_GFS);"
elif BoundaryCondition == "Sommerfeld":
    # Sommerfeld BCs are applied to the gridfunction RHSs directly
    RHS_string += "\n apply_bcs_sommerfeld(&params, xx, &bcstruct, NUM_EVOL_GFS, evol_gf_parity, RK_INPUT_GFS, RK_OUTPUT_GFS);"
    post_RHS_string = ""
else:
    print("Invalid choice of boundary condition")
    sys.exit(1)

MoL.MoL_C_Code_Generation(RK_method, RHS_string = RHS_string, post_RHS_string = post_RHS_string,
                          outdir = os.path.join(Ccodesdir,"MoLtimestepping/"))

# Step 4: Set the coordinate system for the numerical grid
par.set_parval_from_str("reference_metric::CoordSystem",CoordSystem)
rfm.reference_metric()

# Step 5: Set the finite differencing order to 4.
par.set_parval_from_str("finite_difference::FD_CENTDERIVS_ORDER",FD_order)

# Step 6: Copy SIMD/SIMD_intrinsics.h to $Ccodesdir/SIMD/SIMD_intrinsics.h
cmd.mkdir(os.path.join(Ccodesdir,"SIMD"))
shutil.copy(os.path.join("SIMD/")+"SIMD_intrinsics.h",os.path.join(Ccodesdir,"SIMD/"))
# par.set_parval_from_str("indexedexp::symmetry_axes","2")
```

    'MaxwellEvolCurvi_Playground_Ccodes/SIMD/SIMD_intrinsics.h'

## Step 1.a: Output needed C code for finding the minimum proper distance between grid points, needed for [CFL](https://en.wikipedia.org/w/index.php?title=Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&oldid=806430673)-limited timestep \[Back to [top](#toc)\]
$$\label{cfl}$$

In order for our explicit-timestepping numerical solution to Maxwell's equations to be stable, it must satisfy the [CFL](https://en.wikipedia.org/w/index.php?title=Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition&oldid=806430673) condition:
$$
\Delta t \le \frac{\min(ds_i)}{c},
$$
where $c$ is the wavespeed, and
$$ds_i = h_i \Delta x^i$$
is the proper distance between neighboring gridpoints in the $i$th direction (in 3D, there are 3 directions), $h_i$ is the $i$th reference metric scale factor, and $\Delta x^i$ is the uniform grid spacing in the $i$th direction:

```python
# Output the find_timestep() function to a C file.
rfm.out_timestep_func_to_file(os.path.join(Ccodesdir,"find_timestep.h"))
```

# Step 2: Generate symbolic expressions and output C code for evolving Maxwell's equations \[Back to [top](#toc)\]
$$\label{mw}$$

Here we read in the symbolic expressions from the NRPy+ [InitialData](../edit/Maxwell/InitialData.py) and [Maxwell/VacuumMaxwell_Flat_Evol_Curvilinear_rescaled](../edit/Maxwell/VacuumMaxwell_Flat_Evol_Curvilinear_rescaled.py) modules to define the initial data, evolution equations, and constraint equations.

## Step 2.a: Generate symbolic expressions for toroidal dipole field initial data \[Back to [top](#toc)\]
$$\label{mwid}$$

Here we use the NRPy+ [InitialData](../edit/Maxwell/InitialData.py) module, described in [this tutorial](Tutorial-VacuumMaxwell_InitialData.ipynb), to write initial data to the grid functions for both systems.

We define the rescaled quantities $a^i$ and $e^i$ in terms of $A^i$ and $E^i$ in curvilinear coordinates within the NRPy+ [InitialData](../edit/Maxwell/InitialData.py) module (see [this tutorial](Tutorial-VacuumMaxwell_formulation_Curvilinear.ipynb) for more detail);

\begin{align}
a^i &= \frac{A^i}{\text{ReU}[i]},\\ \\
e^i &= \frac{E^i}{\text{ReU}[i]}.
\end{align}

```python
import Maxwell.InitialData as mwid

# Set which system to use, which are defined in Maxwell/InitialData.py
par.set_parval_from_str("Maxwell.InitialData::System_to_use","System_II")

mwid.InitialData()
aidU = ixp.zerorank1()
eidU = ixp.zerorank1()
for i in range(DIM):
    aidU[i] = mwid.AidU[i]/rfm.ReU[i]
    eidU[i] = mwid.EidU[i]/rfm.ReU[i]

Maxwell_ID_SymbExpressions = [\
                 lhrh(lhs="*AU0_exact",rhs=aidU[0]),\
                 lhrh(lhs="*AU1_exact",rhs=aidU[1]),\
                 lhrh(lhs="*AU2_exact",rhs=aidU[2]),\
                 lhrh(lhs="*EU0_exact",rhs=eidU[0]),\
                 lhrh(lhs="*EU1_exact",rhs=eidU[1]),\
                 lhrh(lhs="*EU2_exact",rhs=eidU[2]),\
                 lhrh(lhs="*PSI_exact",rhs=mwid.psi_ID),\
                 lhrh(lhs="*GAMMA_exact",rhs=mwid.Gamma_ID)]

Maxwell_ID_CcodeKernel = fin.FD_outputC("returnstring", Maxwell_ID_SymbExpressions)

```

    Currently using System_II initial data

## Step 2.b: Generate symbolic expressions for evolution equations \[Back to [top](#toc)\]
$$\label{mwevol}$$

Here we use the NRPy+ [Maxwell/VacuumMaxwell_Flat_Evol_Curvilinear_rescaled](../edit/Maxwell/VacuumMaxwell_Flat_Evol_Curvilinear_rescaled.py) module, described in [this tutorial](Tutorial-VacuumMaxwell_Curvilinear_RHSs.ipynb), to ascribe the evolution equations to the grid functions.

```python
import Maxwell.VacuumMaxwell_Flat_Evol_Curvilinear_rescaled as rhs

# Set which system to use, which are defined in Maxwell/InitialData.py
par.set_parval_from_str("Maxwell.InitialData::System_to_use","System_II")

cmd.mkdir(os.path.join(Ccodesdir,"rfm_files/"))
par.set_parval_from_str("reference_metric::enable_rfm_precompute","True")
par.set_parval_from_str("reference_metric::rfm_precompute_Ccode_outdir",os.path.join(Ccodesdir,"rfm_files/"))

rhs.VacuumMaxwellRHSs_rescaled()

Maxwell_RHSs_SymbExpressions =  [\
                  lhrh(lhs=gri.gfaccess("rhs_gfs","aU0"),rhs=rhs.arhsU[0]),\
                  lhrh(lhs=gri.gfaccess("rhs_gfs","aU1"),rhs=rhs.arhsU[1]),\
                  lhrh(lhs=gri.gfaccess("rhs_gfs","aU2"),rhs=rhs.arhsU[2]),\
                  lhrh(lhs=gri.gfaccess("rhs_gfs","eU0"),rhs=rhs.erhsU[0]),\
                  lhrh(lhs=gri.gfaccess("rhs_gfs","eU1"),rhs=rhs.erhsU[1]),\
                  lhrh(lhs=gri.gfaccess("rhs_gfs","eU2"),rhs=rhs.erhsU[2]),\
                  lhrh(lhs=gri.gfaccess("rhs_gfs","psi"),rhs=rhs.psi_rhs),\
                  lhrh(lhs=gri.gfaccess("rhs_gfs","Gamma"),rhs=rhs.Gamma_rhs)]

par.set_parval_from_str("reference_metric::enable_rfm_precompute","False") # Reset to False to disable rfm_precompute.
rfm.ref_metric__hatted_quantities()

Maxwell_RHSs_string = fin.FD_outputC("returnstring",
                          Maxwell_RHSs_SymbExpressions,
                          params="enable_SIMD=True")

```

## Step 2.c: Generate symbolic expressions for constraint equations \[Back to [top](#toc)\]
$$\label{mwcon}$$

We now use the NRPy+ [Maxwell/VacuumMaxwell_Flat_Evol_Curvilinear_rescaled](../edit/Maxwell/VacuumMaxwell_Flat_Evol_Curvilinear_rescaled.py) module, described in [this tutorial](Tutorial-VacuumMaxwell_Curvilinear_RHSs.ipynb), to ascribe the constraint equations.

```python
C = gri.register_gridfunctions("AUX", "C")
G = gri.register_gridfunctions("AUX", "G")

Constraints_string = fin.FD_outputC("returnstring",
                                [lhrh(lhs=gri.gfaccess("aux_gfs", "C"), rhs=rhs.C),
                                 lhrh(lhs=gri.gfaccess("aux_gfs", "G"), rhs=rhs.G)],
                                params="outCverbose=False")
```

## Step 2.d: Generate symbolic expressions for converting $A^i$ and $E^i$ to the Cartesian basis \[Back to [top](#toc)\]
$$\label{mwcart}$$

We now use the NRPy+ [Maxwell/VacuumMaxwell_Flat_Evol_Curvilinear_rescaled](../edit/Maxwell/VacuumMaxwell_Flat_Evol_Curvilinear_rescaled.py) module, described in [this tutorial](Tutorial-VacuumMaxwell_Curvilinear_RHSs.ipynb), to ascribe the coordinate conversion, to make our convergence tests slightly easier.

```python
AUCart = ixp.register_gridfunctions_for_single_rank1("AUX", "AUCart")
EUCart = ixp.register_gridfunctions_for_single_rank1("AUX", "EUCart")

Cartesian_Vectors_string = fin.FD_outputC("returnstring",
                        [lhrh(lhs=gri.gfaccess("aux_gfs", "AUCart0"), rhs=rhs.AU_Cart[0]),
                         lhrh(lhs=gri.gfaccess("aux_gfs", "AUCart1"), rhs=rhs.AU_Cart[1]),
                         lhrh(lhs=gri.gfaccess("aux_gfs", "AUCart2"), rhs=rhs.AU_Cart[2]),
                         lhrh(lhs=gri.gfaccess("aux_gfs", "EUCart0"), rhs=rhs.EU_Cart[0]),
                         lhrh(lhs=gri.gfaccess("aux_gfs", "EUCart1"), rhs=rhs.EU_Cart[1]),
                         lhrh(lhs=gri.gfaccess("aux_gfs", "EUCart2"), rhs=rhs.EU_Cart[2])],
                        params="outCverbose=False")
```

## Step 2.e: Output C codes for initial data and evolution equations \[Back to [top](#toc)\]
$$\label{mwccode}$$

Next we write the C codes for the initial data and evolution equations to files, to be used later by our main C code.

```python
# Step 11: Generate all needed C functions
Part_P1_body = Maxwell_ID_CcodeKernel
desc="Part P1: Declare the function for the exact solution at a single point. time==0 corresponds to the initial data."
name="exact_solution_single_point"
outCfunction(
    outfile  = os.path.join(Ccodesdir,name+".h"), desc=desc, name=name,
    params   ="const REAL xx0,const REAL xx1,const REAL xx2,const paramstruct *restrict params,\
                                        REAL *EU0_exact, \
                                        REAL *EU1_exact, \
                                        REAL *EU2_exact, \
                                        REAL *AU0_exact, \
                                        REAL *AU1_exact, \
                                        REAL *AU2_exact, \
                                        REAL *PSI_exact,\
                                        REAL *GAMMA_exact",
    body     = Part_P1_body,
    loopopts = "")

desc="Part P2: Declare the function for the exact solution at all points. time==0 corresponds to the initial data."
name="exact_solution_all_points"
outCfunction(
    outfile  = os.path.join(Ccodesdir,name+".h"), desc=desc, name=name,
    params   ="const paramstruct *restrict params,REAL *restrict xx[3], REAL *restrict in_gfs",
    body     ="""
REAL xx0 = xx[0][i0]; REAL xx1 = xx[1][i1]; REAL xx2 = xx[2][i2];
exact_solution_single_point(xx0,xx1,xx2,params,&in_gfs[IDX4S(EU0GF,i0,i1,i2)],
                                               &in_gfs[IDX4S(EU1GF,i0,i1,i2)],
                                               &in_gfs[IDX4S(EU2GF,i0,i1,i2)],
                                               &in_gfs[IDX4S(AU0GF,i0,i1,i2)],
                                               &in_gfs[IDX4S(AU1GF,i0,i1,i2)],
                                               &in_gfs[IDX4S(AU2GF,i0,i1,i2)],
                                               &in_gfs[IDX4S(PSIGF,i0,i1,i2)],
                                               &in_gfs[IDX4S(GAMMAGF,i0,i1,i2)]);""",
    loopopts = "AllPoints")

Part_P3_body = Maxwell_RHSs_string

desc="Part P3: Declare the function to evaluate the RHSs of Maxwell's equations"
name="rhs_eval"
outCfunction(
    outfile  = os.path.join(Ccodesdir,name+".h"), desc=desc, name=name,
    params   ="""rfm_struct *restrict rfmstruct,const paramstruct *restrict params,
                 const REAL *restrict in_gfs, REAL *restrict rhs_gfs""",
    body     =Part_P3_body,
    loopopts = "InteriorPoints,enable_SIMD,enable_rfm_precompute")
```

    Output C function exact_solution_single_point() to file MaxwellEvolCurvi_Playground_Ccodes/exact_solution_single_point.h
    Output C function exact_solution_all_points() to file MaxwellEvolCurvi_Playground_Ccodes/exact_solution_all_points.h
    Output C function rhs_eval() to file MaxwellEvolCurvi_Playground_Ccodes/rhs_eval.h

## Step 2.f: Output C code for constraint equations \[Back to [top](#toc)\]
$$\label{mwccode_con}$$

Finally output the C code for evaluating the divergence constraint, described in [this tutorial](Tutorial-VacuumMaxwell_Curvilinear_RHSs.ipynb). In the absence of numerical error, this constraint should evaluate to zero, but due to numerical (typically truncation and roundoff) error it does not. We will therefore measure the divergence constraint violation to gauge the accuracy of our simulation, and ultimately determine whether errors are dominated by numerical finite differencing (truncation) error as expected. Specifically, we take the L2 norm of the constraint violation, via

\begin{align}
    \lVert C \rVert^2 &= \frac{\int C^2 d\mathcal{V}}{\int d\mathcal{V}}.
\end{align}

Numerically approximating this integral, in spherical coordinates for example, then gives us

\begin{align}
    \lVert C \rVert^2   &= \frac{\sum C^2 r^2 \sin^2 (\theta) dr d\theta d\phi}{\sum r^2 \sin^2 (\theta) dr d\theta d\phi}, \\ \\
    &= \frac{\sum C^2 r^2 \sin^2 (\theta)}{\sum r^2 \sin^2 (\theta) } , \\  \\
    &= \frac{\sum C^2 \ \sqrt{\text{det} \ \hat{\gamma}}}{\sum \sqrt{\text{det} \ \hat{\gamma}}},
\end{align}

where $\hat{\gamma}$ is the reference metric. Thus, along with the C code to calculate the constraints, we also print out the code required to evaluate $\sqrt{\text{det} \ \hat{\gamma}}$ at any given point.

```python
# Set up the C function for the calculating the constraints
Part_P4_body = Constraints_string
desc="Evaluate the constraints"
name="Constraints"
outCfunction(
    outfile  = os.path.join(Ccodesdir,name+".h"), desc=desc, name=name,
    params   = """rfm_struct *restrict rfmstruct,const paramstruct *restrict params,
                  REAL *restrict in_gfs, REAL *restrict aux_gfs""",
    body     = Part_P4_body,
    loopopts = "InteriorPoints,enable_rfm_precompute")

# intgrand to be used to calculate the L2 norm of the constraint
diagnostic_integrand_body = outputC(rfm.detgammahat,"*detg",filename='returnstring',
                                    params="includebraces=False")
desc="Evaluate the volume element at a specific point"
name="diagnostic_integrand"
outCfunction(
    outfile  = os.path.join(Ccodesdir,name+".h"), desc=desc, name=name,
    params   ="const REAL xx0,const REAL xx1,const REAL xx2,const paramstruct *restrict params, REAL *restrict detg",
    body     = diagnostic_integrand_body,
    loopopts = "")

```

    Output C function Constraints() to file MaxwellEvolCurvi_Playground_Ccodes/Constraints.h
    Output C function diagnostic_integrand() to file MaxwellEvolCurvi_Playground_Ccodes/diagnostic_integrand.h

## Step 2.g: Output C code for converting $A^i$ and $E^i$ to Cartesian coordinates \[Back to [top](#toc)\]
$$\label{mwccode_cart}$$

Here we write the C code for the coordinate transformation to Cartesian coordinates.

```python
desc="Convert EU and AU to Cartesian basis"
name="Cartesian_basis"
outCfunction(
    outfile  = os.path.join(Ccodesdir,name+".h"), desc=desc, name=name,
    params   = """rfm_struct *restrict rfmstruct,const paramstruct *restrict params, REAL *restrict xx[3],
                  REAL *restrict in_gfs, REAL *restrict aux_gfs""",
    body     = "REAL xx0 = xx[0][i0]; REAL xx1 = xx[1][i1]; REAL xx2 = xx[2][i2];\n"+Cartesian_Vectors_string,
    loopopts = "AllPoints, enable_rfm_precompute")
```

    Output C function Cartesian_basis() to file MaxwellEvolCurvi_Playground_Ccodes/Cartesian_basis.h

## Step 2.h: Output C code for printing 2D data \[Back to [top](#toc)\]
$$\label{mwccode_xzloop}$$

Here we output the neccesary C code to print out a 2D slice of the data in the xz-plane, using the `xz_loop` macro

```python
def xz_loop(CoordSystem):
    ret = """// xz-plane output for """ + CoordSystem + r""" coordinates:
#define LOOP_XZ_PLANE(ii, jj, kk) \
"""
    if "Spherical" in CoordSystem or  "SymTP" in CoordSystem:
        ret += r"""for (int i2 = 0; i2 < Nxx_plus_2NGHOSTS2; i2++) \
  if(i2 == NGHOSTS || i2 == Nxx_plus_2NGHOSTS2/2) \
    for (int i1 = 0; i1 < Nxx_plus_2NGHOSTS1; i1++) \
      for (int i0 = 0; i0 < Nxx_plus_2NGHOSTS0; i0++)
"""
    elif "Cylindrical" in CoordSystem:
        ret += r"""for (int i2 = 0; i2 < Nxx_plus_2NGHOSTS2; i2++) \
  for (int i1 = 0; i1 < Nxx_plus_2NGHOSTS1; i1++) \
    if(i1 == NGHOSTS || i1 == Nxx_plus_2NGHOSTS1/2) \
      for (int i0 = 0; i0 < Nxx_plus_2NGHOSTS0; i0++)
"""
    elif "Cartesian" in CoordSystem:
        ret += r"""for (int i2 = 0; i2 < Nxx_plus_2NGHOSTS2; i2++) \
  for (int i1 = Nxx_plus_2NGHOSTS1/2; i1 < Nxx_plus_2NGHOSTS1/2+1; i1++) \
    for (int i0 = 0; i0 < Nxx_plus_2NGHOSTS0; i0++)
"""
    return ret

with open(os.path.join(Ccodesdir,"xz_loop.h"),"w") as file:
    file.write(xz_loop(CoordSystem))
```

## Step 2.i: Output C codes needed for declaring and setting Cparameters; also set `free_parameters.h` \[Back to [top](#toc)\]
$$\label{cparams_rfm_and_domainsize}$$

Based on declared NRPy+ Cparameters, first we generate `declare_Cparameters_struct.h`, `set_Cparameters_default.h`, and `set_Cparameters[-SIMD].h`.

Then we output `free_parameters.h`, which sets initial data parameters, as well as grid domain & reference metric parameters, applying `domain_size` and `sinh_width`/`SymTP_bScale` (if applicable) as set above

```python
# Generate declare_Cparameters_struct.h, set_Cparameters_default.h, and set_Cparameters[-SIMD].h
par.generate_Cparameters_Ccodes(os.path.join(Ccodesdir))

# Set free_parameters.h
with open(os.path.join(Ccodesdir,"free_parameters.h"),"w") as file:
    file.write("""
// Set free-parameter values.
params.time = 0.0; // Initial simulation time time corresponds to exact solution at time=0.
params.amp = 1.0;
params.lam = 1.0;
params.wavespeed = 1.0;\n""")

# Append to $Ccodesdir/free_parameters.h reference metric parameters based on generic
#    domain_size,sinh_width,sinhv2_const_dr,SymTP_bScale,
#    parameters set above.
rfm.out_default_free_parameters_for_rfm(os.path.join(Ccodesdir,"free_parameters.h"),
                                        domain_size,sinh_width,sinhv2_const_dr,SymTP_bScale)

# Generate set_Nxx_dxx_invdx_params__and__xx.h:
rfm.set_Nxx_dxx_invdx_params__and__xx_h(Ccodesdir, grid_centering="cell")

# Generate xx_to_Cart.h, which contains xx_to_Cart() for
#  (the mapping from xx->Cartesian) for the chosen CoordSystem:
rfm.xx_to_Cart_h("xx_to_Cart","./set_Cparameters.h",os.path.join(Ccodesdir,"xx_to_Cart.h"))

# Step 3.d.v: Generate declare_Cparameters_struct.h, set_Cparameters_default.h, and set_Cparameters[-SIMD].h
par.generate_Cparameters_Ccodes(os.path.join(Ccodesdir))
```

# Step 3: Set up Sommerfeld boundary condition for Cartesian coordinate system \[Back to [top](#toc)\]
$$\label{bc_functs}$$

Next we output the C code necessary to implement the Sommerfeld boundary condition in Cartesian coordinates, [as documented in the corresponding NRPy+ tutorial notebook](Tutorial-SommerfeldBoundaryCondition.ipynb)

```python
import CurviBoundaryConditions.CurviBoundaryConditions as cbcs
cbcs.Set_up_CurviBoundaryConditions(os.path.join(Ccodesdir,"boundary_conditions/"),
                                   Cparamspath=os.path.join("../"),
                                   BoundaryCondition=BoundaryCondition)

if BoundaryCondition == "Sommerfeld":
    bcs = cbcs.sommerfeld_boundary_condition_class(fd_order=4,
                                             vars_radial_falloff_power_default=3,
                                             vars_speed_default=1.,
                                             vars_at_inf_default=0.)
    bcs.write_sommerfeld_file(Ccodesdir)
```

    Wrote to file "MaxwellEvolCurvi_Playground_Ccodes/boundary_conditions/parity_conditions_symbolic_dot_products.h"
    Evolved parity: ( Gamma:0, aU0:1, aU1:2, aU2:3, eU0:1, eU1:2, eU2:3, psi:0
        )
    Auxiliary parity: ( AUCart0:1, AUCart1:2, AUCart2:3, C:0, EUCart0:1,
        EUCart1:2, EUCart2:3, G:0 )

    Wrote to file "MaxwellEvolCurvi_Playground_Ccodes/boundary_conditions/EigenCoord_Cart_to_xx.h"

    Successfully generated Sommerfeld boundary condition C code

# Step 4: `Maxwell_Playground.c`: The Main C Code \[Back to [top](#toc)\]
$$\label{mainc}$$

```python
# Part P0: Define REAL, set the number of ghost cells NGHOSTS (from NRPy+'s FD_CENTDERIVS_ORDER),
#          and set the CFL_FACTOR (which can be overwritten at the command line)

with open(os.path.join(Ccodesdir,"Maxwell_Playground_REAL__NGHOSTS__CFL_FACTOR.h"), "w") as file:
    file.write("""
// Part P0.a: Set the number of ghost cells, from NRPy+'s FD_CENTDERIVS_ORDER
#define NGHOSTS """+str(int(FD_order/2)+1)+"""
// Part P0.b: Set the numerical precision (REAL) to double, ensuring all floating point
//            numbers are stored to at least ~16 significant digits
#define REAL """+REAL+"""
// Part P0.c: Set the CFL Factor. Can be overwritten at command line.
REAL CFL_FACTOR = """+str(default_CFL_FACTOR)+";")
```

```python
%%writefile $Ccodesdir/Maxwell_Playground.c

// Step P0: Define REAL and NGHOSTS; and declare CFL_FACTOR. This header is generated in NRPy+.
#include "Maxwell_Playground_REAL__NGHOSTS__CFL_FACTOR.h"

#include "rfm_files/rfm_struct__declare.h"

#include "declare_Cparameters_struct.h"

// All SIMD intrinsics used in SIMD-enabled C code loops are defined here:
#include "SIMD/SIMD_intrinsics.h"

// Step P1: Import needed header files
#include "stdio.h"
#include "stdlib.h"
#include "math.h"
#include "time.h"
#include "stdint.h" // Needed for Windows GCC 6.x compatibility
#ifndef M_PI
#define M_PI 3.141592653589793238462643383279502884L
#endif
#ifndef M_SQRT1_2
#define M_SQRT1_2 0.707106781186547524400844362104849039L
#endif

// Step P2: Declare the IDX4S(gf,i,j,k) macro, which enables us to store 4-dimensions of
//           data in a 1D array. In this case, consecutive values of "i"
//           (all other indices held to a fixed value) are consecutive in memory, where
//           consecutive values of "j" (fixing all other indices) are separated by
//           Nxx_plus_2NGHOSTS0 elements in memory. Similarly, consecutive values of
//           "k" are separated by Nxx_plus_2NGHOSTS0*Nxx_plus_2NGHOSTS1 in memory, etc.
#define IDX4S(g,i,j,k) \
( (i) + Nxx_plus_2NGHOSTS0 * ( (j) + Nxx_plus_2NGHOSTS1 * ( (k) + Nxx_plus_2NGHOSTS2 * (g) ) ) )
#define IDX4ptS(g,idx) ( (idx) + (Nxx_plus_2NGHOSTS0*Nxx_plus_2NGHOSTS1*Nxx_plus_2NGHOSTS2) * (g) )
#define IDX3S(i,j,k) ( (i) + Nxx_plus_2NGHOSTS0 * ( (j) + Nxx_plus_2NGHOSTS1 * ( (k) ) ) )
#define LOOP_REGION(i0min,i0max, i1min,i1max, i2min,i2max) \
  for(int i2=i2min;i2<i2max;i2++) for(int i1=i1min;i1<i1max;i1++) for(int i0=i0min;i0<i0max;i0++)
#define LOOP_ALL_GFS_GPS(ii) _Pragma("omp parallel for") \
  for(int (ii)=0;(ii)<Nxx_plus_2NGHOSTS_tot*NUM_EVOL_GFS;(ii)++)

// Step P3: Set UUGF and VVGF macros, as well as xx_to_Cart()
#include "boundary_conditions/gridfunction_defines.h"

// Step P4: Defines set_Nxx_dxx_invdx_params__and__xx(const int EigenCoord, const int Nxx[3],
//                                       paramstruct *restrict params, REAL *restrict xx[3]),
//          which sets params Nxx,Nxx_plus_2NGHOSTS,dxx,invdx, and xx[] for
//          the chosen Eigen-CoordSystem if EigenCoord==1, or
//          CoordSystem if EigenCoord==0.
#include "set_Nxx_dxx_invdx_params__and__xx.h"

// Step P5: Include basic functions needed to impose curvilinear
//          parity and boundary conditions.
#include "boundary_conditions/CurviBC_include_Cfunctions.h"

// Step P6: Find the CFL-constrained timestep
#include "find_timestep.h"

// Part P7: Declare the function for the exact solution at a single point. time==0 corresponds to the initial data.
#include "exact_solution_single_point.h"

// Part P8: Declare the function for the exact solution at all points. time==0 corresponds to the initial data.
#include "exact_solution_all_points.h"

// Step P9: Declare function for evaluating constraints (diagnostic)
#include "Constraints.h"

// Step P10: Declare rhs_eval function, which evaluates the RHSs of Maxwells equations
#include "rhs_eval.h"

// Step P11: Declare function to calculate det gamma_hat, used in calculating the L2 norm
#include "diagnostic_integrand.h"

// Step P12: Declare function to transform to the Cartesian basis
#include "Cartesian_basis.h"

// Step P13: Declare macro to print out data along the xz-plane
#include "xz_loop.h"

// Step P14: Define xx_to_Cart(const paramstruct *restrict params,
//                             REAL *restrict xx[3],
//                             const int i0,const int i1,const int i2,
//                             REAL xCart[3]),
//           which maps xx->Cartesian via
//           {xx[0][i0],xx[1][i1],xx[2][i2]}->{xCart[0],xCart[1],xCart[2]}

#include "xx_to_Cart.h"
// main() function:
// Step 0: Read command-line input, set up grid structure, allocate memory for gridfunctions, set up coordinates
// Step 1: Set up initial data to an exact solution
// Step 2: Start the timer, for keeping track of how fast the simulation is progressing.
// Step 3: Integrate the initial data forward in time using the chosen RK-like Method of
//         Lines timestepping algorithm, and output periodic simulation diagnostics
// Step 3.a: Output 2D data file periodically, for visualization
// Step 3.b: Step forward one timestep (t -> t+dt) in time using
//           chosen RK-like MoL timestepping algorithm
// Step 3.c: If t=t_final, output x and y components of evolution variables and the
//           constraint violation to 2D data file
// Step 3.d: Progress indicator printing to stderr
// Step 4: Free all allocated memory
int main(int argc, const char *argv[]) {
    paramstruct params;
#include "set_Cparameters_default.h"

    // Step 0a: Read command-line input, error out if nonconformant
    if((argc != 5 && argc != 6) || atoi(argv[1]) < NGHOSTS || atoi(argv[2]) < NGHOSTS || atoi(argv[3]) < 2 /* FIXME; allow for axisymmetric sims */) {
        fprintf(stderr,"Error: Expected three command-line arguments: ./Maxwell_Playground Nx0 Nx1 Nx2,\n");
        fprintf(stderr,"where Nx[0,1,2] is the number of grid points in the 0, 1, and 2 directions.\n");
        fprintf(stderr,"Nx[] MUST BE larger than NGHOSTS (= %d)\n",NGHOSTS);
        exit(1);
    }
    if(argc == 5) {
        CFL_FACTOR = strtod(argv[5],NULL);
        if(CFL_FACTOR > 0.5 && atoi(argv[3])!=2) {
            fprintf(stderr,"WARNING: CFL_FACTOR was set to %e, which is > 0.5.\n",CFL_FACTOR);
            fprintf(stderr,"         This will generally only be stable if the simulation is purely axisymmetric\n");
            fprintf(stderr,"         However, Nx2 was set to %d>2, which implies a non-axisymmetric simulation\n",atoi(argv[3]));
        }
    }
    // Step 0b: Set up numerical grid structure, first in space...
    const int Nxx[3] = { atoi(argv[1]), atoi(argv[2]), atoi(argv[3]) };
    if(Nxx[0]%2 != 0 || Nxx[1]%2 != 0 || Nxx[2]%2 != 0) {
        fprintf(stderr,"Error: Cannot guarantee a proper cell-centered grid if number of grid cells not set to even number.\n");
        fprintf(stderr,"       For example, in case of angular directions, proper symmetry zones will not exist.\n");
        exit(1);
    }

    // Step 0c: Set free parameters, overwriting Cparameters defaults
    //          by hand or with command-line input, as desired.
#include "free_parameters.h"

   // Step 0d: Uniform coordinate grids are stored to *xx[3]
    REAL *xx[3];
    // Step 0d.i: Set bcstruct
    bc_struct bcstruct;
    {
        int EigenCoord = 1;
        // Step 0d.ii: Call set_Nxx_dxx_invdx_params__and__xx(), which sets
        //             params Nxx,Nxx_plus_2NGHOSTS,dxx,invdx, and xx[] for the
        //             chosen Eigen-CoordSystem.
        set_Nxx_dxx_invdx_params__and__xx(EigenCoord, Nxx, &params, xx);
        // Step 0d.iii: Set Nxx_plus_2NGHOSTS_tot
#include "set_Cparameters-nopointer.h"
        const int Nxx_plus_2NGHOSTS_tot = Nxx_plus_2NGHOSTS0*Nxx_plus_2NGHOSTS1*Nxx_plus_2NGHOSTS2;
        // Step 0e: Find ghostzone mappings; set up bcstruct
#include "boundary_conditions/driver_bcstruct.h"
        // Step 0e.i: Free allocated space for xx[][] array
        for(int i=0;i<3;i++) free(xx[i]);
    }

    // Step 0f: Call set_Nxx_dxx_invdx_params__and__xx(), which sets
    //          params Nxx,Nxx_plus_2NGHOSTS,dxx,invdx, and xx[] for the
    //          chosen (non-Eigen) CoordSystem.
    int EigenCoord = 0;
    set_Nxx_dxx_invdx_params__and__xx(EigenCoord, Nxx, &params, xx);

    // Step 0g: Set all C parameters "blah" for params.blah, including
    //          Nxx_plus_2NGHOSTS0 = params.Nxx_plus_2NGHOSTS0, etc.
#include "set_Cparameters-nopointer.h"
    const int Nxx_plus_2NGHOSTS_tot = Nxx_plus_2NGHOSTS0*Nxx_plus_2NGHOSTS1*Nxx_plus_2NGHOSTS2;

    // Step 0h: Time coordinate parameters
    const REAL t_final = strtod(argv[4],NULL);

    // Step 0i: Set timestep based on smallest proper distance between gridpoints and CFL factor
    REAL dt = find_timestep(&params, xx);
    //fprintf(stderr,"# Timestep set to = %e\n",(double)dt);
    int N_final = (int)(t_final / dt + 0.5); // The number of points in time.
                                             // Add 0.5 to account for C rounding down
                                             // typecasts to integers.
    int output_every_N = (int)((REAL)N_final/800.0);
    if(output_every_N == 0) output_every_N = 1;

    // Step 0j: Error out if the number of auxiliary gridfunctions outnumber evolved gridfunctions.
    //              This is a limitation of the RK method. You are always welcome to declare & allocate
    //              additional gridfunctions by hand.
    if(NUM_AUX_GFS > NUM_EVOL_GFS) {
        fprintf(stderr,"Error: NUM_AUX_GFS > NUM_EVOL_GFS. Either reduce the number of auxiliary gridfunctions,\n");
        fprintf(stderr,"       or allocate (malloc) by hand storage for *diagnostic_output_gfs. \n");
        exit(1);
    }

    // Step 0k: Allocate memory for gridfunctions
#include "MoLtimestepping/RK_Allocate_Memory.h"
    REAL *restrict y_0_gfs = (REAL *)malloc(sizeof(REAL) * NUM_EVOL_GFS * Nxx_plus_2NGHOSTS_tot);
    REAL *restrict diagnostic_output_gfs_0 = (REAL *)malloc(sizeof(REAL) * NUM_AUX_GFS * Nxx_plus_2NGHOSTS_tot);

    // Step 0l: Set up precomputed reference metric arrays
    // Step 0l.i: Allocate space for precomputed reference metric arrays.
#include "rfm_files/rfm_struct__malloc.h"

    // Step 0l.ii: Define precomputed reference metric arrays.
    {
    #include "set_Cparameters-nopointer.h"
    #include "rfm_files/rfm_struct__define.h"
    }

    LOOP_ALL_GFS_GPS(i) {
     y_n_gfs[i] = 0.0/0.0;
}

    // Step 1: Set up initial data to be exact solution at time=0:
    params.time = 0.0;
    exact_solution_all_points(&params, xx, y_n_gfs);

    // Step 2: Start the timer, for keeping track of how fast the simulation is progressing.
#ifdef __linux__ // Use high-precision timer in Linux.
    struct timespec start, end;
    clock_gettime(CLOCK_REALTIME, &start);
#else     // Resort to low-resolution, standards-compliant timer in non-Linux OSs
    // http://www.cplusplus.com/reference/ctime/time/
    time_t start_timer,end_timer;
    time(&start_timer); // Resolution of one second...
#endif

    // Step 3: Integrate the initial data forward in time using the chosen RK-like Method of
    //         Lines timestepping algorithm, and output periodic simulation diagnostics
    for(int n=0;n<=N_final+1;n++) { // Main loop to progress forward in time.

       // At each timestep, set Constraints to NaN in the grid interior
        LOOP_REGION(NGHOSTS,NGHOSTS+Nxx0,
                    NGHOSTS,NGHOSTS+Nxx1,
                    NGHOSTS,NGHOSTS+Nxx2) {
                    const int idx = IDX3S(i0,i1,i2);
                    diagnostic_output_gfs[IDX4ptS(CGF,idx)] = 0.0/0.0;
            }

        // Step 3.a: Output 2D data file periodically, for visualization
        params.time = ((REAL)n)*dt;
        // Evaluate Divergence constraint violation
        Constraints(&rfmstruct, &params, y_n_gfs, diagnostic_output_gfs);
        // log_L2_Norm = log10( sqrt[Integral( [numerical - exact]^2 * dV)] )
        REAL L2Norm_sum_C = 0.;
        int sum = 0;
        LOOP_REGION(NGHOSTS,NGHOSTS+Nxx0,
                    NGHOSTS,NGHOSTS+Nxx1,
                    NGHOSTS,NGHOSTS+Nxx2) {
            const int idx = IDX3S(i0,i1,i2);
            double C = (double)diagnostic_output_gfs[IDX4ptS(CGF,idx)];
            REAL xx0 = xx[0][i0];
            REAL xx1 = xx[1][i1];
            REAL xx2 = xx[2][i2];
            REAL detghat; diagnostic_integrand(xx0, xx1, xx2, &params, &detghat);
            L2Norm_sum_C += C*C*sqrt(detghat);
            sum = sum + sqrt(detghat);
        }
        REAL L2Norm_C = sqrt(L2Norm_sum_C/(sum));
        printf("%e %.15e\n",params.time, L2Norm_C);

    // Step 3.a: Output 2D data file periodically, for visualization
       if(n%20 == 0) {
            exact_solution_all_points(&params, xx, y_0_gfs);
            Cartesian_basis(&rfmstruct, &params, xx, y_n_gfs, diagnostic_output_gfs);
            Cartesian_basis(&rfmstruct, &params, xx, y_0_gfs, diagnostic_output_gfs_0);
            char filename[100];
            sprintf(filename,"out%d-%08d.txt",Nxx[0],n);
            FILE *out2D = fopen(filename, "w");
            LOOP_XZ_PLANE(ii, jj, kk){
            REAL xCart[3];
            xx_to_Cart(&params,xx,i0,i1,i2,xCart);
            int idx = IDX3S(i0,i1,i2);
            REAL Ex_num  = (double)diagnostic_output_gfs[IDX4ptS(EUCART0GF,idx)];
            REAL Ey_num  = (double)diagnostic_output_gfs[IDX4ptS(EUCART1GF,idx)];
            REAL Ax_num  = (double)diagnostic_output_gfs[IDX4ptS(AUCART0GF,idx)];
            REAL Ay_num  = (double)diagnostic_output_gfs[IDX4ptS(AUCART1GF,idx)];
            double C = (double)diagnostic_output_gfs[IDX4ptS(CGF,idx)];

            fprintf(out2D,"%e %e %e %.15e %.15e %.15e %.15e %.15e\n", params.time,
                    xCart[0],xCart[2], Ex_num, Ey_num, Ax_num, Ay_num, C);
       }
        fclose(out2D);
    }

      if(n==N_final-1) {
            exact_solution_all_points(&params, xx, y_0_gfs);
            Cartesian_basis(&rfmstruct, &params, xx, y_n_gfs, diagnostic_output_gfs);
            Cartesian_basis(&rfmstruct, &params, xx, y_0_gfs, diagnostic_output_gfs_0);
            char filename[100];
            sprintf(filename,"out%d.txt",Nxx[0]);
            FILE *out2D = fopen(filename, "w");
            LOOP_XZ_PLANE(ii, jj, kk){
            REAL xCart[3];
            xx_to_Cart(&params,xx,i0,i1,i2,xCart);
            int idx = IDX3S(i0,i1,i2);
            REAL Ex_num  = (double)diagnostic_output_gfs[IDX4ptS(EUCART0GF,idx)];
            REAL Ey_num  = (double)diagnostic_output_gfs[IDX4ptS(EUCART1GF,idx)];

            REAL Ax_num  = (double)diagnostic_output_gfs[IDX4ptS(AUCART0GF,idx)];
            REAL Ay_num  = (double)diagnostic_output_gfs[IDX4ptS(AUCART1GF,idx)];

            REAL Ex_exact  = (double)diagnostic_output_gfs_0[IDX4ptS(EUCART0GF,idx)];
            REAL Ey_exact  = (double)diagnostic_output_gfs_0[IDX4ptS(EUCART1GF,idx)];

            REAL Ax_exact  = (double)diagnostic_output_gfs_0[IDX4ptS(AUCART0GF,idx)];
            REAL Ay_exact  = (double)diagnostic_output_gfs_0[IDX4ptS(AUCART1GF,idx)];

            REAL Ex__E_rel = log10(fabs(Ex_num - Ex_exact));
            REAL Ey__E_rel = log10(fabs(Ey_num - Ey_exact));
            REAL Ax__E_rel = log10(fabs(Ax_num - Ax_exact));
            REAL Ay__E_rel = log10(fabs(Ay_num - Ay_exact));

            fprintf(out2D,"%e %e %.15e %.15e %.15e %.15e\n",xCart[0],xCart[2],
                     Ex__E_rel, Ey__E_rel, Ax__E_rel, Ay__E_rel);
       }
        fclose(out2D);
    }
        // Step 3.b: Step forward one timestep (t -> t+dt) in time using
        //           chosen RK-like MoL timestepping algorithm
#include "MoLtimestepping/RK_MoL.h"

        // Step 3.d: Progress indicator printing to stderr

        // Step 3.d.i: Measure average time per iteration
#ifdef __linux__ // Use high-precision timer in Linux.
        clock_gettime(CLOCK_REALTIME, &end);
        const long long unsigned int time_in_ns = 1000000000L * (end.tv_sec - start.tv_sec) + end.tv_nsec - start.tv_nsec;
#else     // Resort to low-resolution, standards-compliant timer in non-Linux OSs
        time(&end_timer); // Resolution of one second...
        REAL time_in_ns = difftime(end_timer,start_timer)*1.0e9+0.5; // Round up to avoid divide-by-zero.
#endif
        const REAL s_per_iteration_avg = ((REAL)time_in_ns / (REAL)n) / 1.0e9;

        const int iterations_remaining = N_final - n;
        const REAL time_remaining_in_mins = s_per_iteration_avg * (REAL)iterations_remaining / 60.0;

        const REAL num_RHS_pt_evals = (REAL)(Nxx[0]*Nxx[1]*Nxx[2]) * 4.0 * (REAL)n; // 4 RHS evals per gridpoint for RK4
        const REAL RHS_pt_evals_per_sec = num_RHS_pt_evals / ((REAL)time_in_ns / 1.0e9);

        // Step 3.d.ii: Output simulation progress to stderr
        if(n % 10 == 0) {
            fprintf(stderr,"%c[2K", 27); // Clear the line
            fprintf(stderr,"It: %d t=%.2f dt=%.2e | %.1f%%; ETA %.0f s | t/h %.2f | gp/s %.2e\r",  // \r is carriage return, move cursor to the beginning of the line
                   n, n * (double)dt, (double)dt, (double)(100.0 * (REAL)n / (REAL)N_final),
                   (double)time_remaining_in_mins*60, (double)(dt * 3600.0 / s_per_iteration_avg), (double)RHS_pt_evals_per_sec);
            fflush(stderr); // Flush the stderr buffer
        } // End progress indicator if(n % 10 == 0)

    } // End main loop to progress forward in time.
    fprintf(stderr,"\n"); // Clear the final line of output from progress indicator.

    // Step 4: Free all allocated memory
#include "rfm_files/rfm_struct__freemem.h"
#include "boundary_conditions/bcstruct_freemem.h"
#include "MoLtimestepping/RK_Free_Memory.h"
    free(y_0_gfs);
    free(diagnostic_output_gfs_0);
    for(int i=0;i<3;i++) free(xx[i]);

    return 0;
}
```

    Writing MaxwellEvolCurvi_Playground_Ccodes//Maxwell_Playground.c

# Step 5: Compile generated C codes & perform simulation of the propagating toriodal electromagnetic field \[Back to [top](#toc)\]
$$\label{compileexec}$$

To aid in the cross-platform-compatible (with Windows, MacOS, & Linux) compilation and execution, we make use of `cmdline_helper` [(**Tutorial**)](Tutorial-cmdline_helper.ipynb).

```python
import cmdline_helper as cmd
CFL_FACTOR=0.5
cmd.C_compile(os.path.join(Ccodesdir,"Maxwell_Playground.c"),
              os.path.join(outdir,"Maxwell_Playground"),compile_mode="optimized")
```

    Compiling executable...
    (EXEC): Executing `gcc -std=gnu99 -Ofast -fopenmp -march=native -funroll-loops MaxwellEvolCurvi_Playground_Ccodes/Maxwell_Playground.c -o MaxwellEvolCurvi_Playground_Ccodes/output/Maxwell_Playground -lm`...
    (BENCH): Finished executing in 1.0038237571716309 seconds.
    Finished compilation.

```python
# Change to output directory
os.chdir(outdir)
```

```python
# Clean up existing output files
cmd.delete_existing_files("out*.txt")
cmd.delete_existing_files("out*.png")
cmd.delete_existing_files("out-*resolution.txt")
```

```python
# Set tme to end simulation
t_final = str(0.5*domain_size) + ' '

# Run executables
if 'Spherical' in CoordSystem  or 'SymTP' in CoordSystem:
    cmd.Execute("Maxwell_Playground", "64 48 4 "+t_final+str(CFL_FACTOR), "out-lowresolution.txt")
    cmd.Execute("Maxwell_Playground", "96 72 4 "+t_final+str(CFL_FACTOR), "out-medresolution.txt")
    Nxx0_low = '64'
    Nxx0_med = '96'

elif 'Cylindrical' in CoordSystem:
    cmd.Execute("Maxwell_Playground", "50 4 100 "+t_final+str(CFL_FACTOR), "out-lowresolution.txt")
    cmd.Execute("Maxwell_Playground", "80 4 160 "+t_final+str(CFL_FACTOR), "out-medresolution.txt")
    Nxx0_low = '50'
    Nxx0_med = '80'

else:
    # Cartesian
    cmd.Execute("Maxwell_Playground", "64 64 64 "+t_final+str(CFL_FACTOR), "out-lowresolution.txt")
    cmd.Execute("Maxwell_Playground", "128 128 128 "+t_final+str(CFL_FACTOR), "out-medresolution.txt")
    Nxx0_low = '64'
    Nxx0_med = '128'
```

    (EXEC): Executing `taskset -c 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 ./Maxwell_Playground 50 4 100 5.0 0.5`...
    It: 150 t=4.87 dt=3.25e-02 | 97.4%; ETA 0 s | t/h 23955.35 | gp/s 1.64e+07
    (BENCH): Finished executing in 0.8054358959197998 seconds.
    (EXEC): Executing `taskset -c 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 ./Maxwell_Playground 80 4 160 5.0 0.5`...
    It: 240 t=4.87 dt=2.03e-02 | 97.6%; ETA 0 s | t/h 8579.20 | gp/s 2.41e+07
    (BENCH): Finished executing in 2.2058610916137695 seconds.

```python
# Return to root directory
os.chdir(os.path.join("../../"))

print("Finished this code cell.")
```

    Finished this code cell.

# Step 6: Visualize the output! \[Back to [top](#toc)\]
$$\label{visualize}$$

In this section we will generate a movie, plotting the x component of the electric field on a 2D grid..

## Step 6.a: Install `scipy` and download `ffmpeg` if they are not yet installed/downloaded \[Back to [top](#toc)\]
$$\label{installdownload}$$

Note that if you are not running this within `mybinder`, but on a Windows system, `ffmpeg` must be installed using a separate package (on [this site](http://ffmpeg.org/)), or if running Jupyter within Anaconda, use the command: `conda install -c conda-forge ffmpeg`.

```python
!pip install scipy > /dev/null

check_for_ffmpeg = !which ffmpeg >/dev/null && echo $?
if check_for_ffmpeg != ['0']:
    print("Couldn't find ffmpeg, so I'll download it.")
    # Courtesy https://johnvansickle.com/ffmpeg/
    !wget http://astro.phys.wvu.edu/zetienne/ffmpeg-static-amd64-johnvansickle.tar.xz
    !tar Jxf ffmpeg-static-amd64-johnvansickle.tar.xz
    print("Copying ffmpeg to ~/.local/bin/. Assumes ~/.local/bin is in the PATH.")
    !mkdir ~/.local/bin/
    !cp ffmpeg-static-amd64-johnvansickle/ffmpeg ~/.local/bin/
    print("If this doesn't work, then install ffmpeg yourself. It should work fine on mybinder.")
```

## Step 6.b: Generate images for visualization animation \[Back to [top](#toc)\]
$$\label{genimages}$$

Here we loop through the data files output by the executable compiled and run in [the previous step](#mainc), generating a [png](https://en.wikipedia.org/wiki/Portable_Network_Graphics) image for each data file.

**Special thanks to Terrence Pierre Jacques. His work with the first versions of these scripts greatly contributed to the scripts as they exist below.**

```python
## VISUALIZATION ANIMATION, PART 1: Generate PNGs, one per frame of movie ##

import numpy as np
from scipy.interpolate import griddata
import matplotlib.pyplot as plt
from matplotlib.pyplot import savefig
from IPython.display import HTML
import matplotlib.image as mgimg

import glob
import sys
from matplotlib import animation

globby = glob.glob(os.path.join(outdir,'out'+Nxx0_med+'-00*.txt'))
file_list = []
for x in sorted(globby):
    file_list.append(x)

bound = domain_size/2.0
pl_xmin = -bound
pl_xmax = +bound
pl_zmin = -bound
pl_zmax = +bound

for filename in file_list:
    fig = plt.figure()
    t, x, z, Ex, Ey, Ax, Ay, C = np.loadtxt(filename).T #Transposed for easier unpacking

    plotquantity = Ex
    time = np.round(t[0], decimals=3)
    plotdescription = "Numerical Soln."
    plt.title(r"$E_x$ at $t$ = "+str(time))
    plt.xlabel("x")
    plt.ylabel("z")

    grid_x, grid_z = np.mgrid[pl_xmin:pl_xmax:200j, pl_zmin:pl_zmax:200j]
    points = np.zeros((len(x), 2))
    for i in range(len(x)):
        # Zach says: No idea why x and y get flipped...
        points[i][0] = x[i]
        points[i][1] = z[i]

    grid = griddata(points, plotquantity, (grid_x, grid_z), method='nearest')
    gridcub = griddata(points, plotquantity, (grid_x, grid_z), method='cubic')
    im = plt.imshow(gridcub, extent=(pl_xmin,pl_xmax, pl_zmin,pl_zmax))
    ax = plt.colorbar()
    plt.clim(-3,3)
    ax.set_label(plotdescription)
    savefig(os.path.join(filename+".png"),dpi=150)
    plt.close(fig)
    sys.stdout.write("%c[2K" % 27)
    sys.stdout.write("Processing file "+filename+"\r")
    sys.stdout.flush()
```

    Processing file MaxwellEvolCurvi_Playground_Ccodes/output/out80-00000240.txt

## Step 6.c: Generate visualization animation \[Back to [top](#toc)\]
$$\label{genvideo}$$

In the following step, [ffmpeg](http://ffmpeg.org) is used to generate an [mp4](https://en.wikipedia.org/wiki/MPEG-4) video file, which can be played directly from this Jupyter notebook.

```python
## VISUALIZATION ANIMATION, PART 2: Combine PNGs to generate movie ##

# https://stackoverflow.com/questions/14908576/how-to-remove-frame-from-matplotlib-pyplot-figure-vs-matplotlib-figure-frame
# https://stackoverflow.com/questions/23176161/animating-pngs-in-matplotlib-using-artistanimation

fig = plt.figure(frameon=False)
ax = fig.add_axes([0, 0, 1, 1])
ax.axis('off')

myimages = []

for i in range(len(file_list)):
    img = mgimg.imread(file_list[i]+".png")
    imgplot = plt.imshow(img)
    myimages.append([imgplot])

ani = animation.ArtistAnimation(fig, myimages, interval=100,  repeat_delay=1000)
plt.close()
ani.save(os.path.join(outdir,'Maxwell_ToroidalDipole.mp4'), fps=5,dpi=150)
```

```python
## VISUALIZATION ANIMATION, PART 3: Display movie as embedded HTML5 (see next cell) ##

# https://stackoverflow.com/questions/18019477/how-can-i-play-a-local-video-in-my-ipython-notebook
```

```python
# Embed video based on suggestion:
#  https://stackoverflow.com/questions/39900173/jupyter-notebook-html-cell-magic-with-python-variable
HTML("""

""")
```

# Step 7: Plot the numerical error, and confirm that it converges to zero with increasing numerical resolution (sampling) \[Back to [top](#toc)\]
$$\label{convergence}$$

```python
x_med, z_med, Ex__E_rel_med, Ey__E_rel_med, Ax__E_rel_med, Ay__E_rel_med = np.loadtxt(os.path.join(outdir,'out'+Nxx0_med+'.txt')).T #Transposed for easier unpacking

pl_xmin = -domain_size/2.
pl_xmax = +domain_size/2.
pl_ymin = -domain_size/2.
pl_ymax = +domain_size/2.

grid_x, grid_z = np.mgrid[pl_xmin:pl_xmax:100j, pl_ymin:pl_ymax:100j]
points_med = np.zeros((len(x_med), 2))
for i in range(len(x_med)):
    points_med[i][0] = x_med[i]
    points_med[i][1] = z_med[i]

grid_med = griddata(points_med, Ex__E_rel_med, (grid_x, grid_z), method='nearest')
grid_medcub = griddata(points_med, Ex__E_rel_med, (grid_x, grid_z), method='cubic')

plt.clf()
plt.title(r"Nxx0="+Nxx0_med+" Num. Err.: $\log_{10}|Ex|$ at $t$ = "+t_final)
plt.xlabel("x")
plt.ylabel("z")

fig_medcub = plt.imshow(grid_medcub.T, extent=(pl_xmin,pl_xmax, pl_ymin,pl_ymax))
cb = plt.colorbar(fig_medcub)
```

```python
x_low,z_low, EU0__E_rel_low, EU1__E_rel_low, AU0__E_rel_low, AU1__E_rel_low = np.loadtxt(os.path.join(outdir,'out'+Nxx0_low+'.txt')).T #Transposed for easier unpacking

points_low = np.zeros((len(x_low), 2))
for i in range(len(x_low)):
    points_low[i][0] = x_low[i]
    points_low[i][1] = z_low[i]

grid_low = griddata(points_low, EU0__E_rel_low, (grid_x, grid_z), method='nearest')

griddiff__low_minus__med = np.zeros((100,100))
griddiff__low_minus__med_1darray = np.zeros(100*100)
gridx_1darray_yeq0 = np.zeros(100)
grid_low_1darray_yeq0 = np.zeros(100)
grid_med_1darray_yeq0 = np.zeros(100)
count = 0
for i in range(100):
    for j in range(100):
        griddiff__low_minus__med[i][j] = grid_low[i][j] - grid_med[i][j]
        griddiff__low_minus__med_1darray[count] = griddiff__low_minus__med[i][j]
        if j==49:
            gridx_1darray_yeq0[i] = grid_x[i][j]
            grid_low_1darray_yeq0[i] = grid_low[i][j] + np.log10((float(Nxx0_low)/float(Nxx0_med))**4)
            grid_med_1darray_yeq0[i] = grid_med[i][j]
        count = count + 1

fig, ax = plt.subplots()
plt.title(r"4th-order Convergence of $E_x$ at t = "+t_final)
plt.xlabel("x")
plt.ylabel("log10(Absolute error)")

ax.plot(gridx_1darray_yeq0, grid_med_1darray_yeq0, 'k-', label='Nr = '+Nxx0_med)
ax.plot(gridx_1darray_yeq0, grid_low_1darray_yeq0, 'k--', label='Nr = '+Nxx0_low+', mult. by ('+Nxx0_low+'/'+Nxx0_med+')^4')
# ax.set_ylim([-8.5,0.5])

legend = ax.legend(loc='lower right', shadow=True, fontsize='x-large')
legend.get_frame().set_facecolor('C1')
plt.show()
```

# Step 8: Comparison Divergence Constrain Violation  \[Back to [top](#toc)\]
$$\label{div_e}$$

Here we calculate the violation quantity

$$
\mathcal{C} \equiv \nabla^i E_i,
$$

at each point on our grid (excluding the ghost zones) then calculate the normalized L2 norm over the entire volume via

\begin{align}
    \lVert C \rVert &= \left( \frac{\int C^2 d\mathcal{V}}{\int d\mathcal{V}} \right)^{1/2}.
\end{align}

```python
# Plotting the constraint violation
nrpy_div_low = np.loadtxt(os.path.join(outdir,'out-lowresolution.txt')).T
nrpy_div_med = np.loadtxt(os.path.join(outdir,'out-medresolution.txt')).T

plt.plot(nrpy_div_low[0]/domain_size, (nrpy_div_low[1]), 'k-',label='Nxx0 = '+Nxx0_low)
plt.plot(nrpy_div_med[0]/domain_size, (nrpy_div_med[1]), 'k--',label='Nxx0 = '+Nxx0_med)

plt.yscale('log')
plt.xlabel('Light Crossing Times')
plt.ylabel('||C||')
# plt.xlim(0,2.2)
# plt.ylim(1e-4, 1e-5)
plt.title('L2 Norm of the Divergence of E - '+par.parval_from_str("reference_metric::CoordSystem")+' Coordinates')
plt.legend(loc='center right')
plt.show()
```

# Step 9: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-Start_to_Finish-Solving_Maxwells_Equations_in_Vacuum-Curvilinear.pdf](Tutorial-Start_to_Finish-Solving_Maxwells_Equations_in_Vacuum-Curvilinear.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
import cmdline_helper as cmd    # NRPy+: Multi-platform Python command-line interface
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-Start_to_Finish-Solving_Maxwells_Equations_in_Vacuum-Curvilinear")
```

    Created Tutorial-Start_to_Finish-Solving_Maxwells_Equations_in_Vacuum-
        Curvilinear.tex, and compiled LaTeX file to PDF file Tutorial-
        Start_to_Finish-Solving_Maxwells_Equations_in_Vacuum-Curvilinear.pdf


# SpectralDNS

<!-- img src="../images/KHmovie2.gif" width="250"  -->

**Rayleigh-Bénard** convection computed with spectral accuracy using a [spectralDNS solver](https://github.com/spectralDNS/spectralDNS/blob/master/demo/RayleighBenard.py).

The [spectralDNS](https://github.com/spectralDNS) project revolves around
implementing high-performance flow solvers in [Python](https://www.python.org), which is a modern and very
high-level programming language. The project is supported through several grants from [King
Abdullahs University of Science and Technology](https://www.hpc.kaust.edu.sa),
granting access to some of the world's largest supercomputers. The work has
been presented at several conferences and as invited talks

  * [11'th International Conference on Scientific Computing and Applications](http://tianyuan.xmu.edu.cn/activities/19-20/ICSCA2019/index.html)

  * [International Conference on Computational Science and Engineering](https://cseconf2017.simula.no)

  * [I3MS Seminar Series of the Institute for Modeling and Simulation, 6 Nov 2017 RWTH AAchen University](https://www.aices.rwth-aachen.de/en/media-and-seminars/events/mortensen-seminar)

  * [Predictive Complex Computational Fluid Dynamics, KAUST, May 2017](https://pccfd.kaust.edu.sa/speaker?si=4)

  * [MekIT'17 National Conference on computational Mechanics, Trondheim, May 2017](http://arxiv.org/abs/1708.03188)

  * [EuroScipy, Cambridge, August 2015](https://www.euroscipy.org/2015/schedule/presentation/6/)

The *spectralDNS* project on github contains several repositories, each representing a smaller part of the overall project. The most important are presented beolw.

## spectralDNS

**Strong scaling** of triply periodic Navier-Stokes solver on the Shaheen II supercomputer at KAUST.

The [spectralDNS](https://github.com/spectralDNS/spectralDNS) repository is home to several different pseudo-spectral Navier-Stokes and MagnetoHydroDynamics solvers. Most solvers are for triply periodic domains. The simplest possible Navier-Stokes solver is described by {% cite Mortensen2016 %}, who show that a highly efficient solver can be created using no more than 100 lines of code, using nothing more than standard
tools like *Numpy* and *MPI for Python*. The DNS solver has been tested for a
transitional Taylor-Green vortex using a computational box of size $2048^3$. Accuracy is, well spectral, and in benchmark tests on the Shaheen II supercomputer at KAUST it has been found to scale well up to 64,000 cores.
A state-of-the-art spectral channel flow solver that is making extensive use of *shenfun*, has been described by {% cite mortensen2017spectral %}. Turbulent flow at $Re_{\tau}=2000$ is shown in the movie below.

With colleagues at the Extreme Computing Research Center (ECRC), King Abdullah University of Science and Technology (KAUST), we have been using [spectralDNS](https://github.com/spectralDNS) to investigate time integration of Fourier pseudospectral Direct Numerical Simulations {% cite ketcheson2020 %}. We investigate the use of higher‐order Runge‐Kutta pairs and automatic step size control based on local error estimation. We find that the fifth‐order accurate Runge‐Kutta pair of Bogacki and Shampine gives much greater accuracy at a significantly reduced computational cost.

## Shenfun

With the [shenfun](https://github.com/spectralDNS/shenfun) Python module
{% cite shenfun %} an
effort is made towards automating the implementation of the spectral Galerkin
method for simple tensor product domains, consisting of non-periodic and
periodic directions. The user interface to *shenfun* is intentionally made very
similar to [FEniCS](https://fenicsproject.org). Partial Differential Equations
are represented through weak variational forms and solved using efficient direct
solvers where available. MPI decomposition is achieved through the
[mpi4py-fft](https://bitbucket.org/mpi4py/mpi4py-fft) module, and all developed solvers may,
with no additional effort, be run on supercomputers using thousands of
processors.

An introduction to *shenfun* is given in {% cite shenfun %}, on [readthedocs](https://shenfun.readthedocs.io)
and the recent paper {% cite mortensen_joss %}. Introduction to *mpi4py-fft*
is given [here](https://mpi4py-fft.readthedocs.io/en/latest/) and
in {% cite mpi4py-fft_joss jpdc_fft %}. Further documentation is found at

[](https://shenfun.readthedocs.io/en/latest/)

Try shenfun in the computational cell below. Modify to own liking and run interactively.

```python
from sympy import symbols, cos, sin, pi
from shenfun import *
import matplotlib.pyplot as plt
%matplotlib inline

x, y = symbols("x,y")

C = Basis(20, 'Chebyshev')
F = Basis(24, 'Fourier', dtype='d')
T = TensorProductSpace(comm, (C, F))
u = project(sin(2*x)*cos(pi*y), T)
X = T.local_mesh(True)
plt.contourf(X[0], X[1], u.backward())
plt.colorbar()
plt.show()

```

## mpi4py-fft

mpi4py-fft is a Python package for computing Fast Fourier Transforms (FFTs).
Large arrays are distributed and communications are handled under the hood by
MPI for Python (mpi4py). To distribute large arrays we are using a
[new and completely generic algorithm](https://arxiv.org/abs/1804.09536)
that allows for any index set of a multidimensional array to be distributed. We
can distribute just one index (a slab decomposition), two index sets (pencil
decomposition) or even more for higher-dimensional arrays.

mpi4py-fft comes with its own Python interface to the serial
[FFTW](http://www.fftw.org) library. This interface can be used
much like [pyfftw](https://hgomersall.github.io/pyFFTW), and even for
real-to-real transforms, like discrete cosine or sine transforms. Further documentation is found at

[](https://mpi4py-fft.readthedocs.io/en/latest/)

Number of downloads from conda-forge:

[](https://anaconda.org/conda-forge/mpi4py-fft)

## References

{% bibliography --cited %}