# Periodic Signals

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Comunications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Spectrum

Periodic signals are an import class of signals. Many practical signals can be approximated reasonably well as periodic functions. This holds especially when considering only a limited time-interval. Examples of periodic signals are superpositions of harmonic signals, signals captured from vibrating structures or rotating machinery, as well as speech signals or signals from musical instruments. The spectrum of a periodic signal exhibits specific properties which are discussed in the following.

### Representation

A [periodic signal](https://en.wikipedia.org/wiki/Periodic_function) $x(t)$ is a signal that repeats its values in regular periods. It has to fulfill

\begin{equation}
x(t) = x(t + n \cdot T_\text{p})
\end{equation}

for $n \in \mathbb{Z}$ where its period is denoted by $T_\text{p} > 0$. A signal is termed *aperiodic* if is not periodic. One period $x_0(t)$ of a periodic signal is given as

\begin{equation}
x_0(t) = \begin{cases}
x(t) & \text{for } 0 \leq t < T_\text{p} \\
0 & \text{otherwise}
\end{cases}
\end{equation}

A periodic signal can be represented by [periodic summation](https://en.wikipedia.org/wiki/Periodic_summation) of shifted copies of one period $x_0(t)$

\begin{equation}
x(t) = \sum_{\mu = - \infty}^{\infty} x_0(t - \mu T_\text{p})
\end{equation}

which can be rewritten as convolution

\begin{equation}
x(t) = \sum_{\mu = - \infty}^{\infty} x_0(t) * \delta(t - \mu T_\text{p}) = x_0(t) * \sum_{\mu = - \infty}^{\infty} \delta(t - \mu T_\text{p})
\end{equation}

using the sifting property of the Dirac impulse. It can be concluded that a periodic signal can be represented by one period $x_0(t)$ of the signal convolved with a series of Dirac impulses.

**Example**

The cosine signal $x(t) = \cos (\omega_0 t)$ has a periodicity of $T_\text{p} = \frac{2 \pi}{\omega_0}$. One period is given as

\begin{equation}
x_0(t) = \cos (\omega_0 t) \cdot \text{rect} \left( \frac{t}{T_\text{p}} - \frac{T_\text{p}}{2} \right)
\end{equation}

Introduced into above representation of a periodic signal yields

\begin{align}
x(t) &= \cos (\omega_0 t) \cdot \text{rect} \left( \frac{t}{T_\text{p}} - \frac{T_\text{p}}{2} \right) * \sum_{\mu = - \infty}^{\infty} \delta(t - \mu T_\text{p}) \\
&= \cos (\omega_0 t) \sum_{\mu = - \infty}^{\infty} \text{rect} \left( \frac{t}{T_\text{p}} - \frac{T_\text{p}}{2} - \mu T_\text{p} \right) \\
&= \cos (\omega_0 t)
\end{align}

since the sum over the shifted rectangular signals is equal to one.

### The Dirac Comb

The sum of shifted Dirac impulses, as used above to represent a periodic signal, is known as [*Dirac comb*](https://en.wikipedia.org/wiki/Dirac_comb). The Dirac comb is defined as

\begin{equation}
{\bot \!\! \bot \!\! \bot}(t) = \sum_{\mu = - \infty}^{\infty} \delta(t - \mu)
\end{equation}

It is used for the representation of periodic signals and for the modeling of ideal sampling. In order to compute the spectrum of a periodic signal, the Fourier transform of the Dirac comb $\mathcal{F} \{ {\bot \!\! \bot \!\! \bot}(t) \}$ is derived in the following. Fourier transformation of the left- and right-hand side of above definition yields

\begin{equation}
\mathcal{F} \{ {\bot \!\! \bot \!\! \bot}(t) \} = \sum_{\mu = - \infty}^{\infty} e^{-j \mu \omega}
\end{equation}

The exponential function $e^{-j \mu \omega}$ for $\mu \in \mathbb{Z}$ is periodic with a period of $2 \pi$. Hence, the Fourier transform of the Dirac comb is also periodic with a period of $2 \pi$.

In order to gain further insight, the following convolution of a [rectangular signal](../notebooks/continuous_signals/standard_signals.ipynb#Rectangular-Signal) with a Dirac comb is considered

\begin{equation}
{\bot \!\! \bot \!\! \bot}(t) * \text{rect}(t) = 1
\end{equation}

The right hand side follows from the fact that the rectangular signals equals one for $-\frac{1}{2} < t < \frac{1}{2}$ which is then periodically summed up with a period of one. Fourier transform of the left- and right-hand side yields

\begin{equation}
\mathcal{F} \{ {\bot \!\! \bot \!\! \bot}(t) \} \cdot \text{sinc}\left(\frac{\omega}{2}\right) = 2 \pi \delta(\omega)
\end{equation}

For $\text{sinc}( \frac{\omega}{2} ) \neq 0$, which is equal to $\omega \neq 2 n \cdot \pi$ with $n \in \mathbb{Z} \setminus \{0\}$, this can be rearranged to

\begin{equation}
\mathcal{F} \{ {\bot \!\! \bot \!\! \bot}(t) \} = 2 \pi \, \delta(\omega) \cdot \frac{1}{\text{sinc}\left(\frac{\omega}{2}\right)} = 2 \pi \, \delta(\omega)
\end{equation}

Note that the [multiplication property](../continuous_signals/standard_signals.ipynb#Dirac-Impulse) of the Dirac impulse and $\text{sinc}(0) = 1$ has been used to derive the last equality. The Fourier transform is now known in the interval $-2 \pi < \omega < 2 \pi$. It has already been concluded that the Fourier transform is periodic with a period of $2 \pi$. Hence, the Fourier transformation of the Dirac comb can be derived by periodic continuation

\begin{equation}
\mathcal{F} \{ {\bot \!\! \bot \!\! \bot}(t) \} = \sum_{\mu = - \infty}^{\infty} 2 \pi \, \delta(\omega - 2 \pi \mu) = \sum_{\mu = - \infty}^{\infty} 2 \pi \, \left( \frac{\omega}{2 \pi} - \mu \right)
\end{equation}

The last equality follows from the scaling property of the Dirac impulse. Using the definition of the Dirac comb, the Fourier transform can now be rewritten in terms of the Dirac comb

\begin{equation}
\mathcal{F} \{ {\bot \!\! \bot \!\! \bot}(t) \} = {\bot \!\! \bot \!\! \bot} \left( \frac{\omega}{2 \pi} \right)
\end{equation}

The Fourier transform of a Dirac comb with unit distance between the Dirac impulses is a Dirac comb with a distance of $2 \pi$ between the Dirac impulses which are weighted by $2 \pi$.

**Example**

The following example computes the truncated series

\begin{equation}
X(j \omega) = \sum_{\mu = -M}^{M} e^{-j \mu \omega}
\end{equation}

as approximation of the Fourier transform $\mathcal{F} \{ {\bot \!\! \bot \!\! \bot}(t) \}$ of the Dirac comb. For this purpose the sum is defined and plotted in `SymPy`.

```python
%matplotlib inline
import sympy as sym
sym.init_printing()

mu = sym.symbols('mu', integer=True)
w = sym.symbols('omega', real=True)
M = 20

X = sym.Sum(sym.exp(-sym.I*mu*w), (mu, -M, M)).doit()
sym.plot(X, xlabel='$\omega$', ylabel='$X(j \omega)$', adaptive=False, nb_of_points=1000);
```

**Exercise**

* Change the summation limit $M$. How does the approximation change? Note: Increasing $M$ above a certain threshold may lead to numerical instabilities.

### Fourier-Transform

In order to derive the Fourier transform $X(j \omega) = \mathcal{F} \{ x(t) \}$ of a periodic signal $x(t)$ with period $T_\text{p}$, the signal is represented by one period $x_0(t)$ and the Dirac comb. Rewriting above representation of a periodic signal in terms of a sum of Dirac impulses by noting that $\delta(t - \mu T_\text{p}) = \frac{1}{T_\text{p}} \delta(\frac{t}{T_\text{p}} - \mu)$ yields

\begin{equation}
x(t) = x_0(t) * \frac{1}{T_\text{p}} {\bot \!\! \bot \!\! \bot} \left( \frac{t}{T_\text{p}} \right)
\end{equation}

The Fourier transform is derived by application of the [convolution theorem](../fourier_transform/theorems.ipynb#Convolution-Theorem)

\begin{align}
X(j \omega) &= X_0(j \omega) \cdot {\bot \!\! \bot \!\! \bot} \left( \frac{\omega T_\text{p}}{2 \pi} \right) \\
&= \frac{2 \pi}{T_\text{p}} \sum_{\mu = - \infty}^{\infty} X_0 \left( j \, \mu \frac{2 \pi}{T_\text{p}} \right) \cdot
\delta \left( \omega - \mu \frac{2 \pi}{T_\text{p}} \right)
\end{align}

where $X_0(j \omega) = \mathcal{F} \{ x_0(t) \}$ denotes the Fourier transform of one period of the periodic signal. From the last equality it can be concluded that the Fourier transform of a periodic signal consists of a series of weighted Dirac impulses. These Dirac impulse are equally distributed on the frequency axis $\omega$ at an interval of $\frac{2 \pi}{T_\text{p}}$. The weights of the Dirac impulse are given by the values of the spectrum $X_0(j \omega)$ of one period at the locations $\omega = \mu \frac{2 \pi}{T_\text{p}}$. Such a spectrum is termed *line spectrum*.

### Parseval's Theorem

[Parseval's theorem](../fourier_transform/theorems.ipynb#Parseval%27s-Theorem) relates the energy of a signal in the time domain to its spectrum. The energy of a periodic signal is in general not defined. This is due to the fact that its energy is unlimited, if the energy of one period is non-zero. As alternative, the average power of a periodic signal $x(t)$ is used. It is defined as

\begin{equation}
P = \frac{1}{T_\text{p}} \int_{0}^{T_\text{p}} |x(t)|^2 \; dt
\end{equation}

Introducing the Fourier transform of a periodic signal into [Parseval's theorem](../fourier_transform/theorems.ipynb#Parseval%27s-Theorem) yields

\begin{equation}
\frac{1}{T_\text{p}} \int_{0}^{T_\text{p}} |x(t)|^2 \; dt = \frac{1}{T_\text{p}} \sum_{\mu = - \infty}^{\infty} \left| X_0 \left( j \, \mu \frac{2 \pi}{T_\text{p}} \right) \right|^2
\end{equation}

The average power of a periodic signal can be calculated in the time-domain by integrating over the squared magnitude of one period or in the frequency domain by summing up the squared magnitude weights of the coefficients of the Dirac impulses of its Fourier transform.

### Fourier Transform of the Pulse Train

The [pulse train](https://en.wikipedia.org/wiki/Pulse_wave) is commonly used for power control using [pulse-width modulation (PWM)](https://en.wikipedia.org/wiki/Pulse-width_modulation). It is constructed from a periodic summation of a rectangular signal $x_0(t) = \text{rect} (\frac{t}{T} - \frac{T}{2})$

\begin{equation}
x(t) = \text{rect} \left( \frac{t}{T} - \frac{T}{2} \right) * \frac{1}{T_\text{p}} {\bot \!\! \bot \!\! \bot} \left( \frac{t}{T_\text{p}} \right)
\end{equation}

where $0 < T < T_\text{p}$ denotes the width of the pulse and $T_\text{p}$ its periodicity. Its usage for power control becomes evident when calculating the average power of the pulse train

\begin{equation}
P = \frac{1}{T_\text{p}} \int_{0}^{T_\text{p}} | x(t) |^2 dt = \frac{T}{T_\text{p}}
\end{equation}

The Fourier transform of one period $X_0(j \omega) = \mathcal{F} \{ x_0(t) \}$ is derived by applying the scaling and shift theorem of the Fourier transform to the [Fourier transform of the retangular signal](../fourier_transform/definition.ipynb#Transformation-of-the-Rectangular-Signal)

\begin{equation}
X_0(j \omega) = e^{-j \omega \frac{T}{2}} \cdot T \, \text{sinc} \left( \frac{\omega T}{2} \right)
\end{equation}

from which the spectrum of the pulse train follows by application of above formula for the Fourier transform of a periodic signal

\begin{equation}
X(j \omega) = 2 \pi \frac{1}{T_\text{p}} \sum_{\mu = - \infty}^{\infty} e^{-j \mu \pi \frac{T}{T_\text{p}}} \cdot T \, \text{sinc} \left( \mu \pi \frac{T}{T_\text{p}} \right) \cdot \delta \left( \omega - \mu \frac{2 \pi}{T_\text{p}} \right)
\end{equation}

**Example**

The pulse train and its spectrum are illustrated by the subsequent computational example. First the pulse train is defined and plotted in `SymPy`

```python
mu = sym.symbols('mu', integer=True)
t = sym.symbols('t', real=True)
T = 2
Tp = 5

def pulse_train(T, Tp):
    n = sym.symbols('n', integer=True)
    x0 = sym.Piecewise((0, t < 0), (1, t < T), (0, True))
    return sym.summation(x0.subs(t, t+n*Tp), (n, -10, 10))

import warnings
warnings.filterwarnings("ignore", module="sympy.plot")
sym.plot(pulse_train(T, Tp), (t, -5, 20), xlabel='$t$', ylabel='$x(t)$', adaptive=False);
```

The weights of the Dirac impulses are defined for fixed values $T$ and $T_\text{p}$

```python
X_mu = sym.exp(-sym.I * mu * sym.pi * T/Tp) * T * sym.sinc(mu * sym.pi * T/Tp)
X_mu
```

The weights of the Dirac impulses are plotted with [`matplotlib`](http://matplotlib.org/index.html), a Python plotting library. The library expects the values of the function to be plotted at a series of sampling points. In order to create these, the function [`sympy.lambdify`](http://docs.sympy.org/latest/modules/utilities/lambdify.html?highlight=lambdify#sympy.utilities.lambdify) is used which numerically evaluates a symbolic function at given sampling points. The resulting plot illustrates the positions and weights of the Dirac impulses.

```python
import numpy as np
import matplotlib.pyplot as plt

Xn = sym.lambdify(mu, sym.Abs(X_mu), 'numpy')

n = np.arange(-15, 15)
plt.stem(n*2*np.pi/Tp, Xn(n))
plt.xlabel('$\omega$')
plt.ylabel('$|X(j \omega)|$');
```

**Exercise**

* Change the ratio $\frac{T}{T_\text{p}}$. How does the spectrum of the pulse train change?
* Can you derive the periodicity $T_\text{p}$ of the signal from its spectrum?
* Calculate the average power of the pulse train in the frequency domain by applying Parseval's theorem.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Continuous- and Discrete-Time Signals and Systems - Theory and Computational Examples*.


```python
import matplotlib.pyplot as plt
import scipy.stats as st
import seaborn as sns
import pandas as pd
from scipy.stats import norm, uniform, expon, t
from scipy.integrate import quad
from sympy.solvers import solve
from sympy import Symbol
import numpy as np
from pandas import Series, DataFrame
```

```python
fuellungen = Series([71, 69, 67, 68, 73, 72, 71, 71, 68, 72, 69, 72])
```

```python
fuellungen.mean()
```

    70.25

```python
fuellungen_standardisiert = fuellungen
```

```python
norm.ppf(q=[0.05,0.95], loc=70, scale=1.5/np.sqrt(12))
```

    array([69.28775749, 70.71224251])

```python
np.sqrt(12)*((70.25-70)/1.5)
```

    0.5773502691896257

```python
norm.ppf(q=0.05)
```

    -1.6448536269514729

# 6.2

```python
fuellungen2 = Series([71, 69, 67, 68, 73, 72, 71, 71, 68, 72, 69, 72])
```

```python
fuellungen.mean()
```

    70.25

```python
t.ppf(q=[0.05], df=fuellungen2.size-1, loc=70, scale=fuellungen2.std()/np.sqrt(fuellungen2.size))
```

    array([68.98397389])

```python

```

    1.9598237397554634

# 6.3

```python
norm.ppf(q=[0.95], loc=200, scale=10/np.sqrt(16))
```

    array([204.11213407])

```python
norm.cdf(x=205, loc=200, scale=10/np.sqrt(16))
```

    0.9772498680518208

```python
norm.cdf(x=200, loc=200, scale=10/np.sqrt(16))
```

    0.5

```python
t.ppf(q=[0.95], df=16-1, loc=200, scale=10/np.sqrt(16))
```

    array([204.38262589])

```python
t.cdf(x=200, loc=200, df=16-1, scale=10/np.sqrt(16))
```

    0.5

```python
st.ttest_1samp()
```


# Mathematical Preliminaries

**DIVE into Deep Learning**
___

```python
from IPython import display
from manim import *
```

The following is a lecture series that introduces the basic theory of deep learning.

```python
display.IFrame(src="https://www.youtube.com/embed/videoseries?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi", width="100%", height=450)
```

Understanding the lectures requires some background on vector calculus and probability theory. $\def\abs#1{\left\lvert #1 \right\rvert}
\def\Set#1{\left\{ #1 \right\}}
\def\mc#1{\mathcal{#1}}
\def\M#1{\boldsymbol{#1}}
\def\R#1{\mathsf{#1}}
\def\RM#1{\boldsymbol{\mathsf{#1}}}
\def\op#1{\operatorname{#1}}
\def\E{\op{E}}
\def\d{\mathrm{\mathstrut d}}
$

**What to know about vector calculus?**

```python
%%html
<b>Notation:</b><br>

```

- [Vectors](https://en.wikipedia.org/wiki/Vector_(mathematics_and_physics)) in lowercase boldface font:

$$\M{x}=\begin{bmatrix}x_1 \\ x_2 \\ \vdots \end{bmatrix}$$

- [Matrices](https://en.wikipedia.org/wiki/Matrix_(mathematics)) in uppercase boldface font:

$$\M{W}=\begin{bmatrix}w_{11} & w_{12} & \cdots \\
w_{21} & \ddots &  \\
\vdots &  &  \end{bmatrix}$$

- [Matrix multiplication](https://en.wikipedia.org/wiki/Matrix_multiplication):

$$\M{W}\M{x} = \begin{bmatrix}w_{11} & w_{12} & \cdots \\
w_{21} & \ddots &  \\
\vdots &  &  \end{bmatrix}
\begin{bmatrix}x_1 \\ x_2 \\ \vdots \end{bmatrix}
=
\begin{bmatrix}w_{11}x_1 + w_{12}x_2 + \cdots \\ w_{21}x_1+\cdots \\ \vdots \end{bmatrix}
$$

**What to know about Probability Theory?**

```python
%%html
<b>Distribution:</b><br>

```

[Joint distribution](https://en.wikipedia.org/wiki/Joint_probability_distribution#Mixed_case):

$$p_{\RM{x}\R{y}}(\M{x},y)= \underbrace{p_{\R{y}|\RM{x}}(y|\M{x})}_{\underbrace{\Pr}_{\text{probability measure}\kern-3em}\Set{\R{y}=y|\RM{x}=\M{x}}} \cdot \underbrace{p_{\RM{x}}(\M{x})}_{(\underbrace{\partial_{x_1}}_{\text{partial derivative w.r.t. $x_1$}\kern-5em} \partial_{x_2}\cdots)\Pr\Set{\RM{x} \leq \M{x}}\kern-4em}\kern1em \text{where}$$
- $p_{\R{y}|\RM{x}}(y|\M{x})$ is the *probability mass function [(pmf)](https://en.wikipedia.org/wiki/Probability_mass_function)* of $\R{y}=y\in \mc{Y}$ [conditioned](https://en.wikipedia.org/wiki/Conditional_probability_distribution) on $\RM{x}=\M{x}\in \mc{X}$, and
- $p_{\RM{x}}(\M{x})$ is the *(multivariate) probability density function [(pdf)](https://en.wikipedia.org/wiki/Probability_density_function#Densities_associated_with_multiple_variables)* of $\RM{x}=\M{x}\in \mc{X}$.

```python
%%html
<b>Expectation:<br>

```

For any function $g$ of $(\RM{x},y)$, the expectations are:

$$
\begin{align}
\E[g(\RM{x},\R{y})|\RM{x}]&=\sum_{y\in \mc{Y}} g(\RM{x},y)\cdot p_{\R{y}|\RM{x}}(y|\RM{x})\tag{conditional exp.}
\\
\E[g(\RM{x},\R{y})] &=\int_{\mc{X}} \underbrace{\sum_{y\in \mc{Y}} g(\RM{x},y)\cdot \underbrace{p_{\RM{x},\R{y}}(\M{x},y)}_{p_{\R{y}|\RM{x}}(y|\M{x}) p_{\R{x}}(\M{x})}\kern-1.7em}_{\E[g(\RM{x},\R{y})|\RM{x}]}\kern1.4em\,\d \M{x} \tag{exp.}\\
&= \E[\E[g(\RM{x},\R{y})|\RM{x}]] \tag{iterated exp.}
\end{align}
$$

All the videos were created using the python library [mathematical animation engine (`manim`)](https://github.com/3b1b/manim).

```python
from manim import *
```

**Exercise** Run the following cell and see the effect when changing

- Mobjects `Text('Hello, World!')` to
  - `MathTex(r'E=mc^2')` or
  - `Circle()` or
  - `Square()`...
- Animation objects `Write` to
  - `FadeIn` or
  - `GrowFromCenter`...

You may take a look at the documentation [here](https://docs.manim.community/en/v0.2.0/index.html) and a more detailed [tutorial](https://talkingphysics.wordpress.com/2019/01/08/getting-started-animating-with-manim-and-python-3-7/) here to create more complicated animations. Since rending videos take time, create a separate notebook for more intensive work without affecting the autograding of this notebook.

```python
%%manim -ql --progress_bar=none --disable_caching --flush_cache -v ERROR HelloWorld
class HelloWorld(Scene):
    def construct(self):
        self.play(Write(Text('Hello, World!')))
```

To show all the options available to `manim`:

```python
%manim render --help
```


```python
from sympy import symbols, sin, cos, Abs
from galgebra.ga import Ga
from galgebra.printer import Format, Fmt
from IPython.display import Latex
Format()
```

```python
xyz_coords = (x, y, z) = symbols('x y z', real=True)
(o3d, ex, ey, ez) = Ga.build('e', g=[1, 1, 1], coords=xyz_coords, norm=True)
o3d.g
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{matrix}\right]$

```python
f = o3d.mv('f', 'scalar', f=True)
f
```

\begin{equation*} f = f  \end{equation*}

```python
F = o3d.mv('F', 'vector', f=True)
lap = o3d.grad*o3d.grad
lap.Fmt(1,r'\nabla^{2}')
```

\begin{equation*} \nabla^{2} = \frac{\partial^{2}}{\partial x^{2}} + \frac{\partial^{2}}{\partial y^{2}} + \frac{\partial^{2}}{\partial z^{2}} \end{equation*}

```python
lap.Fmt(1,r'\nabla^{2}')
```

\begin{equation*} \nabla^{2} = \frac{\partial^{2}}{\partial x^{2}} + \frac{\partial^{2}}{\partial y^{2}} + \frac{\partial^{2}}{\partial z^{2}} \end{equation*}

```python
lapf = lap*f
lapf
```

\begin{equation*} \partial^{2}_{x} f  + \partial^{2}_{y} f  + \partial^{2}_{z} f  \end{equation*}

```python
lapf = o3d.grad | (o3d.grad * f)
lapf.Fmt(1,r'\nabla \cdot (\nabla f)')
```

\begin{equation*} \nabla \cdot (\nabla f) = \partial^{2}_{x} f  + \partial^{2}_{y} f  + \partial^{2}_{z} f  \end{equation*}

```python
divF = o3d.grad|F
divF.Fmt(1,'x =')
```

\begin{equation*} x = = \partial_{x} F^{x}  + \partial_{y} F^{y}  + \partial_{z} F^{z}  \end{equation*}

```python
gradF = o3d.grad * F
gradF.Fmt(1,r'\nabla F')
```

\begin{equation*} \nabla F = \left ( \partial_{x} F^{x}  + \partial_{y} F^{y}  + \partial_{z} F^{z} \right )  + \left ( - \partial_{y} F^{x}  + \partial_{x} F^{y} \right ) \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + \left ( - \partial_{z} F^{x}  + \partial_{x} F^{z} \right ) \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + \left ( - \partial_{z} F^{y}  + \partial_{y} F^{z} \right ) \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \end{equation*}

```python
sph_coords = (r, th, phi) = symbols('r theta phi', real=True)
(sp3d, er, eth, ephi) = Ga.build('e', g=[1, r**2, r**2 * sin(th)**2], coords=sph_coords, norm=True)
sp3d.g_raw
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & r^{2} & 0\\0 & 0 & r^{2} \sin^{2}{\left(\theta \right)}\end{matrix}\right]$

```python
sp3d.grad.Fmt(1,r'\nabla')
```

\begin{equation*} \nabla = \boldsymbol{e}_{r} \frac{\partial}{\partial r} + \boldsymbol{e}_{\theta } \frac{1}{r} \frac{\partial}{\partial \theta } + \boldsymbol{e}_{\phi } \frac{1}{r \sin{\left (\theta  \right )}} \frac{\partial}{\partial \phi } \end{equation*}

```python
f = sp3d.mv('f', 'scalar', f=True)
F = sp3d.mv('F', 'vector', f=True)
B = sp3d.mv('B', 'bivector', f=True)
sp3d.grad.Fmt(1,r'\nabla')
lap = sp3d.grad*sp3d.grad
lap.Fmt(1,r'\nabla^{2} ')
```

\begin{equation*} \nabla^{2}  = \frac{1}{r^{2} \tan{\left (\theta  \right )}} \frac{\partial}{\partial \theta } + \frac{2}{r} \frac{\partial}{\partial r} + \frac{1}{r^{2} {\sin{\left (\theta  \right )}}^{2}} \frac{\partial^{2}}{\partial \phi ^{2}} + r^{-2} \frac{\partial^{2}}{\partial \theta ^{2}} + \frac{\partial^{2}}{\partial r^{2}} \end{equation*}

```python
Lapf = lap*f
Lapf.Fmt(1,r'\nabla^{2} f')
```

\begin{equation*} \nabla^{2} f = \frac{r^{2} \partial^{2}_{r} f  + 2 r \partial_{r} f  + \partial^{2}_{\theta } f  + \frac{\partial_{\theta } f }{\tan{\left (\theta  \right )}} + \frac{\partial^{2}_{\phi } f }{{\sin{\left (\theta  \right )}}^{2}}}{r^{2}} \end{equation*}

```python
lapf = sp3d.grad | (sp3d.grad * f)
lapf.Fmt(1,r'\nabla \cdot (\nabla f)')
```

\begin{equation*} \nabla \cdot (\nabla f) = \frac{r^{2} \partial^{2}_{r} f  + 2 r \partial_{r} f  + \partial^{2}_{\theta } f  + \frac{\partial_{\theta } f }{\tan{\left (\theta  \right )}} + \frac{\partial^{2}_{\phi } f }{{\sin{\left (\theta  \right )}}^{2}}}{r^{2}} \end{equation*}

```python
dviF = sp3d.grad | F
divF.Fmt(1,r'\nabla F')
```

\begin{equation*} \nabla F = \partial_{x} F^{x}  + \partial_{y} F^{y}  + \partial_{z} F^{z}  \end{equation*}

```python
curlF = sp3d.grad ^ F
curlF.Fmt(1,r'\nabla \wedge F')
```

\begin{equation*} \nabla \wedge F = \frac{r \partial_{r} F^{\theta }  + F^{\theta }  - \partial_{\theta } F^{r} }{r} \boldsymbol{e}_{r}\wedge \boldsymbol{e}_{\theta } + \frac{r \partial_{r} F^{\phi }  + F^{\phi }  - \frac{\partial_{\phi } F^{r} }{\sin{\left (\theta  \right )}}}{r} \boldsymbol{e}_{r}\wedge \boldsymbol{e}_{\phi } + \frac{\frac{F^{\phi } }{\tan{\left (\theta  \right )}} + \partial_{\theta } F^{\phi }  - \frac{\partial_{\phi } F^{\theta } }{\sin{\left (\theta  \right )}}}{r} \boldsymbol{e}_{\theta }\wedge \boldsymbol{e}_{\phi } \end{equation*}

```python
divB = sp3d.grad | B
divB.Fmt(1,r'\nabla \cdot B')
```

\begin{equation*} \nabla \cdot B = - \frac{\frac{B^{r\theta } }{\tan{\left (\theta  \right )}} + \partial_{\theta } B^{r\theta }  + \frac{\partial_{\phi } B^{r\phi } }{\sin{\left (\theta  \right )}}}{r} \boldsymbol{e}_{r} + \frac{r \partial_{r} B^{r\theta }  + B^{r\theta }  - \frac{\partial_{\phi } B^{\theta \phi } }{\sin{\left (\theta  \right )}}}{r} \boldsymbol{e}_{\theta } + \frac{r \partial_{r} B^{r\phi }  + B^{r\phi }  + \partial_{\theta } B^{\theta \phi } }{r} \boldsymbol{e}_{\phi } \end{equation*}

```python
F
```

\begin{equation*} F = F^{r}  \boldsymbol{e}_{r} + F^{\theta }  \boldsymbol{e}_{\theta } + F^{\phi }  \boldsymbol{e}_{\phi } \end{equation*}

```python
F.Fmt(3,'F')
```

 \begin{align*}   F =& F^{r}  \boldsymbol{e}_{r} \\  &  + F^{\theta }  \boldsymbol{e}_{\theta } \\  &  + F^{\phi }  \boldsymbol{e}_{\phi }  \end{align*}

```python
F.norm()
```

$\displaystyle \sqrt{\operatorname{F^{\phi}}^{2}{\left(r,\theta,\phi \right)} + \operatorname{F^{r}}^{2}{\left(r,\theta,\phi \right)} + \operatorname{F^{\theta}}^{2}{\left(r,\theta,\phi \right)}}$


# Section 3.3 $\quad$ Cofactor Expansion

## Definition of Minor

Let $A = [a_{ij}]$ be an $n\times n$ matrix.

## Definition of Cofactor

Let $A = [a_{ij}]$ be an $n\times n$ matrix.

### Example 1

Find the cofactors $A_{12}$ and $A_{23}$ if
\begin{equation*}
  A =
  \left[
    \begin{array}{ccc}
      4 & 3 & 2 \\
      4 & -2 & 5 \\
      2 & 4 & 6 \\
    \end{array}
  \right]
\end{equation*}

```python
from sympy import *

A = Matrix([[4, 3, 2], [4, -2, 5], [2, 4, 6]]);

A.cofactor(0, 1), A.cofactor(1, 2)
```

    (-14, -10)

>**Theorem (cofactor expansion)** Let $A = [a_{ij}]$ be an $n\times n$ matrix. Then

\begin{equation*}
det(A) = \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad
\end{equation*}

and

\begin{equation*}
det(A) = \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad
\end{equation*}

### Example 2

Find the determinant of
\begin{equation*}
  A = \left[
    \begin{array}{cccc}
      1 & 2 & -3 & 4 \\
      -4 & 2 & 1 & 3 \\
      3 & 0 & 0 & -3 \\
      2 & 0 & -2 & 3 \\
    \end{array}
  \right]
\end{equation*}

```python
from sympy import *

A = Matrix([[1, 2, -3, 4], [-4, 2, 1, 3], [3, 0, 0, -3], [2, 0, -2, 3]]);

A.det()
```

    48

### Example 3

Find all values of $t$ for which
\begin{equation*}
  det\left[
    \begin{array}{ccc}
      t-1 & 0 & 1 \\
      2 & t+2 & -1 \\
      0 & 0 & t+1 \\
    \end{array}
  \right] = 0
\end{equation*}

```python
from sympy import *

t = symbols('t');
A = Matrix([[t-1, 0, 1], [2, t+2, -1], [0, 0, t+1]]);

solve(A.det())
```

    [-2, -1, 1]


# Exercise 6.2 in JT: More on pooling in credit markets.

Import packages, classes and settings:

```python
import numpy as np
import math
import itertools
from scipy import optimize
import scipy.stats as stats
import PS1 as func
# For plots:
import matplotlib.pyplot as plt
import matplotlib as mpl
%matplotlib inline
plt.style.use('seaborn-whitegrid')
mpl.style.use('seaborn')
prop_cycle = plt.rcParams["axes.prop_cycle"]
colors = prop_cycle.by_key()["color"]
import ipywidgets as widgets
from ipywidgets import interact, interact_manual
```

Compared to exercise and 6.1, alter the setup as follows:

* Continuum of types instead of good/bad. For entrepreneur $i$ the value $B_i$ is distributed according to the CDF function $H(B)$, with support on $[0, \bar{B}]$

* Monopoly lender offers credit to borrowers (entrepreneurs).

The lender offers $R_b$ for a successfull investment, otherwise 0. Borrower $i$ then behaves **if** the IC constraint holds: $B_i\leq \Delta p R_b$. The expected profits from offering the contract is then defined by:

$$\begin{align}
    \pi(R_b) = H\big(\Delta p R_b\big)p_H(R-R_b)+\left[1-H\big(\Delta p R_b\big)\right] p_L(R-R_b)-I, \tag{Profits}
\end{align} $$

where $H(\Delta p R_b)$ measures the share of borrowers that behave, i.e. with $B_i<\Delta p R_b$. From this note:

* The share of high-quality borrowers increase with $R_b$ (bad types start to behave).

* Same dynamics as before: Adverse selection reduces quality of lending, induces cross-subsidies between types.

### Example with uniform distribution:

```python
Model = func.poolingCredit() # create model instance with default values
Model.plot_distr()
```

### Expected profits as a function of $B$:

```python
Model.plot_exp_profits()
```

## Q: Show a number of results:
1. Proportion of high-quality borrowers is endogenous and depends on $R_b$.

2. Adverse selection reduces quality of lending

3. externality among types: Presence of low-quality borrower forces lender to charge interest rate, generating strictly positive profits on high-quality types.

4. Show an example of 'bad lemons' breaking down the market.

The first 3 are answered straightforwardly:

* A high quality borrower, is one that behaves. The share of high quality borrowers is defined as $H(\Delta p R_b)$. If $R_b$ increases, so does this share.

* Adverse selection means that the lender cannot e.g. **only** offer the contract $R_b$ to certain types / offer individual contracts based on their types $R_b^i(B_i)$.

Lastly, we give an example where the market breaks down. All we need for the market to break down, is to find a scenario where profits of the lender is negative:

* Assume that $p_L=0$, and that borrowers are **uniformly distributed** that is $H(B) = B/\bar{B}$, for $B\in[0,\bar{B}]$.
* In this case the expected profits are given by:

$$\begin{align}
    \pi(R_b) &= \overbrace{\dfrac{\Delta p R_b}{\bar{B}}}^{\equiv H(\Delta p R_b)} p_H (R-R_b)-I  \\
             &= \dfrac{p_H R_b}{\bar{B}}p_H(R-R_b)-I.
\end{align} $$

The profit maximizing level of $R_b$ is then given by: $R_b^* = R/2$, which yields profits of

$$\begin{align}
    \pi(R_b^*) &= \dfrac{p_H R/2}{\bar{B}}p_H\left(R-\dfrac{R}{2}\right)-I \\
               &= \dfrac{p_H^2R^2}{4\bar{B}}-I.
\end{align} $$

Thus as long as $I>p_H^2R^2/(4\bar{B})$ the market breaks down.

We can show this by updating parameter values as follows:
* $p_L = 0$ (this is 0.5 as default),
* $I = 5$ (this is 2 as default).

```python
par = {'pL': 0, # set pL = 0, which is set to 0.5 as default
       'I': 5} #set I=5, which is set to 2 as default
Model.upd_par(par)
```

Next, we can plot the expected profit functions:

```python
# Profits with default values
Model_st = func.poolingCredit()
Model_st.plot_exp_profits()
```

```python
# Profits with updated parameter values:
Model.plot_exp_profits()
```


# INTRODUCTION TO PATTERN RECOGNITION
## PROGRAMMING ASSIGNMENT -1

__1. Import essential libraries__
 * numpy
 * pandas
 * mlxtend.data to import the MNIST dataset
 * cv2 to write the images for mean and standard deviation

```python
import numpy as np
import math
from mlxtend.data import loadlocal_mnist
import pandas as pd
import cv2
```

__2. Import the MNIST dataset__
  * Images path for features for the digits
  * Labels indicate the digits

```python
X_train, y_train= loadlocal_mnist(images_path='train-images-idx3-ubyte',
                                  labels_path='train-labels-idx1-ubyte')
```

```python
import matplotlib.pyplot as plt

a = X_train[0].reshape(28, 28)
plt.imshow(a)
print(y_train[0])
```

__3. Load the training dataset onto a dataframe, which makes it easier to access the data__

```python
df=pd.DataFrame(X_train,y_train)
```

__4. Function that calculates the mean and standard deviation for all of the 10 classes (10 digits).__

* The mean for the data can be calulated by the following equation:

\begin{align}
\mu = \frac{( \sum X_i)}{N}
\end{align}

       * where the N is the length for each class of digits.
       * X is the individual features for each of the classes.

* The standard deviation for the data:</br>

\begin{align}
\sigma = \sqrt{\frac{\sum (  x_{ij} - \mu_i)}{N}}
\end{align}

```python
def mandstd(n):
    x=[]
    y=[]
    s=0
    for i in range(n):
        l=df.loc[i]
        mean=l.sum()/len(l)
        diff=(l-mean)**2
        s=diff.sum()
        var=s/len(l)
        SD=np.sqrt(var)
        x.append(mean)
        y.append(SD)

    return x,y
```

__5. Calling the above function and storing the mean and standard deviation in 2 lists x and y.__

```python
x,y=mandstd(10)
```

```python
print(type(y[0]))
```

    <class 'pandas.core.series.Series'>

__6. The next step is to convert the list and draw an image based on the results of mean and standard deviation.__

    * For the mean and standard deviation images we take all the 10 classes and convert it into a 28x28 pixel  matrix and pass it on to the imwrite() function provided by the opencv library.

```python
x2=[]
y2=[]
for i in range(10):
    x2.append(np.array(x[i]).reshape(28,28))
    y2.append(np.array(y[i]).reshape(28,28))
    cv2.imwrite("mean{}.png".format(i),x2[i])
    cv2.imwrite("sd{}.png".format(i),y2[i])
```

### Images for mean

## Images for Standard Deviation

__7. Covariance matrix__

    * Covariance matrix is required in the LDA function.
    * The values are added along the diagonal.
    * An offset is added to the Variance matrix in order to avoid the Singular matrix ( Where the determinant of the matrix becomes 0.

```python
# Offset so that the determinant of the matrix does not become 0.
for i in range(len(y)):
    for j in range(784):
        y[i][j]+=0.01

cov=[]
inv=[]
for i in range(len(y)):
    a=np.diag(y[i]**2)
    cov.append(a)
    b=np.linalg.inv(cov[i])
    inv.append(b)

```

__8. Loading the Testing data__

```python
X_test, y_test=loadlocal_mnist(images_path='t10k-images-idx3-ubyte',
                                labels_path='t10k-labels-idx1-ubyte')
```

__9. Writing the LDA classifier to predict. __

 * Guassian Distribution - The LDA function is derived by the above equation.

\begin{align}
p(x)=\frac{1}{(2π)^{d/2}|\sum|^{1/2}}exp(−\frac{1}{2}(x−\mu)^{′}\sum^{−1}(x−\mu))
\end{align}

 * Then we take Log Likelyhood of the above equation to get the discriminant function.

 \begin{align}
\boldsymbol{g_{i}(x) = \log p(x|\omega_i) + \log P(\omega_i)}
\end{align}
\begin{align}
\newline = -\frac{1}{2}(x - \mu_{i})^{-1} \sum^{-1}(x - \mu_{i}) - \frac{d}{2} \log (2 \pi) -\frac{1}{2} \log |\sum_i| + \log P(\omega_i)
\end{align}

 * Now we take the case 3 of the LDA, as we have different Standard Deviations for different classes:
   Case 3 is defined by:

\begin{align}
\boldsymbol{g_i = x^{t} W_i x + N_{i}^{t} x + B_{i0}}
\end{align}
\begin{align}
\newline
where, W_i = - \frac{1}{2} \sum_{i}^{-1} , N_{i} = \sum_{i}^{-1} \mu_{i} and B_{i0} = - \frac{1}{2} \mu_{i}^{t} \sum_{i}^{-1} \mu_{i} + ln P(w_i) - \frac{1}{2} ln |\sum_{i}|
\end{align}

```python
W=[]
N=[]
B=[]
g=[]
y_pred=[]
a=np.asarray(inv)
b=np.asarray(x)
for i in range(len(y)):
    W.append(a[i]*(-0.5))
    N.append(np.matmul(a[i],b[i]))
    b1=np.matmul(np.matmul(b[i].T, a[i]), b[i])
    B.append((-0.5)*b1)

N=np.asarray(N)
for i in range(10000):
    g=[]
    for j in range(10):
        g1=B[j]
        g2=np.matmul(N[j].T,X_test[i])
        g3=np.matmul(np.matmul(X_test[i].T, W[j]), X_test[i])
        g.append(g1+g2+g3)

    y_pred.append(np.argmax(g))
```

__10. 0-1 Loss function__
\begin{align}
\boldsymbol{\lambda =}
\begin{cases}
     0, i=j\\
    \newline 1,i \neq j\\
\end{cases}
\end{align}

The 0-1 loss function is used to find the accuracy. This is done by finding the the number of 1's when the predicted value is same as the data from test data and 0's when the predicted value is false.

```python
y_pred=np.asarray(y_pred)
y_test=np.asarray(y_test)
it=np.nditer(y_pred)
it1=np.nditer(y_test)
trues=[]
falses=[]
count=0

while not it.finished:
    if(it[0]==it1[0]):
        trues.append(1)
    it.iternext()
    it1.iternext()

while not it.finished:
    if(it[0]!=it1[0]):
        falses.append(0)
    it.iternext()
    it1.iternext()

count=np.count_nonzero(trues)+np.count_nonzero(falses==0)
print("Accuracy:",(count/len(y_test))*100)
```

    Accuracy: 81.73

__Q. Why it doesn't perform as good as many other methods on LeCuns web page?__

A. There are few reasons why LDA does not perform better than the other methods performed on LeCuns web page:
   * We are performing the LDA function based on the assumption that the MNIST dataset is a guassian distribution,
     but the guassian distribution does not perform well on high dimensional data images.
   * One more reason is that the LDA tries to find a hyperplane such that the variances are maximized which is not
     an optimal hyperplane whereas when we consider a single layer perceptron or a multilayer perceptron, it tries
     find optimal hyperplane to disjoin the labelled dataset.


```python
from IPython.display import Image
from IPython.core.display import HTML
from sympy import *; x,h,y = symbols("x h y")
Image(url= "https://i.imgur.com/f6YYF0w.png")
# source: https://www.math.purdue.edu/files/academic/courses/2015fall/MA15910/MA15910Lesson16Notes(revsp15).pdf

```

```python
#from class notes
#
#  derivative product rule  (capital letter means a Function() )
# # #
```

```python
F0 = 4
dF0 = -10
def dH():
    return  (    F0*0 - 1*dF0  )   /   F0**2                       #remember the derivative of a constant is 0
dH()
```

    0.625

```python
Image(url= "https://i.imgur.com/YnjJXoS.png")
```

```python
Image(url= "https://i.imgur.com/88uLtrE.png")
```

```python

```


# python libs

```python
import gluoncv as gcv
import mxnet as mx
# mxnet NDarray
from mxnet import nd

# MXNet Gluon Neural network layers.
from mxnet.gluon import nn
http://localhost:8888/notebooks/gray_rotate_two.ipynb#
# optional for displaying the image
import matplotlib.pyplot as plt
import numpy as np
import os
import math
# pip install pillow but import PIL
#from PIL import Image
```

# Load image and convert to grayscale

```python
a_file_name = '0.jpg'
nd_array_uint8_HWC3 = mx.image.imread(a_file_name, flag=1)
np_array = nd_array_uint8_HWC3.asnumpy()
gray_aug = mx.image.RandomGrayAug(p=1)
nd_array_f32_HWC3 = nd_array_uint8_HWC3.astype("float32")
nd_array_f3201_HWC3 = nd_array_f32_HWC3/255
aug = mx.image.RandomGrayAug(p=1)
nd_array_f3201_HWC3 = aug(nd_array_f3201_HWC3)
print('nd_array_f3201_HWC3.shape = ', nd_array_f3201_HWC3.shape)
gray_np_array_f3201_HWC3 = nd_array_f3201_HWC3.asnumpy()
#nd_array_f3201_HWC1 = nd_array_f3201_HWC3[:,:,0]  # this makes a shape like (y,x)
nd_array_f3201_HWC1 = nd_array_f3201_HWC3.slice(begin=(0,0,0), end=(428,641,1))  # this give (y,x,1)
print('nd_array_f3201_HWC1.shape = ', nd_array_f3201_HWC1.shape)
gray_np_array_f3201_HWC1 = nd_array_f3201_HWC1.asnumpy()
plt.imshow(gray_np_array_f3201_HWC1, cmap='gray')
```

# rotate

From this source

https://en.wikipedia.org/wiki/Rotation_matrix

\begin{vmatrix}
\cos(\theta) & -\sin(\theta) \\
\sin(\theta) & \cos(\theta)
\end{vmatrix}

rotates points in the xy plane counterclockwise through an angle θ with respect to the positive x axis about the origin of a two-dimensional Cartesian coordinate system. To perform the rotation on a plane point with standard coordinates v = (x, y), it should be written as a column vector, and multiplied by the matrix R:

\begin{align}
\begin{vmatrix}
\cos(\theta) & -\sin(\theta) \\
\sin(\theta) & \cos(\theta)
\end{vmatrix}
\begin{vmatrix} x \\ y \end{vmatrix} =
\begin{vmatrix}
x \cos(\theta) - y \sin(\theta) \\
x \sin(\theta) + y \cos(\theta)
\end{vmatrix}
\end{align}

```python
def build_rot_matrix(angle):
    #  a  b
    #  c  d

    # calculate terms
    a = math.cos(x)
    b = -math.sin(x)
    c = math.sin(x)
    d = math.cos(x)

    # print("a = ", a)
    # print("b = ", b)
    # print("c = ", c)
    # print("d = ", d)

    # build np array
    np_array = np.array([
        [a,b],
        [c,d]
    ])

    # build mx.nd array
    nd_array = mx.nd.array(np_array)

    return nd_array
```

```python
x=0.5*math.pi

rotate_90 = build_rot_matrix(x)
print('rotate by 90 nd array is ', rotate_90)
```

    rotate by 90 nd array is
    [[ 6.123234e-17 -1.000000e+00]
     [ 1.000000e+00  6.123234e-17]]
    <NDArray 2x2 @cpu(0)>

```python
print("image shape = ", nd_array_f3201_HWC1.shape)
print("image shape = ", nd_array_f3201_HWC1.shape)
#print("image = ", nd_array_f3201_HWC1)
#gray_np_array_f3201_HWC1 = nd_array_f3201_HWC1.asnumpy()
#plt.imshow(gray_np_array_f3201_HWC1, cmap='gray')
```

    image shape =  (427, 640, 1)
    image shape =  (427, 640, 1)

```python
print('rotate shape = ', rotate_90.shape)
# add a dimension
#rotate_90_221 = rotate_90.reshape(2,2,1)
#ndarray.stack([a, a]).T
rotate_90_221=nd.expand_dims(rotate_90, axis=0)
print('rotate shape = ', rotate_90_221.shape)
print('rotate = ', rotate_90_221)

```

    rotate shape =  (2, 2)
    rotate shape =  (1, 2, 2)
    rotate =
    [[[ 6.123234e-17 -1.000000e+00]
      [ 1.000000e+00  6.123234e-17]]]
    <NDArray 1x2x2 @cpu(0)>

```python

```

```python
# chance from HWC order to CHW order
nd_array_f3201_C1HW = nd.transpose(nd_array_f3201_HWC1, axes=(2,0,1) )
```

```python
nd_array_f3201_C1HW.shape
```

    (1, 427, 640)

```python
# I didn't know what I was doing.  I need to rethink this on a small scale.
```

```python
# rotate_img = nd_array_f3201_C1HW * rotate_90_221
```

# Reattempt of rotation with small image

This image is already grayscale so the augmentation step is skipped.

Also, I am trying to rotate a grayscale image.  As a result, I am removing the channel dimension.  FWIW, I am using grayscale so that I can learn how to do mxnet.  Later, I will try to rotate a color image.

```python
# Setup the sample image file paths
a_file_name = 'imgs/testy-grid.png'
nd_array_uint8_HWC1 = mx.image.imread(a_file_name, flag=0)
print('nd_array_uint8_HWC1.shape = ', nd_array_uint8_HWC1.shape)
np_array = nd_array_uint8_HWC1.asnumpy()
nd_array_f32_HWC1 = nd_array_uint8_HWC1.astype("float32")
nd_array_f3201_HWC1 = nd_array_f32_HWC1/255
print('nd_array_f3201_HWC1.shape = ', nd_array_f3201_HWC1.shape)
nd_array_f3201_HW = nd_array_f3201_HWC1[:,:,0]  # this makes a shape like (y,x)
print('nd_array_f3201_HW.shape = ', nd_array_f3201_HW.shape)
np_array_f3201_HW = nd_array_f3201_HW.asnumpy()
plt.imshow(np_array_f3201_HW, cmap='gray')
```

# Subset the image

```python
subset_nd_array_f3201_HW = nd_array_f3201_HW[52:56,94:98]
subset_np_array_f3201_HW = subset_nd_array_f3201_HW.asnumpy()
plt.imshow(subset_np_array_f3201_HW, cmap='gray')
```

# Iterate the pixels

A couple of links found

* https://mxnet.apache.org/versions/1.5.0/api/python/io/io.html
* https://mxnet.apache.org/versions/1.1.0/tutorials/basic/data.html
* https://mxnet.apache.org/versions/1.5.0/tutorials/basic/data.html
* https://mxnet.apache.org/versions/0.11.0/tutorials/r/CustomIteratorTutorial.html

Nothing to do with iterators, but it looks interesting for beginners

* https://mxnet.apache.org/versions/1.1.0/tutorials/basic/ndarray.html

A book, what is Julia?  It shows some mxnet.

* https://books.google.com/books?id=JMNiDwAAQBAJ&pg=PA147&lpg=PA147&dq=iterate+mxnet+iterate+a+matrix&source=bl&ots=8tWjjWCzsz&sig=ACfU3U3967qA3cNGQjQxuSoACFgRySqBhA&hl=en&sa=X&ved=2ahUKEwihrNXzsof2AhW1GDQIHVogD4YQ6AF6BAgREAM#v=onepage&q=iterate%20mxnet%20iterate%20a%20matrix&f=false

A channel RGB only image based iterator

* https://mxnet.apache.org/versions/1.5.0/api/python/image/image.html#mxnet.image.ImageIter

Finally one which looks appropriate

* https://mxnet.apache.org/versions/1.6/api/python/docs/api/ndarray/ndarray.html#mxnet.ndarray.broadcast_mul
* https://mxnet.incubator.apache.org/api/faq/new_op
* https://mxnet.apache.org/versions/1.1.0/api/python/io/io.html#mxnet.io.NDArrayIter

```python
subset_nd_array_f3201_HW
```

    [[1.         1.         1.         0.01176471]
     [1.         1.         1.         0.01176471]
     [1.         1.         1.         0.01176471]
     [0.01176471 0.01176471 0.01176471 0.01176471]]
    <NDArray 4x4 @cpu(0)>

# Iteration routines

## Numpy NDArrayIter

I found this via stackoverflow.  Not really interested in it though.  I want to stick with MXNet/NDArray API.

```python
a = np.arange(1,9).reshape(2,4)
b = -a

it = np.nditer((a, b), order='C', flags=('multi_index',))
for i, j in it:
    print(it.multi_index, i, j)
```

    (0, 0) 1 -1
    (0, 1) 2 -2
    (0, 2) 3 -3
    (0, 3) 4 -4
    (1, 0) 5 -5
    (1, 1) 6 -6
    (1, 2) 7 -7
    (1, 3) 8 -8

## NDArrayIter

[docs](https://mxnet.apache.org/versions/1.6/api/python/docs/api/mxnet/io/index.html#mxnet.io.NDArrayIter)

```
mxnet.io.NDArrayIter(data,
                     label=None,
                     batch_size=1,
                     shuffle=False,
                     last_batch_handle='pad',
                     data_name='data',
                     label_name='softmax_label')
```

```python
# batch 1
data = np.arange(16).reshape((1,4,4))
print('data is ', data)
labels = np.ones([1, 1])
dataiter = mx.io.NDArrayIter(data, labels, 1, False, last_batch_handle='discard')
for batch in dataiter:
    print('loop entry')
    #print ('batch.data[0] = ', batch.data[0].asnumpy())
    print ('batch.data[0] = ', batch.data[0].asnumpy())
    print ('batch.data[0].shape = ', batch.data[0].shape)
```

    data is  [[[ 0  1  2  3]
      [ 4  5  6  7]
      [ 8  9 10 11]
      [12 13 14 15]]]
    loop entry
    batch.data[0] =  [[[ 0.  1.  2.  3.]
      [ 4.  5.  6.  7.]
      [ 8.  9. 10. 11.]
      [12. 13. 14. 15.]]]
    batch.data[0].shape =  (1, 4, 4)

```python
# batch 3
data = np.arange(40).reshape((10,2,2))
labels = np.ones([10, 1])
dataiter = mx.io.NDArrayIter(data, labels, 3, True, last_batch_handle='discard')
for batch in dataiter:
    print('loop entry')
    print (batch.data[0].asnumpy())
    print (batch.data[0].shape)
```

    loop entry
    [[[12. 13.]
      [14. 15.]]

     [[32. 33.]
      [34. 35.]]

     [[ 0.  1.]
      [ 2.  3.]]]
    (3, 2, 2)
    loop entry
    [[[ 4.  5.]
      [ 6.  7.]]

     [[28. 29.]
      [30. 31.]]

     [[24. 25.]
      [26. 27.]]]
    (3, 2, 2)
    loop entry
    [[[ 8.  9.]
      [10. 11.]]

     [[16. 17.]
      [18. 19.]]

     [[20. 21.]
      [22. 23.]]]
    (3, 2, 2)

## NDArray Broadcast Multiply

[docs](https://mxnet.apache.org/versions/master/api/python/docs/api/legacy/ndarray/ndarray.html#mxnet.ndarray.broadcast_mul)

```python
# mxnet.ndarray.broadcast_mul(lhs=None, rhs=None, out=None, name=None, **kwargs)

# This is a good choice for submititing a patch

#
# x = [[ 1.,  1.,  1.],
#      [ 1.,  1.,  1.]]
#
# y = [[ 0.],
#     [ 1.]]
#
#mx.nd.broadcast_mul(x, y) = [[ 0.,  0.,  0.],
#                            [ 1.,  1.,  1.]]

x = nd.array([[ 1.,  1.,  1.],[ 1.,  1.,  1.]])
print('x is ', x)
y = nd.array([[ 0.],[ 1.]])
print('y is ', y)
z = mx.ndarray.broadcast_mul(x, y)
print ('z is ', z)
```

    x is
    [[1. 1. 1.]
     [1. 1. 1.]]
    <NDArray 2x3 @cpu(0)>
    y is
    [[0.]
     [1.]]
    <NDArray 2x1 @cpu(0)>
    z is
    [[0. 0. 0.]
     [1. 1. 1.]]
    <NDArray 2x3 @cpu(0)>

# array indexes

[docs](https://mxnet.apache.org/versions/master/api/python/docs/api/legacy/ndarray/contrib/index.html#mxnet.ndarray.contrib.index_array)

```
index_array(data=None,
            axes=_Null,
            out=None,
            name=None,
            **kwargs)
```

#### index_array ex 1

```python
x = mx.nd.ones((3, 2))

mx.nd.contrib.index_array(x)
```

    [[[0 0]
      [0 1]]

     [[1 0]
      [1 1]]

     [[2 0]
      [2 1]]]
    <NDArray 3x2x2 @cpu(0)>

#### index_array ex 2

```python
x = mx.nd.ones((3, 2, 2))

mx.nd.contrib.index_array(x, axes=(1, 0))
```

    [[[[0 0]
       [0 0]]

      [[1 0]
       [1 0]]]

     [[[0 1]
       [0 1]]

      [[1 1]
       [1 1]]]

     [[[0 2]
       [0 2]]

      [[1 2]
       [1 2]]]]
    <NDArray 3x2x2x2 @cpu(0)>

# index_copy

[docs](https://mxnet.apache.org/versions/master/api/python/docs/api/contrib/ndarray/index.html#mxnet.contrib.ndarray.index_copy)

```
index_copy(old_tensor=None,
           index_vector=None,
           new_tensor=None,
           out=None,
           name=None,
           **kwargs)
```

I've learned a little more about this api.  I've elaborated on the example below.

```python
x = mx.nd.zeros((5,3))
index = mx.nd.array([0,4,2])
t = mx.nd.array([[1,2,3],[4,5,6],[7,8,9]])

mx.nd.contrib.index_copy(x, index, t)
```

    [[1. 2. 3.]
     [0. 0. 0.]
     [7. 8. 9.]
     [0. 0. 0.]
     [4. 5. 6.]]
    <NDArray 5x3 @cpu(0)>

####  A better example of this api

```python
template_tensor = nd.zeros([5,3])
template_tensor
```

    [[0. 0. 0.]
     [0. 0. 0.]
     [0. 0. 0.]
     [0. 0. 0.]
     [0. 0. 0.]]
    <NDArray 5x3 @cpu(0)>

```python
source_indices = nd.array([0,2,4])
source_indices
```

    [0. 2. 4.]
    <NDArray 3 @cpu(0)>

```python
source_tensor = nd.array([[0,1,2],[3,4,5],[6,7,8]])
source_tensor
```

    [[0. 1. 2.]
     [3. 4. 5.]
     [6. 7. 8.]]
    <NDArray 3x3 @cpu(0)>

This api uses the src_indices to specify which elements to copy from the source tensor to output.  The output uses
the template tensor for unspecified values misssing from the src_indices array.

```python
nd.contrib.index_copy(template_tensor, source_indices, source_tensor)
```

    [[0. 1. 2.]
     [0. 0. 0.]
     [3. 4. 5.]
     [0. 0. 0.]
     [6. 7. 8.]]
    <NDArray 5x3 @cpu(0)>

Doing again using the named parameter form

```python
out_tensor = nd.ones([5,3])*9 # testing the out parameter with all nines

nd.contrib.index_copy(old_tensor=template_tensor,  # old_tensor is confusing
           index_vector=source_indices,            # index_vector is not as confusing, but not clear
           new_tensor=source_tensor,               # new_tensor is confusing.  Its not new nor is it modifed.
           out=out_tensor,                         # using out paramter as foo, since not sure what it is ...
           name=None,)
```

    [[0. 1. 2.]
     [0. 0. 0.]
     [3. 4. 5.]
     [0. 0. 0.]
     [6. 7. 8.]]
    <NDArray 5x3 @cpu(0)>

```python
# Hmm, this shows that out_tensor is the name of the output.  So, what happens is
# all the values from the template_tensor is copied to the output, with the exception
# of the rows specified by the index_vector.  In that case, the values in the new_tensor
# are are copied to output
out_tensor
```

    [[0. 1. 2.]
     [0. 0. 0.]
     [3. 4. 5.]
     [0. 0. 0.]
     [6. 7. 8.]]
    <NDArray 5x3 @cpu(0)>

## Testing to rotate the images

```python
# batch 1
in_data = nd.arange(4).reshape((1,2,2))
print('in_data is ', in_data)
```

    in_data is
    [[[0. 1.]
      [2. 3.]]]
    <NDArray 1x2x2 @cpu(0)>

```python
# setup an ouptut array with all nines, so I can see overwrites with new values
out_data = nd.ones(4).reshape((2,2))*9
# out_data also needs to be in batch form
out_data = nd.expand_dims(out_data, axis=0)
print('out_data is ', out_data)
```

    out_data is
    [[[9. 9.]
      [9. 9.]]]
    <NDArray 1x2x2 @cpu(0)>

```python
# setup a rotation matrix
rotate_data = rotate_90
print('rotate_90 ', rotate_data)
```

    rotate_90
    [[ 6.123234e-17 -1.000000e+00]
     [ 1.000000e+00  6.123234e-17]]
    <NDArray 2x2 @cpu(0)>

```python
# NDArrayIter(data, label=None, batch_size=1, shuffle=False,
#            last_batch_handle='pad', data_name='data',
#            label_name='softmax_label')
#
# Ignore the label parameter.
dataiter = mx.io.NDArrayIter(in_data, batch_size=1, shuffle=False, last_batch_handle='discard')
#batch_index = [0]
for batch in dataiter:
    print('loop entry - a single batch - a single image in batch.data[0] from what is in in_data')
    #print ('batch.data[0] = ', batch.data[0].asnumpy())
    #print ('batch.data[0].shape = ', batch.data[0].shape)
    # Does this copy or get an alias to the input image?
    input_img = batch.data[0]
    #print ('input_img = ', input_img.asnumpy())
    print ('input_img.shape = ', input_img.shape)

    # this will print the axis including the batch axis
    # print('indicies are ', mx.nd.contrib.index_array(entire_img) )
    # this will print the x,y axis and ignore the batch
    input_img_indexes = mx.nd.contrib.index_array(input_img, axes=(1, 2))
    print('indexes are: ', input_img_indexes)
    print ('input_img_indexes.shape = ', input_img_indexes.shape)

    # copy data from input to output
    # Note, this seems to also use the batch axis?  Its shape (1,2,2)
    #out_data = input_img

    # Try to assign input data to output data based upon indicies
    orig_indexes = mx.nd.reshape(input_img_indexes, shape=(4,2))
    print('new_indexes ', new_indexes)
    orig_indexes = orig_indexes.astype("float32")

    # I've seen one variant where matrix-matrix multiply
    # is done with .T.  why?
    new_indexes = nd.dot(orig_indexes, rotate_data)
    #print('result = ', result)
    new_indexes = new_indexes.astype('int64')
    print('new_indexes = ', new_indexes)
    new_indexes = new_indexes + nd.array(nd.array([0, 1])).astype('int64')
    print('new_indexes after shift to positive', new_indexes)

```

    loop entry - a single batch - a single image in batch.data[0] from what is in in_data
    input_img.shape =  (1, 2, 2)
    indexes are:
    [[[[0 0]
       [0 1]]

      [[1 0]
       [1 1]]]]
    <NDArray 1x2x2x2 @cpu(0)>
    input_img_indexes.shape =  (1, 2, 2, 2)
    new_indexes
    [[0 1]
     [1 1]
     [0 0]
     [1 0]]
    <NDArray 4x2 @cpu(0)>
    new_indexes =
    [[ 0  0]
     [ 1  0]
     [ 0 -1]
     [ 1 -1]]
    <NDArray 4x2 @cpu(0)>
    new_indexes after shift to positive
    [[0 1]
     [1 1]
     [0 0]
     [1 0]]
    <NDArray 4x2 @cpu(0)>

```python
print('out_data is ', out_data)
print('in_data is ', in_data)
print('rotate_data is ', rotate_data)
print('orig_indexes is ', orig_indexes)
print('new_indexes is ', new_indexes)
print('new_indexes.dtype is ', new_indexes.dtype)
```

    out_data is
    [[[9. 9.]
      [9. 9.]]]
    <NDArray 1x2x2 @cpu(0)>
    in_data is
    [[[0. 1.]
      [2. 3.]]]
    <NDArray 1x2x2 @cpu(0)>
    rotate_data is
    [[ 6.123234e-17 -1.000000e+00]
     [ 1.000000e+00  6.123234e-17]]
    <NDArray 2x2 @cpu(0)>
    orig_indexes is
    [[0. 0.]
     [0. 1.]
     [1. 0.]
     [1. 1.]]
    <NDArray 4x2 @cpu(0)>
    new_indexes is
    [[0 1]
     [1 1]
     [0 0]
     [1 0]]
    <NDArray 4x2 @cpu(0)>
    new_indexes.dtype is  <class 'numpy.int64'>

# rewriting to match notes

To avoid the batch dimension, and simply problem

```python
orig_image = in_data[0]
print('orig_image ', orig_image)
```

    orig_image
    [[0. 1.]
     [2. 3.]]
    <NDArray 2x2 @cpu(0)>

```python
rotated_image=out_data[0]
print('rotated_image', rotated_image)
```

    rotated_image
    [[9. 9.]
     [9. 9.]]
    <NDArray 2x2 @cpu(0)>

```python
print('orig_indexes', orig_indexes)
```

    orig_indexes
    [[0. 0.]
     [0. 1.]
     [1. 0.]
     [1. 1.]]
    <NDArray 4x2 @cpu(0)>

```python
print('new_indexes', new_indexes)
```

    new_indexes
    [[0 1]
     [1 1]
     [0 0]
     [1 0]]
    <NDArray 4x2 @cpu(0)>

#### do pixel 0,0

```python
orig_indexes[0,0]  # row 0, column 0 = the first row index is in the first row column 0
```

    [0.]
    <NDArray 1 @cpu(0)>

```python
new_indexes[0,1] # row 0, column 1
```

    [1]
    <NDArray 1 @cpu(0)>

```python
orig_image[new_indexes[0,0], new_indexes[0,1]]  # the new 0,0 pixel should be nine, pulled from 1,1
```

    [1.]
    <NDArray 1 @cpu(0)>

```python
# Did a 1 move ot pixel 0,0?
rotated_image[orig_indexes[0,0], orig_indexes[0,1]] = orig_image[new_indexes[0,0], new_indexes[0,1]]
rotated_image
```

    [[1. 9.]
     [9. 9.]]
    <NDArray 2x2 @cpu(0)>

#### do pixel 0,1

```python
rotated_image[orig_indexes[1,0], orig_indexes[1,1]] = orig_image[new_indexes[1,0], new_indexes[1,1]]
rotated_image
```

    [[1. 3.]
     [9. 9.]]
    <NDArray 2x2 @cpu(0)>

#### do pixel 1,0

```python
rotated_image[orig_indexes[2,0], orig_indexes[2,1]] = orig_image[new_indexes[2,0], new_indexes[2,1]]
rotated_image
```

    [[1. 3.]
     [0. 9.]]
    <NDArray 2x2 @cpu(0)>

#### do pixel 1,1

```python
rotated_image[orig_indexes[3,0], orig_indexes[3,1]] = orig_image[new_indexes[3,0], new_indexes[3,1]]
rotated_image
```

    [[1. 3.]
     [0. 2.]]
    <NDArray 2x2 @cpu(0)>

# Is that rotated 90?

```python
print('orig image', orig_image)
print('rotated_image', rotated_image)
```

    orig image
    [[0. 1.]
     [2. 3.]]
    <NDArray 2x2 @cpu(0)>
    rotated_image
    [[1. 3.]
     [0. 2.]]
    <NDArray 2x2 @cpu(0)>

# Looks like it to me.  I call that success, but not DFD success

## How to do this implicitly?

```python
orig_image[new_indexes] # nope
```

    [[[0. 1.]
      [2. 3.]]

     [[2. 3.]
      [2. 3.]]

     [[0. 1.]
      [0. 1.]]

     [[2. 3.]
      [0. 1.]]]
    <NDArray 4x2x2 @cpu(0)>

# this looks promising

```python
# reset rotated_image
# setup an ouptut array with all nines, so I can see overwrites with new values
out_data = nd.ones(4).reshape((2,2))*9
# out_data also needs to be in batch form
out_data = nd.expand_dims(out_data, axis=0)
#print('out_data is ', out_data)
rotated_image=out_data[0]
print('rotated_image', rotated_image)
```

    rotated_image
    [[9. 9.]
     [9. 9.]]
    <NDArray 2x2 @cpu(0)>

```python
rotated_image = orig_image[new_indexes[:,0],new_indexes[:,1]]
rotated_image
```

    [1. 3. 0. 2.]
    <NDArray 4 @cpu(0)>

```python
rotated_image.reshape(2,2)
```

    [[1. 3.]
     [0. 2.]]
    <NDArray 2x2 @cpu(0)>

# another try with a 3x3

```python
# batch 1
in_data = nd.arange(9).reshape((1,3,3))
print('in_data is ', in_data)
```

    in_data is
    [[[0. 1. 2.]
      [3. 4. 5.]
      [6. 7. 8.]]]
    <NDArray 1x3x3 @cpu(0)>

```python
# setup an ouptut array with all nines, so I can see overwrites with new values
out_data = nd.ones(9).reshape((3,3))*9
# out_data also needs to be in batch form
out_data = nd.expand_dims(out_data, axis=0)
print('out_data is ', out_data)
rotated_image=out_data[0]
print('rotated_image', rotated_image)
```

    out_data is
    [[[9. 9. 9.]
      [9. 9. 9.]
      [9. 9. 9.]]]
    <NDArray 1x3x3 @cpu(0)>
    rotated_image
    [[9. 9. 9.]
     [9. 9. 9.]
     [9. 9. 9.]]
    <NDArray 3x3 @cpu(0)>

```python
# setup a rotation matrix
rotate_data = rotate_90
print('rotate_90 ', rotate_data)
```

    rotate_90
    [[ 6.123234e-17 -1.000000e+00]
     [ 1.000000e+00  6.123234e-17]]
    <NDArray 2x2 @cpu(0)>

```python
# NDArrayIter(data, label=None, batch_size=1, shuffle=False,
#            last_batch_handle='pad', data_name='data',
#            label_name='softmax_label')
#
# Ignore the label parameter.
dataiter = mx.io.NDArrayIter(in_data, batch_size=1, shuffle=False, last_batch_handle='discard')
#batch_index = [0]
for batch in dataiter:

    print('loop entry - a single batch - a single image in batch.data[0] from what is in in_data')
    #print ('batch.data[0] = ', batch.data[0].asnumpy())
    #print ('batch.data[0].shape = ', batch.data[0].shape)
    # Does this copy or get an alias to the input image?
    input_img = batch.data[0]
    #print ('input_img = ', input_img.asnumpy())
    print ('input_img.shape = ', input_img.shape)

    # this will print the axis including the batch axis
    # print('indicies are ', mx.nd.contrib.index_array(entire_img) )
    # this will print the x,y axis and ignore the batch
    input_img_indexes = mx.nd.contrib.index_array(input_img, axes=(1, 2))
    print('indexes are: ', input_img_indexes)
    print ('input_img_indexes.shape = ', input_img_indexes.shape)

    # Try to assign input data to output data based upon indicies
    orig_indexes = mx.nd.reshape(input_img_indexes, shape=(9,2))
    print('orig_indexes ', orig_indexes)
    orig_indexes = orig_indexes.astype("float32")

    # I've seen one variant where matrix-matrix multiply
    # is done with .T.  why?
    new_indexes = nd.dot(orig_indexes, rotate_data)
    print('new_indexes = ', new_indexes)
    new_indexes = new_indexes.astype('int64')
    print('new_indexes = ', new_indexes)
    new_indexes = new_indexes + nd.array(nd.array([0, 2])).astype('int64')
    print('new_indexes after shift to positive', new_indexes)

    print('rotated_image ', rotated_image)
    print('rotated_image.shape ', rotated_image.shape)
    rotated_image = orig_image[new_indexes[:,0],new_indexes[:,1]]
    print('rotated_image ', rotated_image)

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

### Hmm, try this contrib api again

```python
# for reference
x = mx.nd.zeros((5,3))
t = mx.nd.array([[1,2,3],[4,5,6],[7,8,9]])
index = mx.nd.array([0,4,2])  # <--- this is interesting. It has a zero length axes?????
# x is a template for output
# index is the indexes?
# t is "new_tensor?", its the source tensor?
mx.nd.contrib.index_copy(x, index, t)
```

    [[1. 2. 3.]
     [0. 0. 0.]
     [7. 8. 9.]
     [0. 0. 0.]
     [4. 5. 6.]]
    <NDArray 5x3 @cpu(0)>

```python
# hmm, the t has three axis specification. perhaps it requires a batch parameter?
index
```

    [0. 4. 2.]
    <NDArray 3 @cpu(0)>

```python
# Did x change? No, it is just a template for output array?
x
```

    [[0. 0. 0.]
     [0. 0. 0.]
     [0. 0. 0.]
     [0. 0. 0.]
     [0. 0. 0.]]
    <NDArray 5x3 @cpu(0)>

```python
template_img = nd.zeros([2,2])
template_img_batch = nd.expand_dims(template_img, axis=0)
template_img_batch
```

    [[[0. 0.]
      [0. 0.]]]
    <NDArray 1x2x2 @cpu(0)>

```python
# this is the wrong shape.  It gives an error. possibly due to the lack of batch
new_indexes
```

    [[0 1]
     [1 1]
     [0 0]
     [1 0]]
    <NDArray 4x2 @cpu(0)>

```python
new_indexes_batch = nd.expand_dims(new_indexes, axis=0)
new_indexes_batch
```

    [[[0 1]
      [1 1]
      [0 0]
      [1 0]]]
    <NDArray 1x4x2 @cpu(0)>

```python
# perhaps the orignal image also needs a batch?
orig_image_batch = nd.expand_dims(orig_image, axis=0)
orig_image_batch
```

    [[[0. 1.]
      [2. 3.]]]
    <NDArray 1x2x2 @cpu(0)>

```python
# Hmm. adding back batch dims? Or running in circles of confusion?
print('orig_image_batch.shape = ', orig_image_batch.shape )
print('new_indexes_batch.shape = ', new_indexes_batch.shape )
print('template_img_batch.shape = ', template_img_batch.shape )
```

    orig_image_batch.shape =  (1, 2, 2)
    new_indexes_batch.shape =  (1, 4, 2)
    template_img_batch.shape =  (1, 2, 2)

```python
# mx.nd.contrib.index_copy(template_img, new_indexes, orig_image_batch)
# mx.nd.contrib.index_copy(template_img, new_indexes_batch, orig_image_batch)
#mx.nd.contrib.index_copy(template_img_batch, new_indexes_batch, orig_image_batch)
# best guess at the api, would be
# * template img is not batched, but it is three columns?
# *
mx.nd.contrib.index_copy(template_img, new_indexes_batch, orig_image_batch)
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python
orig_indexes
```

```python
in_data[0]
```

```python

```

```python
in_data[0,1]
```

```python

```

```python
orig_indexes
```

```python

```

```python

```

```python
a = np.arange(50).reshape(10, 5) # Array to be indexed
# b = 9 7 5 3 1  which means start with row 9 and step down to
b = np.arange(9, -1, -2) # Indexing array. Start at 9, step down by 2 each time until 1
print('a = ', a)
print('b (indexing) = ', b)
result = a[b, np.arange(a.shape[1])]
print('result = ', result)
```

```python
out_data[0] = in_data[0,new_indexes]
print('out_data is ', out_data)
```

```python
out_data[0,new_indexes] = in_data
print('out_data is ', out_data)
```

```python
a = nd.arange(50).reshape(10, 5) # Array to be indexed
b = nd.arange(9, -1, -2) # Indexing array
print('a = ', a)
print('b (indexing) = ', b)
result = a[b, nd.arange(a.shape[1])]
print('result = ', result)
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python
out_data[new_indexes]
```

```python
new_indexes
```

```python
new_indexes[:,0], new_indexes[:,1]
```

```python
out_data[ 0, new_indexes[:,0]: , new_indexes[:,1]: ]
```

```python
out_data[0, 0-2:, :]
```

```python
out_data[0, np.array([0,1,2]):, :]
```

```python
# x = mx.nd.zeros((5,3))
# t = mx.nd.array([[1,2,3],[4,5,6],[7,8,9]])
# index = mx.nd.array([0,4,2])

mx.nd.contrib.index_copy(out_data[0], new_indexes, in_data[0])
```

```python
out_data[0]
```

```python
in_data[0]
```

```python
new_indexes
```

```python
out_data[0,new_indexes[0,0],0]
```

```python

```


# Documentation for SeisAcoustic

## Theory

### Acoustic wave eqaution
The first order acoustic wave equation can be expressed as
$$\begin{align}
\rho \frac{\partial v_z}{\partial t} &=  \frac{\partial }{\partial z} \left( p_z + p_x \right)  \\
\rho \frac{\partial v_x}{\partial t} &=  \frac{\partial }{\partial x} \left( p_z + p_x \right) \\
\frac{\partial p_z}{\partial t} &= k \frac{\partial v_z}{\partial z} + S(t)\\
\frac{\partial p_x}{\partial t} &= k \frac{\partial v_x}{\partial x} \\
\end{align},$$
where $\rho$ is density, $k$ is bulk modulus, the pressure $p$ is split into two unphysical components $p_z$ and $p_x$, where $p = p_z + p_x$. These two components are used to account for the PML absorbing boundary conditions. s(t) is the pressure source term applied to the sum of the last two equations.

### Born approximation

Assure the density term is constant, the scatter wavefield is produced by the perturbation of bulk modulus $k$, so the new wavefield is governed by the following equations:
$$
\begin{align}
\rho \frac{\partial (v_z + \Delta v_z)}{\partial t} &=  \frac{\partial }{\partial z} \left( p_{z} + p_x + \Delta p_z  + \Delta p_x \right) \\
\rho \frac{\partial (v_x+ \Delta v_x)}{\partial t} &=  \frac{\partial }{\partial x} \left( p_{z} + p_x + \Delta p_z  + \Delta p_x \right) \\
\frac{\partial (p_{z} + \Delta p_{z})}{\partial t} &=(k+\Delta k) \frac{\partial (v_z + \Delta v_z)}{\partial z} + s(t)\\
\frac{\partial (p_{x} + \Delta p_x)}{\partial t} &= (k+\Delta k) \frac{\partial (v_x + \Delta v_x)}{\partial x} \\
\end{align}
$$
Subtracted by the original wave equation without model parameters pertubations and ignore the higher order terms, we get the scattered wave field equations as
$$
\begin{align}
\rho \frac{\partial \Delta v_z}{\partial t} &=  \frac{\partial }{\partial z} \left(\Delta p_z  + \Delta p_x \right) \\
\rho \frac{\partial \Delta v_x}{\partial t} &=  \frac{\partial }{\partial x} \left(\Delta p_z  + \Delta p_x \right) \\
\frac{\partial  \Delta p_{z}}{\partial t} &= k \frac{\partial  \Delta v_z}{\partial z} + \Delta k \frac{\partial v_z}{\partial z}\\
\frac{\partial \Delta p_x}{\partial t} &= k \frac{\partial  \Delta v_x}{\partial x} + \Delta k \frac{\partial v_x}{\partial x} \\
\end{align}
$$
combine the last two equations, we get
$$
\frac{\partial (\Delta p_z + \Delta p_x)}{\partial t} = k (\frac{\partial  \Delta v_z}{\partial z} + \frac{\partial  \Delta v_x}{\partial x} ) + \Delta k (\frac{\partial v_z}{\partial z} + \frac{\partial v_x}{\partial x}) \\
$$
We know that $$\frac{\partial (p_z + p_x)}{\partial t} = k( \frac{\partial v_z}{\partial z} + \frac{\partial v_x}{\partial x}) + S(t)$$
if we $p = p_z + p_x$ and using the equations $$ \frac{\partial v_z}{\partial z} + \frac{\partial v_x}{\partial x} = \frac{ \frac{\partial (p_z + p_x)}{\partial t} - S(t)}{k} $$

So the born approximation is obtained as
$$
\begin{align}
\rho \frac{\partial \Delta v_z}{\partial t} &=  \frac{\partial }{\partial z} \left(\Delta p_z  + \Delta p_x \right) \\
\rho \frac{\partial \Delta v_x}{\partial t} &=  \frac{\partial }{\partial x} \left(\Delta p_z  + \Delta p_x \right) \\
\frac{\partial  \Delta p_{z}}{\partial t} &= k \frac{\partial  \Delta v_z}{\partial z} +  \frac{ \frac{\partial p}{\partial t} - S(t)}{k} \Delta k \\
\frac{\partial \Delta p_x}{\partial t} &= k \frac{\partial  \Delta v_x}{\partial x}  \\
\end{align}
$$
$\frac{ \frac{\partial p}{\partial t} - S(t)}{k}$ is the so-called source-side wavefield and $\Delta k$ is the model parameter permutation.

### Name convention

1. Struct: camel case, for exmaple the struct for model paramters are named as *ModelParams*.
2. function and variable names use lower-case litters and dash line is used to separate words, for example *fd_coefficients*

### The location of sources and receivers are specified by grid index, and the index is start from 1.

## Discretization of Acoustic wave equation

We use stagger grid finite difference to discretize the acoustic wave equation, the particle $v_z$ is put on the stagger grid $[i+\frac{1}{2},j,k]$, $v_x$ is put on the stagger grid $[i, j+\frac{1}{2}, k]$, and the pressure component is put on the grid $[i,j,k+\frac{1}{2}]$. Where $i,j,k$ are the index in vertical, horizontal, time directions, respectively.

### updating ${\bf v}_z$

We move the density term to the right of the equation and expressed as buoyancy.
\begin{equation}
\frac{\partial v_z}{\partial t} + \gamma_z v_z = b \frac{\partial p}{\partial z}
\end{equation}

The partial derivative operator is approximated by finite difference and the center of both side of the equation is on $[i+\frac{1}{2},j,k+\frac{1}{2}]$. Typically, the time step is much smaller than the spatial grid size, The finite difference is second order in time direction, and the spatial derivative is derived in any user-defined order.

$$
\frac{v_z[i+\frac{1}{2},j,k+1] - v_z[i+\frac{1}{2},j,k]}{dt} + \gamma_z[i+\frac{1}{2}]
\frac{v_z[i+\frac{1}{2},j,k+1] + v_z[i+\frac{1}{2},j,k]}{2}  = b[i+\frac{1}{2},j] \frac{1}{dz} \left( \sum_{n=1}^N  a_n (p[i+n,j,k+\frac{1}{2}] - p[i-n+1,j,k+\frac{1}{2}])  \right)
$$

Times $2dt$ on both side of the equation, we get
$$
2(v_z[i+\frac{1}{2},j,k+1] - v_z[i+\frac{1}{2},j,k]) + dt\, \gamma_z[i+\frac{1}{2}]
(v_z[i+\frac{1}{2},j,k+1] + v_z[i+\frac{1}{2},j,k])  = 2b[i+\frac{1}{2},j] \frac{dt}{dz} \left( \sum_{n=1}^N  a_n (p[i+n,j,k+\frac{1}{2}] - p[i-n+1,j,k+\frac{1}{2}])  \right)
$$

The left side only keep the terms involved with $v_z[i+\frac{1}{2},j,k+1]$, and all the other terms are moved to the right hand side.

$$
(2+ dt\, \gamma_z[i+\frac{1}{2}])v_z[i+\frac{1}{2},j,k+1] =  (2- dt\, \gamma_z[i+\frac{1}{2}])v_z[i+\frac{1}{2},j,k] + 2b[i+\frac{1}{2},j]\frac{dt}{dz}
\left( \sum_{n=1}^N  a_n (p[i+n,j,k+\frac{1}{2}] - p[i-n+1,j,k+\frac{1}{2}])  \right)
$$

The final form of the equation become
$$
v_z[i+\frac{1}{2},j,k+1] =  \frac{2- dt\, \gamma_z[i+\frac{1}{2}]}{(2+ dt\, \gamma_z[i+\frac{1}{2}])}v_z[i+\frac{1}{2},j,k] + \frac{2b[i+\frac{1}{2},j] \, dt}{(2+ dt\, \gamma_z[i+\frac{1}{2}])}
\left(\frac{1}{dz} \sum_{n=1}^N  a_n (p[i+n,j,k+\frac{1}{2}] - p[i-n+1,j,k+\frac{1}{2}])  \right)
$$

#### Details about coding:
we create a vector ${\bf M}_{v_z}^{v_z}$ length of $N_z$, which save the elements of pre-computed $\frac{2- dt\, \gamma_z[i+\frac{1}{2}]}{(2+ dt\, \gamma_z[i+\frac{1}{2}])}$, another vector called ${\bf M}_p^{v_z}$ length of $N_z \cdot N_x$, which save the elements of pre-computed $\frac{2b[i+\frac{1}{2},j]}{(2+ dt\, \gamma_z[i+\frac{1}{2}])}\frac{dt}{dz}$. In our implementation, the wavefield are savd in a long vector.

For the spatial derivative, we use a 5 point grid, fourth-order finite difference as an example. We write them into matrix times vector form, which facilitate us to derive its associated adjoint operator.
$$
\begin{bmatrix}
\frac{\partial p}{\partial z}[1+\frac{1}{2}] \\
\frac{\partial p}{\partial z}[2+\frac{1}{2}] \\
\frac{\partial p}{\partial z}[3+\frac{1}{2}] \\
\frac{\partial p}{\partial z}[4+\frac{1}{2}] \\
\frac{\partial p}{\partial z}[5+\frac{1}{2}] \\
\end{bmatrix}
= \begin{bmatrix}
-a_1 & a_1 & a_2 & & \\
-a_2 & -a_1 & a_1 & a_2 & \\
& -a_2 & -a_1 & a_1 & a_2 \\
& & -a_2 & -a_1 & a_1 \\
& & & -a_2 & -a_1 \\
\end{bmatrix}
\begin{bmatrix}
p[1] \\
p[2] \\
p[3] \\
p[4] \\
p[5] \\
\end{bmatrix}
$$

In above expressions, we assum the value of wavefield outside the computation area is zero. The adjoint operator is given as
$$
\begin{bmatrix}
p[1] \\
p[2] \\
p[3] \\
p[4] \\
p[5] \\
\end{bmatrix}
= - \begin{bmatrix}
a_1 & a_2 & & & \\
-a_1 & a_1 & a_2 &  & \\
-a_2 & -a_1 & a_1 & a_2 &  \\
& -a_2& -a_1 & a_1 & a_2 \\
& & -a_2 & -a_1 & a_1 \\
\end{bmatrix}
\begin{bmatrix}
\frac{\partial p}{\partial z}[1+\frac{1}{2}] \\
\frac{\partial p}{\partial z}[2+\frac{1}{2}] \\
\frac{\partial p}{\partial z}[3+\frac{1}{2}] \\
\frac{\partial p}{\partial z}[4+\frac{1}{2}] \\
\frac{\partial p}{\partial z}[5+\frac{1}{2}] \\
\end{bmatrix}
$$

### updating ${\bf v}_x$

With respect to the second equation
$$
\frac{\partial v_x}{\partial t} + \gamma_x v_x = b \frac{\partial p}{\partial x}
$$

We just give the final form of the discretized equation as
$$
v_x[i,j+\frac{1}{2},k+1] =  \frac{2- dt\, \gamma_x[j+\frac{1}{2}]}{(2+ dt\, \gamma_x[j+\frac{1}{2}])}v_x[i,j+\frac{1}{2},k] + \frac{2b[i,j+\frac{1}{2}]}{(2+ dt\, \gamma_x[j+\frac{1}{2}])}\frac{dt}{dx}
\left( \sum_{n=1}^N  a_n (p[i,j+n,k+\frac{1}{2}] - p[i,j-n+1,k+\frac{1}{2}]) \right)
$$

#### Details about coding:
we create a vector ${\bf M}_{v_x}^{v_x}$ length of $N_x$, which save the elements of pre-computed $\frac{2- dt\, \gamma_x[j+\frac{1}{2}]}{(2+ dt\, \gamma_x[j+\frac{1}{2}])}$, another vector called ${\bf M}_p^{v_x}$ length of $N_z \cdot N_x$, which save the elements of pre-computed $\frac{2b[i,j+\frac{1}{2}]}{(2+ dt\, \gamma_x[j+\frac{1}{2}])}\frac{dt}{dx}$.

For the spatial derivative, we use a 5 point grid, fourth-order finite difference as an example. We write them into matrix times vector form, which facilitate us to derive its associated adjoint operator.
$$
\begin{bmatrix}
\frac{\partial p}{\partial x}[1+\frac{1}{2}] \\
\frac{\partial p}{\partial x}[2+\frac{1}{2}] \\
\frac{\partial p}{\partial x}[3+\frac{1}{2}] \\
\frac{\partial p}{\partial x}[4+\frac{1}{2}] \\
\frac{\partial p}{\partial x}[5+\frac{1}{2}] \\
\end{bmatrix}
= \begin{bmatrix}
-a_1 & a_1 & a_2 & & \\
-a_2 & -a_1 & a_1 & a_2 & \\
& -a_2 & -a_1 & a_1 & a_2 \\
& & -a_2 & -a_1 & a_1 \\
& & & -a_2 & -a_1 \\
\end{bmatrix}
\begin{bmatrix}
p[1] \\
p[2] \\
p[3] \\
p[4] \\
p[5] \\
\end{bmatrix}
$$

In above expressions, we assum the value of wavefield outside the computation area is zero. The adjoint operator is given as
$$
\begin{bmatrix}
p[1] \\
p[2] \\
p[3] \\
p[4] \\
p[5] \\
\end{bmatrix}
= - \begin{bmatrix}
a_1 & a_2 & & & \\
-a_1 & a_1 & a_2 &  & \\
-a_2 & -a_1 & a_1 & a_2 &  \\
& -a_2& -a_1 & a_1 & a_2 \\
& & -a_2 & -a_1 & a_1 \\
\end{bmatrix}
\begin{bmatrix}
\frac{\partial p}{\partial x}[1+\frac{1}{2}] \\
\frac{\partial p}{\partial x}[2+\frac{1}{2}] \\
\frac{\partial p}{\partial x}[3+\frac{1}{2}] \\
\frac{\partial p}{\partial x}[4+\frac{1}{2}] \\
\frac{\partial p}{\partial x}[5+\frac{1}{2}] \\
\end{bmatrix}
$$

The last equation is give as
$$
\frac{\partial p}{\partial t} = k(\frac{\partial v_z}{\partial z} + \frac{\partial v_x}{\partial x}) + \delta(z-z_0, x-x_0)f(t)
$$

where $f(t)$ is the source signature and $(z_0,x_0)$ indicate the location of source.

To accomodate the PML obsorbing boundary conditions, the pressure component is splited into two unphysical terms $p_z$ and $p_x$, and $p = p_z + p_x$. We get two equations for pressure component.

$$
\begin{align}
\frac{\partial p_z}{\partial t} + \gamma_z p_z &= k \frac{\partial v_z}{\partial z} \\
\frac{\partial p_x}{\partial t} + \gamma_x p_x &= k \frac{\partial v_x}{\partial x} +  \delta(z-z_0, x-x_0)f(t) \\
\end{align}
$$

As the source term is additive, so it can be added to either one of the above equations.

### updating ${\bf p}_z$

$$
\frac{\partial p_z}{\partial t} + \gamma_z p_z = k \frac{\partial v_z}{\partial z}
$$

Where $k$ is bulk modulus, and $k = \rho v^2$

The partial derivative operator is approximated by finite difference and the center of both side of the equation is on $[i,j,k+1]$.

$$
\frac{p_z[i,j,k+\frac{3}{2}] - p_z[i,j,k+\frac{1}{2}]}{dt} + \gamma_z[i]
\frac{p_z[i,j,k+\frac{3}{2}] + p_z[i,j,k+\frac{1}{2}]}{2} = k[i,j] \frac{1}{dz} \left( \sum_{n=1}^N  a_n (v_z[i+n-\frac{1}{2},j,k+1] - v_z[i-n+\frac{1}{2},j,k+1]) \right)
$$

Times $2dt$ on both side of the equation, we get
$$
2(p_z[i,j,k+\frac{3}{2}] - p_z[i,j,k+\frac{1}{2}]) + dt \, \gamma_z[i]
(p_z[i,j,k+\frac{3}{2}] + p_z[i,j,k+\frac{1}{2}]) = 2k[i,j] \frac{dt}{dz} \left( \sum_{n=1}^N  a_n (v_z[i+n-\frac{1}{2},j,k+1] - v_z[i-n+\frac{1}{2},j,k+1]) \right)
$$

After some simple operations, we can get
$$
(2+dt \, \gamma_z[i])p_z[i,j,k+\frac{3}{2}] = (2-dt \, \gamma_z[i]) p_z[i,j,k+\frac{1}{2}] + 2k[i,j] \frac{dt}{dz} \left( \sum_{n=1}^N  a_n (v_z[i+n-\frac{1}{2},j,k+1] - v_z[i-n+\frac{1}{2},j,k+1]) \right)
$$

the final form for this equation is given as
$$
p_z[i,j,k+\frac{3}{2}] = \frac{2-dt \, \gamma_z[i]}{2+dt \, \gamma_z[i]} p_z[i,j,k+\frac{1}{2}] + \frac{2k[i,j]}{2+dt \, \gamma_z[i]}\frac{dt}{dz} \left( \sum_{n=1}^N  a_n (v_z[i+n-\frac{1}{2},j,k+1] - v_z[i-n+\frac{1}{2},j,k+1]) \right)
$$

#### Details about coding:
we create a vector ${\bf M}_{p_z}^{p_z}$ length of $N_z$, which save the elements of pre-computed $\frac{2- dt\, \gamma_z[i]}{(2+ dt\, \gamma_z[i])}$, another vector called ${\bf M}_{v_z}^{p_z}$ length of $N_z \cdot N_x$, which save the elements of pre-computed $\frac{2k[i,j]}{(2+ dt\, \gamma_z[i])}\frac{dt}{dz}$.

For the spatial derivative, we use a 5 point grid, fourth-order finite difference as an example. We write them into matrix times vector form, which facilitate us to derive its associated adjoint operator.
$$
\begin{bmatrix}
\frac{\partial v_z}{\partial z}[1] \\
\frac{\partial v_z}{\partial z}[2] \\
\frac{\partial v_z}{\partial z}[3] \\
\frac{\partial v_z}{\partial z}[4] \\
\frac{\partial v_z}{\partial z}[5] \\
\end{bmatrix}
= \begin{bmatrix}
a_1 & a_2 & & &\\
-a_1& a_1 & a_2 & & \\
-a_2& -a_1 & a_1 & a_2 & \\
& -a_2 & -a_1 & a_1 & a_2 \\
& & -a_2 & -a_1 & a_1 \\
\end{bmatrix}
\begin{bmatrix}
v_z[1+\frac{1}{2}] \\
v_z[2+\frac{1}{2}] \\
v_z[3+\frac{1}{2}] \\
v_z[4+\frac{1}{2}] \\
v_z[5+\frac{1}{2}] \\
\end{bmatrix}
$$

In above expressions, we assum the value of wavefield outside the computation area is zero. The adjoint operator is given as
$$
\begin{bmatrix}
v_z[1+\frac{1}{2}] \\
v_z[2+\frac{1}{2}] \\
v_z[3+\frac{1}{2}] \\
v_z[4+\frac{1}{2}] \\
v_z[5+\frac{1}{2}] \\
\end{bmatrix}
= - \begin{bmatrix}
-a_1 & a_1 & a_2 & & \\
-a_2 & -a_1 & a_1 & a_2 &   \\
& -a_2 & -a_1 & a_1 & a_2  \\
& & -a_2 & -a_1 & a_1  \\
& & & -a_2 & -a_1  \\
\end{bmatrix}
\begin{bmatrix}
\frac{\partial v_z}{\partial z}[1] \\
\frac{\partial v_z}{\partial z}[2] \\
\frac{\partial v_z}{\partial z}[3] \\
\frac{\partial v_z}{\partial z}[4] \\
\frac{\partial v_z}{\partial z}[5] \\
\end{bmatrix}
$$

### updating ${\bf p}_x$
Similarly, we can get the discretized equations for the last equation

$$
\frac{\partial p_x}{\partial t} + \gamma_x p_x = k \frac{\partial v_x}{\partial x} +  \delta(z-z_0, x-x_0)f(t) \\
$$

$$
\frac{p_x[i,j,k+\frac{3}{2}] - p_x[i,j,k+\frac{1}{2}]}{dt} + \gamma_x[j]
\frac{p_x[i,j,k+\frac{3}{2}] + p_z[i,j,k+\frac{1}{2}]}{2} = k[i,j] \frac{1}{dx} \left( \sum_{n=1}^N  a_n (v_x[i,j+n-\frac{1}{2},k+1] - v_x[i,j-n+\frac{1}{2},k+1]) \right) + \delta(z-z_0, x-x_0) f(t)
$$

Time $2\,dt$ on both side of the equations, we get
$$
2(p_x[i,j,k+\frac{3}{2}] - p_x[i,j,k+\frac{1}{2}]) + dt \, \gamma_x[j]
(p_x[i,j,k+\frac{3}{2}] + p_x[i,j,k+\frac{1}{2}]) = 2 k[i,j] \frac{dt}{dx} \left( \sum_{n=1}^N  a_n (v_x[i,j+n-\frac{1}{2},k+1] - v_x[i,j-n+\frac{1}{2},k+1]) \right) + 2 dt\, \delta(z-z_0, x-x_0)f(t)
$$

Move the terms include $p_x[i,j,k+\frac{3}{2}]$ to the left side of the equation
$$
(2+dt \, \gamma_x[j])p_x[i,j,k+\frac{3}{2}] = (2-dt \, \gamma_x[j]) p_x[i,j,k+\frac{1}{2}] + 2k[i,j] \frac{dt}{dx} \left( \sum_{n=1}^N  a_n (v_x[i,j+n-\frac{1}{2},k+1] - v_x[i,j-n+\frac{1}{2},k+1]) \right) + 2 dt\, \delta(z-z_0, x-x_0)f(t)
$$

divided by the the term $(2+ dt\, \gamma_x[j])$
$$
p_x[i,j,k+\frac{3}{2}] = \frac{2-dt \, \gamma_x[j]}{2+dt \, \gamma_x[j]} p_x[i,j,k+\frac{1}{2}] + \frac{2k[i,j]}{2+dt \, \gamma_x[j]}\frac{dt}{dx} \left( \sum_{n=1}^N  a_n (v_x[i,j+n-\frac{1}{2},k+1] - v_x[i,j-n+\frac{1}{2},k+1]) \right) + dt \,\delta(z-z_0, x-x_0)f(t)
$$

Outside of the PML area, $\gamma_x$ is always equal to zeros, so we have
$$
\frac{2\,dt}{2+dt\, \gamma_x[j]}\,\delta(z-z_0, x-x_0)f(t)  = dt \,\delta(z-z_0, x-x_0)f(t)
$$

#### Details about coding:
we create a vector ${\bf M}_{p_x}^{p_x}$ length of $N_x$, which save the elements of pre-computed $\frac{2- dt\, \gamma_x[j]}{(2+ dt\, \gamma_x[j])}$, another vector called ${\bf M}_{v_x}^{p_x}$ length of $N_z \cdot N_x$, which save the elements of pre-computed $\frac{2k[i,j]}{(2+ dt\, \gamma_x[j])}\frac{dt}{dx}$.

For the spatial derivative, we use a 5 point grid, fourth-order finite difference as an example. We write them into matrix times vector form, which facilitate us to derive its associated adjoint operator.
$$
\begin{bmatrix}
\frac{\partial v_x}{\partial x}[1] \\
\frac{\partial v_x}{\partial x}[2] \\
\frac{\partial v_x}{\partial x}[3] \\
\frac{\partial v_x}{\partial x}[4] \\
\frac{\partial v_x}{\partial x}[5] \\
\end{bmatrix}
= \begin{bmatrix}
a_1 & a_2 & & &\\
-a_1& a_1 & a_2 & & \\
-a_2& -a_1 & a_1 & a_2 & \\
& -a_2 & -a_1 & a_1 & a_2 \\
& & -a_2 & -a_1 & a_1 \\
\end{bmatrix}
\begin{bmatrix}
v_x[1+\frac{1}{2}] \\
v_x[2+\frac{1}{2}] \\
v_x[3+\frac{1}{2}] \\
v_x[4+\frac{1}{2}] \\
v_x[5+\frac{1}{2}] \\
\end{bmatrix}
$$

In above expressions, we assum the value of wavefield outside the computation area is zero. The adjoint operator is given as
$$
\begin{bmatrix}
v_x[1+\frac{1}{2}] \\
v_x[2+\frac{1}{2}] \\
v_x[3+\frac{1}{2}] \\
v_x[4+\frac{1}{2}] \\
v_x[5+\frac{1}{2}] \\
\end{bmatrix}
= - \begin{bmatrix}
-a_1 & a_1 & a_2 & & \\
-a_2 & -a_1 & a_1 & a_2 &   \\
& -a_2 & -a_1 & a_1 & a_2  \\
& & -a_2 & -a_1 & a_1  \\
& & & -a_2 & -a_1  \\
\end{bmatrix}
\begin{bmatrix}
\frac{\partial v_x}{\partial x}[1] \\
\frac{\partial v_x}{\partial x}[2] \\
\frac{\partial v_x}{\partial x}[3] \\
\frac{\partial v_x}{\partial x}[4] \\
\frac{\partial v_x}{\partial x}[5] \\
\end{bmatrix}
$$

We define the the state field vector at two neighbouring time step as
$$
{\bf u}^{k} = \begin{bmatrix}
{\bf v}_z^{k} \\
{\bf v}_x^{k} \\
{\bf p}_z^{k+\frac{1}{2}} \\
{\bf p}_x^{k+\frac{1}{2}} \\
\end{bmatrix}, \,\,\,
{\bf u}^{k+1} = \begin{bmatrix}
{\bf v}_z^{k+1} \\
{\bf v}_x^{k+1} \\
{\bf p}_z^{k+1+\frac{1}{2}} \\
{\bf p}_x^{k+1+\frac{1}{2}} \\
\end{bmatrix},
$$

With these expressions, one step forward modeling is given as
$$
\begin{bmatrix}
{\bf v}_z^{k+1} \\
{\bf v}_x^{k+1} \\
{\bf p}_z^{k+1+\frac{1}{2}} \\
{\bf p}_x^{k+1+\frac{1}{2}} \\
\end{bmatrix}
=\begin{bmatrix}
\bf{I}   & \bf{0}   & \bf{0} & \bf{0}   \\
\bf{0}   & \bf{I}   & \bf{0} & \bf{0}   \\
{\bf M}_{v_z}^{p_z} & \bf{0} & {\bf M}_{p_z}^{p_z} & \bf{0}   \\
\bf{0} & {\bf M}_{v_x}^{p_x} & \bf{0} & {\bf M}_{p_x}^{p_x} \\
\end{bmatrix}
\begin{bmatrix}
{\bf M}_{v_z}^{v_z} & \bf{0} & {\bf M}_{p}^{v_z} & {\bf M}_{p}^{v_z} \\
\bf{0} & {\bf M}_{v_x}^{v_x} & {\bf M}_{p}^{v_x} & {\bf M}_{p}^{v_x} \\
\bf{0}   & \bf{0}   & \bf{I}   & \bf{0}   \\
\bf{0}   & \bf{0}   & \bf{0}  & \bf{I}    \\
\end{bmatrix}
\begin{bmatrix}
{\bf v}_z^{k} \\
{\bf v}_x^{k} \\
{\bf p}_z^{k+\frac{1}{2}} \\
{\bf p}_x^{k+\frac{1}{2}}
\end{bmatrix}
$$

Or simplified as $${\bf u}^{k+1} = {\bf L} {\bf u}^{k}$$, where $${\bf L} = {\bf L_p}{\bf L_v}$$

The multi-step forward modelling can be expressed as, here I just use four time steps as an exmaple which make the derivation of inversion processe much easier.
$$
\begin{bmatrix}
{\bf u}^1 \\
{\bf u}^2 \\
{\bf u}^3 \\
{\bf u}^4 \\
\end{bmatrix}
=
\begin{bmatrix}
{\bf I} & & & \\
 & {\bf I} & & \\
 & &  {\bf I}& \\
 & & {\bf L}& {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
 & {\bf I} & & \\
 & {\bf L} &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
{\bf L} & {\bf I} & & \\
 & &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
& {\bf I} & & \\
 & &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf f}_1 \\
{\bf f}_2 \\
{\bf f}_3 \\
{\bf f}_4 \\
\end{bmatrix}
$$

The synthetic data at one time step ${\bf d}^{k}= {\bf S} {\bf u}^{k}$, where ${\bf S}$ is the sampling operator which depends on the loation of receivers. For FWI, we estimate the model parameters by minimizing the cost function as $$
J = \frac{1}{2}|| {\bf d} - {\bf d}_{obs}||_2^2$$, where ${\bf d}_{obs}$ is the observed data.
The next part is to deriving the gradient of the objective function with repect to one model parameter $m_i$.
$$\frac{\partial J}{\partial m_i} = (\frac{\partial {\bf u}}{\partial m_i})^T \frac{\partial J}{\partial {\bf u}}$$

where $\frac{\partial J}{\partial {\bf u}} = {\bf S}^T ({\bf d} - {\bf d}_{obs})$. Usually, people name the difference between synthetic data and the observed data as residue ${\bf r}$, so we have $$
\frac{\partial J}{\partial {\bf u}} = {\bf S}^T {\bf r}
$$

The we derivae the derivative of $\frac{\partial {\bf u}}{\partial m_i}$
$$
\begin{align}
\frac{\partial {\bf u}}{\partial m_i} &= \begin{bmatrix}
{\bf 0} & & & \\
 & {\bf 0} & & \\
 & &  {\bf 0}& \\
 & & \frac{\partial {\bf L}}{\partial m_i}& {\bf 0} \\
\end{bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
 & {\bf I} & & \\
 & {\bf L} &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
{\bf L} & {\bf I} & & \\
 & &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
& {\bf I} & & \\
 & &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf f}_1 \\
{\bf f}_2 \\
{\bf f}_3 \\
{\bf f}_4 \\
\end{bmatrix}                  \\
&+ \begin{bmatrix}
{\bf I} & & & \\
 & {\bf I} & & \\
 & &  {\bf I}& \\
 & & {\bf L}& {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf 0} & & & \\
 & {\bf 0} & & \\
 & \frac{\partial {\bf L}}{\partial m_i} & {\bf 0}& \\
 & & & {\bf 0} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
{\bf L} & {\bf I} & & \\
 & &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
& {\bf I} & & \\
 & &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf f}_1 \\
{\bf f}_2 \\
{\bf f}_3 \\
{\bf f}_4 \\
\end{bmatrix}                 \\
&+ \begin{bmatrix}
{\bf I} & & & \\
 & {\bf I} & & \\
 & &  {\bf I}& \\
 & & {\bf L}& {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
 & {\bf I} & & \\
 & {\bf L} &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
\frac{\partial {\bf L}}{\partial m_i} & {\bf I} & & \\
 & &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
& {\bf I} & & \\
 & &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf f}_1 \\
{\bf f}_2 \\
{\bf f}_3 \\
{\bf f}_4 \\
\end{bmatrix}
\end{align}
$$

The above expression can be simplified as
$$
\begin{align}
\frac{\partial {\bf u}}{\partial m_i} &= \begin{bmatrix}
{\bf 0} & & & \\
 & {\bf 0} & & \\
 & &  {\bf 0}& \\
 & & \frac{\partial {\bf L}}{\partial m_i}& {\bf 0} \\
\end{bmatrix}
\begin{bmatrix}
{\bf u}_1 \\
{\bf u}_2 \\
{\bf u}_3 \\
{\bf f}_4 \\
\end{bmatrix}                  \\
&+ \begin{bmatrix}
{\bf I} & & & \\
 & {\bf I} & & \\
 & &  {\bf I}& \\
 & & {\bf L}& {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf 0} & & & \\
 & {\bf 0} & & \\
 & \frac{\partial {\bf L}}{\partial m_i} & {\bf 0}& \\
 & & & {\bf 0} \\
\end {bmatrix}
\begin{bmatrix}
{\bf u}_1 \\
{\bf u}_2 \\
{\bf f}_3 \\
{\bf f}_4 \\
\end{bmatrix}                 \\
&+ \begin{bmatrix}
{\bf I} & & & \\
 & {\bf I} & & \\
 & &  {\bf I}& \\
 & & {\bf L}& {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf I} & & & \\
 & {\bf I} & & \\
 & {\bf L} &  {\bf I}& \\
 & & & {\bf I} \\
\end {bmatrix}
\begin{bmatrix}
{\bf 0} & & & \\
\frac{\partial {\bf L}}{\partial m_i} & {\bf 0} & & \\
 & &  {\bf 0}& \\
 & & & {\bf 0} \\
\end {bmatrix}
\begin{bmatrix}
{\bf u}_1 \\
{\bf f}_2 \\
{\bf f}_3 \\
{\bf f}_4 \\
\end{bmatrix}
\end{align}
$$

We record the adjoint wavefield as ${\bf b}^k$, so the gradient can be simplified as
$$
\begin{align}
\frac{\partial J}{\partial m_i} &= (\frac{\partial {\bf u}}{\partial m_i})^T \frac{\partial J}{\partial {\bf u}} \\
&= \begin{bmatrix}
{\bf u}_1^T & {\bf u}_2^T & {\bf u}_3^T & {\bf f}_4^T
\end{bmatrix}
\begin{bmatrix}
{\bf 0} & & & \\
 & {\bf 0} & & \\
 & &  {\bf 0}& (\frac{\partial {\bf L}}{\partial m_i})^T \\
 & &  & {\bf 0} \\
\end{bmatrix}
\begin{bmatrix}
{\bf S}^T {\bf r}_1 \\
{\bf S}^T {\bf r}_2 \\
{\bf S}^T {\bf r}_3 \\
{\bf b}_4 \\
\end{bmatrix}            \\
&+ \begin{bmatrix}
{\bf u}_1^T & {\bf u}_2^T & {\bf f}_3^T & {\bf f}_4^T
\end{bmatrix}
\begin{bmatrix}
{\bf 0} & & & \\
 & {\bf 0} & (\frac{\partial {\bf L}}{\partial m_i})^T & \\
 & &  {\bf 0}&  \\
 & &  & {\bf 0} \\
\end{bmatrix}
\begin{bmatrix}
{\bf S}^T {\bf r}_1 \\
{\bf S}^T {\bf r}_2 \\
 {\bf b}_3 \\
 {\bf b}_4 \\
\end{bmatrix}            \\
&+ \begin{bmatrix}
{\bf u}_1^T & {\bf f}_2^T & {\bf f}_3^T & {\bf f}_4^T
\end{bmatrix}
\begin{bmatrix}
{\bf 0} & (\frac{\partial {\bf L}}{\partial m_i})^T   & & \\
 & {\bf 0} & & \\
 & &  {\bf 0}& \\
 & &  & {\bf 0} \\
\end{bmatrix}
\begin{bmatrix}
{\bf S}^T {\bf r}_1 \\
 {\bf b}_2 \\
 {\bf b}_3 \\
 {\bf b}_4 \\
\end{bmatrix}            \\
\end{align}
$$

So the final form is
$$\frac{\partial J}{\partial m_i} = {\bf u}_3^T (\frac{\partial L}{\partial m_i})^T {\bf b}_4
+ {\bf u}_2^T (\frac{\partial L}{\partial m_i})^T {\bf b}_3
+ {\bf u}_1^T (\frac{\partial L}{\partial m_i})^T {\bf b}_2 $$

```julia
h
```


```python
import sympy as sym
import numpy as np
from sympy.integrals import integrate
```

```python
x = sym.Symbol('x')
integrate(x**2, x)
```

    x**3/3

```python
y = x**2
a = integrate(y, x)
?a
```

    Type:            Mul
    String form:     x**3/3
    File:            c:\programdata\anaconda3\lib\site-packages\sympy\core\mul.py
    Docstring:       <no docstring>
    Class docstring:
    Base class for algebraic expressions.

    Everything that requires arithmetic operations to be defined
    should subclass this class, instead of Basic (which should be
    used only for argument storage and expression manipulation, i.e.
    pattern matching, substitutions, etc).

    See Also
    ========

    sympy.core.basic.Basic

```python
#sympy seems only to output symbolic solutions; not good for something we need to run by itself (without a human to translate)
```

```python
a = lambda x: sym.cosh(x)
b = a(125)
c = b.evalf()
d = b**1/2
e = d.evalf()
```

```python
inf = np.inf
```

```python

```

    inf

```python

```