# Stochastic Differential Equations: Lab 1

```
from IPython.core.display import HTML
css_file = '../ipython_notebook_styles/ngcmstyle.css'
HTML(open(css_file, "r").read())
```

<link href='http://fonts.googleapis.com/css?family=Open+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 1000px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1200px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Open Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:900px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Arvo', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Arvo', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Arvo', serif;
    margin-top:16px;
	font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Arvo', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Arvo', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 90%;
}

</style>

This background for these exercises is article of D Higham, [*An Algorithmic Introduction to Numerical Simulation of Stochastic Differential Equations*, SIAM Review 43:525-546 (2001)](http://epubs.siam.org/doi/abs/10.1137/S0036144500378302).
Higham provides Matlab codes illustrating the basic ideas at <http://personal.strath.ac.uk/d.j.higham/algfiles.html>, which are also given in the paper.

For random processes in `python` you should look at the `numpy.random` module. To set the initial seed (which you should *not* do in a real simulation, but allows for reproducible testing), see `numpy.random.seed`.

## Brownian processes

Simulate a Brownian process over $[0, 1]$ using a step length $\delta t = 1/N$ for $N = 500, 1000, 2000$. Use a fixed seed of `100`. Compare the results.

```

```

Evaluate the function $u(B(t)) = \sin^2(t + B(t))$, where $B(t)$ is a Brownian process, on $M$ Brownian paths for $M = 500, 1000, 2000$. Compare the *average* path for each $M$.

```

```

## Stochastic integrals

Write functions to compute the Itô and Stratonovich integrals of a function $h(t, B(t))$ of a *given* Brownian process $B(t)$ over the interval $[0, 1]$.

```

```

Test the functions on $h = B(t)$ for various $N$. Compare the limiting values of the integrals.

```

```

## Euler-Maruyama's method

Apply the Euler-Maruyama method to the stochastic differential equation

$$
\begin{equation}
  dX(t) = \lambda X(t) + \mu X(t) \ dB(t), \qquad X(0) = X_0.
\end{equation}
$$

Choose any reasonable values of the free parameters $\lambda, \mu, X_0$.

The exact solution to this equation is $X(t) = X(0) \exp \left[ \left( \lambda - \tfrac{1}{2} \mu^2 \right) t + \mu B(t) \right]$. Fix the timetstep and compare your solution to the exact solution.

```

```

Vary the timestep of the Brownian path and check how the numerical solution compares to the exact solution.

## Convergence

Investigate the weak and strong convergence of your method, applied to the problem above.

```

```

## Milstein's method

Implement Milstein's method, applied to the problem above.

```

```

Check the convergence again.

```

```

Compare the *performance* of the Euler-Maruyama and Milstein method using eg `timeit`. At what point is one method better than the other?

```

```

## Population problem

Apply the algorithms, convergence and performance tests to the SDE

$$
\begin{equation}
  dX(t) = r X(t) (K - X(t)) \ dt + \beta X(t) \ dB(t), \qquad X(0) = X_0.
\end{equation}
$$

Use the parameters $r = 2, K = 1, \beta = 0.25, X_0 = 0.5$.

```

```

Investigate how the behaviour varies as you change the parameters $r, K, \beta$.

```

```

FEEG6016 Simulation and Modelling/2014/SDEs Lab 1.ipynb


# 実践データ科学入門 2020年度木曜4限

# 第3回 その1 重線形回帰

```python
%matplotlib inline
#%matplotlib notebook # if necessary to rotate figures in 3D plot
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as patches
from mpl_toolkits.mplot3d import Axes3D
from mpl_toolkits.mplot3d import art3d
from ipywidgets import interact
```

## 重線形回帰とは

重回帰は目的変数に対して説明変数が複数ある場合に用いる手法で，

$$
y = a_0 + a_1 x_1 + a_2 x_2 + \ldots + a_M x_M = a_0 + \sum_{j=1}^M a_j x_j + \xi
$$

というモデルを当てはめる方法である．ただし，

- $y$ は目的変数（データとして与えられるもの）
- $x_j \ (j=1,2,\ldots,M)$ は説明変数（データとして与えられるもの）
- $a_j \ (j=0,1,2,\ldots,M)$ は回帰係数（データから求めるもの）
- $\xi$ はノイズ（モデルでは当てはめられないランダムな要因）

以下，特に注意しない限り__データは実数値__とする．

$M$ 個の説明変数と $M$ 個の回帰係数を

$$
\begin{align}
& \mathbf X = (x_1, x_2, \ldots, x_M) \in \mathbb R^M,\\
& \mathbf A = (a_1, a_2, \ldots, a_M) \in \mathbb R^M
\end{align}
$$

とそれぞれ $M$ 次元ベクトルで表すことで

$$
y = a_0 + \mathbf X \cdot \mathbf A + \xi
$$

と内積の形で表示することができる．さらに定数の部分もまとめて

$$
\begin{align}
& \tilde{\mathbf X} = (1, x_1, x_2, \ldots, x_M) \in \mathbb R^M,\\
& \tilde{\mathbf A} = (a_0, a_1, a_2, \ldots, a_M) \in \mathbb R^M
\end{align}
$$

とおくことで

$$
y = \tilde{\mathbf X} \cdot \tilde{\mathbf A} + \xi
$$

と表すこともできる．

ノイズはしばしば平均 $0$ の正規分布 $\mathrm N(0, \sigma^2)$ に従う確率変数とみなして用いる．

### 単回帰のとき（復習）

回帰変数1つの単回帰では，データ空間 $(x, y)$ の2次元平面に回帰直線を引き，
データから直線までの高さの二乗和が最小になるようにパラメータを選んだ．

```python
# 真のパラメータ
A0 = 1.2
A1 = 2.6

# dataset
X = np.arange(0, 3, 0.3)
Y = A0 + A1*X + np.random.randn(X.size)
```

```python
# 回帰直線のプロット
def plot_XY_and_regressionline(a0=0.0, a1=1.0):
    fig = plt.figure()
    ax = plt.axes()
    ax.set_xlabel("X", size=20)
    ax.set_ylabel("Y", size=20)
    ax.set_xticks
    ax.set_ylim(-1, 9)
    ax.scatter(X, Y)
    ax.plot([np.min(X), np.max(X)], [a0+a1*np.min(X), a0+a1*np.max(X)], linewidth=3, color='tab:red')
    ax.set_title('MSE = %f'%(np.sum((Y-a0-a1*X)**2)/Y.size), size=20)
    ax.tick_params(labelsize=12)
    for x, y in zip(X, Y):

        ypred = a0 + a1*x
        sq = patches.Rectangle(xy=(x, ypred), width=np.abs(y-ypred), height=np.abs(y-ypred), ec='k', fc='lime', alpha=0.8)
        ax.add_patch(sq)
```

```python
interact(plot_XY_and_regressionline, a0=(-5.0, 5.0, 0.1), a1=(-5.0, 10.0, 0.1))

# 青点がデータ点
# 赤が回帰直線
# 緑の長方形が二乗誤差：長方形の総面積が誤差二乗和
# MSE = Mean Squared Error = 平均二乗誤差

# 必ずしも真のパラメータのときに平均二乗誤差最小になるわけではないことに注意しよう．
```

### 重回帰では

回帰変数2つの重回帰では，データ空間 $(x_1, x_2, y)$ の3次元空間に回帰平面を引き，データから平面までの高さの二乗和が最小になるように選ぶ．

```python
# 真のパラメータ
B0 = 1.2
B1 = 1.8
B2 = 0.2

# dataset
N = 1000
X1 = np.random.rand(N) * 3
X2 = np.random.rand(N) * 3
Y = B0 + B1*X1 + B2*X2 + np.random.randn(X1.size)
```

```python
# 回帰平面のプロット
def plot_X1X2Y_and_regressionplane(b0=0.0, b1=0.0, b2=0.0, azimuth=30, elevation=30):
    fig = plt.figure()
    ax = Axes3D(fig)
    ax.view_init(azim=azimuth, elev=elevation)
    ax.set_xlabel("X1")
    ax.set_ylabel("X2")
    ax.set_zlabel("Y")

    ax.set_zlim(-1, 9)
    ax.scatter(X1, X2, Y, s=200)

    X2mesh = X1mesh = np.arange(0, 3, 0.3)

    X1mesh, X2mesh = np.meshgrid(X1mesh, X2mesh)
    Ymesh = b0 + b1*X1mesh + b2*X2mesh
    ax.plot_wireframe(X1mesh, X2mesh, Ymesh, linewidth=3, color='tab:red')
    ax.set_title('MSE = %f'%(np.sum((Y-b0-b1*X1-b2*X2)**2)/Y.size), size=20)
```

```python
interact(plot_X1X2Y_and_regressionplane, b0=(-5.0, 5.0, 0.1), b1=(-5.0, 10.0, 0.1), b2=(-5.0, 10.0, 0.1), azimuth=(0, 360, 1), elevation=(0, 90, 1))

# 青点がデータ点
# 赤が回帰平面
# MSE = Mean Squared Error = 平均二乗誤差

# 必ずしも真のパラメータのときに平均二乗誤差最小になるわけではないことに注意しよう．
```

## 学習方法

上では手動で回帰係数を動かして回帰モデルを作った．最適なパラメータを計算してみよう．

---
### モデル推定値

$N$ 組みのデータセット

$$
\left\{ (x_1^{(i)}, x_2^{(i)}, \ldots, x_M^{(i)}, y^{(i)}) \right\}_{i=1}^N \quad \left( = \left\{ (\mathbf x^{(i)}, y^{(i)}) \right\}_{i=1}^N \right)
$$

があるとしよう．パラメータ $\mathbf A$ で作った重回帰モデルの推定値は

$$
a_0 + \sum_{j=1}^M a_j x_j^{(i)} + \xi^{(i)}
$$

である．ここで $\xi^{(i)}$ は $i$ 番目のデータに対するランダムさ $\xi$ の実現値である．

$\xi$ は平均 $0$ だったので，推定値の期待値は

$$
a_0 + \sum_{j=1}^M a_j x_j^{(i)}
$$

である．

---
### 回帰残差

したがって，パラメータ $\mathbf A$ で作った重回帰モデルによって得られる $i$ 番目のデータに対する推定値の誤差（回帰残差）の期待値は

$$
e^{(i)} := y^{(i)} - a_0 - \sum_{j=1}^M a_j x_j^{(i)}
$$

である．

---
### 平均二乗誤差

誤差平方（二乗誤差）は

$$
(e^{(i)})^2 = \left( y^{(i)} - a_0 - \sum_{j=1}^M a_j x_j^{(i)} \right)^2
$$

なので，平均二乗誤差は，$N$ 個のデータセットに対する誤差平方の平均値であるから，

$$
E(a_0, a_1, \ldots, a_N) := \frac1N \sum_{i=1}^N (e^{(i)})^2 = \frac1N \sum_{i=1}^N \left( y^{(i)} - a_0 - \sum_{j=1}^M a_j x_j^{(i)} \right)^2
$$

となる．

平均二乗誤差は，モデルのパラメータ $\tilde{\mathbf A} = (a_0, a_1, \ldots, a_M)$ が変わるごとに変化するので，$a_0, a_1, \ldots, a_M$ の関数 $E(a_0, a_1, \ldots, a_M)$ として置くことにする．

---
## 最小二乗法

重回帰モデルのパラメータを決定するために，平均二乗誤差を最小にするパラメータを求めることにする．これを最小二乗法と呼ぶ．（平均二乗誤差最小以外のパラメータフィッティングの方法もある．）

つまり，平均二乗誤差の関数 $E(a_0, a_1, \ldots, a_M)$ が最小の値となるパラメータ $a_0, a_1, \ldots, a_M$ を求める．計算方法は単純に多変数関数の極値問題を解けば良い．

極値の候補を求めるには，$a_0, a_1, \ldots, a_M$ それぞれの変数に関する偏導関数を計算して，全ての偏導関数の値が $0$ になる点（勾配が零ベクトルとなる点）を求めれば良い．

---
### 回帰残差の勾配

先に回帰残差について偏導関数を計算しておこう．

$$
\begin{align}
& \frac{\partial e^{(i)}}{\partial a_0} = \frac{\partial}{\partial a_0} \left( y^{(i)} - a_0 - \sum_{j=1}^M a_j x_j^{(i)} \right) = -1,
\\
(k=1, 2, \ldots, M) \quad & \frac{\partial e^{(i)}}{\partial a_k} = \frac{\partial}{\partial a_k} \left( y^{(i)} - a_0 - \sum_{j=1}^M a_j x_j^{(i)} \right) = -\sum_{j=1}^M \frac{\partial}{\partial a_k} a_j x_j^{(i)} = -\sum_{j=1}^M \delta_{jk} x_j^{(i)}
= -x_k^{(i)}.
\end{align}
$$

---
### 平均二乗誤差の勾配

これにより平均二乗誤差の偏導関数は

$$
\begin{align}
& \frac{\partial E}{\partial a_0} = \frac{\partial}{\partial a_0} \frac1N \sum_{i=1}^N (e^{(i)})^2
= \frac2N \sum_{i=1}^N e^{(i)} \frac{\partial e^{(i)}}{\partial a_0} = -\frac2N \sum_{i=1}^N e^{(i)} \\
& \phantom{\frac{\partial E}{\partial a_0}} = -\frac2N \sum_{i=1}^N \left( y^{(i)} - a_0 - \sum_{j=1}^M a_j x_j^{(i)} \right) = -2 \left\{ \frac1N \sum_{i=1}^N y^{(i)} - a_0 - \sum_{j=1}^M a_j \left( \frac1N \sum_{i=1}^N x_j^{(i)} \right) \right\} \\
& \phantom{\frac{\partial E}{\partial a_0}} := -2 \left( \langle y \rangle - a_0 - \sum_{j=1}^M a_j \langle x_j \rangle \right),
\\
(k=1, 2, \ldots, M) \quad & \frac{\partial E}{\partial a_k} = \frac{\partial}{\partial a_k} \frac1N \sum_{i=1}^N (e^{(i)})^2
= \frac2N \sum_{i=1}^N e^{(i)} \frac{\partial e^{(i)}}{\partial a_k} = -\frac2N \sum_{i=1}^N e^{(i)} x_k^{(i)} = \frac2N \sum_{i=1}^N e^{(i)} \frac{\partial e^{(i)}}{\partial a_k} = -\frac2N \sum_{i=1}^N e^{(i)} x_k^{(i)} \\
& \phantom{\frac{\partial E}{\partial a_k}} = -\frac2N \sum_{i=1}^N \left( y^{(i)} - a_0 - \sum_{j=1}^M a_j x_j^{(i)} \right) x_k^{(i)} = -2 \left\{ \frac1N \sum_{i=1}^N y^{(i)} x_k^{(i)} - a_0 \frac1N \sum_{i=1}^N x_k^{(i)} - \sum_{j=1}^M a_j \left( \frac1N \sum_{i=1}^N x_j^{(i)} x_k^{(i)} \right) \right\} \\
& \phantom{\frac{\partial E}{\partial a_0}} := -2 \left( \langle y x_k \rangle - a_0 \langle x_k \rangle - \sum_{j=1}^M a_j \langle x_k x_j \rangle \right),
\end{align}
$$

となる．

---
### 平均二乗誤差の極値

$\frac{\partial E}{\partial a_k}=0$ が全ての $k=0, 1, 2, \ldots, M$ で成り立つとすると

$$
\begin{align}
&
\begin{cases}
\displaystyle \frac{\partial E}{\partial a_0} = 0 \\[5pt]
\displaystyle \frac{\partial E}{\partial a_k} = 0 & (k = 1, 2, \ldots, M)
\end{cases}
\quad \Longleftrightarrow \quad
\begin{cases}
\displaystyle \langle y \rangle - a_0 - \sum_{j=1}^M a_j \langle x_j \rangle = 0 \\
\displaystyle \langle y x_k \rangle - a_0 \langle x_k \rangle - \sum_{j=1}^M a_j \langle x_k x_j \rangle = 0  & (k = 1, 2, \ldots, M)
\end{cases}
\\
&
\Longleftrightarrow \quad
\begin{cases}
\displaystyle a_0 + \sum_{j=1}^M a_j \langle x_j \rangle = \langle y \rangle \\
\displaystyle a_0 \langle x_k \rangle + \sum_{j=1}^M a_j \langle x_k x_j \rangle = \langle y x_k \rangle  & (k = 1, 2, \ldots, M)
\end{cases}
\\
&
\Longleftrightarrow \quad
\underbrace{\begin{pmatrix}
1 & \langle x_1 \rangle & \langle x_2 \rangle & \ldots & \langle x_M \rangle \\
\langle x_1 \rangle & \langle (x_1)^2 \rangle & \langle x_1 x_2 \rangle & \ldots & \langle x_1 x_M \rangle \\
\langle x_2 \rangle & \langle x_1 x_2 \rangle & \langle (x_2)^2 \rangle & \ldots & \langle x_2 x_M \rangle \\
\vdots & \vdots & & \ddots & \vdots \\
\langle x_M \rangle & \langle x_M x_2 \rangle & \langle x_M x_2 \rangle & \ldots & \langle (x_M)^2 \rangle
\end{pmatrix}
}_{=: \mathbb M \\ \scriptsize \mbox{given by data}}
\underbrace{\begin{pmatrix}
a_0 \\ a_1 \\ a_2 \\ \vdots \\ a_M
\end{pmatrix}
}_{= \tilde{\mathbf A}^{\mathrm T}\\ \scriptsize \mbox{parameters} \\ \scriptsize \mbox{to be fitted}}
=
\underbrace{\begin{pmatrix}
\langle y \rangle \\ \langle y x_1 \rangle \\ \langle y x_2 \rangle \\ \vdots \\ \langle y x_M \rangle
\end{pmatrix}
}_{=: \mathbf B \\ \scriptsize\mbox{given by data}}
\\
&
\Longleftrightarrow \quad
\mathbb M \tilde{\mathbf A}^{\mathrm T} = \mathbf B
\end{align}
$$

という連立一次方程式の解となることがわかる．

ただし，

- $\displaystyle \langle y \rangle = \frac1N \sum_{i=1}^M y^{(i)}$ は目的変数 $y$ の標本平均
- $\displaystyle \langle x_k \rangle = \frac1N \sum_{i=1}^M x_k^{(i)}$ $(k=1, 2, \ldots, M)$ は各回帰変数 $x_k$ の標本平均
- $\displaystyle \langle x_k x_j \rangle = \frac1N \sum_{i=1}^M x_j^{(i)} x_k^{(i)}$ $(j, k=1, 2, \ldots, M)$ は回帰変数 $x_k$ と $x_j$ の積の標本平均
- $\displaystyle \langle y x_k \rangle = \frac1N \sum_{i=1}^M y x_k^{(i)}$ $(j, k=1, 2, \ldots, M)$ は目的変数 $y$ と回帰変数 $x_k$ の積の標本平均

この連立一次方程式の係数行列 $\mathbb M$ は非負定値対称行列（固有値は全て非負）である．

---
### グラム行列

$N \times (M+1)$ 行列を以下のように

$$
\mathbb X =
\begin{pmatrix}
\tilde{\mathbf X}^{(1)} \\
\tilde{\mathbf X}^{(2)} \\
\vdots \\
\tilde{\mathbf X}^{(N)}
\end{pmatrix}
=
\begin{pmatrix}
1 & x_1^{(1)} & x_2^{(1)} & \ldots & x_M^{(1)} \\
1 & x_1^{(2)} & x_2^{(2)} & \ldots & x_M^{(2)} \\
\vdots \\
1 & x_1^{(N)} & x_2^{(N)} & \ldots & x_M^{(N)} \\
\end{pmatrix}
$$

置くと，上の連立一次方程式の係数行列は

$$
\mathbb M = \frac1N \mathbb X^{\mathrm T} \mathbb X
$$

と表すことができる．ただし ${}^{\mathrm T}$ は転置を表す．$\mathbb X^{\mathrm T} \mathbb X$ の形の行列を $\mathbb X$ の__グラム行列__という．

$\mathbb M$ は，$\mathbb X$ のグラム行列ではなく，正確には $\frac1{\sqrt N}\mathbb X$ のグラム行列であるが，以下では $\mathbb M$ もグラム行列とよぶことにする．

$N\times(M+1)$-実行列 $\mathbb X$ のグラム行列に対して一般に成り立つ重要な性質としては

- $\mathbb X^{\mathrm T} \mathbb X$ は非負定値対称行列
- $\mathbb X^{\mathrm T} \mathbb X$ の階数は $\mathbb X$ の階数に等しい．つまり $\mathrm{rank}(\mathbb X^{\mathrm T} \mathbb X) = \mathrm{rank}(\mathbb X)$
- $\mathbb X^{\mathrm T} \mathbb X$ が正定値となるための必要十分条件は $\mathbb X$ の階数 (rank) が $M+1$ となること（つまり $\mathbb X$ は full rank）

グラム行列が正定値になるには，データ数が回帰変数の数 $M$ を上回る必要がある．（もちろん上回るだけでは必ずしも正定値になるとは限らない．線型独立なインスタンスが $M+1$ 個必要ということ．）

---
### グラム行列の非負定値性（参考）

$\mathbb X$ を

$$
\mathbb X = (\mathbf x_0, \mathbf x_1, \mathbf x_2, \ldots, \mathbf x_M), \quad
\mathbf x_0 = \begin{pmatrix} 1 \\ 1 \\ \vdots \\ 1 \end{pmatrix}, \
\mathbf x_j = \begin{pmatrix} x_j^{(1)} \\ x_j^{(2)} \\ \vdots \\ x_j^{(N)} \end{pmatrix} \ (j=1, 2, \ldots, M)
$$

と $N$ 次の列ベクトル $M+1$ 個に分割しておくと，グラム行列 $\mathbb M$ の $(i, j)$ 成分 $M_{ij}$ $(i, j = 1, 2, \ldots, M+1)$ は

$$
M_{ij} = \frac1N \mathbf x_{i+1}^{\mathrm T} \mathbf x_{j+1} \quad \left( = \langle x_{i+1} x_{j+1} \rangle \right)
$$

というように，$\mathbb X$ の列ベクトルの内積で表される．

データを実数値に限っておくと，$\mathbb X^{\mathrm T} \mathbb X$ は実対称行列なので適当な直交行列 $\mathbb U$ を用いて

$$
\mathbb U^{\mathrm T} \mathbb M \mathbb U = \mathbb U^{\mathrm T} (\mathbb X^{\mathrm T} \mathbb X) \mathbb U
= \begin{pmatrix}
\lambda_0 &  & & O \\ & \lambda_1 \\ & & \ddots \\ O & & & \lambda_M
\end{pmatrix}
$$

と対角行列に変形できる．

$\mathbb X \mathbb U =: \mathbb V$ と置き，$\mathbb V$ を

$$
\mathbb V = (\mathbf v_0, \mathbf v_1, \ldots, \mathbf v_M)
$$

のように列ベクトル分割をしておくと，上の左辺は

$$
\mathbb U^{\mathrm T} (\mathbb X^{\mathrm T} \mathbb X) \mathbb U
= (\mathbb X \mathbb U)^{\mathrm T} (\mathbb X \mathbb U) = \mathbb V^{\mathrm T} \mathbb V
= \begin{pmatrix}
\mathbf v_0^{\mathrm T} \mathbf v_0 & \mathbf v_0^{\mathrm T} \mathbf v_1 & \ldots & \mathbf v_0^{\mathrm T} \mathbf v_M \\
\mathbf v_1^{\mathrm T} \mathbf v_0 & \mathbf v_1^{\mathrm T} \mathbf v_1 & \ldots & \mathbf v_1^{\mathrm T} \mathbf v_M \\
\vdots & \vdots & \ddots & \vdots
\\
\mathbf v_M^{\mathrm T} \mathbf v_0 & \mathbf v_M^{\mathrm T} \mathbf v_1 & \ldots & \mathbf v_M^{\mathrm T} \mathbf v_M \end{pmatrix}
$$

となる．両辺比較すると

$$
\mathbf v_i^{\mathrm T} \mathbf v_j = \delta_{ij} \lambda_i,
$$

特に

$$
\lambda_i = \| \mathbf v_i \|^2 \ge 0
$$

とわかる．ただし $\| \cdot \|$ はユークリッドノルム（2-ノルム）である．

したがって係数行列 $\mathbb X^{\mathrm T} \mathbb X$ の固有値は全て非負とわかった．

---
### グラム行列が正定値になるとき

$\mathbb X^{\mathrm T}\mathbb X$ は $(M+1)$ 次正方行列であり，

$$
\mathrm{rank}{\mathbb X^{\mathrm T}\mathbb X} = \mathrm{rank}(\mathbb X)
$$

であることから，$\mathrm{rank}(\mathbb X) = M+1$ であることが，$\mathbb X^{\mathrm T}\mathbb X$ の正定値性の必要十分条件であることがわかる．

---
### $\mathbb X$ が full rank のときの最適パラメータ

$\mathbb X$ が full rank であるとしよう．このとき，連立一次方程式の係数行列 $\mathbb M$ は正則なので，連立一次方程式 $\mathbb M \tilde{\mathbf A}^{\mathrm T} = \mathbf B$ の解はただ一つである．つまり，平均二乗誤差を与える関数 $E(a_0, a_1, \ldots, a_N)$ の極値の候補は一つだけということがわかる．実際にこの点で極小点を与え，さらに最小であることもわかる．そのパラメータ点は

$$
\mathbb M \tilde{\mathbf A}^{\mathrm T} = \mathbf B \ \Longleftrightarrow \
\tilde{\mathbf A}^{\mathrm T} = \mathbb M^{-1} \mathbf B
$$

と表される．この $\tilde{\mathbf A}^{\mathrm T} = \mathbb M^{-1} \mathbf B$ で定められるパラメータ $\tilde{\mathbf A} = (a_0, a_1, a_2, \ldots, a_M)$ が，重回帰モデルにおいて平均二乗誤差最小を与えるパラメータである．

ここで

$$
\tilde{\mathbf A}^{\mathrm T}_* = \mathbb M^{-1} \mathbf B
$$

と区別しておこう．

---
### $\tilde{\mathbf A}^{\mathrm T}_* = \mathbb M^{-1} \mathbf B$ において極小を取ること（参考）

$\tilde{\mathbf A}^{\mathrm T}_* = \mathbb M^{-1} \mathbf B$ が極小値となることを示すには，$E(\tilde{\mathbf A})$ のヘッセ行列が $\tilde{\mathbf A}^{\mathrm T}_* = \mathbb M^{-1} \mathbf B$ において正定値であることを示せばよい．

ヘッセ行列 $\mathbb H = (H_{kl})$ の $(k,l)$ 成分は $\frac{\partial^2 E}{\partial a_k \partial a_l}$ で与えられる．すると

$$
\begin{align}
& H_{11} = \frac{\partial^2 E}{\partial (a_0)^2} = -2 \frac{\partial}{\partial a_0} \left( \langle y \rangle - a_0 - \sum_{j=1}^M a_j \langle x_j \rangle \right) = 2 = 2 M_{11},
\\
(k = 1, 2, \ldots, M) \quad & H_{1,k+1} = \frac{\partial^2 E}{\partial a_0 \partial a_k} = -2 \frac{\partial}{\partial a_k} \left( \langle y \rangle - a_0 - \sum_{j=1}^M a_j \langle x_j \rangle \right) = 2 \langle x_k \rangle = 2M_{1, k+1},
\\
(k = 1, 2, \ldots, M) \quad & H_{k+1, 1} = H_{1, k+1} = 2 \langle x_k \rangle = 2M_{k+1, 1},
\\
(k, l = 1, 2, \ldots, M) \quad & \frac{\partial^2 E}{\partial a_k \partial a_l} = -2 \frac{\partial}{\partial a_l} \left( \langle y x_k \rangle - a_0 \langle x_k \rangle - \sum_{j=1}^M a_j \langle x_k x_j \rangle \right) = 2 \langle x_k x_l \rangle = 2 M_{kl}
\end{align}
$$

となることから，

$$
\mathbb H = 2 \mathbb M
$$

とわかる．$\mathbb H$ は $a_0, a_1, \ldots, a_M$ には依存しない定行列である．

$\mathbb M$ が正定値であれば $\mathbb H$ も正定値となるので，$\tilde{\mathbf A}^{\mathrm T}_* = \mathbb M^{-1} \mathbf B$ は $E(a_0, a_1, \ldots, a_M)$ の極小を与える．

---
### 最小値となること

$\mathbb X$ が full rank のとき，$E(a_0, a_1, \ldots, a_M)$ のヘッセ行列 $\mathbb H$ は全ての点で正定値なので，$E(a_0, a_1, \ldots, a_M)$ は狭義凸関数である．

狭義凸関数の極小点は最小点となるため，上で得られた $\tilde{\mathbf A}_*$ は $E(a_0, a_1, \ldots, a_M)$ の最小を与えるパラメータであることがわかる．

---
### 凸関数の補足（参考）

ここで $n$ 変数関数 $f$ が（広義）凸関数とは，

$$
f((1-c)\mathbf x+c\mathbf y) \le (1-c)f(\mathbf x) + c f(\mathbf y)
$$

が全ての $\mathbf x, \mathbf y \in \mathbb R^n$ と $0<c<1$ なる全ての $c$ に対して成り立つときをいう．

この不等号において等号が成り立つのが $\mathbf x = \mathbf y$ のときだけであるとき，特に狭義凸関数とよぶ．

$f$ が狭義凸となる十分条件として，$f$ のヘッセ行列が全ての点で正定値となることが知られている．したがって，$E(a_0, a_1, \ldots, a_M)$ は狭義凸関数である．

凸関数のもう一つの重要な性質は

$$
f(\mathbf a) + \nabla f(\mathbf a) (\mathbf x - \mathbf a) \le f(\mathbf x)
$$

が全ての $\mathbf a, \mathbf x \in \mathbb R^N$ において成り立つということ．

これを $E(a_0, a_1, \ldots, a_M)$ に適用すると，

$$
E(\tilde{\mathbf A}_*) + \nabla E(\tilde{\mathbf A}_*) (\tilde{\mathbf A} - \tilde{\mathbf A}_*) \le E(\tilde{\mathbf A})
$$

が全ての $\tilde{\mathbf A} \in \mathbb R^{M+1}$ に対して成り立つ．
$\tilde{\mathbf A}_*$ は $E$ の極値なので $\nabla E(\tilde{\mathbf A}_*)= \mathbf 0$ が成り立つ．したがって

$$
E(\tilde{\mathbf A}_*) \le E(\tilde{\mathbf A})
$$

となるため，$E(\tilde{\mathbf A}_*)$ が最小値であることがわかる．

---
## では計算してみよう

```python
# 真のパラメータ
B0 = 1.2
B1 = 1.8
B2 = 0.2

# dataset
N = 10
X1 = np.random.rand(N) * 3
X2 = np.random.rand(N) * 3
Y = B0 + B1*X1 + B2*X2 + np.random.randn(X1.size)

# この場合では N = 10, M = 2 である
M = 2
```

$$
\mathbb X =
\begin{pmatrix}
\tilde{\mathbf X}^{(1)} \\
\tilde{\mathbf X}^{(2)} \\
\vdots \\
\tilde{\mathbf X}^{(N)}
\end{pmatrix}
=
\begin{pmatrix}
1 & x_1^{(1)} & x_2^{(1)} \\
1 & x_1^{(2)} & x_2^{(2)} \\
\vdots \\
1 & x_1^{(N)} & x_2^{(N)} \\
\end{pmatrix}
$$

```python
# データ行列
XX = np.ones((N, M+1))

# XX の第0列は全て1，第1列は X1, 第2列は X2
XX[:, 1] = X1
XX[:, 2] = X2

# M = (X^T)X / N
MM = np.dot(XX.T, XX) / N # 行列の積には np.dot を用いる．XX.T で XX の転置が取れる

BB = np.zeros(M+1)
BB[0] = np.mean(Y)
BB[1] = np.mean(Y*X1)
BB[2] = np.mean(Y*X2)

# 連立一次方程式 MA = B を解く
AA = np.linalg.solve(MM, BB)

print('最小二乗法で得られた重回帰モデル： %9.6f + %9.6f * x1 + %9.6f * x2'%(AA[0], AA[1], AA[2]))
print('真の重回帰モデル：                         %9.6f + %9.6f * x1 + %9.6f * x2'%(B0, B1, B2))
```

学習で得られたパラメータは必ずしも真のパラメータに近いとは言えない．（ただし，与えられたデータセットに対して平均二乗誤差が最小であることは間違いない．）

---
scikit-learn の結果と一致することを確かめてみよう

```python
from sklearn import linear_model
reg = linear_model.LinearRegression()
X_train = XX[:, 1:3]
Y_train = Y
reg.fit(X_train, Y_train)

print('scikit-learn の LinearRegression で得られた重回帰モデル： %9.6f + %9.6f * x1 + %9.6f * x2'
      %(reg.intercept_, reg.coef_[0], reg.coef_[1]))
```

データ数 $N$ を変えてフィッテイングの精度を調べてみよう．

```python
# 真のパラメータ
B0 = 1.2
B1 = 1.8
B2 = 0.2

# 回帰変数の数
M = 2

# データ数の常用対数の最大
Lmax = 7
B0_pred = np.zeros(Lmax)
B1_pred = np.zeros(Lmax)
B2_pred = np.zeros(Lmax)

reg = linear_model.LinearRegression()
for L in range(1, Lmax):
    # データ数を変える
    N = 10**L

    # dataset
    X1 = np.random.rand(N) * 3
    X2 = np.random.rand(N) * 3
    Y = B0 + B1*X1 + B2*X2 + np.random.randn(X1.size)

    X_train = np.zeros((N, M))
    X_train[:, 0] = X1
    X_train[:, 1] = X2

    # sklearn の LinearRegression で fitting
    reg.fit(X_train, Y)
    B0_pred[L] = reg.intercept_
    B1_pred[L] = reg.coef_[0]
    B2_pred[L] = reg.coef_[1]
    print('scikit-learn の LinearRegression で得られた重回帰モデル： %9.6f + %9.6f * x1 + %9.6f * x2'
          %(reg.intercept_, reg.coef_[0], reg.coef_[1]))

fig = plt.figure()
ax = plt.axes()
ax.set_xlabel("L", size=20)
ax.plot( np.arange(1, Lmax+1, 1), np.log10(np.abs(B0-B0_pred)), 'o-', label='$\log|B0 - B0_{pred}|$')
ax.plot( np.arange(1, Lmax+1, 1), np.log10(np.abs(B1-B1_pred)), 'o-', label='$\log|B1 - B1_{pred}|$')
ax.plot( np.arange(1, Lmax+1, 1), np.log10(np.abs(B2-B2_pred)), 'o-', label='$\log|B2 - B2_{pred}|$')
ax.tick_params(labelsize=12)
ax.legend()
```

# 演習問題 3-1

第2回その3で Iris データセットを用いて線形回帰モデルを適用した．そこでは目的変数として `Petal Length` を取り，単回帰では説明変数に `Sepal Length` を，重回帰では説明変数に `Sepal Length` と `Petal Width` を用いてモデルを立てた．さらには `Species` でデータを分けて種別に回帰することでよりフィッティングが合うようにした．

では，目的変数と説明変数を全ての組み合わせを取り替えて線形回帰モデルを試し，回帰変数の数とフィッティング精度に傾向があるか確かめよ．（植物学的に意味があるかはさておき）一番フィッティングの合う線形回帰モデルは何か．その理由も考えよ．

その2につづく

```python

```

```python

```


# Stability of a Multistep method

#### John S Butler
john.s.butler@tudublin.ie

[Course Notes](https://johnsbutler.netlify.com/files/Teaching/Numerical_Analysis_for_Differential_Equations.pdf)    [Github](https://github.com/john-s-butler-dit/Numerical-Analysis-Python)

## Overview
A one-step or multistep method is used to approximate the solution of an initial value problem of the form
\begin{equation} \frac{dy}{dt}=f(t,y),\end{equation}
with the initial condition
\begin{equation} y(a)=\alpha.\end{equation}
The method should only be used if it satisfies the three criteria:
1. that difference equation is consistent with the differential equation;
2. that the numerical solution converges to the exact answer of the differential equation;
3. that the numerical solution is stable.

In the notebooks in this folder we will illustate examples of consistent and inconsistent, convergent and non-convergent, and stable and unstable methods.

This notebook focuses on stable and unstable methods. The video below outlines the notebook.

```python
from IPython.display import HTML
HTML('')
```

## Introduction to unstable
This notebook illustrates an unstable multistep method for numerically approximating an initial value problem
\begin{equation} \frac{dy}{dt}=f(t,y), \end{equation}
with the initial condition
\begin{equation} y(a)=\alpha,\end{equation}
using the Modified Abysmal Kramer-Butler method. The method is named after the great [Cosmo Kramer]( https://en.wikipedia.org/wiki/Cosmo_Kramer) and myself [John Butler](https://johnsbutler.netlify.com).

## 2-step Abysmal Kramer-Butler Method

The 2-step Abysmal Kramer-Butler difference equation is given by
\begin{equation}w_{i+1} = w_{i-1} + h(4f(t_i,w_i)-2f(t_{i-1},w_{i-1})) \end{equation}
by changing $F$, the Modified Abysmal Butler Method (see convergent and consistent notebooks), the Abysmal Kramer-Butler method is consistent with the differential equation and convergent with the exact solution (see below for proof).
But the most important thing is that method is weakly stable, it fluctuates widely around the exact answer, just like it's name sake Kramer (for examples see any Seinfeld episode).

## Definition of Stability
The stability of a numerical method is not as tangable as consistency and convergence but when you see an unstable solution it is obvious.

To determine the stabilty of a multistep method we need three definitions:

### Definition: Characteristic Equation
Associated with the difference equation
\begin{equation} w_0=\alpha \ \ \ w_1=\alpha_1 \ \ \ ... \ \ \ w_{m-1}=\alpha_{m-1} \end{equation}
\begin{equation}w_{i+1} = a_{m-1}w_{i}+a_{m-2}w_{i-1}+...+a_{0}w_{i+1-m} +hF(t_i,h,w_{i+1},...,w_{i+1-m}),\end{equation}
is the __characteristic equation__ given by
\begin{equation}\lambda^{m} - a_{m-1}\lambda^{m-1}-a_{m-2}\lambda^{m-2}-...-a_{0} =0. \end{equation}

### Definition: Root Condition

Let $\lambda_1,...,\lambda_m$ denote the roots of the that characteristic equation
\begin{equation}\lambda^{m} - a_{m-1}\lambda^{m-1}-a_{m-2}\lambda^{m-2}-...-a_{0} =0 \end{equation}
associated with the multi-step difference method
\begin{equation} w_0=\alpha \ \ \ w_1=\alpha_1 \ \ \ ... \ \ \ w_{m-1}=\alpha_{m-1} \end{equation}
\begin{equation} w_{i+1} = a_{m-1}w_{i}+a_{m-2}w_{i-1}+...+a_{0}w_{i+1-m} +hF(t_i,h,w_{i+1},...,w_{i+1-m}),\end{equation}
If $|\lambda_{i}|\leq 1$ for each $i=1,...,m$ and all roots with absolute value 1
are simple roots then the difference equation is said to satisfy the __root condition__.

### Definition: Stability
1. Methods that satisfy the root condition and have $\lambda=1$ as the only root
of the characteristic equation of magnitude one and all other roots are 0 are called __strongly stable__;
2. Methods that satisfy the root condition and have more than one distinct root
with magnitude one are called __weakly stable__;
3. Methods that do not satisfy the root condition are called __unstable__.

All one step methods, Adams-Bashforth and Adams-Moulton methods are all stongly stable.

## Intial Value Problem
To illustrate stability we will apply the method to a linear intial value problem given by
\begin{equation}y^{'}=t-y, \ \ (0 \leq t \leq 2), \end{equation}
with the initial condition
\begin{equation}y(0)=1,\end{equation}
with the exact solution
\begin{equation}y(t)= 2e^{-t}+t-1.\end{equation}

## Python Libraries

```python
import numpy as np

%matplotlib inline
import matplotlib.pyplot as plt # side-stepping mpl backend
import matplotlib.gridspec as gridspec # subplots
import warnings
import pandas as pd

warnings.filterwarnings("ignore")
```

### Defining the function
$$ f(t,y)=t-y.\end{equation}

```python
def myfun_ty(t,y):
    return t-y
```

## Discrete Interval
Defining the step size  $h$  from the interval range  $a \leq t \leq b$  and number of steps  $N$
\begin{equation}h=\frac{b-a}{N}.\end{equation}

This gives the discrete time steps,
\begin{equation}t_i=t_0+ih,\end{equation}
where  $t_0=a.$

```python
# Start and end of interval
b=2
a=0
# Step size
N=16
h=(b-a)/(N)
t=np.arange(a,b+h,h)
fig = plt.figure(figsize=(10,4))
plt.plot(t,0*t,'o:',color='red',label='Coarse Mesh')
plt.xlim((0,2))
plt.ylim((-0.1,.1))

plt.legend()
plt.title('Illustration of discrete time points')
plt.show()
```

## 2-step  Abysmal Kramer-Butler Method

For this initial value problem 2-step Abysmal Kramer-Butler difference equation is
\begin{equation}w_{i+1} = w_{i-1} + h(4(t_i- w_i)-2(t_{i-1}-w_{i-1})) \end{equation}
by changing $F$, the Modified Abysmal Butler Method, is consistent and convergent.

For $i=0$ the system of difference equation is:
\begin{equation}w_{1} = w_{-1} + h(4(t_0-w_0)-2(t_{-1}-w_{-1})) \end{equation}
this is not solvable as  $w_{-1}$  is unknown.

For $i=1$ the difference equation is:
\begin{equation}w_{2} = w_{0} + h(4(t_1-w_1)-2(t_{0}-w_{0})) \end{equation}
this is not solvable as  $w_{1}$   is unknown. $w_1$ can be  approximated using a one step method. Here, as the exact solution is known,
\begin{equation}w_1=2e^{-t_1}+t_1-1.\end{equation}

```python
### Initial conditions
IC=1
w=np.zeros(len(t))
y=(2)*np.exp(-t)+t-1
w[0]=IC
w[1]=y[1]
```

### Loop

```python
for i in range (1,N):
    w[i+1]=(w[i-1]+h*(4*myfun_ty(t[i],w[i])-2*myfun_ty(t[i-1],w[i-1])))

```

### Plotting solution

```python
def plotting(t,w,y):

    fig = plt.figure(figsize=(10,4))
    plt.plot(t,w,'^:',color='red',label='Abysmal Kramer-Butler (N)')
    plt.plot(t,y, 'o-',color='black',label='Exact')
    plt.xlabel('time')
    plt.legend()
    plt.title(' Abysmal Kramer-Butler')
    plt.show
```

The plot below shows the Abysmal Kramer-Butler approximation $w_i$ (red) and the exact solution $y(t_i)$ (black) of the intial value problem.

The numerically solution initially approximates the exact solution $(t<.5)$ reasonably but then the instability creeps in such that the numerical approximation starts to widely oscilate around the exact solution.

```python
plotting(t,w,y)
```

The table below illustrates the absolute error and the signed error of the numerical method.

```python
n=10
d = {'time': t[0:n], 'Abysmal Kramer-Butler w': w[0:n],'Exact Error abs':np.abs(y[0:n]-w[0:n]),
     'Exact Error':(y[0:n]-w[0:n])}
df = pd.DataFrame(data=d)
df
```

      time
      Abysmal Kramer-Butler w
      Exact Error abs
      Exact Error

      0
      0.000
      1.000000
      0.000000
      0.000000

      1
      0.125
      0.889994
      0.000000
      0.000000

      2
      0.250
      0.867503
      0.059902
      -0.059902

      3
      0.375
      0.772491
      0.022912
      -0.022912

      4
      0.500
      0.823134
      0.110072
      -0.110072

      5
      0.625
      0.710297
      0.014774
      -0.014774

      6
      0.750
      0.861269
      0.166535
      -0.166535

      7
      0.875
      0.675986
      0.032738
      0.032738

      8
      1.000
      0.988592
      0.252834
      -0.252834

      9
      1.125
      0.631937
      0.142368
      0.142368

# Theory
## Consistent
The Abysmal Kramer-Butler method does satisfy the consistency condition
\begin{equation}\tau_{i}(h)=\frac{y_{i+1}-y_{i-1}}{2h}-\frac{1}{2}[4(f(t_i,y_i)-2f(t_{i-1},y_{i-1})] \end{equation}
As $h \rightarrow 0$
\begin{equation}\frac{1}{2}[4(f(t_i,y_i)-2f(t_{i-1},y_{i-1})] \rightarrow f(t_i,y_i).\end{equation}
While as $h \rightarrow 0$
\begin{equation}\frac{y_{i+1}-y_{i-1}}{2h} \rightarrow \frac{y^{'}}{1}=\frac{f(t_i,y_i)}{1}.\end{equation}
Hence as $h \rightarrow 0$ $$\frac{y_{i+1}-y_{i}}{h}-\frac{1}{2}[4(f(t_i,y_i)-2f(t_{i-1},y_{i-1})]\rightarrow f(t_i,y_i)-f(t_i,y_i)=0,\end{equation}
which means it is consistent.

## Convergent
The Abysmal Kramer-Butler method does satisfy the Lipschitz condition:
\begin{equation}F(t,w:h)-F(t,\hat{w}:h)=\frac{4}{2}f(t,w_i)-\frac{2}{2}f(t-h,w_{i-1}))-(\frac{4}{2}f(t,\hat{w}_{i})-\frac{2}{2}f(t-h,\hat{w}_{i-1}))),\end{equation}
\begin{equation}F(t,w:h)-F(t,\hat{w}:h)=\frac{4}{2}(f(t,w_i)-f(t,\hat{w}_i))-\frac{2}{2}(f(t-h,w_{i-1}))-f(t-h,\hat{w}_{i-1}))),\end{equation}
\begin{equation}F(t,w:h)-F(t,\hat{w}:h)\leq\frac{4}{2}L|w_i-\hat{w_i}|+\frac{2}{2}L|w-\hat{w}|\leq \frac{6}{2} L|w_i-\hat{w_i}|,\end{equation}
This means it is internally convergent,
\begin{equation}|w_i-\hat{w_i}|\rightarrow 0\end{equation}
as $h \rightarrow 0$.
## Stability
The Abysmal Kramer-Butler method does __not__ satisfy the stability condition.
The characteristic equation of the
\begin{equation}w_{i+1} = w_{i-1} + h(4f(t_i,w_i)-2f(t_{i-1},w_{i-1})) \end{equation}
is
\begin{equation}\lambda^2 = 1, \end{equation}
This has two roots $\lambda=1$ and   $\lambda=-1$, hence the method is weakly stable.

```python

```

Chapter 05 - IVP Consistent Convergence Stability/503_Stability.ipynb


# Parametric Model Document Formulas

This notebook attempts to generate the equations as shown in the parametric model document. Useful for checking whether the parametric model as implemented actually matches the document.

```python
import sys

from IPython.display import display, Math, Latex, HTML, Markdown
from sympy import latex, simplify, Lambda, Symbol, Function

sys.path+=['..']
from sdp_par_model import evaluate
from sdp_par_model import reports as iapi
from sdp_par_model.config import PipelineConfig
from sdp_par_model.parameters.definitions import *
from sdp_par_model.parameters.container import BLDep
```

```python
# Telescope and band should not matter
cfg = PipelineConfig(telescope=Telescopes.SKA1_Mid, band=Bands.Mid1, pipeline=Pipelines.ICAL)
cfg_rcal = PipelineConfig(telescope=Telescopes.SKA1_Mid, band=Bands.Mid1, pipeline=Pipelines.RCAL)
cfg_ingest = PipelineConfig(telescope=Telescopes.SKA1_Mid, band=Bands.Mid1, pipeline=Pipelines.Ingest)
cfg_specfit = PipelineConfig(telescope=Telescopes.SKA1_Mid, band=Bands.Mid1, pipeline=Pipelines.DPrepA_Image)
tp = cfg.calc_tel_params(symbolify='product')
tp2 = cfg.calc_tel_params(symbolify='helper')
tp3 = cfg.calc_tel_params(symbolify='all')
tp_rcal = cfg_rcal.calc_tel_params(symbolify='product')
tp_ingest = cfg_ingest.calc_tel_params(symbolify='product')
tp_ingest2 = cfg_ingest.calc_tel_params(symbolify='helper')
tp_specfit = cfg_specfit.calc_tel_params(symbolify='product')
def show_sympy(*exprs):
    l = "=".join([latex(e) for e in exprs])
    display(Math("\\begin{aligned}" + l + "\\end{aligned}"))
```

```python
b = Symbol("b")
tp_ingest.Rviscorr = tp_ingest.Nbeam * tp_ingest.Npp * tp_ingest.Rvis_ingest
tp_ingest2.Rviscorr = tp_ingest2.Nbeam * tp_ingest2.Npp * tp_ingest2.Rvis_ingest
tp.RvisfullFoV = tp_ingest2.Nbeam * tp_ingest2.Npp * tp.Rvis_predict
# Definition of Rvis consistent with most uses in the document, and a helper to introduce it into formulas
RvisfullFoV = tp.Nbeam * tp.Nbl * tp.Npp * tp.Nf_vis / tp.Tint_used
def rvisfullfov(expr):
    return expr / RvisfullFoV(b) * Symbol("R_vis,fullFoV")
```

## Imaging

```python
# Ingest rate over all polarisations and beams.
show_sympy(Symbol("R_vis,corr"), tp_ingest.Rviscorr, tp_ingest2.Rviscorr)
```

```python
# Pixel production rate (after reprojection), assumed to be summed over the entire observation
show_sympy(Symbol("R_pix"), tp.products[Products.Reprojection]['Rout'] * tp.Tsnap / tp.Tobs)
```

## Receive

```python
show_sympy(Symbol("C_Receive"), tp_ingest.products[Products.Receive]['Rflop'])
```

## Flagging

```python
show_sympy(Symbol("N_flop/vis"), tp_ingest.products[Products.Flag]['Rflop'] / tp_ingest.Rviscorr)
show_sympy(Symbol("C_Flag"), tp_ingest.products[Products.Flag]['Rflop'])
```

## Demix

```python
show_sympy(Symbol("N_flop/vis"), tp_ingest.products[Products.Demix]['Rflop'] / tp_ingest.Rviscorr)
show_sympy(Symbol("C_Demix"), tp_ingest.products[Products.Demix]['Rflop'])
```

## Averaging

```python
show_sympy(Symbol("C_Average"), tp_ingest.products[Products.Average]['Rflop'])
```

## Predict via Direct Fourier Transform

```python
show_sympy(Symbol("N_flop/vis^predict,DFT"), tp.products[Products.DFT]['Rflop'] / RvisfullFoV(b) / tp.Nmajortotal)
show_sympy(Symbol("C_Predict"), rvisfullfov(tp.products[Products.DFT]['Rflop']))
```

## Solve

```python
show_sympy(Symbol("N_{flop/solution}^{StEFCal}"), tp.NFlop_solver)
show_sympy(Symbol("C_{Solve}^{RCAL}"), tp_rcal.products[Products.Solve]['Rflop'])
show_sympy(Symbol("C_{Solve}^{ICAL}"), tp.products[Products.Solve]['Rflop'])
```

## Subtract

```python
show_sympy(Symbol("C_Subtract"), tp.products[Products.Subtract_Visibility]['Rflop'])
```

## Correct

```python
show_sympy(Symbol("C_Correct"), tp.products[Products.Correct]['Rflop'])
```

## Phase Rotation

```python
show_sympy(Symbol("C_phrot,back"), simplify(tp.products[Products.PhaseRotation]['Rflop']), RvisfullFoV(b))
show_sympy(Symbol("C_phrot,predict"), rvisfullfov(tp.products[Products.PhaseRotationPredict]['Rflop']))
```

## Grid and Degrid

```python
show_sympy(Symbol("C_{grid}^{ICAL}"), tp.products[Products.Grid]['Rflop'])
show_sympy(Symbol("C_{degrid}^{ICAL,A}"), tp.products[Products.Degrid]['Rflop'])
```

## Gridding Kernel Update

```python
show_sympy(tp.Nkernel_AW_predict(b), tp2.Nkernel_AW_predict.term)
show_sympy(Function("N_cvff")(b), (tp.Ngcf_used_backward * tp.Nkernel_AW_predict).term)
show_sympy(Function("C_{Kernels,back}"), tp.products[Products.Gridding_Kernel_Update]['Rflop'])
show_sympy(Function("C_{Kernels,predict}"), tp.products[Products.Degridding_Kernel_Update]['Rflop'])
```

## FFT and iFFT

```python
show_sympy(tp.Nf_FFT_backward, tp3.Nf_FFT_backward)
show_sympy(Symbol("C_{FFT}^{ICAL}"), tp.products[Products.FFT]['Rflop'])
show_sympy(tp.Nf_FFT_predict, tp3.Nf_FFT_predict)
show_sympy(Symbol("C_{IFFT}^{ICAL,A}"), tp.products[Products.IFFT]['Rflop'])
```

## Reprojection

```python
show_sympy(Symbol("C_{Reproj}^{ICAL}"), tp.products[Products.Reprojection]['Rflop'])
```

## Image Spectral Fitting

```python
show_sympy(Symbol("C_{SpecFit}^{ICAL}"), tp_specfit.products[Products.Image_Spectral_Fitting]['Rflop'])
```

## Subtract Image Component

```python
show_sympy(Symbol("C_{grid}^{ICAL}"), tp.products[Products.Identify_Component]['Rflop'])
show_sympy(Symbol("C_{degrid}^{ICAL,A}"), tp.products[Products.Subtract_Image_Component]['Rflop'])
```

## Source find

```python
show_sympy(Symbol("C_{Source Find}"), tp.products[Products.Source_Find]['Rflop'])
```

# Time Smearing Limit, tsmear

```python
show_sympy(Function("t_{smear}")(b), tp2.Tint_used * tp2.combine_time_samples_facet.term)
```

# Number of Channels at the Frequency Smearing Limit, Nf,smear

```python
show_sympy(Symbol("Theta_PSF"), tp3.Theta_beam)
show_sympy(Symbol("Q_bw"), tp3.Qbw)
show_sympy(Function("N_f,smear")(b), tp2.Nf_no_smear_backward.term)
```

## Image Size

```python
from numpy import pi
# Replace wl_sb_max and pi by symbol in formula
show_sympy(Symbol("Theta_FoV"), tp3.Theta_fov_total /tp3.wl_sb_max*tp.wl_sb_max * pi/Symbol("pi"))
```

## Image Plane Pixel Size

```python
# Replace wl_sb_min by symbol in formula
show_sympy(Symbol("Theta_pix"), tp3.Theta_pix /tp3.wl_sb_min*tp.wl_sb_min)
```

## Number of Pixels on Image or Grid Size

```python
# All replacements from above
show_sympy(Symbol("N_pix"), tp3.Npix_linear_fov_total
            /tp3.wl_sb_max*tp.wl_sb_max *tp3.wl_sb_min/tp.wl_sb_min *pi/Symbol("pi"))
```

## Update Frequency Scale for Convolution Kernels

```python
show_sympy(Function("N_f,kernel")(b), tp2.Nf_gcf_backward.term)
```

## Size of the w-kernel

```python
show_sympy(1/(2*Symbol("pi")),1/2/pi)
show_sympy(Function("N_GW")(b), tp2.Ngw_backward.term)
show_sympy(tp.DeltaW_max.term, tp3.DeltaW_max.term)
```

## Imaging Pipeline Geometry Assumptions

```python
show_sympy(Function("\Delta w_min")(b), tp3.DeltaW_Earth.term)
```


```python
from devito import *
from examples.seismic.source import WaveletSource, TimeAxis
from examples.seismic import plot_image
import numpy as np

from sympy import init_printing, latex
init_printing(use_latex=True)
```

```python
# Initial grid: 1km x 1km, with spacing 100m
extent = (2000., 2000.)
shape = (81, 81)
x = SpaceDimension(name='x', spacing=Constant(name='h_x', value=extent[0]/(shape[0]-1)))
z = SpaceDimension(name='z', spacing=Constant(name='h_z', value=extent[1]/(shape[1]-1)))
grid = Grid(extent=extent, shape=shape, dimensions=(x, z))
```

```python
class DGaussSource(WaveletSource):

    def wavelet(self, f0, t):
        a = 0.004
        return -2.*a*(t - 1./f0) * np.exp(-a * (t - 1./f0)**2)

# Timestep size from Eq. 7 with V_p=6000. and dx=100
t0, tn = 0., 200.
dt = 1e2*(1. / np.sqrt(2.)) / 60.
time_range = TimeAxis(start=t0, stop=tn, step=dt)

src = DGaussSource(name='src', grid=grid, f0=0.01, time_range=time_range)
src.coordinates.data[:] = [1000., 1000.]
```

```python
#NBVAL_SKIP

src.show()
```

```python
# Now we create the velocity and pressure fields
p = TimeFunction(name='p', grid=grid, staggered=NODE, space_order=2, time_order=1)
vx= TimeFunction(name='vx', grid=grid, staggered=x, space_order=2, time_order=1)
vz = TimeFunction(name='vz', grid=grid, staggered=z, space_order=2, time_order=1)
```

```python
t = grid.stepping_dim
time = grid.time_dim

# We need some initial conditions
V_p = 4.0
#V_s = 1.0
density = 3.

dx = 100.
rox = 1/density * dt
roz = 1/density * dt
l2m = V_p*V_p*density * dt

c1 = 9.0/8.0;
c2 = -1.0/24.0;

# The source injection term
src_p = src.inject(field=p.forward, expr=src)

# 2nd order acoustic according to fdelmoc
u_vx_2 = Eq(vx.forward, vx + rox * p.dx)
u_vz_2 = Eq(vz.forward, vz + roz * p.dz)
u_p_2 = Eq(p.forward, p + l2m * (vx.forward.dx + vz.forward.dz))

op_2 = Operator([u_vx_2, u_vz_2, u_p_2] + src_p)

# 4th order acoustic according to fdelmoc
# Now we create the velocity and pressure fields
p4 = TimeFunction(name='p', grid=grid, staggered=NODE,space_order=4, time_order=1)
vx4= TimeFunction(name='vx', grid=grid, staggered=x, space_order=4, time_order=1)
vz4 = TimeFunction(name='vz', grid=grid, staggered=z, space_order=4, time_order=1)
u_vx_2 = Eq(vx4.forward, vx4 + rox * p4.dx)
u_vz_2 = Eq(vz4.forward, vz4 + roz * p4.dz)
u_p_2 = Eq(p4.forward, p4 + l2m * (vx4.forward.dx + vz4.forward.dz))

op_4 = Operator([u_vx_2, u_vz_2, u_p_2] + src_p)
```

```python
### 2nd order acoustic

# Reset the fields
vx.data[:] = 0.
vz.data[:] = 0.
p.data[:] = 0.

```

```python
#NBVAL_IGNORE_OUTPUT

# Propagate the source
op_2(time=src.time_range.num-1)
```

    Operator `Kernel` run in 0.01 s

```python
#NBVAL_SKIP

# Let's see what we got....
plot_image(vx.data[0])
plot_image(vz.data[0])
plot_image(p.data[0])
```

```python
#NBVAL_IGNORE_OUTPUT

### 4th order acoustic

# Propagate the source
op_4(time=src.time_range.num-1)
```

    Operator `Kernel` run in 0.02 s

```python
#NBVAL_SKIP

# Let's see what we got....
plot_image(vx4.data[0])
plot_image(vz4.data[0])
plot_image(p4.data[0])
```


# The $z$-Transform

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Comunications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Definition

The [$z$-transform](https://en.wikipedia.org/wiki/Z-transform), as the [discrete-time Fourier transform](../discrete_time_fourier_transform/definition.ipynb) (DTFT), is a transform which represents a discrete signal $x[k]$ in the spectral domain. Unlike the DTFT which uses the harmonic complex exponential function $e^{-j \Omega k}$ with $\Omega \in \mathbb{R}$ as kernel, the $z$-transformation bases on the complex exponential function $z^{-k}$ with $z \in \mathbb{C}$ as kernel.

### Two-Sided $z$-Transform

The two-sided (or bilateral) $z$-transform is defined as

\begin{equation}
X(z) = \sum_{k = -\infty}^{\infty} x[k] \, z^{-k}
\end{equation}

where $X(z) = \mathcal{Z} \{ x[k] \}$ denotes the $z$-transform of $x[k]$. A complex signal $x[k] \in \mathbb{C}$ with discrete index $k \in \mathbb{Z}$ is represented by its complex valued $z$-transform $X(z) \in \mathbb{C}$ with complex dependent variable $z \in \mathbb{C}$. The variable $z$ can be interpreted as [complex frequency](../discrete_signals/standard_signals.ipynb#Complex-Exponential-Signal) $z = e^{\Sigma + j \Omega}$ with $\Sigma, \Omega \in \mathbb{R}$.

Whether a $z$-transform $X(z) = \mathcal{Z} \{ x[k] \}$ exists depends on the complex frequency $z$ and the signal $x[k]$ itself. All values $z$ for which the $z$-transform converges form a region of convergence (ROC). The $z$-transforms of two different signals may differ only with respect to their ROCs. Consequently, the ROC needs to be explicitly given for a unique inversion of the $z$-transform.

### One-Sided $z$-Transform

Causal signals play an important role in the theory of signals and systems. For a causal signal with $x[k] = 0$ for $k <0$, the relation $x[k] = x[k] \cdot \epsilon[k]$ holds. Introducing this into the definition of the two-sided $z$-transform results in

\begin{equation}
X(z) = \sum_{k = -\infty}^{\infty} x[k] \cdot \epsilon[k] \, z^{-k} = \sum_{k = 0}^{\infty} x[k] \, z^{-k}
\end{equation}

This motivates the definition of the one-sided (or unilateral) $z$-transform

\begin{equation}
X(z) = \sum_{k = 0}^{\infty} x[k] \, z^{-k}
\end{equation}

In the literature both the one- and two-sided $z$-transform are termed as $z$-transform. For causal signals both give the same result. The one-sided $z$-transform is also useful for the solution of initial value problems, for instance as defined by [linear difference equations with constant coefficients](../discrete_systems/difference_equation.ipynb) where the initial values are given for $k=0$.

### Signals of Finite Duration

The $z$-transform of a generic signal $x[k]$ of finite duration $x[k] = 0$ for $\{k : k < M_1 \wedge k \geq M_2\}$ with $M_1 < M_2$ reads

\begin{equation}
X(z) = \sum_{k=M_1}^{M_2 - 1} x[k] \, z^{-k} = x[M_1] \, z^{-M_1} + x[M_1 + 1] \, z^{- (M_1 + 1)} + \dots + x[M_2 - 1] \, z^{- (M_2-1)}
\end{equation}

The transform of a generic finite-length signal is given as a polynomial in $z$. Depending on the particular limits $M_1$ and $M_2$, the polynomial may contain powers of $z$ and $z^{-1}$. For a causal finite-length signal, above result specializes to the case $M_1 = 0$ and $M_2 > 0$ as

\begin{equation}
X(z) = x[0] + x[1] \, z^{-1} + \dots + x[M_2] \, z^{- M_2}
\end{equation}

The transform of a causal finite-length signal is given as a polynomial in $z^{-1}$. The ROC for this case is given as $z \in \mathbb{C} \setminus \{ 0 \}$. Similar considerations yield the $z$-transform and ROC of an anticausal signal. The ROCs for a generic finite-length signal can be summarized as

* $z \in \mathbb{C}$ for an anticausal signal ($M_1 < 0$, $M_2 \leq 0$)
* $z \in \mathbb{C}_\infty$ if $x[k] = 0$ for $k \neq 0$ ($M_1 = 0$, $M_2 = 1$)
* $z \in \mathbb{C} \setminus \{ 0 \}$ for a causal signal ($M_1 = 0$, $M_2 > 0$)

where $\mathbb{C}_\infty = \mathbb{C} \cup \{ \infty \}$ denotes the [set of extended complex numbers](https://en.wikipedia.org/wiki/Riemann_sphere).

### Transformation of the Rectangular Signal

The rectangular signal $x[k] = \text{rect}_N[k]$ is a causal signal of finite duration. Using above result its $z$-transform can be derived straightforward as

\begin{equation}
\mathcal{Z} \{ \text{rect}_N[k] \} = \sum_{k=0}^{N-1} z^{-k} = 1 + z^{-1} + \dots + z^{-N + 1}
\end{equation}

for $z \in \mathbb{C} \setminus \{ 0 \}$. Above sum can also be interpreted as [finite geometrical series](https://en.wikipedia.org/wiki/Geometric_series) with common ratio $\frac{1}{z}$, resulting in an alternative form of the $z$-transform of the rectangular signal

\begin{equation}
\mathcal{Z} \{ \text{rect}_N[k] \} =  \begin{cases}
\frac{1 - z^{-N}}{1 - z^{-1}} & \text{for } z \neq 1 \\
N & \text{for } z = 1
\end{cases}
\end{equation}

### Transformation of the Dirac Impulse

The transform $\mathcal{Z} \{ \delta[k] \}$ of the [Dirac impulse](../discrete_signals/standard_signals.ipynb#Dirac-Impulse) is derived by introducing $\delta[k]$ into the definition of the two-sided $z$-transform and exploiting the sifting property of the Dirac impulse

\begin{equation}
\mathcal{Z} \{ \delta[k] \} = \sum_{k = -\infty}^{\infty} \delta[k] \, z^{-k} = 1
\end{equation}

for $z \in \mathbb{C}$. The ROC covers the entire complex plane.

The transform of the Dirac impulse is equal to one. Hence, all complex frequencies $z$ are present with equal weight. This is an important property in the theory of discrete signals and systems, since the Dirac impulse is used to characterize linear time-invariant (LTI) systems by their [impulse response](../discrete_systems/impulse_response.ipynb) $h[k] = \mathcal{H} \{ \delta[k] \}$.

**Example**

The $z$-transform of the Dirac impulse is computed by direct evaluation of its definition. The Dirac impulse is represented by the [Kronecker delta](https://en.wikipedia.org/wiki/Kronecker_delta) $\delta[k] = \delta_{k 0}$ in `SymPy`.

```python
%matplotlib inline
import sympy as sym
sym.init_printing()

k = sym.symbols('k', integer=True)
z = sym.symbols('z', complex=True)

X = sym.summation(sym.KroneckerDelta(k, 0) * z**(-k), (k, -sym.oo, sym.oo))
X
```

**Exercise**

* Derive the two-sided $z$-transform of a shifted Dirac impulse $\delta[t - \kappa]$ for $\kappa \in \mathbb{Z}$ by manual evaluation of its definition and by modification of above example. Provide the ROC for the three cases $\kappa<0$, $\kappa=0$ and $\kappa>0$.

* Derive the one-sided $z$-transform of a shifted Dirac impulse $\delta[t - \kappa]$. Hint: Differentiate between the cases $\kappa < 0$ and $\kappa \geq 0$.

### Transformation of the Causal Complex Exponential Signal

The transform $X(z) = \mathcal{Z} \{ x[k] \}$ of the causal complex exponential signal

\begin{equation}
x[k] = \epsilon[k] \cdot z_0^k
\end{equation}

with complex frequency $z_0 \in \mathbb{C}$ is derived by evaluation of the definition of the one-sided $z$-transform

\begin{equation}
X(z) = \sum_{k=0}^{\infty} z_0^k \cdot z^{-k} = \sum_{k=0}^{\infty} \left( \frac{z_0}{z} \right)^k = \frac{z}{z - z_0}
\end{equation}

The last equality has been derived by noting that the sum constitutes an [infinite geometrical series](https://en.wikipedia.org/wiki/Geometric_series) with common ratio $\frac{z_0}{z}$ which converges for $\left| \frac{z_0}{z} \right| < 1$. The ROC is consequently given as

\begin{equation}
|z| > |z_0|
\end{equation}

Combining above results, the transformation of the causal complex exponential signal reads

\begin{equation}
\mathcal{Z} \{ \epsilon[k] \cdot z_0^k \} = \frac{z}{z - z_0} \qquad \text{for } |z| > |z_0|
\end{equation}

**Example**

The $z$-transform of the causal complex exponential signal $x[k] = z_0^k \cdot \epsilon[k]$ with $z_0 \in \mathbb{C}$ is computed by direct evaluation of its definition.

```python
z0 = sym.symbols('z0')

X = sym.summation(z0**k * z**(-k), (k, 0, sym.oo))
X
```

Note that the sum is returned in case that it cannot be evaluated. It hence can be concluded that the series converges only for $|\frac{z_0}{z}| < 1$. This is in line with the analytic result derived above.

## Convergence

The definition of the $z$-transform constitutes an infinite series. A sufficient but not necessary condition for the convergence of an infinite series is that its elements are absolutely summable. Applying this to the definition of the two-sided $z$-transform $X(z) = \mathcal{Z} \{ [k] \}$ of a given signal $x[k]$ yields

\begin{equation}
\sum_{k = -\infty}^{\infty} | x[k] \cdot z^{- k} | = \sum_{k = -\infty}^{\infty} | x[k] | \cdot | z |^{- k}  < \infty
\end{equation}

It can be concluded from this result that the ROC is determined solely by the magnitude $|z|$ of the complex frequency. It follows further that the ROC is given as a ring in the $z$-plane which may include $z=0$ and/or $z=\infty$. The phase $\Omega$ of the complex frequency $z$ has no effect in terms of convergence, as $|z| = e^{\Sigma} \cdot e^{j \Omega}$.

For a right-sided signal of infinite length with $x[k] = 0$ for $k < M$, the ROC is given in the form of $|z| > a$ with $a \in \mathbb{R}^+$. This can be concluded from the decay of the term $| z |^{- k}$ for $k \to \infty$, which ensures convergence for a given $a$. However this holds only for signals $x[k]$ with [exponential growth](https://en.wikipedia.org/wiki/Exponential_growth). Please refer to the $z$-transform of the causal exponential signal derived above.

The same reasoning leads to the ROCs of a left-sided and two-sided signal. The resulting ROCs are illustrated in the following

The gray areas denote the values $z$ for which the $z$-transform converges. The borders $a$ of these areas (dashed lines) depend on the signal $x[k]$. In case that the $z$-transform $X(z)$ is given in terms of a rational function in $z$, the ROC has to be chosen such that it does not include zeros of the denominator polynomial. A more detailed discussion of the ROCs for the $z$-transform can be found in the literature, e.g. [[Girod et al.](index.ipynb#Literature)].

## Relation to the Laplace Transform of a Sampled Signal

The link between the Laplace transform of a sampled signal $x_\text{s}(t)$ and the $z$-transform of its discrete counterpart $x[k] = x(k T)$ is established in the following. Under the assumption of [ideal sampling](../sampling/ideal.ipynb#Model-of-Ideal-Sampling), the sampled signal reads

\begin{equation}
x_\text{s}(t) = \sum_{k = -\infty}^{\infty} x(k T) \cdot \delta(t - k T) = \sum_{k = -\infty}^{\infty} x[k] \cdot \delta(t - k T)
\end{equation}

where $x(t)$ denotes the continuous signal and $T$ the sampling interval. Introducing the sampled signal into the [definition of the Laplace transform](../laplace_transform/definition.ipynb) yields the transform of the sampled signal

\begin{equation}
X_\text{s}(s) = \int_{-\infty}^{\infty} \sum_{k = -\infty}^{\infty} x[k] \cdot \delta(t - k T) \, e^{- s t} \; dt = \sum_{k = -\infty}^{\infty} x[k] \, e^{-s k T}
\end{equation}

where the last equality has been derived by changing the order of summation/integration and exploiting the [sifting property of the Dirac impulse](../continuous_signals/standard_signals.ipynb#Dirac-Impulse). Comparison with the definition of the $z$-transform yields

\begin{equation}
X_\text{s}(s) = X(z) \big\rvert_{z = e^{s T}}
\end{equation}

The spectrum of the sampled signal $X_\text{s}(s)$ is equal to the $z$-transform of the discrete Signal $X(z)$ for $z = e^{s T}$. The resulting mapping from the $s$-plane to the $z$-plane is illustrated by the shading and the colors in the following figure

The Laplace transform of a sampled signal is peridoic with respect to the frequency $\omega$. It sufficient to consider the strip $-\frac{\pi}{T} < \Im \{ s \} < \frac{\pi}{T}$ of the $s$-plane for the mapping. The left half-plane of the $s$-plane is mapped into the unit circle of the $z$-plane. The corresponding right half-plane is mapped to the outside of the unit circle. The imaginary axis of the $s$-plane is mapped onto the unit circle $|z|=1$. The frequency $s=0$ is mapped onto $z=1$ and the frequencies $s=\pm j \frac{\pi}{T}$ are mapped onto $z=-1$.

## Relation to the Discrete-Time Fourier-Transform

The two-sided $z$-transform $X(z) = \mathcal{z} \{ x[k] \}$ of a signal $x[k]$ can be related to its [discrete-time Fourier transform](../discrete_time_fourier_transform/definition.ipynb) (DTFT) $X(e^{j \Omega}) = \mathcal{F}_* \{ x[k] \}$ by inspecting the kernels of both transforms. The $z$-transform has the complex exponential function $z^{-k}$ with $z \in \mathbb{C}$ as kernel. The DTFT has the harmonic exponential function $e^{- j \Omega k}$ with $\Omega \in \mathbb{R}$ as kernel. Both can be related to each other by considering that $z = e^{\Sigma + j \Omega}$. Hence, if the ROC includes the unit circle $|z| = 1$ of the $z$-plane, the DTFT of a signal $x[k]$ can be derived from its $z$-transform by

\begin{equation}
\mathcal{F}_* \{ x[k] \} = \mathcal{Z} \{ x[k] \} \bigr\rvert_{z = e^{j \Omega}}
\end{equation}

If the ROC does not include the unit circle, the DTFT of a given signal does not exist. A benefit of the $z$-transform over the DTFT is that it can be applied to a wider class of signals.

**Example**

The DTFT $X(e^{j \Omega}) = \mathcal{F}_* \{ x[k] \}$ of the causal exponential signal

\begin{equation}
x[k] = \epsilon[k] \cdot e^{- \Sigma_0 k}
\end{equation}

for $\Sigma_0 \in \mathbb{R}^+$ is derived from the $z$-transform of the [causal complex exponential signal](#Transformation-of-the-Causal-Complex-Exponential-Signal). Using the substituting $z_0 = e^{-\Sigma_0}$ yields

\begin{equation}
X(z) = \frac{z}{z - e^{-\Sigma_0}}
\end{equation}

with the ROC $|z| > e^{-\Sigma_0}$. The ROC includes the unit circle for $0 < \Sigma_0 < \infty$. The DTFT can be derived from the $z$-transform by substituting $z$ with $e^{j \Omega}$ as

\begin{equation}
X(e^{j \Omega}) = \frac{e^{j \Omega}}{e^{j \Omega} - e^{-\Sigma_0}}
\end{equation}

## Representation

The $z$-transform $X(z) = \mathcal{Z} \{ x[k] \}$ depends on the complex frequency $z \in \mathbb{C}$ and is in general complex valued $X(z) \in \mathbb{C}$. It can be illustrated by plotting its magnitude $|X(z)|$ and phase $\varphi(z)$ or real $\Re \{ X(z) \}$ and imaginary $\Im \{ X(z) \}$ part in the complex $z$-plane. The resulting three-dimensional plots are often not very illustrative.

However, many $z$-transforms of interest in the theory of signals and systems are [rational functions](https://en.wikipedia.org/wiki/Rational_function) in $z$. The polynomials of the numerator and denominator can be represented by their complex [roots](https://en.wikipedia.org/wiki/Zero_of_a_function#Polynomial_roots) and a constant factor. The roots of the numerator are termed as *zeros* while the roots of the denominator a termed as *poles* of $X(z)$. The polynomial and the zero/pole representation of a rational $z$-transform are equivalent

\begin{equation}
X(z) = \frac{\sum_{m=0}^{M} \beta_m \, z^{-m}}{\sum_{n=0}^{N} \alpha_n \, z^{-n}} = K \cdot \frac{\prod_{\mu=0}^{Q} (z - z_{0 \mu})}{\prod_{\nu=0}^{P} (z - z_{\infty \nu})}
\end{equation}

where $M$ and $N$ denote the order of the numerator/denominator polynomial, $z_{0 \mu}$ and $z_{\infty \nu}$ the $\mu$-th zero/$\nu$-th pole of $X(z)$, and $Q = M-1$ and $P = N-1$ the total number of zeros and poles, respectively. For $M=N$ the factor $K = \frac{\beta_M}{\alpha_N}$. If

* $M > N$ at least one pole is located at $|z| = \infty$,
* $M < N$ at least one pole is located at $z = 0$.

It is common to illustrate the poles and zeros in a [pole-zero plot](https://en.wikipedia.org/wiki/Pole%E2%80%93zero_plot). Here the locations of the complex poles and zeros, their degrees and the factor $K$ are plotted in the $z$-plane. It is common to include the unit circle $|z| = 1$ in the plot due to its relevance in the theory of discrete signals and systems. An example for a pole-zero plot is shown in the following

The locations of the poles and zeros provide insights into the composition of a signal. For instance, the $z$-transform of the [complex exponential signal](#Transformation-of-the-Causal-Complex-Exponential-Signal) has a zero at $z=0$ and a pole at $z = z_0$. A signal which is composed from a superposition of complex exponential signals will have multiple poles whose positions are related to the complex frequencies of the signals it is composed of.

**Exercise**

* Derive $X(z)$ together with its ROC from above pole-zero plot under the assumption that it represents a right-handed signal $x[k]$.

**Example**

The $z$-transform of the causal complex exponential signal

\begin{equation}
X(z) = \frac{z}{z - z_0} \qquad \text{for } |z| > |z_0|
\end{equation}

derived above is illustrated by plotting its magnitude $|X(z)|$ over the $z$-plane for $z_0 = 1 + j$

```python
rez, imz = sym.symbols('rez imz', real=True)

X = z / (z - z0)
X1 = X.subs({z: rez+sym.I*imz, z0: 1+sym.I})

sym.plotting.plot3d(abs(X1), (rez, -2, 2), (imz, -2, 2),
                    xlabel=r'$\Re\{z\}$', ylabel=r'$\Im\{z\}$', title=r'$|X(z)|$');
```

Alternatively, the $z$-transform $X(z)$ is illustrated by its pole-zero plot. First the poles

```python
poles = sym.roots(sym.denom(X), z)
poles
```

and zeros are computed

```python
zeros = sym.roots(sym.numer(X), z)
zeros
```

Above dictionaries are composed from the poles and zeros, and their degrees.

In order to illustrate the location of poles and zeros in the $z$-plane, the pole-zero plot is shown for $z_0 = 1 + j$

```python
from pole_zero_plot import pole_zero_plot

X2 = X.subs(z0, 1+sym.I)
pole_zero_plot(sym.roots(sym.denom(X2), z), sym.roots(sym.numer(X2), z))
```

**Copyright**

The notebooks are provided as [Open Educational Resource](https://de.wikipedia.org/wiki/Open_Educational_Resources). Feel free to use the notebooks for your own educational purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Lecture Notes on Signals and Systems* by Sascha Spors.


## Burgers Equation - Forward Euler/0-estimation
#### Parameter estimation for Burgers' Equation using Gaussian processes (Forward Euler scheme)

#### Problem Setup

$u_t + u u_{x} = \nu u_{x}$

$u(x,t) = \frac{x}{1+t}$ => We'd expect $\nu = 0$

$u_0(x) := u(x,0) = x$

$x \in [0, 1], t \in \{0, \tau \}$

Using the forward Euler scheme, the equation can be re-written as:

$\frac{u_n - u_{n-1}}{\tau} + u_{n-1} \frac{d}{dx}u_{n-1}  = \nu \frac{d^2}{dx}u_{n-1}$

and setting the factor $u_{n-1} = \mu_{n-1}$ to deal with the non-linearity:

$\tau \nu \frac{d^2}{dx}u_{n-1} - \tau \mu_{n-1} \frac{d}{dx}u_{n-1} + u_{n-1} = u_{n}$

Consider $u_{n-1}$ to be a Gaussian process.

$u_{n-1} \sim \mathcal{GP}(0, k_{uu}(x_i, x_j, \theta))$

And the linear operator:

$\mathcal{L}_x^\nu = \cdot + \tau \nu \frac{d}{dx}\cdot - \tau \mu_{n-1} \frac{d}{dx} \cdot$

so that

$\mathcal{L}_x^\nu u_{n-1} = u_n$

Problem at hand: estimate $\nu$ (should be $\nu = 0$ in the end).

For the sake of simplicity, take $u := u_{n-1}$ and $f := u_n$.

#### step 1: Simulate data

Take data points at $t = 0$ for $(u_{n-1})$ and $t = \tau$ for $(u_n)$, where $\tau$ is the time step.

```python
import numpy as np
import sympy as sp
import warnings
from scipy.optimize import minimize
import matplotlib.pyplot as plt
```

```python
tau = 0.001
def get_simulated_data(tau, alpha, beta):
    # Random numbers in [alpha, beta]
    x = np.random.rand(20)*(beta - alpha) + alpha
    y_u = x
    y_f = x/(1+tau)
    return (x, y_u, y_f)
```

#### Step 2:Evaluate kernels

$k_{nn}(x_i, x_j; \theta) = \theta exp(-\frac{1}{2l}(x_i-x_j)^2)$

```python
x_i, x_j, theta, l, nu, mu = sp.symbols('x_i x_j theta l nu mu')
kuu_sym = theta*sp.exp(-1/(2*l)*((x_i - x_j)**2))
kuu_fn = sp.lambdify((x_i, x_j, theta, l), kuu_sym, "numpy")
def kuu(x, theta, l):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kuu_fn(x[i], x[j], theta, l)
    return k
```

$k_{ff}(x_i,x_j;\theta,\phi) \\
= \mathcal{L}_{x_i}^\nu \mathcal{L}_{x_j}^\nu k_{uu}(x_i, x_j; \theta) \\
= k_{uu} + \tau \nu \frac{d}{dx_i}k_{uu} - \tau \mu_{n-1} \frac{d}{dx_i}k_{uu} + \tau \nu \frac{d}{dx_j}k_{uu} + \tau^2 \nu^2 \frac{d}{dx_i} \frac{d}{dx_j}k_{uu} - \tau^2 \nu \mu_{n-1}\frac{d^2}{dx_i dx_j} k_{uu} - \tau \mu_{n-1} \frac{d}{dx_j}k_{uu} - \tau^2 \nu \mu_{n-1} \frac{d^2}{dx_i dx_j} k_{uu} + \tau^2 \mu_{n-1}^2 \frac{d^2}{dx_i dx_j}k_{uu}$

```python
kff_sym = kuu_sym \
        + tau*nu*sp.diff(kuu_sym, x_i) \
        - tau*mu*sp.diff(kuu_sym, x_i) \
        + tau*nu*sp.diff(kuu_sym, x_j) \
        + tau**2*nu**2*sp.diff(kuu_sym, x_j, x_i) \
        - tau**2*nu*mu*sp.diff(kuu_sym, x_j, x_i) \
        - tau*mu*sp.diff(kuu_sym, x_j) \
        - tau**2*nu*mu*sp.diff(kuu_sym, x_j, x_i) \
        + tau**2*mu**2*sp.diff(kuu_sym, x_j, x_i)
kff_fn = sp.lambdify((x_i, x_j, theta, l, nu, mu), kff_sym, "numpy")
def kff(x, theta, l, nu, mu):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kff_fn(x[i], x[j], theta, l, nu, mu)
    return k
```

$k_{fu}(x_i,x_j;\theta,\phi) \\
= \mathcal{L}_{x_i}^\nu k_{uu}(x_i, x_j; \theta) \\
= k_{uu} + \tau \nu \frac{d}{dx_i}k_{uu} - \tau \mu_{n-1}\frac{d}{dx_i}k_{uu}$

```python
kfu_sym = kuu_sym + tau*nu*sp.diff(kuu_sym, x_i) - tau*mu*sp.diff(kuu_sym, x_i)
kfu_fn = sp.lambdify((x_i, x_j, theta, l, nu, mu), kfu_sym, "numpy")
def kfu(x, theta, l, nu, mu):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kfu_fn(x[i], x[j], theta, l, nu, mu)
    return k
```

```python
def kuf(x, theta, l, nu, mu):
    return kfu(x,theta, l, nu, mu).T
```

#### Step 3: Compute NLML

```python
def nlml(params, x, y1, y2, s, mu):
    theta_exp = np.exp(params[0])
    l_exp = np.exp(params[1])
    K = np.block([
        [kuu(x, theta_exp, l_exp) + s*np.identity(x.size), kuf(x, theta_exp, l_exp, params[2], mu)],
        [kfu(x, theta_exp, l_exp, params[2], mu), kff(x, theta_exp, l_exp, params[2], mu) + s*np.identity(x.size)]
    ])
    y = np.concatenate((y1, y2))
    val = 0.5*(np.log(abs(np.linalg.det(K))) + np.mat(y) * np.linalg.inv(K) * np.mat(y).T)
    return val.item(0)
```

#### Step 4: Optimise hyperparameters

```python
%%timeit
(x, y_u, y_f) = get_simulated_data(tau, 0, 1)
mu = np.mean(y_u)
m = minimize(nlml, np.random.rand(3), args=(x, y_u, y_f, 1e-7, mu), method="Nelder-Mead")
```

    13.2 s ± 2.01 s per loop (mean ± std. dev. of 7 runs, 1 loop each)

```python
results = []
warnings.filterwarnings("ignore")
for n in range(10):
    (x, y_u, y_f) = get_simulated_data(tau, n, n+1)
    mu = np.mean(y_u)
    m = minimize(nlml, np.random.rand(3), args=(x, y_u, y_f, 1e-7, mu), method="Nelder-Mead", options = {'maxiter' : 1000})
    if m.status == 0:
        results.append(m.x[2])
```

```python
results
```

    [0.0002450294171610081,
     0.0005594256273098408,
     0.0009660283606295254,
     0.0017380149626413878,
     0.002002554197617694,
     0.0027096515506198128,
     0.003396823903370382,
     0.00396511030564704,
     0.004044037981253901,
     0.004589675043895319]

```python
lin = np.linspace(1,len(results),len(results))
plt.plot(lin, results)
```

- The result of the first run (with an increasing length of the interval [alpha, beta]):

results =
[0.0003267148093141276,
 0.0003488675338190198,
 0.011209156241128872,
 0.0796449338251744,
 -0.08602889885832063,
 -0.04599509119150834,
 0.043607183833924854]

with an average error of 0.038,

where beta is given by the values on the x-axis.

- The result of the second run (with an increasing shift of the interval [beta - 1, beta]):

results =
[0.0002450294171610081,
 0.0005594256273098408,
 0.0009660283606295254,
 0.0017380149626413878,
 0.002002554197617694,
 0.0027096515506198128,
 0.003396823903370382,
 0.00396511030564704,
 0.004044037981253901,
 0.004589675043895319]

with an average error of 0.002,

where beta is given by the values on the x-axis.

- When comparing Burgers Equation with mean and with no mean, we might assume that when keeping the interval low, the approximation with the mean is a good approximation which should also lead to the desired coefficient

```python

```


# Tracé des modes propre du tuyau pour différentes valeurs de $\beta$ et $\gamma$

```javascript
%%javascript
MathJax.Hub.Config({
    TeX: { equationNumbers: { autoNumber: "AMS" } }
});
```

    <IPython.core.display.Javascript object>

```python
from IPython.core.display import HTML
HTML("""
<style>
.output_png {
    display: table-cell;
    text-align: center;
    vertical-align: middle;
}
</style>
""")
```

On cherche ici à résoudre l'équation linéaire du tuyau encastré-libre et notamment à apprendre plus sur ses modes propres.

On rappelle l'équation linéaire du tuyau encastré-libre:
\begin{equation}
    \alpha \dot{\eta}^{''''}+\eta^{''''}+(u^2-\gamma)(1-\xi)\eta^{''}+2u\sqrt{\beta}\dot{\eta}^{'}+\gamma\eta^{'}+\ddot{\eta} \end{equation}

Où $( )^{'} = \frac{\partial ( )}{\partial \xi}, \quad \dot{( )} = \frac{\partial ()}{\partial \tau} \quad$
Avec $\xi=\frac{x}{L},\quad \eta=\frac{w}{L},\quad \tau=\sqrt{\frac{EI}{M+m}}\frac{t}{L^2}\quad $
Et $u = \sqrt{\frac{M}{EI}}LU, \quad \beta=\frac{M}{M+m}, \quad \gamma = \frac{(M+m)L^3}{EI}g $

Pour résoudre cette équation on utilise la méthode de Galerkin consistant à écrire la solution comme une combinaison linéaire des modes propres d'une poutre:

\begin{equation}
\eta(\xi,\tau)=\sum_{r=1}^{\infty}\phi_r(\xi)q_r(\tau)
\end{equation}

Où $\phi_r(\xi)$ est la r-ième fonction propre d'une poutre encastrée-libre:

\begin{equation}
\phi_r(x)=cosh(\lambda_r x)-cos(\lambda_r x)-\sigma_r(sinh(\lambda_r x)-sin(\lambda_r x))
\end{equation}

Avec

\begin{equation}
\sigma_r = \frac{(sinh(\lambda_r L)-sin(\lambda_r L)}{cosh(\lambda_r L)+cos(\lambda_r L)}
\end{equation}

Et $\lambda_r$ la r-iéme valeur propre adimentionnée d'une poutre encastrée libre vérifiant:

\begin{equation}
cos(\lambda_r L)cosh(\lambda_r L)+1=0
\end{equation}

Ces résultats peuvent se retrouver facilement en appliquant la théorie des poutres sur une poutre vibrante.

On injecte alors la solution (2) dans l'equation (1) et on intégre sur l'intervalle [0,1]. Comme les fonctions propres $\phi_r$ sont orthonormées on peut écrire:

\begin{equation}
\int_0^1\phi_s\phi_rd\xi=\delta_{sr}
\end{equation}

De plus l'équation de la poutre encastrée libre donne:

\begin{equation}
\phi_r^{''''}=\lambda_r^4\phi_r
\end{equation}

Ainsi on obtient l'équation suivante pour s de 1 à N:

\begin{equation}
\sum^N_{r=1} \left( \delta_{sr}\ddot{q_r}+\left[2u\sqrt{\beta}\int_0^1\phi_s\phi_r^{''}d\xi\right]\dot{q_r}+ \left[ \lambda_r^4\delta_{sr}+(u^2\gamma)\int^1_0\phi_s\phi_r^{''}d\xi+\gamma\int^1_0\phi_s\phi_r^{'}d\xi+\gamma\int^1_0\xi\phi_s\phi_r^{''}d\xi \right] q_r \right) =0
\end{equation}

On peut poser:

\begin{equation}
b_{sr}=\int^1_0\phi_s\phi_r^{'}d\xi \quad c_{sr}=\int^1_0\phi_s\phi_r^{''}d\xi \quad d_{sr}=\int^1_0\xi\phi_s\phi_r^{''}d\xi
\end{equation}

Avec les conditions aux limites $\phi(0)=\phi(0)^{'}=\phi(L)^{''}=\phi(L)^{'''}=0$ on obtient ainsi:

\begin{equation}
b_{sr}=\frac{4}{(\frac{\lambda_s}{\lambda_r})^2+(-1)^{r+s}} \quad (s\ne r) \quad
b_{rr}=2   \\
c_{sr}=\frac{4(\lambda_r \sigma_r-\lambda_s \sigma_s)} {(-1)^{r+s}-(\frac{\lambda_s}{\lambda_r})^2} \quad (s\ne r) \quad
c_{rr}=\lambda_r \sigma_r(2-\lambda_r \sigma_r)  \\
d_{sr}=\frac{4(\lambda_r \sigma_r-\lambda_s \sigma_s+2)} {(1-(\frac{\lambda_s}{\lambda_r})^4}(-1)^{r+s}-\frac{3+(\frac{\lambda_s}{\lambda_r})^4}{1-(\frac{\lambda_s}{\lambda_r})^4}b_sr \quad (s\ne r) \quad d_{rr}=\frac{1}{2}c_{rr}
\end{equation}

On peut ainsi réécrire (8) sous forme matricielle:

\begin{equation}
\ddot{q}+2u\sqrt{\beta}B\dot{q}+[\Delta+\gamma B+(u^2-\gamma)C+\gamma D]q=0
\end{equation}

Où les matrices B, C et D sont des matrices de diamension (NxN) respectivement constituées des termes $b_{sr}$ $c_{sr}$ et $d_{sr}$ et où la matrice $\Delta$ est une matrice diagonale dont les termes sont les $\lambda_r^4$

On obtient ainsi un système du type:

\begin{equation}
M\ddot{q}+S\dot{q}+Kq
\end{equation}

On veut ramener ce problème d'ordre 2 à un problème d'ordre 1, pour cela on pose: $z=(\dot{q}, q)$ et l'equation (12) se réécrit alors:

\begin{equation}
F\dot{z}+Ez=0
\end{equation}

Avec $F=\begin{pmatrix}
0 & M\\
M & S
\end{pmatrix}$ et $E=\begin{pmatrix}
-M & 0\\
0 & K
\end{pmatrix}$

On propose $z=Ae^{\lambda \tau}$ comme solution et on l'injecte dans l'équation (13). On obtient alors:

\begin{equation}
\lambda FAe^{\lambda\tau}+EAe^{\lambda \tau}= 0 \iff (\lambda F+E)A=0 \iff \lambda I_dA = (-F^{-1}E)A
\end{equation}

Il nous suffit ainsi de trouver les valeurs propres de la matrice $-F^{-1}E$ pour trouver $\lambda$

Le code suivant permet de résoudre numériquement le problème définit précédemment et de visualiser les modes propres. Il faut exécuter chaque bloc dans l'ordre pour résoudre le problème. Lorsque le symbole de l'onglet est un sablier cela signifie qu'au moins une des cellules est en cours d'exécution. Les cellules en cours d'exécution sont marquées par une astérix entre crochets: [*] et et celles qui ont été exécutées sont marquées par un numéro entre crochets: [n].

Ce premier bloc permet d'importer les bibliothéques nécessaires à la résolution du problème. La bibliothéque numpy permet de travailler avec des vecteurs et des matrices. La bibliothéque linalg de scipy permet de calculer les valeurs propres de la matrice. Enfin la bibliothéque matplotlib.pyplot permet de tracer les résultats.

```python
import numpy as np
from scipy import linalg
import matplotlib.pyplot as mpl
import matplotlib.ticker as mpt
```

Ce second bloc permet de définir les paramètres du calcul. Si vous sélectionnez un cas entre 1 et 4 alors les paramètres sont déjà définis pour obtenir une comparaison avec des résultats obtenus par Paidoussis.
Si vous sélectionnez le cas 0 alors vous devez renseigner les paramètres en début de cellule (tous égaux à ceux du cas 1 par défaut) pour obtenir les résultats correspondant au système de votre choix. Il peut être intéressant de jouer avec ce cas pour s'intéresser à l'influence des difféerents paramètres.

```python
L = 0.5  ##Longueur du tuyau en m

##Parametres à définir pour le cas Custom
N = 10  ##Nombre de modes de poutre utilisé avec la méthode de Galerkin
umax = 20  ##Vitesse maximale du fluide
du = 0.01  ##Valeur utilisée pour l'incrémentation de la vitesse du fluide
gamma = 0  ##Valeur du paramètre de gravité gamma
beta = 0.2  ##Valeur du paramètre de masse beta
xmin = -50
xmax = 150
ymin = -25
ymax = 50

Custom = [umax,gamma,beta,du,N,xmin,xmax,ymin,ymax,1,"https://raw.githubusercontent.com/MorganDemenois/Pipe-conveying-fluid/main/Images/Blank.jpg"]
P1 = [20,0,0.2,0.01,10,0,130,-20,30,1.2,"https://raw.githubusercontent.com/MorganDemenois/Pipe-conveying-fluid/main/Images/beta0.2.PNG"]
P2 = [20,0,0.295,0.01,10,0,130,-20,35,1.2,"https://raw.githubusercontent.com/MorganDemenois/Pipe-conveying-fluid/main/Images/beta0.295.PNG"]
P3 = [20,0,0.5,0.01,10,0,65,-10,25,1.2,"https://raw.githubusercontent.com/MorganDemenois/Pipe-conveying-fluid/main/Images/beta0.5.PNG"]
P4 = [20,10,0.65,0.01,10,0,70,-10,40,1,"https://raw.githubusercontent.com/MorganDemenois/Pipe-conveying-fluid/main/Images/beta0.65.PNG"]
Parametres = [Custom,P1,P2,P3,P4]

##Choix du cas à résoudre
Cas = 1

##Parametres en fonction du cas choisi
umax = Parametres[Cas][0]
gamma = Parametres[Cas][1]
beta = Parametres[Cas][2]
du = Parametres[Cas][3]
N = Parametres[Cas][4]
xmin = Parametres[Cas][5]
xmax = Parametres[Cas][6]
ymin = Parametres[Cas][7]
ymax = Parametres[Cas][8]
asp = Parametres[Cas][9]
url = Parametres[Cas][10]
```

Ce bloc sert à calculer les N valeurs propres de poutre encastrée-libre $(\lambda)$. On cherche des solutions de l'équation (3) obtenues avec la théorie des poutres. On commence par définir une fonction qui renvoit la valeur de l'équation caractéristique (3). On cherche ensuite les N premiéres racines de cette fonction. Pour cela on utilise la méthode des sécantes qui reprend le principe de la méthode de Newton. On utilise la relation de récurrence suivante: $x_{n+1}=x_n-\frac{x_n-x_{n-1}}{f(x_n)-f(x_{n-1})}f(x_n)$ qui converge vers une racine de f. De plus la pratique a montré que $\lambda_r \approx \frac{2k+1}{2}\pi$. On choisit donc $x_0=\frac{2k+1}{2}\pi+0.1$ et $x_1=\frac{2k+1}{2}\pi-0.1$ pour chaque valeur propre pour que la méthode converge vers la bonne racine.

```python
def Eq_Cara(Lambda):
    return np.cos(Lambda)*np.cosh(Lambda)+1

LAMBDA = []

for i in range(N):
    LL_Guess = np.pi*(2*i+1)/2  ##Guess initial de la solution en utilisant la formule approchée
    x0 = LL_Guess + 0.1
    x1 = LL_Guess - 0.1
    while abs(x0-x1)>10**-16:  ##Condition de convergence du code
        xnew = x0 - (x0-x1)*Eq_Cara(x0)/(Eq_Cara(x0)-Eq_Cara(x1))  ##Calcul des nouveaux termes à l'aide de la formule de récurrence
        x1 = x0
        x0 = xnew
    LAMBDA.append(x0)

##Affichage des résultats
print()
for i in range(len(LAMBDA)):
    print("Valeur propre du mode "+str(i+1)+" d'une poutre: ")
    print(str(LAMBDA[i])+'\n')
```

    Valeur propre du mode 1 d'une poutre:
    1.8751040687119611

    Valeur propre du mode 2 d'une poutre:
    4.694091132974175

    Valeur propre du mode 3 d'une poutre:
    7.854757438237613

    Valeur propre du mode 4 d'une poutre:
    10.995540734875467

    Valeur propre du mode 5 d'une poutre:
    14.13716839104647

    Valeur propre du mode 6 d'une poutre:
    17.278759532088237

    Valeur propre du mode 7 d'une poutre:
    20.42035225104125

    Valeur propre du mode 8 d'une poutre:
    23.561944901806445

    Valeur propre du mode 9 d'une poutre:
    26.7035375555183

    Valeur propre du mode 10 d'une poutre:
    29.845130209102816

Ce bloc définit des fonctions permetant de calculer les termes $\sigma_r$, $b_{sr}$, $c_{sr}$ et $d_{sr}$ définis précédemment.

```python
def sigma(r):
    return ((np.sinh(L*LAMBDA[r])-np.sin(L*LAMBDA[r]))/(np.cosh(L*LAMBDA[r])+np.cos(L*LAMBDA[r])))

def bsr(s,r):
    if s == r:
        return 2
    else:
        return 4/((LAMBDA[s]/LAMBDA[r])**2+(-1)**(r+s))

def csr(s,r):
    if s == r:
        return LAMBDA[r]*sigma(r)*(2-LAMBDA[r]*sigma(r))
    else:
        return 4*(LAMBDA[r]*sigma(r)-LAMBDA[s]*sigma(s))/((-1)**(r+s)-(LAMBDA[s]/LAMBDA[r])**2)

def dsr(s,r):
    if s == r:
        return csr(s,r)/2
    else:
        return (4*(LAMBDA[r]*sigma(r)-LAMBDA[s]*sigma(s)+2)*(-1)**(r+s))/(1-(LAMBDA[s]/LAMBDA[r])**4)-((3+(LAMBDA[s]/LAMBDA[r])**4)/(1-((LAMBDA[s]/LAMBDA[r])**4)))*bsr(s,r)

```

La fonction MAC définie dans ce bloc permet de comparer deux vecteurs propres. Cette fonction est nécessaire car la fonction qui calcule les vecteurs propres et les valeurs propres d'une matrice les renvoit dans un ordre aléatoire. Ainsi lorsqu'on modifit la valeur de la vitesse u, les modes ne sont pas renvoyés dans le même ordre. Il est donc nécessaire de trouver une méthode permettant de trier les modes pour pouvoir suivre le même mode sur toute la plage de vitesse u. La méthode utilisée consiste à calculer le modal assurance criterion (MAC) entre les vecteurs propres obtenus aux vitesses $u_n$ et $u_{n-1}$. On obtient alors une matrice de taille (NxN) contenant les MAC entre chaque vecteurs. On associt ainsi les vecteurs (et donc les modes) ayant la valeur maximale de MAC. La MAC est définit comme suit: $MAC(X,Y) = \frac{|\{X\}^T|\{Y\}^*|^2}{(\{X\}^T\{X\}^*)(\{Y\}^T\{Y\}^*)}$

```python
def MAC(X,Y):
    return ((np.dot(X,np.conj(Y)).real)**2+(np.dot(X,np.conj(Y)).imag)**2)/(np.dot(X,np.conj(X))*np.dot(Y,np.conj(Y))).real
```

Le bloc suivant permet de définir les matrices B, C, D, $\Delta$ et M.

```python
B = np.zeros((N,N))
C = np.zeros((N,N))
D = np.zeros((N,N))
M = np.eye(N)
for i in range(N):
    for j in range(N):
        B[i,j] = bsr(i,j)
        C[i,j] = csr(i,j)
        D[i,j] = dsr(i,j)

Delta = np.zeros((N,N))
for i in range(N):
    Delta[i,i] = LAMBDA[i]**4

```

Dans ce bloc la fonction définie permet de calculer les matrices S et K en fonction de u puis de construire par blocs les matrices E et F. On utilise ensuite la fonction scypi.linalg.eig() pour calculer les vecteurs et valeurs propres du problème.

```python
def result(u):
    S = 2*(beta**0.5)*u*B
    K = Delta + gamma*B + (u**2-gamma)*C + gamma*D
    F = np.block([[np.zeros((N,N)),M],[M,S]])
    E = np.block([[-M,np.zeros((N,N))],[np.zeros((N,N)),K]])
    eigenValues, eigenVectors = linalg.eig(-np.dot(np.linalg.inv(F),E))
    return eigenValues, eigenVectors
```

Le bloc suivant initialise les tableaux qui vont contenir les vecteurs propres de chaque mode ainsi que les parties imaginaires et réelles des valeurs propres de chaque mode. Les modes sont ordonnées par partie réelle croissante de la pulsation pour u = 0. La pulsation est définie telle que $\Omega_r = -i*\lambda_r$

```python
##Initialisation des modes propres pour u = 0
u_array = np.array([0])
u = 0
eigenValues, eigenVectors = result(u)

##Tri des modes propres par partie réelle croissante de Omega
arg = np.argsort(np.array([(-1j*eigenValues).real]))[0]
IM_Omega = np.array([(-1j*eigenValues).imag[arg]])
RE_Omega = np.array([(-1j*eigenValues).real[arg]])
Vectors = np.array([eigenVectors[:,arg]])
```

Ce bloc contient la boucle qui va résoudre le problème pour toutes les valeurs de u. Son exécution peut prendre jusqu'à quelques minutes selon les paramétres utilisés au début du code. La boucle continue tant que la valeur de u n'a pas atteint la valeur finale définie au début du code. Un pas de vitesse adaptatif est également utilisé dans la boucle. Si les valeurs du MAC sont inférieures à une certaine valeur alors cela signifie qu'il faut réduire le pas u pour être certain d'avoir une bonne corrélation entre les vcecteurs propres et d'associer les bons modes ensembles.

```python
##Début de la boule sur la vitesse
i = 0
while u < umax:
    i += 1
    u_tempo = u + du
    du_tempo = du  ##Pas adaptatif

    eigenValues, eigenVectors = result(u_tempo)

    ##Construction de la matrice contenant les MAC entre chaque vecteurs propres de l'itération i et i+1
    diff = np.zeros((2*N,2*N))
    for l in range(2*N):
        for k in range(2*N):
            diff[l,k] = MAC(Vectors[i-1,:,l],eigenVectors[:,k])

    ##Si la valeur de la MAC entre 2 vecteurs du même mode est inférieure à 0.99 on réduit le pas de moitié pour obtenir la corrélation souhaitée
    while min(np.max(diff,1)) < 0.99 and du_tempo > 10**-10:
        u_tempo = u+du_tempo/2
        du_tempo = du_tempo/2  ##Réduction du pas
        eigenValues, eigenVectors = result(u)  ##Nouveau calcul de la solution
        for l in range(2*N):
            for k in range(2*N):
                diff[l,k] = MAC(Vectors[i-1,:,l],eigenVectors[:,k])  ##Nouvelle construction de la matrice contenant les MAC

    ##Remplissage des matrices contenant les valeurs de u, de Omega et des vecteurs propres
    u = u_tempo
    u_array = np.append(u_array,u)
    arg_Max = np.argmax(diff,1)
    IM_Omega = np.append(IM_Omega,[(-1j*eigenValues[arg_Max]).imag],axis=0)
    RE_Omega = np.append(RE_Omega,[(-1j*eigenValues[arg_Max]).real],axis=0)
    Vectors = np.append(Vectors,np.array([eigenVectors[:,arg_Max]]),axis=0)
```

Finalement ce dernier bloc permet de tracer les diagrammes d'évolutions des modes souhaités.

```python
###Choix des modes à tracer
mode1 = N
mode2 = N+1
mode3 = N+2
mode4 = N+3

fig = mpl.figure(figsize=(7, 7))
ax = fig.add_subplot(1, 1, 1, aspect=asp)
ax.plot(RE_Omega[:,mode1],IM_Omega[:,mode1],label="Mode 1")
ax.plot(RE_Omega[:,mode2],IM_Omega[:,mode2],label="Mode 2")
ax.plot(RE_Omega[:,mode3],IM_Omega[:,mode3],label="Mode 3")
ax.plot(RE_Omega[:,mode4],IM_Omega[:,mode4],label="Mode 4")
ax.set_xlim((xmin,xmax))
ax.set_ylim((ymin,ymax))
ax.xaxis.set_major_locator(mpt.MultipleLocator(10))
ax.yaxis.set_major_locator(mpt.MultipleLocator(5))
ax.set_xlabel("Re($\Omega$)")
ax.set_ylabel("Im($\Omega$)")
ax.set_title("Evolution de $\Omega$ en faisant varier la vitesse u pour beta = " + str(beta) + " et gamma = " + str(gamma))
ax.legend()
ax.grid()
```

Si un cas entre 1 et 4 (un cas traité par Paidoussis) a été sélectionné alors le bloc suivant affiche une image du diagramme de l'évolution de $\Omega$ obtenu dans la recherche de Paidoussis pour les paramètres $\gamma$ et $\beta$ choisis. Si le cas 0 a été sélectionné (cas custom) alors ce bloc n'affiche rien.

```python
from IPython.display import Image
if Cas != 0:
    print("Evolution de Omega en faisant varier la vitesse u pour beta = " + str(beta) + " et gamma = " + str(gamma))
Image(url)

```

```python

```

```python

```