## Predator-Prey Population Differential Equations

In this notebook we will discuss stability points, Isoclines and the Jaobian matrix for the Predator-Prey non-linear differential equation [1,2,3,4].

```python
## Library
import numpy as np
import math
import matplotlib.pyplot as plt
```

## Predators and prey
The cyclical relationship of Predators and Prey in the wild  means that when there is an abundance of prey, the predators will eat more and their population will increase. On the other hand, if there is a shortage in the number of prey, the number of predators will decrease. One of the of the most famous examples of this was observed by the fur traders  Hudson's Bay Company of Canada. The company collected data about the number of lynx and hare pelts they purchased over a 90 year period. Their data showed the cylical nature of the predator prey relationship.
This kind of behaviour has been observed in many pairs of predators and prey from mammals to fish to beyond.

## Snow Hares and Lynxes
In a balanced eco system the relationship between snow hares (prey) and lynxes (predator) can be described mathematically using a Loketta-Volterra model [2-4]
\begin{equation}
\begin{array}{cl}
\frac{d H}{dt}=a_{Birth} H-b_{Con}LH,\\
\frac{d L}{dt}=-c_{Death}L+d_{Food} L H,\\
\end{array}
\end{equation}
this set of equations models the interaction between snow hares (H) and lynxes (L). The lynes are the predators who eat the snow hares to survive. Where there parameters
* $a_{Birth}$ is the birth rate of the snow hares;
* $b_{Con}$ is the consumation rate of snow hares by lynxes (interaction);
* $c_{Death}$ is the death rate of lynxes if there is no food source;
* $d_{Food}$ is the rate that lynxes eat the snow hares to survive (interaction).

```python
a_Birth=4.0
b_Con=0.1
c_Death=0.9
d_Food=0.001

```

## Discrete Domain
To numerically approximate the solution we discretise the continuous domain using a step size, $h=0.001$,
which gives
\begin{equation}
t_i=0+i h,
\end{equation}
for $i=0,...10$.
The figure below illustrate the discrete domain.

```python
h=0.001
t=np.arange(1850,1925+h/2,h)
fig = plt.figure(1,figsize=(5,4))
ax = fig.add_subplot(1,1,1)
ax.plot(t,0*t,'o:',color='k')
ax.set_xlabel('Time (yrs)')
plt.title('Discrete Time Domain')
plt.show()
```

```python
N=len(t) # Number of Steps
H=np.zeros(N) # Numerical Solution Hares
H[0]=90
L=np.zeros(N) # Numerical Solution Lynxes
L[0]=35

```

We numerically approximate the differential equation using Eulers method to give the difference equation
\begin{equation}
H[i+1] = H[i]+ h*\big(a_{Birth}*H[i]-b_{Con}*H[i]*L[i] \big)
\end{equation}

\begin{equation}
L[i+1] = L[i]+ h*\big(-c_{Death}*H[i]+d_{Food}*H[i]*L[i] \big)
\end{equation}
where $h$ is the stepsize and where $P_i$ is the numerical approximation of $p(t_i)$ at distance $t_i$, for $i=0,...,N$

```python
a_Birth=0.4
b_Con=0.01
c_Death=0.09
d_Food=0.001

for i in range (0,N-1):
    H[i+1]=H[i]+h*(a_Birth-b_Con*L[i])*H[i]# INSERT EQUATION HERE
    L[i+1]=L[i]+h*(-c_Death+d_Food*H[i])*L[i]# INSERT EQUATION HERE

```

## Results
Figure 2 shows a simulation of the populations of salmon and small fish from 2000 to 2100, assuming that nothing is added to enviroment.  The salmon population fluxuates between 45 and 55 billion while the small fish population fluctuates from 25 to 85 billion. The equation illustates a relative equilibrium between the two populations ensuring neither get too big or too small.

```python
## Plotting Figure
fig = plt.figure(1,figsize=(10,4))

ax = fig.add_subplot(1,1,1)

ax.plot(t,H,':',color='green',label='Hares')
ax.plot(t,L,':',color='red',label='Lynxes')

ax.legend()
ax.set_xlabel('Time (yrs)')
ax.set_ylabel('Population (tens of thousands)')
#ax.legend(loc='best')
ax.set_title('Predator Prey Relationship')
plt.tight_layout()
plt.subplots_adjust(top=0.85)
plt.show()

```

## Phase Plane Plot

```python
## Plotting Figure
fig = plt.figure(1,figsize=(5,5))

ax = fig.add_subplot(1,1,1)

ax.plot(L,H,'-',color='black')

ax.set_xlabel('Lynx Population (tens of thousands)')
ax.set_ylabel('Hare Population (tens of thousands)')
#ax.legend(loc='best')
ax.set_title('Predator Prey Relationship')
plt.tight_layout()
plt.subplots_adjust(top=0.85)
plt.xlim(0.9*np.min(L),1.1*np.max(L))
plt.ylim(0.9*np.min(H),1.1*np.max(H))

plt.show()
```

## Equilibrium
The Loketta-Volterra model [4] can be written in matrix frorm
\begin{equation}
\begin{pmatrix}
\frac{dH}{dt}\\
\frac{dL}{dt}
\end{pmatrix}
=\begin{pmatrix}
a_{Birth} & -b_{Con}H \\
 d_{Food}L &-c_{Death}
\end{pmatrix}
\begin{pmatrix}
H \\
L
\end{pmatrix}.
\end{equation}
The point of equilibrium (stability) is when $\frac{dH}{dt}=0$ and $\frac{dL}{dt}=0$,
\begin{equation}
\begin{pmatrix}
0\\
0
\end{pmatrix}
=\begin{pmatrix}
a_{Birth} & -b_{Con}H \\
 d_{Food}L &-c_{Death}
\end{pmatrix}
\begin{pmatrix}
H \\
L
\end{pmatrix},
\end{equation}
this gives two solutions for $H$ and $L$:
1. the trivial solution is $H=0$ and $L=0$;
2. the non-trivial solution is $H=\frac{c_{Death}}{d_{Food}}$ and $L=\frac{a_{Birth}}{b_{Con}}$.

```python
## Plotting Figure
fig = plt.figure(1,figsize=(5,5))

ax = fig.add_subplot(1,1,1)

ax.plot(L,H,'-',color='black',label='Predator - Prey Cycle')

ax.plot(a_Birth/b_Con,c_Death/d_Food,'o',markersize=7,color='g',markeredgecolor='k',label='Stable Point')
ax.plot(0,0,'o',markersize=7,color='g',markeredgecolor='b',label='Trivial Stable Point Zero')

ax.set_xlabel('Lynx Population (tens of thousands)')
ax.set_ylabel('Hare Population (tens of thousands)')
#ax.legend(loc='best')
ax.set_title('Predator Prey Relationship')
plt.tight_layout()
plt.subplots_adjust(top=0.85)
plt.xlim(0.9*np.min(L),1.1*np.max(L))
plt.ylim(0.9*np.min(H),1.1*np.max(H))
plt.legend(bbox_to_anchor=(1.2, 1.0))
plt.show()
```

## Nullclines
The Nullclines are defined as curves when $\frac{dH}{dt}=0$ or $\frac{dH}{dt}=0$.
Here we have two simple nullcline;
1. For $\frac{dH}{dt}=0$, gives the vertical nullcline
$$L=\frac{a_{Birth}}{b_{Con}};$$
2. For $\frac{dL}{dt}=0$, gives the horizontal nullcline
$$H=\frac{a_{Birth}}{b_{Con}}.$$

The nullclines divides the graph into quadrantes:
1. The first quadrant (Q1) is when $\frac{dH}{dt}<0$ and $\frac{dL}{dt}>0$, the Hare population is decreasing while the Lynx population is increasing;
2. The second quadrant (Q2) is when $\frac{dH}{dt}>0$ and $\frac{dL}{dt}>0$, the Hare population and the Lynx population are increasing;

3. The third quadrant (Q3) is when $\frac{dH}{dt}>0$ and $\frac{dL}{dt}<0$, the Hare population is increasing while the Lynx population is decreasing;

4. The fourth quadrant (Q1) is when $\frac{dH}{dt}<0$ and $\frac{dL}{dt}<0$, the Hare and Lynx populations are decreasing.

```python
## Plotting Figure
fig = plt.figure(1,figsize=(5,5))

ax = fig.add_subplot(1,1,1)

ax.plot(L,H,'-',color='black',label='Predator - Prey Cycle')

ax.plot(a_Birth/b_Con,c_Death/d_Food,'o',markersize=7,color='g',markeredgecolor='k',label='Stable Point')
ax.plot(0,0,'o',markersize=7,color='g',markeredgecolor='b',label='Trivial Stable Point')

ax.plot(a_Birth/b_Con,c_Death/d_Food,'o',markersize=7,color='g',markeredgecolor='k',label='Stable Point Zero')

ax.axvline(a_Birth/b_Con,linewidth=4, color='r',label=r'nullcline $\frac{dH}{dt}=0$')

ax.axhline(c_Death/d_Food,linewidth=4, color='y',label=r'nullcline $\frac{dL}{dt}=0$')

ax.set_xlabel('Lynx Population (tens of thousands)')
ax.set_ylabel('Hare Population (tens of thousands)')
#ax.legend(loc='best')
ax.set_title('Predator Prey Relationship')
plt.tight_layout()
plt.subplots_adjust(top=0.85)
plt.text(np.min(L),np.min(H),r'Q1')
plt.text(np.max(L),np.min(H),r'Q2')
plt.text(np.max(L),np.max(H),r'Q3')
plt.text(np.min(L),np.max(H),r'Q4')
plt.xlim(0.9*np.min(L),1.1*np.max(L))
plt.ylim(0.9*np.min(H),1.1*np.max(H))

plt.legend(bbox_to_anchor=(1.1, 1.05))

plt.show()
```

## Phase Plane

The nullclines divides the graph into quadrantes:
1. The first quadrant (Q1) is when $\frac{dH}{dt}<0$ and $\frac{dL}{dt}>0$, the Hare population is decreasing while the Lynx population is increasing;
2. The second quadrant (Q2) is when $\frac{dH}{dt}>0$ and $\frac{dL}{dt}>0$, the Hare population and the Lynx population are increasing;

3. The third quadrant (Q3) is when $\frac{dH}{dt}>0$ and $\frac{dL}{dt}<0$, the Hare population is increasing while the Lynx population is decreasing;

4. The fourth quadrant (Q1) is when $\frac{dH}{dt}<0$ and $\frac{dL}{dt}<0$, the Hare and Lynx populations are decreasing.

```python
def dX_dt(H,L):
   # """ Return the growth rate of fox and rabbit populations. """
    return np.array([ a_Birth*H - b_Con*L*H ,
                     -c_Death*L + d_Food*L*H
    ])

nbpoints=20
Hsp = np.linspace(np.min(H), np.max(H), nbpoints)
Lsp = np.linspace(np.min(L), np.max(L), nbpoints)

L1 , H1  = np.meshgrid(Lsp, Hsp)                       # create a grid
DL1, DH1 = dX_dt(H1, L1)                      # compute growth rate on the gridt
M = (np.hypot(DL1, DH1))/(np.hypot(DL1, DH1))                        # Norm of the growth rate
M[ M == 0] = 1.                              # Avoid zero division errors
DL1 /= M                                        # Normalize each arrows
DH1 /= M

## Plotting Figure
fig = plt.figure(1,figsize=(5,5))

ax = fig.add_subplot(1,1,1)

ax.plot(L,H,'-',color='black',label='Predator - Prey Cycle')

ax.plot(a_Birth/b_Con,c_Death/d_Food,'o',markersize=7,color='g',markeredgecolor='k',label='Stable Point')
ax.plot(0,0,'o',markersize=7,color='g',markeredgecolor='b',label='Trivial Stable Point')

ax.plot(a_Birth/b_Con,c_Death/d_Food,'o',markersize=7,color='g',markeredgecolor='k',label='Stable Point')

ax.axvline(a_Birth/b_Con,linewidth=4, color='r',label=r'Isocline $\frac{dH}{dt}=0$')

ax.axhline(c_Death/d_Food,linewidth=4, color='y',label=r'Isocline $\frac{dL}{dt}=0$')

ax.set_xlabel('Lynx Population (tens of thousands)')
ax.set_ylabel('Hare Population (tens of thousands)')
#ax.legend(loc='best')
ax.set_title('Predator Prey Relationship')
plt.tight_layout()
plt.subplots_adjust(top=0.85)
plt.text(np.min(L),np.min(H),r'Q1')
plt.text(np.max(L),np.min(H),r'Q2')
plt.text(np.max(L),np.max(H),r'Q3')
plt.text(np.min(L),np.max(H),r'Q4')
plt.xlim(0.9*np.min(L),1.1*np.max(L))
plt.ylim(0.9*np.min(H),1.1*np.max(H))
plt.quiver(L1, H1, DH1, DL1, M, pivot='mid', cmap=plt.cm.jet)
plt.legend(bbox_to_anchor=(1.1, 1.05))
plt.show()
```

## Jacobian Matrix
The Loketta-Volterra model [4] can be written in matrix frorm
\begin{equation}
\begin{pmatrix}
\frac{dH}{dt}\\
\frac{dL}{dt}
\end{pmatrix}
=\begin{pmatrix}
a_{Birth} & -b_{Con}L \\
-c_{Death} & d_{Food}H
\end{pmatrix}
\begin{pmatrix}
H \\
L
\end{pmatrix}.
\end{equation}
The point of equilibrium (stability) is when $\frac{dH}{dt}=0$ and $\frac{dL}{dt}=0$,
\begin{equation}
\begin{pmatrix}
0\\
0
\end{pmatrix}
=\begin{pmatrix}
a_{Birth} & -b_{Con}L \\
-c_{Death} & d_{Food}H
\end{pmatrix}
\begin{pmatrix}
H \\
L
\end{pmatrix},
\end{equation}
this gives two solutions for $H$ and $L$:
1. the trivial solution is $H=0$ and $L=0$;
2. the non-trivial solution is $H=\frac{c_{Death}}{d_{Food}}$ and $L=\frac{a_{Birth}}{b_{Con}}$.

# References

[1] Stover, Christopher and Weisstein, Eric W. "Population Growth." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/PopulationGrowth.html

[2]  Vito Volterra.  Fluctuations in the abundance of a species considered mathematically. Nature,118:558–560,1926.

[3] Alfred J Lotka.   Analytical note on certain rhythmic  relations inorganic  systems.Proceedings of the National Academy of Sciences,6(7):410–415,1920.

[4] Strogatz, S.  Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering (studies in nonlinearity), Westview Press; 2 edition (29 July 2014)

```python

```

08_Snow Hares and Lynxes-Stability-Jacobian.ipynb


```python
import numpy
%matplotlib notebook
import matplotlib.pyplot
import sympy
```

# Elliptical Turns

## Overview

At an intersection, a car has to make a smooth turn between two road endpoints. To approximate this effect, I will attempt to fit an ellipse between the two roads. The ellipse must intersect at the endpoint of the intersection and the ellipse must also be tangential to the roads at those points.

## Defining Roads

We can define the roads parametrically in two dimensions. The general equation for a line from point $\langle x_a, y_a \rangle$ to $\langle x_b, y_b \rangle$ in parametric form is

$$ \ell(t) = \langle x_a + (x_b - x_a) t, y_a + (y_b - y_a) t \rangle \quad \forall t \in [0, 1] $$

In particular, we will focus on two perpendicular, linear roads. The first goes from $\langle 3, 2 \rangle$ to $\langle 3, 3 \rangle$. The second from $\langle 4, 4 \rangle$ to $\langle 5, 4 \rangle$. These are defined by the equations

$$ \ell_a(t) = \langle 3, 2 + t \rangle \quad \forall t \in [0, 1] $$
$$ \ell_b(t) = \langle 4 + t, 4 \rangle \quad \forall t \in [0, 1] $$

```python
# Each component must have a dependence on `t` for NumPy to broadcast correctly
l_a = lambda t: numpy.array([[3 + 0 * t], [2 + t]])
l_b = lambda t: numpy.array([[4 + t], [4 + 0 * t]])
```

Now we can visualize the problem a little better by plotting the roads.

```python
fig, ax = matplotlib.pyplot.subplots()

t_a = t_b = numpy.linspace(0, 1, 100)

ax.plot(l_a(t_a)[0].flatten(), l_a(t_a)[1].flatten(), 'r')
ax.plot(l_b(t_b)[0].flatten(), l_b(t_b)[1].flatten(), 'b')

matplotlib.pyplot.show()
```

    <IPython.core.display.Javascript object>

We also need to determine the partial derivatives of these roads. The general equation for the partial derivatives of a line from point $\langle x_a, y_a \rangle$ and $\langle x_b, y_b \rangle$ in parametric form is

$$ \frac{\partial \ell(t)}{\partial t} = \langle x_b - x_a, y_b - y_a \rangle \quad \forall t \in [0, 1] $$

For this example case in particular, the partial derivatives evaluate to

$$ \frac{\partial \ell_a(t)}{\partial t} = \langle 0, 1 \rangle \quad \forall t \in [0, 1] $$
$$ \frac{\partial \ell_b(t)}{\partial t} = \langle 1, 0 \rangle \quad \forall t \in [0, 1] $$

```python
dl_adt = lambda t: numpy.array([[0], [1]])
dl_bdt = lambda t: numpy.array([[1], [0]])
```

## Defining Constraints

With the road equations defined, we need to determine what constraints to apply to a general ellipse equation so it fits our needs. The general equation for an ellipse in parametric form is

$$ e(t) = \langle h + a \cos t, k + b \sin t \rangle \quad \forall t \in [0, 2 \pi) $$

```python
ellipse = lambda t, a, b, h, k: numpy.array([
    [h + a * numpy.sin(t)],
    [k + b * numpy.cos(t)]
])
```

We know that we want it to intersect the roads at their endpoints $\langle x_a, y_a \rangle$ and $\langle x_b, y_b \rangle$. Therefore, one set of constraints on the ellipse must be

$$ \langle h + a \cos t_a, k + b \sin t_a \rangle = \langle x_a, y_a \rangle $$
$$ \langle h + a \cos t_b, k + b \sin t_b \rangle = \langle x_b, y_b \rangle $$

Furthermore, the ellipse must be tangential to the roads that the same points $\langle x_a, y_a \rangle$ and $\langle x_b, y_b \rangle$. To determine this, the derivatives of the ellipse are needed. They are

$$ \frac{\partial e}{\partial t}(t) = \langle - a \sin t, b \cos t \rangle \quad \forall t \in [0, 2\pi) $$

From this, the final set of constraints follows from Lagrangian multipliers

$$ \langle -a \sin t_a, b \cos t_a \rangle = \lambda \frac{\partial \ell_a(t_a)}{\partial t} = \lambda \left\langle \frac{\partial \ell_a(t_a)}{\partial x}, \frac{\partial \ell_a(t_a)}{\partial y} \right\rangle $$
$$ \langle -a \sin t_b, b \cos t_b \rangle = \mu \frac{\partial \ell_b(t_b)}{\partial t} = \mu \left\langle \frac{\partial \ell_b(t_b)}{\partial x}, \frac{\partial \ell_b(t_b)}{\partial y} \right\rangle$$

## Solving

While it might be possible to solve these eight equations analytically, we will save ourselves some algebra and use numerical methods. To do so, we have to stage our problem a little differently: We are going to determine a root of a vector function using Newton's Method.

The vector function comes directly from our constraints determined above, but offset to equal zero. Let the vector $\vec{x}$ be equal to

$$ \vec{x} = \left\langle a, b, h, k, t_a, t_b, \lambda, \mu, x_a, y_a, x_b, y_b, \frac{\partial \ell_a(t_a)}{\partial x}, \frac{\partial \ell_a(t_a)}{\partial y}, \frac{\partial \ell_b(t_b)}{\partial x}, \frac{\partial \ell_b(t_b)}{\partial y} \right\rangle $$

Then the vector function $\vec{F}(\vec{x})$, from the constraints above, is defined to be

$$ \vec{F}(\vec{x}) = \begin{pmatrix}
    x_2 + x_0 \cos x_4 - x_8 \\
    x_3 + x_1 \sin x_4 - x_9 \\
    x_2 + x_0 \cos x_5 - x_{10} \\
    x_3 + x_1 \sin x_5 - x_{11} \\
    -x_0 \sin x_4 - x_6 x_{12} \\
    x_1 \cos x_4 - x_6 x_{13} \\
    -x_0 \sin x_5 - x_7 x_{14} \\
    x_1 \cos x_5 - x_7 x_{15}
\end{pmatrix} $$

```python
F = lambda x: numpy.array([
    x[2] + x[0] * numpy.cos(x[4]) - x[8],
    x[3] + x[1] * numpy.sin(x[4]) - x[9],
    x[2] + x[0] * numpy.cos(x[5]) - x[10],
    x[3] + x[1] * numpy.sin(x[5]) - x[11],
    -x[0] * numpy.sin(x[4]) - x[6] * x[12],
    x[1] * numpy.cos(x[4]) - x[6] * x[13],
    -x[0] * numpy.sin(x[5]) - x[7] * x[14],
    x[1] * numpy.cos(x[5]) - x[7] * x[15]
])
```

To apply Newton's Method, the Jacobian matrix is needed. I would much prefer not to take 64 derivatives, so we will use SymPy to analytically determine the matrix so we don't have to compute it each time.

```python
x0, x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, x11, x12, x13, x14, x15 = sympy.symbols('x:16')

sympy.Matrix([
    x2 + x0 * sympy.cos(x4) - x8,
    x3 + x1 * sympy.sin(x4) - x9,
    x2 + x0 * sympy.cos(x5) - x10,
    x3 + x1 * sympy.sin(x5) - x11,
    -x0 * sympy.sin(x4) - x6 * x12,
    x1 * sympy.cos(x4) - x6 * x13,
    -x0 * sympy.sin(x5) - x7 * x14,
    x1 * sympy.cos(x5) - x7 * x15
]).jacobian(sympy.Matrix([x0, x1, x2, x3, x4, x5, x6, x7]))
```

    Matrix([
    [ cos(x4),       0, 1, 0, -x0*sin(x4),           0,    0,    0],
    [       0, sin(x4), 0, 1,  x1*cos(x4),           0,    0,    0],
    [ cos(x5),       0, 1, 0,           0, -x0*sin(x5),    0,    0],
    [       0, sin(x5), 0, 1,           0,  x1*cos(x5),    0,    0],
    [-sin(x4),       0, 0, 0, -x0*cos(x4),           0, -x12,    0],
    [       0, cos(x4), 0, 0, -x1*sin(x4),           0, -x13,    0],
    [-sin(x5),       0, 0, 0,           0, -x0*cos(x5),    0, -x14],
    [       0, cos(x5), 0, 0,           0, -x1*sin(x5),    0, -x15]])

Using the SymPy output, we can copy the Jacobian matrix. I think you could use `sympy.lambdify` here instead, but I decided to copy it by hand.

$$ J(\vec{x}) = \begin{pmatrix}
    \cos x_4 & 0 & 1 & 0 & -x_0 \sin x_4 & 0 & 0 & 0 \\
    0 & \sin x_4 & 0 & 1 & x_1 \cos x_4 & 0 & 0 & 0 \\
    \cos x_5 & 0 & 1 & 0 & 0 & -x_0 \sin x_5 & 0 & 0 \\
    0 & \sin x_5 & 0 & 1 & 0 & x_1 \cos x_5 & 0 & 0 \\
    -\sin x_4 & 0 & 0 & 0 & -x_0 \cos x_4 & 0 & -x_{12} & 0 \\
    0 & \cos x_4 & 0 & 0 & -x_1 \sin x_4 & 0 & -x_{13} & 0 \\
    -\sin x_5 & 0 & 0 & 0 & 0 & -x_0 \cos x_5 & 0 & -x_{14} \\
    0 & \cos x_5 & 0 & 0 & 0 & -x_1 \sin x_5 & 0 & -x_{15}
\end{pmatrix} $$

```python
J = lambda x: numpy.array([
    [  numpy.cos(x[4]),               0, 1, 0, -x[0] * numpy.sin(x[4]),                       0,      0,      0 ],
    [                0, numpy.sin(x[4]), 0, 1,  x[1] * numpy.cos(x[4]),                       0,      0,      0 ],
    [  numpy.cos(x[5]),               0, 1, 0,                       0, -x[0] * numpy.sin(x[5]),      0,      0 ],
    [                0, numpy.sin(x[5]), 0, 1,                       0,  x[1] * numpy.cos(x[5]),      0,      0 ],
    [ -numpy.sin(x[4]),               0, 0, 0, -x[0] * numpy.cos(x[4]),                       0, -x[12],      0 ],
    [                0, numpy.cos(x[4]), 0, 0, -x[1] * numpy.sin(x[4]),                       0, -x[13],      0 ],
    [ -numpy.sin(x[5]),               0, 0, 0,                       0, -x[0] * numpy.cos(x[5]),      0, -x[14] ],
    [                0, numpy.cos(x[5]), 0, 0,                       0, -x[1] * numpy.sin(x[5]),      0, -x[15] ]
]).astype(numpy.float64)  # This will remove the nested np.arrays
```

Now we are ready to apply Newton's Method. The steps for solving a root problem in multiple dimensions are

1. Initialize an $\vec{x}_0$ to a point close to the desired root. In our case, a vector selected from the Gaussian distribution will do well most of the time.
2. Calculate $J(\vec{x}_k)$ and $\vec{F}(\vec{x}_k)$.
3. Solve the linear system of equations $J(\vec{x}_k) \vec{y}_k = - \vec{F}(\vec{x}_k)$ for $\vec{y}_k$ using Gaussian elimination (or appropriate method).
4. Calculate $\vec{x}_{k + 1} = \vec{x}_k + \vec{y}_k$.
5. Repeat until $\lVert \vec{x}_{k + 1} - \vec{x}_k \rVert < \varepsilon$.

```python
def newtons_method(F, J, x_0, epsilon):

    xs, Js, Fs, ys = [x_0], [], [], []

    do = True
    while do:

        Js.append(J(xs[-1]))
        Fs.append(F(xs[-1]))
        xs.append(xs[-1] + numpy.linalg.solve(Js[-1], -Fs[-1]))

        do = numpy.linalg.norm(xs[-1] - xs[-2]) >= epsilon

    return xs, Js, Fs
```

It is important to realize that our vector function depends on several constants that we are not trying to solve for. To handle this, we will pass lambda functions into Newton's method to concatenate the constants.

```python
consts = numpy.vstack([l_a(1), l_b(0), dl_adt(1), dl_bdt(0)])
x_0 = numpy.random.randn(8, 1)

xs, Js, Fs = newtons_method(lambda x: F(numpy.vstack([x, consts])), lambda x: J(numpy.vstack([x, consts])), x_0, 0.01)

numpy.around(xs[-1], 3)
```

    array([[ 1.000000e+00],
           [-1.000000e+00],
           [ 4.000000e+00],
           [ 3.000000e+00],
           [ 4.853761e+03],
           [ 4.849048e+03],
           [ 1.000000e+00],
           [ 1.000000e+00]])

Now we have a solution to our system of equations. Let's plot it to confirm that it is in fact a solution.

```python
fig, ax = matplotlib.pyplot.subplots()

t_a = t_b = numpy.linspace(0, 1, 100)
t_e = numpy.linspace(0, 2 * numpy.pi, 100)

ax.plot(l_a(t_a)[0].flatten(), l_a(t_a)[1].flatten(), 'r')
ax.plot(l_b(t_b)[0].flatten(), l_b(t_b)[1].flatten(), 'b')

a, b, h, k = xs[-1][0], xs[-1][1], xs[-1][2], xs[-1][3]

ax.plot(ellipse(t_e, a, b, h, k)[0].flatten(), ellipse(t_e, a, b, h, k)[1].flatten(), 'g')

matplotlib.pyplot.show()
```

    <IPython.core.display.Javascript object>

Perfect! There is an ellipse that smoothly connects these two roads.

## Issues

Unfortunately, this doesn't always work as smoothly. There are many ways to form roads that can not have an ellipse connecting them, even though a smooth turn does exist.

Let's now consider the roads defined by the fuctions

$$ \ell_a(t) = \left\langle 3 + \frac{t}{2}, \frac{5}{2} + t \right\rangle \quad \forall t \in [0, 1] $$
$$ \ell_b(t) = \langle 4 + t, 4 \rangle \quad \forall t \in [0, 1] $$

```python
l_a = lambda t: numpy.array([[3 + t / 2], [2.5 + t]])
l_b = lambda t: numpy.array([[4 + t], [4 + 0 * t]])
```

The partial derivatives of these roads are easily determined by hand

$$ \frac{\partial \ell_a(t)}{\partial t} = \left\langle \frac{1}{2}, 1 \right\rangle \quad \forall t \in [0, 1] $$
$$ \frac{\partial \ell_b(t)}{\partial t} = \langle 1, 0 \rangle \quad \forall t \in [0, 1] $$

```python
dl_adt = lambda t: numpy.array([[1/2], [1]])
dl_bdt = lambda t: numpy.array([[1], [0]])
```

Let's plot them to see if this is a meaningful intersection.

```python
fig, ax = matplotlib.pyplot.subplots()

t_a = t_b = numpy.linspace(0, 1, 100)

ax.plot(l_a(t_a)[0].flatten(), l_a(t_a)[1].flatten(), 'r')
ax.plot(l_b(t_b)[0].flatten(), l_b(t_b)[1].flatten(), 'b')

matplotlib.pyplot.show()
```

    <IPython.core.display.Javascript object>

Clearly, a smooth turn is possible between these two roads. Let's see if our method will determine it.

```python
consts = numpy.vstack([l_a(1), l_b(0), dl_adt(1), dl_bdt(0)])
x_0 = numpy.random.randn(8, 1)

xs, Js, Fs = newtons_method(lambda x: F(numpy.vstack([x, consts])), lambda x: J(numpy.vstack([x, consts])), x_0, 0.01)

numpy.around(xs[-1], 3)
```

Nope. Newton's method fails to converge. Let's confirm that moving the endpoint a little farther down will yield convergence. Instead of ending at $t_a = 1$, we will continue to $t_a = 1.01$.

```python
consts = numpy.vstack([l_a(1.01), l_b(0), dl_adt(1.01), dl_bdt(0)])
x_0 = numpy.random.randn(8, 1)

xs, Js, Fs = newtons_method(lambda x: F(numpy.vstack([x, consts])), lambda x: J(numpy.vstack([x, consts])), x_0, 0.01)

numpy.around(xs[-1], 3)
```

    array([[  2.487],
           [ 24.5  ],
           [  4.   ],
           [-20.5  ],
           [  1.771],
           [ -4.712],
           [ -4.875],
           [ -2.487]])

This time, Newton's method converged. Let's plot the solution to confirm its legitimacy.

```python
fig, ax = matplotlib.pyplot.subplots()

t_a = numpy.linspace(0, 1.01, 100)
t_b = numpy.linspace(0, 1, 100)
t_e = numpy.linspace(0, 2 * numpy.pi, 1000)

ax.plot(l_a(t_a)[0].flatten(), l_a(t_a)[1].flatten(), 'r')
ax.plot(l_b(t_b)[0].flatten(), l_b(t_b)[1].flatten(), 'b')

a, b, h, k = xs[-1][0], xs[-1][1], xs[-1][2], xs[-1][3]

ax.plot(ellipse(t_e, a, b, h, k)[0].flatten(), ellipse(t_e, a, b, h, k)[1].flatten(), 'g')

matplotlib.pyplot.show()
```

    <IPython.core.display.Javascript object>

That certainly looks like a valid curve.

## Conclusion

This is unfortunate information. In order to apply this method, carefully selected endpoints must be chosen to yield convergence. However, I am not aware of a simple way to detect this issue or its fix. Thus, I will not be using this method.


# Mine management model

**Randall Romero Aguilar, PhD**

This demo is based on the original Matlab demo accompanying the  Computational Economics and Finance 2001 textbook by Mario Miranda and Paul Fackler.

Original (Matlab) CompEcon file: **demddp03.m**

Running this file requires the Python version of CompEcon. This can be installed with pip by running

    !pip install compecon --upgrade

Last updated: 2021-Oct-01

## About

A mine operator must decide how much ore to extract from a mine that will be shut down and abandoned after $T$ years of operation. The price of extracted ore is $p$ dollars per ton, and the total cost of extracting $x$ tons of ore in any year, given that the mine contains $s$ tons at the beginning of the year, is $c(s, x)$ dollars. The mine currently contains $\bar{s}$ tons of ore. Assuming the amount of ore extracted in any year must be an integer number of tons, what extraction schedule maximizes profits?

## Initial tasks

```python
import numpy as np
import pandas as pd
from compecon import DDPmodel, getindex
```

## Model Parameters

Assuming a market price $p=1$, initial stock of ore $\bar{s}=100$, and annual discount factor $\delta = 0.9$

```python
price = 1
sbar  = 100
delta = 0.9
```

### State Space

This is a finite horizon, deterministic model with time $t$ measured in years. The state
variable $s \in \{0, 1, 2, \dots, \bar{s}\}$ is the amount of ore remaining in the mine at the beginning of the year, measured in tons.

```python
S = np.arange(sbar + 1)      # vector of states
n = S.size                   # number of states
```

### Action Space

The action variable $x \in \{0, 1, 2, \dots, s\}$ is the amount of ore extracted over the year, measured in tons.

```python
X = np.arange(sbar + 1)      # vector of actions
m = X.size                   # number of actions
```

### Reward Function

The reward function is $f(s, x) = px − c(s, x)$. The cost of extraction is $c(s, x) = \frac{x^2}{1+s}$.

Here, the reward is set to negative infinity if the extraction level exceeds the available stock in order to preclude the choice of an infeasible
action:

```python
f = np.full((m, n), -np.inf)
for c, s in enumerate(S):
    for r, x in enumerate(X):
        if x <= s:
            f[r, c] = price * x - (x ** 2) / (1 + s)
```

### State Transition Function

The state transition function is $g(s, x) = s − x$

Here, the routine `getindex` is used to find the index of the following period’s state.

```python
g = np.empty_like(f)
for r, x in enumerate(X):
    snext = S - x
    g[r] = getindex(snext, S)
```

## Model Structure

The value of the mine, given that it contains $s$ tons of ore at the beginning of year $t$, satisfies the Bellman equation

\begin{equation} V_t(s) = max_{x\in\{0,1,\dots,s\}} \left\{px−c(s, x) + \delta V_{t+1}(s−x)\right\} \end{equation}

subject to the terminal condition $V_{T+1}(s) = 0$

To solve and simulate this model, use the CompEcon class ```DDPmodel```.

```python
model = DDPmodel(f, g, delta)
model.solve()
```

    A deterministic discrete state, discrete action, dynamic model.
    There are 101 possible actions over 101 possible states

By default, the `solve()` method uses Newton's algorithm. This and other default settings can be changed when solving the model. For example,
```python
model.solve(algorithm='funcit', print=True)
```
solves the model by function iteration, printing a summary of each iteration to screen.

In either case, `solve()` updates the model itself, storing the $n$ vector of values `.value`, the $n$ vector of optimal actions `.policy`, and the $n\times n$ controlled state `.transition` probability.

```python
solution = pd.DataFrame({
    'Stock': S,
    'Extraction': X[model.policy],
    'Value': model.value}).set_index('Stock')
```

## Analysis

### Simulate Model
The path followed by the stock level over time is computed by the `simulate()` method. Here, the simulation assumes an initial stock level of 100 and 15 years in duration.

```python
sinit = S.max()
nyrs = 15
t = np.arange(nyrs + 1)
spath, xpath = model.simulate(sinit, nyrs)

simul = pd.DataFrame({
    'Year': t,
    'Stock': S[spath],
    'Extraction': X[xpath]}).set_index('Year')
simul
```

      Stock
      Extraction

      Year

      0
      100
      24

      1
      76
      18

      2
      58
      14

      3
      44
      11

      4
      33
      8

      5
      25
      6

      6
      19
      5

      7
      14
      3

      8
      11
      3

      9
      8
      2

      10
      6
      2

      11
      4
      1

      12
      3
      1

      13
      2
      1

      14
      1
      1

      15
      0
      0

### Plot Optimal Policy

```python
ax = solution['Extraction'].plot(title='Optimal Extraction Policy')
ax.set(ylabel='Extraction');
```

### Plot Value Function
The value of the firm is very nearly proportional to the stock level.

```python
ax = solution['Value'].plot(title='Optimal Value Function')
ax.set(ylabel='Value');
```

### Plot State and Extraction Path
As seen in this figure, the content of the mine is optimally exhausted in 15 years.

```python
ax = simul.plot(subplots=True, layout=[1,2], legend=None)
for i, lab in enumerate(simul.columns):
    ax[0, i].set(ylabel=lab);
```

_build/html/_sources/notebooks/ddp/01 Mine management model.ipynb


# Surge hull equation

```python
%load_ext autoreload
%autoreload 2
%matplotlib inline
```

```python
import sympy as sp
from sympy.plotting import plot as plot
from sympy.plotting import plot3d as plot3d
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
sp.init_printing()
from IPython.core.display import HTML
```

```python
import seaman.helpers
import seaman_symbol as ss
import surge_hull_equations as equations
import surge_hull_lambda_functions as lambda_functions
from bis_system import BisSystem
```

## Coordinate system

## Symbols

```python
from seaman_symbols import *
```

```python
HTML(ss.create_html_table(symbols=equations.surge_hull_equation_SI.free_symbols))
```

## Surge equation

```python
sp.latex(equations.surge_hull_equation)
```

```python
equations.surge_hull_equation_SI
```

### Plotting the total surge hull equation

```python
df = pd.DataFrame()
df['v_w'] = np.linspace(-0.3,0.3,20)
df['r_w'] = 0.0
df['rho'] = 1025
df['L'] = 1.0
df['g'] = 9.81
df['X_vv'] = -1.0
df['X_vr'] = 1.0
df['X_rr'] = 1.0
df['X_res'] = 0.0
df['disp'] = 23

result = df.copy()
result['fx'] = lambda_functions.X_h_function(**df)

result.plot(x = 'v_w',y = 'fx');
```

### Plotting with coefficients from a real seaman ship model

```python
import generate_input
shipdict = seaman.ShipDict.load('../../tests/test_ship.ship')
```

```python
df = pd.DataFrame()
df['v_w'] = np.linspace(-0.3,0.3,20)
df['r_w'] = 0.0
df['rho'] = 1025
df['g'] = 9.81
df['X_res'] = 0.0

df_input = generate_input.add_shipdict_inputs(lambda_function=lambda_functions.X_h_function,
                                              shipdict = shipdict,
                                              df = df)
df_input
```

```python
result = df_input.copy()
result['fx'] = lambda_functions.X_h_function(**df_input)
```

```python
result.plot(x = 'v_w',y = 'fx');
```

## Real seaman++
Run real seaman in C++ to verify that the documented model is correct.

```python
import run_real_seaman
```

```python
df = pd.DataFrame()
df['v_w'] = np.linspace(-0.3,0.3,20)
df['r_w'] = 0.0
df['rho'] = 1025
df['g'] = 9.81
df['X_res'] = 0.0

result_comparison = run_real_seaman.compare_with_seaman(lambda_function=lambda_functions.X_h_function,
                                                        shipdict = shipdict,
                                                        df = df,label='fx',
                                                        seaman_function=run_real_seaman.calculate_resistance)

fig,ax = plt.subplots()
result_comparison.plot(x = 'v_w',y = ['fx','fx_seaman'],ax = ax)
ax.set_title('Drift angle variation');
```

```python
df = pd.DataFrame()
df['r_w'] = np.linspace(-0.3,0.3,20)
df['v_w'] = 0.0
df['rho'] = 1025
df['g'] = 9.81
df['X_res'] = 0.0

shipdict2 = shipdict.copy()
shipdict2.resistance_data['xrr'] = -0.01

result_comparison = run_real_seaman.compare_with_seaman(lambda_function=lambda_functions.X_h_function,
                                                        shipdict = shipdict2,
                                                        df = df,label='fx',
                                                        seaman_function=run_real_seaman.calculate_resistance)

fig,ax = plt.subplots()
result_comparison.plot(x = 'r_w',y = ['fx','fx_seaman'],ax = ax)
ax.set_title('Yaw rate variation');
```

```python
df = pd.DataFrame()
df['v_w'] = np.linspace(-0.3,0.3,20)
df['r_w'] = 0.01
df['rho'] = 1025
df['g'] = 9.81
df['X_res'] = 0.0

result_comparison = run_real_seaman.compare_with_seaman(lambda_function=lambda_functions.X_h_function,
                                                        shipdict = shipdict,
                                                        df = df,label='fx',
                                                        seaman_function=run_real_seaman.calculate_resistance)

fig,ax = plt.subplots()
result_comparison.plot(x = 'v_w',y = ['fx','fx_seaman'],ax = ax)
ax.set_title('Drift angle variation, yaw rate = 0.01 rad/s');
```

```python
df = pd.DataFrame()
df['r_w'] = np.linspace(-0.3,0.3,20)
df['v_w'] = 0.01
df['rho'] = 1025
df['g'] = 9.81
df['X_res'] = 0.0

result_comparison = run_real_seaman.compare_with_seaman(lambda_function=lambda_functions.X_h_function,
                                                        shipdict = shipdict,
                                                        df = df,label='fx',
                                                        seaman_function=run_real_seaman.calculate_resistance)

fig,ax = plt.subplots()
result_comparison.plot(x = 'r_w',y = ['fx','fx_seaman'],ax = ax)
ax.set_title('Yaw rate variation');
```

```python
import save_lambda_functions

function_name = 'X_h_function'
lambda_function = lambda_functions.X_h_function

save_lambda_functions.save_lambda_to_python_file(lambda_function = lambda_function,
                                                 function_name = function_name)

save_lambda_functions.save_lambda_to_matlab_file(lambda_function = lambda_function,
                                                 function_name = function_name)
```


# Taylor series comparison
a) Finden Sie heraus ab welcher Ordnung der Approximationsfehler der Taylorreihe von `exp(x)` um `x0 = 0` an der Stelle `x = 3` kleiner als 10^−4 ist. Taylorreihenentwicklungen bekommen Sie in sympy via `f.series(x,x0,Ordnung)`; verwenden Sie `.removeO()` um das Landau Symbol aus den Entwicklungen zu entfernen und diese als Funktion auswerten zu können. Speichern Sie die Approximationsfehler verschiedener Ordnungen in eine Liste `ep` und wählen Sie eine geeignete Kontrollstruktur um die gesuchte Ordnung zu bestimmen. Geben Sie außerdem die Liste `ep` aus.
b) Plotten Sie in einer Figur mit Wertebereich [−1,10]) die Ausgangsfunktion 𝑓 sowie die Taylorreihenentwicklungen der Ordnungen 5,10 und 15 mit verschiedenen Farben (zB line_color = 'r'). Die Ausgabe einzelner Plots kann mit `show = False` unterdrückt werden und mit `p1.extend(p2)` etc können Plots zusammengefügt werden. p1.show() gibt dann den Gesamtplot aus.

```python
from sympy import Symbol
from sympy.functions import exp

def taylor_expansion(function, x, x0, order):
    return function.series(x, x0, order).removeO()

def taylor_approximation(point, x0, max_order = 25, epsilon = 1e-4):
    ep = []
    x = Symbol('x')
    e_x = exp(x).evalf(subs = {'x': point})
    limit_order = 0
    for order in range(max_order):
        expansion = taylor_expansion(exp(x), x, x0, order)
        ep.append(
            abs(expansion.evalf(subs = {'x': point}) - e_x)
        )

        if not limit_order and ep[-1] <= epsilon:
            limit_order = order

    return ep, limit_order

ep, order = taylor_approximation(3, 0)
print(f'The order of a taylor expansion of exp(x) with a deviation from the true value smaller or equal to 10^-4 is: {order}')
for i, deviation in enumerate(ep):
    print(f'{i}: {deviation}')
```

    The order of a taylor expansion of exp(x) with a deviation from the true value smaller or equal to 10^-4 is: 14
    0: 20.0855369231877
    1: 19.0855369231877
    2: 16.0855369231877
    3: 11.5855369231877
    4: 7.08553692318767
    5: 3.71053692318767
    6: 1.68553692318767
    7: 0.673036923187667
    8: 0.239108351759096
    9: 0.0763851374733804
    10: 0.0221440660448096
    11: 0.00587174461623974
    12: 0.00143383877208336
    13: 0.000324362311044268
    14: 0.0000683292815750747
    15: 0.0000134650609737719
    16: 0.00000249221685422185
    17: 4.34808580251911E-7
    18: 7.17365331581732E-8
    19: 1.12245253092169E-8
    20: 1.66999569728432E-9
    21: 2.36816788401484E-10
    22: 3.20774518058897E-11
    23: 4.16022771787539E-12
    24: 5.18696197104873E-13

```python
from sympy.plotting import plot
p = plot(
    xlim = (-1, 10),
#    ylim = (-100, 5000),
    legend = True,
    loc = 'best',
    show = False
)
for order in [5, 10, 15]:
    x = Symbol('x')
    expansion = taylor_expansion(exp(x), x, 0, order)
    p.extend(
        plot(
            expansion,
            label = f'order = {order}',
            show = False)
    )
p.show()
```

# System of equations
Seien
$$f(x, y) = x^3 + 3x^2y - x^2 + 3xy^2 - 2xy + y^3 - y^2$$

$$g(x, y) = x^2 + y^2$$

Geben Sie zuerst $f$ vereinfacht (faktorisiert) aus. Lösen Sie dann das Gleichungssystem $f(x, y) = 0, g(x, y) = 0$ und geben Sie die Lösungsmenge aus. Stellen Sie weiters die zweiten partiellen Ableitungen in der Hesse-Matrix von $f$ dar und geben Sie diese aus. Die Hesse-Matrix H ist gegeben durch

$$
  H =
  \begin{pmatrix}
    f_{xx} & f_{xy}\\
    f_{yx} & f_{yy}\\
  \end{pmatrix}
$$

wobei $f_{xx}$ einfach bedeutet, dass die Ableitung von $f$ nach $x$ nochmals nach $x$ abgeleitet wird (also die 2. Ableitung nach 𝑥 gebildet wird), $f_{xy}$ dass die 1.Ableitung von $f$ nach $x$ danach nach $y$ abgeleitet wird, etc.

```python
from sympy.abc import x, y
f = x*x*x + 3*x*x*y - x*x + 3*x*y*y - 2*x*y + y*y*y - y*y
g = x*x + y*y
f.factor()
```

$\displaystyle \left(x + y\right)^{2} \left(x + y - 1\right)$

```python
from sympy import solve
solve([f, g], [x, y])
```

    [(0, 0), (1/2 - I/2, 1/2 + I/2), (1/2 + I/2, 1/2 - I/2)]

```python
from sympy import Matrix
H = Matrix(
    [
        [f.diff(x).diff(x), f.diff(x).diff(y)],
        [f.diff(y).diff(x), f.diff(y).diff(y)]
    ]
)
H
```

$\displaystyle \left[\begin{matrix}6 x + 6 y - 2 & 6 x + 6 y - 2\\6 x + 6 y - 2 & 6 x + 6 y - 2\end{matrix}\right]$

# Toeplitz matrix
a) Eine $n \times n$ tridiagonale Toeplitz-Matrix hat die Form
$$
  m =
  \begin{pmatrix}
    a & c & 0 & \cdots & 0 \\
    b & a & c & \ddots & \vdots \\
    0 & b & \ddots & \ddots & 0 \\
    \vdots & \ddots & \ddots & \ddots & c \\
    0 & \cdots & 0 & b & a \\
  \end{pmatrix}
$$

Schreiben Sie eine Funktion, die eine solche $n \times n$ Matrix für eine beliebige Größe $n$ und Variablen $a, b, c$ zurückgibt. Geben Sie die Matrix für verschiedene $n$ aus und vergleichen Sie die Standardausgabe mit jener Ausgabe, die Sie erhalten, wenn Sie display() aus dem Modul IPython.display verwenden.

```python
from sympy import Matrix
from sympy.abc import a, b, c
from IPython.display import display

def toeplitz_matrix(n):
    m = [
        [a, c] + [0] * (n - 2)
    ]
    for i in range(n - 2):
        m.append(
            [0] * i + [b, a, c] + [0] * (n - i - 3)
        )

    m.append(
        [0] * (n - 2) + [b, a]
    )
    return Matrix(m)

for n in range(4, 9, 2):
    print(f'{n} x {n} Toeplitz matrix')
    m = toeplitz_matrix(n)
    print(m)
    display(m)
    print()
```

    4 x 4 Toeplitz matrix
    Matrix([[a, c, 0, 0], [b, a, c, 0], [0, b, a, c], [0, 0, b, a]])

$\displaystyle \left[\begin{matrix}a & c & 0 & 0\\b & a & c & 0\\0 & b & a & c\\0 & 0 & b & a\end{matrix}\right]$

    6 x 6 Toeplitz matrix
    Matrix([[a, c, 0, 0, 0, 0], [b, a, c, 0, 0, 0], [0, b, a, c, 0, 0], [0, 0, b, a, c, 0], [0, 0, 0, b, a, c], [0, 0, 0, 0, b, a]])

$\displaystyle \left[\begin{matrix}a & c & 0 & 0 & 0 & 0\\b & a & c & 0 & 0 & 0\\0 & b & a & c & 0 & 0\\0 & 0 & b & a & c & 0\\0 & 0 & 0 & b & a & c\\0 & 0 & 0 & 0 & b & a\end{matrix}\right]$

    8 x 8 Toeplitz matrix
    Matrix([[a, c, 0, 0, 0, 0, 0, 0], [b, a, c, 0, 0, 0, 0, 0], [0, b, a, c, 0, 0, 0, 0], [0, 0, b, a, c, 0, 0, 0], [0, 0, 0, b, a, c, 0, 0], [0, 0, 0, 0, b, a, c, 0], [0, 0, 0, 0, 0, b, a, c], [0, 0, 0, 0, 0, 0, b, a]])

$\displaystyle \left[\begin{matrix}a & c & 0 & 0 & 0 & 0 & 0 & 0\\b & a & c & 0 & 0 & 0 & 0 & 0\\0 & b & a & c & 0 & 0 & 0 & 0\\0 & 0 & b & a & c & 0 & 0 & 0\\0 & 0 & 0 & b & a & c & 0 & 0\\0 & 0 & 0 & 0 & b & a & c & 0\\0 & 0 & 0 & 0 & 0 & b & a & c\\0 & 0 & 0 & 0 & 0 & 0 & b & a\end{matrix}\right]$

b) Geben Sie nun eine tridiagonale Toeplitz-Matrix der Größe $n = 5$ mit Zahlenwerten aus, indem Sie eine allgemeine Toeplitz-Matrix erstellen und konkrete Werte für die Symbole übergeben: $a, b, c$ sollen hier auf zwei Nachkommastellen gerundete Zufallszahlen aus [−1,1] sein. Erstellen Sie dann eine Funktion, die eine Liste mit den Eigenwerten ohne algebraische Vielfachheit zurückgibt. Geben Sie dann die Eigenwerte für unterschiedlichen Fälle von $b$ und $c$ aus, wobei letztendlich unterschieden werden soll, ob $b*c \geq 0$ ist. Was fällt Ihnen auf?

```python
import random
m = toeplitz_matrix(5)
m.evalf(
    subs = {
        var: round(random.uniform(-1, 1), 2) for var in 'abc'
    }
)
```

$\displaystyle \left[\begin{matrix}-0.22 & 0.44 & 0 & 0 & 0\\-0.59 & -0.22 & 0.44 & 0 & 0\\0 & -0.59 & -0.22 & 0.44 & 0\\0 & 0 & -0.59 & -0.22 & 0.44\\0 & 0 & 0 & -0.59 & -0.22\end{matrix}\right]$

```python
def eigenvalues(m):
    return [eigval for eigval, mult in m.eigenvals().items()]

subs = {
    key1: {key2: val for key2, val in zip('abc', vals)}
    for key1, vals in zip(['geq0', 'leq0'], [[0.54, 0.26, 0.80], [0.54, -0.26, 0.80]])
}

for k, sub in subs.items():
    print(
        k,
        eigenvalues(
            m.evalf(subs = sub)
        )
    )
```

    geq0 [-0.249936706325260, 0.540000000000000, 1.32993670632526, 0.996070170039655, 0.0839298299603448]
    leq0 [0.54 + 0.78993670632526*I, 0.54 - 6.81490126339204e-65*I, 0.54 - 0.78993670632526*I, 0.54 - 0.456070170039655*I, 0.54 + 0.456070170039655*I]

c) Berechnen Sie die Determinante einer zufälligen $5 \times 5$ Toeplitz-Matrix mit $a, b, c \neq 0$. Was sagt dies über den Rang der Matrix aus? Überlegen und erklären Sie wann die Matrix nicht vollen Rang hat. Belegen Sie Ihre Überlegung durch die Berechnung der Determinante von einer allgemeinen tridiagonalen $5 \times 5$ Toeplitz-Matrix aus (a). Wann ist die Determinante 0? Illustrieren Sie die verschiedenen Fälle mit geeignet gewählten Beispielen.

```python
subs = {k: random.randint(1, 10) for k in 'abc'}
m_sub = m.evalf(subs = subs)
m_sub
```

$\displaystyle \left[\begin{matrix}5.0 & 6.0 & 0 & 0 & 0\\7.0 & 5.0 & 6.0 & 0 & 0\\0 & 7.0 & 5.0 & 6.0 & 0\\0 & 0 & 7.0 & 5.0 & 6.0\\0 & 0 & 0 & 7.0 & 5.0\end{matrix}\right]$

```python
m_sub.det()
```

$\displaystyle 8585.0$

Die Determinante einer quadratischen Matrix gibt Aufschluss darüber ob es sich bei der betrachteten Matrix um eine reguläre, also invertierbare oder eine singuläre, nicht invertierbare, Matrix handelt. Sei $A$ eine quadratische Matrix der Dimension $n \times n$. Ist die Determinante $\det(A) \neq 0$, so ist $A$ invertierbar und $\text{rang}(A) = n$. Ist $\det(A) = 0$ ist $A$ nicht invertierbar und $\text{rang}(A) < n$. Weiters lässt sich über den Rang einer quadratischen Matrix feststellen ob ihre Zeilen- bzw. Spaltenvektoren linear unabhängig voneinander sind. Ist dies der Fall so haben wir eine reguläre Matrix anderfalls ist die Matrix singulär. Daher ist die Determinante einer quadratischen Matrix genau dann 0 wenn nicht alle Zeilen- oder Spaltenvektoren linear unabhängig sind d.h. wenn einer der Vektoren eine Linearkombination der anderen ist. Die Linearkombination zweier Vektoren $x = (x_1, \ldots, x_n)^T$ und $y = (y_1, \ldots, y_n)^T$ zu einem Vektor $z = (z_1, \ldots, z_n)^T$ lässt sich schreiben als

$$
  z = \lambda*x + \mu*y =
  \begin{pmatrix}
    \lambda * x_1 + \mu*y_1 \\
    \vdots \\
    \lambda * x_n + \mu*y_n \\
  \end{pmatrix}
$$

Die Determinante einer $5 \times 5$ Toeplitzmatrix

$$
  m =
  \begin{pmatrix}
    a & c & 0 & 0 & 0 \\
    b & a & c & 0 & 0 \\
    0 & b & a & c & 0 \\
    0 & 0 & b & a & c \\
    0 & 0 & 0 & b & a \\
  \end{pmatrix}
$$

ist definiert als

$$
  \det(m) = a^5 - 4a^3bc + 3ab^2c^2
$$

und daher

$$
  \det(A) = 0 \Longleftrightarrow a = 0 \vee 4a^3bc = a^5 + 3ab^2c^2
$$

wobei

$$
  4a^3bc = a^5 + 3ab^2c^2 \Longleftrightarrow a = b = c
$$

womit wir obige Aussage umschreiben können auf

$$
  \det(A) = 0 \Longleftrightarrow a = 0 \vee a = b = c
$$

Nehmen wir unser Beispiel mit den Linearkombinationen von weiter oben her und setzen $a = b = c$ dann sehen wir sofort bei betrachten von $m$ dass sich z.B. der letzte Zeilenvektor als Linearkombination des erste, zweiten und vierten Zeilenvektors schreiben lässt mit

$$
  zv_5 = 1 * zv_1 + - 1 * zv_2 + zv_4 =
  \begin{pmatrix}
    a - b + 0 \\
    c - a + 0 \\
    0 - c + b \\
    0 - 0 + b \\
    0 - 0 + a \\
  \end{pmatrix} =
  \begin{pmatrix}
    0 \\
    0 \\
    0 \\
    b \\
    a \\
  \end{pmatrix}
$$

Ähnliches gilt für $a = 0$ wobei wir hier den dritten Zeilenvektor leicht durch den ersten und den letzten Zeilenvektor ausdrücken können mit

$$
  zv_3 = \lambda * zv_1 + \mu * zv_5 =
    \begin{pmatrix}
    \lambda * a + 0 \\
    \lambda * c + 0 \\
    \lambda * 0 + \lambda * 0 \\
    0 + \mu * b \\
    0 + \mu * a \\
  \end{pmatrix} =
  \begin{pmatrix}
    0 \\
    \lambda * c \\
    0 \\
    \mu * b \\
    0 \\
  \end{pmatrix}
$$
mit $\lambda, \mu \in \mathbb{R}$

```python
subs = [
    {'a': 2, 'b': 2, 'c': 2},
    {'a': 0, 'b': 2, 'c': 2}
]
for sub in subs:
    m_sub = m.evalf(subs = sub)
    display(m_sub)
    print('The determinat of the above matrix is: ', m_sub.det())
```

$\displaystyle \left[\begin{matrix}2.0 & 2.0 & 0 & 0 & 0\\2.0 & 2.0 & 2.0 & 0 & 0\\0 & 2.0 & 2.0 & 2.0 & 0\\0 & 0 & 2.0 & 2.0 & 2.0\\0 & 0 & 0 & 2.0 & 2.0\end{matrix}\right]$

    The determinat of the above matrix is:  0

$\displaystyle \left[\begin{matrix}0 & 2.0 & 0 & 0 & 0\\2.0 & 0 & 2.0 & 0 & 0\\0 & 2.0 & 0 & 2.0 & 0\\0 & 0 & 2.0 & 0 & 2.0\\0 & 0 & 0 & 2.0 & 0\end{matrix}\right]$

    The determinat of the above matrix is:  0


# Stationarity of stochastic processes II

Author: [Zeel B Patel](https://patel-zeel.github.io/)

https://bookdown.org/gary_a_napier/time_series_lecture_notes/ChapterThree.html

```python
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import rc
from matplotlib.animation import FuncAnimation

rc('font', size=16)
rc('text', usetex=True)
rc('animation', html='jshtml')
```

The function below generates multiple samples of $n$ length, $d$ order stationary process using the following AR equation (AR stands for auto-regressive),

$$
y_t = \sum\limits_{i=1}^{d}y_{t-d}\phi_{t-d} + \mathcal{N}(0, 1)
$$

```python
def AR(phi, n):
    order = len(phi)
    x = [0 for _ in range(order)]
    for i in range(n-order):
        tmp = np.sum(np.array(x[-order:])*phi+np.random.normal(0,1))
        x.append(tmp)
    return x
```

Let us visualize many samples from AR($d=1,n=100,\phi=\{0.8\}$) stochastic process.

```python
def plot_AR(Phi, n):
    X = []
    for i in range(201):
        X.append(AR(Phi, n))
    for sample in X:
        plt.plot(sample, alpha=0.2);

    means = np.mean(X, axis=0)
    std2 = np.std(X, axis=0)*2
    plt.plot(means, label='Mean')
    plt.fill_between(range(n), means-std2, means+std2, label='Mean $\pm\;2\sigma$');
    plt.legend(bbox_to_anchor=(1,1));
    plt.xlabel('$t$')
    plt.ylabel('$f(t)$')
```

```python
plot_AR([0.8], 100)
```

Let us try changing $\phi=\{-0.8\}$

```python
plot_AR([-0.8], 100)
```

We can see that mean and variance of samples at any time $t$ is almost constant.

Let us draw some samples from a AR($d=1,n=100,\phi=\{1.01\}$).

```python
plot_AR([1.01], 100)
```

We can see that mean is almost constant over time but variance is varying significantly.

What are the insights? For AR(1) process,

\begin{align}
y_t &= y_{t-1}\phi_{t-1}+Z_t , \;\;\;Z_t \sim \mathcal{N}(0,1)\\
y_t - y_{t-1}\phi_{t-1} &= Z_t\\
(1 - B\phi_{t-1})y_t &= Z_t
\end{align}

We define charesterstic equation as following,

$$
f_c(B) = 1 - B\phi_{t-1} = 0
$$

If roots (magnitude of roots in case of imaginary roots) of $f_c(B)$ are outside unit circle or in other words, $|B|>1$, AR(1) process is stationary.

* For $\phi=0.8$, $B=1.25$ so, AR($d=1, \phi=0.8$) is a stationary process.
* For $\phi=1.01$, $B=0.99$ so, AR($d=1, \phi=1.01$) is a non-stationary process.

Same rule is applicable for AR($d$) process for any value of $d$.

Let us try an AR($d=3$) process now.

We take AR($d=3,n=100,\phi=\{0.1, 0.2, 0.3\}$).

```python
plot_AR([0.1, 0.2, 0.3], 100)
```

From the graph, process looks stationary, let us find the roots of the following charesterstic equation,

$$
f_c(B) = 1 - 0.1B - 0.2B^2 - 0.3B^3 = 0
$$

```python
np.roots([-0.3, -0.2, -0.1, 1]), np.abs(np.roots([-0.3, -0.2, -0.1, 1]))
```

    (array([-0.95244656+1.3359895j, -0.95244656-1.3359895j,
             1.23822645+0.j       ]),
     array([1.64073837, 1.64073837, 1.23822645]))

We see that all the roots are outside the unit circle, so the process must be stationary.

Let us try one more example.

AR($d=3, n=10, \phi=\{0.6, 0.8, 0.9\}$)

```python
plot_AR([0.6, 0.8, 0.9], 10)
```

```python
np.roots([-0.9, -0.8, -0.6, 1]), np.abs(np.roots([-0.9, -0.8, -0.6, 1]))
```

    (array([-0.77387424+1.04284911j, -0.77387424-1.04284911j,
             0.6588596 +0.j        ]),
     array([1.29862066, 1.29862066, 0.6588596 ]))

We can see that atleast one root is inside the unit circle, thus the process is non-stationary.


```python
%matplotlib inline
from __future__ import division, print_function
from matplotlib import pyplot as plt
import numpy as np
from ipywidgets import interact
```

```python
def play(bandits, n_repetitions, mu, metric='average', noise=.3):
    mu_star = np.max(mu)
    rewards = {name: np.zeros(n_repetitions) for name in bandits}
    for i in range(n_repetitions):
        for name, bandit in bandits.items():
            arm = bandit.play()
            rewards[name][i] = mu[arm]
            bandit.feedback(arm, np.clip(mu[arm] + noise * np.random.randn(), 0, 1))

    plt.hold(True)
    for name, reward in rewards.items():
        if metric == 'regret':
            data = mu_star - reward
        elif metric == 'cumulative':  # Also known as total regret.
            data = np.cumsum(mu_star - reward)
        else:
            assert metric == 'average'
            data = np.cumsum(mu_star - reward) / np.linspace(1, n_repetitions, n_repetitions)
        plt.plot(data, '--', label=name)

    plt.xlabel("Rounds")
    plt.ylabel("Metric (%s)" % metric)
    plt.legend()
    plt.show()
```

# $\epsilon$-greedy

This strategy involves picking a (small) $\epsilon$ and then at any stage after every arm has been played at least once, explore with a probability of $\epsilon$, and exploit otherwise.

For a suitable choice of $\epsilon_t$, $R_T = O(k \log T)$, which means that $\frac{R_T}{T} = O\left( \frac{\log T}{T} \right)$, which goes to 0 as T goes to $\infty$.

```python
class EpsGreedy:
    def __init__(self, n_arms, eps=0):
        self.eps = eps
        self.n_arms = n_arms
        self.payoffs = np.zeros(n_arms)
        self.n_plays = np.zeros(n_arms)

    def play(self):
        # Note that the theory tells us to pick epsilon as O(1/t), not constant (which we use here).
        idx = np.argmin(self.n_plays)
        if self.n_plays[idx] == 0:
            return idx
        if np.random.rand() <= self.eps:
            return np.random.randint(self.n_arms)
        else:
            return np.argmax(self.payoffs / self.n_plays)

    def feedback(self, arm, reward):
        self.payoffs[arm] += reward
        self.n_plays[arm] += 1
```

# UCB1

This algorithms keeps track of the upper confidence bound for every arm, and always picks the arm with the best upper confidence bound.

At any point in time $t$, we know each arm's draw count $n_i^{(t)}$, as well as its average payoff $\hat{\mu}_i^{(t)}$. Based on this, we can compute every arm's upper confidence bound (or UCB):

\begin{equation}
    \operatorname{UCB}(i) = \hat{\mu}_i + \sqrt{\frac{2\ln t}{n_i}}
\end{equation}

Also **no-regret**, just like $\epsilon$-greedy. The math is just a bit fluffier (see slide dm-11:20).

```python
class UCB:
    def __init__(self, n_arms, tau):
        self.n_arms = n_arms
        self.means = np.zeros(n_arms)
        # Note that the UCB1 algorithm has tau=1.
        self.n_plays = np.zeros(n_arms)
        self.tau = tau
        self.t = 0

    def play(self, plot=True):
        # If plot is true, it will plot the means + bounds every 100 iterations.
        self.t += 1
        idx = np.argmin(self.n_plays)
        if self.n_plays[idx] == 0:
            return idx

        ub = self.tau * np.sqrt(2 * np.log(self.t) / self.n_plays)
        ucb = self.means + ub

        if plot and self.t % 100 == 0:
            plt.errorbar(list(range(self.n_arms)), self.means, yerr=ub)
            plt.show()
            print('chose arm', np.argmax(ucb))

        return np.argmax(ucb)

    def feedback(self, arm, reward):
        self.n_plays[arm] += 1
        self.means[arm] += 1 / (self.n_plays[arm]) * (reward - self.means[arm])
```

```python
@interact(n_arms=(10, 100, 1), n_rounds=(100, 1000, 10), eps=(0, 1, .01) , tau=(0, 1, .01))
def run(n_arms, n_rounds, eps, tau):
    np.random.seed(123)
    # Initialize the arm payoffs.
    mu = np.random.randn(n_arms)
    # Some other strategies for sampling.
    # mu = np.random.standard_cauchy(n_arms)
    # mu = np.random.gamma(shape=.1, size=(n_arms, 1))
    mu = np.abs(mu)
    mu /= np.max(mu)
    plt.bar(list(range(n_arms)), mu)
    plt.xlabel('arms')
    plt.ylabel('rewards')
    plt.show()
    bandits = {
        'eps-{0}'.format(eps) : EpsGreedy(n_arms, eps=eps),
        'ucb-{0}'.format(tau) : UCB(n_arms, tau=tau)
    }
    play(bandits, n_rounds, mu)
    # Hint: You can also plot the upper bound from UCB1 and see how tight it is.
```

```python

```


# Inference in Gaussian Networks

The [junction tree algorithm](https://en.wikipedia.org/wiki/Junction_tree_algorithm) (`JTA`) is a widely used algorithm for exact inference in Bayesian Belief Networks (`BBNs`). A great paper to learn the mechanics of JTA is authored by [Huang and Darwiche](http://pages.cs.wisc.edu/~dpage/ijar95.pdf). The Huang and Darwiche paper focuses only on discrete variables and leaves a lot to be desired if one has continuous variables.

The JTA may also be applied to continuous variables, and in particular, a set of Gaussian variables (multivariate Gaussian). The key idea in JTA is to transform a directed acylic graph (`DAG`) into a junction tree (`JT`). Once a JT is created, probabilistic inference to extract marginal probabilities (with or without evidence) is possible. Just like a BBN, which is composed of a graph (structure), `G`, and joint probability (parameters), `P`, a JT also has this type of duality. The graph of a JT is composed of nodes (`cliques` and `separation sets`) and edges (structure), and associated with each node is a `potential` (parameters).

## Potentials, discrete variables

Potentials are the key to `inference` in JTA. Inference is conducted in JTA by passing `messages` (potentials) around. The representation of potentials is one of the two key differences between using JTA for discrete versus continuous variables (the other being how evidence is injected into the JT). For JTA with discrete variables, a potential is essentially a table. Below, we have a clique associated with the binary variables `A` and `B`. The potential associated with this clique would be the cross-product of the domains and a value (typically between $[0, 1]$) associated to each combination of the instantiations.

| A   | B   | value |
|-----|-----|-------|
| on  | on  | 0.2   |
| on  | off | 0.3   |
| off | on  | 0.4   |
| off | off | 0.8   |

## Potentials, Gaussian variables

The representation of potentials in a JT for Gaussian variables is entirely different but based on the multivariate Gaussian density function. Let's look at the multivariate Gaussian density function.

$f_X(x_1, x_2, \ldots, x_n)=\cfrac{\exp\left(\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^T\Sigma^{-1}(\mathbf{x}-\mathbf{\mu})\right)}{(2\pi)^{\frac{n}{2}}|\Sigma|^{\frac{1}{2}}}$

Where

* $n$ is the number of Gaussian variables
* $\Sigma$ is the $n \times n$ covariance matrix
* $\mathbf{\mu}$ is a $n$ dimensional vector of means
* $\mathbf{x}$ is a $n$ dimensional vector of values

This form of the multivariate Gaussian is called the `covariance form`. In expanded form, the density function may be rewritten as follows.

$\begin{equation}
\begin{split}
f_X(x_1, x_2, \ldots, x_n) & = \cfrac{\exp\left(\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^T\Sigma^{-1}(\mathbf{x}-\mathbf{\mu})\right)}{(2\pi)^{\frac{n}{2}}|\Sigma|^{\frac{1}{2}}} \\
  & = \exp\left(-\frac{1}{2}\mathbf{x}^T\Sigma^{-1}\mathbf{x} + \mathbf{\mu}^T\Sigma^{-1}\mathbf{x} - \frac{1}{2}\mathbf{\mu}^T\Sigma^{-1}\mathbf{\mu} - \log\left((2\pi)^{\frac{n}{2}}|\Sigma|^{\frac{1}{2}}\right)\right)
\end{split}
\end{equation}$

Let's denote the following.

* $K = \Sigma^{-1}$
* $\mathbf{h} = \Sigma^{-1}\mathbf{\mu}$
* $g = -\frac{1}{2}\mathbf{\mu}^T\Sigma^{-1}\mathbf{\mu} - \log\left((2\pi)^{\frac{n}{2}}|\Sigma|^{\frac{1}{2}}\right)$

$K$ is called the `precision matrix` or `information matrix` , and it is the inverse of the covariance matrix. $h$ has a special name and depending on the context, is called the `information vector`, `observation vector` or even `shift vector`. Note that $\mathbf{h}$ is a $n$ dimensional vector and $g$ evaluates to a scalar (a single number). Substituting $K$, $h$ and $g$ into the `covariance form`, we can rewrite the multivariate density function as follows.

$\begin{equation}
\begin{split}
f_X(x_1, x_2, \ldots, x_n) & = \cfrac{\exp\left(\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^T\Sigma^{-1}(\mathbf{x}-\mathbf{\mu})\right)}{(2\pi)^{\frac{n}{2}}|\Sigma|^{\frac{1}{2}}} \\
  & = \exp\left(-\frac{1}{2}\mathbf{x}^T\Sigma^{-1}\mathbf{x} + \mathbf{\mu}^T\Sigma^{-1}\mathbf{x} - \frac{1}{2}\mathbf{\mu}^T\Sigma^{-1}\mathbf{\mu} - \log\left((2\pi)^{\frac{n}{2}}|\Sigma|^{\frac{1}{2}}\right)\right) \\
  & = \exp\left(-\frac{1}{2}\mathbf{x}^TK\mathbf{x} + \mathbf{h}^T\mathbf{x} + g\right)
\end{split}
\end{equation}$

This last form of the multivariate Gaussian is the potential, $\phi$, of a node in a JT of all Gaussian variables. This form is also called the `canonical form` or `information form` of the multivariate Gaussian.

$\phi(\mathbf{x}; K, \mathbf{h}, g) = \exp\left(-\frac{1}{2}\mathbf{x}^TK\mathbf{x} + \mathbf{h}^T\mathbf{x} + g\right)$

When marginalizing over a multivariate Gaussian distribution, it is easier to do so using the `covariance form`. When conditioning against a multivariate Gaussian distribution, it is easier to do so using the `information form` .

## Operations on $\phi$

The three main operations we would like to do against $\phi$ are

- multiplication,
- division, and
- marginalization.

### Multiplication

We may `multiply two potentials` as follows (omitting the reference to $\mathbf{x}$).

$\phi(K_1,\mathbf{h}_1, g_1) \cdot \phi(K_2,\mathbf{h}_2, g_2) = \phi(K_1 + K_2, \mathbf{h}_1 + \mathbf{h}_2, g_1 + g_2)$

### Division

We may `divide two potentials` as follows.

$\cfrac{\phi(K_1,\mathbf{h}_1, g_1)}{\phi(K_2,\mathbf{h}_2, g_2)} = \phi(K_1 - K_2, \mathbf{h}_1 - \mathbf{h}_2, g_1 - g_2)$

### Marginalization

`Marginalization` over $\phi$ is integration and used in message passing. Assume two sets of disjoint variables $X$ and $Y$, then, the potential is written as $\phi(X,Y; K, \mathbf{h}, g)$. We may integrate $\phi(X,Y; K, \mathbf{h}, g)$ over $Y$ to give us $\phi(X; K', \mathbf{h}', g')$.

$\begin{equation}
\begin{split}
\int \phi(X,Y; K, \mathbf{h}, g) dY & = \phi(X; K', \mathbf{h}', g')
\end{split}
\end{equation}$

Where

* $K' = K_{XX} - K_{XY}K_{YY}^{-1}K_{YX}$
* $\mathbf{h}' = \mathbf{h}_{X} - K_{XY}K_{YY}^{-1}\mathbf{h}_{Y}$
* $g' = g + \frac{1}{2}\left(\log|2\pi K_{YY}^{-1}| + \mathbf{h}_Y^TK_{YY}^{-1}\mathbf{h}_Y\right)$

### Evidence

When there is `evidence`, $\phi(X,Y; K, \mathbf{h}, g)$ may be reduced to $\phi(X; K', \mathbf{h}', g')$, where

* $K' = K_{XX}$
* $h' = \mathbf{h}_X - K_{XY}\mathbf{y}$
* $g' = g + \mathbf{h}_Y^T \mathbf{y} - \frac{1}{2} \mathbf{y}^T K_{YY} \mathbf{y}$

## Message passing

For two cliques $X$ and $Y$ with a separation set $R$ in between them, message passing from $X$ to $Y$ through $R$ consists of `projection` and `absorption`. The cliques and separation set will have the following denoted potentials: $\phi_X$, $\phi_Y$, and $\phi_R$.

Projection involves saving $\phi_R$ and replacing it through marginalization of $\phi_X$.

* $\phi_R^{0} \leftarrow \phi_R$
* $\phi_R \leftarrow \displaystyle \sum_{X \setminus R} \phi_X$

Absorption involves multiplication and division of the potentials.

* $\phi_Y \leftarrow \phi_Y \cfrac{\phi_R}{\phi_R^0}$

In JTA, `global propagation` uses message passing in two phases, `collect-evidence` and `distribute-evidence` to make the potentials locally consistent. In collect-evidence, we start by picking an initial clique (not separation set) in the JT and recursively travel to the terminal cliques (marking each one on the way). Once a terminal clique is reached, we pass messages back to the clique that came before it. In distribute-evidence, we start with the same initial clique and pass messages to its neighboring cliques, and then recursively call distribute-evidence to those neighboring cliques after.

After global propagation, for any variable whose density function we are interested, we pick a clique containing the variable and marginalize (over the other variables).


# Métodos para sistemas no lineales

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
def newtonMultivariate(F, J, x_0, n, tol=1e-10):
    x = np.zeros((n + 1, x_0.shape[0]))
    x[0] = x_0
    for k in range(n):
        w = np.linalg.solve(J(*x[k]), -F(*x[k]))
        x[k+1] = x[k] + w
        if np.linalg.norm(F(*x[k+1])) < tol:
            x = x[:k+2]
            break
    return x
```

```python
def broyden1(F, A_0, x_0, n, tol=1e-10):
    x = np.zeros((n + 1, x_0.shape[0]))
    x[0] = x_0
    A_k = A_0
    for k in range(n):
        delta = np.linalg.solve(A_k, -F(*x[k]))
        x[k+1] = x[k] + delta
        Delta = F(*x[k+1]) - F(*x[k])
        A_k = A_k + np.outer(Delta - np.dot(A_k, delta), delta) / np.dot(delta, delta)
        if np.linalg.norm(F(*x[k+1])) < tol:
            x = x[:k+2]
            break
    return x
```

```python
def broyden2(F, B_0, x_0, n, tol=1e-10):
    x = np.zeros((n + 1, x_0.shape[0]))
    x[0] = x_0
    B_k = B_0
    for k in range(n):
        x[k+1] = x[k] - np.dot(B_k, F(*x[k]))
        delta = x[k+1] - x[k]
        Delta = F(*x[k+1]) - F(*x[k])
        B_k = B_k + np.dot(np.outer(delta - np.dot(B_k, Delta), delta), B_k) / np.dot(delta, np.dot(B_k, Delta))
        if np.linalg.norm(F(*x[k+1])) < tol:
            x = x[:k+2]
            break
    return x
```

```python
def error(X, x):
    err = np.linalg.norm(X - x, axis=1, ord=np.inf) / np.linalg.norm(x, np.inf)
    return err
```

```python
def plotCompare(error_n, error_b1, error_b2):
    iter_n = np.arange(error_n.shape[0])
    iter_b1 = np.arange(error_b1.shape[0])
    iter_b2 = np.arange(error_b2.shape[0])
    plt.figure(figsize=(12, 6))
    plt.plot(iter_n, error_n, 'go',label="Newton")
    plt.plot(iter_b1, error_b1, 'rs', label="Broyden 1")
    plt.plot(iter_b2, error_b2, 'kd', label="Broyden 2")
    plt.xlabel("# Iteraciones")
    plt.ylabel("Error")
    plt.yscale('log')
    plt.grid(True)
    plt.legend()
    plt.show()
```

## Ejemplo de sistema de ecuaciones no lineales

Resolver
\begin{equation}
    \begin{split}
        y - x^3 & = 0 \\
        x^2 + y^2 - 1 & = 0
    \end{split}
\end{equation}

Se define $f_1(x,y)=y - x^3$, $f_2(x,y)=x^2 + y^2 - 1$,
\begin{equation}
    \mathbf{F}(x,y)=
    \begin{bmatrix}
        f_1(x,y) \\ f_2(x,y)
    \end{bmatrix} =
    \begin{bmatrix}
        y - x^3 \\
        x^2 + y^2 - 1
    \end{bmatrix},
\end{equation}

y
\begin{equation}
    J_{\mathbf{F}}(x,y)=
    \begin{bmatrix}
        -3x^2 & 1 \\
        2x & 2y
    \end{bmatrix}
\end{equation}

La solución real de este sistema es: $x=\pm 0.826031357654187$, $y=\pm 0.826031357654187^3$.

```python
F1 = lambda x, y: np.array([y - x ** 3, x ** 2 + y ** 2 - 1])
J1 = lambda x, y: np.array([[-3 * x ** 2, 1], [2 * x, 2 * y]])
x1_0 = np.array([1, 2])
x_sol = np.array([0.826031357654187, 0.826031357654187**3])
```

```python
n1_n = 7
x1_n = newtonMultivariate(F1, J1, x1_0, n1_n)
```

```python
A_0 = np.eye(x1_0.shape[0]) # J1(*x1_0)
n1_b = 50
x1_b1 = broyden1(F1, A_0, x1_0, n1_b)
```

```python
B_0 = np.eye(x1_0.shape[0]) # np.linalg.inv(J1(*x1_0))
n1_b2 = 50
x1_b2 = broyden2(F1, B_0, x1_0, n1_b2)
```

```python
error_n = error(x1_n, x_sol)
error_b1 = error(x1_b1, x_sol)
error_b2 = error(x1_b2, x_sol)
plotCompare(error_n, error_b1, error_b2)
```

## Ejemplo sistema de ecuaciones lineales

Resolver

\begin{equation}
    \begin{split}
        x - 8y - 2z & = 1 \\
        x + y + 5z & = 4 \\
        3x - y + z & = -2
    \end{split}
\end{equation}

Debemos llevarlo a la forma $\mathbf{F}(\mathbf{x})=\mathbf{0}$, con $\mathbf{x}=(x,y,z)$.

\begin{equation}
    \mathbf{F}(x,y,z)=
    \begin{bmatrix}
        x - 8y - 2z - 1 \\
        x + y + 5z - 4 \\
        3x - y + z +2
    \end{bmatrix},
\end{equation}

y,

\begin{equation}
    J_{\mathbf{F}}(x,y,z) =
    \begin{bmatrix}
        1 & -8 & -2 \\
        1 & 1 & 5 \\
        3 & -1 & 1
    \end{bmatrix}
\end{equation}

```python
F2 = lambda x, y, z: np.array([x - 8 * y - 2 * z - 1, x + y + 5 * z - 4, 3 * x - y + z + 2])
J2 = lambda x, y, z: np.array([[1, -8, -2], [1, 1, 5], [3, -1, 1]])
x2_0 = np.array([1, 3, 0])
```

### Solución

```python
A = np.array([
    [1, -8, -2],
    [1, 1, 5],
    [3, -1, 1]
]) # Se parece a otra matriz?
b = np.array([1, 4, -2])
x_2 = np.linalg.solve(A, b)
```

```python
x_2
```

    array([-1.24489796, -0.57142857,  1.16326531])

### Aproximación con Newton

```python
n2_n = 10
x2_n = newtonMultivariate(F2, J2, x2_0, n2_n)
```

```python
x2_n
```

    array([[ 1.        ,  3.        ,  0.        ],
           [-1.24489796, -0.57142857,  1.16326531]])

```python
np.linalg.norm(x2_n[-1] - x_2, np.inf)
```

    2.220446049250313e-16

Notar que al tener un sistema de ecuaciones lineales y utilizar Newton en $\mathbb{R}^n$ ocurre lo siguiente.

\begin{equation}
    \mathbf{F}(\mathbf{x})=
    \begin{bmatrix}
        a_{1,1}x_1 + a_{1, 2}x_2 + \cdots a_{1, n}x_n - b_1 \\
        a_{2,1}x_1 + a_{2, 2}x_2 + \cdots a_{2, n}x_n - b_2 \\
        \quad \vdots \\
        a_{n,1}x_1 + a_{n, 2}x_2 + \cdots a_{n, n}x_n - b_n \\
    \end{bmatrix} =
    A\mathbf{x}-\mathbf{b}
\end{equation}

y al calcular la matriz Jacobiana tenemos:

\begin{equation}
    J_{\mathbf{F}}(\mathbf{x}) =
    \begin{bmatrix}
        a_{1,1} & a_{1, 2} & \cdots & a_{1, n} \\
        \vdots  & \vdots   & \vdots & \vdots \\
        a_{n,1} & a_{n, 2} & \cdots & a_{n, n}
    \end{bmatrix} =
    A.
\end{equation}

Al aplicar el algoritmo tenemos:

\begin{equation}
    \begin{split}
        J_{\mathbf{F}}(\mathbf{x}_k)\, \mathbf{w} &= -\mathbf{F}(\mathbf{x}_k) \\
        A\,(\mathbf{x}_{k+1}-\mathbf{x}_{k}) & = \mathbf{b} - A\mathbf{x}_k \\
        A\,\mathbf{x}_{k+1}-A\,\mathbf{x}_{k} & = \mathbf{b} - A\mathbf{x}_k \\
        A\,\mathbf{x}_{k+1} & = \mathbf{b}
    \end{split},
\end{equation}

es decir, se resuelve el mismo sistema de ecuaciones lineales del cual estamos buscando solución en un paso.


# Algorithms Exercise 2

## Imports

```python
%matplotlib inline
from matplotlib import pyplot as plt
import seaborn as sns
import numpy as np
```

## Peak finding

Write a function `find_peaks` that finds and returns the indices of the local maxima in a sequence. Your function should:

* Properly handle local maxima at the endpoints of the input array.
* Return a Numpy array of integer indices.
* Handle any Python iterable as input.

```python
def find_peaks(a):
    """Find the indices of the local maxima in a sequence."""
    a = np.array(a)
    s = np.sign(np.diff(a))
    d = np.diff(s)
    ind  = [i for i in range(len(d)) if d[i] == 2]
    if s[-1] == 1:
         ind.append(len(a)-1)
    return(np.array(ind))

```

```python
p1 = find_peaks([2,0,1,0,2,0,1])
assert np.allclose(p1, np.array([0,2,4,6]))
p2 = find_peaks(np.array([0,1,2,3]))
assert np.allclose(p2, np.array([3]))
p3 = find_peaks([3,2,1,0])
assert np.allclose(p3, np.array([0]))
```

```python
find_peaks([2,2,2,1,2,2,2])
```

    array([2])

Here is a string with the first 10000 digits of $\pi$ (after the decimal). Write code to perform the following:

* Convert that string to a Numpy array of integers.
* Find the indices of the local maxima in the digits of $\pi$.
* Use `np.diff` to find the distances between consequtive local maxima.
* Visualize that distribution using an appropriately customized histogram.

```python
from sympy import pi, N
pi_digits_str = str(N(pi, 10001))[2:]
```

```python
pi_int = np.array(list(pi_digits_str), dtype="int")
pks = find_peaks(pi_int)
pks_diff = np.diff(pks)
plt.hist(pks_diff, bins = range(0,max(pks_diff)+1))
min(pks_diff), max(pks_diff)
```

```python
pks
```

    array([   1,    4,    7, ..., 9993, 9995, 9999])

```python
assert True # use this for grading the pi digits histogram
```


# Table of Contents

1  Generating dataset

1.1  System equation

1.2  System input

1.3  System simulation

1.4  Adding output noise

1.5  Validation data and identification data

2  Neural Network Parameter Estimation

2.1  Set problem

2.2  Solve optimization problem

2.3  Simulate on Identification Interval

3  Validation

```python
import numpy as np
import matplotlib.pyplot as plt
from oeshoot import narx
% matplotlib inline
```

```python
def tic():
    #Homemade version of matlab tic and toc functions
    import time
    global startTime_for_tictoc
    startTime_for_tictoc = time.time()

def toc():
    import time
    if 'startTime_for_tictoc' in globals():
        print("Elapsed time is " + str(time.time() - startTime_for_tictoc) + " seconds.")
    else:
        print("Toc: start time not set")
```

# Generating dataset

##  System equation
The data set used for identification will be generated using the following equation:

\begin{equation}
y[n] = (0.8-0.5\text{exp}(-y^2[n-1]))y[n-1]-(0.3+0.9\text{exp}(-y^2[n-1]))y[n-2]+u[n-1]+0.2u[n-2]+0.1u[n-1]u[n-2]
\label{eq:equation}
\end{equation}

```python
# Define System
def system_equation(y, u):
    ynext = (0.8 - 0.5*np.exp(-y[0]**2))*y[0] - \
        (0.3+0.9*np.exp(-y[0]**2))*y[1] + u[0] + \
        0.2*u[1]+0.1*u[0]*u[1]
    return ynext

# System Parameters
N = 2
M = 2
Ny = 1
Nu = 1
```

##  System input
The system input $u[n]$ is generated such that each
value is hold for ten samples and have gaussian distribution with zero
mean and unitary variance.

```python
# Defining parameters
rept = 5
random_samples = 200
total_time = rept*random_samples

# Time vector
n = np.arange(0, total_time)

# Generating input
u = np.random.normal(size=(random_samples,)).repeat(rept)
```

```python
# Plot
_, ax = plt.subplots(figsize=(12, 6))
ax.step(n, u)
ax.set_xlabel("$n$", fontsize=18)
ax.set_ylabel("$u[n]$", fontsize=18)
ax.grid()
plt.show()
```

##  System simulation

The defined system will be simulated for the given input.

```python
# Initialize input
y = np.empty_like(u)

# Initial Conditions
y0 = [0, 0]

# Simulate it
y[0:2] = y0
for i in range(2, len(y)):
    y[i] = system_equation([y[i-1], y[i-2]], [u[i-1], u[i-2]])+np.random.normal(scale=0.1)

# Plot
_, ax = plt.subplots(figsize=(12, 6))
ax.plot(n, y)
ax.set_xlabel("$n$", fontsize=18)
ax.set_ylabel("$y[n]$", fontsize=18)
ax.grid()
plt.show()
```

## Adding output noise

Some gaussian white output noise with standart deviation $1.0$ will be added to the data.

```python
# Noise free data
ynoise_free = y

# Generate noise
v = np.random.normal(scale=0.5, size=y.shape)

# Data corrupted with noise
ynoisy = ynoise_free + v

# Plot
_, ax = plt.subplots(figsize=(14, 8))
ax.plot(n, ynoise_free, label='Noise-free data')
ax.plot(n, ynoisy, label='Corrupted data')
ax.set_xlabel("$n$", fontsize=18)
ax.set_ylabel("$y[n]$", fontsize=18)
ax.grid()
ax.legend(fontsize=18)
plt.show()
```

## Validation data and identification data
The data will be divided into validation and identification data. The first 600 values will be used
for identification and the last 400 values for validation. The validation data will be free of noise,
so the model quality may be better assessed.

```python
Ni = 700
Nv = len(y) - Ny

# Identification data
yi = ynoisy[:Ni]
ui = u[:Ni]

# Validation data
yv = ynoise_free[Ni:]
uv = u[Ni:]
```

# Neural Network Parameter Estimation

## Set problem
A neural network model will be fit to the identification data, minimizing simulation error. The system will consider $y[n-1]$, $y[n-2]$, $u[n-1]$ and $u[n-2]$ as inputs for the neural network. Two hidden layers with
10 hidden nodes each will be considered.

The simulation error will be minimized in order to find the parameters.

```python
# Feed forward model
mdl = narx.FeedforwardNetwork(2, 2, [30])

# Reshape data
yi = yi.reshape((-1, 1))
ui = ui.reshape((-1, 1))

# Define Simulation Error
sim_error = narx.SimulationError(mdl, yi, ui, maxlength=6000)
predict_error = narx.PredictionError(mdl, yi, ui)

# Define Input and Output bounds
ybounds = [[-6, 6]]
ubounds = [[-2, 2]]

# Get initial guess for the parameters
initial_guess = mdl.params_random_guess(ybounds, ubounds)
```

## Solve optimization problem

```python
tic()
params_predict, info = predict_error.lsq_estimate_parameters(initial_guess, max_nfev=100)
toc()
```

    Elapsed time is 17.8364839554 seconds.

```python
tic()
params, info = sim_error.lsq_estimate_parameters(initial_guess, use_sparse=False,max_nfev=100)
toc()
```

    Elapsed time is 21.6483960152 seconds.

# Validation

Simulate data on Validation Interval

```python
ys = narx.simulate(mdl, [[yv[0]], [yv[1]]], uv.reshape((-1, 1)), params)
ys = np.hstack(([yv[0], yv[1]],ys[:-1].flatten()))
yp = narx.simulate(mdl, [[yv[0]], [yv[1]]], uv.reshape((-1, 1)), params_predict)
yp = np.hstack(([yp[0,0], yp[1,0]],yp[:-1].flatten()))

mse_s = np.mean((yv.flatten()-ys.flatten())**2)
print(mse_s)
mse_p = np.mean((yv.flatten()-yp.flatten())**2)
print(mse_p)

# Plot
_, ax = plt.subplots(figsize=(14, 8))
ax.plot(n[Ni:], yv, label='Data')
ax.plot(n[Ni:], ys, label='Simulation')
ax.plot(n[Ni:], yp, label='Prediction')
ax.set_xlabel("$n$", fontsize=18)
ax.set_ylabel("$y[n]$", fontsize=18)
ax.grid()
ax.legend(fontsize=18)
plt.show()
```


```python
import numpy as np
from sympy import *
init_printing(use_latex='mathjax')
```

```python
def get_jacobian(expressions, symbols):
    rows = len(expressions)
    columns = len(symbols)

    print("Expressions:")
    for expression in expressions:
        display(expression)

    results = [[0 for x in range(columns)] for y in range(rows)]
    for row, expression in enumerate(expressions):
        for column, symbol in enumerate(symbols):
#             print('Row %d, column %d, expression: %s, symbol: %s' % (row, column, expression, symbol))
            df = diff(expression, symbol)
#             print("DF: %s" % df)
            results[row][column] = df
    return results
```

```python
def get_hessians(jacobian, symbols):
    jacobian = np.array(jacobian)
    rows, columns = jacobian.shape
    hessians = [[[None for z in range(len(symbols))] for x in range(columns)] for y in range(rows)]
    for row in range(rows):
        for column in range(columns):
            for index, symbol in enumerate(symbols):
                df = diff(jacobian[row, column], symbol)
                hessians[row][column][index] = df

    return hessians
```

```python
x, y = symbols('x y')
j = get_jacobian([
    y * x ** 3 + x + 2 * y
], [x, y])
display(j)
h = get_hessians(j, [x, y])
display(h)
```

    Expressions:

$$x^{3} y + x + 2 y$$

$$\left [ \left [ 3 x^{2} y + 1, \quad x^{3} + 2\right ]\right ]$$

$$\left [ \left [ \left [ 6 x y, \quad 3 x^{2}\right ], \quad \left [ 3 x^{2}, \quad 0\right ]\right ]\right ]$$

```python
x, y = symbols('x y')
j = get_jacobian([
    exp(x) * cos(y)
], [x, y])
display(j)
h = get_hessians(j, [x, y])
display(h)
```

    Expressions:

$$e^{x} \cos{\left (y \right )}$$

$$\left [ \left [ e^{x} \cos{\left (y \right )}, \quad - e^{x} \sin{\left (y \right )}\right ]\right ]$$

$$\left [ \left [ \left [ e^{x} \cos{\left (y \right )}, \quad - e^{x} \sin{\left (y \right )}\right ], \quad \left [ - e^{x} \sin{\left (y \right )}, \quad - e^{x} \cos{\left (y \right )}\right ]\right ]\right ]$$

```python
x, y = symbols('x y')
j = get_jacobian([
    x ** 2 /2 + x * y + y ** 2 / 2
], [x, y])
display(j)
h = get_hessians(j, [x, y])
display(h)
```

    Expressions:

$$\frac{x^{2}}{2} + x y + \frac{y^{2}}{2}$$

$$\left [ \left [ x + y, \quad x + y\right ]\right ]$$

$$\left [ \left [ \left [ 1, \quad 1\right ], \quad \left [ 1, \quad 1\right ]\right ]\right ]$$

```python
x, y, z = symbols('x y z')
j = get_jacobian([
    x ** 2 * exp(-y) * cos(z)
], [x, y, z])
display(j)
h = get_hessians(j, [x, y, z])
display(h)
```

    Expressions:

$$x^{2} e^{- y} \cos{\left (z \right )}$$

$$\left [ \left [ 2 x e^{- y} \cos{\left (z \right )}, \quad - x^{2} e^{- y} \cos{\left (z \right )}, \quad - x^{2} e^{- y} \sin{\left (z \right )}\right ]\right ]$$

$$\left [ \left [ \left [ 2 e^{- y} \cos{\left (z \right )}, \quad - 2 x e^{- y} \cos{\left (z \right )}, \quad - 2 x e^{- y} \sin{\left (z \right )}\right ], \quad \left [ - 2 x e^{- y} \cos{\left (z \right )}, \quad x^{2} e^{- y} \cos{\left (z \right )}, \quad x^{2} e^{- y} \sin{\left (z \right )}\right ], \quad \left [ - 2 x e^{- y} \sin{\left (z \right )}, \quad x^{2} e^{- y} \sin{\left (z \right )}, \quad - x^{2} e^{- y} \cos{\left (z \right )}\right ]\right ]\right ]$$

```python
x, y, z = symbols('x y z')
j = get_jacobian([
    x * exp(y) + y ** 2 * cos(z)
], [x, y, z])
display(j)
h = get_hessians(j, [x, y, z])
display(h)
```

    Expressions:

$$x e^{y} + y^{2} \cos{\left (z \right )}$$

$$\left [ \left [ e^{y}, \quad x e^{y} + 2 y \cos{\left (z \right )}, \quad - y^{2} \sin{\left (z \right )}\right ]\right ]$$

$$\left [ \left [ \left [ 0, \quad e^{y}, \quad 0\right ], \quad \left [ e^{y}, \quad x e^{y} + 2 \cos{\left (z \right )}, \quad - 2 y \sin{\left (z \right )}\right ], \quad \left [ 0, \quad - 2 y \sin{\left (z \right )}, \quad - y^{2} \cos{\left (z \right )}\right ]\right ]\right ]$$

```python

```

Certification 2/Week2.4 - Hessians.ipynb


# Fitting a transfer function with CurveFit and TransferFunctionModel
In this tutorial, we will use ``kontrol.curvefit.CurveFit`` to fit a measured transfer function.
As is mentioned in previous tutorial, ``kontrol.curvefit.CurveFit`` requires a few things, the independent variable data ``xdata``, the dependent variable ``ydata``, the model ``model``, the cost function ``cost``, and the optimizer.
They have the following signature:
```
xdata : array
ydata : array
model : func(x: array, *args, **kwargs) -> array
cost : func(args: array, model: func, xdata: array, ydata: array, model_kwargs: dict) -> float
optimizer : func(cost: func, **kwargs) -> scipy.optimize.OptimzeResult
```
To save you a lot of troubles, ``kontrol`` library provides various models classes ``kontrol.curvefit.model``, cost functions (error functions). And optimizers are readily available in ``scipy.optimize``. So we typically just need to prepare ``xdata`` and ``ydata``.
Of course, knowing what model, error function, and optimizer to use is crucial in a curve fitting task.

This time, we will use the model ``kontrol.curvefit.model.TransferFunctionModel`` as our model this time.
This model can be defined by the number of zeros ``nzero`` and ``npole``. The parameters are simply concatenated coefficients of numerator and denominator arranging from high to low order.

Here, let's consider the transfer function

\begin{equation}
    H(s) = \frac{s^2 + 3s + 2}{s^3 + 12s^2 + 47s + 60}
\end{equation}

So the parameters we would like to recover are the coefficients ``[1, 3, 2, 1, 12, 47, 60]``.

For the sake of demonstration, let's assume that we know there are 2 zeros and 3 poles as this is required to define a model.

We will use ``kontrol.curvefit.error_func.tf_error`` as the error function and it is defined as
\begin{equation}
E_\mathrm{tf\_error}\left(H_1(f), H_2(f); w(f), \epsilon\right) = \frac{1}{N}\sum_{i=0}^{N} \log_{10}(\lvert(H_1(f_i) - H_2(f_i)+\epsilon)w(f_i)\rvert)\,,
\end{equation}
where $H_1(f)$ and $H_2(f)$ are the frequency responses values (complex array) of the measured system and the model, $\epsilon$ is a small number to prevent $\log_{10}$ from exploding, $w(f)$ is a weighting function, and $N$ is the total number of data points.

```python
import control
import numpy as np
import matplotlib.pyplot as plt

f = np.logspace(-3, 3, 100000)
s = 1j*2*np.pi*f
tf = (s**2 + 3*s + 2) / (s**3 + 12*s**2 + 47*s + 60)

plt.figure(figsize=(15, 5))
plt.subplot(121)
plt.loglog(f, abs(tf))
plt.grid(which="both")
plt.ylabel("Amplitude")
plt.xlabel("Frequency (Hz)")
plt.subplot(122)
plt.semilogx(f, np.angle(tf))
plt.grid(which="both")
plt.ylabel("Phase (rad)")
plt.xlabel("Frequency (Hz)")
plt.show()
```

Now, let's everything that ``kontrol.curvefit.CurveFit`` needs, namely ``xdata``, ``ydata``, ``model``, ``cost``, and ``optimizer``.

```python
import kontrol.curvefit
import scipy.optimize

xdata = f
ydata = tf
model = kontrol.curvefit.model.TransferFunctionModel(nzero=2, npole=3, log_args=False)
error_func = kontrol.curvefit.error_func.tf_error
cost = kontrol.curvefit.Cost(error_func=error_func)
optimizer = scipy.optimize.minimize
```

Since we're using ``scipy.optimize.minimize``, it requires an initial guess. Let's start with all ones.

```python
x0 = np.ones(7)  # There are 7 coefficients, 3 for numerator and 4 for denominator
optimizer_kwargs = {"x0": x0}
```

Now let's use ``kontrol.curvefit.CurveFit`` to fit the data.

```python
a = kontrol.curvefit.CurveFit()
a.xdata = xdata
a.ydata = ydata
a.model = model
a.cost = cost
a.optimizer = optimizer
a.optimizer_kwargs = optimizer_kwargs
res = a.fit()
```

```python
plt.figure(figsize=(15, 5))
plt.subplot(121)
plt.loglog(f, abs(tf), label="Data")
plt.loglog(f, abs(a.model(f, x0)), label="Initial guess")
plt.loglog(f, abs(a.yfit), label="Fit")
plt.legend(loc=0)
plt.grid(which="both")
plt.ylabel("Amplitude")
plt.xlabel("Frequency (Hz)")
plt.subplot(122)
plt.semilogx(f, np.angle(tf), label="Data")
plt.semilogx(f, np.angle(a.model(f, x0)), label="Initial guess")
plt.semilogx(f, np.angle(a.yfit), label="Fit")
plt.legend(loc=0)
plt.grid(which="both")
plt.ylabel("Phase (rad)")
plt.xlabel("Frequency (Hz)")
plt.show()
```

```python
res.x/res.x[0]
```

    array([ 1.        ,  1.78627675,  0.84976028,  1.00034299, 11.024541  ,
           35.31604716, 25.492815  ])

Looks great, right? But, this is not typically that easy. This above case was easy because I have set the frequency array to logspace whereas we typically get a linspace array from fourier transform of linearly spaced time domain data!

Now let's see what happens if we switch back to linspace.

```python
f = np.linspace(0.001, 1000, 100000)
s = 1j*2*np.pi*f
tf = (s**2 + 3*s + 2) / (s**3 + 12*s**2 + 47*s + 60)

xdata = f
ydata = tf
model = kontrol.curvefit.model.TransferFunctionModel(nzero=2, npole=3, log_args=False)
error_func = kontrol.curvefit.error_func.tf_error
cost = kontrol.curvefit.Cost(error_func=error_func)
optimizer = scipy.optimize.minimize

x0 = np.ones(7)  # There are 7 coefficients, 3 for numerator and 4 for denominator
optimizer_kwargs = {"x0": x0}

a = kontrol.curvefit.CurveFit()
a.xdata = xdata
a.ydata = ydata
a.model = model
a.cost = cost
a.optimizer = optimizer
a.optimizer_kwargs = optimizer_kwargs
res = a.fit()
```

```python
plt.figure(figsize=(15, 5))
plt.subplot(121)
plt.loglog(f, abs(tf), label="Data")
plt.loglog(f, abs(a.model(f, x0)), label="Initial guess")
plt.loglog(f, abs(a.yfit), label="Fit")
plt.legend(loc=0)
plt.grid(which="both")
plt.ylabel("Amplitude")
plt.xlabel("Frequency (Hz)")
plt.subplot(122)
plt.semilogx(f, np.angle(tf), label="Data")
plt.semilogx(f, np.angle(a.model(f, x0)), label="Initial guess")
plt.semilogx(f, np.angle(a.yfit), label="Fit")
plt.legend(loc=0)
plt.grid(which="both")
plt.ylabel("Phase (rad)")
plt.xlabel("Frequency (Hz)")
plt.show()
```

Not great! Terrible!

It turns out that there're a few tricks that can be used fit linspace transfer function data.

1. Weighing function (1/f).
2. Parameter scaling (log).
3. Use other optimizers (differential evolution, Nelder-Mead, Powell, etc).
4. Use better initial guess.
5. Tighter acceptable tolerance for convergence during minimization (e.g. ``ftol``, ``xtol``. Different optimizers use different convention)

We will use all of them!

```python
a = kontrol.curvefit.CurveFit()
a.xdata = f
a.ydata = tf
# Use 1/f weighting
weight = 1/f
error_func_kwargs = {"weight": weight}
a.cost = kontrol.curvefit.Cost(error_func=error_func, error_func_kwargs=error_func_kwargs)

# Use log scaling. Models have an argument for enabling that.
a.model = kontrol.curvefit.model.TransferFunctionModel(nzero=2, npole=3, log_args=True)

# Use better initial guess and Nelder-Mead optimizer.
np.random.seed(123)
true_args = np.array([1, 3, 2, 1, 12, 47, 60])  # These are the true parameters
noise = np.random.normal(loc=0, scale=true_args/10, size=7)
x0 = true_args + noise  # Now the initial guess is assumed to be some deviation from the true values.
# x0 = np.ones(7)
a.optimizer_kwargs = {"x0": np.log10(x0), "method": "powell", "options": {"xtol": 1e-12, 'ftol': 1e-12}}  # Note the inital guess is log.
a.optimizer = scipy.optimize.minimize
res = a.fit()
```

```python
10**a.optimize_result.x
```

    array([ 2.25124892,  3.66597884,  1.51466572,  2.25137845, 23.92566764,
           77.41975222, 45.43996825])

```python
plt.figure(figsize=(15, 5))
plt.subplot(121)
plt.loglog(f, abs(tf), label="Data")
plt.loglog(f, abs(a.model(f, np.log10(x0))), label="Initial guess")
plt.loglog(f, abs(a.yfit), label="Fit")
plt.legend(loc=0)
plt.grid(which="both")
plt.ylabel("Amplitude")
plt.xlabel("Frequency (Hz)")
plt.subplot(122)
plt.semilogx(f, np.angle(tf), label="Data")
plt.semilogx(f, np.angle(a.model(f, np.log10(x0))), label="Initial guess")
plt.semilogx(f, np.angle(a.yfit), label="Fit")
plt.legend(loc=0)
plt.grid(which="both")
plt.ylabel("Phase (rad)")
plt.xlabel("Frequency (Hz)")
plt.show()
```

```python
fitted_args = 10**a.optimized_args
fitted_args /= fitted_args[0]
fitted_args
```

    array([ 1.        ,  1.62842003,  0.6728113 ,  1.00005754, 10.62773089,
           34.38968995, 20.18433765])

```python
print("True arguments: ", true_args)
print("Initial arguments: ", x0/x0[0])
print("Arguments optained from fitting: ", fitted_args)
```

    True arguments:  [ 1  3  2  1 12 47 60]
    Initial arguments:  [ 1.          3.70099497  2.30705685  0.95281056 12.68253445 61.43087558
     50.97379581]
    Arguments optained from fitting:  [ 1.          1.62842003  0.6728113   1.00005754 10.62773089 34.38968995
     20.18433765]

```python
print("True transfer function")
s = control.tf("s")
tf = (s**2 + 3*s + 2) / (s**3 + 12*s**2 + 47*s + 60)
tf
```

    True transfer function

$$\frac{s^2 + 3 s + 2}{s^3 + 12 s^2 + 47 s + 60}$$

```python
print("Fitted transfer function")
a.model.args = a.optimized_args
a.model.tf.minreal()
```

    Fitted transfer function

$$\frac{0.9999 s^2 + 1.628 s + 0.6728}{s^3 + 10.63 s^2 + 34.39 s + 20.18}$$

The fit looks good.
But, apparently it has found a set of different parameters.
See further tutorials for alternative ways of fitting the same transfer function.

Nevertherless, we demonstrated the use of ``kontrol.curvefit.CurveFit`` for transfer function fitting.


\iffalse

```
from scipy import *
from scipy.integrate import quad

from IPython.core.display import HTML
def css_styling():
    styles = open("custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
    }
    div.cell{
        width:800px;
        margin-left:10% !important;
        margin-right:auto;
    }
    h1 {
        font-family: Candara, Cambria, serif;
    }
    h2 { text-align:center; }
    h4{
        margin-top:12px;
        margin-bottom: 3px;
       }
    div.text_cell_render{
        font-family: Candara, Calibri, Trebuchet, Geneva, sans-serif;
        line-height: 155%;
        font-size: 130%;
        width:800px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            background-color: #FDFCFF;
            font-family:"Aurulent Sans Mono","Source Code Pro", source-code-pro,Consolas, monospace;
    }
    .prompt{
        display: None;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 16pt;
        color: #3035A1;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }

    BODY {background-color: rgb( 252, 255, 220 )}

</style>

\fi
# The water tower

The tower is in reinforced concrete,

```
g  = 2500.0
E  = 30E9
g0 = 9.80665 # standard acceleration of gravity
```

the heigth of the tower `H` and the lumped mass `M` at its top (representing the
mass of the tank and of its content) are respectively

```
H = 45.0
M = 1.2E6
```

the cross-section is annular, the outer radius varies linearly from `R0` to `RH`
and the thickness varies linearly from `T0` to `TH`

```
R0 = 3.20 ; RH = 2.40
T0 = 0.25 ; TH = 0.20
```

We can define the radius, the thickness, the cross section area and flexural
inertia of the section as functions of the vertical coordinate `z`,

```
def R(z):
    return R0+(RH-R0)*z/H

def T(z):
    return T0+(TH-T0)*z/H

def A(z):
    t = T(z)
    return pi*(2*t*R(z)-t*t)

def J(z):
    r_ext = R(z)
    r_int = r_ext - T(z)
    return pi/4*(r_ext**4-r_int**4)
```

## Total mass of the beam

We are ready to compute the total mass of the beam and to print aside of the lumped mass, so that we can judge the relevance of the distributed mass on our results.

To compute the beam mass, `mass`, we have to integrate $\gamma A(z)\,\text{d}z$
over the length of the beam. To this purpose we use the library function
`quad` that has 3 mandatory arguments:

  1. a function, library or user defined, to be numerically integrated,
  2. the lower limit of the interval of integration,
  3. the upper limit of the interval of integration.

The integrand is here _defined on the spot_ using the `lambda` syntax
(in `matlab` it is `@` syntax).

Note that the function `quad` returns both the definite integral value
and an estimate of the error, and that we discard the error estimate
by assigning it to the dummy variable `_`.

```
mass, _ = quad(lambda z:A(z)*g, 0, H)
print M, mass
print mass/M
```

    1200000.0 429710.970149
    0.358092475124

The mass of the beam represent a significant fraction of the lumped mass,
we can expect a small, but significant correction from the naive estimate
$\omega^2 \approx EJ/3ML^3$

# Choice of the shape function

I will simply use the one-minus-quarter-cosine shape function, no discussion, sorry...

\begin{align}
    \psi(z)   & = 1 - \cos\left(\frac\pi2\frac zH\right),\\
    \psi'(z)  & = \frac\pi{2H} \sin\left(\frac\pi2\frac zH\right),
    &\delta(z) &= \int_0^z \psi'^2(s) ds =
    \frac18\frac\pi H\left(\frac\pi H z- \sin\left(\frac\pi Hz\right)\right),\\
    \psi''(z) & = \frac{\pi^2}{4H^2} \cos\left(\frac\pi2\frac zH\right).
\end{align}

### Definitions

We define here the shape function and its derivatives
in terms of `Z`, a constant proportional to the wave length.

While we're at it,
$\int_0^z \psi'^2(s)\,\text{d}s= \frac{\pi}{8H} \left(\frac\pi{H}z
    - \sin\left(\frac\pi{H}z\right)\right)$
is the increment of vertical displacement at quote `z` for a unit $\delta Z_0$.

```
Z = 2*H/pi

def psi0(z): return 1-cos(z/Z)
def psi1(z): return sin(z/Z)/Z
def psi2(z): return cos(z/Z)/Z/Z
def de_u(z):  return pi*(-sin(pi*z/H) + pi*z/H)/(8*H)
```

# Generalised properties

We compute the generalized mass and stiffness, and also the contributions to the geometric stiffness due to the lumped mass
and the distributed mass

```
mstar = quad(lambda z: psi0(z)**2*A(z)*g, 0, H)[0]
kstar = quad(lambda z: psi2(z)**2*E*J(z), 0, H)[0]
kgeoM = quad(lambda z: psi1(z)**2,        0, H)[0]*M*g0
kgeom = quad(lambda z: de_u(z)*A(z)*g,    0, H)[0]*g0
```

# Results

Using a helper function, we display the results in terms of eigenvalues, frequencies, periods under different assumptions
regarding the geometrical stiffness

```
def pr_res(s,w2):
    wn = sqrt(w2)
    fn = wn/2/pi
    Tn = 1.0/fn
    print "%36s%12.6f%8.5f%8.5f%8.5f" % (s, w2, wn, fn, Tn)
    return None

print
print "                 k*:", kstar,"N/m"
print "                 m*:", mstar,"kg"
print "                  M:    ", M,"kg"
print "           k_G*M*g0:", kgeoM,"N/m"
print "    g0 \int de_u dm: ", kgeom,"N/m"

print
print " "*39+"      w^2 "+"      w "+"      f "+"      T"
pr_res("k*/M ", kstar/M)
pr_res("k*/(M+m*)", kstar/(M+mstar))
pr_res("(k*-kg*M*g0)/(M+m*)", (kstar-kgeoM)/(M+mstar))
pr_res("(k*-kg M g0-\\int(kg dm)g0)/(M+m*)", (kstar-kgeoM-kgeom)/(M+mstar))
```

                     k*: 17347995.0148 N/m
                     m*: 81844.8215573 kg
                      M:     1200000.0 kg
               k_G*M*g0: 322625.853333 N/m
        g0 \int de_u dm:  29517.750381 N/m

                                                 w^2       w       f       T
                                   k*/M    14.456663 3.80219 0.60514 1.65252
                               k*/(M+m*)   13.533616 3.67881 0.58550 1.70794
                     (k*-kg*M*g0)/(M+m*)   13.281927 3.64444 0.58003 1.72405
       (k*-kg M g0-\int(kg dm)g0)/(M+m*)   13.258899 3.64128 0.57953 1.72554

## Comments

We have a large dependency of $\omega^2$ on the account of distributed mass, a small but not negligible dependency
on the geometrical stiffness associated with the weight of the lumped mass and a very small dependency on the geometrical stiffness
associated with the distributed weight of the beam