# Lecture 4: Linear systems

## Syllabus
**Week 1:** Python intro
**Week 2:** Matrices, vectors, norms, ranks
**Week 3:** Linear systems, eigenvectors, eigenvalues

##Recap of the previous lecture
- Scalar product
- Unitary/orthogonal matrices and norm conservation
- Matrix rank definition
- Skeleton approximation and dyadic representation of a rank-$r$ matrix
- Singular value decomposition

## Today lecture
Today we will talk about:
- Linear systems, inverse matrices
- Gaussian elimination
- Sparse matrices
- Condition numbers

Before that I will spend a few minutes on certain techinal notes on the HW1.

## Use numpy arrays in homework!
-  Use
 ```
     a = np.zeros((n, n)) #Note for the double brackets
 ```
 instead of
 ```
     a = [[0 for i in xrange(n)] for j in xrange(n)]
 ```
 In this case you can not add arrays like
 ```
     a = a + a
 ```
 and have to write down functions/cycles instead one line

- Use numpy arrays indexing like ```a[i, j]``` instead of ```a[i][j]```
- Use ```%matplotib inline``` and ```plt.plot()``` (without ```plt.show()```) in the Notebook
- For code writing, there is a [PEP8 style guide for Python code](http://legacy.python.org/dev/peps/pep-0008/) - try to follow

## Motivational video on algebra
And before we start - a very simple and intuitive video on algebra. It also contains a short ove

```
from IPython.display import YouTubeVideo
YouTubeVideo('8CX-Q0gtSp8')
```

## Linear equations and matrices
A linear system of equations can be written in the form
\begin{equation}
\begin{split}
    &2 y + 3 x = 5 \longrightarrow \quad &3x + 2 y + 0 z = 5\\
    &2 x + 3z = 5 \longrightarrow\quad &2 x + 0 y + 3 z = 5\\
    &x + y = 2 \longrightarrow\quad  & 1 x + 1 y + 0 z = 2\\
\end{split}
\end{equation}

##Matrix form

$$
\begin{pmatrix}
3 & 2 & 0 \\
2 & 0 & 3 \\
1 & 1 & 0 \\
\end{pmatrix}\begin{pmatrix}
x \\
y \\
z
\end{pmatrix} =
\begin{pmatrix}
5 \\
5 \\
2
\end{pmatrix}
$$
The "matrix form" is
$$
A x = f,
$$

where $A$ is a $3 \times 3$ matrix.

## Linear systems in applications
Linear systems are everywhere - in modelling, even if you have nonlinear systems, by **linearization** they are reduced to a sequence of linear systems. They appear in different applications:
- Circuit modelling (Kirchoffs law)
- Photonics (Maxwell equation, electrodynamics)
- Computational fluid dynamics (Navier-Stokes equation)
- +$\infty$ more

## Linear systems are big
We take a continious problem, discretize on a mesh with $N$ elements and get a linear system.
Example of a mesh around A319 aircraft
(taken from [GMSH website](http://geuz.org/gmsh/)).

The main difficulty is that these systems are big: millions or billions of unknowns!

## Linear systems are structured
Storing $N^2$ elements of a matrix is prohibitive even for $N = 100000$. How to work with such matrices?
Fortunately, those matrices are **structured** and require $\mathcal{O}(N)$ parameters to be stored.
The most widespread structure are **sparse matrices**: in the matrix there are $\mathcal{O}(N)$ non-zeros!
Example (one of the famous matrices around for $n = 5$):
$$
  \begin{pmatrix}
  2 & -1 & 0 & 0 & 0 \\
  -1 & 2 & -1 & 0 & 0 \\
  0 & -1 & 2 & -1 & 0 \\
  0 & 0 &-1& 2 & -1  \\
  0 & 0 & 0 & -1 & 2 \\
  \end{pmatrix}
$$
At least you can store such matrices, and multiply by vector fast; but how to solve linear systems (and that is typically the final goal).

## How to solve linear systems
**Important**: forget about determinants and the **Cramer rule** (it is  are good for $2 \times 2$ matrices still).

## How to solve linear systems

The main tool is variable elimination.
\begin{equation}
\begin{split}
    &2 y + 3 x = 5 \longrightarrow \quad &y = 5/2 -  3/2 x \\
    &2 x + 3z = 5 \longrightarrow\quad &z = 5/3 - 2/3 x\\
    &x + y = 2 \longrightarrow\quad  & 5/2 + 5/3 - (3/2 + 2/3) x = 2,\\
\end{split}
\end{equation}
and that is how you find $x$ (and all previous ones).
This process is called **Gaussian elimination** and is one of the most widely used algorithms.

## Gaussian elimination
Gaussian elimination consists of two steps:
1. Forward step
2. Backward step

## Forward step
In the forward step, we eliminate $x_1$:
$$
   x_1 = f_1 - (a_{12} x_2 + \ldots + a_{1n} x_n)/a_{11},
$$
and then substitute this into the equations $2, \ldots, n$.
Then we eliminate $x_2$ and so on from the second equation. The important thing is that the **pivots** (that we divide over) are not equal to $0$.

## Backward step
In the backward step, we solve equation for $x_n$, put it into the equation for $x_{n-1}$ and so on, until we
compute all $x_i, i=1,\ldots, n$.

## Complexity of the Gaussian elimination
Each elimination step requires $\mathcal{O}(n^2)$ operations. Thus, the cost of the naive algorithm is $\mathcal{O}(n^3)$.
**Think a little bit**: Can Strassen help here?

## A short demo
Now we can do a short demo (with a Hilbert matrix)

```
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

n = 500
a = [[1.0/(i + j + 1) for i in xrange(n)] for j in xrange(n)]
a = np.array(a)
rhs = np.ones(n) #Right-hand side
x = np.linalg.solve(a, rhs)

#And check if everything is fine
er = np.linalg.norm(a.dot(x) - rhs) / np.linalg.norm(rhs)
print er
plt.xkcd()
plt.plot(x)
plt.title('Yea')
```

As you see, the error grows with larger $n$, and we have to find out why.
**Important point** is that it is not a problem of the algorithm: it is a problem of representing
the matrix in the memory. The error occurs in the moment when the matrix elements are evaluated approximately.

## Linear systems and inverse matrix
What was the problem in the previous example? Why the error grows so quickly?
And here is one of the main concepts of numerical linear algebra: the concept of **condition number** of a matrix.
But before that we have to define the **inverse**.

The inverse of a matrix $A$ is defined as a matrix $X$ denoted by $A^{-1}$ such that
$$
   AX = XA = I,
$$
where $I$ is the identity matrix (i.e., $I_{ij} = 0$ if $i \ne j$ and $1$ otherwise).
The computation of the inverse is linked to the solution of linear systems.  Indeed, $i$-th column of the product gives
$$
A x_i = e_i,
$$
where $e_i$ is the $i$-th column of the identity matrix. Thus, we can apply Gaussian elimination to solve this system. Moreover, if there are no divisions by zero in this process (and the pivots do not depend on the right-hand side), the it is possible to solve the system.

## Inverse matrix and linear systems
If we have computed $A^{-1}$, the solution of linear system
$$Ax = f$$
is just $x = A^{-1} f$.
Indeed,
$$
    A(A^{-1} f) = (AA^{-1})f = I f = f.
$$

## Neumann series
To study, why there can be such big errors in a solution (see the example above on the Hilbert matrix)  we need an important auxilary result:  **Neumann series**:
If for a matrix $\Vert F \Vert < 1$ then the matrix $(I - F)$ is invertible and
$$(I - F)^{-1} = I + F + F^2 + F^3 + \ldots = \sum_{k=0}^{\infty} F^k.$$
Note that it is a matrix version of the geometric progression.

##Proof
The proof is constructive. First of all, the show prove that the series $\sum_{k=0}^{\infty} F^k$ converges.
Like in the scalar case, we have
$$
   (I - F) \sum_{k=0}^N F^k = (I - F^{N+1}) \rightarrow I
$$
We can also estimate the **norm of the inverse**:
$$
  \Vert \sum_{k=0}^N F^k \Vert \leq \sum_{k=0}^N \Vert F \Vert^k \Vert I \Vert = \frac{\Vert I \Vert}{I - \Vert F \Vert}
$$

## Small perturbation of the inverse
Using this result, we can estimate, how the perturbation of the matrix influences the inverse matrix. We assume
that the perturbatin $E$ is small in the sense that $\Vert A^{-1} E \Vert < 1$. Then
$$(A + E)^{-1} = \sum_{k=0}^{\infty} (-A^{-1} E)^k A^{-1}$$
and moreover,
$$
  \frac{\Vert (A + E)^{-1} - A^{-1} \Vert}{\Vert A^{-1} \Vert} \leq \frac{\Vert A^{-1} \Vert \Vert E \Vert}{1 - \Vert A^{-1} E \Vert}.
$$
As you see, the norm of the inverse enters the estimate.

## Condition number of a linear system
Now consider the **perturbed** linear system:
$$
   (A + \Delta A) \widehat{x} = f + \Delta f.
$$
Then the algebra begins!

##Estimates
$\widehat{x} - x = (A + \Delta A)^{-1} (f + \Delta f) - A^{-1} f = ((A + \Delta A)^{-1} - A^{-1})f + (A + \Delta A)^{-1} \Delta f = $
$ = \Big[\sum_{k=0}^{\infty} (-A^{-1} \Delta A)^k\Big] A^{-1} f + \Big[\sum_{k=0}^{\infty} (A^{-1} \Delta A)^k \Big] A^{-1} \Delta f,$
therefore
$\frac{\Vert \widehat{x} - x \Vert}{\Vert x \Vert} \leq \frac{\Vert A \Vert \Vert A^{-1} \Vert}{1 - \Vert A^{-1}\Delta A\Vert}\Big(\frac{\Vert\Delta A\Vert}{\Vert A \Vert} + \frac{\Vert \Delta f \Vert}{ \Vert f \Vert}\Big)$.

The crucial role is played by the **condition number** $\mathrm{cond}(A) = \Vert A \Vert \Vert A^{-1} \Vert$.
The larger the condition number, the less number of digits we can recover.

The spectral norm of the matrix is equal to **largest singular value**, and the singular values of the inverse matrix are equal to the inverses of the singular values.  Thus,  the condition number is equal to the ratio of the largest singular value and the smallest singular value.
$$
    \mathrm{cond(A)} = \Vert A \Vert \Vert A^{-1} \Vert = \frac{\sigma_{\max}}{\sigma_{\min}}
$$

## Hilbert matrix (again)
We can also try to test how tight is the estimate, both with ones in the right-hand side, and with a random vector in the right-hand side. The results are strickingly different

```
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

n = 100
a = [[1.0/(i + j + 1) for i in xrange(n)] for j in xrange(n)]
a = np.array(a)
rhs = np.ones(n) #Right-hand side
f = np.linalg.solve(a, rhs)

#And check if everything is fine
er = np.linalg.norm(a.dot(f) - rhs) / np.linalg.norm(rhs)
cn = np.linalg.cond(a)
print 'Error:', er, 'Condition number:', cn
```

    Error: 2.81450551264e-08 Condition number: 3.33972881047e+19

And with random right-hand side...

```
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

n = 500
a = [[1.0/(i + j + 1) for i in xrange(n)] for j in xrange(n)]
a = np.array(a)
rhs = np.random.randn(n) #Right-hand side
f = np.linalg.solve(a, rhs)

#And check if everything is fine
er = np.linalg.norm(a.dot(f) - rhs) / np.linalg.norm(rhs)
cn = np.linalg.cond(a)
print 'Error:', er, 'Condition number:', cn
```

    Error: 292.997678103 Condition number: 3.5650102203e+20

** Can you think about an explanation?**

Todays lecture
- Gaussian elimination
- Condition number

##### Questions?

```
from IPython.core.display import HTML
def css_styling():
    styles = open("./styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
    }
    div.cell{
        /*width:80%;*/
        /*margin-left:auto !important;
        margin-right:auto;*/
    }
    h1 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    h2 {
        font-family: 'Fenix', serif;
    }
    h3{
		font-family: 'Fenix', serif;
        margin-top:12px;
        margin-bottom: 3px;
       }
	h4{
		font-family: 'Fenix', serif;
       }
    h5 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    div.text_cell_render{
        font-family: 'Alegreya Sans',Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 1.2;
        font-size: 160%;
        /*width:70%;*/
        /*margin-left:auto;*/
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro";
			font-size: 90%;
    }
/*    .prompt{
        display: None;
    }*/
    .text_cell_render h1 {
        font-weight: 200;
        font-size: 50pt;
		line-height: 100%;
        color:#CD2305;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
        display: block;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 16pt;
        color: #CD2305;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>


```python
import cmath, random, numpy
import functools
import  matplotlib.pyplot as plt
import sys
import os
import math
from qutip import*

from sympy import*
#from sympsi import*
from scipy import optimize

from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
import time
import math
from qutip import *
from qutip.ipynbtools import plot_animation
import numpy as np
import matplotlib.pyplot as plt
import qutip
%matplotlib inline
import matplotlib.pylab as plt
import matplotlib as mpl
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
from IPython.display import display, Math, Latex
import cmath
from mpl_toolkits.axes_grid1 import AxesGrid
from scipy.special import factorial
```

```python
"""Define the operators for MZI - we will deal in Heisenberg """
T = Symbol('T')
xvec = np.arange(-30.,30.)*5./40
X,Y = np.meshgrid(xvec, xvec)  ##Some plotting params
N_dim = 35##Dimenstion of the Hilbert spac

a1 = destroy(N_dim) ##This is for single-photon field
a2 = destroy(N_dim) ##for coherent field
a3 = destroy(N_dim) ##for vacuum field
def n_choose_k(n,k):
    return factorial(n)/(factorial(n-k)*factorial(k))
def D(state,alpha):
    Rho_new=displace(N_dim,alpha)*state*displace(N_dim,alpha).dag()
    return Rho_new
'''Define a rotation in phase space, or phase shifter operation'''
def Phase(theta):
    b=-1j*theta*a1.dag()*a1;
    return b.expm()
'''Squeezing operation, inputs a density matrix and outputs the squeezed density matrix for squeezing parameter r'''
def Sq(state,r):
    Rho_new=squeeze(N_dim,r)*state*squeeze(N_dim,r).dag();
    return Rho_new
'''The function below creates a beamsplitter operation that acts on two modes.
The value for k determines what number Fock state could be filtered out of the first state
based on a single photon input for the second BS port, followed by single photon detection.'''
def BS_operator_filtering(a1, a2, k):
    theta_k = np.arctan(1/np.sqrt(k))
    T = np.sin(theta_k)*np.sin(theta_k)
    R = np.cos(theta_k)*np.cos(theta_k)
    print('I am filtering', k, 'and:', theta_k*180/math.pi)
    print('BS T is : ', T, 'and : ', R)
    b = theta_k*(tensor(a1,a2.dag()) - tensor(a1.dag(),a2))
    return b.expm()
def SSV_plus(r,alpha):
    state = ket2dm((displace(N_dim,alpha)+displace(N_dim,-alpha))*squeeze(N_dim,r)*fock(N_dim,0))
    norm_state = state/state.tr()
    return norm_state
def SSV_minus(r,alpha):
    state = ket2dm((displace(N_dim,alpha)-displace(N_dim,-alpha))*squeeze(N_dim,r)*fock(N_dim,0))
    norm_state = state/state.tr()
    return norm_state
def cat_plus(alpha):
    cat = (1/(np.sqrt(2)*np.sqrt(1+np.e**(-alpha*alpha.conj()))))*(coherent(N_dim,-alpha)+(coherent(N_dim,alpha)))
    return cat
def cat_minus(alpha):
    cat = (1/(np.sqrt(2)*np.sqrt(1-np.e**(-alpha*alpha.conj()))))*(-coherent(N_dim,-alpha)+(coherent(N_dim,alpha)))
    return cat
def pnr_resolution_detector(eta, click, n_truc):
    pi_n = 0;
    l = np.arange(click,n_truc)
    for i in l:
        pi_n +=  n_choose_k(i,click)*math.pow((1-eta),(i-click))*math.pow(eta,click)*fock(N_dim,i)*fock(N_dim,i).dag()
        #print("The final Povm element is: ", pi_0)
    return Qobj(pi_n)
def Fock_Filter_povm(in_state,in_fock,refl,num_det,eta,n_truc):
    Projector = tensor(pnr_resolution_detector(eta, num_det, n_truc),qeye(N_dim));
    Initial_state=tensor(in_state,ket2dm(fock(N_dim,in_fock)));
    theta_k=np.arccos(np.sqrt(refl));

    BS1= ((theta_k)*(tensor(a1,a2.dag()) - tensor(a1.dag(),a2))).expm()

    Rho=BS1*Initial_state*BS1.dag();

    Rho_filtered = ((Rho*Projector).ptrace(1))/((Rho*Projector).tr())
    '''The operation .ptrace(m) takes the partial trace over every mode EXCEPT m, where the numbering
    startes at 0.  So .ptrace(1) means you keep mode 1, which is actually the 2nd mode'''
    print('BS has reflectivity',refl,' and I am detecting the |',num,'> state, where my detector has efficiency', eta)
    return Rho_filtered
def Fock_Filter_prob(in_state,in_fock,refl,num_det,eta,n_truc):
    Projector = tensor(pnr_resolution_detector(eta, num_det, n_truc),qeye(N_dim));
    Initial_state=tensor(in_state,ket2dm(fock(N_dim,in_fock)));
    theta_k=np.arccos(np.sqrt(refl));

    BS1= ((theta_k)*(tensor(a1,a2.dag()) - tensor(a1.dag(),a2))).expm()

    Rho=BS1*Initial_state*BS1.dag();
    P=(Rho*Projector).tr()
    print('The probability of a sucessful detection is:',P)
    Rho_filtered = ((Rho*Projector).ptrace(1))/((Rho*Projector).tr())
    #Rho_filtered=Rho*Projector
    '''The operation .ptrace(m) takes the partial trace over every mode EXCEPT m, where the numbering
    startes at 0.  So .ptrace(1) means you keep mode 1, which is actually the 2nd mode'''
    print('BS has reflectivity',refl,' and I am detecting the |',num,'> state, where my detector has efficiency', eta)
    return Rho_filtered

def fid(state1,state2):
    F=np.absolute((state1.sqrtm()*state2*state1.sqrtm()).sqrtm().tr())
    return F

```

```python
'''Two-step, check: {

 {P -> 0.0152557, 0.999653, r[2.] -> 0.852986,
  r[1.] -> 0.603152, \[Alpha] -> 0.901576,
  sq -> -0.224664, \[Beta] -> 0.5, delta -> 1.19655, n1 -> 1,
  n2 -> 2}
}'''
refl1=0.603152**2; refl2=0.852986**2; delta=1.19655; beta=0.5; sq=0.224664; alpha=0.901576; n1=1;n2=2
```

```python
ssv=SSV_plus(sq,alpha)
#ssv=SSV_minus(0,1)

W_ssv=wigner(ssv,xvec,xvec);
eta=1;

first=Fock_Filter_prob(ket2dm(coherent(N_dim,delta)),1,refl1,n1,eta,N_dim)
W1=wigner(first,xvec,xvec);

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W_ssv, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W_ssv, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Wigner function of ssv state')

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W1, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W1, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Wigner function of filtered state')
fid = fidelity(ssv,first)
print('fidelity with ssv:',fid*fid)

fig, axes = plt.subplots(1, 3, figsize=(12,4))
bar0 = axes[0].bar(range(N_dim), first.diag())

lbl0 = axes[0].set_title("Filtered State")

lim0 = axes[0].set_xlim([-.5, 20])
bar1 = axes[1].bar(range(N_dim), ssv.diag())

lbl1 = axes[1].set_title("SSV")

lim1 = axes[1].set_xlim([-.5, 20])

bar2 = axes[2].bar(range(N_dim), ket2dm(coherent(N_dim,delta)).diag())

lbl2 = axes[2].set_title("Initial Coherent State")

lim2 = axes[2].set_xlim([-.5, 20])

plt.show()
```

```python
second=Fock_Filter_prob(first,1,refl2,n2,eta,N_dim);
W2=wigner(second,xvec,xvec);

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W2, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W2, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Wigner function of filtered state')
fid = fidelity(ssv,second)
print('fidelity with cat:',fid*fid)

fig, axes = plt.subplots(1, 3, figsize=(12,4))
bar0 = axes[0].bar(range(N_dim), second.diag())

lbl0 = axes[0].set_title("Twice Filtered State")

lim0 = axes[0].set_xlim([-.5, 20])
bar1 = axes[1].bar(range(N_dim), first.diag())

lbl1 = axes[1].set_title("First Filtered State")

lim1 = axes[1].set_xlim([-.5, 20])

bar2 = axes[2].bar(range(N_dim), ket2dm(coherent(N_dim,delta)).diag())

lbl2 = axes[2].set_title("Initial Coherent State")

lim2 = axes[2].set_xlim([-.5, 20])
plt.show()
```

```python
final_state=D(second,beta*(-1))

W_final=wigner(final_state,xvec,xvec)

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W_ssv, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W_ssv, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Wigner function of squeezed cat state')

fig = plt.figure(figsize=(16,7))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot
ax = fig.add_subplot(2, 2, 2)
p = ax.contourf(X, Y, W_final, 20, cmap=cm.RdBu)
cb = fig.colorbar(p, shrink = 0.7)
# surface_plot with color grading and color bar
ax = fig.add_subplot(2, 2, 1, projection='3d')
p = ax.plot_surface(X, Y, W_final, rstride=1, cstride=1, cmap=cm.RdBu, linewidth=0.5)
cb = fig.colorbar(p,shrink = .7)
plt.title('Displaced Wigner function of |5>, |3> and |2> filtered coherent state')
fid = fidelity(ssv,final_state)
print('fidelity with ssv:',fid*fid)
plt.show()
fig, axes = plt.subplots(1, 5, figsize=(16,4))

bar0 = axes[0].bar(range(N_dim), ket2dm(coherent(N_dim,delta)).diag())

lbl0 = axes[0].set_title("Initial Coherent State")

lim0 = axes[0].set_xlim([-.5, 10])

bar1 = axes[1].bar(range(N_dim), first.diag())

lbl1 = axes[1].set_title("First Filtered")

lim1 = axes[1].set_xlim([-.5, 10])

bar2 = axes[2].bar(range(N_dim), second.diag())

lbl2 = axes[2].set_title("Twice Filtered State")

lim2 = axes[2].set_xlim([-.5, 10])

bar3 = axes[3].bar(range(N_dim), final_state.diag())

lbl3 = axes[3].set_title("Final")

lim3 = axes[3].set_xlim([-.5, 10])

bar4 = axes[4].bar(range(N_dim), ssv.diag())

lbl4 = axes[4].set_title("SSV")

lim4 = axes[4].set_xlim([-.5, 10])
plt.show()

```

```python
from mpl_toolkits.axes_grid1 import AxesGrid

def shiftedColorMap(cmap, start=1, midpoint=0, stop=0, name='shiftedcmap'):
    '''
    Function to offset the "center" of a colormap. Useful for
    data with a negative min and positive max and you want the
    middle of the colormap's dynamic range to be at zero.

    Input
    -----
      cmap : The matplotlib colormap to be altered
      start : Offset from lowest point in the colormap's range.
          Defaults to 0.0 (no lower offset). Should be between
          0.0 and `midpoint`.
      midpoint : The new center of the colormap. Defaults to
          0.5 (no shift). Should be between 0.0 and 1.0. In
          general, this should be  1 - vmax / (vmax + abs(vmin))
          For example if your data range from -15.0 to +5.0 and
          you want the center of the colormap at 0.0, `midpoint`
          should be set to  1 - 5/(5 + 15)) or 0.75
      stop : Offset from highest point in the colormap's range.
          Defaults to 1.0 (no upper offset). Should be between
          `midpoint` and 1.0.
    '''
    cdict = {
        'red': [],
        'green': [],
        'blue': [],
        'alpha': []
    }

    # regular index to compute the colors
    reg_index = np.linspace(start, stop, 257)

    # shifted index to match the data
    shift_index = np.hstack([
        np.linspace(0, midpoint, 128, endpoint=False),
        np.linspace(midpoint, 1, 129, endpoint=True)
    ])

    for ri, si in zip(reg_index, shift_index):
        r, g, b, a = cmap(ri)

        cdict['red'].append((si, r, r))
        cdict['green'].append((si, g, g))
        cdict['blue'].append((si, b, b))
        cdict['alpha'].append((si, a, a))

    newcmap = cm.colors.LinearSegmentedColormap(name, cdict)
    plt.register_cmap(cmap=newcmap)

    return newcmap
#from matplotlib import rc,rcParams
#from pylab import *
```

```python
#activate latex text rendering
#rc('text', usetex=True)
#rc('axes', linewidth=2)
#rc('font', weight='bold')
#rcParams['text.latex.preamble'] = [r'\usepackage{sfmath} \boldmath']

xvec = np.arange(-28.,28.)*5./40
yvec = np.arange(-35.,35.)*5./40
X,Y = np.meshgrid(xvec, xvec)  ##Some plotting params
W=wigner(final_state,xvec,xvec);
orig_cmap = cm.seismic

shifted_cmap1 = shiftedColorMap(orig_cmap, midpoint=1-(W.max()/(W.max()-W.min())), name='shifted')

fig = plt.figure(figsize=(8,5))
# `ax` is a 3D-aware axis instance, because of the projection='3d' keyword argument to add_subplot

ax1 = fig.add_subplot(1,1,1, projection='3d')
p = ax1.plot_surface(X, Y, W, rstride=1,cstride=1, cmap=shifted_cmap1, linewidth=0.5)
#lim=ax1.set_xlim([-4,2])
#lim=ax1.set_ylim([-1,1])

cb = fig.colorbar(p, shrink = 0.5)
cb.ax.tick_params(labelsize=12)
cb.set_ticks([-0.1, 0,0.1,0.2,0.3]);

ax1.contour(X, Y, W, cmap=shifted_cmap1, linestyles="solid", offset=-0.15)
ax1.set_xlim([-4,4])
ax1.set_ylim([-4,4])
ax1.set_zticks([-0.1,0,0.1,0.2,0.3])

ax1.view_init(15, -40)
ax1.contour(X, Y, W,[0], zdir='y', cmap=cm.bwr, offset=4)
ax1.contour(X, Y, W,[0], zdir='x', cmap=cm.RdBu, offset=-4)
#plt.axis('off')
ax1.grid(b=False)
#plt.xlabel('P',fontsize=16,fontweight='heavy')
plt.xlabel('P',fontsize=16,fontweight='heavy')
plt.ylabel('X',fontsize=16,fontweight='heavy')
ax1.xaxis.set_tick_params(labelsize=12)
ax1.yaxis.set_tick_params(labelsize=12)
ax1.zaxis.set_tick_params(labelsize=12)

fig2 = plt.figure(figsize=(8,4))
ax1=fig2.add_subplot(1,2,1)
ax1.bar(range(N_dim),final_state.diag())
lim1 = ax1.set_xlim([-.5, 10.5])
lim1y = ax1.set_ylim([0,0.7])
plt.ylabel('Probability',fontsize='large')
plt.xlabel('Photon Number',fontsize='large')
plt.title('Approximate SSV')

ax2=fig2.add_subplot(1,2,2)
ax2.bar(range(N_dim),ssv.diag())
lim2 = ax2.set_xlim([-.5, 10.5])
lim2y = ax2.set_ylim([0,0.7])
plt.xlabel('Photon Number',fontsize='large')
plt.title('Ideal SSV')
#plt.subplots_adjust(wspace=0)
plt.show()
```

```python
'''Now plot the density matrix elements of the ideal state with that of
the approximate state resulting from photon-catalysis'''
s=np.abs((ssv.full()))
temp=np.delete(s,np.s_[10:],0)
state=np.delete(temp,np.s_[10:],1)

s2=np.abs((final_state.full()))
temp1=np.delete(s2,np.s_[10:],0)
state2=np.delete(temp1,np.s_[10:],1)

#plt.imshow(state)
fig = plt.figure(figsize=(6,4))
plt.pcolormesh(state, vmin=0.0, vmax=0.7)
cbar=plt.colorbar()
cbar.ax.tick_params(labelsize=14)
plt.tick_params(labelsize=14)
plt.show()

#plt.imshow(state2)
fig = plt.figure(figsize=(6,4))
plt.pcolormesh(state2, vmin=0.0, vmax=0.7)
cbar=plt.colorbar()
cbar.ax.tick_params(labelsize=14)
plt.tick_params(labelsize=14)
plt.show()

fig = plt.figure(figsize=(6,4))
plt.pcolormesh(np.abs(state2-state), vmin=0.0, vmax=0.7)
cbar=plt.colorbar()
cbar.ax.tick_params(labelsize=14)
plt.tick_params(labelsize=14)
plt.show()
```

```python
'''photon number distributions after each photon catalysis step'''

from pylab import *
rc('axes', linewidth=2)
fontsize=14
fig, axes = plt.subplots(1, 4, figsize=(16,4))

bar0 = axes[0].bar(np.arange(0, N_dim), abs(ket2dm(coherent(N_dim,delta)).diag()))

lbl0 = axes[0].set_title("Coherent State", fontsize=22)

lim0 = axes[0].set_xlim([-.5, N_dim])

bar1 = axes[1].bar(np.arange(0, N_dim), abs(first.diag()))

lbl1 = axes[1].set_title("First Step", fontsize=22)

lim1 = axes[1].set_xlim([-.5, N_dim])

bar2 = axes[2].bar(np.arange(0, N_dim), abs(second.diag()))

lbl2 = axes[2].set_title("Second Step",fontsize=22)

lim2 = axes[2].set_xlim([-.5, N_dim])

bar3 = axes[3].bar(np.arange(0, N_dim), abs(final_state.diag()))

lbl3 = axes[3].set_title("Final State",fontsize=22)

lim3 = axes[3].set_xlim([-.5, N_dim])

axes[0].set_xticks([0,1,2,3,4,5,6,7,8,9,10])
axes[1].set_xticks([0,1,2,3,4,5,6,7,8,9,10])
axes[2].set_xticks([0,1,2,3,4,5,6,7,8,9,10])
axes[3].set_xticks([0,1,2,3,4,5,6,7,8,9,10])

axes[0].set_xlim(-0.5, 6.5)
axes[1].set_xlim(-0.5, 6.5)
axes[2].set_xlim(-0.5, 6.5)
axes[3].set_xlim(-0.5, 6.5)

axes[0].set_ylim(0, 0.8)
axes[1].set_ylim(0, 0.8)
axes[2].set_ylim(0, 0.8)
axes[3].set_ylim(0, 0.8)

axes[0].tick_params(axis='both', labelsize=16)
axes[1].tick_params(axis='both', labelsize=16)
axes[2].tick_params(axis='both', labelsize=16)
axes[3].tick_params(axis='both', labelsize=16)
for tick in ax.xaxis.get_major_ticks():
    tick.label1.set_fontsize(fontsize)
    tick.label1.set_fontweight('bold')
for tick in ax.yaxis.get_major_ticks():
    tick.label1.set_fontsize(fontsize)
    tick.label1.set_fontweight('bold')

fig.tight_layout()

plt.show()
```

```python
'''photon number distributions from ideal and approximate state'''

fig, axes = plt.subplots(1, 2, figsize=(9.5,4.1))
indices = range(N_dim)
#Calculate optimal width
width = np.min(np.diff(indices))/2

bar0 = axes[0].bar(np.arange(0, N_dim), abs(ssv.diag()))

lbl0 = axes[0].set_title("Even SSV State", fontsize=22)

#ax = fig.add_subplot(111)
axes[1].bar(indices-width/2,ssv.diag(),width, color='r',label = 'Target SSV')
#bar1 = axes[1].bar(np.arange(0, N_dim),sq_cat.diag())
axes[1].bar(indices+width/2,final_state.diag(),width,label = 'Photon-Catalyzed')

plt.xticks(fontsize=14, rotation=0)
plt.yticks(fontsize=14,)
plt.legend(fontsize=13,shadow='True')
lbl1 = axes[1].set_title("Final Result", fontsize=22)

axes[0].set_xlim(-0.5, 6.5)
axes[1].set_xlim(-0.5, 6.5)

axes[0].set_ylim(0, 0.8)
axes[1].set_ylim(0, 0.8)

axes[0].set_xticks([0,1,2,3,4,5,6,7,8,9,10])
axes[1].set_xticks([0,1,2,3,4,5,6])

axes[0].tick_params(axis='both', labelsize=16)
axes[1].tick_params(axis='both', labelsize=16)

plt.show()
```

```python

```


# One-Qubit-Classifier

With the following code a linear separable 2dim dataset can be classified.

The original idea and some code snippets stem from "NISQAI: One-Qubit Quantum Classifier" by
Ryan LaRose, Yousif Almulla, Nic Ezzell, Joe Iosue, Arkin Tikku, https://github.com/QuantumAI-lib/NISQAI/tree/master/examples/one-qubit-classifier

They however used pyQuil for implementation.

The modification and transformation to Qiskit has been done by Gerhard Hellstern, ghellstern@gmx.de

```python
from qiskit import *
import numpy as np
from scipy.optimize import minimize

import matplotlib.pyplot as plt
%matplotlib inline
```

```python
from qiskit import Aer
# See a list of available local simulators
print("Aer backends: ", Aer.backends())
backend_state = Aer.get_backend('statevector_simulator')
backend_sim = Aer.get_backend('qasm_simulator')
```

    Aer backends:  [<QasmSimulator('qasm_simulator') from AerProvider()>, <StatevectorSimulator('statevector_simulator') from AerProvider()>, <UnitarySimulator('unitary_simulator') from AerProvider()>]

### Generating data

Generate 2 dimensional random date in [0,1]x[0,1]. Label the left half of the square with 1 and the second one with 0.

```python
np.random.seed(seed=12345)
n = 100
data = np.random.rand(n, 2)

label=np.zeros(n)

for p in range(0,n):
    if data[p,0] <= 0.5:
        label[p]=1

# plot the line y = x
xs = np.linspace(0, 1, 100)
ys = 0.5 * np.ones_like(xs)
plt.plot(ys, xs, '--k')

for i in range(n):
    if label[i] == 0:
        ckey = 'r'
    else:
        ckey = 'g'
    plt.scatter(data[i, 0], data[i, 1], color=ckey)

# title and axis lables
plt.title("Data Points in Unit Square", fontweight="bold", fontsize=16)

# put on a grid and show the plot
plt.grid()
plt.show()

```

### Qubit Encoding

Here we implement a method of data encoding that we call "qubit encoding." Qubit encoding works by writing two bits of classical information into one bit of quantum information. How is this done? Note that any qubit can be written

\begin{equation}
|\psi\rangle = \alpha |0\rangle + \beta |1\rangle
\end{equation}

where $\alpha, \beta \in \mathbb{C}$ satisfying $|\alpha|^2 + |\beta|^2 = 1$. Because of this normalization condition, we may equivalently write

\begin{equation}
|\psi\rangle = \cos(\theta / 2) |0\rangle + e^{i \phi} \sin(\theta / 2)|1\rangle
\end{equation}

We then encode information into the phases $0 \le \theta \le \pi$ and $ 0 \le \phi \le 2 \pi$.

For the $(x, y)$ coordinates of our points, there are several possibilities for encoding:

\begin{align}
\theta &= \pi x \\
\phi &= 2 \pi y
\end{align}

Another encoding is:

\begin{align}
\theta &= \frac{1}{2} \tan^{-1}\left( \frac{y}{x} \right) \\
\phi &= \pi (x^2 + y^2),
\end{align}

In code, we may make these transformations as follows:

```python
qdata1 = np.zeros((n, 2))

for (index, point) in enumerate(data):
    qdata1[index][0] = np.pi * point[0]
    qdata1[index][1] = 2 * np.pi * point[1]
```

```python
qdata2 = np.zeros((n, 2))

for (index, point) in enumerate(data):
    qdata2[index][0] = 0.5*np.arctan(point[1]/point[0])
    qdata2[index][1] = np.pi*(point[0]**2 + point[1]**2)
```

```python
## To switch between the two encoding possibilities
qdata=qdata1
```

### State Preparation
Starting from the ground state $|0\rangle$ state and given angles $\theta, \phi$ state preparatiob can be performed via:

\begin{equation}
S(\theta, \phi) |0\rangle \rightarrow |\psi\rangle = \cos(\theta / 2) |0\rangle + e^{i \phi} \sin(\theta / 2)|1\rangle
\end{equation}

where $S$ is a _state preparation unitary_ or _state preparation circuit_.

It is clear from the equation above that the matrix representation for $S$ in the computational basis is

\begin{equation}
S(\theta, \phi) = \left[ \begin{matrix}
\cos(\theta / 2) & e^{-i \phi} \sin(\theta / 2)\\
e^{i \phi} \sin(\theta / 2) & - \cos(\theta / 2)  \\
  \end{matrix} \right]
\end{equation}

Using Qiskit, one possibility for stae preparation is using the u3-gate:
\begin{equation}
u_3 = u_3(\theta,\pi, \pi-\phi)
\end{equation}

After the classical data has been encoded and prepared into the quantum system, the next step is to perform unitary evolution on the quantum state representing the data. In the language of classical learning theory, this corresponds to implementing a layer of the neural network.

In the quantum neural network case, we simply need to implement a sequence of parameterized gates.

Again, in Qiskit the u3-gate can be used for this purpose.

```python
qr = QuantumRegister(1,'q0')
cr = ClassicalRegister(1,'c0')

def make_program(pangles, uangles):
    """Returns a program that prepares the state according
    to pangles and applies the unitary according to uangles.
    """
    # instantiate a program
    circuit0=QuantumCircuit(qr, cr)

    # define a gate in Qiskit for the state preparation
    circuit0.u3(pangles[0], pangles[1],np.pi-pangles[1],qr)

#    unitary(circuit0,uangles)
    circuit0.u3(uangles[0], uangles[1],np.pi-uangles[1],qr)
    return circuit0
```

```python
def obj_simple(angles):
    """Returns the number of zero outputs of a single training example."""
    # make the program
    circuit1=make_program([theta, phi], angles)

    circuit1.measure(qr, cr)
    job_sim = execute(circuit1, backend_sim)
    result_sim = job_sim.result()
    result_counts = result_sim.get_counts(circuit1)
    res_1=result_counts.get('1')
    if res_1==None:
        res_1=0
    res_0=result_counts.get('0')
    if res_0==None:
        res_0=0
    obj=res_1/(res_0+res_1)

    print("The current value of the objective function is:", obj, end="\r")
    return obj
```

### Training

To faciliate training an  _indicator function_ $I(z_i = \hat{z}_i)$ = 0 if $z_i = \hat{z}_i$ and 1 otherwise is definied. Here, $z_i$ is the exact label of the $i$th training data $(x_i, y_i)$ and $\hat{z}_i$ is the prediction of this label by our neural network. (In code, we use the variables `labels` for $z_i$ and `predictions` for $\hat{z}_i$.)

To define the total cost, sum over all points in the training data:

\begin{equation}
C = \sum_{i = 1}^{M} I(z_i = \hat{z}_i)
\end{equation}

```python
# fraction of total data to use as training data
train_frac = 0.7

def obj(uangles):
    """Returns the objective function C defined above over all training data.

    Args:
        uangles [type: list<float>]
            the angles in the unitary evolution.

    rtype: float
    """
    # grab some training data from the overall data set
    tpoints = int(train_frac * len(qdata))
    tdata = qdata[:tpoints]
    tlabels = label[:tpoints]

    # initialize a variable to store the output predictions of the neural net
    predictions = np.zeros_like(tlabels, dtype=int)

    # loop over all training data to get the predictions
    for i, pangles in enumerate(tdata):
        circuit1=make_program(pangles, uangles)
        circuit1.measure(qr, cr)
        job_sim = execute(circuit1, backend_sim)
        result_sim = job_sim.result()
        result_counts = result_sim.get_counts(circuit1)
        res_1=result_counts.get('1')
        if res_1==None:
            res_1=0
        res_0=result_counts.get('0')
        if res_0==None:
            res_0=0
        p0=res_0/(res_0+res_1)
        p1=res_1/(res_0+res_1)
        # take the prediction to be max(p0, p1)
        if p0 >= p1:
            predictions[i] = 0
        else:
            predictions[i] = 1

    # compute the difference of the labels and return the cost
    cost = sum(abs(predictions - tlabels)) / tpoints
    print("The current value of the cost function is:", cost, end="\r")
    return cost
```

```python
# train the quantum neural network and time how long it takes
# get some random angles
angs = 2 * np.pi * np.random.rand(2)
out = minimize(fun=obj, x0=angs, method="Powell")

```

    The current value of the cost function is: 0.014285714285714285

```python
# grab the optimal angles and minimal cost value
optimal_angles = out['x']
fval = out['fun']

# print them out
print(fval)
print(optimal_angles)
```

    0.0
    [3.17067419 3.64657332]

### Predictions

Calculatee predictions on the whole dataset.

```python
def get_all_predictions(angles):
    """Returns a numpy array of all predictions."""
    # initialize a variable to store the output predictions of the neural net
    zhats = np.zeros_like(label, dtype=int)

    # loop over all data to get predictions
    for i, pangles in enumerate(qdata):
        circuit1=make_program(pangles, angles)
        circuit1.measure(qr, cr)
        job_sim = execute(circuit1, backend_sim)
        result_sim = job_sim.result()
        result_counts = result_sim.get_counts(circuit1)
        res_1=result_counts.get('1')
        if res_1==None:
            res_1=0
        res_0=result_counts.get('0')
        if res_0==None:
            res_0=0
        p0=res_0/(res_0+res_1)
        p1=res_1/(res_0+res_1)

        # take the prediction to be max(p0, p1)
        if p0 >= p1:
            zhats[i] = 0
        else:
            zhats[i] = 1
    return zhats
```

```python
# compute all the predictions of the quantum neural network
predictions = get_all_predictions(optimal_angles)
```

```python
# compute statistics of the QNN
ntrain = int(train_frac * n)
ncorrect = n - sum(abs(predictions - label))
acc = ncorrect / n * 100

# print them out
print(" Results of quantum neural network classification ".center(80, "="))
print("Out of {} total data points:".format(n))
print("The QNN was trained on {}% of the total data ({} training points).".format(train_frac * 100, ntrain))
print("The QNN classified {} data points correctly ({}% accuracy).".format(ncorrect, acc))
print("".center(80, "="))

# plot the points, line y = x, and prediction
plt.plot(ys, xs, '--k')
for i in range(n):
    if predictions[i] == 0:
        ckey = 'g'
    else:
        ckey = 'b'
    plt.scatter(data[i, 0], data[i, 1], color=ckey)

plt.grid()
plt.title("QNN Predictions", fontsize=16, fontweight="bold")
plt.show()
```

```python

```


```python
import numpy as np
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt
import sympy
import torch

%matplotlib inline
```

```python
df = pd.read_csv('../data/raw/cities.csv', index_col=['CityId'])
```

```python
x = torch.from_numpy(df[['X', 'Y']].values)
x.div_(x.max())
```

    tensor([[0.0621, 0.4319],
            [0.8584, 0.0660],
            [0.6774, 0.5530],
            ...,
            [0.9365, 0.6087],
            [0.5872, 0.3788],
            [0.2657, 0.6311]], dtype=torch.float64)

```python
n = x.shape[0] * 8
network = torch.from_numpy(np.random.rand(n, 2))
```

```python
iterations=10000
learning_rate=0.8
```

```python
def get_neighborhood(center, radix, domain):
    if radix < 1:
        radix = 1
    deltas = torch.abs(center - torch.arange(domain))
    distances = torch.min(deltas, domain - deltas).double()
    return torch.exp(-(distances*distances) / (2*(radix*radix)))
```

```python
for i in range(iterations):
    if not i % 100:
        print('\t> Iteration {}/{}'.format(i, iterations), end="\r")
    city = x[torch.randint(x.shape[0], (1,1)).item()]
    _, ind = torch.min(torch.norm(network - city, p=2, dim=1), 0)
    gaussian = get_neighborhood(ind, n//10, network.shape[0])
    network += gaussian[:,np.newaxis] * learning_rate * (city - network)
    learning_rate = learning_rate * 0.99997
    n = n * 0.9997

    if n < 1:
        print('Radius has completely decayed, finishing execution', 'at {} iterations'.format(i))
        break
    if learning_rate < 0.001:
        print('Learning rate has completely decayed, finishing execution', 'at {} iterations'.format(i))
        break
else:
    print('Completed {} iterations.'.format(iterations))
```

```python
plt.plot(network.numpy())
```

```python
def select_closest_np(candidates, origin):
    """Return the index of the closest candidate to a given point."""
    return euclidean_distance_np(candidates, origin).argmin()

def euclidean_distance_np(a, b):
    """Return the array of distances of two numpy arrays of points."""
    return np.linalg.norm(a - b, axis=1)

def get_route_np(cities, network):
    """Return the route computed by a network."""
    cities['winner'] = cities[['X', 'Y']].apply(
        lambda c: select_closest_np(network, c),
        axis=1, raw=True)

    return cities.sort_values('winner')
```

```python
route = get_route_np(df[:1000], network.numpy())
```

```python
route = cities.reindex(route)
```

    Int64Index([0, 4, 2, 3, 1], dtype='int64', name='CityId')

```python
def plot_route(cities, route, name='diagram.png', ax=None):
    """Plot a graphical representation of the route obtained"""
    mpl.rcParams['agg.path.chunksize'] = 10000

    if not ax:
        fig = plt.figure(figsize=(5, 5), frameon = False)
        axis = fig.add_axes([0,0,1,1])

        axis.set_aspect('equal', adjustable='datalim')
        plt.axis('off')

        axis.scatter(cities['X'], cities['Y'], color='red', s=4)
        route = cities.reindex(route)
        route.loc[route.shape[0]] = route.iloc[0]
        axis.plot(route['X'], route['Y'], color='purple', linewidth=1)

        plt.savefig(name, bbox_inches='tight', pad_inches=0, dpi=200)
        plt.close()

    else:
        ax.scatter(cities['X'], cities['Y'], color='red', s=4)
        route = cities.reindex(route)
        route.loc[route.shape[0]] = route.iloc[0]
        ax.plot(route['X'], route['Y'], color='purple', linewidth=1)
        return ax
```

```python
fig = plt.figure(figsize=(5, 5), frameon = False)
axis = fig.add_axes([0,0,1,1])

plot_route(df[:10], route, ax=axis)
```

```python

```


```python
import sympy as sym
```

```python
alpha = 1; beta = 0.01;
delta = 0.02; gam = 1;
eps = 0.1; lamda = 0.1;
```

```python
alpha = sym.Symbol('alpha')
beta = sym.Symbol('beta')
delta = sym.Symbol('delta')
gam = sym.Symbol('gam')
eps = sym.Symbol('eps')
lamda = sym.Symbol('lamda')

x = (beta+(eps*gam))/(eps*(delta+(alpha*lamda)))
y = ((alpha*lamda)+delta)/(lamda*(beta+(eps*gam)))

a = sym.simplify(delta*x - gam + alpha*lamda*x - beta*lamda*x*y)
print(a)
```

    0


# Ecuaciones Diferenciales Ordinarias

Como es sabido en la [historia de la computación](https://youtu.be/jWum9IJPyks), uno de los problemas que motivo más fuertemente la construcción de computadoras digitales programables fue la solucionar las ecuaciones diferenciales ordinarias que describen la trayectoria de misiles balísticos (esto, claro, durante la segunda guerra mundial).

Dichas ecuaciones, así como muchas otras ecuaciones diferenciales ordinarias, son imposibles de resolver analíticamente, por lo que la única manera de resolverlas es acudir a métodos numéricos que nos permitan obtener una solución aproximada al problema.

Existen [libros enteros](https://books.google.com.mx/books?id=lZysESfSWwgC&printsec=frontcover&dq=leveque+finite+differences&hl=en&sa=X&ved=2ahUKEwi9hJaQ-arrAhVN5awKHXjgDnkQ6AEwAHoECAUQAg#v=onepage&q=leveque%20finite%20differences&f=false), y cursos también, dedicados al estudio de la solución numérica de ecuaciones diferenciales tanto ordinarias como parciales.

## Preludio Computacional 1: prealocación de memoria para arreglos

Durante el curso, hemos, en muchas ocasiones, construido arreglos que representan a los elementos de una sucesión, **iniciando con un arreglo que contiene un solo elemento y añadiéndole elementos uno a uno**. Esto se puede ya que, como ya mencionamos anteriormente, los arreglos en Julia son **dinámicos**.

En general, es mala idea tratar con arreglos dinámicos pues es computacionalmente costoso ir asignando o  **alocando** memoria a elementos nuevos. Lo más conveniente, de ser posible, es construir un arreglo de la longitud que necesitemos desde un inicio. Así, podemos tratar al arreglo de manera **estática**: solo cambiamos sus elementos a nuestro gusto pero no cambiamos su longitud

En Julia, existen varias funciones para crear arreglos:
* La función `zeros(n)` crea un arreglo 1D de longitud $n$ en la que cada elemento es un $0$.

* La función `ones(n)` crea un arreglo 1D de longitud $n$ en la que cada elemento es un $1$.

* La función `rand(n)` crea un arreglo 1D de longitud $n$ en la que cada elemento es un número aleatorio en el intervalo $[0,1]$

```julia
arr1 = zeros(10)
println("ceros")
println(arr1)
arr2 = ones(10)
println("unos")
println(arr2)
arr3 = rand(10)
println("aleatorios")
println(arr3)
```

    ceros
    [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
    unos
    [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
    aleatorios
    [0.8003510631840209, 0.4101320439113849, 0.8579331097464615, 0.5234381958176033, 0.44733476848457143, 0.00810758358575625, 0.453898058253688, 0.6616717600235262, 0.3933356452364807, 0.5079435731409363]

Noten también que `rand()` te genera un solo número aleatorio

```julia
println(rand())
```

    0.1535703856332895

Por default, los arreglos creados con `zeros` y `ones` tienen elementos de tipo `Float64`. Sin embargo, podemos especificar que sean de otro tipo utilizando la sintaxis `zeros(T,n)` o `ones(T,n)` con `T` el tipo de interés

```julia
arr4 = zeros(5)
println(arr4)
println(typeof(arr4))
arr5 = zeros(Int32,5)
println(arr5)
println(typeof(arr5))
```

    [0.0, 0.0, 0.0, 0.0, 0.0]
    Array{Float64,1}
    Int32[0, 0, 0, 0, 0]
    Array{Int32,1}

Podemos usar la prealocación a la hora de construir sucesiones recursivas como las que usamos en la mayoría de métodos numéricos

```julia
# crea $n$ términos de la sucesión de fibonacci
function fibonacciPrealocado(n)
    # creamos arreglo para guardar los $n$ términos
    final = zeros(Int64,n)
    # definimos los primeros elementos
    final[1] = 1
    final[2] = 1
    # creamos los demás, iniciando en el tercero
    for i in 3:n
        final[i] = final[i-1] + final[i-2]
    end
    return final
end
```

    fibonacciPrealocado (generic function with 1 method)

```julia
fibonacciPrealocado(20)
```

    20-element Array{Int64,1}:
        1
        1
        2
        3
        5
        8
       13
       21
       34
       55
       89
      144
      233
      377
      610
      987
     1597
     2584
     4181
     6765

## Preludio Computacional II: tuplas

Hasta ahora, hemos trabajado con objetos iterables como listas, arreglos y rangos, los cuales se pueden describir a grandes rasgos como sucesiones ordenadas de objetos. Sabemos que, en caso de las listas y los arreglos podemos cambiar sus elementos

```julia
nombres = ["aldo","david","crhistian"]
nombres[1] = "sayeg"
println(nombres)
```

    ["sayeg", "david", "crhistian"]

Podemos cambiar sus entradas por que las listas y los arreglos son **objetos mutables**, es decir, que pueden cambiar sus elementos. Existen objetos **no mutables** que también representan sucesiones de objetos pero que no podemos cambiar sus elementos. Dichos objetos son las **tuplas** (de tipo `Tuple` en Julia). Una tupla se define con la siguiente sintaxis

```julia
tup1 = (obj1,obj2,obj3)
```

```julia
tupNombres = ("aldo","david","crhistian")
println(tupNombres)
println(typeof(tupNombres))
```

    ("aldo", "david", "crhistian")
    Tuple{String,String,String}

```julia
# al intentar cambiar un elemento arrojará un error
tupNombres[1] = "sayeg"
```

`length` funciona de igual manera con las tuplas

```julia
println(length(tupNombres))
```

    3

Nuevamente, por ser inmutable, no podemos añadirle objetos y `push!` fracasará

```julia
push!(tupNombres,"miguel")
```

### Preludio Computacional III: productos cartesianos

En las siguientes clases trabajaremos con objetos como campos vectoriales y otros análogos, los cuales viven en espacios de varias dimensiones.

Muchas veces nos va a interesar poder obtener el producto cartesiano de de dos arreglos, es decir, un objeto de la forma

$$\{(x,y) \; \mid \; x \in xs \; , \; y \in ys\}$$

Podemos utilizar comprensión de arreglos en Julia para crear ese arreglo con la siguiente sintaxis

```julia
prodCart = [[x,y] for x in xs for y in ys]
```

```julia
# Ejemplo simple
xs = [1,2,3]
ys = [8,9]
# se empieza a iterar por el úlimo arreglo
prodCart = [[x,y] for x in xs for y in ys]
display(prodCart)
```

    6-element Array{Array{Int64,1},1}:
     [1, 8]
     [1, 9]
     [2, 8]
     [2, 9]
     [3, 8]
     [3, 9]

Podemos hacer esto mismo con tuplas:

```julia
nombres = ["aldo","david","miguel"]
apellidos = ["perez","lopez","medrano"]
# ahora tendremos tuplas en el producto cartesiano
prodCar = [(x,y) for x in nombres for y in apellidos]
display(prodCar)
```

    9-element Array{Tuple{String,String},1}:
     ("aldo", "perez")
     ("aldo", "lopez")
     ("aldo", "medrano")
     ("david", "perez")
     ("david", "lopez")
     ("david", "medrano")
     ("miguel", "perez")
     ("miguel", "lopez")
     ("miguel", "medrano")

## Preludio Matemático: Ecuciones Diferenciales I

Sea $x(t): I \subseteq \mathbb{R} \to \mathbb{R}$ una función de clase $C^{1}$, es decir, una función derivable en todo $I$ con derivada continua en $I$. Una ecuación diferencial de primer orden es una ecuación **funcional**  de la siguiente forma

$$
\dot{x}(t) = f(x(t),t)
$$

Donde $\dot{x}(t) = \frac{dx}{dt} (t)$ y $f$ una función continua de dos variables. Notemos que, a diferencia de una ecuación **algebraica** donde las variables representan números o miembros de algún otra estructura algebraica "simple" (vectores, matrices, etc) que solo se operan con operadores algebraicos como suma, producto etc, en una ecuación diferencial algunas variables representan **funciones**.

Más aún, las operaciones que se les realizan a nuestros objetos no son operaciones algebraicas simples si no operaciones más delicadas una derivada o la aplicación de una función arbitraria.

La técnica más estándar para resolver una ecuación diferencial consiste en utilizar el teorema fundamental del cálculo para integrar ambos lados de la ecuación respecto al tiempo. Sin embargo, a la hora de realizar dicha integración, es probable que surgan **constantes de integración**. Para poder determinar dichas constantes, será necesario tener una **condición** sobre la función $x$, es decir, conocer un punto en el que sepamos su valor.

$$
x(t_i) = C
$$

### Problemas de valores iniciales

Aunque el punto $t_i$ puede ser un punto arbitrario del dominio de la función, en general, lo que tenemos que es que ese tiempo es un tiempo que llamamos **inicial** y denotamos $t_0$, y la solución $x(t)$ nos interesa conocerla para tiempos $t \geq t_0$. Cuando ese es el caso, decimos que tenemos un **problema de valores iniciales** descrito por las dos ecuaciones

$$
\begin{cases}
\dot{x}(t) = f(x(t),t) \\
x(t_0) = x_0 \\
\end{cases}
$$

### Campo de direcciones o "retrato fase"

Antes de resolver una ecuación diferencial, podemos darnos una idea del comportamiento que tendrán las soluciones si graficamos el campo de direcciones o retrato fase de la curva: el campo vectorial de las pendientes de las funciones que son solución a la ecuación diferencial pero no necesariamente a a la condición inicial. Dicho campo corresponde al campo vectorial $P:\mathbb{R}^2 \to \mathbb{R}^2$ definido por

$$
P(t,x) = (1,f(x,t))
$$

Recordando el tutorial de Plots para Julia, podemos graficar un campo vectorial utilizando la función `quiver` del paquete Plots. La sintaxis es

```julia
quiver(xs,ys,vectorfield=(us,vs))
```

La función hace una grafica consistente en flechas entre los puntos `(xs[i],ys[i])` y `(xs[i]+us[i],ys[i]+vs[i])`

### Ejemplo 1:

Sea $\dot{x}(t) = \alpha x(t) $. Claramente, eso indica que $f(x(t),t) = \alpha x(t) $. El campo vectorial entonces es

$$
P(t,x) = ( 1,\alpha x)
$$

```julia
using Plots, LinearAlgebra
```

```julia
function campoVecEjemplo1(X,alpha)
    # X es un vector en cuya primera entrada representa a t y la segunda a x
    # es pura notación para poder definir la función
    return [1,X[2]*alpha]
end
function campoDireccionesEjemplo1(alpha)
    # definimos nuestros puntos ts yxs donde van a empezar las flechas
    ts = range(-1,stop=1,length=15)
    xs = range(0,stop=2,length=15)
    # hacemos el producto carteasiano ts x xs para tener puntos en R^2 donde poner flechas
    # la siguiente sintaxis nos construye el producto cartesiano de dos listas en otro
    dom =[[t,x] for t in ts for x in xs]
    # aplicamos el campo vectorial al los elementos del dominio
    imagen = [campoVecEjemplo1(X,alpha) for X in dom]
    # normalizamos cada elemento del campo vectorial para que tenga norma 0.1
    # normalize(arr) es una funcion de LinearAlgebra que nos regresa el vector `arr` normalizado
    imagen = [0.1*normalize(ima) for ima in imagen]
    # hacemos la gráfica del campo vectorial
    quiver([d[1] for d in dom],[d[2] for d in dom],vectorfield=([ima[1] for ima in imagen],[ima[2] for ima in imagen]))
    # otra manera de poner títulos
    # poner `$(var)` en un string remplaza ese pedazo por lo que haya en la variable `var`
    title!("Campo de direcciones para alpha = $(alpha)")
    # otra manera de poner título a los ejes
    xlabel!("t")
    ylabel!("x")
end
```

    campoDireccionesEjemplo1 (generic function with 1 method)

```julia
campoDireccionesEjemplo1(2.0)
```

## El método de Euler

El método más sencillo para resolver numéricamente un problema de valores iniciales es el **método de Euler**, que consiste en aproximar la derivada $\dot{x}$ con una diferencia finita hacia adelante. Así, obtenemos

$$
\frac{x(t+h) - x(t)}{h} = f(x(t),t)
$$

Despejando encontramos que

$$
x(t+h) = x(t) + h \; f(x(t),t)
$$

Es claro que, si conocemos $x(t_0) = x_0$ y $t_0$, esta regla nos permite calcular $x(t_0 +h)$ y, al tener $x(t_0+h)$ y también $t_0 + h$, posteriormente podemos calcular $x(t_0 + 2h)$ y así sucesivamente para obtener $x(t_0 + k h)$  para cualquier natural $k$. Planteandolo en términos de una sucesión recursiva, sea $t_k = t+ k h$ y entonces $x_{k} := x(t_k)$, entonces la sucesión recursiva del método de Euler es

$$
x_k = \begin{cases}
x_0 & k = 0\\
x_{k-1} + h f(x_{k-1},t_{k-1}) & k > 0\\
\end{cases}
$$

Notemos que esto solo nos dará los valores de $x(t)$ en un montón de tiempos discretos $t_0,t_1,t_2,\ldots$. Todos los métodos numéricos para EDO regresan la solución como un conjunto de puntos discretos. Si nosotros insitieramos en querer conocer, en lugar de puntos discretos, la función completa, podriamos utilizar interpolación para convertir los puntos discretos en una función.

### Ejercicio 1

implementa una función `primerEuler(f,x_0,t_0,t_f,h)` con `f` la función **vectorial** tal que $\dot{x}(t) = f(x(t),t)$, `x_0`, `t_0` la condición inicial, `t_f` el tiempo máximo al que queremos obtener la solución y `h` el tamaño de paso. La función debe de implementar el método de euler y debe de regresar el arreglo de los tiempos `[t_0,t_1, t_2,....,t_n]` con $t_{n} \leq t_f < t_{n} + h$ y el de la solución a la ecuación diferencial `[x_0,x_1,...,x_n]`.

**Sugerencia:** prealoquen los arreglos que van a guardar los tiempos y los puntos de la solución. utilicen una tupla para regresar ambos arreglos.

### Ejercicio 2

Utiliza tu función `primerEuler` para resolver las siguientes ecuaciones diferenciales (con las condiciones inciales y tiempo final de tu preferencia):

1. $\dot{x}(t) = 0.3 x(t) $

2. $\dot{x}(t) = 3t^2 - 5t $

3. $\dot{x}(t) = x(t)(1-x(t))$

Grafica el campo de direcciones para cada ecuación. Realiza una gráfica de la solución obtenida y compárala con la solución analítica en la misma gráfica

**Sugerencia**: ten cuidado pues no las condiciones iniciales que pueden utilizarse no son arbitrarias. (Por ejemplo, en la ecuación 1, no pueden tomar $t_0 = 0$, $x_0 = 0$)

### Ejercicio 3

Podemos definir el error absoluto de la solución numérica como la diferencia entre el último valor de la función obtenido numéricamente y su correspondiente valor analítico, es decir

$$
\text{error} = \mid x_n - x_{\text{analítica}}(t_n) \mid
$$

Para cada una de las funciones del ejercicio anterior, analiza el error absoluto como función del parámetro $h$. Realiza su gráfica y menciona algo sobre su comportamiento

## Ecuaciones de orden mayor: reducción

En la práctica del día a día, en particular en la física, por lo general tratamos con ecuaciones diferenciales de orden mayor a uno. En general, una ecuación diferencial de orden $l$ es una ecuación funcional de la forma

$$
y^{(l)} (t) = g(y^{(l-1)} (t),y^{(l-2)} (t), \ldots, y^{(1)}(t), y(t),t)
$$

Donde $y^{(l)} (t) = \frac{d^l y}{dt^l}$ es la $l$-ésima derivada de $y(t)$. Aunque podríamos desarrollar un método para atacar ecuaciones diferenciales de ese orden, existe un [**truco**](http://mathonline.wikidot.com/converting-nth-order-odes-to-systems-of-n-first-order-odes) para convertir una ecuación de orden $n$ en un sistema de ecuaciones de primer orden con $n$ variables. Definimos

$$
\begin{align}
 x_1(t) &= y(t) \\
 x_2(t) &= y^{(1)}(t) \\
 &\vdots \\
 x_i(t) &= y^{(i-1)}(t) \\
 &\vdots \\
 x_l(t) &= y^{(l-1)}(t)
\end{align}
$$

Entonces la ecuación diferencial se reduce al siguiente sistema

$$
\begin{align}
 \dot{x_1}(t) &= y^{(1)}(t) = x_2(t) \\
 \dot{x_2}(t) &= y^{(2)}(t) = x_3(t) \\
 &\vdots \\
 \dot{x_i}(t) &= y^{(i)}(t) = x_{i+1}(t) \\
 &\vdots \\
 \dot{x}_{l-1}(t) &= y^{(l-1)}(t) = x_{l}(t) \\
\dot{x_l}(t) &= y^{(l)}(t)
\end{align}
$$

Pero

$$
y^{(l)}(t) = g(y^{(l-1)} (t),y^{(l-2)} (t), \ldots, y^{(1)}(t),y(t),t) = g(x_{l}(t),x_{l-1}(t),\ldots,x_1(t),t)
$$

Por lo que el sistema se vuelve

$$
\begin{align}
 \dot{x_1}(t) &= x_2(t) \\
 \dot{x_2}(t) &= x_3(t) \\
 &\vdots \\
 \dot{x_i}(t) &= x_{i+1}(t) \\
 &\vdots \\
 \dot{x}_{l-1}(t) &= x_{l}(t) \\
\dot{x_l}(t) &= g(x_{l}(t),\ldots,x_1(t),t)
\end{align}
$$

### Ejercicio 4

Analíticamente, convierte las siguientes ecuaciones diferenciales de orden $n$ en sistemas de $n$ ecuaciones diferenciales:

1. $y''(t) = -\frac{10}{m}$

2. $y'''(t) = t + 2y(t) + 3y''(t)$

3. $y''(t) = -\frac{k}{m} y(t) - \frac{c}{m} y'(t)$

## Método de euler vectorial

Sea entonces **el vector de funciones**  $\mathbf{x}(t) : \mathbb{R} \to \mathbb{R}^{l}$

$$\mathbf{x}(t) := (x_1(t),\ldots,x_{l}(t))$$

y la función vectorial $\mathbf{f}(\mathbf{x}(t),t) : \mathbb{R}^{l+1} \to \mathbb{R}^{l}$

$$\mathbf{f}(\mathbf{x}(t),t) = (x_2(t),\ldots,x_{l}(t),g(x_{l}(t),x_{l-1}(t),\ldots,x_1(t),t))$$

Podemos escribir el sistema de ecuaciones diferenciales de forma vectorial como

$$
\dot{\mathbf{x}}(t) = \mathbf{f}(\mathbf{x}(t),t)
$$

Que nuevamente admite una solución mediante el método de euler al aproximar la derivada del vector $\mathbf{x}(t)$ con una diferencia hacia adelante:

$$
\mathbf{x}(t+h) = \mathbf{x}(t) + h \; \mathbf{f}(\mathbf{x}(t),t)
$$

### Ejercicio 5

Implementa una función `eulerVec(f,x_0,t_0,t_f,h)` con `f` la función **vectorial** tal que $\dot{\mathbf{x}}(t) = \mathbf{f}(\mathbf{x}(t),t)$, `x_0` (un vector de $n$ entradas) y `t_0` la condición inicial, `t_f` el tiempo máximo al que queremos obtener la solución y `h` el tamaño de paso. La función debe de implementar el método de euler y debe de regresar el arreglo de los tiempos `[t_0,t_1, t_2,....,t_n]` con $t_{n} \leq t_f < t_{n} + h$ y el de la solución a la ecuación diferencial `[x_0,x_1,...,x_n]`.

**Sugerencia 1:** Prealoca los arreglos de los tiempos y las soluciones. Utiliza una tupla para regresar ambos arreglos.

**Sugerencia 2** puedes utilizar `zeros(n,l)` para prealocar una matriz de zeros de $n \times l$ en la cual puedes guardar como renglones cada vector $\mathbf{x}_i$

### Ejercicio 6

Utiliza tu función `eulerVec` para resolver los sistemas de ecuaciones obtenidos en el ejercicio 4. Nuevamente, puedes escoger arbitrariamente las condiciones iniciales, el tiempo final y las constantes de las ecucaiones ($m$,$k$,etc).

Grafica  el **espacio fase** de tus soluciones, es decir, el conjunto

$$\{ \mathbf{x}(t) \; \mid \; t_0 \leq t \leq t_f\} \subset \mathbb{R}^2, \mathbb{R}^3$$

Si es posible, compara con la solución analítica

### Ejercicio 7

El inciso 1 y 3 del ejercicio 4, la variable $y(t)$ corresponde a la posición como función del tiempo para un objeto en **caída libre** o en un **oscilador armónico amortiguado**. Para unas condiciones iniciales, tiempo final y valor de los parámetros, soluciona el sistema y realiza una **animación** del movimiento del objeto (i.e animar $y(t)$ para distintos tiempos). Verifica que coincida con las soluciones que has visto en los cursos de mecánica.


# Lynx Motion AL5D

Here is the serial manipulator we are using for our lab. Note all of the dimension at the bottom center.

The toy RC servos for the robot arm work off of a pulse width modulation (PWM) where a pulse emmited with the following width will move the servo to a desired position.

| Dir    | PWM  |
|--------|------|
|CCW_max | 1200 |
|Center  | 1500 |
|CW_max  | 1800 |

Since this interface was developed around the 1970's, and toy RC servos are built for price not performance, different servos behave differently to the same signal. Thus, the user typically has to find the true limits of each servo to know how to command it. But the times shown above are approximate (close enough for hand grenades and horseshoes).

```python
%matplotlib inline
```

```python
from __future__ import print_function
from __future__ import division
import numpy as np
from matplotlib import pyplot as plt
from sympy import symbols, sin, cos, simplify, trigsimp, pi
from math import radians as d2r
from math import degrees as r2d
from math import atan2, sqrt, acos, fabs
```

```python
class mDH(object):
    """
    This uses the modified DH parameters
    see Craig, eqn (3.6)
    """
    def __init__(self):
        pass

    def fk(self, params):
        t = np.eye(4)
        for p in params:
            t = t.dot(self.makeT(*p))
        return t

    def makeT(self, a, alpha, d, theta):
        return np.array([  # classic DH
            [           cos(theta),           -sin(theta),           0,             a],
            [sin(theta)*cos(alpha), cos(theta)*cos(alpha), -sin(alpha), -d*sin(alpha)],
            [sin(theta)*sin(alpha), cos(theta)*sin(alpha),  cos(alpha),  d*cos(alpha)],
            [                    0,                     0,           0,             1]
        ])

def eval(f):
    """
    This allows you to simplify the trigonomic mess that kinematics can
    create and also substitute in some inputs in the process
    """
    c = []
    for row in f:
        r = []
        for col in row:
            # use python symbolic toolbox to simplify the trig mess above
            r.append(simplify(col))
        c.append(r)
    return np.array(c)

def subs(f, m):
    """
    This allows you to simplify the trigonomic mess that kinematics can
    create and also substitute in some inputs in the process
    """
    c = []
    for row in f:
        r = []
        for col in row:
            r.append(col.subs(m))
        c.append(r)
    return np.array(c)

def rplot(t1, t2, t3, t4):
    """Given the 4 joint angles (in rads), plot the arm in the x-y and w-z planes"""
    l1 = 5.75
    l2 = 7.375
    l3 = 3.375
    ptsx = [0]
    ptsy = [0]

    # our definition is reverse or these joints
    t3 = -t3
    t4 = -t4

    # link 1
    x1 = l1*cos(t2)
    y1 = l1*sin(t2)
    ptsx.append(x1)
    ptsy.append(y1)

    # link 2
    x2 = x1 + l2*cos(t2 + t3)
    y2 = y1 + l2*sin(t2 + t3)
    ptsx.append(x2)
    ptsy.append(y2)

    # link 3
    x3 = x2 + l3*cos(t2 + t3 + t4)
    y3 = y2 + l3*sin(t2 + t3 + t4)
    ptsx.append(x3)
    ptsy.append(y3)

    plt.subplot(1,2,1,projection='polar')
    plt.plot([0, t1], [0, 1.0])
    plt.grid(True)
    plt.title('Azimuth Angle (x-y plane)')

    plt.subplot(1,2,2)
    plt.plot(ptsx, ptsy, 'b-', marker='o')
    plt.axis('equal')
    plt.grid(True)
    plt.title('w-z Plane')
```

The DH parameters are:

| i |$a_i$        | $\alpha_i$   | $d_i$   | $theta_i$  |
|---|-------------|--------------|---------|------------|
| 1 | 0           | 0            | $d_2$   | $\theta_1$ |
| 2 | 0           |$\frac{\pi}{2}$| 0       | $\theta_2$ |
| 3 | $a_3$       | 0            | 0       | $\theta_3$ |
| 4 | $a_4$       | 0            | 0       | $\theta_4$ |
| 5 | $a_5$       | 0            | 0       |          0 |

```python
from sympy import pi
t1, t2, t3, t4 = symbols('t1 t2 t3 t4')
# a2, a3, a4, a5 = symbols('a2 a3 a4 a5')

# t1 - base
# t2 - shoulder
# t3 - elbow
# t4 - wrist

# a2 = 2.75   # base to shoulder
a2 = 0.0
a3 = 5.75   # shoulder to elbow
a4 = 7.375  # elbow to wrist
a5 = 3.375  # wrist to end effector

# put all angles in rads
# sybolic can't handle converting a symbol using d2r()
# a, alpha, d, theta
params = [
    [    0,    0,    0, t1],
    [    a3, -pi/2,   0, t2],
    [   a4,    0,    0, t3],
    [   a5,    0,    0, t4]
]
# params = [
#     [       0,    0, 2.75, t1],
#     [       0, pi/2,    0, t2],
#     [    5.75,    0,    0, t3],
#     [   7.375,    0,    0, t4],
#     [   3.375,    0,    0,  0]
# ]
dh = mDH()
al5d = dh.fk(params)
al5d = eval(al5d)
```

```python
def printT(tt):
    """Print out the entire T matrix"""
    R = tt[0:3,0:3]
    D = tt[0:3, 3]
    print('-'*30)
    print('Position:')
    print('  x:', D[0])
    print('  y:', D[1])
    print('  z:', D[2])
    # R(n, o, a)
    print('-'*30)
    print('Orientation')
    print('  nx:', R[0,0])
    print('  ny:', R[0,1])
    print('  nz:', R[0,2])
    print('')
    print('  ox:', R[1,0])
    print('  oy:', R[1,1])
    print('  oz:', R[1,2])
    print('')
    print('  ax:', R[2,0])
    print('  ay:', R[2,1])
    print('  az:', R[2,2])

def printP(dh):
    """dh is the forward kinematics equations matrix"""
    pos = [float(x) for x in dh[0:3, 3]]
    print('Pos (x,y,z): {:5.2f} {:5.2f} {:5.2f}'.format(*pos))
    return pos

def printDegrees(angles):
    """angles are in radians"""
    a = [r2d(x) for x in angles]
    print('Angles: {:6.1f} {:6.1f} {:6.1f} {:6.1f}'.format(*a))
```

```python
# the symbolic forward kinematics of our robot arm
printT(al5d)
```

    ------------------------------
    Position:
      x: (7.375*cos(t2) + 3.375*cos(t2 + t3) + 5.75)*cos(t1)
      y: (7.375*cos(t2) + 3.375*cos(t2 + t3) + 5.75)*sin(t1)
      z: -7.375*sin(t2) - 3.375*sin(t2 + t3)
    ------------------------------
    Orientation
      nx: 1.0*cos(t1)*cos(t2 + t3 + t4)
      ny: -1.0*sin(t2 + t3 + t4)*cos(t1)
      nz: -1.0*sin(t1)

      ox: 1.0*sin(t1)*cos(t2 + t3 + t4)
      oy: -1.0*sin(t1)*sin(t2 + t3 + t4)
      oz: 1.0*cos(t1)

      ax: -1.0*sin(t2 + t3 + t4)
      ay: -1.0*cos(t2 + t3 + t4)
      az: 0

```python
# Let's set some angles
simp=subs(al5d, [(t1,0.0), (t2,pi/2), (t3, pi/2), (t4, 0.0)])
printT(simp)
```

    ------------------------------
    Position:
      x: 2.37500000000000
      y: 0
      z: -7.37500000000000
    ------------------------------
    Orientation
      nx: -1.00000000000000
      ny: 0
      nz: 0

      ox: 0
      oy: 0
      oz: 1.00000000000000

      ax: 0
      ay: 1.00000000000000
      az: 0

```python
simp=subs(al5d, [(t1,0.0),(t2,d2r(111.5)), (t3, d2r(-127)), (t4, d2r(-74.5))])
printP(simp)
```

    Pos (x,y,z):  5.00  0.00  0.00

    [4.999392538748885, 0.0, 0.004017992319495178]

```python

```

# Inverse Kinematics

## Law of Cosines

$$
a^2 = b^2 + c^2 - 2bc \cos(A) \rightarrow \cos(A)=\frac{-a^2+b^2+c^2}{2bc}\\
b^2 = a^2 + c^2 - 2ac \cos(B) \rightarrow \cos(B)=\frac{a^2-b^2+c^2}{2ac}\\
c^2 = a^2 + b^2 - 2ab \cos(C) \rightarrow \cos(C)=\frac{a^2+b^2-c^2}{2ab}
$$

- [Wolfram: law of cosines](http://mathworld.wolfram.com/LawofCosines.html)

## Law of Sines

$$
\frac{a}{\sin(A)} = \frac{b}{\sin(B)} = \frac{c}{\sin(C)}
$$

- [Wolfram: law of sines](http://mathworld.wolfram.com/LawofSines.html)

## Arm

We are going to solve this using geometry! So, given a point (w,z) and the
orientation of the end effector, we will calculate the joint angles ($\theta_2$,
$\theta_3$, $\theta_4$).

You will also want to make some checks:

- $z \geq 0$
- $-1 \geq \arctan(\frac{y}{x}) \leq 1$
- $\|(x,y,z) \| \leq \|(l_1, l_2, l_3) \|$

```python
from math import atan2, acos, sqrt, pi, cos, sin
```

```python
def cosine_law(a, b, c, phase=False):
    if phase:
        angle = ((c**2 - (a**2 + b**2))/(2*a*b))
    else:
        angle = ((c**2 - (a**2 + b**2))/(-2*a*b))

#     print('cosine_law', angle)

    if angle > 1 or angle < -1:
        raise Exception('angle outside range')
    return acos(angle)

def line(x1, y1, x2, y2):
    return sqrt((x2-x1)**2 + (y2-y1)**2)

def mag(a, b):
    return sqrt(a**2 + b**2)

def mag3(a, b, c):
    return sqrt(a**2 + b**2 + c**2)
```

```python
def inverse(x, y, z, orient):
    """
    Azimuth angle is between x and w and lies in the x-y plane

               ^ x
         w     |
           \   |
            \  |
             \ |
              \|
    <----------+ (z is out of the page - right hand rule)
    y

    Most of the robot arm move in the plane defined by w-z

    ^ z
    |   o-----o
    |  /       \
    | /         E
    |/
    +----------------> w

    All joint angles returned are in radians: (t1, t2, t3, t4)

    """
    l1 = 5.75
    l2 = 7.375
    l3 = 3.375

    # check workspace constraints
    if z < 0:
        raise Exception('z in ground')
    elif mag3(x,y,z) > (l1 + l2 + l3):
        raise Exception('out of reach')

    # get x-y plane azimuth
    t1 = atan2(y, x)

    # Now, most of the arm operates in the w-z frame
    w = mag(x, y)         # new frame axis
    gamma = atan2(z, w)
    r = mag(z, w)

    c = mag(w-l3*cos(orient), z-l3*sin(orient))

    t3 = cosine_law(l1, l2, c, True)

    d = cosine_law(l2, c, l1)
    e = cosine_law(c, l3, r)
    t4 = pi - d - e

    alpha = cosine_law(l1, c, l2)
    beta = cosine_law(c,r,l3)

    t2 = alpha + beta + gamma

    return (t1, t2, t3, t4)
```

```python
def checkPts(x, y, z, orient):
    """Given a point (in inches) and orientation (in rads), this calculates
    the joint angles, then uses those angles to calculate the forward solution
    and prints out the error. It also plots the arm.
    """
    angles = inverse(x, y, z, orient)

    a,b,c,d = angles

    simp=subs(al5d, [(t1, a), (t2, b), (t3, -c), (t4, -d)])
    pts = printP(simp)

    printDegrees(angles)
    rplot(*angles)

    error = [a-b for a,b in zip((x,y,z), pts)]
    print('Error: {:6.3f} {:6.3f} {:6.3f}'.format(*error))
```

```python
checkPts(10.75, 0, 5.75, 0.0)
```

```python
# 0 90 -90 0
checkPts(7.385, 0, 5.75-3.375, -pi/2)
```

```python
x,y,z = (7.385*cos(pi/4), 7.385*sin(pi/4), 5.75-3.375)
checkPts(x,y,z, -pi/2)
```

```python
x,y,z = (7.385*cos(-pi/4), 7.385*sin(-pi/4), 5.75-3.375)
checkPts(x,y,z, -pi/2)
```

```python
x,y,z = (7.385*cos(pi/2), 7.385*sin(pi/2), 5.75-3.375)
checkPts(x,y,z, -pi/2)
```

```python
checkPts(5,0,0, -pi/2)
```

```python
checkPts(7,-3,0, -pi/2)
```

```python
checkPts(7,6,4, -pi/2)
```

# Phasing

Let's show $\cos(\pi-\theta)$ and $\cos(\theta - \pi)$ are the same thing.

```python
print(cos(pi-1))
print(cos(1-pi))
print(cos(1))
```

    -0.540302305868
    -0.540302305868
    0.540302305868

```python
# let's plot cos from -pi to pi
def draw(p, title):
    test = []
    for i in range(-pi*100, pi*100):
        if p == pi:
            ans = cos(pi - i/100)
        elif p == -pi:
            ans = cos(i/100 - pi)
        else:
            ans = cos(i/100)
        test.append(ans)
    x = [x/100 for x in range(-pi*100, pi*100)]
    plt.plot(x,test)
    plt.title(title)
    plt.grid(True)
```

```python
draw(pi, 'pi')
```

```python
draw(-pi, '-pi')
```

```python
draw(0, '0')
```

```python

```


# Upper envelope

This notebook shows how to use the **upperenvelope** module from the **consav** package.

# Model

Consider a **standard consumption-saving** model

\begin{align}
v_{t}(m_{t})&=\max_{c_{t}}\frac{c_{t}^{1-\rho}}{1-\rho}+\beta v_{t+1}(m_{t+1})
\end{align}

where

\begin{align}
a_{t}	&=m_{t}-c_{t} \\
m_{t+1}	&=Ra_{t}+y \\
\end{align}

The **Euler equation** is

\begin{align}
c_{t}^{-\rho} &=\beta Rc_{t+1}^{-\rho}
\end{align}

Assume that the **t+1 consumption and value functions** are given by

\begin{align}
c_{t+1}(m_{t}) &= \sqrt{m_{t}}-\eta_{c} \cdot 1\{m_{t}\geq\underline{m}\} \\
v_{t+1}(m_{t}) &= \sqrt{m_{t}}+\eta_{v}\sqrt{m_{t}-\underline{m}} \cdot 1\{m_{t}\geq\underline{m}\}
\end{align}

This **notebook** shows how to find the **t consumption and value function** using an **upper envelope** code despite the **kink** in the next-period value function.

# Algorithm

1. Specify an increasing grid of $m_t$ indexed by $j$ {${m_1,m_2,...,m_{\#_m}}$}

2. Specify an increasing grid of $a_t $ indexed by $i$ {${a^1,a^2,...,a^{\#_a}}$}

3. For each $i$ compute (using linear interpolation):

  a. Post-decision value function: $w^i = \beta \breve{v}_{t+1}(Ra^i+y)$

  b. Post-decision marginal value of cash: $q^i = \beta R\breve{c}_{t+1}(Ra^i+y)^{-\rho}$

  c. Consumption: $c_i = (q^i)^{-1/\rho}$

  d. Cash-on-hand: $m^i = a^i + c^i$

4. For each $j$:

  a. Constraint: If $m_j < m^1$ then set $c_j = m_j$

  b. Find best segment: If $m_j \geq m^1$ then set $c_j =c_j^{i^{\star}(j)} $ where

  $$
  \begin{align}
  c_j^i=c_j^i+\frac{c^{i+1}-c^i}{m^{i+1}-m^i}(m_j-m^i)
  \end{align}
  $$

  and

  $$
  \begin{align}
  i^{\star}(j)=\arg\max_{i\in\{1,\dots\#_{A}-1\}}\frac{(c_{j}^{i})^{1-\rho}}{1-\rho}+\beta w_{j}^{i} \\
  \end{align}
  $$

  subject to

  $$
  \begin{align}
  m_{j} &\in [m^{i},m^{i+1}] \\
  a_{j}^{i} &= m_{j}-c_{j}^{i} \\
  w_{j}^{i} &= w^{i}+\frac{w^{i+1}-w^{i}}{a^{i+1}-a^{i}}(a_{j}^{i}-a^{i})
  \end{align}
  $$

# Setup

```python
import yaml
yaml.warnings({'YAMLLoadWarning': False})

import numpy as np
from numba import njit

import matplotlib.pyplot as plt
plt.style.use('seaborn-whitegrid')
prop_cycle = plt.rcParams["axes.prop_cycle"]
colors = prop_cycle.by_key()["color"]
import ipywidgets as widgets
```

Choose parameters and create grids:

```python
def setup():

    par = dict()

    # a. model parameters
    par['beta'] = 0.96
    par['rho'] = 2
    par['R'] = 1.02
    par['y'] = 1

    # b. cash-on-hand (exogenous grid)
    par['Nm'] = 10000
    par['m_max'] = 10

    # c. end-of-period assets (exogenous grid)
    par['Na'] = 1000
    par['a_max'] = 10

    # d. next-period consumption and value function
    par['eta_v'] = 0.5
    par['eta_c'] = 0.5
    par['x_ubar'] = 5

    return par

def create_grids(par):

    par['grid_a'] = np.linspace(0,par['a_max'],par['Na'])
    par['grid_m'] = np.linspace(1e-8,par['m_max'],par['Nm'])

    return par

par = setup()
par = create_grids(par)
```

# Next-period functions

Calculate the next-period consumption and value functions:

```python
sol = dict()

# a. consumption function
sol['c_next'] = np.sqrt(par['grid_m']) - par['eta_c']*(par['grid_m'] >= par['x_ubar']);

# b. value function
sol['v_next'] = np.sqrt(par['grid_m']) + par['eta_v']*np.sqrt(np.fmax(par['grid_m']-par['x_ubar'],0))*(par['grid_m'] >= par['x_ubar'])
```

## Figures

Plot them to see the jump in consumption and the kink in the value function.

```python
# a. next-period consumption function
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

ax.plot(par['grid_m'],sol['c_next'],'o',MarkerSize=0.5)

ax.set_title('next-period consumption function')
ax.set_xlabel('$m_t$')
ax.set_ylabel('$c_t$')

# b. next-period value function
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

ax.plot(par['grid_m'],sol['v_next'],'o',MarkerSize=0.5)

ax.set_title('next-period value function')
ax.set_xlabel('$m_t$')
ax.set_ylabel('$v_t$');
```

# EGM

```python
from consav import linear_interp # linear interpolation
```

Apply the EGM algorithm.

```python
@njit
def u(c,rho):
    return c**(1-2)/(1-2)

def marg_u(c,par):
    return c**(-par['rho'])

def inv_marg_u(u,par):
    return u**(-1.0/par['rho'])

def EGM(par,sol):

    # a. next-period cash-on-hand
    m_plus = par['R']*par['grid_a'] + par['y']

    # b. post-decision value function
    sol['w_vec'] = np.empty(m_plus.size)
    linear_interp.interp_1d_vec(par['grid_m'],sol['v_next'],m_plus,sol['w_vec'])

    # c. post-decision marginal value of cash
    c_next_interp = np.empty(m_plus.size)
    linear_interp.interp_1d_vec(par['grid_m'],sol['c_next'],m_plus,c_next_interp)
    q = par['beta']*par['R']*marg_u(c_next_interp,par)

    # d. EGM
    sol['c_vec'] = inv_marg_u(q,par)
    sol['m_vec'] = par['grid_a'] + sol['c_vec']

    return sol

sol = EGM(par,sol)
```

## Figures

Plot the result of the EGM algorithm to see that the its does not define a consumption function.

```python
# a. raw consumption function
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

ax.plot(sol['m_vec'],sol['c_vec'],'o',MarkerSize=0.5)

ax.set_title('raw consumption points')
ax.set_xlabel('$m_t$')
ax.set_ylabel('$c_t$')

# b. raw value function
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

ax.plot(sol['m_vec'],sol['w_vec'],'o',MarkerSize=0.5)

ax.set_title('raw value function points')
ax.set_xlabel('$m_t$')
ax.set_ylabel('$w_t$');
```

# Upper envelope

```python
from consav import upperenvelope

# a. create
myupperenvelope = upperenvelope.create(u) # where is the utility function

# b. apply
c_ast_vec = np.empty(par['grid_m'].size) # output
v_ast_vec = np.empty(par['grid_m'].size) # output
myupperenvelope(par['grid_a'],sol['m_vec'],sol['c_vec'],sol['w_vec'],par['grid_m'],c_ast_vec,v_ast_vec,par['rho'])
```

## Figures

```python
# a. consumption function
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

ax.plot(par['grid_m'],c_ast_vec,'o',MarkerSize=0.5)

ax.set_title('consumption function')
ax.set_xlabel('$m_t$')
ax.set_ylabel('$c_t$')

# b. value function
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

ax.plot(par['grid_m'],v_ast_vec,'o',MarkerSize=0.5)

ax.set_title('value function')
ax.set_xlabel('$m_t$')
ax.set_ylabel('$v_t$')

ax.set_ylim((-5,5));
```


## **Airport daily percentage of cancelled flights**

In this notebook we compute the percentage of cancelled flights per day, throughout the entire data set.

\begin{equation}
PercentageOfCancelledFlights(dayOfYear) = \frac{CancelledFlights(dayOfYear)}{NumberOfFlights(dayOfYear)} * 100
\end{equation}

#### Execution mode

If development mode is turned on, all the calculations will be performed on a reduced instance of the data set.

This allows to *save computation time* while developing.

```python
# development mode
# False when performing real analytics
DEV = False

### threads to be used to run spark worker nodes locally
spark_local_threads = 4

# default transformed dataset path

transformed_dataset_path = '../dataset/01_cache_daily_cancelled_analytics.parquet'
missings_dataframe_path = '../dataset/01_cache_daily_cancelled_analytics_missings.pkl'
```

#### Find and import Apache Spark on current machine

```python
import findspark
findspark.init()
```

```python
# imports
from pyspark.sql import SparkSession
import pyspark.sql.functions as F
from pyspark.sql.types import *

from pathlib import Path

from missing_values_utils import *
```

#### Create a new session

```python
# Build a Spark SQL Session for DataFrames
master = 'local[{}]'.format(spark_local_threads)
appName = 'Daily cancelled filghts'
spark = SparkSession  \
    .builder          \
    .appName(appName) \
    .master(master)   \
    .getOrCreate()
```

### Load data

If there exists a cached version of the already transformed dataset, just load it and skip all the unnecessary computation.

```python
do_computation = True

if not DEV and Path(transformed_dataset_path).is_dir():
    df = spark.read.load(transformed_dataset_path)
    do_computation = False
```

Try to load the optimized parquet format data set. If parquet data set is not found, load full compressed data sets, reduce and save them.

```python
from preprocessing_utils import *

if do_computation:
    if DEV:
        # DEV preprocessing
        perform_DEV_dataset_preprocessing(spark)
    else:
        # Production preprocessing
        perform_dataset_preprocessing(spark)
```

    Starting preprocessing of ../dataset/*.csv.bz2
    Preprocessing NOT performed.
    Preprocessed dataset already exists: ../dataset/preprocessed_dataset.parquet

```python
# Load the parquet dataset
if do_computation:
    if DEV:
        # Load DEV dataset
        df = load_DEV_preprocessed_dataset(spark)
    else:
        # Load production dataset
        df = load_preprocessed_dataset(spark)
```

    Peprocessed dataset loaded.
    ../dataset/preprocessed_dataset.parquet

### Attributes selection

```python
if do_computation:
    # Keep only the dimensions we need
    df = df.select(df['Year'], df['Month'], df['DayofMonth'], df['Cancelled'])
    # Explore the data
    df.printSchema()
```

    root
     |-- Year: integer (nullable = true)
     |-- Month: integer (nullable = true)
     |-- DayofMonth: integer (nullable = true)
     |-- Cancelled: integer (nullable = true)

```python
# get years from dataset
if do_computation:
    df_years = df.select('Year').distinct().toPandas()['Year']
else:
    df_years = list(df.columns)
    df_years.remove('DayOfMonth')
    df_years.remove('Month')

df_years = sorted(df_years)
print(df_years)
```

    [1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008]

### Missing values exploration and exclusion

```python
df.count()
```

    91469371

```python
df, df_missing = get_df_missing_and_drop_all_null(df, do_computation, missings_dataframe_path)
```

    Years examined:
     [1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008]

    Missing values inspection:
    0  null values in:  Year
    0  null values in:  Month
    0  null values in:  DayofMonth
    0  null values in:  Cancelled

            Count  MissingsCount  MissingsPercentage
    1994  5180048              0                 0.0
    1995  5327435              0                 0.0
    1996  5351983              0                 0.0
    1997  5411843              0                 0.0
    1998  5384721              0                 0.0
    1999  5527884              0                 0.0
    2000  5683047              0                 0.0
    2001  5967780              0                 0.0
    2002  5271359              0                 0.0
    2003  6488540              0                 0.0
    2004  7129270              0                 0.0
    2005  7140596              0                 0.0
    2006  7141922              0                 0.0
    2007  7453215              0                 0.0
    2008  7009728              0                 0.0

```python
df.count()
```

    91469371

### Compute daily-percentage analytics

```python
if do_computation:
    df.describe('Cancelled').show()
    df.describe('Year').show()
```

    +-------+--------------------+
    |summary|           Cancelled|
    +-------+--------------------+
    |  count|            91469371|
    |   mean|0.021325903727926587|
    | stddev|  0.1444683695010413|
    |    min|                   0|
    |    max|                   1|
    +-------+--------------------+

    +-------+------------------+
    |summary|              Year|
    +-------+------------------+
    |  count|          91469371|
    |   mean|2001.5266289411786|
    | stddev| 4.332419506702069|
    |    min|              1994|
    |    max|              2008|
    +-------+------------------+

```python
if do_computation:
    df = df.withColumn("Count", F.lit(1))

    df = df.groupBy(['Year', 'Month', 'DayOfMonth'])                \
           .sum('Count', 'Cancelled')                               \
           .withColumnRenamed('sum(Count)', 'Count')                \
           .withColumnRenamed('sum(Cancelled)', 'Cancelled')

    df.show(10)
```

    +----+-----+----------+-----+---------+
    |Year|Month|DayOfMonth|Count|Cancelled|
    +----+-----+----------+-----+---------+
    |2007|    1|        18|20815|      650|
    |2007|    1|        19|20867|      511|
    |2007|    9|        22|17136|      150|
    |2006|    2|         3|19592|      413|
    |2006|    8|        23|20386|      233|
    |2004|    8|        14|18184|      863|
    |2005|    1|        15|16826|      315|
    |2005|   12|        25|16618|      177|
    |2005|   12|         1|19246|      286|
    |1995|    3|        20|15186|      232|
    +----+-----+----------+-----+---------+
    only showing top 10 rows

#### Leap years management

```python
leap_years = range(1996,2009,4)
list(leap_years)
```

    [1996, 2000, 2004, 2008]

```python
# insert a zero flights count and a zero cancelled flights count for all the leap years
if do_computation:
    for i in range(1994, 2009):
        if i not in leap_years:
            new_row = spark.createDataFrame([[i, 2, 29, 0, 0]])
            df = df.union(new_row)
```

```python
# Compute the percentage
if do_computation:
    df = df.withColumn('DailyCancelledFlightsPercentage', (df.Cancelled / df.Count) * 100)
    df = df.select('Year', 'Month', 'DayOfMonth', 'DailyCancelledFlightsPercentage')
```

```python
if do_computation:
    df.show(20)
```

    +----+-----+----------+-------------------------------+
    |Year|Month|DayOfMonth|DailyCancelledFlightsPercentage|
    +----+-----+----------+-------------------------------+
    |2007|    1|        18|              3.122748018256065|
    |2007|    1|        19|             2.4488426702448844|
    |2007|    9|        22|             0.8753501400560224|
    |2006|    2|         3|             2.1080032666394444|
    |2006|    8|        23|             1.1429412341803198|
    |2004|    8|        14|              4.745930488341399|
    |2005|    1|        15|             1.8721026982051587|
    |2005|   12|        25|             1.0651101215549403|
    |2005|   12|         1|             1.4860230697287748|
    |1995|    3|        20|              1.527722902673515|
    |1995|    5|        24|              1.339165545087483|
    |1994|   10|        28|            0.24920859432882064|
    |1994|   12|        26|              1.077893024201749|
    |2000|    1|         3|             2.8468627698522915|
    |2000|    5|        15|             1.6666666666666667|
    |2000|    8|         9|             3.5416280363434085|
    |2002|    6|        30|             0.3976813157185225|
    |2002|   12|        28|            0.30716999434160536|
    |1999|    6|        25|             2.2761856454476286|
    |1999|    9|        20|              2.866885577165951|
    +----+-----+----------+-------------------------------+
    only showing top 20 rows

#### Data format optimazion

```python
# dataset transformation so to have:
# - for each year a column with the daily cancelled percentages
# - have the dataset rows beign 366: days in a year
if do_computation:
    for y in df_years:
        df = df.withColumn(str(y) + '_val', df.Year == y)
        df = df.withColumn(str(y) + '_val', df[str(y) + '_val'].cast(IntegerType()))
        df = df.withColumn(str(y), df[str(y) + '_val'] * df.DailyCancelledFlightsPercentage)
        df = df.drop(str(y) + '_val')

    df = df.drop('DailyCancelledFlightsPercentage')

    # do the sum so that the columns will shrink all to the same size of 366 rows
    df = df.groupBy('Month', 'DayOfMonth')                     \
           .sum()                                              \
           .drop('sum(Year)', 'sum(Month)', 'sum(DayOfMonth)') \
           .orderBy('Month', 'DayOfMonth')

    # rename colums as year name
    for y in df_years:
        df = df.withColumnRenamed('sum({})'.format(y), str(y))

```

```python
if do_computation:
    df.write.mode('overwrite').save(transformed_dataset_path, format='parquet')
```

```python
df.printSchema()
```

    root
     |-- Month: long (nullable = true)
     |-- DayOfMonth: long (nullable = true)
     |-- 1994: double (nullable = true)
     |-- 1995: double (nullable = true)
     |-- 1996: double (nullable = true)
     |-- 1997: double (nullable = true)
     |-- 1998: double (nullable = true)
     |-- 1999: double (nullable = true)
     |-- 2000: double (nullable = true)
     |-- 2001: double (nullable = true)
     |-- 2002: double (nullable = true)
     |-- 2003: double (nullable = true)
     |-- 2004: double (nullable = true)
     |-- 2005: double (nullable = true)
     |-- 2006: double (nullable = true)
     |-- 2007: double (nullable = true)
     |-- 2008: double (nullable = true)

```python
df.select('1994', '1996', '2001', '2007').show(10)
```

    +------------------+------------------+------------------+------------------+
    |              1994|              1996|              2001|              2007|
    +------------------+------------------+------------------+------------------+
    |0.5264023688106596|3.0900723208415517| 7.400037311112493|1.8350968665337628|
    |0.4492230650268797|  6.72579453067258| 4.008630479218715|0.9954238807378403|
    | 1.541819205857505|7.1923102776097325|3.2720650998691174| 0.839897504033406|
    |15.560882746950574|2.8392652896454282|2.6977803073420605|  2.37118320610687|
    | 4.765613935703159|2.9633113828786453| 4.989758761948111|2.7675716944219118|
    | 5.416755640970887|  5.07185122569738|3.2939955907145637| 1.141539672721138|
    | 8.754974417282547| 25.97874919774656| 2.738993523854792| 1.124558749369642|
    | 9.567631892106307|29.014598540145986|2.8929163569835916|1.7102229054573406|
    |1.1112791049289386|16.927559802127803| 3.116957392050329|1.0085954190887862|
    |1.8481801444554598| 5.157224179994579| 4.055368071841217|0.7662458863401935|
    +------------------+------------------+------------------+------------------+
    only showing top 10 rows

```python
pdf = df.toPandas()
```

```python
# rows?
assert(366 == pdf.shape[0])
pdf.shape
```

    (366, 17)

```python
# update index to match the day of the year!
pdf.index = pdf.index + 1
```

```python
#pdf.head(65)
pdf.head()
```

      Month
      DayOfMonth
      1994
      1995
      1996
      1997
      1998
      1999
      2000
      2001
      2002
      2003
      2004
      2005
      2006
      2007
      2008

      1
      1
      1
      0.526402
      1.058201
      3.090072
      2.313218
      0.841365
      2.745820
      5.616711
      7.400037
      0.824279
      0.817641
      1.156372
      1.225079
      1.565285
      1.835097
      2.887372

      2
      1
      2
      0.449223
      1.085271
      6.725795
      4.256721
      1.040861
      22.986174
      1.992935
      4.008630
      4.502776
      1.343912
      3.201092
      0.931881
      4.043623
      0.995424
      2.438675

      3
      1
      3
      1.541819
      1.775262
      7.192310
      2.808102
      1.617592
      15.144628
      2.846863
      3.272065
      6.581214
      2.024377
      1.500753
      1.617964
      3.611637
      0.839898
      2.268711

      4
      1
      4
      15.560883
      2.272874
      2.839265
      1.771453
      1.676289
      6.294622
      3.369008
      2.697780
      2.311980
      1.314240
      6.338884
      2.966507
      1.429391
      2.371183
      3.674327

      5
      1
      5
      4.765614
      2.155619
      2.963311
      3.017984
      3.157408
      4.057876
      2.185376
      4.989759
      0.379687
      1.349053
      3.019592
      7.438643
      0.901448
      2.767572
      2.523659

```python
pdf.tail()
```

      Month
      DayOfMonth
      1994
      1995
      1996
      1997
      1998
      1999
      2000
      2001
      2002
      2003
      2004
      2005
      2006
      2007
      2008

      362
      12
      27
      0.890943
      1.028069
      4.551761
      0.508069
      3.134514
      1.167192
      3.008695
      1.319929
      0.738084
      0.838833
      3.959081
      1.084935
      1.199437
      2.305393
      4.748991

      363
      12
      28
      0.564023
      1.143824
      3.058104
      0.635164
      3.417621
      2.440258
      2.413772
      1.395477
      0.307170
      0.658997
      2.752881
      0.952381
      2.024370
      4.510253
      1.570976

      364
      12
      29
      0.361882
      1.192008
      3.448754
      2.978921
      3.567425
      1.436436
      3.298360
      0.319788
      0.323078
      0.980549
      1.453488
      1.026196
      6.553163
      0.940749
      0.744337

      365
      12
      30
      0.564127
      0.484541
      4.328033
      2.598183
      2.187953
      1.449644
      16.860670
      0.570613
      2.808713
      0.707927
      1.843114
      1.124755
      3.250604
      1.522896
      1.537383

      366
      12
      31
      0.992063
      1.730665
      2.537848
      1.085432
      1.165880
      2.786458
      9.149301
      0.576302
      0.942484
      0.565900
      0.993871
      1.337856
      2.125939
      2.970016
      3.517907

##### Data visualization functions

Scroll down to see the charts

```python
# Hide warnings if there are any
import warnings
warnings.filterwarnings('ignore')

%matplotlib inline

from ipywidgets import interact_manual
import ipywidgets as widgets
import matplotlib.pyplot as plt
from matplotlib.font_manager import FontProperties
import pandas as pd
import numpy as np

import calendar
```

```python
fig_size=[15,10]
title_size=20
```

```python
def get_years_df(years):
    """
    Return the dataframe of the selected years
    """
    years = [str(y) for y in range(*years)]
    return pdf[years]

def plot_cancelled_time_series(years):
    # select requested years from dataframe
    df = get_years_df(years)

    # xticks are the last day of the month counted as day of the year
    xticks = [0]
    for m in range(1,13):
        new_val = xticks[-1] + calendar.monthrange(2016, m)[1]
        xticks.append(new_val)

    # xlabels are the months abbreviations
    xlabels = list(calendar.month_abbr)[1:]

    plt.rcParams['figure.figsize'] = fig_size
    plt.rcParams['axes.titlesize'] = title_size
    fig, ax = plt.subplots()

    ax.set_xticks(xticks)
    ax.set_xticklabels(xlabels)

    ax.set_xlabel('Months', size=16)
    ax.set_ylabel('Cancellation percentage', size=16)
    ax.set_title('Daily cancelled flights percentage', pad=20)

    plt.xlim(xmin=-4, xmax=370)
    df.plot(grid=True, rot=45, ax=ax)
    plt.legend(loc='best')

def plot_average_cancelled_flights(years):
    # select requested years from dataframe
    df = get_years_df(years)
    averages = []
    for y in range(*years):
        averages.append(df[str(y)].mean())

    labels = [str(y) for y in range(*years)]

    plt.rcParams['figure.figsize'] = fig_size
    plt.rcParams['axes.titlesize'] = title_size
    fig, ax = plt.subplots()
    indexes = np.arange(len(averages))

    rects = ax.bar(indexes, averages, label=labels, color='orangered')

    # add percentages on bars
    for rect in rects:
        height = rect.get_height()
        ax.text(rect.get_x() + rect.get_width()/2., height - 0.18,
                str(np.round(height, decimals=2)) + '%',
                ha='center', va='bottom', color='w', size=13)

    ax.set_xlabel('Years', size=16)
    ax.set_ylabel('Cancellation percentage', size=16)
    ax.set_title('Average of cancelled flights percentages', pad=20)

    ax.set_xticks(indexes)
    ax.set_xticklabels(labels)
    ax.xaxis.set_tick_params(labelsize=14)
    ax.legend(loc='best', labels=['Cancellation percentage'])
    plt.show()
```

```python
def ui_callback(years):
    years = (int(years[0]), int(years[1]) + 1)  #due to range() function

    plot_cancelled_time_series(years)

    plot_average_cancelled_flights(years)

# Years selection range
years_w = widgets.SelectionRangeSlider(options=[(str(y), y) for y in df_years],
                                       index=(0, 2),
                                       description='Years',
                                       continuous_update=False)
ui = widgets.HBox([years_w])
```

## Data Visualization

Analytics showhing the percentage of daily cancelled flights are reported below.

### Missing values and valid data

```python
plot_missing_values_stacked_bar(df_missing)
```

### Percentage of daily cancelled flights

A line plot was used to display percentages as a time series.

- _x axis_ is the day number, from day 1 (1st of January of the selected year) to 366 (30th of December of the selected year)
- _y axis_ reports the daily percentage of cancelled flights

Moreover, also bar plot is generated to display the yearly average of cancelled flights.

#### Data discoveries

- The analytics is performed over the totality of the data. There're no missing values for the analized attributes.

```python
out = widgets.interactive_output(ui_callback, {'years': years_w})
display(ui, out)
```

    HBox(children=(SelectionRangeSlider(continuous_update=False, description='Years', index=(0, 2), options=(('199…

    Output()