# Finite Difference Methods for the Laplacian Equation
#### John S Butler john.s.butler@tudublin.ie   [Course Notes](https://johnsbutler.netlify.com/files/Teaching/Numerical_Analysis_for_Differential_Equations.pdf)    [Github](https://github.com/john-s-butler-dit/Numerical-Analysis-Python)
## Overview

This notebook will focus on numerically approximating a homogenous second order Poisson Equation which is the Laplacian Equation.
## The Differential Equation
The general two dimensional Poisson Equation is of the form:
\begin{equation} \frac{\partial^2 u}{\partial y^2} + \frac{\partial^2 u}{\partial x^2}=f(x,y), \ \ \ (x,y) \in \Omega=(0,1)\times (0,1),\end{equation}
with boundary conditions
\begin{equation}U(x,y) = g(x,y), \ \ \  (x,y)\in\delta\Omega\text{ - boundary}. \end{equation}
## Homogenous Poisson Equation
This notebook will implement a finite difference scheme to approximate the homogenous form of the Poisson Equation $f(x,y)=0$:
\begin{equation} \frac{\partial^2 u}{\partial y^2} + \frac{\partial^2 u}{\partial x^2}=0.\end{equation}
with the Boundary Conditions:
\begin{equation} u(x,0)=\sin(2\pi x), \ \ \ \ \ 0 \leq x \leq 1, \text{ lower},\end{equation}
\begin{equation}u(x,1)=\sin(2\pi x), \ \ \ \ \ 0 \leq x \leq 1, \text{ upper},\end{equation}
\begin{equation} u(0,y)=2\sin(2\pi y), \ \ \ \ \ 0 \leq y \leq 1, \text{ left},\end{equation}
\begin{equation} u(1,y)=2\sin(2\pi y), \ \ \ \ \ 0 \leq y \leq 1, \text{ right}.\end{equation}

```python
# LIBRARY
# vector manipulation
import numpy as np
# math functions
import math

# THIS IS FOR PLOTTING
%matplotlib inline
import matplotlib.pyplot as plt # side-stepping mpl backend
import warnings
warnings.filterwarnings("ignore")
from IPython.display import HTML
from mpl_toolkits.mplot3d import axes3d
import matplotlib.pyplot as plt

```

## Discete Grid
The region $\Omega=(0,1)\times(0,1)$ is discretised into a uniform mesh $\Omega_h$. In the  $x$ and $y$ directions into $N$ steps giving a stepsize of
\begin{equation} h=\frac{1-0}{N},\end{equation}
resulting in
\begin{equation}x[i]=0+ih, \ \ \  i=0,1,...,N,$\end{equation}
and
\begin{equation}x[j]=0+jh, \ \ \  j=0,1,...,N,\end{equation}
The Figure below shows the discrete grid points for $N=10$,  the known boundary conditions (green),  and the unknown values (red) of the Poisson Equation.

```python
N=10
h=1/N
x=np.arange(0,1.0001,h)
y=np.arange(0,1.0001,h)
X, Y = np.meshgrid(x, y)
fig = plt.figure()
plt.plot(x[1],y[1],'ro',label='unknown');
plt.plot(X,Y,'ro');
plt.plot(np.ones(N+1),y,'go',label='Boundary Condition');
plt.plot(np.zeros(N+1),y,'go');
plt.plot(x,np.zeros(N+1),'go');
plt.plot(x, np.ones(N+1),'go');
plt.xlim((-0.1,1.1))
plt.ylim((-0.1,1.1))
plt.xlabel('x')
plt.ylabel('y')
plt.gca().set_aspect('equal', adjustable='box')
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))
plt.title(r'Discrete Grid $\Omega_h,$ h= %s'%(h),fontsize=24,y=1.08)
plt.show();
```

## Boundary Conditions

The  discrete boundary conditions are
\begin{equation} w_{i0}=w[i,0]=\sin(2\pi x[i]), \text{ for } i=0,...,10, \text{ upper},\end{equation}

\begin{equation}  w_{iN}=w[i,N]=\sin(2\pi x[i]), \text{ for } i=0,...,10,  \text{ lower},\end{equation}

\begin{equation}  w_{0j}=w[0,j]=2\sin(2\pi y[j]), \text{ for } j=0,...,10,   \text{ left},\end{equation}
\begin{equation}  w_{Nj}=w[N,j]=2\sin(2\pi y[j]), \text{ for } i=0,...,10,\text{ right}. \end{equation}

The Figure below plots the boundary values of $w[i,j]$.

```python
w=np.zeros((N+1,N+1))

for i in range (0,N):
        w[i,0]=np.sin(2*np.pi*x[i]) #left Boundary
        w[i,N]=np.sin(2*np.pi*x[i]) #Right Boundary

for j in range (0,N):
        w[0,j]=2*np.sin(2*np.pi*y[j]) #Lower Boundary
        w[N,j]=2*np.sin(2*np.pi*y[j]) #Upper Boundary

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
# Plot a basic wireframe.
ax.plot_wireframe(X, Y, w,color='r', rstride=10, cstride=10)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('w')
plt.title(r'Boundary Values',fontsize=24,y=1.08)
plt.show()
```

## Numerical Method
The Poisson Equation  is discretised using
$\delta_x^2$ is the central difference approximation of the second derivative in the $x$ direction
\begin{equation}\delta_x^2=\frac{1}{h^2}(w_{i+1j}-2w_{ij}+w_{i-1j}), \end{equation}
and $\delta_y^2$ is the central difference approximation of the second derivative in the $y$ direction
\begin{equation}\delta_y^2=\frac{1}{h^2}(w_{ij+1}-2w_{ij}+w_{ij-1}). \end{equation}
The gives the Poisson Difference Equation,
\begin{equation}-(\delta_x^2w_{ij}+\delta_y^2w_{ij})=f_{ij} \ \ (x_i,y_j) \in \Omega_h, \end{equation}
\begin{equation}w_{ij}=g_{ij} \ \ (x_i,y_j) \in \partial\Omega_h, \end{equation}
where $w_ij$ is the numerical approximation of $U$ at $x_i$ and $y_j$.
Expanding the  the Poisson Difference Equation gives the five point method,
\begin{equation}-(w_{i-1j}+w_{ij-1}-4w_{ij}+w_{ij+1}+w_{i+1j})=h^2f_{ij} \end{equation}
for $i=1,...,N-1$ and $j=1,...,N-1$ which can be written

\begin{equation}\nabla^2_h w_{ij}=f_{ij}. \end{equation}

### Matrix form
This can be written as a system of $(N-1)\times(N-1)$ equations can be arranged in matrix form
\begin{equation} A\mathbf{w}=\mathbf{r},\end{equation}
where $A$ is an $(N-1)^2\times(N-1)^2$  matrix made up of the following block tridiagonal structure
\begin{equation}\left(\begin{array}{ccccccc}
T&I&0&0&.&.&.\\
I&T&I&0&0&.&.\\
.&.&.&.&.&.&.\\
.&.&.&0&I&T&I\\
.&.&.&.&0&I&T\\
\end{array}\right),
\end{equation}
where $I$ denotes an $N-1 \times N-1$ identity matrix and $T$ is the tridiagonal matrix of the form:
\begin{equation} T=\left(\begin{array}{ccccccc}
-4&1&0&0&.&.&.\\
1&-4&1&0&0&.&.\\
.&.&.&.&.&.&.\\
.&.&.&0&1&-4&1\\
.&.&.&.&0&1&-4\\
\end{array}\right).
\end{equation}
The plot below shows the matrix $A$ and its inverse $A^{-1}$ as a colourplot.

```python
N2=(N-1)*(N-1)
A=np.zeros((N2,N2))
## Diagonal
for i in range (0,N-1):
    for j in range (0,N-1):
        A[i+(N-1)*j,i+(N-1)*j]=-4

# LOWER DIAGONAL
for i in range (1,N-1):
    for j in range (0,N-1):
        A[i+(N-1)*j,i+(N-1)*j-1]=1
# UPPPER DIAGONAL
for i in range (0,N-2):
    for j in range (0,N-1):
        A[i+(N-1)*j,i+(N-1)*j+1]=1

# LOWER IDENTITY MATRIX
for i in range (0,N-1):
    for j in range (1,N-1):
        A[i+(N-1)*j,i+(N-1)*(j-1)]=1

# UPPER IDENTITY MATRIX
for i in range (0,N-1):
    for j in range (0,N-2):
        A[i+(N-1)*j,i+(N-1)*(j+1)]=1
Ainv=np.linalg.inv(A)
fig = plt.figure(figsize=(12,4));
plt.subplot(121)
plt.imshow(A,interpolation='none');
clb=plt.colorbar();
clb.set_label('Matrix elements values');
plt.title('Matrix A ',fontsize=24)
plt.subplot(122)
plt.imshow(Ainv,interpolation='none');
clb=plt.colorbar();
clb.set_label('Matrix elements values');
plt.title(r'Matrix $A^{-1}$ ',fontsize=24)

fig.tight_layout()
plt.show();
```

The vector $\mathbf{w}$ is of length $(N-1)\times(N-1)$ which made up of $N-1$ subvectors $\mathbf{w}_j$  of length $N-1$ of the form
\begin{equation}\mathbf{w}_j=\left(\begin{array}{c}
w_{1j}\\
w_{2j}\\
.\\
.\\
w_{N-2j}\\
w_{N-1j}\\
\end{array}\right).
\end{equation}
The vector $\mathbf{r}$ is of length $(N-1)\times(N-1)$ which made up of $N-1$ subvectors of the form $\mathbf{r}_j=-h^2\mathbf{f}_j-\mathbf{bx}_{j}-\mathbf{by}_j$,
where $\mathbf{bx}_j $ is the vector of left and right boundary conditions
\begin{equation}\mathbf{bx}_j =\left(\begin{array}{c}
w_{0j}\\
0\\
.\\
.\\
0\\
w_{Nj}
\end{array}\right),
\end{equation}

for $j=1,..,N-1$, where $\mathbf{by}_j$ is the vector of the lower boundary condition for $j=1$,

\begin{equation}
\mathbf{by}_{1} =\left(\begin{array}{c}
w_{10}\\
w_{20}\\
.\\
.\\
w_{N-20}\\
w_{N-10}\\
\end{array}\right),
\end{equation}
upper boundary condition for $j=N-1$

\begin{equation}
\mathbf{by}_{N-1} =\left(\begin{array}{c}
w_{1N}\\
w_{2N}\\
.\\
.\\
w_{N-2N}\\
w_{N-1N}\\
\end{array}\right),
\end{equation}
for $j=2,...,N-2$
\begin{equation}\mathbf{by}_j=0,\end{equation}
and
\begin{equation}\mathbf{f}_j =-\left(\begin{array}{c}
0\\
0\\
.\\
.\\
0\\
0\\
\end{array}\right)
\end{equation}
for $j=1,...,N-1$.

```python
r=np.zeros(N2)

# vector r
for i in range (0,N-1):
    for j in range (0,N-1):
        r[i+(N-1)*j]=-h*h*0
# Boundary
b_bottom_top=np.zeros(N2)
for i in range (0,N-1):
    b_bottom_top[i]=np.sin(2*np.pi*x[i+1]) #Bottom Boundary
    b_bottom_top[i+(N-1)*(N-2)]=np.sin(2*np.pi*x[i+1])# Top Boundary

b_left_right=np.zeros(N2)
for j in range (0,N-1):
    b_left_right[(N-1)*j]=2*np.sin(2*np.pi*y[j+1]) # Left Boundary
    b_left_right[N-2+(N-1)*j]=2*np.sin(2*np.pi*y[j+1])# Right Boundary

b=b_left_right+b_bottom_top
```

## Results

To solve the system for $\mathbf{w}$ invert the matrix $A$
\begin{equation} A\mathbf{w}=\mathbf{r},\end{equation}
such that
\begin{equation} \mathbf{w}=A^{-1}\mathbf{r}.\end{equation}
Lastly, as $\mathbf{w}$ is in vector it has to be reshaped into grid form to plot.

The figure below shows the numerical approximation of the homogeneous Equation.

```python
C=np.dot(Ainv,r-b)
w[1:N,1:N]=C.reshape((N-1,N-1))

fig = plt.figure(figsize=(8,6))
ax = fig.add_subplot(111, projection='3d');
# Plot a basic wireframe.
ax.plot_wireframe(X, Y, w,color='r');
ax.set_xlabel('x');
ax.set_ylabel('y');
ax.set_zlabel('w');
plt.title(r'Numerical Approximation of the Poisson Equation',fontsize=24,y=1.08);
plt.show();
```

# Consistency and Convergence
We now ask how well the grid function determined by the five point scheme approximates the exact solution of the Poisson problem.
## Consistency

### Consitency (Definition)
Let
\begin{equation}\nabla^2_h(\varphi)=-(\varphi_{i-1j}+\varphi_{ij-1}-4\varphi_{ij}+\varphi_{ij+1}+\varphi_{i+1j})\end{equation}
denote the finite difference approximation associated with the grid $\Omega_h$ having the mesh size $h$, to a partial differential operator
\begin{equation}\nabla^2(\varphi)=\frac{\partial^2 \varphi}{\partial x^2}+\frac{\partial^2 \varphi}{\partial y^2}\end{equation}
defined on
a simply connected, open set $\Omega \subset R^2$. For a given function $\varphi\in C^{\infty}(\Omega)$,
the truncation error of $\nabla^2_h$ is
\begin{equation}\tau_{h}(\mathbf{x})=(\nabla^2-\nabla^2_h)\varphi(\mathbf{x}) \end{equation}
The approximation $\nabla^2_h$ is consistent with $\nabla^2$ if
\begin{equation} \lim_{h\rightarrow 0}\tau_h(\mathbf{x})=0,\end{equation}
for all $\mathbf{x} \in D$ and all $\varphi \in C^{\infty}(\Omega)$. The approximation is consistent to order $p$ if $\tau_h(\mathbf{x})=O(h^p)$.

_In other words a method is consistent with the differential equation it is approximating._

## Proof of Consistency
The five-point difference analog $\nabla^2_h$ is consistent to order 2 with $\nabla^2$.

__Proof__

Pick $\varphi \in C^{\infty}(D)$, and let $(x,y) \in \Omega$ be a point such that $(x\pm h, y),(x,y \pm h) \in \Omega\bigcup \partial\Omega$.  By the Taylor Theorem
\begin{eqnarray*}
\varphi(x\pm h,y)&=&\varphi(x,y) \pm h \frac{\partial \varphi}{\partial x}(x,y)+\frac{h^2}{2!}\frac{\partial^2 \varphi}{\partial x^2}(x,y) \pm\frac{h^3}{3!}\frac{\partial^3 \varphi}{\partial x^3}(x,y)+\frac{h^4}{4!}\frac{\partial^4 \varphi}{\partial x^4}(\zeta^{\pm},y)
\end{eqnarray*}
where $\zeta^{\pm} \in (x-h,x+h)$. Adding this pair of equation together and rearranging , we get
\begin{equation}\frac{1}{h^2}[\varphi(x+h,y)-2\varphi(x,y)+\varphi(x-h,y) ] -\frac{\partial^2 \varphi}{\partial x^2}(x,y)=\frac{h^2}{4!}\left[\frac{\partial^4 \varphi}{\partial x^4}(\zeta^{+},y)+
\frac{\partial^4 \varphi}{\partial x^4}(\zeta^{-},y)
 \right]
\end{equation}
By the intermediate value theorem
\begin{equation}\left[\frac{\partial^4 \varphi}{\partial x^4}(\zeta^{+},y)+
\frac{\partial^4 \varphi}{\partial x^4}(\zeta^{-},y)
 \right]
=2\frac{\partial^4 \varphi}{\partial x^4}(\zeta,y),\end{equation}
for some $\zeta \in (x-h,x+h)$.  Therefore,
\begin{equation}\delta_x^2(x,y)
=\frac{\partial^2 \varphi}{\partial x^2}(x,y)+\frac{h^2}{2!}\frac{\partial^4 \varphi}{\partial x^4}(\zeta,y)\end{equation}
Similar reasoning shows that
\begin{equation}\delta_y^2(x,y)
=\frac{\partial^2 \varphi}{\partial y^2}(x,y)+\frac{h^2}{2!}\frac{\partial^4 \varphi}{\partial y^4}(x,\eta)
\end{equation}
for some $\eta \in (y-h,y+h)$. We conclude that $\tau_h(x,y)=(\nabla-\nabla_h)\varphi(x,y)=O(h^2).$

## Convergence
### Definition
Let $\nabla^2_hw(\mathbf{x}_j)=f(\mathbf{x}_j)$ be a finite difference approximation, defined on a grid mesh size $h$, to a PDE $\nabla^2U(\mathbf{x})=f(\mathbf{x})$ on a simply connected set $D \subset R^n$. Assume that $w(x,y)=U(x,y)$ at all points $(x,y)$ on the boundary $\partial\Omega$.  The finite difference scheme converges (or is convergent) if
\begin{equation} \max_j|U(\mathbf{x}_j)-w(\mathbf{x}_j)| \rightarrow 0 \mbox{  as  } h \rightarrow 0.\end{equation}

### Theorem (DISCRETE MAXIMUM PRINCIPLE).
If $\nabla^2_hV_{ij}\geq 0$ for all points $(x_i,y_j) \in \Omega_h$, then
\begin{equation} \max_{(x_i,y_j)\in\Omega_h}V_{ij}\leq  \max_{(x_i,y_j)\in\partial\Omega_h}V_{ij},\end{equation}
If $\nabla^2_hV_{ij}\leq 0$ for all points $(x_i,y_j) \in \Omega_h$, then
\begin{equation} \min_{(x_i,y_j)\in\Omega_h}V_{ij}\geq  \min_{(x_i,y_j)\in\partial\Omega_h}V_{ij}.\end{equation}

### Propositions
1. The zero grid function for which $U_{ij}=0$ for all $(x_i,y_j) \in \Omega_h \bigcup \partial\Omega_h$
is the only solution to the finite difference problem
\begin{equation}\nabla_h^2U_{ij}=0 \mbox{ for }(x_i,y_j)\in\Omega_h,\end{equation}
\begin{equation}U_{ij}=0 \mbox{ for }(x_i,y_j)\in\partial\Omega_h.\end{equation}

2. For prescribed grid functions $f_{ij}$ and $g_{ij}$, there exists a unique solution to the problem
\begin{equation}\nabla_h^2U_{ij}=f_{ij} \mbox{ for }(x_i,y_j)\in\Omega_h,\end{equation}
\begin{equation}U_{ij}=g_{ij} \mbox{ for }(x_i,y_j)\in\partial\Omega_h.\end{equation}

### Definition
For any grid function $V:\Omega_h\bigcup\partial\Omega_h \rightarrow R$,
\begin{equation}||V||_{\Omega} =\max_{(x_i,y_j)\in\Omega_h}|V_{ij}|, \end{equation}
\begin{equation}||V||_{\partial\Omega} =\max_{(x_i,y_j)\in\partial\Omega_h}|V_{ij}|. \end{equation}

### Lemma
If the grid function $V:\Omega_h\bigcup\partial\Omega_h\rightarrow R$ satisfies the boundary condition $V_{ij}=0$ for $(x_i,y_j)\in \partial\Omega_h$, then
\begin{equation}||V_||_{\Omega}\leq \frac{1}{8}||\nabla_h^2V||_{\Omega}. \end{equation}

Given these Lemmas and Propositions, we can now prove that the solution to the five point scheme $\nabla^2_h$ is convergent to the exact solution of the Poisson Equation $\nabla^2$.
### Convergence Theorem
Let $U$ be a solution to the Poisson equation and let $w$ be the grid function
that satisfies the discrete analog
\begin{equation}-\nabla_h^2w_{ij}=f_{ij} \ \ \mbox{ for } (x_i,y_j)\in\Omega_h, \end{equation}
\begin{equation}w_{ij}=g_{ij} \ \ \mbox{ for } (x_i,y_j)\in\partial\Omega_h. \end{equation}
Then there exists a positive constant $K$ such that
\begin{equation}||U-w||_{\Omega}\leq KMh^2, \end{equation}
where
\begin{equation} M=\left\{
\left|\left|\frac{\partial^4 U}{\partial x^4} \right|\right|_{\infty},
\left|\left|\frac{\partial^4 U}{\partial y^4} \right|\right|_{\infty}
 \right\}\end{equation}

 __Proof__

 The statement of the theorem assumes that $U\in C^4(\bar{\Omega})$. This assumption
holds if $f$ and $g$ are smooth enough.
\begin{proof}
Following from the proof of the Proposition we have
\begin{equation} (\nabla_h^2-\nabla^2)U_{ij}=\frac{h^2}{12}\left[ \frac{\partial^4 U}{\partial x^4}(\zeta_i,y_j)+\frac{\partial^4 U}{\partial y^4}(x_i,\eta_j) \right],\end{equation}
for some $\zeta \in (x_{i-1},x_{i+1})$ and $\eta_j\in(y_{j-1},y_{j+1})$.  Therefore,
\begin{equation} -\nabla_h^2U_{ij}=f_{ij}-\frac{h^2}{12}\left[ \frac{\partial^4 U}{\partial x^4}(\zeta_i,y_j)+\frac{\partial^4 U}{\partial y^4}(x_i,\eta_j) \right].\end{equation}
If we subtract from this the identity equation $-\nabla_h^2w_{ij}=f_{ij}$ and note
that $U-w$ vanishes on $\partial\Omega_h$, we find that
\begin{equation} \nabla_h^2(U_{ij}-w_{ij})=\frac{h^2}{12}\left[ \frac{\partial^4 U}{\partial x^4}(\zeta_i,y_j)+\frac{\partial^4 U}{\partial y^4}(x_i,\eta_j) \right].\end{equation}
It follows that

\begin{equation} ||U-w||_{\Omega}\leq\frac{1}{8}||\nabla_h^2(U-w)||_{\Omega}\leq KMh^2.\end{equation}

```python

```

```python

```

Chapter 09 - Elliptic Equations/901_Poisson Equation-Laplacian.ipynb


# Getting Started: fitting a Lasso model

The purpose of this tutorial is to show the basics of `glum`. It assumes a working knowledge of python, regularized linear models, and machine learning. The API is very similar to scikit-learn. After all, `glum` is based on a fork of scikit-learn.

If you have not done so already, please refer to our [installation instructions](../install.rst) for installing `glum`.

```python
import pandas as pd
import sklearn
from sklearn.datasets import fetch_openml
from glum import GeneralizedLinearRegressor, GeneralizedLinearRegressorCV
```

## Data

We start by loading the King County housing dataset from openML and splitting it into training and test sets. For simplicity, we don't go into any details regarding exploration or data cleaning.

```python
house_data = fetch_openml(name="house_sales", version=3, as_frame=True)

# Use only select features
X = house_data.data[
    [
        "bedrooms",
        "bathrooms",
        "sqft_living",
        "floors",
        "waterfront",
        "view",
        "condition",
        "grade",
        "yr_built",
    ]
].copy()

# Targets
y = house_data.target
```

```python
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(
    X, y, test_size = 0.3, random_state=5
)
```

## GLM basics: fitting and predicting using the normal family

We'll use `glum.GeneralizedLinearRegressor` to predict the house prices using the available predictors.

We set three key parameters:

- `family`: the family parameter specifies the distributional assumption of the GLM and, as a consequence, the loss function to be minimized. Accepted strings are 'normal', 'poisson', 'gamma', 'inverse.gaussian', and 'binomial'. You can also pass in an instantiated `glum` distribution (e.g. `glum.TweedieDistribution(1.5)` )
- `alpha`: the constant multiplying the penalty term that determines regularization strength. (*Note*: `GeneralizedLinearRegressor` also has an alpha-search option. See the `GeneralizedLinearRegressorCV` example below for details on how alpha-search works).
- `l1_ratio`: the elastic net mixing parameter (`0 <= l1_ratio <= 1`). For `l1_ratio = 0`, the penalty is the L2 penalty (ridge). ``For l1_ratio = 1``, it is an L1 penalty (lasso).  For ``0 < l1_ratio < 1``, the penalty is a combination of L1 and L2.

To be precise, we will be minimizing the function with respect to the parameters, $\beta$:

\begin{equation}
\frac{1}{N}(\mathbf{X}\beta - y)^2 + \alpha\|\beta\|_1
\end{equation}

```python
glm = GeneralizedLinearRegressor(family="normal", alpha=0.1, l1_ratio=1)
```

The `GeneralizedLinearRegressor.fit()` method follows typical sklearn API style and accepts two primary inputs:

1. `X`: the design matrix with shape `(n_samples, n_features)`.
2. `y`: the `n_samples` length array of target data.

```python
glm.fit(X_train, y_train)
```

    GeneralizedLinearRegressor(alpha=0.1, l1_ratio=1)

Once the model has been estimated, we can retrieve useful information using an sklearn-style syntax.

```python
# retrieve the coefficients and the intercept
coefs = glm.coef_
intercept = glm.intercept_

# use the model to predict on our test data
preds = glm.predict(X_test)

preds[0:5]
```

    array([ 482648.22861066,  142902.68859995,  539452.61266391,
            569693.78048569, 1042446.90903451])

## Fitting a GLM with cross validation

Now, we fit using automatic cross validation with `glum.GeneralizedLinearRegressorCV`. This mirrors the commonly used `cv.glmnet` function.

Some important parameters:

- `alphas`: for `GeneralizedLinearRegressorCV`, the best `alpha` will be found by searching along the regularization path. The regularization path is determined as follows:
    1. If `alpha` is an iterable, use it directly. All other parameters
        governing the regularization path are ignored.
    2. If `min_alpha` is set, create a path from `min_alpha` to the
        lowest alpha such that all coefficients are zero.
    3. If `min_alpha_ratio` is set, create a path where the ratio of
        `min_alpha / max_alpha = min_alpha_ratio`.
    4. If none of the above parameters are set, use a `min_alpha_ratio`
        of 1e-6.
- `l1_ratio`: for `GeneralizedLinearRegressorCV`, if you pass `l1_ratio` as an array, the `fit` method will choose the best value of `l1_ratio` and store it as `self.l1_ratio_`.

```python
glmcv = GeneralizedLinearRegressorCV(
    family="normal",
    alphas=None,  # default
    min_alpha=None,  # default
    min_alpha_ratio=None,  # default
    l1_ratio=[0, 0.5, 1.0],
    fit_intercept=True,
    max_iter=150
)
glmcv.fit(X_train, y_train)
print(f"Chosen alpha:    {glmcv.alpha_}")
print(f"Chosen l1 ratio: {glmcv.l1_ratio_}")
```

    Chosen alpha:    0.0003274549162877732
    Chosen l1 ratio: 0.0

Congratulations! You have finished our getting started tutorial. If you wish to learn more, please see our other tutorials for more advanced topics like Poisson, Gamma, and Tweedie regression, high dimensional fixed effects, and spatial smoothing using Tikhonov regularization.

```python

```


# Alunos

### Daniel Cunha Oliveira
### Lucas Iria
### Leandro do Nascimento Silva

# Questão 1

Considere os seguintes dados de um Experimento Completamente Aleatorizado que avaliou a expressão de três genes (E1, E2 e E3) em tecidos tumorais submetidos a quatro tratamentos definidos de acordo com o cruzamento dos níveis baixo (=0) e alto (=1) de dois medicamentos A e B, tais que: T1=A0B0, T2=A0B1, T3=A1B0 e T4=A1B1. O centroide dos dados são dados por:

|Tratamento |  n    |E1    | E2 | E3  |
|--|-------|------|----|-----|
|T1| (n=10)| 1.62 |2.50| 3.60|
|T2| (n=10)| 1.40 |2.43| 3.90|
|T3| (n=10)| 1.80 |2.80| 4.30|
|T4| (n=10)| 1.00 |2.00| 4.00|

a matriz de covariânicia total:
\begin{bmatrix}
   2.65 & 1.08 & 2.51\\
   1.08 & 4.74 & 2.87\\
   2.51 & 2.87 & 6.67
\end{bmatrix}

## item A
Calcule os correspondentes números de graus de liberdade e as Somas de Quadrados e Produtos Cruzados (SQPC) das fontes de variação A, B, A*B.

#### Resolução

Inicialmente começamos carregando na memória o vetor de médias e a matriz de covariancia

```python
import numpy as np
from scipy.stats import chi2
from scipy.stats import f
```

```python
mean_A0B0 = np.array([[1.62,2.50,3.60]]).T
mean_A0B1 = np.array([[1.40,2.43,3.90]]).T
mean_A1B0 = np.array([[1.80,2.80,4.30]]).T
mean_A1B1 = np.array([[1.00,2.00,4.00]]).T

Sigma = np.array([[2.65,1.08,2.51],
                  [1.08,4.74,2.87],
                  [2.51,2.87,6.67]])
```

O nosso modelo é da forma

$y_{jik} = \mu_{jk} + e_{ijk} = \mu + \tau_j + \beta_l + \gamma_{jk} + e_{ijk} $

A partir dos vetores fornecidos no enunciado vamo calcular os vetores da média do tratamento A, B, intereção e a média populacional

```python
mean_A = np.array(mean_A1B0+mean_A1B1)/2
mean_B = np.array(mean_A0B1+mean_A1B1)/2
mean_AxB = np.array(mean_A0B1+mean_A1B0)/2
mean =(mean_A0B0 + mean_A0B1 + mean_A1B0 + mean_A1B1)/4
```

Abaixo calculamos as matrizes de covariância intra grupos e a matriz de Residuos

```python
HF1 = 2* 20 * (mean_A-mean).dot((mean_A-mean).T)
print(HF1)
```

    [[ 0.121    0.0715  -0.44   ]
     [ 0.0715   0.04225 -0.26   ]
     [-0.44    -0.26     1.6    ]]

```python
HF2 = 2* 20 * (mean_B-mean).dot((mean_B-mean).T)
print(HF2)
```

    [[ 2.601    2.2185  -0.     ]
     [ 2.2185   1.89225 -0.     ]
     [-0.      -0.       0.     ]]

```python
HInt = 20 * (mean_AxB-mean).dot((mean_AxB-mean).T)
print(HInt)
```

    [[0.4205   0.52925  0.435   ]
     [0.52925  0.666125 0.5475  ]
     [0.435    0.5475   0.45    ]]

```python
Hres = Sigma - HF1 - HF2 - HInt
print (Hres)
```

    [[-0.4925   -1.73925   2.515   ]
     [-1.73925   2.139375  2.5825  ]
     [ 2.515     2.5825    4.62    ]]

|Tratamento   |  Graus de liberdade    |Matriz de SSCP  |
|-------------|------------------------|----------------|
|Fator A      |  1  |\begin{bmatrix}   2.65 & 1.08 & 2.51\\1.08 & 4.74 & 2.87\\ 2.51 & 2.87 & 6.67 \end{bmatrix} |
|Fator B      |  1  |\begin{bmatrix}   2.65 & 1.08 & 2.51\\1.08 & 4.74 & 2.87\\ 2.51 & 2.87 & 6.67 \end{bmatrix}| |Interação    |  1  |\begin{bmatrix}   2.65 & 1.08 & 2.51\\1.08 & 4.74 & 2.87\\ 2.51 & 2.87 & 6.67 \end{bmatrix}|
||||
|Residuos     | 76  |\begin{bmatrix}   2.65 & 1.08 & 2.51\\1.08 & 4.74 & 2.87\\ 2.51 & 2.87 & 6.67 \end{bmatrix}|
||||
|Total        | 79  |\begin{bmatrix}   2.65 & 1.08 & 2.51\\1.08 & 4.74 & 2.87\\ 2.51 & 2.87 & 6.67 \end{bmatrix}|

## Item b
Há indicação de diferenças significantes no padrão de expressão gênica dos diferentes
tratamentos?
Faça suposições necessárias (independência, homocedasticidade e Normalidade)e teste o efeito de interação entre os medicamentos A e B. No caso da não existência de interação teste os efeitos principais dos medicamentos. Interprete os resultados em termos das possíveis diferenças entre os centroides populacionais.

#### Resolução
Para medir a significancia das diferença entre as médias vamos proceder com o teste de wilk inicialmente para a testar a hipótese:
$H_0: \gamma_{jk} = 0$

```python
 Lambda = np.linalg.det(Hres)/np.linalg.det(Hres + HInt)
```

```python
g =2
b =2
p =3
n =20
```

```python
mi1= (np.abs((g-1)*(b -1)-p) +1)
mi2= (g*b*(n-1)-p+1)

((1 - np.log(Lambda))/np.log(Lambda)) * (mi2/mi1) > chi2.ppf(q= 0.95, df= 3)
```

    False

Não rejeitamos a hipótese de hipótese nula, logo o modelo não tem interação. Agora vamos testar o efeito dos fatores

```python
Lambda_f1 = np.linalg.det(Hres)/np.linalg.det(Hres + HF1)
```

```python
mi1= np.abs(b -1 -p) +1
mi2= g*b*(n-1)-p+1

((1 - Lambda_f1)/Lambda_f1) * (mi2/mi1)  > f.ppf(q= 0.95, dfn=mi1,dfd=mi2)
```

    False

```python
Lambda_f2 = np.linalg.det(Hres)/np.linalg.det(Hres + HF2)

Lambda_f2
```

    -10.603261220290216

```python
mi1= np.abs(g -1 -p) +1
mi2= g*b*(n-1)-p+1

((1 - Lambda_f2)/Lambda_f2) * (mi2/mi1)  > f.ppf(q= 0.95, dfn=mi1,dfd=mi2)
```

    False

## Item C

Obtenha também a tabela MANOVA para o fator Tratamento (em quatro níveis).
Compare as duas abordagens, isto é, aquela considerando um único fator em quatro níveis e aquela sob a estrutura fatorial cruzada dos medicamentos (fatorial 2x2).

Sabemos que as matrizes de soma de quadrados para o problema em questao sao dadas por

$$
SSP_{A} = \sum_{l=1}^{g} bn(\boldsymbol{\bar{x}}_l - \boldsymbol{\bar{x}})(\boldsymbol{\bar{x}}_l - \boldsymbol{\bar{x}})^{'} = 20\sum_{l=1}^{2} (\boldsymbol{\bar{x}}_l - \boldsymbol{\bar{x}})(\boldsymbol{\bar{x}}_l - \boldsymbol{\bar{x}})^{'}
$$

$$
SSP_{B} = \sum_{k=1}^{b} gn(\boldsymbol{\bar{x}}_k - \boldsymbol{\bar{x}})(\boldsymbol{\bar{x}}_k - \boldsymbol{\bar{x}})^{'} = 20\sum_{k=1}^{2} (\boldsymbol{\bar{x}}_k - \boldsymbol{\bar{x}})(\boldsymbol{\bar{x}}_k - \boldsymbol{\bar{x}})^{'}
$$

$$
SSP_{AxB} = \sum_{l=1}^{g}\sum_{k=1}^{b} n(\boldsymbol{\bar{x}}_{lk} - \boldsymbol{\bar{x}}_{l} - \boldsymbol{\bar{x}}_{k} - \boldsymbol{\bar{x}})(\boldsymbol{\bar{x}}_{lk} - \boldsymbol{\bar{x}}_{l} - \boldsymbol{\bar{x}}_{k} - \boldsymbol{\bar{x}})^{'} = 10\sum_{l=1}^{2}\sum_{k=1}^{2} (\boldsymbol{\bar{x}}_{lk} - \boldsymbol{\bar{x}}_{l} - \boldsymbol{\bar{x}}_{k} - \boldsymbol{\bar{x}})(\boldsymbol{\bar{x}}_{lk} - \boldsymbol{\bar{x}}_{l} - \boldsymbol{\bar{x}}_{k} - \boldsymbol{\bar{x}})^{'}
$$

$$
SSP_{res} = \sum_{l=1}^{g}\sum_{k=1}^{b}\sum_{i=1}^{n} (\boldsymbol{{x}}_{lki} - \boldsymbol{\bar{x}}_{lk})(\boldsymbol{{x}}_{lki} - \boldsymbol{\bar{x}}_{lk})^{'} =  10\sum_{l=1}^{2}\sum_{k=1}^{2} (\boldsymbol{{x}}_{lk} - \boldsymbol{\bar{x}}_{lk})(\boldsymbol{{x}}_{lk} - \boldsymbol{\bar{x}}_{lk})^{'}
$$

$$
SSP_{tot} = \sum_{l=1}^{g}\sum_{k=1}^{b}\sum_{i=1}^{n} (\boldsymbol{{x}}_{lki} - \boldsymbol{\bar{x}})(\boldsymbol{{x}}_{lki} - \boldsymbol{\bar{x}})^{'} = 10\sum_{l=1}^{2}\sum_{k=1}^{2}(\boldsymbol{{x}}_{lk} - \boldsymbol{\bar{x}})(\boldsymbol{{x}}_{lk} - \boldsymbol{\bar{x}})^{'}
$$

```python
mean_A1 = (mean_A1B0+mean_A1B1)/2
mean_A0 = (mean_A0B0+mean_A0B1)/2

mean_B1 = (mean_A0B1+mean_A1B1)/2
mean_B0 = (mean_A0B0+mean_A1B0)/2
```

```python
## soma dos quadrados e produtos cruzados
SSPf1 = 2 * 20 * ((mean_A1 - mean).dot((mean_A1 - mean).T) + (mean_A0 - mean).dot((mean_A0 - mean).T))
SSPf2 = 2 * 20 * ((mean_B1 - mean).dot((mean_B1 - mean).T) + (mean_B0 - mean).dot((mean_B0 - mean).T))
SSPint = 10 * ((mean_A0B1- mean_A0 - mean_B1 + mean).dot((mean_A0B1- mean_A0 - mean_B1 + mean).T) + (mean_A1B0 - mean_A1 - mean_B0 + mean).dot((mean_A1B0 - mean_A1 - mean_B0 + mean).T))
SSPres = Sigma - SSPf1 - SSPf2 - SSPint
SSPTotal = Sigma
```

```python
SSPtrat = 10 * ((mean_A0B0 - mean).dot((mean_A0B0 - mean).T) +
                (mean_A0B1 - mean).dot((mean_A0B1 - mean).T) +
                (mean_A1B0 - mean).dot((mean_A1B0 - mean).T) +
                (mean_A1B1 - mean).dot((mean_A1B1 - mean).T))
SSPRes = SSPTotal- SSPtrat
```

```python
SSPtrat
```

    array([[3.563  , 3.3485 , 0.43   ],
           [3.3485 , 3.26675, 0.835  ],
           [0.43   , 0.835  , 2.5    ]])

```python
SSPRes
```

    array([[-0.913  , -2.2685 ,  2.08   ],
           [-2.2685 ,  1.47325,  2.035  ],
           [ 2.08   ,  2.035  ,  4.17   ]])

# Questão 2
Considere o modelo de MANOVA de um delineamento experimental completamente aleatorizado com um fator em G níveis.

## Item A
Como podem ser obtidas (apresente um estimador) as matrizes de Soma de Quadrados e Produtos Cruzados Entre e Dentro de grupos, SSb e SSw, respectivamente? Essas matrizes aleatórias seguem qual distribuição?

#### Resolução
O estimador para SSb é dado por
\begin{equation}
\text{SSb} = \sum_{g = 1} ^G n_g (\bar x_g - \bar x )(\bar x_g - \bar x)^T \sim \chi^2(G-1)
\end{equation}
e para SSw temos:
\begin{equation}
\text{SSw} = \sum_{g = 1} ^G \sum_{j=1}^{n_g}(\bar x_{jg} - \bar x_g ) \left(\bar x_{jg} - \bar x_g\right)^T \sim \chi^2(\sum_{g = 1}^G n_g - G)
\end{equation}

## Item B
 Qual é o papel da decomposição espectral de $SSw^{-1}SSb$ na discriminação entre os grupos em um modelo MANOVA?
#### Resolução
 A decomposição espectral de $SSw^{-1}SSb$ tem dois papéis, seus autovalores são utilizados no cálculo das estatísticas Pilai, Wilks, Roy e Hotelling-Lawley. E os autovetores correspondentes a cada autovetor nos dão a contribuição das variáveis para a tabela de ANOVA.

 ## Item C

```python

```

```python

```

```python

```


```python
from matplotlib import pyplot as plt
import numpy as np
from scipy.integrate import quad
from qiskit.quantum_info.states import Statevector, DensityMatrix
from qiskit.quantum_info.operators import Operator
from qiskit.quantum_info.operators import SuperOp, Choi, Kraus, PTM

from qiskit_dynamics import solve_ode, solve_lmde
from qiskit_dynamics.signals import Signal
from qiskit_dynamics.models import HamiltonianModel, LindbladModel

def gaussian(amp, sig, t0, t):
    return amp * np.exp( -(t - t0)**2 / (2 * sig**2) )
```

# In this demo

This notebook demonstrates some of the core model building and differential equation solving elements:
- Hamiltonian and signal construction
- Model transformations: entering a frame, making a rotating wave approximation
- Defining and solving differential equations

Sections
1. `Signal`s
2. Constructing a `HamiltonianModel`
3. Setting `frame` and `cutoff_freq` in `HamiltonianModel`
4. Integrating the Schrodinger equation
5. Adding dissipative dynamics with a `LindbladModel` and simulating density matrix evolution
6. Simulate the Lindbladian to get a `SuperOp` representation of the quantum channel

# 1. `Signal`

A `Signal` object represents a complex mixed signal, i.e. a function of the form:
\begin{equation}
    s(t) = f(t)e^{i2 \pi \nu t},
\end{equation}
where $f(t)$ is the *envelope* and $\nu$ is the *carrier frequency*.

Here we define a signal with a Gaussian envelope:

```python
amp = 1. # amplitude
sig = 2. # sigma
t0 = 3.5*sig # center of Gaussian
T = 7*sig # end of signal

gaussian_envelope = lambda t: gaussian(amp, sig, t0, t)

gauss_signal = Signal(envelope=gaussian_envelope, carrier_freq=0.5)
```

```python
print(gauss_signal.envelope(0.25))
print(gauss_signal(0.25))
```

    0.0033616864879322562
    0.0023770713118400873

```python
gauss_signal.draw(0, T, 100, function='envelope')
```

```python
gauss_signal.draw(0, T, 200)
```

# 2. The `HamiltonianModel` class

A `HamiltonianModel` is specified as a list of Hermitian operators with `Signal` coefficients. Here, we use a classic qubit model:
\begin{equation}
    H(t) = 2 \pi \nu \frac{Z}{2} + 2 \pi r s(t) \frac{X}{2}.
\end{equation}

Generally, a `HamiltonianModel` represents a linear combination:
\begin{equation}
    H(t) = \sum_j s_j(t) H_j,
\end{equation}
where:
 - $H_j$ are Hermitian operators given as `terra.quantum_info.Operator` objects, and
 - $s_j(t) = Re[f_j(t)e^{i2 \pi \nu_j t}]$, where the complex functions $f_j(t)e^{i2 \pi \nu_j t}$ are specified as `Signal` objects.

Constructing a `HamiltonianModel` requires specifying lists of the operators and the signals.

```python
#####################
# construct operators
#####################
r = 0.5
w = 1.
X = Operator.from_label('X')
Y = Operator.from_label('Y')
Z = Operator.from_label('Z')

operators = [2 * np.pi * w * Z/2,
             2 * np.pi * r * X/2]

###################
# construct signals
###################

# Define gaussian envelope function to have max amp and area approx 2
amp = 1.
sig = 0.399128/r
t0 = 3.5*sig
T = 7*sig
gaussian_envelope = lambda t: gaussian(amp, sig, t0, t)

signals = [1.,
           Signal(envelope=gaussian_envelope, carrier_freq=w)]

#################
# construct model
#################

hamiltonian = HamiltonianModel(operators=operators, signals=signals)
```

## 2.1 Evaluation and drift

Evaluate at a given time.

```python
print(hamiltonian.evaluate(0.12))
```

    [[ 3.14159265+0.j  0.00419151+0.j]
     [ 0.00419151+0.j -3.14159265+0.j]]

Get the drift (terms corresponding to constant coefficients).

```python
hamiltonian.drift
```

    Array([[ 3.14159265+0.j,  0.        +0.j],
           [ 0.        +0.j, -3.14159265+0.j]], backend='numpy')

```python
def plot_qubit_hamiltonian_components(hamiltonian, t0, tf, N=200):
    t_vals = np.linspace(t0, tf, N)
    model_vals = np.array([hamiltonian.evaluate(t) for t in t_vals])
    x_coeff = model_vals[:, 0, 1].real
    y_coeff = -model_vals[:, 0, 1].imag
    z_coeff = model_vals[:, 0, 0].real
    plt.plot(t_vals, x_coeff, label='X component')
    plt.plot(t_vals, y_coeff, label='Y component')
    plt.plot(t_vals, z_coeff, label='Z component')
    plt.legend()

plot_qubit_hamiltonian_components(hamiltonian, 0., T)
```

## 2.2 Enter a rotating frame

We can specify a frame to enter the Hamiltonian in. Given a Hermitian operator $H_0$, *entering the frame* of $H_0$ means transforming a Hamiltonian $H(t)$:
\begin{equation}
    H(t) \mapsto \tilde{H}(t) = e^{i H_0 t}H(t)e^{-iH_0 t} - H_0
\end{equation}

Here, we will enter the frame of the drift Hamiltonian, resulting in:
\begin{equation}
\begin{aligned}
    \tilde{H}(t) &= e^{i2 \pi \nu \frac{Z}{2} t}\left(2 \pi \nu \frac{Z}{2} + 2 \pi r s(t) \frac{X}{2}\right)e^{-i2 \pi \nu \frac{Z}{2} t} - 2 \pi \nu \frac{Z}{2} \\
                &= 2 \pi r s(t) e^{i2 \pi \nu \frac{Z}{2} t}\frac{X}{2}e^{-i2 \pi \nu \frac{Z}{2} t}\\
                &= 2 \pi r s(t) \left[\cos(2 \pi \nu t) \frac{X}{2} - \sin(2 \pi \nu t) \frac{Y}{2} \right]
\end{aligned}
\end{equation}

```python
hamiltonian.frame = hamiltonian.drift
```

Evaluate again.

```python
print(hamiltonian.evaluate(0.12))
```

    [[0.        +0.j         0.00305548+0.00286929j]
     [0.00305548-0.00286929j 0.        +0.j        ]]

```python
# validate with independent computation
t = 0.12
2 * np.pi * r * np.real(signals[1](t)) * (np.cos(2*np.pi * w * t) * X / 2
                                                - np.sin(2*np.pi * w * t) * Y / 2 )
```

    Operator([[0.        +0.j        , 0.00305548+0.00286929j],
              [0.00305548-0.00286929j, 0.        +0.j        ]],
             input_dims=(2,), output_dims=(2,))

Replot the coefficients of the model in the Pauli basis over time.

```python
plot_qubit_hamiltonian_components(hamiltonian, 0., T)
```

# 3. Set cutoff frequency (rotating wave approximation)

A common technique to simplify the dynamics of a quantum system is to perform the *rotating wave approximation* (RWA), in which terms with high frequency are averaged to $0$.

The RWA can be applied on any `HamiltonianModel` (in the given `frame`) by setting the `cutoff_freq` attribute, which sets any fast oscillating terms to $0$, effectively performing a moving average on terms with carrier frequencies above `cutoff_freq`.

For our model, the classic `cutoff_freq` is $2 \nu$ (twice the qubit frequency). This approximates the Hamiltonian $\tilde{H}(t)$ as:
\begin{equation}
    \tilde{H}(t) \approx 2 \pi \frac{r}{2} \left[Re[f(t)] \frac{X}{2} + Im[f(t)] \frac{Y}{2} \right],
\end{equation}
where $f(t)$ is the envelope of the on-resonance drive. On our case $f(t) = Re[f(t)]$, and so we simply have
\begin{equation}
    \tilde{H}(t) \approx 2 \pi \frac{r}{2} Re[f(t)] \frac{X}{2},
\end{equation}

```python
# set the cutoff frequency
hamiltonian.cutoff_freq = 2*w
```

```python
# evaluate again
print(hamiltonian.evaluate(0.12))
```

    [[0.        +0.00000000e+00j 0.00287496-2.16840434e-19j]
     [0.00287496+2.16840434e-19j 0.        +0.00000000e+00j]]

We also plot the coefficients of the model in the frame of the drift with the RWA applied. We now expect to see simply a plot of $\pi \frac{r}{2} f(t)$ for the $X$ coefficient.

```python
plot_qubit_hamiltonian_components(hamiltonian, 0., T)
```

# 4. Solve Shrodinger equation with a `HamiltonianModel`

To solve the Schrodinger Equation for the given Hamiltonian, we construct a `SchrodingerProblem` object, which specifies the desired simulation.

```python
# reset the frame and cutoff_freq properties
hamiltonian.frame = None
hamiltonian.cutoff_freq = None
```

```python
# solve the problem, with some options specified
y0 = Statevector([0., 1.])
%time sol = solve_lmde(hamiltonian, t_span=[0., T], y0=y0, atol=1e-10, rtol=1e-10)
y = sol.y[-1]
print("\nFinal state:")
print("----------------------------")
print(y)
print("\nPopulation in excited state:")
print("----------------------------")
print(np.abs(y.data[0])**2)
```

    CPU times: user 453 ms, sys: 6.67 ms, total: 459 ms
    Wall time: 454 ms

    Final state:
    ----------------------------
    Statevector([0.96087419-0.27193942j, 0.0511707 +0.01230027j],
                dims=(2,))

    Population in excited state:
    ----------------------------
    0.9972302627366899

When specifying a problem, we can specify which frame to solve in, and a cutoff frequency to solve with.

```python
%time sol = solve_lmde(hamiltonian, t_span=[0., T], y0=y0, solver_cutoff_freq=2*w, atol=1e-10, rtol=1e-10)
y = sol.y[-1]
print("\nFinal state:")
print("----------------------------")
print(y)
print("\nPopulation in excited state:")
print("----------------------------")
print(np.abs(y.data[0])**2)
```

    CPU times: user 150 ms, sys: 14.7 ms, total: 165 ms
    Wall time: 153 ms

    Final state:
    ----------------------------
    Statevector([ 9.62205827e-01-2.72323239e-01j,
                 -2.36278329e-08+8.34847474e-08j],
                dims=(2,))

    Population in excited state:
    ----------------------------
    0.9999999999756191

# 4.1 Technical solver notes:
- Behind the scenes, the `SchrodingerProblem` constructs an `OperatorModel` from the `HamiltonianModel`, representing the generator in the Schrodinger equation:
\begin{equation}
    G(t) = \sum_j s_j(t)\left[-iH_j\right]
\end{equation}
which is then pass used in a generalized routine for solving DEs of the form $y'(t) = G(t)y(t)$
- The generalized solver routine will automatically solve the DE in the drift frame, as well as in the basis in which the drift is diagonal (relevent for non-diagonal drift operators, to save on exponentiations for the frame operator).

# 5. Solving with dissipative dynamics

To simulate with noise operators, we define a `LindbladModel`, containing:
- a model of a Hamiltonian (specified with either a `HamiltonianModel` object, or in the standard decomposition of operators and signals)
- an optional list of noise operators
- an optional list of time-dependent coefficients for the noise operators

Such a system is simulated in terms of the Lindblad master equation:
\begin{equation}
    \dot{\rho}(t) = -i[H(t), \rho(t)] + \sum_j g_j(t) \left(L_j \rho L_j^\dagger - \frac{1}{2}\{L_j^\dagger L_j, \rho(t)\}\right),
\end{equation}
where
- $H(t)$ is the Hamiltonian,
- $L_j$ are the noise operators, and
- $g_j(t)$ are the noise coefficients

Here we will construct such a model using the above `Hamitonian`, along with a noise operator that drives the state to the ground state.

```python
# construct quantum model with noise operators
noise_ops = [np.array([[0., 0.],
                       [1., 0.]])]
noise_signals = [0.001]
lindblad_model = LindbladModel.from_hamiltonian(hamiltonian=hamiltonian,
                                                noise_operators=noise_ops,
                                                noise_signals=noise_signals)

# density matrix
y0 = DensityMatrix([[0., 0.], [0., 1.]])

%time sol = solve_lmde(lindblad_model, t_span=[0., T], y0=y0, atol=1e-10, rtol=1e-10)
sol.y[-1]
```

    CPU times: user 1.04 s, sys: 316 ms, total: 1.35 s
    Wall time: 567 ms
    DensityMatrix([[0.99473642-9.02056208e-17j, 0.04620048-2.49934328e-02j],
                   [0.04620048+2.49934328e-02j, 0.00526358+0.00000000e+00j]],
                  dims=(2,))

We may also simulate the Lindblad equation with a cutoff frequency.

```python
%time sol = solve_lmde(lindblad_model, t_span=[0., T], y0=y0, solver_cutoff_freq=2*w, atol=1e-10, rtol=1e-10)
sol.y[-1]
```

    CPU times: user 475 ms, sys: 21 ms, total: 496 ms
    Wall time: 481 ms
    DensityMatrix([[0.99614333-7.63278329e-17j, 0.01049406-3.50564736e-02j],
                   [0.01049406+3.50564736e-02j, 0.00252446+0.00000000e+00j]],
                  dims=(2,))

## 5.1 Technical notes

- Similarly to the flow of `SchrodingerProblem`, `LindbladProblem` constructs an `OperatorModel` representing the *vectorized* Lindblad equation, which is then used to simulate the Lindblad equation on the vectorized density matrix.
- Frame handling and cutoff frequency handling are handled at the `OperatorModel` level, and hence can be used here as well.

## 5.2 Simulate the Lindbladian/SuperOp

```python
# identity quantum channel in superop representation
y0 = SuperOp(np.eye(4))

%time sol = solve_lmde(lindblad_model, t_span=[0., T], y0=y0, atol=1e-10, rtol=1e-10)
print(sol.y[-1])
```

    CPU times: user 571 ms, sys: 19.4 ms, total: 590 ms
    Wall time: 575 ms
    SuperOp([[ 0.00523988-1.46642940e-16j,  0.05230901-2.08651875e-03j,
               0.05230901+2.08651875e-03j,  0.99473642+1.76741057e-16j],
             [-0.04508833-2.62771380e-02j,  0.00223689+1.07542007e-03j,
              -0.84671078-5.20988650e-01j,  0.04620048+2.49934329e-02j],
             [-0.04508833+2.62771380e-02j, -0.84671078+5.20988650e-01j,
               0.00223689-1.07542007e-03j,  0.04620048-2.49934329e-02j],
             [ 0.99476012+5.25552053e-18j, -0.05230901+2.08651875e-03j,
              -0.05230901-2.08651875e-03j,  0.00526358-5.58790089e-17j]],
            input_dims=(2,), output_dims=(2,))

```python
print(PTM(y))
```

    PTM([[ 5.00000000e-01+0.j, -4.54696754e-08+0.j,  7.38951024e-08+0.j,
           5.00000000e-01+0.j],
         [-4.54696754e-08+0.j,  4.13498276e-15+0.j, -6.71997263e-15+0.j,
          -4.54696754e-08+0.j],
         [ 7.38951024e-08+0.j, -6.71997263e-15+0.j,  1.09209723e-14+0.j,
           7.38951024e-08+0.j],
         [ 5.00000000e-01+0.j, -4.54696754e-08+0.j,  7.38951024e-08+0.j,
           5.00000000e-01+0.j]],
        input_dims=(2,), output_dims=(2,))

```python

```


```python
# Header starts here.
from sympy.physics.units import *
from sympy import *

# Rounding:
import decimal
from decimal import Decimal as DX
from copy import deepcopy
def iso_round(obj, pv, rounding=decimal.ROUND_HALF_EVEN):
    import sympy
    """
    Rounding acc. to DIN EN ISO 80000-1:2013-08
    place value = Rundestellenwert
    """
    assert pv in set([
        # place value   #  round to:
        1,              #  1
        0.1,            #  1st digit after decimal
        0.01,           #  2nd
        0.001,          #  3rd
        0.0001,         #  4th
        0.00001,        #  5th
        0.000001,       #  6th
        0.0000001,      #  7th
        0.00000001,     #  8th
        0.000000001,    #  9th
        0.0000000001,   # 10th
        ])
    objc = deepcopy(obj)
    try:
        tmp = DX(str(float(objc)))
        objc = tmp.quantize(DX(str(pv)), rounding=rounding)
    except:
        for i in range(len(objc)):
            tmp = DX(str(float(objc[i])))
            objc[i] = tmp.quantize(DX(str(pv)), rounding=rounding)
    return objc

# LateX:
kwargs = {}
kwargs["mat_str"] = "bmatrix"
kwargs["mat_delim"] = ""

# kwargs["symbol_names"] = {FB: "F^{\mathsf B}", }

# Units:
(k, M, G ) = ( 10**3, 10**6, 10**9 )
(mm, cm) = ( m/1000, m/100 )
Newton = kg*m/s**2
Pa     = Newton/m**2
MPa    = M*Pa
GPa    = G*Pa
kN     = k*Newton
deg    = pi/180

half = S(1)/2

# Header ends here.
#
# https://colab.research.google.com/github/kassbohm/wb-snippets/blob/master/ipynb/TEM_10/ESA/a1_cc.ipynb

F,l = var("F,l")
R = 3*F/2
lu = l/sqrt(3)

Ah,Av,Bh,Bv,Ch,Cv = var("Ah,Av,Bh,Bv,Ch,Cv")

e1 = Eq(Ah + Bh + F)
e2 = Eq(Av + Bv - R)
e3 = Eq(Bv*l - Bh*l - F*l/2 - R*7/18*l)

e4 = Eq(Ch - Bh)
e5 = Eq(Cv - F - Bv)
e6 = Eq(F*lu/2 + Bv*lu + Bh*l)

eqs = [e1,e2,e3,e4,e5,e6]
unknowns = [Ah,Av,Bh,Bv,Ch,Cv]

pprint("\nEquations:")
for e in eqs:
    pprint(e)
    pprint("\n")

# Alternative Solution (also correct):
# Ah,Av,Bh,Bv,Gh,Gv = var("Ah,Av,Bh,Bv,Gh,Gv")
#
# e1 = Eq(Av + Gv - R)
# e2 = Eq(Ah + F - Gh)
# e3 = Eq(F/2 + 7*R/18 - Gv - Gh)
# e4 = Eq(-Gv -F + Bv)
# e5 = Eq(Gh - Bh)
# e6 = Eq(Gh - sqrt(3)*F/6 - Gv/sqrt(3))
#
# eqs = [e1,e2,e3,e4,e5,e6]
# unknowns = [Ah,Av,Bh,Bv,Gh,Gv]

sol = solve(eqs,unknowns)

pprint("\nReactions:")
pprint(sol)

pprint("\nReactions / F (rounded to 0.01):")
for v in  sorted(sol,key=default_sort_key):
    pprint("\n\n")
    s = sol[v]
    tmp = (s/F)
    tmp = tmp.simplify()
    # pprint(tmp)
    pprint([v, tmp, iso_round(tmp,0.01)])

# Reactions / F:
#
# ⎡      43   19⋅√3       ⎤
# ⎢Ah, - ── + ─────, -0.42⎥
# ⎣      24     24        ⎦
#
#
# ⎡      3   19⋅√3     ⎤
# ⎢Av, - ─ + ─────, 1.0⎥
# ⎣      8     24      ⎦
#
#
# ⎡      19⋅√3   19       ⎤
# ⎢Bh, - ───── + ──, -0.58⎥
# ⎣        24    24       ⎦
#
#
# ⎡      19⋅√3   15     ⎤
# ⎢Bv, - ───── + ──, 0.5⎥
# ⎣        24    8      ⎦
#
#
# ⎡      19⋅√3   19       ⎤
# ⎢Ch, - ───── + ──, -0.58⎥
# ⎣        24    24       ⎦
#
#
# ⎡      19⋅√3   23     ⎤
# ⎢Cv, - ───── + ──, 1.5⎥
# ⎣        24    8      ⎦

```


# Taming math and physics using `SymPy`

Tutorial based on the [No bullshit guide](http://minireference.com/) series of textbooks by [Ivan Savov](mailto:ivan.savov+SYMPYTUT@gmail.com)

## Abstract

Most people consider math and physics to be scary
beasts from which it is best to keep one's distance. Computers,
however, can help us tame the complexity and tedious arithmetic
manipulations associated with these subjects. Indeed, math and
physics are much more approachable once you have the power of
computers on your side.

This tutorial serves a dual purpose. On one hand, it serves
as a review of the fundamental concepts of mathematics for
computer-literate people. On the other hand, this tutorial serves
to demonstrate to students how a computer algebra system can
help them with their classwork. A word of warning is in order.
Please don't use `SymPy` to avoid the suffering associated with your
homework! Teachers assign homework problems to you because
they want you to learn. Do your homework by hand, but if you
want, you can check your answers using `SymPy`. Better yet, use
`SymPy` to invent extra practice problems for yourself.

## Contents

* [Fundamentals of mathematics](Fundamentals-of-mathematics.ipynb)
* [Complex numbers](Complex-numbers.ipynb)
* [Calculus](Calculus.ipynb)
* [Vectors](Vectors.ipynb)
* [Mechanics](Mechanics.ipynb)
* [Linear algebra](Linear-algebra.ipynb)

## Introduction

You can use a computer algebra system (CAS) to compute complicated
math expressions, solve equations, perform calculus procedures,
and simulate physics systems.

All computer algebra systems offer essentially the same functionality,
so it doesn't matter which system you use: there are free
systems like `SymPy`, `Magma`, or `Octave`, and commercial systems like
`Maple`, `MATLAB`, and `Mathematica`. This tutorial is an introduction to
`SymPy`, which is a *symbolic* computer algebra system written in the
programming language `Python`. In a symbolic CAS, numbers and
operations are represented symbolically, so the answers obtained are
exact. For example, the number √2 is represented in `SymPy` as the
object `Pow(2,1/2)`, whereas in numerical computer algebra systems
like `Octave`, the number √2 is represented as the approximation
1.41421356237310 (a `float`). For most purposes the approximation
is okay, but sometimes approximations can lead to problems:
`float(sqrt(2))*float(sqrt(2))` = 2.00000000000000044 ≠ 2. Because
`SymPy` uses exact representations, you'll never run into such
problems: `Pow(2,1/2)*Pow(2,1/2)` = 2.

This tutorial is organized as follows. We'll begin by introducing the
`SymPy` basics and the bread-and-butter functions used for manipulating
expressions and solving equations. Afterward, we'll discuss the
`SymPy` functions that implement calculus operations like differentiation
and integration. We'll also introduce the functions used to deal with
vectors and complex numbers. Later we'll see how to use vectors and
integrals to understand Newtonian mechanics. In the last section,
we'll introduce the linear algebra functions available in `SymPy`.

This tutorial presents many explanations as blocks of code. Be sure
to try the code examples on your own by typing the commands into
`SymPy`. It's always important to verify for yourself!

## Using SymPy

The easiest way to use `SymPy`, provided you're connected to the
Internet, is to visit http://live.sympy.org. You'll be presented with
an interactive prompt into which you can enter your commands—right
in your browser.

If you want to use `SymPy` on your own computer, you must install
`Python` and the python package `sympy`. You can then open a command
prompt and start a `SymPy` session using:

```
you@host$ python
Python X.Y.Z
[GCC a.b.c (Build Info)] on platform
Type "help", "copyright", or "license" for more information.
>>> from sympy import *
>>>
```

The `>>>` prompt indicates you're in the Python shell which accepts
Python commands. The command `from sympy import *` imports all
the `SymPy` functions into the current namespace. All `SymPy` functions
are now available to you. To exit the python shell press `CTRL+D`.

I highly recommend you also install `ipython`, which is an improved
interactive python shell. If you have `ipython` and `SymPy` installed,
you can start an `ipython` shell with `SymPy` pre-imported using the
command `isympy`. For an even better experience, you can try `ipython notebook`,
which is a web frontend for the `ipython` shell.

You can start your session the same way as `isympy` do, by running following commands, which will be detaily described latter.

```python
from sympy import init_session
init_session()
```

    IPython console for SymPy 0.7.6 (Python 3.4.2-64-bit) (ground types: gmpy)

    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()

    Documentation can be found at http://www.sympy.org

## Conclusion

I would like to conclude with some words of caution about the overuse of computers.
Computer technology is very powerful and is everywhere around us,
but let's not forget that computers are actually very dumb:
computers are mere calculators and they depend on your knowledge to direct them.
It's important that you learn how to do complicated math by hand in order to be
able to instruct computers to do math for you and to check the results of your computer calculations.
I don't want you to use the tricks you learned in this tutorial to avoid math problems from now on
and simply rely blindly on `SymPy` for all your math needs.
I want both you and the computer to become math powerhouses!
The computer will help you with tedious calculations (they're good at that)
and you'll help the computer by guiding it when it gets  stuck (humans are good at that).

## Links

* [Installation instructions for `ipython notebook`](http://ipython.org/install.html)
* [The official `SymPy` tutorial](http://docs.sympy.org/latest/tutorial/intro.html)
* [A list of `SymPy` gotchas](http://docs.sympy.org/dev/gotchas.html)
* [`SymPy` video tutorials by Matthew Rocklin](http://pyvideo.org/speaker/583/matthew-rocklin)

## Book plug

The examples and math explanations in this tutorial are sourced from the
*No bullshit guide* series of books published by Minireference Co.
We publish textbooks that make math and physics accessible and affordable for everyone.
If you're interested in learning more about the math, physics, and calculus topics discussed in this tutorial,
check out the **No bullshit guide to math and physics**.
The book contains the distilled information that normally comes in two first-year university books:
the introductory physics book (1000+ pages) and the first-year calculus book (1000+ pages).
Would you believe me if I told you that you can learn the
same material from a single book that is 1/7th the size and 1/10th of the
price of mainstream textbooks?

This book contains short lessons on math and physics, calculus.
Often calculus and mechanics are taught as separate subjects.
It shouldn't be like that.
If you learn calculus without mechanics, it will be boring.
If you learn mechanics without calculus, you won't truly understand what is going on.
This textbook covers both subjects in an integrated manner.

Contents:

* High school math
* Vectors
* Mechanics
* Differential calculus
* Integral calculus
* 250+ practice problems

For more information, see the book's website at [minireference.com](http://minireference.com/)

The presented linear algebra examples are
sourced from the [**No bullshit guide to linear algebra**](https://gum.co/noBSLA).
Check out the book if you're taking a linear algebra course of if you're missing the prerequisites
for learning machine learning, computer graphics, or quantum mechanics.

I'll close on a note for potential readers who suffer from math-phobia.
Both books start with an introductory chapter that reviews all
high school math concepts needed to make math and physics
accessible to everyone.
Don't worry, we'll fix this math-phobia thing right up for you;
**when you've got `SymPy` skills, math fears *you*!**

To stay informed about upcoming titles,
follow [@minireference](https://twitter.com/minireference) on twitter
and check out the facebook page at [fb.me/noBSguide](http://fb.me/noBSguide).


# How to use different estimators and how to interpret the results

```python
import pandas  as pd
import numpy as np
import biogeme.database as db
import biogeme.biogeme as bio
import biogeme.models as models
import biogeme.optimization as opt
from biogeme.expressions import Beta, DefineVariable
import seaborn as sns
import matplotlib.pyplot as plt
```

**Import Swissmetro data**

```python
pandas = pd.read_csv("../../Data/6-Discrete Choice Models/swissmetro.dat",sep='\t')
database = db.Database("data/swissmetro", pandas)
```

**Use collumn names as variables**

```python
globals().update(database.variables)
```

**Exclude some unwanted entries**

```python
exclude = (( PURPOSE != 1 ) * ( PURPOSE != 3 ) + ( CHOICE == 0 )) > 0

database.remove(exclude)
```

**Define some dummy variables**

```python
SM_COST = SM_CO * ( GA == 0 )
TRAIN_COST = TRAIN_CO * ( GA == 0 )

CAR_AV_SP = DefineVariable ('CAR_AV_SP', CAR_AV * ( SP !=0 ), database)
TRAIN_AV_SP = DefineVariable ('TRAIN_AV_SP', TRAIN_AV * ( SP != 0 ), database)
```

**Rescale some data**

```python
TRAIN_TT_SCALED   = DefineVariable('TRAIN_TT_SCALED',   TRAIN_TT / 100.0, database)
TRAIN_COST_SCALED = DefineVariable('TRAIN_COST_SCALED', TRAIN_COST / 100, database)
SM_TT_SCALED      = DefineVariable('SM_TT_SCALED',      SM_TT / 100.0   , database)
SM_COST_SCALED    = DefineVariable('SM_COST_SCALED',    SM_COST / 100   , database)
CAR_TT_SCALED     = DefineVariable('CAR_TT_SCALED',     CAR_TT / 100    , database)
CAR_CO_SCALED     = DefineVariable('CAR_CO_SCALED',     CAR_CO / 100    , database)
```

**Create parameters to be estimated**

```python
ASC_CAR = Beta('ASC_CAR',0,None ,None ,0)
ASC_TRAIN = Beta('ASC_TRAIN',0,None ,None ,0)
ASC_SM = Beta('ASC_SM',0,None ,None ,1)
B_TIME = Beta('B_TIME',0,None ,None ,0)
B_COST = Beta('B_COST',0,None ,None ,0)
```

**Define the utility functions**

```python
V1 = ASC_TRAIN + \
     B_TIME * TRAIN_TT_SCALED + \
     B_COST * TRAIN_COST_SCALED
V2 = ASC_SM + \
     B_TIME * SM_TT_SCALED + \
     B_COST * SM_COST_SCALED
V3 = ASC_CAR + \
     B_TIME * CAR_TT_SCALED + \
     B_COST * CAR_CO_SCALED
```

**Associate utility functions with alternatives and associate availability of alternatives**

```python
V = {1: V1,
     2: V2,
     3: V3}

av = {1: TRAIN_AV_SP,
      2: SM_AV,
      3: CAR_AV_SP}
```

**Define the model**

```python
logprob = models.loglogit(V, av, CHOICE)
```

**Define the Biogeme object**

```python
biogeme  = bio.BIOGEME(database, logprob)

biogeme.modelName = "swissmetro_logit_estimators"
```

**Define the algorithms to estimat the maximum likelihood**

```python
algos = {'CFSQP                ': None,
         'scipy                ': opt.scipy,
         'Trust region (dogleg)': opt.newtonTrustRegionForBiogeme,
         'Trust region (cg)    ': opt.newtonTrustRegionForBiogeme,
         'TR-BFGS              ': opt.bfgsTrustRegionForBiogeme,
         'Simple bounds Newton ': opt.simpleBoundsNewtonAlgorithmForBiogeme,
         'Simple bounds BFGS   ': opt.simpleBoundsNewtonAlgorithmForBiogeme,
         'Simple bounds hybrid ': opt.simpleBoundsNewtonAlgorithmForBiogeme}

algoParameters = {'Trust region (dogleg)': {'dogleg':True},
                  'Trust region (cg)': {'dogleg':False},
                  'Simple bounds Newton ': {'proportionAnalyticalHessian': 1.0},
                  'Simple bounds BFGS   ': {'proportionAnalyticalHessian': 0.0},
                  'Simple bounds hybrid ': {'proportionAnalyticalHessian': 0.5}}
```

**Estimate the model**

```python
biogeme.generateHtml = False
biogeme.generatePickle = False

results = {}
for name,algo in algos.items():
    p = algoParameters.get(name)
    results[name] = biogeme.estimate(algorithm=algo,algoParameters=p)
    g = results[name].data.g
    biogeme.createLogFile()
```

**Print results for the different algorithms**

```python
print("Algorithm\t\tloglike\t\tnormg\ttime")
print("+++++++++\t\t+++++++\t\t+++++\t++++")
for name,algo in algos.items():
    print(f'{name}\t{results[name].data.logLike:.2f}\t'
          f'{results[name].data.gradientNorm:.2g}\t'
          f'{results[name].data.optimizationMessages["Optimization time"]}')
```

    Algorithm		loglike		normg	time
    +++++++++		+++++++		+++++	++++
    CFSQP                	-5331.25	0.0004	0:00:00.279782
    scipy                	-5331.25	6.5e-05	0:00:00.085049
    Trust region (dogleg)	-5331.25	0.03	0:00:00.070050
    Trust region (cg)    	-5331.25	0.03	0:00:00.068124
    TR-BFGS              	-5331.25	0.013	0:00:00.251144
    Simple bounds Newton 	-5331.25	0.00063	0:00:00.069949
    Simple bounds BFGS   	-5331.25	0.015	0:00:00.269823
    Simple bounds hybrid 	-5331.25	0.00038	0:00:00.081564

**Print results**

```python
results_1 = results['scipy                ']
results_2 = results['CFSQP                ']

betas_1 = results_1.getBetaValues()
betas_2 = results_2.getBetaValues()

for k in betas_1.keys():
    b_1 = betas_1[k]
    b_2 = betas_2[k]
    print(f"{k:10}=\t{b_1:.8g}\t{b_2:.8g}")
```

    ASC_CAR   =	-0.15463245	-0.15463283
    ASC_TRAIN =	-0.70118664	-0.70118664
    B_COST    =	-1.0837907	-1.0837909
    B_TIME    =	-1.2778603	-1.2778601

**Explore the results**

```python
results = results['scipy                ']

Results = results.getEstimatedParameters()
display(Results)
```

      Value
      Std err
      t-test
      p-value
      Rob. Std err
      Rob. t-test
      Rob. p-value

      ASC_CAR
      -0.154632
      0.043235
      -3.576518
      0.000348
      0.058163
      -2.658586
      0.007847

      ASC_TRAIN
      -0.701187
      0.054874
      -12.778137
      0.000000
      0.082562
      -8.492846
      0.000000

      B_COST
      -1.083791
      0.051830
      -20.910412
      0.000000
      0.068225
      -15.885522
      0.000000

      B_TIME
      -1.277860
      0.056883
      -22.464577
      0.000000
      0.104254
      -12.257125
      0.000000

**Get the general statistics**
* Number of estimated parameters($K$)
* Sample size ($N$)
* Number of excluded observations
* Log likelihood of the sample with the default values for the parameters ($\mathcal{L}^i)$)
* Log likelihood for the final estimated model ($\mathcal{L}^*)$)
* Likelihood ratio:
\begin{align}
-2 (\mathcal{L}^i-\mathcal{L}^*)
\end{align}
* Rho-square for the init model
\begin{align}
\rho^2 = 1- \frac{\mathcal{L}^*}{\mathcal{L}^i}
\end{align}
* Rho-square adjusted for the init model
\begin{align}
\rho^2 = 1- \frac{\mathcal{L}^* - K}{\mathcal{L}^i}
\end{align}
* Akaike Information Criterion
\begin{align}
2 K - 2 \mathcal{L}^*
\end{align}
* Bayesian Information Criterion
\begin{align}
K N- 2 \mathcal{L}^*
\end{align}
* Final gradient norm
* Iterations
* Optimization Time

```python
gs = results.getGeneralStatistics()

for k,v in gs.items():
    print("{}= {}".format(k.ljust(45),v[0]))
```

    Number of estimated parameters               = 4
    Sample size                                  = 6768
    Excluded observations                        = 3960
    Init log likelihood                          = -6964.662979192295
    Final log likelihood                         = -5331.252006915832
    Likelihood ratio test for the init. model    = 3266.8219445529267
    Rho-square for the init. model               = 0.23452835796311466
    Rho-square-bar for the init. model           = 0.23395403010088356
    Akaike Information Criterion                 = 10670.504013831664
    Bayesian Information Criterion               = 10697.783857436807
    Final gradient norm                          = 6.477959424450974e-05
    Nbr of threads                               = 8


# "Solow Model"
> "Simple Solow model without technological progress"

- toc:true
- branch: master
- badges: true
- comments: false
- author: João B. Duarte
- categories: [Economic growth, python, dash]

The Solow model is described by the following system of equations:

\begin{align}
Y_t &= A_t F(K_t, L_t) & & \mathbf{\text{Production Function}}\\
\\
\Delta K_t &= I_t - \delta K_t  & & \mathbf{\text{Capital Accumulation}}\\
\\
S_t &= s(Y_t - \delta K_t) & & \mathbf{\text{Net Saving Behaviour}}\\
\\
I_t - \delta K_t &= S_t & & \mathbf{\text{Closed Economy}}
\end{align}

Lets simplify it.

The Closed Economy and Net Saving Behaviour equations imply that net investment is equal to a fixed fraction $s$ of net income:
$I_t - \delta K_t = s(Y_t - \delta K_t)$. We can then replace this into the Capital Accumulation equation to get:

$$\Delta K_t = s(Y_t - \delta K_t)$$

Lets think about this equation for a moment. If there is some net income leftover after paying for restocking the
 capital that gets lost during the latest production ($Y_t - \delta K_t >0$), then we invest part of it into building
 new capital ($\Delta K_t >0$). If, on the other hand, income is not enough to even pay for capital replacement, then
 capital falls next period ($\Delta K_t <0$).

 Finally, lets replace the production function into $Y_t$ to get the equation that summarizes all of the Solow model
 dynamics:

 $$\Delta K_t = s(A_t F(K_t, L_t) - \delta K_t) \quad \quad \text{(In levels)}$$

 And we can rewrite it in growth rates when dividing both sides by $K_t$:

 $$\frac{\Delta K_t}{K_t} = s\frac{A_t F(K_t, L_t)}{K_t} - s\delta \quad \quad \text{(In growth rates)}$$

Hence, so long as output per unit of capital is larger than what gets lost per unit of capital, capital will grow.

So the crucial part to understand capital growth is to figure out how output per unit of capital behaves.

**Assumptions**

1. $$\frac{\partial F(K_t, L_t)}{\partial K_t} > 0 \quad  \frac{\partial^2 F(K_t, L_t)}{\partial K_t^2} < 0 \quad \text{Decreasing returns to scale}$$

2. $$\frac{\Delta L_t}{L_t} = n$$

With assumption (1) in place, as the economy grows, output per unit of capital $\frac{A_t F(K_t, L_t)}{K_t}$ falls.
Which means that at some point it will be lower than depreciation $\delta$. Assumption (2) implies that population growth is constant.

***Solow model set in per capita variable***

Lets also assume a Cobb-Douglas production function: $Y_t = A_t K_t^\alpha L_t^{1-\alpha}$. Note that assumption 1 is verified
under this specific production function.

Let $k_t=\frac{K_t}{L_t}$ and $y_t = \frac{Y_t}{L_t}$ be per capita capital and output, respectively.

Lets use a log approximation $\ln{(1+x)} \approx x$ for small $x$ (the model can also hold exactly, ie. without approximation, but the
algebra does not match the class slides)

Using this approximation, we have that
$$\frac{\Delta k_t}{k_t} \approx \frac{\Delta K_t}{K_t} - \frac{\Delta L_t}{L_t}$$

Why?

$$\ln{k_t} = \ln{K_t} - \ln{L_t}$$

and so

$$\ln{k_{t+1} - \ln{k_t}}  = \ln{K_{t+1}} - \ln{K_t} - (\ln{L_{t+1}} -  \ln{L_t}) $$
$$\ln{\frac{k_{t+1}}{k_t}}  = \ln{\frac{K_{t+1}}{K_t}} - \ln{\frac{L_{t+1}}{L_t}}$$

$$\ln{(1 + \frac{\Delta k_{t}}{k_t})}  = \ln{(1 + \frac{\Delta K_{t}}{K_t})} - \ln{(1+\frac{\Delta L_{t}}{L_t})}$$

Hence,

$$\frac{\Delta k_t}{k_t} \approx \frac{\Delta K_t}{K_t} - \frac{\Delta L_t}{L_t}$$

Now, we have all the ingredients to write the main Solow equation in per capita terms:

$$\frac{\Delta K_t}{K_t} = s\frac{A_t F(K_t, L_t)}{K_t} - s\delta $$

$$\frac{\Delta k_t}{k_t} + \frac{\Delta L_t}{L_t} = s\frac{A_t k_t^\alpha}{k_t} - s\delta $$

 $$ \boxed{\frac{\Delta k_t}{k_t} = s\frac{A_t k_t^\alpha}{k_t} - (s\delta+n)}$$

```python
#hide
import numpy as np
import plotly.graph_objects as go
from jupyter_dash import JupyterDash
JupyterDash.infer_jupyter_proxy_config()
import dash_core_components as dcc
import dash_html_components as html
from dash.dependencies import Input, Output
import plotly.io as pio
pio.renderers.default = "plotly_mimetype+notebook_connected"

class Solow:
    r"""
    Implements the Solow growth model with the update rule

        k_{t+1} = [(s z k^α_t) + (1 - δ)k_t] /(1 + n)

    """
    def __init__(self, n=0.05,  # population growth rate
                       s=0.3,  # savings rate
                       δ=0.1,   # depreciation rate
                       α=0.3,   # share of labor
                       z=2.0,   # productivity
                       k=1.0):  # current capital stock

        self.n, self.s, self.δ, self.α, self.z = n, s, δ, α, z
        self.k = k

    def f(self, k):
        return self.z * k**self.α

    def savings(self, k):
        return self.s*self.f(k)/k

    def depreciation(self):
        return self.δ*self.s + self.n
solow = Solow()
solow2 = Solow()
k = np.linspace(1,100,100)
fig = go.Figure()
fig.add_trace(go.Scatter(x=k, y=solow.savings(k), name = 'Savings benchmark',
              line=dict(color='black', width=1)))
fig.add_trace(go.Scatter(x=k, y=solow.depreciation()*np.ones(100), name = 'Depreciation benchmark',
              line=dict(color='black', width=1)))
fig.add_trace(go.Scatter(x=k, y=solow2.savings(k), name = 'Savings',
              line=dict(color='blue', width=2)))
fig.add_trace(go.Scatter(x=k, y=solow2.depreciation()*np.ones(100), name = 'Depreciation',
              line=dict(color='red', width=2)))
fig.update_layout(title='Simulations',
                   xaxis_title='k',
                   yaxis_title='Value')
fig.update_layout(transition_duration=1000)
fig.show()

```

```python
#hide

# Build App
app = JupyterDash(__name__)
server = app.server
app.layout = html.Div([
    html.H1(
        children='Solow Model',
        style={
            'textAlign': 'center'
        }
    ),
     html.Label('Savings rate'),
    dcc.Slider(
        id='s-slider',
        min=0.1,
        max=0.9,
        step=0.1,
        marks={i: str(round(i,1)) for i in np.linspace(0.1,0.9,9)},
        value=0.3
    ),
    html.Label('Depreciation rate'),
    dcc.Slider(
        id='delta-slider',
        min=0.1,
        max=0.9,
        step=0.1,
        marks={i: str(round(i,1)) for i in np.linspace(0.1,0.9,9)},
        value=0.1
    ),
    html.Label('Population growth rate'),
    dcc.Slider(
        id='n-slider',
        min=0.01,
        max=0.09,
        step=0.01,
        marks={i: str(round(i,2)) for i in np.linspace(0.01,0.09,9)},
        value=0.05
    ),
    html.Label('Alpha'),
    dcc.Slider(
        id='alpha-slider',
        min=0.1,
        max=0.9,
        step=0.1,
        marks={i: str(round(i,1)) for i in np.linspace(0.1,0.9,9)},
        value=0.3
    ),
    html.Label('Technology'),
    dcc.Slider(
        id='tec-slider',
        min=1,
        max=9,
        step=1,
        marks={i: str(round(i,1)) for i in range(1,10)},
        value=2
    ),
    dcc.Graph(id='graph', figure = fig)
])
# Define callback to update graph
@app.callback(
    Output('graph', 'figure'),
    [Input("s-slider", "value"), Input("delta-slider", "value"),
     Input("n-slider", "value"), Input("alpha-slider", "value"), Input("tec-slider", "value")]
)
def update_figure(selected_s, selected_delta, selected_n, selected_α, selected_z):
    k = np.linspace(1,100,100)
    solow2.s = selected_s
    solow2.δ = selected_delta
    solow2.n = selected_n
    solow2.α = selected_α
    solow2.z = selected_z
    fig = go.Figure()
    fig.add_trace(go.Scatter(x=k, y=solow.savings(k), name = 'Savings benchmark',
                  line=dict(color='black', width=1)))
    fig.add_trace(go.Scatter(x=k, y=solow.depreciation()*np.ones(100), name = 'Depreciation benchmark',
                  line=dict(color='black', width=1)))
    fig.add_trace(go.Scatter(x=k, y=solow2.savings(k), name = 'Savings',
                  line=dict(color='blue', width=2)))
    fig.add_trace(go.Scatter(x=k, y=solow2.depreciation()*np.ones(100), name = 'Depreciation',
                  line=dict(color='red', width=2)))
    fig.update_layout(title='Simulations',
                   xaxis_title='k',
                   yaxis_title='Value')
    fig.update_layout(transition_duration=1000)
    return fig
# Run app and display result inline in the notebook
app.run_server(mode='inline')

```

```python

```


###### Material desenvolvido para o minicurso: Introdução à solução numérica de EDP's, ministrado no ERMAC/2018 de 5 a 6 de abril de 2018 na Universidade Federal de Lavras, Lavras/MG, Brasil. Autor: [Jonas Laerte Ansoni](http://jonasansoni.blogspot.com.br/).

#  Minicurso: Introdução à solução numérica de EDP's

##  Equação parabólica

### 3.2. Equação transiente de difusão de calor
######  Olhar Fortuna página 172

A equação clássica para problemas parabólicos é a equação transiente de difusão de calor:

\begin{equation}
\frac{\partial T}{\partial t} = \alpha \frac{\partial^2 T}{\partial x^2}
\tag{1}
\end{equation}

onde, $\alpha$ é o ceficiente de difusividade térmica (propriedade do material), e $T$ é a temperatura.

No _notebook_ anterior, nós discretizamos a equação de difusão com o esquema forward-time, centered-space, sujeita à seguinte restrição de estabilidade:

\begin{equation}
\alpha \frac{\Delta t}{(\Delta x)^2} \leq \frac{1}{2}
\tag{2}
\end{equation}

Vamos examiná-lo mais profundamente agora, usando um problema de evolução de temperatura 1D.

### Configuração do problema

Digamos que temos uma vareta de grafite com [difusividade térmica](http://en.wikipedia.org/wiki/Thermal_diffusivity) de $\alpha = 1,22 \times10^{-3} {\rm m}^2/{\rm s}$, comprimento $L=1{\rm m}$ e temperatura $T = 0{\rm C}$ em todos os lugares. No momento $t = 0$, aumentamos a temperatura no lado esquerdo, $x = 0$, para $T = 100{\rm C}$, e o mantemos constate. **Como a temperatura irá evoluir na haste?**

####  Figura 1.  Vareta de grafite, temperaturas fixas nas extremidades.

### Discretização

Como discutido na aula anterior para este tipo de problema podemos aplicar o método de Euler explícito.

\begin{equation}\frac{T_{i}^{n+1}-T_{i}^{n}}{\Delta t}=\alpha\frac{T_{i+1}^{n}-2T_{i}^{n}+T_{i-1}^{n}}{\Delta x^2}\tag{3} \label{eq3}\end{equation}

Para obter a temperatura no próximo passo de tempo, $T^{n+1}_i$, a partir das informações conhecidas no passo de tempo corrente, nós calculamos:

\begin{equation}T_{i}^{n+1}=T_{i}^{n}+\frac{\alpha\Delta t}{\Delta x^2}(T_{i+1}^{n}-2T_{i}^{n}+T_{i-1}^{n})\tag{4}\label{eq4}\end{equation}

As seguintes funções implementam o esquema numérico para resolver isto:

```python
import numpy
from matplotlib import pyplot
%matplotlib inline
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
```

```python
L = 1
nt = 100
nx = 51
alpha = 1.22e-3

dx = L/(nx-1)

Ti = numpy.zeros(nx)
Ti[0] = 100
```

```python
def ftcs(T, nt, dt, dx, alpha):
    """Solves the diffusion equation with forward-time, centered scheme

    Parameters:
    ----------
    T: array of float
        Initial temperature profile
    nt: int
        Number of time steps
    dt: float
        Time step size
    dx: float
        Mesh size
    alpha: float
        Diffusion coefficient (thermal diffusivity)

    Returns:
    -------
    T: array of float
        Temperature profile after nt time steps with forward in time scheme

    """

    for n in range(nt):
        Tn = T.copy()
        T[1:-1] = Tn[1:-1] + alpha*dt/dx**2*(Tn[2:] -2*Tn[1:-1] + Tn[0:-2])

    return T
```

Estamos todos prontos para executar! Primeiro, vamos usar um passo de tempo `dt` que satisfaz a restrição de estabilidade.

```python
sigma = 1/2.0
dt = sigma * dx*dx/alpha
T = ftcs(Ti.copy(), nt, dt, dx, alpha)
```

NOTA:  Estamos enviando uma *cópia* do `Ti` para a função `ftcs`. Enviar uma cópia garante que o valor de `Ti` permaneça inalterado para nós reutilizarmos.

```python
pyplot.plot(numpy.linspace(0,1,nx), T, color='#003366', ls='-', lw=3)
pyplot.ylim(0,100)
pyplot.xlabel('Length of Rod')
pyplot.ylabel('Temperature');
```

#### 3.2.1 Condições de contorno

No problema da barra acima, afirmamos que o lado esquerdo da haste é mantido a uma temperatura de $T = 100{\rm C}$. Este é um exemplo de uma *condição de contorno* (Boundary Condition - BC): uma regra que define como o problema interage com as bordas do domínio. Nesse caso, o domínio abrange o comprimento da haste, de $0 \leq x \leq 1$.

Existem vários tipos de condições de contorno (Dirichlet, Neumann, Robin, periódica...), mas iremos abordar aqui duas das mais utilizadas.

##### Condição de contorno de Dirichlet

* O valor de $\phi$ é especificado na fronteira.

O exemplo da haste acima usa um *Dirichlet* BC no lado esquerdo. Condição de contorno de Dirichlet é aquele em que a borda é mantida em um valor específico da variável de solução.

E quanto à extremidade direita da haste $(x = 1)$? Na discretização que estabelecemos acima, o problema não evoluiu por tempo suficiente para o calor percorrer todo o comprimento da haste. Vamos aumentar o número de timesteps, `nt` e ver o que acontece.

```python
nt = 1000
T = ftcs(Ti.copy(), nt, dt, dx, alpha)
```

```python
pyplot.plot(numpy.linspace(0,1,nx), T, color='#003366', ls='-', lw=3)
pyplot.ylim(0,100)
pyplot.xlabel('Length of Rod')
pyplot.ylabel('Temperature');
```

Hmmm ... parece que estamos colocando o BC do lado direito o valor $T = 0{\rm C}$, você concorda?
Agora estude o código para a função `ftbs`, acima, e tente descobrir o que acontece no ponto mais à direita da grade espacial.

Você descobriu?

O valor em `T [-1]` nunca atualiza!
Esse valor é definido como zero na condição inicial e permanece assim durante toda a simulação. Nós efetivamente temos condições de contorno do tipo Dirichlet em *ambas* extremidades. O lado esquerdo da haste é mantido a uma temperatura de $T = 100{\rm C}$ e o lado direito é mantido em $T = 0{\rm C}$. Ambas as temperaturas finais são "fixas" definindo valores, a distribuição de temperatura dentro da haste convergirá (após tempo suficiente) para um gradiente de temperatura linear em todo o comprimento da haste.

####  Figura 2.  Condução de calor e uma haste com temperatura fixa nas extremidades.

 Quando a solução estiver em equilíbrio, nenhum número de *timesteps* extras fará com que ela seja alterada. Este é um exemplo de uma <font color=`blue`>*solução de estado estacionário*.

As condições de contorno de Dirichlet aparecem em muitas aplicações de engenharia, entre eles a termodinâmica (por exemplo, a definição de uma temperatura superficial) e a dinâmica do fluido (por exemplo, as condições de não deslizamento nas paredes em fluidos viscosos).

NOTA:

Em geral, não se deve aplicar condições de fronteira do tipo Direchlet, simultaneamente, nas fronteiras de entrada e de saída do escoamento a menos que se conheça a pressão real nessas fronteiras. Pode-se originar soluções fisicamente incorretas ou mesmo divergência do método numérico.

##### Condição de contorno de Neumann

Em problemas de dinâmica dos fluidos computacional, é muito comum resolvermos problemas que exigem condições de fronteira, em que a derivada normal (gradiente) da variável dependente é especificada na fronteira.

* O gradiente normal $\frac{\partial \phi}{\partial n}=f$ ou tangencial $\frac{\partial \phi}{\partial s}=g$ é especificado na fronteira. Quando $f=0$, dizemos que a condição utilizada é uma __condição de fronteira natural__ ou __homogênia__.

Por exemplo, quando temos uma parede adiabática, o fluxo de calor através dessa parede é zero:

\begin{equation}
q=k \frac{\partial T}{\partial n}=0,
\end{equation}
onde $n$ é a direção normal à parede.

Assim, uma vez que $k>0,$ um fluxo nulo de calor na parede implica

\begin{equation}
\frac{\partial T}{\partial n}=0.
\end{equation}

Diferentemente da condição de Dirichlet, onde os valores de uma grandeza $\phi$ na fronteira são conhecidos, na condição de Neumann, apenas o gradiente normal de $\phi$ na fronteira é especificado. Assim, $\phi$ na fronteira também é uma incógnita que __deve ser determinada__ como parte do processo de solução.

### _Pergunta:_ Como podemos utilizar está condição de contorno no nosso problema da haste?

### Resposta:
(Fortuna, página 121, 177) Supondo que a extremidade direita da haste ($x=1$) esteja termicamente isolada, de tal modo que não haja fluxo de calor através da extremidade da mesma:

\begin{equation}
\label{eq5}
\left. \frac{\partial T}{\partial x} \right|_{x=1} = 0
\tag{5}
\end{equation}

Utilizando o método de Euler explícito, juntamente com diferenças centrais de segunda ordem para a derivada espacial conforme apresentado nas equações \ref{eq3} e \ref{eq4}.

Note-se que como já discutido, temos que calcular o valor da varialvel $T$ no ponto $x=1$, e a expressão \ref{eq4}, faz referência à temperatura em $T^n_{N+1}$. Assim, para o cálcularmos o valor de $T$ na extremidade direita da haste dependemos de um valor $N+1$ que estará fora do nosso domínio. __Ponto fantasma.__

Para tratarmos este problema de ponto fantasma, podemos discretizar a condição de fronteira Eq. \ref{eq5} utilizando diferenças centrais de segunda ordem aplicadas ao ponto $x_N$:

\begin{equation}
\left. \frac{\partial T}{\partial x} \right|_{x=1} \approx \frac{T^n_{N+1} - T^n_{N-1}}{2\Delta x}=0
\end{equation}

Essa discretização implica que

\begin{equation}
T^n_{N-1}=T^n_{N+1}.
\end{equation}

Pense no que a equação está dizendo: a mudança na temperatura ao se mover na direção $x$ é zero na borda mais à direita da haste. Isso significa que a temperatura deve ser igual nos dois últimos pontos da grade espacial.

Para reforçar essa condição de limite de Neumann na extremidade direita da haste, adicionamos a seguinte linha de código:

```Python
T[-1] = T[-2]
```

Ou seja, a temperatura no limite (`T [-1]`) é igual à temperatura diretamente à sua esquerda (`T [-2]`). O gradiente espacial é zero e uma condição de Neumann é satisfeita lá.

As condições de Dirichlet no exemplo da haste foram aplicadas sem nenhuma entrada extra, mas a condição Neumann exigirá uma atualização em cada iteração do loop.

Finalmente, os BCs de Dirichlet e Neumann às vezes podem ser misturados para representar melhor a física de um problema.

Vamos revisitar o problema da vara aquecida. Desta vez, a temperatura em $x = 0$ permanecerá fixa em $T = 100{\rm C}$ - essa é a condição de Dirichlet - e um Neumann BC se aplica a $x = 1$.

```python
def ftcs_mixed(T, nt, dt, dx, alpha):
    """Solves the diffusion equation with forward-time, centered scheme using
    Dirichlet b.c. at left boundary and Neumann b.c. at right boundary

    Parameters:
    ----------
    u: array of float
        Initial temperature profile
    nt: int
        Number of time steps
    dt: float
        Time step size
    dx: float
        Mesh size
    alpha: float
        Diffusion coefficient (thermal diffusivity)

    Returns:
    -------
    u: array of float
        Temperature profile after nt time steps with forward in time scheme

    """
    for n in range(nt):
        Tn = T.copy()
        T[1:-1] = Tn[1:-1] + alpha*dt/dx**2*(Tn[2:] -2*Tn[1:-1] + Tn[0:-2])
        T[-1] = T[-2]

    return T
```

```python
nt = 1000
T = ftcs_mixed(Ti.copy(), nt, dt, dx, alpha)
```

```python
pyplot.plot(numpy.linspace(0,1,nx), T, color='#003366', ls='-', lw=3)
pyplot.ylim(0,100)
pyplot.title('Rod with LHS Dirichlet B.C. and RHS Neumann B.C.\n')
pyplot.xlabel('Length of Rod')
pyplot.ylabel('Temperature');
```

Agora o calor está se acumulando no domínio. Nosso isolador funciona! Se você aumentar ainda mais o número de timesteps `nt`, verá que a temperatura de saída em $x = 1$ continua a aumentar.

Fortuna 113 - problema discretização explícita

#### 3.2.2 Esquemas explícitos e condições de contorno

Os esquemas numéricos que aprendemos até agora são chamados _explícitos_, porque calculamos a solução atualizada em $t^{n + 1}$ usando apenas informações conhecidas no tempo $t^n$. Isto é simples, mas tem limitações, particularmente com os pequenos tamanhos de tempo necessários para obter estabilidade.

Aqui está outro problema com esquemas explícitos. A Figura 3 mostra o estêncil sobreposto de vários pontos da malha em três etapas de tempo calculadas usando o esquema já apresentado anteriormente. Sabemos que para calcular $T_i^{n + 1}$ utilizamos as informações dos pontos de grade $i-1, i, i + 1$ no passo de tempo anterior. Pense no que acontece na fronteira: qualquer alteração na condição de fronteira alimentará a solução apenas na próxima etapa, não imediatamente. Mas isso contradiz a física do problema, pois qualquer mudança na fronteira deve ser sentida imediatamente sob a equação de difusão. Para incluir efeitos de fronteira no mesmo nível de tempo, podemos usar uma atualização implícita (assunto da próxima lição).

####  Figura 3. Propagação da condição de contornoem uma discretização explícita.

```python
from IPython.core.display import HTML
css_file = '../styles/custom.css'
HTML(open(css_file, "r").read())
```


```python
%pylab inline
pylab.rcParams['figure.figsize'] = (16.0, 8.0)
```

    Populating the interactive namespace from numpy and matplotlib

# Summarizing the Monte Carlo output

The result of a Monte Carlo simulation is a set of samples from the probability distribution associated with the measurand
$$ \{\mathbf{y}^{(k)},\,k=1,\ldots,M\} $$

The aim is to derive characteristic information about the measurand from this set:

1) best estimate

2) uncertainty associated with the best estimate

3) intervals/regions with a prescribed coverage probability

## Univariate measurand

1) best estimate
\begin{align}
y =& mean \{ y^{(k)}, k=1,\ldots,M\} \\
=& \frac{1}{M} \sum_{k=1}^M y^{(k)}
\end{align}
2) uncertainty associated with the best estimate
\begin{align}
u_y =& std\{ y^{(k)}, k=1,\ldots,M\} \\
=& \frac{1}{M-1} \sum_{k=1}^M (y-y^{(k)})^2
\end{align}

### Exercise 4.1

Draw randomly from the normal distribution with mean $\mu=1.3$ and standard deviation $\sigma=0.4$ and calculate best estimate and its uncertainy using 100, 200, 300, ..., 100 000 draws. Compare estimate and uncertainty with mean and standard deviation of the original distribution for the different number of draws.

```python
from scipy.stats import norm

Y = norm.rvs(loc=1.3, scale=0.4, size=100000)

draws = arange(100,len(Y)+100,100)
means = array([Y[:M].mean() for M in draws])
uncs  = array([Y[:M].std() for M in draws])

figure()
subplot(121)
plot(draws, means)
axhline(1.3,color="k",linestyle="--")
subplot(122)
plot(draws, uncs)
axhline(0.4,color="k",linestyle="--");
```

Intervals with a prescribed coverage probability can be calculated from the Monte Carlo outcome as follows

1) Sort the Monte Carlo outcome in ascending order
``` python
    sort(Y)
```
2) For propability $P$ calculate the corresponding fraction of Monte Carlo draws $q=int(PM)$

3) Set lower bound of interval as $r=int(0.5(M-q))$ for a probabilistically symmetrical interval

4) Calculate upper bound of interval as $r+q$

### Exercise 4.2

Draw randomly from the distribution from Exercise 4.1 and calculate the 95% probabilistally symetrical coverage interval from 100000 draws. Compare the result to the exact 95% interval.

```python
# solution for exercise 4.2

from scipy.stats import norm

P = 0.95 # sought probability of coverage interval

draws = 100000
Y = norm.rvs(loc=1.3, scale=0.4, size=draws)

# step 1 - sort random draws
Ys = sort(Y)
# step 2 - calculate portion of draws corresponding to P
q  = int(draws*P)
# step 3 - set lower bound of coverage interval
r  = int(0.5*(draws-q))
# interval bounds
ylow = Ys[r]
yhigh = Ys[r+q]
interval = [ylow, yhigh]
true_interval = [1.3 - 1.96*0.4, 1.3 + 1.96*0.4]

print("calculated interval")
print(interval)
print("true interval")
print(true_interval)
```

    calculated interval
    [0.51555327864031619, 2.0847763045218146]
    true interval
    [0.516, 2.084]

## Multivariate measurand

1) best estimate
\begin{align}
\mathbf{y} =& mean \{ \mathbf{y}^{(k)}, k=1,\ldots,M\} \\
=& \frac{1}{M} \sum_{k=1}^M \mathbf{y}^{(k)}
\end{align}
2) uncertainty associated with the best estimate
\begin{align}
U_\mathbf{y} =& cov\{ \mathbf{y}^{(k)}, k=1,\ldots,M\} \\
=& \frac{1}{M-1} \sum_{k=1}^M (\mathbf{y}-\mathbf{y}^{(k)})(\mathbf{y}-\mathbf{y}^{(k)})^T
\end{align}

### Exercise 4.3

Draw randomly from the normal distribution with mean
$$\mathbf{\mu}=\left( \begin{array}{c}
0.4 \\ -1.5
\end{array}\right)
$$
and covariance
$$
\Sigma=\left(\begin{array}{cc}
0.09 & -0.2 \\ -0.2 & 1.44
\end{array}\right)
$$
and calculate best estimate and its uncertainy using 1000, 10000 and 100 000 draws. Compare estimate and uncertainty with mean and covariance of the original distribution for the different number of draws.

```python
# solution for exercise 4.3

from scipy.stats import multivariate_normal

Y = multivariate_normal.rvs(mean=[0.4, -1.5],
                            cov = array([[0.09, -0.2],[-0.2, 1.44]]),
                            size = 100000)
draws = [1000, 10000, 100000]
means = array([Y[:M].mean(axis=0) for M in draws])
uncs  = array([cov(Y[:M], rowvar=0) for M in draws])

print("calculated estimates")
print(means)
print("calculated covariances")
print(uncs)
```

    calculated estimates
    [[ 0.39823368 -1.47816826]
     [ 0.39785645 -1.49430727]
     [ 0.39923487 -1.49887348]]
    calculated covariances
    [[[ 0.0946245  -0.1966798 ]
      [-0.1966798   1.37136819]]

     [[ 0.09177784 -0.20437818]
      [-0.20437818  1.43619558]]

     [[ 0.09024844 -0.19873001]
      [-0.19873001  1.42682509]]]

Regions with a prescribed coverage probability can be calculated from the multivariate Monte Carlo outcome as follows

1) Calculate the Cholesky decomposition of the sample covariance matrix $U_{\mathbf{y}}=\mathbf{LL}^T$

2) Transform the Monte Carlo outcomes
$$ \mathbf{y}_{(k)} = \mathbf{L}^{-1}(\mathbf{y}^{(k)}-\mathbf{y})$$
and sort according to the distance measure
$$ d^2_{(k)} = \mathbf{y}_{(k)}^T\mathbf{y}_{(k)} $$

3) calculate $k_P$ such that a fraction $P$ of all Monte Carlo outcomes satisfies $d_{(k)}<k_P$

This defines the ellipsoidal region $(\mathbf{\eta}-\mathbf{y})^TU_{\mathbf{y}}^{-1}(\mathbf{\eta}-\mathbf{y})<k^2_P$

For a bivariate normal distribution, the factor for a 95% coverage ellipsoidal region is given as the 95% quantile of the $\chi^2$ distribution with 2 degrees of freedom.

```python
## !! Run this cell before continuing !!

# helper function to calculate ellipsoidal parameters from mean and covariance
def calculate_ellipse(mu, Sigma, kP):
    vals, vecs = linalg.eigh(Sigma)
    order = vals.argsort()[::-1]
    vals = vals[order]
    vecs = vecs[:,order]
    theta = degrees(np.arctan2(*vecs[:,0][::-1]))
    width, height = kP * sqrt(vals)
    return width, height, theta

```

### Exercise 4.4

Calculate 100 000 random draws from the distribution from Exercise 4.3 and calculate the 95% coverage region. Compare to the true 95% coverage region.

```python
# solution for exercise 4.4

from scipy.stats import multivariate_normal, chi2
from matplotlib.patches import Ellipse

mu = array([0.4, -1.5])
Sigma = array([[0.09, -0.2],[-0.2, 1.44]])

P = 0.95
draws = 10000
q = int(P*draws)

Y = multivariate_normal.rvs(mean = mu, cov = Sigma, size = draws)

y = Y.mean(axis=0)
Uy= cov(Y, rowvar=0)

# step 1 - Cholesky decomposition of covariance matrix
L = cholesky(Uy)
# step 2 - Transform the Monte Carlo outcomes
Ys = linalg.solve(L, (Y - tile(y,(draws,1))).T).T
d = diag(dot(Ys,Ys.T))
# and sort distances
ds = sort(d)
# step 3 - calculate kP as the scale parameter of the ellipse
kP = ds[q]
width, height, theta = calculate_ellipse(y, Uy, kP)

figure()
scatter(Y[:,0],Y[:,1],edgecolor="none",alpha=0.5)
ellip=Ellipse(xy=y, width=width, height=height, angle=theta,facecolor="none")
gca().add_artist(ellip)
w,h,t = calculate_ellipse(mu, Sigma, chi2.ppf(0.95,2))
ellip_true = Ellipse(xy=mu,width=w,height=h,angle=t,facecolor="none",edgecolor="r")
gca().add_artist(ellip_true);
```

04 Summarizing the Monte Carlo output.ipynb