Text provided under a Creative Commons Attribution license, CC-BY.  All code is made available under the FSF-approved BSD-3 license.  (c) Lorena A. Barba, Gilbert F. Forsyth 2017. Thanks to NSF for support via CAREER award #1149784.
[@LorenaABarba](https://twitter.com/LorenaABarba)

12 steps to Navier–Stokes
=====
***

We continue our journey to solve the Navier–Stokes equation with Step 4. But don't continue unless you have completed the previous steps! In fact, this next step will be a combination of the two previous ones. The wonders of *code reuse*!

Step 4: Burgers' Equation
----
***

You can read about Burgers' Equation on its [wikipedia page](http://en.wikipedia.org/wiki/Burgers'_equation).

Burgers' equation in one spatial dimension looks like this:

$$\frac{\partial u}{\partial t} + u \frac{\partial u}{\partial x} = \nu \frac{\partial ^2u}{\partial x^2}$$

As you can see, it is a combination of non-linear convection and diffusion. It is surprising how much you learn from this neat little equation!

We can discretize it using the methods we've already detailed in Steps [1](./01_Step_1.ipynb) to [3](./04_Step_3.ipynb).  Using forward difference for time, backward difference for space and our 2nd-order method for the second derivatives yields:

$$\frac{u_i^{n+1}-u_i^n}{\Delta t} + u_i^n \frac{u_i^n - u_{i-1}^n}{\Delta x} = \nu \frac{u_{i+1}^n - 2u_i^n + u_{i-1}^n}{\Delta x^2}$$

As before, once we have an initial condition, the only unknown is $u_i^{n+1}$. We will step in time as follows:

$$u_i^{n+1} = u_i^n - u_i^n \frac{\Delta t}{\Delta x} (u_i^n - u_{i-1}^n) + \nu \frac{\Delta t}{\Delta x^2}(u_{i+1}^n - 2u_i^n + u_{i-1}^n)$$

### Initial and Boundary Conditions

To examine some interesting properties of Burgers' equation, it is helpful to use different initial and boundary conditions than we've been using for previous steps.

Our initial condition for this problem is going to be:

\begin{eqnarray}
u &=& -\frac{2 \nu}{\phi} \frac{\partial \phi}{\partial x} + 4 \\\
\phi &=& \exp \bigg(\frac{-x^2}{4 \nu} \bigg) + \exp \bigg(\frac{-(x-2 \pi)^2}{4 \nu} \bigg)
\end{eqnarray}

This has an analytical solution, given by:

\begin{eqnarray}
u &=& -\frac{2 \nu}{\phi} \frac{\partial \phi}{\partial x} + 4 \\\
\phi &=& \exp \bigg(\frac{-(x-4t)^2}{4 \nu (t+1)} \bigg) + \exp \bigg(\frac{-(x-4t -2 \pi)^2}{4 \nu(t+1)} \bigg)
\end{eqnarray}

Our boundary condition will be:

$$u(0) = u(2\pi)$$

This is called a *periodic* boundary condition. Pay attention! This will cause you a bit of headache if you don't tread carefully.

### Saving Time with SymPy

The initial condition we're using for Burgers' Equation can be a bit of a pain to evaluate by hand.  The derivative $\frac{\partial \phi}{\partial x}$ isn't too terribly difficult, but it would be easy to drop a sign or forget a factor of $x$ somewhere, so we're going to use SymPy to help us out.

[SymPy](http://sympy.org/en/) is the symbolic math library for Python.  It has a lot of the same symbolic math functionality as Mathematica with the added benefit that we can easily translate its results back into our Python calculations (it is also free and open source).

Start by loading the SymPy library, together with our favorite library, NumPy.

```python
import numpy
import sympy
```

We're also going to tell SymPy that we want all of its output to be rendered using $\LaTeX$. This will make our Notebook beautiful!

```python
from sympy import init_printing
init_printing(use_latex=True)
```

Start by setting up symbolic variables for the three variables in our initial condition and then type out the full equation for $\phi$.  We should get a nicely rendered version of our $\phi$ equation.

```python
x, nu, t = sympy.symbols('x nu t')
phi = (sympy.exp(-(x - 4 * t)**2 / (4 * nu * (t + 1))) +
       sympy.exp(-(x - 4 * t - 2 * sympy.pi)**2 / (4 * nu * (t + 1))))
phi
```

It's maybe a little small, but that looks right.  Now to evaluate our partial derivative $\frac{\partial \phi}{\partial x}$ is a trivial task.

```python
phiprime = phi.diff(x)
phiprime
```

If you want to see the unrendered version, just use the Python print command.

```python
print(phiprime)
```

    -(-8*t + 2*x)*exp(-(-4*t + x)**2/(4*nu*(t + 1)))/(4*nu*(t + 1)) - (-8*t + 2*x - 4*pi)*exp(-(-4*t + x - 2*pi)**2/(4*nu*(t + 1)))/(4*nu*(t + 1))

### Now what?

Now that we have the Pythonic version of our derivative, we can finish writing out the full initial condition equation and then translate it into a usable Python expression.  For this, we'll use the *lambdify* function, which takes a SymPy symbolic equation and turns it into a callable function.

```python
from sympy.utilities.lambdify import lambdify

u = -2 * nu * (phiprime / phi) + 4
print(u)
```

    -2*nu*(-(-8*t + 2*x)*exp(-(-4*t + x)**2/(4*nu*(t + 1)))/(4*nu*(t + 1)) - (-8*t + 2*x - 4*pi)*exp(-(-4*t + x - 2*pi)**2/(4*nu*(t + 1)))/(4*nu*(t + 1)))/(exp(-(-4*t + x - 2*pi)**2/(4*nu*(t + 1))) + exp(-(-4*t + x)**2/(4*nu*(t + 1)))) + 4

### Lambdify

To lambdify this expression into a useable function, we tell lambdify which variables to request and the function we want to plug them in to.

```python
ufunc = lambdify((t, x, nu), u)
print(ufunc(1, 4, 3))
```

    3.49170664206

### Back to Burgers' Equation

Now that we have the initial conditions set up, we can proceed and finish setting up the problem.  We can generate the plot of the initial condition using our lambdify-ed function.

```python
from matplotlib import pyplot
%matplotlib inline

###variable declarations
nx = 101
nt = 100
dx = 2 * numpy.pi / (nx - 1)
nu = .07
dt = dx * nu

x = numpy.linspace(0, 2 * numpy.pi, nx)
un = numpy.empty(nx)
t = 0

u = numpy.asarray([ufunc(t, x0, nu) for x0 in x])
u
```

    array([ 4.        ,  4.06283185,  4.12566371,  4.18849556,  4.25132741,
            4.31415927,  4.37699112,  4.43982297,  4.50265482,  4.56548668,
            4.62831853,  4.69115038,  4.75398224,  4.81681409,  4.87964594,
            4.9424778 ,  5.00530965,  5.0681415 ,  5.13097336,  5.19380521,
            5.25663706,  5.31946891,  5.38230077,  5.44513262,  5.50796447,
            5.57079633,  5.63362818,  5.69646003,  5.75929189,  5.82212374,
            5.88495559,  5.94778745,  6.0106193 ,  6.07345115,  6.136283  ,
            6.19911486,  6.26194671,  6.32477856,  6.38761042,  6.45044227,
            6.51327412,  6.57610598,  6.63893783,  6.70176967,  6.76460125,
            6.82742866,  6.89018589,  6.95176632,  6.99367964,  6.72527549,
            4.        ,  1.27472451,  1.00632036,  1.04823368,  1.10981411,
            1.17257134,  1.23539875,  1.29823033,  1.36106217,  1.42389402,
            1.48672588,  1.54955773,  1.61238958,  1.67522144,  1.73805329,
            1.80088514,  1.863717  ,  1.92654885,  1.9893807 ,  2.05221255,
            2.11504441,  2.17787626,  2.24070811,  2.30353997,  2.36637182,
            2.42920367,  2.49203553,  2.55486738,  2.61769923,  2.68053109,
            2.74336294,  2.80619479,  2.86902664,  2.9318585 ,  2.99469035,
            3.0575222 ,  3.12035406,  3.18318591,  3.24601776,  3.30884962,
            3.37168147,  3.43451332,  3.49734518,  3.56017703,  3.62300888,
            3.68584073,  3.74867259,  3.81150444,  3.87433629,  3.93716815,  4.        ])

```python
pyplot.figure(figsize=(11, 7), dpi=100)
pyplot.plot(x, u, marker='o', lw=2)
pyplot.xlim([0, 2 * numpy.pi])
pyplot.ylim([0, 10]);
```

This is definitely not the hat function we've been dealing with until now. We call it a "saw-tooth function".  Let's proceed forward and see what happens.

### Periodic Boundary Conditions

One of the big differences between Step 4 and the previous lessons is the use of *periodic* boundary conditions.  If you experiment with Steps 1 and 2 and make the simulation run longer (by increasing `nt`) you will notice that the wave will keep moving to the right until it no longer even shows up in the plot.

With periodic boundary conditions, when a point gets to the right-hand side of the frame, it *wraps around* back to the front of the frame.

Recall the discretization that we worked out at the beginning of this notebook:

$$u_i^{n+1} = u_i^n - u_i^n \frac{\Delta t}{\Delta x} (u_i^n - u_{i-1}^n) + \nu \frac{\Delta t}{\Delta x^2}(u_{i+1}^n - 2u_i^n + u_{i-1}^n)$$

What does $u_{i+1}^n$ *mean* when $i$ is already at the end of the frame?

Think about this for a minute before proceeding.

```python
for n in range(nt):
    un = u.copy()
    for i in range(1, nx-1):
        u[i] = un[i] - un[i] * dt / dx *(un[i] - un[i-1]) + nu * dt / dx**2 *\
                (un[i+1] - 2 * un[i] + un[i-1])
    u[0] = un[0] - un[0] * dt / dx * (un[0] - un[-2]) + nu * dt / dx**2 *\
                (un[1] - 2 * un[0] + un[-2])
    u[-1] = u[0]

u_analytical = numpy.asarray([ufunc(nt * dt, xi, nu) for xi in x])
```

```python
pyplot.figure(figsize=(11, 7), dpi=100)
pyplot.plot(x,u, marker='o', lw=2, label='Computational')
pyplot.plot(x, u_analytical, label='Analytical')
pyplot.xlim([0, 2 * numpy.pi])
pyplot.ylim([0, 10])
pyplot.legend();
```

***

What next?
----

The subsequent steps, from 5 to 12, will be in two dimensions. But it is easy to extend the 1D finite-difference formulas to the partial derivatives in 2D or 3D. Just apply the definition — a partial derivative with respect to $x$ is the variation in the $x$ direction *while keeping $y$ constant*.

Before moving on to [Step 5](./07_Step_5.ipynb), make sure you have completed your own code for steps 1 through 4 and you have experimented with the parameters and thought about what is happening. Also, we recommend that you take a slight break to learn about [array operations with NumPy](./06_Array_Operations_with_NumPy.ipynb).

```python
from IPython.core.display import HTML
def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
    }
    div.cell{
        width:800px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    h2 {
        font-family: 'Fenix', serif;
    }
    h3{
		font-family: 'Fenix', serif;
        margin-top:12px;
        margin-bottom: 3px;
       }
	h4{
		font-family: 'Fenix', serif;
       }
    h5 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    div.text_cell_render{
        font-family: 'Alegreya Sans',Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 135%;
        font-size: 120%;
        width:600px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro";
			font-size: 90%;
    }
/*    .prompt{
        display: None;
    }*/
    .text_cell_render h1 {
        font-weight: 200;
        font-size: 50pt;
		line-height: 100%;
        color:#CD2305;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
        display: block;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 16pt;
        color: #CD2305;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>


# Tutoriel MID Contrôle

On traite le quasipolynôme suivant :
$$
Q(s) = s^n+a_{n-1}s^{n-1}+...+a_0 + \left[\alpha_ms^m+...+\alpha_0\right]e^{-s\tau}
$$

Cette fois-ci, on considère que l'on connait les paramètres du systèmes $a_i$ ainsi que l'un des hyperparamètres $s_0$ ou $\tau$. Ainsi, nous allons utiliser la MID pour trouver les $\alpha_j$ et l'hyperparamètre manquant.

## Packages nécessaires

On utilise numpy pour les opérations mathématiques numériques telles que les opérations matricielles (produit matriciel par exemple).

Sympy permet de réaliser des calculs en symbolique sur Python et donc de travailler avec des expressions mathématiques et en numérique.

```python
# pip install sympy si nécessaire (module not found), pareil avec cxroots
%matplotlib notebook
import numpy as np
import sympy as sp
import cxroots as cx
import matplotlib.pyplot as plt
import traceback
```

## Paramètres
$n$ est le degré du polynôme non retardé, $m$ est celui du polynôme retardé (celui multiplié par l'exponentielle).

$s_0$ est le taux de décroissance exponentiel. Il s'agit de la racine dominante que l'on souhaite assigner (celle la plus à droite lorsqu'on plot les racines). Celui-ci doit être négatif pour que le système soit stable. Un $s_0$ positif signifie donc que le système est instable.

$\tau$ est le retard considéré.

```python
n = 2
m = 1

```

## Création de symboles

Les variables s, tau, a et alpha vont désormais correspondre à des symboles et non des variables numériques.

On considère dans nos cas concrets que les $a_i$ correspondent aux paramètres du système (de son équation) et les $\alpha_j$ correspondent aux gains de la loi de commande retardée.

```python
s = sp.symbols('s')  # define variable s for our problem to be solved
tau = sp.symbols('tau')  # define variable tau : delay

a = sp.symbols(["a{:d}".format(i) for i in range(n)], real=True)
alpha = sp.symbols(["alpha{:d}".format(i) for i in range(m + 1)], real=True)
avalue = [2, 1]

```

## Création du quasipolynome

```python
Polynomial = s**n + np.array(a).dot([s**i for i in range(n)]) # Revient à faire s^n + a_{n-1}^{n-1}...
Delayed = np.array(alpha).dot([s**i for i in range(m+1)])*sp.exp(-s*tau) # Revient à faire
#b^m*s^m + b_{m-1}^{m-1}...
Q = Polynomial + Delayed
Q
```

$\displaystyle a_{0} + a_{1} s + s^{2} + \left(\alpha_{0} + \alpha_{1} s\right) e^{- s \tau}$

## Créer le système à résoudre

Dans ce cadre de MID orientée contrôle, on a $m+2$ inconnues qui sont les $\alpha_j$ ($m+1$ variables de $0$ à $m$ inclus) et l'hyperparamètre manquant

On parle donc de multiplicité de $m+2$ et on souhaite donc avoir $m+2$ équations. Pour l'instant nous avons une première équation qui correspond à notre quasipolynôme (=0).

Il y a deux possibilités :

- $s_0$ est donné
- $\tau$ est donné

Dans ces deux cas, il faudra procéder en deux étapes en considérant d'abord les $m+1$ premières équations (toutes sauf la dernière) que l'on résoudra facilement en $\alpha_j$ (système linéaire en $\alpha$) et on remplacera ensuite les $\alpha_j$ dans la dernière équation avec les expressions trouvées ce qui nous fera une équation permettant de trouver l'hyperparamètre manquant.

Il faut donc calculer les $m+1$ dérivées suivantes ce qui se fait très simplement avec Sympy en utilisant la méthode <i>diff</i> en spécifiant en argument la variable de différenciation.

```python
SysDerivatif = [Q]
print(Q)
for i in range(m+1):
    DerniereDerivee = SysDerivatif[-1]
    SysDerivatif.append(DerniereDerivee.diff(s)) # Dérivée par rapport à s
    print(SysDerivatif[-1])
```

    a0 + a1*s + s**2 + (alpha0 + alpha1*s)*exp(-s*tau)
    a1 + alpha1*exp(-s*tau) + 2*s - tau*(alpha0 + alpha1*s)*exp(-s*tau)
    -2*alpha1*tau*exp(-s*tau) + tau**2*(alpha0 + alpha1*s)*exp(-s*tau) + 2

## Résolution du système

Après avoir créé le système dérivatif, il suffit de le résoudre avec Sympy en spécifiant bien que l'on souhaite obtenir les expressions des $a_i$ et des $\alpha_j$.

Ici, on a bien $n+m+1 = 4$ ce qui nous retourne $a_0, a_1, \alpha_0$ et $\alpha_1$ en gardant bien à l'esprit que $a_2=1$.

Le ".args[0]" sert à extraire les solutions à partir du solveur qui retourne un set.

```python
sol = sp.linsolve(SysDerivatif[:-1], alpha).args[0] # Solveur selon les alpha et les a
sol
```

$\displaystyle \left( \left(a_{0} s \tau - a_{0} + a_{1} s^{2} \tau + s^{3} \tau + s^{2}\right) e^{s \tau}, \  - \left(a_{0} \tau + a_{1} s \tau + a_{1} + s^{2} \tau + 2 s\right) e^{s \tau}\right)$

### $\tau$ connu

On cherche donc à trouver $s_0$ et on choisit le $s_0$ réel **négatif** qui est le **plus proche de $0$** s'il existe.

```python
def solve_tau_connu(tau_val, acoef, Q):
    sys = [Q]
    for i in range(m+1):
        DerniereDerivee = sys[-1]
        sys.append(DerniereDerivee.diff(s)) # Dérivée par rapport à s
    sol = sp.linsolve(SysDerivatif[:-1], alpha).args[0]
    finaleq = SysDerivatif[-1].subs({alph : alphacoef for alph, alphacoef in zip(alpha, sol)})
    finaleq = finaleq.subs({asymb: aval for asymb, aval in zip(a, acoef)})
    solS0 = finaleq.subs({tau : tau_val})
    solS0 = sp.solve(solS0)
    solS0eval = [i.evalf() for i in solS0]
    try :
        solution = max([i for i in solS0eval if i<0])
    except Exception :
        traceback.print_exc()
    return solution
```

```python
value_tau = 0.738

finaleq = SysDerivatif[-1].subs({alph : alphacoef for alph, alphacoef in zip(alpha, sol)}) #remplace les coeffs
finaleq = finaleq.subs({asymb: aval for asymb, aval in zip(a, avalue)})
print(finaleq)
solS0 = finaleq.subs({tau : value_tau})
solS0 = sp.solve(solS0)
print(solS0)
solS0eval = [i.evalf() for i in solS0]
```

    tau**2*(-s*(s**2*tau + s*tau + 2*s + 2*tau + 1)*exp(s*tau) + (s**3*tau + s**2*tau + s**2 + 2*s*tau - 2)*exp(s*tau))*exp(-s*tau) + 2*tau*(s**2*tau + s*tau + 2*s + 2*tau + 1) + 2
    [-4.59643376575614, -1.82362043478586]

```python
solve_tau_connu(0.738, [2, 1], Q) # Safety check on function
```

$\displaystyle -1.82362043478586$

```python
computedS0 = solS0[1]
alpha_num = sol.subs({asymb: aval for asymb, aval in zip(a, avalue)})
alpha_num = alpha_num.subs({s : computedS0})
alpha_num = alpha_num.subs({tau : value_tau})
alpha_num_eval = [i.evalf() for i in alpha_num]
alpha_num_eval
alpha_sens = alpha_num_eval
```

### $s_0$ connu

On cherche donc à trouver $\tau$ et on choisit le $\tau$ réel **positif** qui est le **plus proche de $0$** s'il existe.

```python
value_s0 = -2

finaleq = SysDerivatif[-1].subs({alph : alphacoef for alph, alphacoef in zip(alpha, sol)}) #remplace les coeffs
finaleq = finaleq.subs({asymb: aval for asymb, aval in zip(a, avalue)})
solTau = finaleq.subs({s : value_s0})
solTau = sp.solve(solTau)
print(solTau)
```

    [1/2, 1]

```python
computedTau = solTau[0]
alpha_num = sol.subs({asymb: aval for asymb, aval in zip(a, avalue)})
alpha_num = alpha_num.subs({tau : computedTau})
alpha_num = alpha_num.subs({s : value_s0})
alpha_num_eval = [i.evalf() for i in alpha_num]
alpha_num_eval
```

    [-0.735758882342885, 0.367879441171442]

On retrouve logiquement les même valeurs de gain lorsque nous avons le couple $s_0$ et $\tau$ correspondant. Dans cet exemple on a soit fixé $\tau$ à $0.738$ et calculé $s_0$ qui est environ égal à $-1.5$ selon nos critères soit, nous avons fixé $s_0$ à $-1.5$ et le $\tau$ calculé est environ égal à $0.738$.

Le choix de fixer $s_0$ ou $\tau$ se fait selon si l'on connait le retard de commande ou si l'on souhaite fixer la vitesse de convergence $s_0$.

Ces hyperparamètres ($s_0$ ou $\tau$) dépend évidemment des coefficients du système $a_i$ et donc, selon les valeurs des coefficients, certaines valeurs d'hyperparamètres sont possibles ou non. Pour cela, on peut tracer le graphe d'admissibilité.

```python
polyAdm = SysDerivatif[-1].subs({alph : alphacoef for alph, alphacoef in zip(alpha, sol)})
polyAdm = polyAdm.subs({asymb: aval for asymb, aval in zip(a, avalue)})
sp.simplify(polyAdm)
```

$\displaystyle s^{2} \tau^{2} + s \tau^{2} + 4 s \tau + 2 \tau^{2} + 2 \tau + 2$

# Admissibilité

```python
s0range = np.arange(-10, 0, 0.01)
taurange = np.arange(0, 10, 0.01)

func = sp.lambdify([s, tau], polyAdm)

fig, ax = plt.subplots()
X, Y = np.meshgrid(s0range, taurange)
z = func(X, Y)
CS = ax.contour(X, Y, z, [0])
ax.grid()
plt.xlabel(r"$s_0$")
plt.ylabel(r"$\tau$")
plt.title("Plot Admissibilité")
```

    <IPython.core.display.Javascript object>

    Text(0.5, 1.0, 'Plot Admissibilité')

# Sensibilité en $\tau$

```python
tau_nominal = value_tau
step = 1e-2
nbIt = 10
```

```python
values = [0] + [-step * i for i in range(1, nbIt + 1)] + [step * i for i in range(1, nbIt + 1)]
```

```python
values.sort()
```

```python
tau_sens = []
s0_sens = []
for value in values :
    tau_sens.append(tau_nominal + value)
    s0_sens.append(solve_tau_connu(tau_sens[-1], [2, 1], Q))
```

```python
s0_sens
```

    [-1.85618056091116,
     -1.85062211334813,
     -1.84560644930057,
     -1.84111940350753,
     -1.83714828538220,
     -1.83368182726193,
     -1.83071014356143,
     -1.82822470069588,
     -1.82621829782361,
     -1.82468505865446,
     -1.82362043478586,
     -1.82302122127399,
     -1.82288558543080,
     -1.82321311017491,
     -1.82400485367054,
     -1.82526342748842,
     -1.82699309614456,
     -1.82919990165969,
     -1.83189181778737,
     -1.83507893986148,
     -1.83877371791975]

```python
def getRoots(m, Q, dominancy, delay, avalue, alphavalue, xwindow, ywindow):
    derivees = [Q, Q.diff(s)]
    for i in range(len(derivees)) :
        derivees[i] = derivees[i].subs({ai: ai_num for ai, ai_num in zip(a, avalue)})
        derivees[i] = derivees[i].subs({alphai: alphai_num for alphai, alphai_num in zip(alpha, alphavalue)})
        derivees[i] = derivees[i].subs({tau : delay})
    func = [sp.lambdify(s, i) for i in derivees]
    rect = cx.Rectangle(xwindow, ywindow)
    root_count = rect.count_roots(func[0])
    roots = rect.roots(func[0], func[1], rootErrTol=1e-5, absTol=1e-5, M=m+2)
    xroot, yroot = np.real(roots[0]), np.imag(roots[0])
    return xroot, yroot
```

```python
sensIterations = len(tau_sens)
bList = np.linspace(start=255, stop=0, num=sensIterations//2) #Blue to black
rList = np.linspace(start=0, stop=255, num=sensIterations//2) #black to red
if len(bList)+len(rList)==sensIterations-1 :
    zerosToFill = [0] * (sensIterations//2 + 1)
else :
    zerosToFill = [0] * (sensIterations//2 )
bList = [*bList, *zerosToFill]
rList = [*zerosToFill, *rList]

```

```python
colorArray = []
for i in range(len(tau_sens)):
    colorArray.append([rList[i], 0, bList[i]])
colorArray = np.array(colorArray)
```

```python
import matplotlib.cm as cm
scaler = lambda x : [i / 255 for i in x]
plt.figure()
for i in range(len(tau_sens)):
    xroot, yroot = getRoots(1, Q, s0_sens[i], tau_sens[i],
                            avalue, alpha_sens, [-5, 5], [-5, 5])
    plt.scatter(xroot, yroot, c=[scaler(colorArray[i])])
plt.grid()
plt.xlabel(r"$Re(s)$")
plt.ylabel(r"$Im(s)$")
tau_sens.sort()
plt.title(r"Sensitivity plot, $\tau \in $"+f"[{tau_sens[0]}, {tau_sens[-1]}]")
```

    <IPython.core.display.Javascript object>

    Text(0.5, 1.0, 'Sensitivity plot, $\\tau \\in $[0.638, 0.838]')


$$ \\ $$
2.1:
$$ \\ $$
When using an $\epsilon$-greedy action selection criterion, in the case of two actions and $\epsilon = 0.5$, actions would be selected at random half of the time.  Of the times when actions are selected at random, the greedy action will be select half of the time.  Meaning the greedy action will be selected a total of $75\%$ of the time.
$$ \\ $$
2.2:
$$ \\ $$
Given that all of the reward estimates are initialized to 0, the first action will be selected arbitrarily, possible at random.
$$ \\ $$
The following actions must have been selected at random as a result of the $\epsilon$ case: $A_2$, $A_5$.
$$ \\ $$
This can be demonstrated by observing that $Q_1(1) = 1 > Q_1(2) = 0$ and $Q_4(2) = \frac{R_2 + R_3 + R_4}{3} = \frac{5}{3} > Q_4(3) = 0$.
$$ \\ $$
Regarding the question of when random $\epsilon$ selection could have occured, it could have occured at any time step after the first.
$$ \\ $$
2.3:
$$ \\ $$
In the long run $\epsilon=0.01$ will achieve the highest cumulative reward and probability of selecting the correct answer.  The disadvantage of $\epsilon=0.01$ relative to $\epsilon=0.1$ is that it will cause our agent to take longer to find the optimal action, but it will select that action more consistently once it has been found.
$$ \\ $$
The optimal action has a reward of $1.55$. The expected reward per action once the agent has found the best possible action is equal to the sum of the greedy case and the $\epsilon$ random case:  $1.55 (1 - \epsilon) + \epsilon \sum^{10}_{i=1}\frac{q_*(i)}{10}$.
$$ \\ $$
So the difference between the expected long term reward per action in the $\epsilon = .01$ and $\epsilon = .1$ cases is $1.55(.1 - .01) = 0.1395$.
$$ \\ $$
2.4:
$$ \\ $$
If $\alpha$ is not constant, but instead varies with with each update step $n$, then our incremental update rule utilizing $\alpha_n$ can be expressed as follows:
$$ \\ $$
\begin{align}
Q_{n+1} & = Q_n + \alpha_n [R_n - Q_n] \\
& = \alpha_n R_n + (1 - \alpha_n) Q_n \\
& = \alpha_n R_n + (1 - \alpha_n) (\alpha_{n-1}R_{n-1} + (1 - \alpha_{n-1})Q_{n-1})  \\
& = \alpha_n R_n + (1 - \alpha_n)\alpha_{n-1} R_{n-1} + (1 - \alpha_n)(1 - \alpha_{n-1})(Q_{n-2} + \alpha_{n-2}[R_{n-2} - Q_{n-2}]) \\
& = \alpha_n R_n + (1 - \alpha_n)\alpha_{n-1} R_{n-1} + (1 - \alpha_n)(1 - \alpha_{n-1})((1 - \alpha_{n-2})Q_{n-2} + \alpha_{n-2}R_{n-2})) \\
& = \alpha_n R_n + (1 - \alpha_n)\alpha_{n-1} R_{n-1} + (1 - \alpha_n)(1 - \alpha_{n-1})\alpha_{n-2}R_{n-2} +  \\
& ...  +  Q_1 \prod_{i=1}^n(1 - \alpha_i) + \alpha_{1} R_1 \prod_{j = 1}^{n - 1} (1 - \alpha_{n - j + 1}) \\
& = Q_1 \prod_{i=1}^n(1 - \alpha_i) + \sum^{n}_{i = 1}\alpha_{i} R_i \prod_{j = 1}^{n - i} (1 - \alpha_{n - j + 1}) \end{align}
$$ \\ $$
2.5:

```python
%matplotlib inline

import numpy as np
import time
import random
import math
import numpy.random
from enum import Enum
import matplotlib.pyplot as plt

# EXERCISE 2.5

runs = 2000

methods = Enum("BanditMethod", "SAMPLE_AVERAGE EPSILON_GREEDY UCB GRADIENT_BANDIT OPTIMISTIC_GREEDY")

"""
returns data in the format

{
    "rewards": np.array([1.5, .2, ...]),
    "optimal": np.array([1, 0, ...])
}
"""
def ten_armed_bandit(steps, method = methods.EPSILON_GREEDY, constant_alpha = .1, Q0 = 0.0, c = 0, epsilon = .1):
    q = np.zeros(10, dtype = np.dtype("float64"))

    if method == methods.GRADIENT_BANDIT:
        average_reward = 0
        H = np.zeros(10, dtype = np.dtype("float64"))

    else:
        Q = np.array(list(map(lambda x : Q0, range(10))), dtype = np.dtype("float64"))

    N_a = np.zeros(10)

    rewards = []
    optimal = []

    for i in range(steps):
        arg_max_reward = None
        arg_max_actions = []

        if method == methods.GRADIENT_BANDIT:
            N_total = 0

            for H_a in H:
                N_total = N_total + math.e ** H_a

            prob_A = []

            for H_a in H:
                prob_A.append((math.e ** H_a) / N_total)

        for j in range(len(q)):
            if method == methods.UCB:
                if N_a[j] == 0:
                    arg_max_actions = [j]
                    break
                else:
                    estimated_reward = Q[j] + c * ((math.log(i + 1)/N_a[j])**.5)

            elif method == methods.GRADIENT_BANDIT:
                estimated_reward = prob_A[j]
            else:
                estimated_reward = Q[j]

            if arg_max_reward == None or estimated_reward > arg_max_reward:
                arg_max_reward = estimated_reward
                arg_max_actions = [j]

            elif estimated_reward == arg_max_reward:
                arg_max_actions.append(j)

        if (method in [methods.SAMPLE_AVERAGE, methods.EPSILON_GREEDY]) and random.uniform(0, 1) < epsilon:
            selected_action = random.choice(range(10))
        else:
            selected_action = random.choice(arg_max_actions)

        N_a[selected_action] = N_a[selected_action] + 1
        actual_reward = q[selected_action]
        was_action_optimal = 1 if actual_reward == max(q) else 0

        rewards.append(actual_reward)
        optimal.append(was_action_optimal)

        if method == methods.GRADIENT_BANDIT:
            average_reward = average_reward + (1/(i + 2))*(actual_reward - average_reward)

            for j in range(10):
                if j == selected_action:
                    H[j] = H[j] + constant_alpha * (actual_reward - average_reward) * (1 - prob_A[j])
                else:
                    H[j] = H[j] - constant_alpha * (actual_reward - average_reward) * prob_A[j]

        else:
            alpha = 1/(i + 1) if (method in [methods.SAMPLE_AVERAGE, methods.UCB]) else constant_alpha
            Q[selected_action] = Q[selected_action] + alpha * (actual_reward - Q[selected_action])

        random_walk_values = np.random.normal(0, .01, len(q))
        q = q + random_walk_values

    return {
        "rewards": np.array(rewards),
        "optimal": np.array(optimal)
    }

def average_bandit_results(sample_average = False, steps = 10000):
    rewards = np.zeros(steps)
    optimal = np.zeros(steps)
    method = methods.SAMPLE_AVERAGE if sample_average else methods.EPSILON_GREEDY

    for i in range(runs):
        results = ten_armed_bandit(steps, method = method)
        rewards = rewards + (results["rewards"] / runs)
        optimal = optimal + (results["optimal"] / runs)

    return {
        "rewards": rewards,
        "optimal": optimal
    }

def plot_average_rewards(with_sample_average, without_sample_average):
    plt.plot(with_sample_average, "r--", label = "sample average")
    plt.plot(without_sample_average, "g", label = "constant")
    plt.xlabel("steps")
    plt.ylabel("average reward")
    plt.legend()
    plt.show()

def plot_average_optimality(with_sample_average, without_sample_average):
    plt.plot(with_sample_average, "r--", label = "sample average")
    plt.plot(without_sample_average, "g", label = "constant")
    plt.xlabel("steps")
    plt.ylabel("percent optimal")
    plt.legend()
    plt.show()
```

```python
with_sample_average = average_bandit_results(True)
```

```python
without_sample_average = average_bandit_results()
```

```python
plot_average_rewards(with_sample_average["rewards"], without_sample_average["rewards"])
```

```python
plot_average_optimality(with_sample_average["optimal"], without_sample_average["optimal"])
```

We can observe above that the sample average method performs poorly for non-stationary reward distributions.

2.6:
$$ \\ $$
The optimistic greedy agent's initial spike in rewards can be explained as follows.
$$ \\ $$
$q_{*}(a)$ is normally distributed with a mean of $0$ and a variance of $1$.  This means $Q_1 > max\{q_{*}(a)\}$ in the vast majority of cases.
$$ \\ $$
For the reward distributions where the above is true, agents with the optimistic greedy action-value method will attempt every action at least once at the onset.
$$ \\ $$
The spike we observe is a product of agents selecting the optimal action once during this preliminary period.
$$ \\ $$
2.7:
$$ \\ $$
The following demonstrates that $Q_1$, the initial reward estimates, does not impact the step size $Q_n$ when using a step size coefficient,
$$ \beta_n = \frac{\alpha}{\overline{o}_n} $$
where,
$$ \overline{o}_n = \overline{o}_{n-1} + \alpha (1 - \overline{o}_{n-1}) $$
for $ n > 0$.  If you expand $\overline{o}_n$ you arrive at,
$$ \overline{o}_n = \alpha \sum_{i=1}^n (1 - \alpha)^{i-1} . $$
So,
$$ \beta_n = \frac{1}{\sum_{i=0}^n (1 - \alpha)^{i-1}} .$$
If we plug this coefficient into our solution for exercise 2.4,
$$ \\ $$
\begin{align}
Q_n & = Q_1 \prod_{i=1}^n (1 - \beta_i) + \sum^{n}_{i = 1}\beta_{i} R_i \prod_{j = 1}^{n - i} (1 - \beta_{n - j + 1}) \\
& = Q_1 \prod_{i=1}^n ( 1 - \frac{1}{\sum_{j=1}^i (1 - \alpha)^{j-1}} ) + ... \\
& = Q_1 (1 - \frac{1}{(1- \alpha)^{1-1}}) \prod_{i=2}^n ( 1 - \frac{1}{\sum_{j=1}^i (1 - \alpha)^{j-1}} ) + ... \\
& = Q_1 (0) + ... \\
& = \sum^{n}_{i = 1}\beta_{i} R_i \prod_{j = 1}^{n - i} (1 - \beta_{n - j + 1})
\end{align}
$$ \\ $$

So $Q_n$ is not biased by $Q_1$.

2.8:
$$ \\ $$
The spike observed on step 11 in figure 2.4 is a consequence of the $N_t(a)$ term in the UCB action selection criterion.  Up until $N_t(a) > 0$ for each action, UCB will behave similarly to the optimistic greedy criterion, selecting each action once before resuming its standard behavior.  This is a consequence of UCB treating each eaction as reward maximizing until it has been selected once.
$$ \\ $$
2.9:
$$ \\ $$
The softmax distribution with two actions consist of,
$$ \\ \pi_t(a) = \frac{e^{H_t(a)}}{e^{H_t(1)} + e^{H_t(2)}}. \\ $$
Logistic classifiers are binary classifiers.  In this case let $L(a)$ represent the probability assigned by a logistic classifier of selecting $a$, where $a \in \{ 1 , 2 \}$.  $L(2)$ is formulated as,
$$ \\ L(2) = \frac{1}{1+ e^{- f(2)} } \\ $$
where $f(a)$ is a function which accepts an action and outputs a preference score.  In the case of the softmax distribution, the probability of selection action $a=2$ is,
\begin{align}
\pi_t(2) &= \frac{e^{H_t(2)}}{e^{H_t(1)} + e^{H_t(2)}} \\
&= \frac{1}{1 + e^{H_t(1) - H_t(2)}}.
\end{align}
Now let $f(a)$ be a function which outputs a preference score such that $f(2) = -(H_t(1) - H_t(2))$, so $\pi_t(2)$ can be re-expressed as,
$$ \\ \pi_t(2) = \frac{1}{1 + e^{-f(2)}} = L(2). \\ $$
The above equality demonstrates the equivalence of logistic classifiers and the softmax distributions with two actions, or classes.
$$ \\ $$
2.10:
$$ \\ $$
We are presented with a 2-armed bandit problem, and our two actions vary between two sets of reward values for scenario $A$ and scenario $B$, $Q_A(1) = .1$, $Q_B(2) = .2$, and $Q_B(1) = .9$, $Q_B(2) = .8$, where both scenarios occur with a probability of $.5$.
$$ \\ $$
In the case that we are unable to determine what scenario we are in, the expected reward of each action is,
$$ Q(a) = \frac{Q_A(a) + Q_B(a)}{2} \\ $$
In this case $Q(1) = Q(2) = .5$. The expected rewards are the same for both action, meaning we can achieve the best possible result of $.5$ by picking actions arbitrarily, possible at random.
$$ \\ $$
In the case that we are told at each time step whether we are facing scenario A or scenario B, we should select action $a=2$ in scenario A, and action $a=1$ in scenario B. So the average reward at each time step will be,
$$ Q = \frac{Q_A(2) + Q_B(1)}{2} = .55 $$

```python
# EXERCISE 2.11

def compare_bandits_parameters(steps = 200000):
    # epsilon greedy varied over epsilon
    # ucb varied over c
    # gradient bandit varied over alpha
    # optimistic epsilon greedy varied over Q0
    average_range = int((steps / 5) - 1)
    epsilon_greedy = []
    optimistic_greedy = []
    ucb = []
    gradient_bandit = []

    for i in range(10):
        parameter_value = 2 ** (i - 7)

        if parameter_value < 1:
            epsilon_greedy.append(np.average(ten_armed_bandit(steps, epsilon = parameter_value)["rewards"][average_range:]))
        else:
            epsilon_greedy.append(0)

        optimistic_greedy.append(np.average(ten_armed_bandit(steps, Q0 = parameter_value, method = methods.OPTIMISTIC_GREEDY)["rewards"][average_range:]))

        ucb.append(np.average(ten_armed_bandit(steps, c = parameter_value, method = methods.UCB)["rewards"][average_range:]))

        gradient_bandit.append(np.average(ten_armed_bandit(steps, constant_alpha = parameter_value, method = methods.GRADIENT_BANDIT)["rewards"][average_range:]))

    return {
        "epsilon_greedy": np.array(epsilon_greedy),
        "optimistic_greedy": np.array(optimistic_greedy),
        "ucb": np.array(ucb),
        "gradient_bandit": np.array(gradient_bandit),
    }

def plot_bandits_parameter_comparison(comparison):
    parameter_values = list(map(lambda i: 2 ** (i - 7), range(10)))
    plt.plot(parameter_values, comparison["epsilon_greedy"], "r", label = "epsilon greedy")
    plt.plot(parameter_values, comparison["ucb"], "b", label = "ucb")
    plt.plot(parameter_values, comparison["optimistic_greedy"], "g", label = "optimistic greedy")
    plt.plot(parameter_values, comparison["gradient_bandit"], "k", label = "gradient bandit")
    plt.xscale('log')
    plt.xlabel("Parameter values: epsilon, c, alpha, Q0")
    plt.ylabel("Average reward")
    plt.legend()
    plt.show()

```

```python
bandits_parameter_comparison = compare_bandits_parameters(200000)
```

```python
plot_bandits_parameter_comparison(bandits_parameter_comparison)
```

As expected the UCB method performs poorly for non-stationary reward distributions, and epsilon greedy and gradient bandit perform the best.

```python

```


# NUME

$\newcommand{\e}[1]{\mathrm{e}^{#1}}$

**Métodos Numéricos**

Instructor: Dr. Keny ORDAZ

Contenido:
==========

1. Representación de números en punto flotante
2. Errores por redondeo y precisión de la máquina
3. Operaciones con números en punto flotante
4. Operaciones con matrices: niveles 1, 2 y 3 en BLAS
http://numerical.recipes/cpppages/chap0sel.pdf
http://www.netlib.org/blas/#_level_1

5. Descomposición LU
https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.lu.html

6. Factorización de matrices: Cholesky y QR
https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.cholesky.html?highlight=cholesky#scipy.linalg.cholesky
https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.qr.html?highlight=qr#scipy.linalg.qr

7. Mínimos cuadrados y SVD
https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.svd.html?highlight=svd#scipy.linalg.svd
https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.lstsq.html#scipy.linalg.lstsq

8. Gradiente conjugado lineal
https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.cg.html

https://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf
https://en.wikipedia.org/wiki/Conjugate_gradient_methodnewto

9. Método de Newton-Raphson (root-finding algorithm )
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.newton.html
https://www.math.ubc.ca/~pwalls/math-python/roots-optimization/newton/#implementation

10. Método de Newton inexacto ??
https://www.sciencedirect.com/science/article/pii/S0377042705007958

11. Método secante (cuasi-Newton)
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.newton.html
"The Newton-Raphson method is used if the derivative fprime of func is provided, otherwise the secant method is used. If the second order derivative fprime2 of func is also provided, then Halley’s method is used."

Optimización
<https://www.fio.unicen.edu.ar/usuario/cgely/q13-0/Apuntes/unidad4b.pdf>

12. Búsqueda lineal de primer orden: Método del gradiente
<https://en.wikipedia.org/wiki/Line_search>

13. Búsqueda lineal de segundo orden: Método de Newton
14. Diferencias finitas: izquierda, derecha y central: <https://pythonnumericalmethods.berkeley.edu/notebooks/chapter23.03-Finite-Difference-Method.html>

15. Integración de ecuaciones diferenciales: Método de Euler
https://en.wikipedia.org/wiki/Euler_method ODEs

16. Integración de ecuaciones diferenciales: Método de Runge-Kutta
https://en.wikipedia.org/wiki/Runge–Kutta_methods

## Recursos en línea

### Libros

1. First Semester in Numerical Analysis with Python
    <http://digital.auraria.edu/content/IR/00/00/01/95/00001/NumAnPython_Aug_09_2020.pdf>

2. Python Programming And Numerical Methods: A Guide For Engineers And Scientists
    <https://pythonnumericalmethods.berkeley.edu/notebooks/Index.html>

### Cursos y Artículos

1. Numerical Computing in Python <https://nbviewer.jupyter.org/github/phelps-sg/python-bigdata/blob/master/src/main/ipynb/numerical-slides.ipynb>

<https://github.com/phelps-sg/python-bigdata/blob/master/src/main/ipynb/numerical-slides.ipynb>

1. Floating Point Arithmetic: Issues and Limitations <https://docs.python.org/3/tutorial/floatingpoint.html>

2. Practical <https://github.com/numerical-mooc/numerical-mooc>

3. CSC231 An Introduction to Fixed- and Floating-Point Numbers <http://dominiquethiebaut.com/dftwiki3/index.php/CSC231_An_Introduction_to_Fixed-_and_Floating-Point_Numbers#Range_of_Floating-Point_Numbers>

4. Is Your Model Susceptible to Floating-Point Errors? <http://jasss.soc.surrey.ac.uk/9/4/4.html>

```python
.1 + .1 + .1
```

    0.30000000000000004

```python
x = 0.1
Decimal.from_float(x)
```

    Decimal('0.1000000000000000055511151231257827021181583404541015625')

```python
tol = 1E-9
import math
math.isclose(.1 + .1 + .1, 0.3, rel_tol=tol, abs_tol=tol)
```

    True

```python
help(math.isclose)
```

    Help on built-in function isclose in module math:

    isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0)
        Determine whether two floating point numbers are close in value.

          rel_tol
            maximum difference for being considered "close", relative to the
            magnitude of the input values
          abs_tol
            maximum difference for being considered "close", regardless of the
            magnitude of the input values

        Return True if a is close in value to b, and False otherwise.

        For the values to be considered close, the difference between them
        must be smaller than at least one of the tolerances.

        -inf, inf and NaN behave similarly to the IEEE 754 Standard.  That
        is, NaN is not close to anything, even itself.  inf and -inf are
        only close to themselves.

```python
from decimal import Decimal
```

```python
x = 1/3
x.hex()
```

    '0x1.5555555555555p-2'

```python
x.as_integer_ratio()
```

    (6004799503160661, 18014398509481984)

```python
(x, type(x))
```

    (0.3333333333333333, float)

```python
print(f'x: {x:0.13f}') # 'sencillas' "doble"
```

    x: 0.3333333333333

```python
Decimal.from_float(x)
```

    Decimal('0.333333333333333314829616256247390992939472198486328125')

```python
print(0.3); Decimal.from_float(0.3)
```

    0.3

    Decimal('0.299999999999999988897769753748434595763683319091796875')

### Calcular la aproximación de $\e{-7}$

$$\e{-7} = 1 + \frac{-7}{1} + \frac{(-7)^2}{2!} + \frac{(-7)^3}{3!} + \ldots +\frac{(-7)^n}{n!}\;.$$

$$\exp{x} = \e{x} = \sum_{k=0}^{n} {\frac{x^k}{k!}}\;.$$

```python
import math

math.exp( -7 )
```

    0.0009118819655545162

```python
import numpy as np
from scipy.special import factorial

def exp_naive(x, n=20):
    sum = 1.0
    for k in range(1, n + 1): # 1, 2, ..., n
        sum += (x ** k) / factorial(k)
    return sum
```

```python
exp_naive(-7)
```

    0.009183673977218275

```python
abs(_2 - _4) / abs(_2)
```

    9.071121399614121

El error relativo es $\approx 9.1$. Es posible disminuir el error si simplemente rescribimos la suma como:

$$\e{-7} = \frac{1}{\e{7}} = \frac{1}{1 + \frac{7}{1} + \frac{(7)^2}{2!} + \frac{(7)^3}{3!} + \ldots +\frac{(7)^n}{n!}}\;.$$

```python
def exp_improv(x, n=20):
    isneg = x < 0.0
    if isneg:
        x = abs(x)
    sum = 1.0
    for k in range(1, n + 1): # 1, 2, ..., n
        sum += (x ** k) / factorial(k)
    if isneg:
        sum = 1 / sum
    return sum
```

```python
exp_improv(-7)
```

    0.0009118951837867185

```python
abs(_2 - _8) / abs(_2)
```

    1.4495551728807734e-05

En este caso, el error relativo es $\approx 1.4 \times 10^{-5}$.

## Ejercicio 1.3-5

Considerar el polinomio escrito en forma estándar: $$5 x^4 + 3 x^3 + 4 x^2 + 7 x - 5$$.

1. Escribir el polinomio en su forma anidada.

2. ¿Cuántas multiplicaciones requiere la forma anidada cuando evaluamos el polinomio en un número real? ¿Cuántas multiplicaciones requiere la forma estándar? ¿Es posible generalizar la respuesta para cualquier polinomio de grado $n$?

```python
import sympy as sp
x = sp.symbols('x')
forma_est = 5 * x ** 4 + 3 * x ** 3 + 4 * x ** 2 + 7 * x - 5
forma_est
forma_ani = (((5 * x + 3 ) * x + 4) * x + 7) * x - 5
forma_ani
```

$\displaystyle x \left(x \left(x \left(5 x + 3\right) + 4\right) + 7\right) - 5$

```python
sp.expand(forma_ani) == forma_est
```

    True

```python
sp.srepr(forma_ani)
```

    "Add(Mul(Symbol('x'), Add(Mul(Symbol('x'), Add(Mul(Symbol('x'), Add(Mul(Integer(5), Symbol('x')), Integer(3))), Integer(4))), Integer(7))), Integer(-5))"

```python
_26.count('Mul'), _26.count('Pow')
```

    (4, 0)

```python
sp.srepr(forma_est)
```

    "Add(Mul(Integer(5), Pow(Symbol('x'), Integer(4))), Mul(Integer(3), Pow(Symbol('x'), Integer(3))), Mul(Integer(4), Pow(Symbol('x'), Integer(2))), Mul(Integer(7), Symbol('x')), Integer(-5))"

```python
_30.count('Mul'), _30.count('Pow')
```

    (4, 3)

```python

```


The Monty Hall Problem
=====

####Introduction
Thinking _conditionally_ is a fundamental ingredient in Bayesian statistics. When applied correctly it can give rise to surprising outcomes and might result in competitetive advantages. This is the case in the classic Monty Hall problem.

Monty Hall?
------

Monty Hall was a game show host in the 70's(?) and gave his participants the following choice

> There are 3 doors. Behind one door is a car, behind the two others are goats. The participant is allowed to choose a door. Monty Hall then open one of the doors that has not been chosen, thus revealing a goat (he obviously won't open the car door). The participant is then given the opportunity to switch doors.

Without deep conditional thinking one might suspect that the opportunity of a second choice does not matter. However, is that true, or might we gain a competitetive advantage by observing the door Monty opens?

Before we give a more mathematical answer, let's investigate the problem numerically.

Let's play the game over and over and over again
------

To get a statistical sense of the success distribution we simulate the game repeatedly and collect the results.

####Python imports

```python
import matplotlib
import matplotlib.pyplot as plt
import random as rd
import numpy as np
from numpy.random import choice

%matplotlib inline
matplotlib.style.use('ggplot')
matplotlib.rc_params_from_file("../styles/matplotlibrc" ).update()
```

Let's first define some helper function for chosing the doors behind which the car lives and which the contestant chooses

```python
def carBehindDoor(prob = [1./3, 1./3, 1./3]):
    list_of_doors = ["A", "B", "C"]
    return str(choice(list_of_doors, p = prob))

def contestantChoosesRandomDoor(prob = [1./3, 1./3, 1./3]):
    list_of_doors = ["A", "B", "C"]
    return str(choice(list_of_doors, p = prob))
```

As a result of these choices, Monty opens another door

```python
def montyOpensDoor(contestantChoice, carBehindDoor, pref_monty = [0.5, 0.5]):
    list_of_doors = ["A", "B", "C"]

    # Monty will not open the car door, so we remove it
    list_of_doors.remove(carBehindDoor)

    # If the contestant chooses the door which contains the car,
    # Monty can choose which door to open
    if contestantChoice is carBehindDoor:
        return str(choice(list_of_doors, p = pref_monty))

    # If the car and contestant door do not coincide,
    # Monty has only a single choice
    elif contestantChoice is list_of_doors[0]:
        return list_of_doors[1]
    else:
        return list_of_doors[0]
```

Now we have all the moves of the game ready. We only need to count how often the contestant wins if he switches.

```python
def successUponSwitch(car_door, contestant_door, monty_door):
    list_of_doors = ["A", "B", "C"]

    # Remove Monty's door
    list_of_doors.remove(monty_door)

    # The contestant switches, so we remove his door
    list_of_doors.remove(contestant_door)

    # check if final door is the door with the car
    if list_of_doors[0] == car_door:
        return True
    elif list_of_doors[0] != car_door:
        return False
```

Let's string together the experiment

```python
def runExperiment(n_trials):
    success = 0
    for k in range(n_trials):
        car_behind_door = carBehindDoor()
        contestant_door = contestantChoosesRandomDoor()
        monty_door = montyOpensDoor(contestant_door, car_behind_door)
        if successUponSwitch(car_behind_door, contestant_door, monty_door):
            success += 1
    return float(success)/float(n_trials)
```

To get a reasonable estimate we run the experiment $10000$ times and obtain the success probability as

```python
print "We succeed with a probability: %0.3f" % runExperiment(10000)
```

    We succeed with a probability: 0.669

We win in about $2/3$ of the cases and hence have a distinct advantage over Monty by observing which door he opens. On average switching is good and get's us a car.

Mathematical reasoning
------

To understand the outcome we use conditional probabilities. To simplify notations we denote the doors now by numbers $1, 2, 3$ and denote the event of finding the car behind door $j$ as $C_j$, Monty opening door $j$ as $M_j$ and the participant choosing door $j$ as $X_j$. Finally we call the event of success after switching doors $S$. We look for $P(S)$.

To gain some intuition let's look at a decision tree

 Source: [Wikimedia Commons](http://upload.wikimedia.org/wikipedia/commons/9/9e/Monty_tree.svg)

It now becomes very obvious that switching the doors after observing Monty's door yields a better outcome for us! To reason through the problem we observe that the problem is completely symmetrical with respect to permutations of the doors. So we assume that the car is behind door $3$. Making use of the [law of total probability](http://en.wikipedia.org/wiki/Law_of_total_probability) and _conditioning on opening a specific door_ we can easily write
\begin{align}
P(S) = P(S|X_1) P(X_1) + P(S|X_2) P(X_2) + P(S|X_3) P(X_3) = 1 \cdot \frac{1}{3} + 1 \cdot \frac{1}{3} + 0 = \frac{2}{3}
\end{align}
which is the result we found numerically. This is easily verified by noting that if we choose door $3$ we will never find the car, whereas in the other two cases switching will always win.

Note that this is completely independent of Monty's choice. No matter what he does in 2 out of 3 cases we win the car! Of course in reality we will not make the choice independent of Monty ... he will try to keep us away from switching, by making us doubt.

###Dependence on Monty's preference for a door

We saw that observing Monty opening a door does not give us any additional information about where the car might be if we ask for the _total_ probability of success. This sounds surprising! Maybe we might gain even more advantage if we knew something about Monty's preferences for opening a door? Let's run some experiments. Assume Monty is always standing next to door $A$ and opens the closest goat-door with probability $p$.

Stringing the experiment together we get:

```python
def runExperiment(n_trials, p_monty):
    success = 0
    for k in range(n_trials):
        car_behind_door = carBehindDoor()
        contestant_door = contestantChoosesRandomDoor()
        monty_door = montyOpensDoor(contestant_door, car_behind_door, pref_monty=[p_monty, 1-p_monty])
        if successUponSwitch(car_behind_door, contestant_door, monty_door):
            success += 1
    return float(success)/float(n_trials)
```

```python
p_monty = 0.9
print "We succeed with a probability: %0.3f" % runExperiment(10000, p_monty)
```

    We succeed with a probability: 0.660

Indeed! Monty's preferences do not play a role in the grand scheme of things. But are we asking the right question here?

Lazy Monty and conditional probability
------

What we should really be asking is the question of how often we succeed _given that Monty opens a specific door_, i.e.
\begin{align}
P(S | M_2) = P(C_3 | M_2)
\end{align}

Using the definition of conditional probability this can be re-expressed as
\begin{align}
P(C_3 | M_2) = \frac{P(C_3\cap M_2)}{P(M_2)} = \frac{P(S\cap M_2)}{P(M_2)}
\end{align}

Let's simulate this!

What we need is the count of cases where we won _and_ Monty opened a specific door

```python
def runConditionalExperiment(n_trials, observed_monty_door, p_monty):
    cond_success = 0
    count_monty_observerd_door = 0
    for k in range(n_trials):
        car_behind_door = 'C'
        contestant_door = contestantChoosesRandomDoor()
        monty_door = montyOpensDoor(contestant_door, car_behind_door, pref_monty=[p_monty, 1-p_monty])

        if monty_door == observed_monty_door:
            count_monty_observerd_door += 1

        if successUponSwitch(car_behind_door, contestant_door, monty_door) and monty_door == observed_monty_door:
            cond_success += 1
    return float(cond_success)/float(count_monty_observerd_door)
```

Note that in the return argument we returned the ratio between the counts and not the probabilities. This is correct since we get the probabilities by normalizing with the number of trials. However, this normalization drops out when calculating the ratio.

What do we get?

```python
p_monty = .75
print "We succeed with a probability: %0.3f" % runConditionalExperiment(10000, 'B', p_monty)
```

    We succeed with a probability: 0.802

Surprise! We do get some additional advantage if we ask the question what our success rate is _given Monty opens a specific door_. How does this probability depend on his preferences?

```python
p_monty = np.linspace(0, 1, 100)
cond_probabilities = []
for prob in p_monty:
    cond_probabilities.append(runConditionalExperiment(10000, 'B', prob))
```

```python
plt.plot(p_monty, cond_probabilities)
```

This is quite surprising! If Monty never opens door $2$ we have no advantage and win the car only half of the time. However, if Monty opens door $2$ always, we are guaranteed to win the car every single time!

To understand this in more formal terms we make use of [Bayes rule](http://en.wikipedia.org/wiki/Bayes%27_theorem) and find
\begin{align}
P(C_3 | M_2) \,=\, \frac{P(M_2 | C_3) P(C_3)}{P(M_2)}
\end{align}

Let's look at the different probabilities individually:

- $P(M_2 | C_3)=1$, since Monty is forced to open door $2$ in this constellation (the contestant has to choose door $1$ or the strategy fails)
- $P(C_j) = 1/3$, since the contestant has no prior knowledge of where the car is
- $P(M_2)$ is slightly more complicated. To simplify this we expand this into disjoint cases using the [law of total probability](http://en.wikipedia.org/wiki/Law_of_total_probability). We then have (given the contestant chooses door $1$):
\begin{align}
P(M_2) = P(M_2|C_1) P(C_1) + P(M_2|C_2) P(C_2) + P(M_2|C_3) P(C_3)
 = (1-p) \cdot \frac{1}{3} + 0 + 1\cdot \frac{1}{3}
\end{align}

Putting this all together we find
\begin{align}
P(C_3 | M_2) \,=\, \frac{1}{2-p}
\end{align}
which is exactly what we found in the graph above! Note that in the symmetric case $p=0.5$ we obtain the result of total probability, meaning that at this point the events are independent.

Summary
------

The Monty Hall problem has several surprising and non-intuitive twists. First of all, we can come up with a dominant stragegy that always beats Monty, by simply switching the doors independent of Monty's choice. This is surprising as naively one would think the chances are equal. Second, when conditioning on Monty's behavior, we do see a dependence of our success strategy on Monty's preference of doors. If we stick with our original choice always, then Monty can come up with a strategy that will equalize our chances.

Enjoy this notebook and have a lot of fun playing some more with it, such as exploring different amount of doors or a different strategy. Contributions are more than welcome!

###Miscellenea

####References
are given throughout the text

####Stylesheet

```python
from IPython.core.display import HTML

def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunss.otf');
    }
    @font-face {
        font-family: "Computer Modern";
        font-weight: bold;
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunsx.otf');
    }
    @font-face {
        font-family: "Computer Modern";
        font-style: oblique;
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunsi.otf');
    }
    @font-face {
        font-family: "Computer Modern";
        font-weight: bold;
        font-style: oblique;
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunso.otf');
    }
    div.cell{
        width:800px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: Helvetica, serif;
    }
    h4{
        margin-top:12px;
        margin-bottom: 3px;
       }
    div.text_cell_render{
        font-family: Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 145%;
        font-size: 130%;
        width:800px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro", source-code-pro,Consolas, monospace;
    }
    .prompt{
        display: None;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 22pt;
        color: #4057A1;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>

```python

```

Numerical-Experimentation/The Monty Hall Problem.ipynb


# An Introduction to Bayesian Statistical Analysis

Before we jump in to model-building and using MCMC to do wonderful things, it is useful to understand a few of the theoretical underpinnings of the Bayesian statistical paradigm. A little theory (and I do mean a *little*) goes a long way towards being able to apply the methods correctly and effectively.

## What *is* Bayesian Statistical Analysis?

Though many of you will have taken a statistics course or two during your undergraduate (or graduate) education, most of those who have will likely not have had a course in *Bayesian* statistics. Most introductory courses, particularly for non-statisticians, still do not cover Bayesian methods at all, except perhaps to derive Bayes' formula as a trivial rearrangement of the definition of conditional probability. Even today, Bayesian courses are typically tacked onto the curriculum, rather than being integrated into the program.

In fact, Bayesian statistics is not just a particular method, or even a class of methods; it is an entirely different paradigm for doing statistical analysis.

> Practical methods for making inferences from data using probability models for quantities we observe and about which we wish to learn.
*-- Gelman et al. 2013*

A Bayesian model is described by parameters, uncertainty in those parameters is described using probability distributions.

All conclusions from Bayesian statistical procedures are stated in terms of *probability statements*

This confers several benefits to the analyst, including:

- ease of interpretation, summarization of uncertainty
- can incorporate uncertainty in parent parameters
- easy to calculate summary statistics

## Bayesian vs Frequentist Statistics: What's the difference?

Any statistical paradigm, Bayesian or otherwise, involves at least the following:

1. Some **unknown quantities** about which we are interested in learning or testing. We call these *parameters*.
2. Some **data** which have been observed, and hopefully contain information about (1).
3. One or more **models** that relate the data to the parameters, and is the instrument that is used to learn.

### The Frequentist World View

- The data that have been observed are considered **random**, because they are realizations of random processes, and hence will vary each time one goes to observe the system.
- Model parameters are considered **fixed**. The parameters' values are unknown, but they are fixed, and so we *condition* on them.

In mathematical notation, this implies a (very) general model of the following form:

$$f(y | \theta)$$

Here, the model $f$ accepts data values $y$ as an argument, conditional on particular values of $\theta$.

Frequentist inference typically involves deriving **estimators** for the unknown parameters. Estimators are formulae that return estimates for particular estimands, as a function of data. They are selected based on some chosen optimality criterion, such as *unbiasedness*, *variance minimization*, or *efficiency*.

> For example, lets say that we have collected some data on the prevalence of autism spectrum disorder (ASD) in some defined population. Our sample includes $n$ sampled children, $y$ of them having been diagnosed with autism. A frequentist estimator of the prevalence $p$ is:

> $$\hat{p} = \frac{y}{n}$$

> Why this particular function? Because it can be shown to be unbiased and minimum-variance.

It is important to note that new estimators need to be derived for every estimand that is introduced.

### The Bayesian World View

- Data are considered **fixed**. They used to be random, but once they were written into your lab notebook/spreadsheet/Jupyter notebook they do not change.
- Model parameters themselves may not be random, but Bayesians use probability distribtutions to describe their uncertainty in parameter values, and are therefore treated as **random**. In some cases, it is useful to consider parameters as having been sampled from probability distributions.

This implies the following form:

$$p(\theta | y)$$

This formulation used to be referred to as ***inverse probability***, because it infers from observations to parameters, or from effects to causes.

Bayesians do not seek new estimators for every estimation problem they encounter. There is only one estimator for Bayesian inference: **Bayes' Formula**.

# Computational Methods in Bayesian Analysis

The process of conducting Bayesian inference can be broken down into three general steps (Gelman *et al.* 2013):

### Step 1: Specify a probability model

As was noted above, Bayesian statistics involves using probability models to solve problems. So, the first task is to *completely specify* the model in terms of probability distributions. This includes everything: unknown parameters, data, covariates, missing data, predictions. All must be assigned some probability density.

This step involves making choices.

- what is the form of the sampling distribution of the data?
- what form best describes our uncertainty in the unknown parameters?

### Step 2: Calculate a posterior distribution

The mathematical form $p(\theta | y)$ that we associated with the Bayesian approach is referred to as a **posterior distribution**.

> posterior /pos·ter·i·or/ (pos-tēr´e-er) later in time; subsequent.

Why posterior? Because it tells us what we know about the unknown $\theta$ *after* having observed $y$.

This posterior distribution is formulated as a function of the probability model that was specified in Step 1. Usually, we can write it down but we cannot calculate it analytically. In fact, the difficulty inherent in calculating the posterior distribution for most models of interest is perhaps the major contributing factor for the lack of widespread adoption of Bayesian methods for data analysis. Various strategies for doing so comprise this tutorial.

**But**, once the posterior distribution is calculated, you get a lot for free:

- point estimates
- credible intervals
- quantiles
- predictions

### Step 3: Check your model

Though frequently ignored in practice, it is critical that the model and its outputs be assessed before using the outputs for inference. Models are specified based on assumptions that are largely unverifiable, so the least we can do is examine the output in detail, relative to the specified model and the data that were used to fit the model.

Specifically, we must ask:

- does the model fit data?
- are the conclusions reasonable?
- are the outputs sensitive to changes in model structure?

## Example: binomial calculation

Binomial model is suitable for data that are generated from a sequence of exchangeable Bernoulli trials. These data can be summarized by $y$, the number of times the event of interest occurs, and $n$, the total number of trials. The model parameter is the expected proportion of trials that an event occurs.

$$p(Y|\theta) = \frac{n!}{y! (n-y)!} \theta^{y} (1-\theta)^{n-y}$$

where $y \in \{0, 1, \ldots, n\}$ and $p \in [0, 1]$.

To perform Bayesian inference, we require the specification of a prior distribution. A reasonable choice is a uniform prior on [0,1] which has two implications:

1. makes all probability values equally probable *a priori*
2. makes calculation of the posterior easy

The second task in performing Bayesian inference is, given a fully-specified model, to calculate a posterior distribution. As we have specified the model, we can calculate a posterior distribution up to a proportionality constant (that is, a probability distribution that is **unnormalized**):

$$P(\theta | n, y) \propto P(y | n, \theta) P(\theta) = \theta^y (1-\theta)^{n-y}$$

We can present different posterior distributions as a function of different realized data.

We can also calculate posterior estimates for $\theta$ by maximizing the unnormalized posterior using optimization.

For example, let's plot this unnormalized posterior for data comprising of 3 observed events from 3 trials:

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

p_range = np.linspace(0, 1, num=100, endpoint=False)

def binomial_posterior(n, y):
    plt.plot(p_range, (p_range**y) * (1 - p_range)**(n-y))
    plt.xlabel('p');
```

```python
binomial_posterior(n=5, y=3)
```

What we see here is a function that characterizes the relatve evidence for different values of $p$, conditional on the assumed binomial model (with a uniform prior) and the observed data. Since the data are small, there is still reasonable support across the (0,1) range.

Let's increase the sample size to 12 events from 20 trials:

```python
binomial_posterior(n=20, y=12)
```

And again to 1200 observations. Note that for this contrived example, the proportion of observed events is the same across sample sizes.

```python
binomial_posterior(n=750, y=450)
```

So as the sample size gets large, the posterior narrows, representing the reduction in residual uncertainty. Notice also that the posterior distributions start to look more and more Gaussian!

# Evaluating Hypotheses with Bayes

Statistical inference is a process of learning from incomplete or imperfect (error-contaminated) data. Can account for this "imperfection" using either a sampling model or a measurement error model.

### Statistical hypothesis testing

The *de facto* standard for statistical inference is statistical hypothesis testing. The goal of hypothesis testing is to evaluate a **null hypothesis**. There are two possible outcomes:

- reject the null hypothesis
- fail to reject the null hypothesis

Rejection occurs when a chosen test statistic is higher than some pre-specified threshold valuel; non-rejection occurs otherwise.

Notice that neither outcome says anything about the quantity of interest, the **research hypothesis**.

Setting up a statistical test involves several subjective choices by the user that are rarely justified based on the problem or decision at hand:

- statistical test to use
- null hypothesis to test
- significance level

Choices are often based on arbitrary criteria, including "statistical tradition" (Johnson 1999). The resulting evidence is indirect, incomplete, and typically overstates the evidence against the null hypothesis (Goodman 1999).

Most importantly to applied users, the results of statistical hypothesis tests are very easy to misinterpret.

### Estimation

Instead of testing, a more informative and effective approach for inference is based on **estimation** (be it frequentist or Bayesian). That is, rather than testing whether two groups are different, we instead pursue an estimate of *how different* they are, which is fundamentally more informative.

Additionally, we include an estimate of **uncertainty** associated with that difference which includes uncertainty due to our lack of knowledge of the model parameters (*epistemic uncertainty*) and uncertainty due to the inherent stochasticity of the system (*aleatory uncertainty*).

## One Group

Before we compare two groups using Bayesian analysis, let's start with an even simpler scenario: statistical inference for one group.

For this we will use Gelman et al.'s (2007) radon dataset. In this dataset the amount of the radioactive gas radon has been measured among different households in all counties of several states. Radon gas is known to be the highest cause of lung cancer in non-smokers. It is believed to be more strongly present in households containing a basement and to differ in amount present among types of soil.

>  the US EPA has set an action level of 4 pCi/L. At or above this level of radon, the EPA recommends you take corrective measures to reduce your exposure to radon gas.

Let's import the dataset:

```python
import pandas as pd
import seaborn as sns
sns.set_context('notebook')

RANDOM_SEED = 20090425
```

```python
radon = pd.read_csv('../data/radon.csv', index_col=0)
radon.head()
```

      idnum
      state
      state2
      stfips
      zip
      region
      typebldg
      floor
      room
      basement
      ...
      pcterr
      adjwt
      dupflag
      zipflag
      cntyfips
      county
      fips
      Uppm
      county_code
      log_radon

      0
      5081.0
      MN
      MN
      27.0
      55735
      5.0
      1.0
      1.0
      3.0
      N
      ...
      9.7
      1146.499190
      1.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      0.832909

      1
      5082.0
      MN
      MN
      27.0
      55748
      5.0
      1.0
      0.0
      4.0
      Y
      ...
      14.5
      471.366223
      0.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      0.832909

      2
      5083.0
      MN
      MN
      27.0
      55748
      5.0
      1.0
      0.0
      4.0
      Y
      ...
      9.6
      433.316718
      0.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      1.098612

      3
      5084.0
      MN
      MN
      27.0
      56469
      5.0
      1.0
      0.0
      4.0
      Y
      ...
      24.3
      461.623670
      0.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      0.095310

      4
      5085.0
      MN
      MN
      27.0
      55011
      3.0
      1.0
      0.0
      4.0
      Y
      ...
      13.8
      433.316718
      0.0
      0.0
      3.0
      ANOKA
      27003.0
      0.428565
      1
      1.163151

5 rows × 29 columns

Let's focus on the (log) radon levels measured in a single county (Hennepin).

Suppose we are interested in:

- whether the mean log-radon value is greater than 4 pCi/L in Hennepin county
- the probability that any randomly-chosen household in Hennepin county has a reading of greater than 4

```python
hennepin_radon = radon.query('county=="HENNEPIN"').log_radon
sns.distplot(hennepin_radon);
```

### The model

Recall that the first step in Bayesian inference is specifying a **full probability model** for the problem.

This consists of:

- a likelihood function(s) for the observations
- priors for all unknown quantities

The measurements look approximately normal, so let's start by assuming a normal distribution as the sampling distribution (likelihood) for the data.

$$y_i \sim N(\mu, \sigma^2)$$

(don't worry, we can evaluate this assumption)

This implies that we have 2 unknowns in the model; the mean and standard deviation of the distribution.

#### Prior choice

How do we choose distributions to use as priors for these parameters?

There are several considerations:

- discrete vs continuous values
- the support of the variable
- the available prior information

While there may likely be prior information about the distribution of radon values, we will assume no prior knowledge, and specify a **diffuse** prior for each parameter.

Since the mean can take any real value (since it is on the log scale), we will use another normal distribution here, and specify a large variance to allow the possibility of very large or very small values:

$$\mu \sim N(0, 10^2)$$

For the standard deviation, we know that the true value must be positive (no negative variances!). I will choose a uniform prior bounded from below at zero and from above at a value that is sure to be higher than any plausible value the true standard deviation (on the log scale) could take.

$$\sigma \sim U(0, 10)$$

We can encode these in a Python model, using the PyMC3 package, as follows:

```python
from pymc3 import Model, Normal, Uniform

with Model() as radon_model:

    μ = Normal('μ', mu=0, sigma=5)
    σ = Uniform('σ', 0, 5)
```

    WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.

> ## Software
> Today there is an array of software choices for Bayesians, including both open source software (*e.g.*, Stan, PyMC, Pyro, TensorFlow Probability) and commercial (*e.g.*, SAS, Stata). These examples can be replicated in any of these environments.

All that remains is to add the likelihood, which takes $\mu$ and $\sigma$ as parameters, and the log-radon values as the set of observations:

```python
with radon_model:

    dist = Normal('dist', mu=μ, sigma=σ, observed=hennepin_radon)
```

Before we go ahead and estimate the model paramters from the data, it's a good idea to perform a **prior predictive check**. This involves sampling from the model before data are incorporated, and gives you an idea of the range of observations that would be considered reasonable within the scope of the modeling assumptions (including choice of priors). If the simnulations generate too many extreme observations relative to our expectations based on domain knowledge, then it can be an indication of problems with model formulation.

```python
from pymc3 import sample_prior_predictive

with radon_model:

    prior_sample = sample_prior_predictive(1000)
```

```python
plt.hist(prior_sample['dist'].ravel(), bins=30);
```

```python
plt.hist(radon.log_radon, bins=30);
```

Now, we will fit the model using **Markov chain Monte Carlo (MCMC)**, which will be covered in detail in an upcoming section. This will draw samples from the posterior distribution (which cannot be calculated exactly).

```python
from pymc3 import sample

with radon_model:

    samples = sample(1000, tune=1000, cores=2, random_seed=RANDOM_SEED)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [σ, μ]

  <progress value='0' class='' max='4000' style='width:300px; height:20px; vertical-align: middle;'></progress>

    Sampling 2 chains for 1_000 tune and 1_000 draw iterations (2_000 + 2_000 draws total) took 8 seconds.

```python
from arviz import plot_posterior

plot_posterior(samples, var_names=['μ'], ref_val=np.log(4), credible_interval=0.95, kind='kde');
```

The plot shows the posterior distribution of $\mu$, along with an estimate of the 95% posterior **credible interval**.

The output

    85% < 1.38629 < 15%

informs us that the probability of $\mu$ being less than log(4) is 85% and the corresponding probability of being greater than log(4) is 15%.

> The posterior probability that the mean level of household radon in Henneprin County is greater than 4 pCi/L is 0.15.

### Prediction

What is the probability that a given household has a log-radon measurement larger than one? To answer this, we make use of the **posterior predictive distribution**.

$$p(z |y) = \int_{\theta} p(z |\theta) p(\theta | y) d\theta$$

where here $z$ is the predicted value and y is the data used to fit the model.

We can estimate this from the posterior samples of the parameters in the model.

```python
mus = samples['μ']
sigmas = samples['σ']
```

```python
radon_samples = Normal.dist(mus, sigmas).random()
```

```python
(radon_samples > np.log(4)).mean()
```

    0.462

> The posterior probability that a randomly-selected household in Henneprin County contains radon levels in excess of 4 pCi/L is about 0.46.

### Model checking

But, ***how do we know this model is any good?***

Its important to check the fit of the model, to see if its assumptions are reasonable. One way to do this is to perform **posterior predictive checks**. This involves generating simulated data using the model that you built, and comparing that data to the observed data.

One can choose a particular statistic to compare, such as tail probabilities or quartiles, but here it is useful to compare them graphically.

We already have these simulations from the previous exercise!

```python
sns.distplot(radon_samples, label='simulated')
sns.distplot(hennepin_radon, label='observed')
plt.legend()
```

### Exercise: Prior sensitivity

Its important to check the sensitivity of your choice of priors to the resulting inference. Specify the radon model using different priors for the mean and stanard deviation. [For reference, here is a link to the probability distributions API for PyMC3](https://docs.pymc.io/api/distributions.html), which lists the available distributions.

Think hard about the desirable properties of priors for each variable.

```python
from pymc3 import # Import distributions here

with Model() as prior_sensitivity:

    μ = ?
    σ = ?

    dist = Normal('dist', mu=μ, sigma=σ, observed=hennepin_radon)

    sensitivity_samples = sample(1000, tune=1000)
```

```python
plot_posterior(sensitivity_samples, var_names=['μ'], ref_val=np.log(4));
```

Here is the original model for comparison:

```python
plot_posterior(samples, var_names=['μ'], ref_val=np.log(4));
```

## Two Groups with Continuous Outcome

To illustrate how this Bayesian estimation approach works in practice, we will use a fictitious example from Kruschke (2012) concerning the evaluation of a clinical trial for drug evaluation. The trial aims to evaluate the efficacy of a "smart drug" that is supposed to increase intelligence by comparing IQ scores of individuals in a treatment arm (those receiving the drug) to those in a control arm (those recieving a placebo). There are 47 individuals and 42 individuals in the treatment and control arms, respectively.

```python
drug = pd.DataFrame(dict(iq=(101,100,102,104,102,97,105,105,98,101,100,123,105,103,100,95,102,106,
        109,102,82,102,100,102,102,101,102,102,103,103,97,97,103,101,97,104,
        96,103,124,101,101,100,101,101,104,100,101),
                         group='drug'))
placebo = pd.DataFrame(dict(iq=(99,101,100,101,102,100,97,101,104,101,102,102,100,105,88,101,100,
           104,100,100,100,101,102,103,97,101,101,100,101,99,101,100,100,
           101,100,99,101,100,102,99,100,99),
                            group='placebo'))

trial_data = pd.concat([drug, placebo], ignore_index=True)
trial_data.hist('iq', by='group');
```

Since there appear to be extreme ("outlier") values in the data, we will choose a Student-t distribution to describe the distributions of the scores in each group. This sampling distribution adds **robustness** to the analysis, as a T distribution is less sensitive to outlier observations, relative to a normal distribution.

The three-parameter Student-t distribution allows for the specification of a mean $\mu$, a precision (inverse-variance) $\lambda$ and a degrees-of-freedom parameter $\nu$:

$$f(x|\mu,\lambda,\nu) = \frac{\Gamma(\frac{\nu + 1}{2})}{\Gamma(\frac{\nu}{2})} \left(\frac{\lambda}{\pi\nu}\right)^{\frac{1}{2}} \left[1+\frac{\lambda(x-\mu)^2}{\nu}\right]^{-\frac{\nu+1}{2}}$$

the degrees-of-freedom parameter essentially specifies the "normality" of the data, since larger values of $\nu$ make the distribution converge to a normal distribution, while small values (close to zero) result in heavier tails.

Thus, the likelihood functions of our model are specified as follows:

$$\begin{align}
y^{(drug)}_i &\sim T(\nu, \mu_1, \sigma_1) \\
y^{(placebo)}_i &\sim T(\nu, \mu_2, \sigma_2)
\end{align}$$

As a simplifying assumption, we will assume that the degree of normality $\nu$ is the same for both groups.

### Prior choice

Since the means are real-valued, we will apply normal priors. Since we know something about the population distribution of IQ values, we will center the priors at 100, and use a standard deviation that is more than wide enough to account for plausible deviations from this population mean:

$$\mu_k \sim N(100, 10^2)$$

```python
with Model() as drug_model:

    μ_0 = Normal('μ_0', 100, sigma=10)
    μ_1 = Normal('μ_1', 100, sigma=10)
```

Similarly, we will use a uniform prior for the standard deviations, with an upper bound of 20.

```python
with drug_model:
    σ_0 = Uniform('σ_0', lower=0, upper=20)
    σ_1 = Uniform('σ_1', lower=0, upper=20)
```

For the degrees-of-freedom parameter $\nu$, we will use an **exponential** distribution with a mean of 30; this allocates high prior probability over the regions of the parameter that describe the range from normal to heavy-tailed data under the Student-T distribution.

```python
from pymc3 import Exponential

with drug_model:
    ν = Exponential('ν_minus_one', 1/29.) + 1

```

```python
sns.distplot(Exponential.dist(1/29).random(size=10000), kde=False);
```

```python
from pymc3 import StudentT

with drug_model:

    drug_like = StudentT('drug_like', nu=ν, mu=μ_1, lam=σ_1**-2, observed=drug.iq)
    placebo_like = StudentT('placebo_like', nu=ν, mu=μ_0, lam=σ_0**-2, observed=placebo.iq)
```

Now that the model is fully specified, we can turn our attention to tracking the posterior quantities of interest. Namely, we can calculate the difference in means between the drug and placebo groups.

As a joint measure of the groups, we will also estimate the "effect size", which is the difference in means scaled by the pooled estimates of standard deviation. This quantity can be harder to interpret, since it is no longer in the same units as our data, but it is a function of all four estimated parameters.

```python
from pymc3 import Deterministic

with drug_model:

    diff_of_means = Deterministic('difference of means', μ_1 - μ_0)

    effect_size = Deterministic('effect size',
                        diff_of_means / np.sqrt((σ_1**2 + σ_0**2) / 2))

```

```python
with drug_model:

    drug_trace = sample(1000, random_seed=RANDOM_SEED)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [ν_minus_one, σ_1, σ_0, μ_1, μ_0]

  <progress value='0' class='' max='4000' style='width:300px; height:20px; vertical-align: middle;'></progress>

    Sampling 2 chains for 1_000 tune and 1_000 draw iterations (2_000 + 2_000 draws total) took 14 seconds.
    The acceptance probability does not match the target. It is 0.7034689607965747, but should be close to 0.8. Try to increase the number of tuning steps.

```python
plot_posterior(drug_trace[100:], var_names=['μ_0', 'μ_1', 'σ_0', 'σ_1', 'ν_minus_one']);
```

```python
plot_posterior(drug_trace[100:],
          var_names=['difference of means', 'effect size'],
          ref_val=0);
```

> The posterior probability that the mean IQ of subjects in the treatment group is greater than that of the control group is 0.99.

---

# References

Gelman, Andrew, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin. 2013. Bayesian Data Analysis, Third Edition. CRC Press.

Pilon, Cam-Davidson. [Probabilistic Programming and Bayesian Methods for Hackers](http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/)