Contenido bajo licencia Creative Commons BY 4.0 y código bajo licencia MIT. © Juan Gómez y Nicolas Guarín-Zapata 2019-2020. Este material es parte del curso Mecánica de los Medios Continuos en el programa de Ingeniería Civil de la Universidad EAFIT.

# Soluciones analíticas para presas triangulares

Este Notebook permite visualizar la solución analítica de una presa triangular
para un ángulo interno de 45° sometida al empuje de un fluido de peso específico
$\gamma$ en condiciones hidrostáticas. Adicionalmente, también permite visualizar
la solución para una presa triangular con ángulo interno $\alpha$ sometida al
empuje de un fluido de densidad $\rho$ en condiciones hidrostáticas, y peso
propio para concreto de densidad $\rho_c$.

Estas soluciones pueden usarse como apoyo en la actividad de diseño de la presa
trapezoidal.

## Importación de paquetes

```python
import numpy as np
from numpy import pi, sin, cos, sqrt
import matplotlib.pyplot as plt
from meshzoo import triangle
from ipywidgets import interact
```

```python
%matplotlib notebook
gris = '#757575'
plt.rcParams["mathtext.fontset"] = "cm"
plt.rcParams["text.color"] = gris
plt.rcParams["font.size"] = 12
plt.rcParams["xtick.color"] = gris
plt.rcParams["ytick.color"] = gris
plt.rcParams["axes.labelcolor"] = gris
plt.rcParams["axes.edgecolor"] = gris
plt.rcParams["axes.spines.right"] = False
plt.rcParams["axes.spines.top"] = False
plt.rcParams["image.cmap"] = "RdYlBu_r"
```

## Presa sin peso propio

El siguiente esquema muestra el problema que se está estudiando.

Para este problema, la solución de tensiones está dada por:

\begin{align}
\sigma_{xx} &= \gamma(y-H)\\
\sigma_{yy} &= -2\gamma x-\gamma(y-H)\\
\tau_{yy} &= -\gamma x
\end{align}

La siguiente función (`plot_dam`) permite graficar los contornos para la presa.
Tiene como parámetro opcional `plot_type` que permite graficar los contornos
para los esfuerzos principales y cortantes máximos — cuando es igual a `"principal"` —
y graficar los esfuerzos en el sistema coordenado dado — cuando es igual a
`"original"`.

```python
def plot_dam(plot_type="principal"):
    npts = 30
    pts, tri = triangle(npts, corners=[[0, 0], [1, 0], [0, 1]])
    x, y = pts.T
    γ = 1
    σ_xx = γ*(y - 1)
    σ_yy = -2*γ*x - γ*(y - 1)
    τ_xy = -γ*x
    τ_max = sqrt((σ_xx - σ_yy)**2/4 + τ_xy**2)
    σ_m = 0.5*(σ_xx + σ_yy)
    σ1 = σ_m + τ_max
    σ2 = σ_m - τ_max

    if plot_type == "principal":
        fields = [σ1, σ2, τ_max]
        titles = [r"$\sigma_{1}/\gamma$",
                  r"$\sigma_{2}/\gamma$",
                  r"$\tau_\max/\gamma$"]
    else:
        fields = [σ_xx, σ_yy, τ_xy]
        titles = [r"$\sigma_{xx}/\gamma$",
                  r"$\sigma_{yy}/\gamma$",
                  r"$\tau_{xy}/\gamma$"]

    plt.clf()
    fig = plt.gcf()
    axes = fig.subplots(1, 3)
    for cont, ax in enumerate(axes):
        f_max = max(np.max(fields[cont]), -np.min(fields[cont]))
        im = ax.tricontourf(x, y, tri, fields[cont], levels=12,
                            vmin=-f_max, vmax=f_max)
        fig.colorbar(im, ax=ax, shrink=0.5)
        ax.axis("image")
        ax.axis("off")
        ax.set_title(titles[cont])
    plt.tight_layout()
    return None
```

Por facilidad de uso, hemos creado un _widget_ que permite variar el parámetro
de entrada con un menú desplegable.

Pruebe a cambiar el parámetro de entrada para ver cómo difieren los campos para las componentes
del tensor en el sistema coordenado dado y en sus direcciones principales.

```python
plt.figure(figsize=(6, 3))
widget0 = interact(plot_dam,
                  plot_type=["original",
                             "principal"])
```

    <IPython.core.display.Javascript object>

    interactive(children=(Dropdown(description='plot_type', index=1, options=('original', 'principal'), value='pri…

## Presa con peso propio

El siguiente esquema muestra el problema que se está estudiando.

Este es un problema más general que el anterior, ya que la solución es válida
para diferentes ángulos (no solo 45°), y además permite considerar el peso de
la presa. Cuando $\rho_c = 0$ y $\alpha = \pi/4 \approx 0.79$ se tiene el caso
anterior.

En este caso, la solución analítica de tensiones está dada por:

\begin{align}
\frac{\sigma_{rr}}{\rho g} =& - \left(- \frac{\rho_{c} \cot(\alpha)}{4} + \frac{\cot^{3}(\alpha)}{2}\right) \sin(3\theta)
  + \left(\frac{\rho_{c} \cot(\alpha)}{4} - \frac{\cot^{3}(\alpha)}{2}\right) \sin(\theta)\\
  &+ \left(- \frac{\rho_{c}}{4} + \frac{3 \cot^{2}(\alpha)}{4} + \frac{1}{4}\right) \cos(3\theta)
  + \left(\frac{\rho_{c}}{4} + \frac{\cot^{2}(\alpha)}{4} - \frac{1}{4}\right) \cos(\theta)
  - \rho_{c} \cos(\theta)\\
\frac{\sigma_{\theta\theta}}{\rho g} =& \left(- \frac{\rho_{c} \cot(\alpha)}{4}
  + \frac{\cot^{3}(\alpha)}{2}\right) \sin(3\theta) + \left(\frac{3 \rho_{c} \cot(\alpha)}{4}
  - \frac{3 \cot^{3}(\alpha)}{2}\right) \sin(\theta)\\
  &+ \left(\frac{\rho_{c}}{4} - \frac{3 \cot^{2}(\alpha)}{4} - \frac{1}{4}\right) \cos(3\theta)
  + \left(\frac{3 \rho_{c}}{4} + \frac{3 \cot^{2}(\alpha)}{4} - \frac{3}{4}\right) \cos(\theta)
  - \rho_{c} \cos(\theta)\\
\frac{\sigma_{r\theta}}{\rho g} =& - \left(- \frac{\rho_{c} \cot(\alpha)}{4}
  + \frac{\cot^{3}(\alpha)}{2}\right) \cos(3\theta) - \left(\frac{\rho_{c} \cot(\alpha)}{4}
  - \frac{\cot^{3}(\alpha)}{2}\right) \cos(\theta)\\
  &+ \left(\frac{\rho_{c}}{4}  - \frac{3 \cot^{2}(\alpha)}{4} - \frac{1}{4}\right) \sin(3\theta)
  + \left(\frac{\rho_{c}}{4} + \frac{\cot^{2}(\alpha)}{4} - \frac{1}{4}\right) \sin(\theta)
\end{align}

La siguiente función (`plot_dam_weight`) permite graficar los contornos para la presa
con peso propio. Tiene como parámetros opcionales: `α`, `ρ_c` y `plot_type` que permiten
cambiar el ángulo, densidad de la presa y el tipo de gráfica — al igual que en el
caso anterior —.

```python
def plot_dam_weight(α=pi/4, ρ_c=1.0, plot_type="principal"):
    npts = 30
    pts, tri = triangle(npts, corners=[[0, 0], [cos(α), 0], [cos(α), sin(α)]])
    x, y = pts.T
    r = sqrt(x**2 + y**2)
    θ = np.arctan2(y, x)
    #r = 1
    ρ = 1
    g = 1
    cot = lambda x: cos(x)/sin(x)
    A =  (ρ_c - ρ)*g/24 - ρ*g *cot(α)**2/8
    B = ρ*cot(α)**3/12 - ρ_c*g*cot(α)/24
    C = (ρ_c - ρ)*g/8 + ρ*g*cot(α)**2/8
    D = ρ_c*g*cot(α)/8 - ρ*g*cot(α)**3/4

    σ_rr = -6*A*r*cos(3*θ) - 6*B*r*sin(3*θ) + 2*C*r*cos(θ) \
        + 2*D*r*sin(θ) - ρ_c*g*r*cos(θ)
    τ_rθ = 6*A*r*sin(3*θ) - 6*B*r*cos(3*θ) + 2*C*r*sin(θ) \
        - 2*D*r*cos(θ)
    σ_θθ = 6*A*r*cos(3*θ) + 6*B*r*sin(3*θ) + 6*C*r*cos(θ) \
        + 6*D*r*sin(θ) - ρ_c*g*r*cos(θ)

    τ_max = sqrt((σ_rr - σ_θθ)**2/4 + τ_rθ**2)
    σ_m = 0.5*(σ_rr + σ_θθ)
    σ1 = σ_m + τ_max
    σ2 = σ_m - τ_max

    if plot_type == "principal":
        fields = [σ1, σ2, τ_max]
        titles = [r"$\sigma_{1}/\rho g$",
                  r"$\sigma_{2}/\rho g$",
                  r"$\tau_\max/\rho g$"]
    else:
        fields = [σ_rr, σ_θθ, τ_rθ]
        titles = [r"$\sigma_{rr}/\rho g$",
                  r"$\sigma_{\theta\theta}/\rho g$",
                  r"$\tau_{r\theta}/\rho g$"]

    plt.clf()
    fig = plt.gcf()
    axes = fig.subplots(1, 3)
    for cont, ax in enumerate(axes):
        f_max = max(np.max(fields[cont]), -np.min(fields[cont]))
        im = ax.tricontourf(y, x, tri, fields[cont], levels=12, vmin=-f_max, vmax=f_max)
        fig.colorbar(im, ax=ax, shrink=0.5)
        ax.invert_yaxis()
        ax.axis("image")
        ax.axis("off")
        ax.set_title(titles[cont])
    plt.tight_layout()
    return None
```

Por facilidad de uso, hemos creado un _widget_ que permite variar los parámetros.
El ángulo ($\alpha$) y densidad ($\rho_c$), pueden variarse usando deslizadores
y el tipo de gráfico con un menú desplegable.

Pruebe a cambiar los parámetros de entrada para ver cómo difieren los campos.

```python
plt.figure(figsize=(6, 3))
widget = interact(plot_dam_weight,
                  α=(0.08,1.0),
                  ρ_c=(0.0,10.),
                  plot_type=["original",
                             "principal"])
```

    <IPython.core.display.Javascript object>

    interactive(children=(FloatSlider(value=0.7853981633974483, description='α', max=1.0, min=0.08), FloatSlider(v…

## Referencias

* Juan Gómez, César Augusto Sierra, Juan Carlos Vergara, Mario Sáenz, y Nicolás Guarín-Zapata (2018). Notas de clase: Mecánica del medio continuo. Universidad EAFIT, Disponible en: https://github.com/AppliedMechanics-EAFIT/Notas-MMC.

* J.R. Barber (2010). Elasticity. Springer Netherlands.

## Formato del Notebook

La siguiente celda da formato al Notebook.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('./nb_style.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>


# Métodos Avanzados en Estadística
# Entrega 3: Regresión
## José Antonio Álvarez Ocete

```R
options(warn=-1)
shhh <- suppressPackageStartupMessages # It's a library, so shhh!

shhh(library(glmnet))
shhh(library(tidyverse))
library(gapminder)
library(comprehenr)
library(ggplot2)
library(dplyr)
library(ggpubr)
shhh(library(KernSmooth))

theme_set(theme_bw())
options(warn=0)
```

### Ejercicio 1

Los datos del fichero *Datos-geyser.txt* corresponden al día de la observación (primera columna), el tiempo medido en minutos (segunda columna $Y$ ) y el tiempo hasta la siguiente erupción (tercera columna $X$) del geyser *Old Faithful* en el parque norteamericano de Yellowstone.

**a)** Representa gráficamente los datos, junto con el estimador de Nadaraya-Watson de la función de regresión de $Y$ sobre $X$.

**b)** Representa gráficamente los datos, junto con el estimador localmente lineal de la función de regresión de $Y$ sobre $X$.

Representaremos ambos estimadores en el mismo gráfico para apreciar mejor las disferencias entre ambos.

```R
geyser_df <- read.table('datos/Datos-geyser.txt', header=TRUE, sep=' ', )
head(geyser_df)
```

A data.frame: 6 × 3

	D	Y	X

	<int>	<int>	<dbl>

1	1	78	4.4

2	1	74	3.9

3	1	68	4.0

4	1	76	4.0

5	1	80	3.5

6	1	84	4.1

```R
n <- length(geyser_df$X)
loc_lineal <- locpoly(geyser_df$X, geyser_df$Y, degree = 1, gridsize=n,
                      bandwidth = dpill(geyser_df$X, geyser_df$Y))

ggplot(geyser_df, aes(X, Y)) +
  geom_point() +
  geom_smooth(formula=y~x, method = 'loess', se = FALSE, span = 0.25, method.args = list(degree=0), aes(col='Nadaraya-Watson')) +
  geom_smooth(formula=y~x, method = 'loess', se = FALSE, span = 0.25, method.args = list(degree=1), aes(col='Localmente lineal')) +
  scale_colour_manual("", breaks = c('Nadaraya-Watson', 'Localmente lineal'), values = c('red', 'blue')) +
  labs(x = 'Duración de erupción (minutos)', y = 'Tiempo entre erupciones') +
  theme(legend.justification=c(1,0), legend.position=c(1,0))
```

### Ejercicio 4

Se considera el siguiente modelo de regresión lineal múltiple:

$$
\begin{equation} \label{eq:model} \tag{1}
    Y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \beta_3 x_{i3} + \epsilon_i, \quad \epsilon_i \sim N(0, \sigma^2), \quad i\in\{1,\ldots,n\}
\end{equation}
$$

Se dispone de $n=20$ observaciones con las que se ajustan todos los posibles submodelos del modelo \eqref{eq:model}, obteniéndose para cada uno de ellos las siguientes sumas de cuadrados de los residuos (todos los submodelos incluyen un término independiente):

| Variables incluidas en el modelo |          Coeficientes de regresión          |    SCR   |
|:--------------------------------:|:-------------------------------------------:|:--------:|
|    Sólo término independiente    |                  $\beta_0$                  | 42644.00 |
|               $x_1$              |            $\beta_0$ y $\beta_1$            |  8352.28 |
|               $x_2$              |            $\beta_0$ y $\beta_2$            | 36253.69 |
|               $x_3$              |            $\beta_0$ y $\beta_3$            | 36606.19 |
|           $x_1$ y $x_2$          |       $\beta_0$, $\beta_1$ y $\beta_2$      |  7713.13 |
|           $x_1$ y $x_3$          |       $\beta_0$, $\beta_1$ y $\beta_3$      |  762.55  |
|           $x_2$ y $x_3$          |       $\beta_0$, $\beta_2$ y $\beta_3$      | 32700.17 |
|       $x_1$, $x_2$ y $x_3$       | $\beta_0$, $\beta_1$, $\beta_2$ y $\beta_3$ |  761.41  |

**a)** Calcula la tabla de análisis de la varianza para el modelo \eqref{eq:model} y contrasta a nivel $\alpha = 0,05$ la hipótesis nula $H_0: \beta_1 = \beta_2 = \beta_3 = 0$.

Sabemos que para realizar este tipo de contraste hemos de hacer uso del siguiente estimador:

$$
    F = \frac{\mbox{SCE}/p}{\mbox{SCR}/(n-p-1)}
$$

Y puesto que bajo $H_0$, $F$ sigue una distribución $F_{p, n-p-1}$, nuestro *p-value* para este contraste será $\mathbb P[F_{p, n-p-1} > F]$. El problema es que no disponemos de $\mbox{SCE}$ ni de $\mbox{SCT}$. Hemos de darnos cuenta de que la $\mbox{SCT}$ del modelo completo es la suma de cuadrados residual del modelo que utiliza únicamente el término independiente:

$$
\begin{align}
    \mbox{SCR}_0 & = \sum_{i=1}^n (\hat y_i - y_i)^2 \\
    & = \sum_{i=1}^n (\hat \beta_0 - y_i)^2 \\
    & = \sum_{i=1}^n (\bar y - y_i)^2 = \mbox{SCT}
\end{align}
$$

Podemos calcular la suma de cuadrados explicada utilizando la ortogonalidad de los residuos con las variables regresoras:

$$
    \mbox{SCT} = \mbox{SCE} + \mbox{SCR} \rightarrow \mbox{SCE} = \mbox{SCR} - \mbox{SCT} = \mbox{SCR} - \mbox{SCR}_0
$$

Con este desarrollo en mente es sencillo computar la tabla de varianzas del modelo completo.

```R
n <- 20
p <- 3
all_scrs <- c(42644, 8352.28, 36253.69, 36606.19, 7713.13, 762.55, 32700.17, 761.41)
scr <- all_scrs[8]
sce <- all_scrs[1] - scr

source <- c("Complete model", "Residuals")
df <- c(p, n - p - 1)
sum_sq <- c(sce, scr)
mean_sq <- sum_sq/df
f <- c(mean_sq[1]/mean_sq[2], NA)
pr_f <- c(pf(f[1], df1=p, df2=n-p-1, lower.tail=FALSE), NA)

variance_table <- data.frame(Source=source, Df=df, Sum_Sq=sum_sq, Mean_Sq=mean_sq, F_value=f, p_value=pr_f)
variance_table
```

A data.frame: 2 × 6

Source	Df	Sum_Sq	Mean_Sq	F_value	p_value

<chr>	<dbl>	<dbl>	<dbl>	<dbl>	<dbl>

Complete model	 3	41882.59	13960.86333	293.3686	3.421043e-14

Residuals     	16	  761.41	   47.58812	      NA	          NA

Obtenemos un *p-value* de $\approx 3.42 \cdot 10^{-14} \ll 0.05$, por lo que tenemos suficiente evidencia para rechazar la hipótesis nula.

**b)** En el modelo $(1)$ , contrasta a nivel $\alpha = 0,05$ las dos hipótesis nulas siguientes:

- $H_0: \beta_2 = 0$
- $H_0: \beta_1 = \beta_3 = 0$

En este caso hemos de utilizar el siguiente estadístico:

$$
    F = \frac{\mbox{(SCR_0 - SCR}/k}{\mbox{SCR}/(n-p-1)}
$$

Donde $k$ es el número de variables del modelo reducido. Puesto que bajo $H_0$, $F$ sigue una distribución $F_{k, n-p-1}$, nuestro *p-value* para este contraste será $\mathbb P[F_{k, n-p-1} > F]$. Hacemos una función que, dados los $\mbox{SCR}$'s del modelo completo y reducido, computa el p-valor:

```R
reduced_model_p_value <- function(total_scr, reduced_scr, n, p, k) {
    numerator <- (reduced_scr - total_scr) / k
    denominator <- total_scr / (n - p - 1)
    pf(numerator/denominator, df1=k, df2=n-p-1, lower.tail=FALSE)
}
```

**Caso $H_0: \beta_2=0$.**

```R
k <- 1
cat('p-value: ', reduced_model_p_value(all_scrs[8], all_scrs[6], n, p, k))
```

    p-value:  0.8789337

Puesto que obtenemos un *p-value* de $\approx 0.8789 > 0.05$, no tenemos suficiente evidencia para rechazar la hipótesis nula. Es decir, **la variable $x_2$ puede no ser significativa para la predicción de $y$**.

**b) Caso $H_0: \beta_1 = \beta_3 = 0$.**

```R
k <- 2
cat('p-value: ', reduced_model_p_value(all_scrs[8], all_scrs[3], n, p, k))
```

    p-value:  3.785566e-14

Puesto que obtenemos un *p-value* de $\approx 3.79 \cdot 10^{-14} \ll 0.05$, rechazamos la hipótesis nula. Es decir, **las variable $x_1$ y $x_3$ son significativas para la predicción de $y$**.

### Ejercicio 6

Sean $Y_1$, $Y_2$ e $Y_3$ tres variables aleatorias independientes con distribución normal y varianza $\sigma^2$. Supongamos que $\mu$ es la media de $Y_1$, $\lambda$ es la media de $Y_2$ y $\lambda + \mu$ es la media de $Y_3$, donde $\lambda, mu \in \mathbb R$.

**a)** Demuestra que el vector $Y = (Y_1, Y_2, Y_3)'$0 verifica el modelo de regresión múltiple $Y = X\beta + \epsilon$. Para ello, determina la matriz de diseño $X$, el vector de parámetros $\beta$ y la distribución de las variables de error $\epsilon$.

Podemos expresar el vector de variables independientes de la siguiente forma:

$$\begin{pmatrix}
    Y_1 \\
    Y_2 \\
    Y_3 \\
\end{pmatrix} =
\underbrace{ \begin{pmatrix}
    1 & 0 \\
    0 & 1 \\
    1 & 1
\end{pmatrix} }_{X}
\underbrace{ \begin{pmatrix}
    \mu \\
    \lambda
\end{pmatrix} }_{\beta} +
\begin{pmatrix}
    \epsilon_1 \\
    \epsilon_2 \\
    \epsilon_3
\end{pmatrix} \quad
\epsilon_i \sim N(0, \sigma^2)
$$

Obteniendo el siguiente modelo de regresión múltiple:

$$
   Y \sim N(X\beta, \sigma^2 I)
$$

**b)** Calcula los estimadores de máxima verosimilitud (equivalentemente, de mínimos cuadrados) de $\lambda$ y $\mu$.

Sabemos que el estimador de mínimos cuadrados puede calcular utilizando la siguiente expresión:

$$
    \hat \beta = (X'X)^{-1}X'Y
$$

Sin embargo, para demostrar esta expresión en clase nos basamos en que la matriz de diseño tenía una columna de unos (el modelo tenía un término independiente $\beta_0$). ¿Es ésta expresión cierta si el modelo carece de término independiente?

La respuesta es afirmativa. Para obtenerla sin hacer uso de dicha hipótesis podemos simplemente obtener el valor que minimiza el error cuadrado mínimo derivando e igualando a $0$ en la siguiente expresión:

$$
    L(\sigma, \beta) = \; \parallel Y - X\beta \;\parallel^2_2
$$

Las cuentas no son complicadas, puede consultarse la demostración completa en [Wikipedia](https://en.wikipedia.org/wiki/Least_squares#Linear_least_squares). Calculemos estos valores para nuestro caso particular:

$$
X'X =
\begin{pmatrix}
    1 & 0 & 1 \\
    0 & 1 & 1 \\
\end{pmatrix}
\begin{pmatrix}
    1 & 0 \\
    0 & 1 \\
    1 & 1
\end{pmatrix} =
\begin{pmatrix}
    2 & 1 \\
    1 & 2
\end{pmatrix}
$$

$$
(X'X)^{-1} = \frac{1}{3}
\begin{pmatrix}
    2 & -1 \\
    -1 & 2
\end{pmatrix}
$$

$$
(X'X)^{-1}X' = \frac{1}{3}
\begin{pmatrix}
    2 & -1 \\
    -1 & 2
\end{pmatrix}
\begin{pmatrix}
    1 & 0 & 1 \\
    0 & 1 & 1 \\
\end{pmatrix} = \frac{1}{3}
\begin{pmatrix}
    2 & -1 & 1 \\
    -1 & 2 & 1 \\
\end{pmatrix}
$$

$$
\hat \beta = (X'X)^{-1}X'Y =
\begin{pmatrix}
    2 & -1 & 1 \\
    -1 & 2 & 1 \\
\end{pmatrix}
\begin{pmatrix}
    Y_1 \\
    Y_2 \\
    Y_3 \\
\end{pmatrix} = \frac{1}{3}
\begin{pmatrix}
    2 Y_1 - Y_2 + Y_3 \\
    -Y_1 + 2 Y_2 + Y_3
\end{pmatrix}
$$

**c)** Calcula la distribución del vector $(\hat \lambda, \hat \mu)'$, formado por los estimadores calculados en el apartado anterior.

Puesto que el vector de variables independientes cumple

$$
   Y \sim N(X\beta, \sigma^2 I)
$$

Y sabiendo que $\hat \beta = (X'X)^{-1}X'Y$, obtenemos que $\hat \beta$ sigue una distribución normal multivariante:

$$
   \hat \beta \sim N( ((X'X)^{-1}X') \cdot X\beta, ((X'X)^{-1}X') \cdot \sigma^2 I \cdot ((X'X)^{-1}X')')
$$

Para la esperanza:

$$
    \mathbb E [\hat\beta] = (X'X)^{-1}X' \cdot X\beta = (X'X)^{-1} \cdot (X'X) \beta = \beta
$$

Luego el estimador es insesgado. Por otro lado, para la varianza hemos de ver que $X'X$ es simétrica (y por lo tanto su inversa también lo será):

$$
    (X'X)' = X'X'' = X'X
$$

Por lo tanto, la varianza cumple:

$$
\begin{align}
    \text{Var}[\hat\beta] & = ((X'X)^{-1}X') \cdot \sigma^2 I \cdot ((X'X)^{-1}X')' \\
     & = \sigma^2 (X'X)^{-1} (X'X) (X'X)^{-1} \\
     & = \sigma^2 (X'X)^{-1} \\
\end{align}
$$

Obtenemos así la expresión reducida de la distribución que siguen los estimadores:

$$
   \hat \beta \sim N( \beta, \sigma^2(X'X)^{-1})
$$

### Ejercicio 10

Los datos *fuel2001* del fichero *combustible.RData* corresponden al consumo de combustible (y otras variables relacionadas) en los estados de EE.UU.
Se desea explicar la variable ***FuelC*** en función del resto de la información.

```R
datos <- 'http://verso.mat.uam.es/~joser.berrendero/datos/combustible.RData'
load(url(datos))
head(fuel2001)
```

A data.frame: 6 × 7

	Drivers	FuelC	Income	Miles	MPC	Pop	Tax

	<int>	<int>	<int>	<int>	<dbl>	<int>	<dbl>

AL	 3559897	 2382507	23471	 94440	12737.00	 3451586	18.0

AK	  472211	  235400	30064	 13628	 7639.16	  457728	 8.0

AZ	 3550367	 2428430	25578	 55245	 9411.55	 3907526	18.0

AR	 1961883	 1358174	22257	 98132	11268.40	 2072622	21.7

CA	21623793	14691753	32275	168771	 8923.89	25599275	18.0

CO	 3287922	 2048664	32949	 85854	 9722.73	 3322455	22.0

**a)** Representa en un plano las dos primeras componentes principales de estos datos estandarizados (consulta la ayuda de *prcomp*). ¿Son suficientes estas dos componentes para explicar un alto porcentaje de la varianza?

En primer lugar, hemos de aclarar que como nuestro objetivo final es predecir la variable *FuelC*, la quitamos de nuestro conjunto de datos para hacer PCA. Si la mantuviésemos, las componentes principales no se podrían utilizar para predecir dicha variable ya que ya la contienen. Si simplemente quisiésemos realizar una análisis de PCA sobre los datos para hacer, por ejemplo, cluestering, mantendríamos dicha variable. He tomado esta decisión porque en el enunciado no aclara que debamos quitarlo, pero tiene sentido para nuestra aplicación final.

Utilizamos la función de R *prcomp* sugerida en el enunciado con los parámetros *center* y *scale.* para normalizar las variables antes de computar el análisos. Pintamos a continuación las dos primeras componentes principales:

```R
fuel2001_without_FuelC <- data.frame(fuel2001)
fuel2001_without_FuelC$FuelC <- NULL
pca <- prcomp(fuel2001_without_FuelC, center=TRUE, scale.=TRUE)

ggplot(as.data.frame(pca$x), aes(PC1, PC2)) +
    geom_point(color="steelblue") +
    labs(x='PC1', y='PC2') +
    theme_minimal()
```

Como se aprecia en el gráfico, los puntos están relativamente separados. Este es el objetivo principal de PCA: maximizar la distancia entre puntos (la varianza) utilizando un conjunto mínimo de variables, siendo éstas combinaciones lineales de las orginales. Utilizamos *summary* para ver los resultados numéricos del PCA.

```R
summary(pca)

pov <- summary(pca)$importance[2,]
names <- c(paste('PC', 1:6))
proportions_df <- data.frame(PC=names, Proportion_of_variance=pov)

ggplot(proportions_df, aes(x=PC, y=Proportion_of_variance)) +
    geom_bar(stat="identity", fill="steelblue") +
    theme_minimal()
```

Obtenemos así un $71.90\%$ de varianza explicada utilizando únicamente 2 componentes principales.

Si repetimos este análisis añadiendo la variable *FuelC* obtenemos 7 componentes principales en vez de 6, y la dos primeras explican un $75.47\%$ de la varianza.

**b)** Ajusta el modelo completo con todas las variables. En este modelo completo, contrasta la hipótesis nula de que los coeficientes de las variables **Income**, **MPC** y **Tax** son simultáneamente iguales a cero.

Como vimos en el ejercicio 4, para comprobar este tipo de hipótesis hemos de ajustar ambnos modelos (completo y reducido), y utilizar el siguiente estadístico:

$$
    F = \frac{ (\mbox{SCR_0}- \mbox{SCR}) / k }{ \mbox{SCR} / (n - p - 1) }
$$

Nuestro *p-value* será:

$$
    \mathbb P [F_{k;n-p-1} \ge F]
$$

```R
complete_model <- lm(FuelC ~ ., data=fuel2001)
summary(complete_model)
```

    Call:
    lm(formula = FuelC ~ ., data = fuel2001)

    Residuals:
         Min       1Q   Median       3Q      Max
    -1480910  -158802    19267   174208  1090089

    Coefficients:
                  Estimate Std. Error t value Pr(>|t|)
    (Intercept) -4.902e+05  8.199e+05  -0.598 0.552983
    Drivers      6.368e-01  1.452e-01   4.386 7.09e-05 ***
    Income       7.690e+00  1.632e+01   0.471 0.639793
    Miles        5.850e+00  1.621e+00   3.608 0.000784 ***
    MPC          4.562e+01  3.565e+01   1.280 0.207337
    Pop         -1.945e-02  1.245e-01  -0.156 0.876586
    Tax         -2.087e+04  1.324e+04  -1.576 0.122235
    ---
    Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

    Residual standard error: 398400 on 44 degrees of freedom
    Multiple R-squared:  0.9808,	Adjusted R-squared:  0.9782
    F-statistic: 374.6 on 6 and 44 DF,  p-value: < 2.2e-16

```R
reduced_model <- lm(FuelC ~ Drivers + Miles + Pop, data=fuel2001)
summary(reduced_model)
```

    Call:
    lm(formula = FuelC ~ Drivers + Miles + Pop, data = fuel2001)

    Residuals:
         Min       1Q   Median       3Q      Max
    -1486667  -209534    25497   186536  1083358

    Coefficients:
                  Estimate Std. Error t value Pr(>|t|)
    (Intercept) -2.374e+05  1.047e+05  -2.268 0.027964 *
    Drivers      7.156e-01  1.446e-01   4.949 9.98e-06 ***
    Miles        6.125e+00  1.472e+00   4.161 0.000134 ***
    Pop         -8.879e-02  1.229e-01  -0.722 0.473593
    ---
    Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

    Residual standard error: 408800 on 47 degrees of freedom
    Multiple R-squared:  0.9784,	Adjusted R-squared:  0.977
    F-statistic: 709.9 on 3 and 47 DF,  p-value: < 2.2e-16

```R
scr_complete <- summary(complete_model)$sigma
scr_reduced <- summary(reduced_model)$sigma
anova(reduced_model, complete_model)
```

A anova: 2 × 6

	Res.Df	RSS	Df	Sum of Sq	F	Pr(>F)

	<dbl>	<dbl>	<dbl>	<dbl>	<dbl>	<dbl>

1	47	7.855597e+12	NA	          NA	      NA	       NA

2	44	6.984313e+12	 3	871284719267	1.829649	0.1556908

Puesto que obtenemos un *p-value* de $0.1557 \gg 0.05$, no tenemos evidentas suficientes para rechazar la hipótesis nula.

**c)** De acuerdo con el método iterativo hacia adelante y el criterio BIC, ¿cúal es el modelo óptimo?

```R
iterative_model <- leaps::regsubsets(FuelC ~ ., data=fuel2001, method='forward')

summary(iterative_model)
plot(summary(iterative_model)$bic,
     xlab="Número de variables",
     ylab="BIC", type="l")
```

El modelo que minimiza el criterio BIC es aquel con tres variables. Como podemos ver en el resumen, el mejor modelo con variables en el método iterativo es aquel que utiliza unicamente ***Drivers***, ***Miles*** y ***Tax***.

**d)** Ajusta el modelo usando lasso, con el parámetro de regularización seleccionado mediante validación cruzada.

```R
set.seed(123)
x <- as.matrix(fuel2001_without_FuelC)
y <- fuel2001$FuelC
lasso_model_cv <- cv.glmnet(x, y, alpha=1) # alpha=1 es lasso
lasso_model_cv
```

    Call:  cv.glmnet(x = x, y = y, alpha = 1)

    Measure: Mean-Squared Error

        Lambda Index   Measure        SE Nonzero
    min  15772    56 3.307e+11 1.484e+11       4
    1se 234210    27 4.655e+11 1.998e+11       2

Vemos así los valores de $\lambda$ obtenidos en validación cruzada, donde *Measure* es la desviación típica, el valor por defecto.

El $\lambda_{min}$ es aquel valor de $\lambda$ que minimiza la media del error en validación cruzada. El $\lambda_{1se}$ es tal que el error esta a una desviación típica del error. Utilizaremos $\lambda_{min}$ ya que minimiza el error a pesar de ser más restrictivo (introducir más sesgo).

Merece la pena destacar que hemos fijado la semilla. Al aplicar validación cruzada obteníamos valores muy variables para ambos lambdas, de esta forma los resultados son completamente reproducibles.

```R
lambda_lasso <- lasso_model_cv$lambda.min
final_lasso_model <- glmnet(x, y, alpha=1, lambda=lambda_lasso) # alpha=1 es lasso
coef(final_lasso_model)
```

    7 x 1 sparse Matrix of class "dgCMatrix"
                           s0
    (Intercept) -1.048244e+05
    Drivers      6.132198e-01
    Income       .
    Miles        5.525452e+00
    MPC          2.987686e+01
    Pop          .
    Tax         -1.945966e+04

Como podemos ver, ajustado lasso de esta forma mantenemos las variables ***Drivers***, ***Miles***, ***MPC*** y ***Tax***.

**e)** Ajusta el modelo usando **ridge**, con el parámetro de regularización seleccionado mediante validación cruzada.

```R
set.seed(123)
ridge_model_cv <- cv.glmnet(x, y, alpha=0) # alpha=0 es ridge
ridge_model_cv
```

    Call:  cv.glmnet(x = x, y = y, alpha = 0)

    Measure: Mean-Squared Error

        Lambda Index   Measure        SE Nonzero
    min 263094   100 2.909e+11 1.179e+11       6
    1se 732075    89 4.033e+11 1.869e+11       6

```R
lambda_ridge <- lasso_model_cv$lambda.min
final_ridge_model <- glmnet(x, y, alpha=0, lambda=lambda_ridge) # alpha=0 es ridge
coef(final_ridge_model)
```

    7 x 1 sparse Matrix of class "dgCMatrix"
                           s0
    (Intercept) -4.336052e+05
    Drivers      4.627274e-01
    Income       7.555968e+00
    Miles        5.861386e+00
    MPC          4.889985e+01
    Pop          1.265333e-01
    Tax         -2.366901e+04

Obtenemos un valor de $\lambda_{min}$ mucho mayor: $263094$ en vez de $15772$. El modelo final es mucho menos restrictivo, manteniendo todas las variables.


Godfrey Beddard 'Applying Maths in the Chemical & Biomolecular Sciences' Chapter 9

```python
# import all python add-ons etc that will be needed later on
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from sympy import *
from scipy.integrate import quad
init_printing()                      # allows printing of SymPy results in typeset maths format
plt.rcParams.update({'font.size': 14})  # set font size for plots
```

# Answers 7-15

**Q7 answer**

The transform equation is $\displaystyle g(k)=\frac{1}{\sqrt{2\pi}}\int_0^\infty \frac{\exp(-t/\tau)}{\tau} \exp(-ikt)dt$ which can be integrated directly to give

$$\displaystyle g(k)=\frac{1}{\tau\sqrt{2\pi}} \left[ -\frac{exp(-ikt-t/\tau)}{ik+1/\tau}  \right]_0^\infty =\frac{1}{\sqrt{2\pi}}\frac{1} {ik\tau+1} $$

The absolute value of a complex number $z$, is $\sqrt{z^*z}$ and its square is clearly $z^*z$, where $z^*$ is the conjugate of the complex number formed by replacing $i$ by $-i$. Doing the same with $g(k)$ gives $\displaystyle |g(k)|^2= \frac{1}{2\pi (k^2\tau^2+1)}$ which is the shape of the spectral line.

The exponential decay with $\tau = 2$ and the corresponding square of the transform, are shown in Fig. 54. In this case $k$ corresponds to a frequency in s$^{-1}$% because $t$ is in seconds.

Figure 54. An exponential decay and (right) the complex conjugate square of its Fourier transform which is a Lorentzian curve.

_____

The half-width at half-height of the transformed signal squared, is calculated as half the value at $k = 0$, which is $| g(0) |^2 /2 = 1/4\pi$ giving the half-width as $k_{1/2} = 1/\tau$ which is in inverse proportion to the excited-state lifetime. The shorter the lifetime, the wider will be the spectral line in accordance with $\Delta t \Delta \nu  = 1$, if we take the change in time to be the lifetime and the frequency to be proportional to $k$, which is reasonable because $k$ has units of s$^{-1}$.

The widening of the spectral line with a shortening of the excited state lifetime, is sometimes considered an example
of the 'time-energy' uncertainty principle, but this is not the same as the Heisenberg uncertainty principle. Spontaneous emission is caused by coupling the excited state to the states in the 'vacuum field', which have energy levels capable of being filled with photons in a similar way to how electronic energy levels can hold zero, one or two electrons or vibrational or rotational energy levels contain their respective quanta. The difference is that any number of photons can be contained in each of the energy levels because photons are bosons: rather like buses, they like to go around in groups.

The energy levels in this vacuum field fluctuate and so the coupling between these states and the excited state varies which causes the transition to have a real width. If, in addition, the excited state is short lived due to interactions within the atom or molecule or with other nearby species, or with external electric or magnetic fields, this also leads to fluctuations in the atom's energy levels and the greater these interactions are, the shorter the lifetime and the wider the energy width. This is called _lifetime broadening_. However, the average energy is always at exactly the transition frequency, and in our example, this is zero because the energy of the excited state above the ground state has not been specified.

The Heisenberg uncertainty principle arises because operators, such as position and momentum, do not commute, but in quantum mechanics time is not an operator, but an observable, therefore commutation does not apply. See Atkins & Friedmann (1997, p. 198) for a detailed discussion of the distinction between Heisenberg uncertainty and time-energy uncertainty.

**Q8 answer** **(a)** With the forward transform $\displaystyle g(\omega) =\frac{1}{\sqrt{2\pi}}\int_0^{t_0} e^{i\omega_0 t}e^{-i\omega t}dt$ which, being exponential, is easily integrated to

$$\displaystyle  g(\omega)=\frac{1}{\sqrt{2\pi}} \left. \frac{e^{i(\omega_0-\omega)t}}{ i(\omega_0-\omega) }  \right|_0^{t_0} = \frac{i}{\sqrt{2\pi}}\frac{1-e^{i(\omega_0 - \omega)t_0}}{\omega_0 - \omega}$$

The real and imaginary parts part can be obtained by converting the exponential to sin and cosine

$$\displaystyle  \frac{i}{\sqrt{2\pi}}\frac{1-e^{i(\omega_0 - \omega)t_0}}{\omega_0 - \omega} = \frac{1}{\sqrt{2\pi}(\omega_0 - \omega)}\left (i-ie^{i(\omega_0 - \omega)t_0}  \right ) = \frac{1}{\sqrt{2\pi}(\omega_0 - \omega)}\left (i-i\cos((\omega_0 - \omega)t_0)+\sin((\omega_0 - \omega)t_0)  \right ) $$

The square of the absolute value is obtained by multiplying the complex conjugate $g^*$ with $g$:

$$\displaystyle  |g(\omega)|^2= g(\omega)^*g(\omega)=\left[\frac{-i}{\sqrt{2\pi}}\frac{1-e^{-i(\omega_0 - \omega)t_0}}{\omega_0 - \omega} \right]\left[\frac{i}{\sqrt{2\pi}}\frac{1-e^{i(\omega_0 - \omega)t_0}}{\omega_0 - \omega}\right]=\frac{1-\cos((\omega_0-\omega)t_0)}{\pi (\omega_0-\omega)^2}$$

Plots of the pulse and its transforms are shown in fig 55. The real part becomes the sinc function if it is written as $\displaystyle \frac{t_0}{\sqrt{2\pi}}\frac{\sin(t_0(\omega_0-\omega)}{t_0(\omega_0-\omega))}$, see fig 15.

Figure 55. Left top: The electric field of a square pulse of 0.1 ps duration. Bottom right: the square of absolute value of the Fourier transform of the pulse; this is its spectrum. The small vertical line shows the position of the first minima, see part (b). Bottom left: The imaginary component of the Fourier transform, the sinc function, and right, the real part.

______

**(b)** Differentiating the transform finds the maxima and minima, either side of these are the missing frequencies as the period of the transform's oscillations is constant. Differentiating gives

$$\displaystyle \frac{d}{d\omega}\frac{\cos((\omega_0-\omega)t_0)-1}{\pi(\omega_0-\omega)^2} = -\frac{t_0\sin((\omega_0-\omega)t_0)}{\pi(\omega_0-\omega)^2}-2\frac{\cos((\omega_0-\omega)t_0)-1}{\pi(\omega_0-\omega)^3} $$

which must be zero at each of its minima. Multiplying out factors gives $\displaystyle (\omega_0-\omega)\sin((\omega_0-\omega)t_0)=2(1-\cos((\omega_0-\omega)t_0))$ which is solved when $t_0(\omega_0-\omega)=2\pi n$ because here the sine is zero, and when $\cos((\omega_0-\omega)t_0)=1$ which is when $n$ is an integer and $n=\pm 1$,  $\pm 2,\cdots$ or $\omega = \omega_0-2\pi n/t_0$ thus a series of zeros is expected. The pulse has a width of $0.1$ ps and zero's occur at $\omega=600\cdot 10^{12}\pm 2\pi n\cdot 10^{13}$ Hz and so the first zeros are at $\approx 662.8 \cdot 10^{12}$ Hz.

**Q9 answer** **(a)** As the pulse is square and of finite duration, it lasts only from time 0 to $\tau$, therefore its Fourier transform is

$$\displaystyle g(\omega)=\frac{1}{\sqrt{2\pi}}\int_0^\tau ae^{-i\omega t}dt =\frac{-a}{\sqrt{2\pi}} \left. \frac{e^{i\omega t}}{i\omega}  \right|_0^\tau = \frac{a}{\sqrt{2\pi}} \left( \frac{1-e^{i\omega \tau}}{i\omega}  \right)$$

which is similar to that in Q8 but with  the change $\omega\rightarrow \omega_0-\omega$.

**(b)** The real and imaginary parts are calculated using Euler's formula for the complex exponential giving

$$\left( \frac{1-e^{i\omega \tau}}{i\omega}  \right) =\frac{-i}{\omega}(1-\cos(\omega \tau)) +\frac{\sin(\omega \tau)}{\omega}$$

The real part $\displaystyle Re[g(\omega)]= \frac{a\tau}{\sqrt{2\pi}}\frac{\sin(\omega \tau)}{\omega}$ which can be converted to the sinc function by multiplying by $\tau/\tau$, see figs 15 and 55.

**(c, d)** The pulse is zero, $Re[g(\omega)]$ = 0, when $\displaystyle \frac{\sin(\omega \tau)}{\omega\tau}$ = 0 which does not occur when $\omega$ = 0 but for the first time when $\omega\tau = \pm \pi$ , because although the sine is zero, the denominator is $\pi$. L'Hopital's rule can also be used to determine the value when $\omega \to 0$:

$$\displaystyle \lim_{\omega \to 0}\frac{\sin(\omega\tau)}{\omega\tau}\to \frac{\tau\cos(\omega\tau)}{\tau} \to 1$$

The pulse has a frequency spread to the first minima of $2\pi/\tau$ which corresponds to $2\pi$ MHz for a $1\,\mu$s duration pulse. Suppose that only half this frequency range has useful intensity then this extends from $399.5 \to 400.5$ MHz in a $400$ MHz instrument. This gives a potential chemical shift of $\pm 50000/400 = 125$ ppm, which is large compared to most proton shifts which are typically $\lt 20$ ppm.

**Q 10 answer** **(a)** The FID is caused,in a semi-classical interpretation of NMR, by the rotating magnetization cutting the wire
coil of the detector, and inducing the current that is subsequently detected. The two different classes of spins move with different frequencies due to their different chemical environments, and hence, experience slightly different magnetic fields. These are the two cosines mentioned in the question. The decay of magnetization is due to the interaction of the spins with one another, with other atoms in the molecule and with dipoles in the solvent. These interactions have the effect of transferring more spins from the upper spin level to the lower one, than in the opposite direction, and hence back towards equilibrium. These effects cause the population to decay with a lifetime T1, and the loss of the spin coherence determines the T2 lifetime. The $\tau = 100$ in the question represents both these decay processes. In a real experiment the decay time would be of the order of a few seconds and the frequencies in MHz.

**(b)** Using the FID equation given in the question the Python code below can be used. You should modify this as you wish.

```python
fig1= plt.figure(figsize=(8, 8))
ax0 = fig1.add_subplot(2,2,1)    # make space for four plots 2 x 2 label axes 1 , 2, 3 & 4
ax1 = fig1.add_subplot(2,2,2)
ax2 = fig1.add_subplot(2,2,3)
ax3 = fig1.add_subplot(2,2,4)

nmax = 2**14                        # number of data points
T = 1000                            # max time
x = np.linspace(0, T, nmax)

tp = T/(1.0*nmax)                   # 1.0 to ensure real number division
freq = np.zeros(nmax,dtype=float)   # numpy array to hold data

for i in range(nmax):               # frequency is index/( (time/point)*number of points ), values 0 to (N-1)/N
    freq[i]= i/(tp*nmax)
    pass
nu1 = 1/11.0                        # frequency Hz 1        # use for fig 9.14 also
nu2 = 1/12.0
tau = 100.0
f01= lambda x: np.exp(-x/tau)*(np.cos(2*np.pi*nu1*x)+ np.cos(2*np.pi*nu2*x)) # lambda function

the_fft = np.fft.rfft(f01(x))       # use numpy to do the transform, has both imag and real parts
afft = the_fft.real
bfft = the_fft.imag
cfft = [ np.arctan( bfft[i]/afft[i]) for i in range( len(the_fft) ) ]     # phase

ax0.plot(x,f01(x))                  # now plot data and transforms
ax0.set_xlabel( 'time/s')
ax0.set_title('FID')
ax0.set_xlim([0,500])

ax1.plot(freq[0:nmax//2],afft[0:nmax//2],color='red')
ax1.set_title('spectrum')

ax2.plot(freq[0:nmax//2],bfft[0:nmax//2],color='red')
ax2.set_title('Im part of transform')

ax3.plot(freq[0:nmax//2],cfft[0:nmax//2],color='red')
ax3.set_ylim([-np.pi,np.pi])
ax3.set_title('spectral phase /radians')
ax3.set_ylabel('radian')

for ax in [ax0,ax1,ax2,ax3]:                # plot common features
    ax.axhline(0,color='gray',linewidth=1)  # plot zero line on each plot
    ax.spines['right'].set_visible(False)
    ax.spines['top'].set_visible(False)
    if ax != ax0:
        ax.axvline(nu1,linestyle='dashed',linewidth=1)
        ax.axvline(nu2,linestyle='dashed',linewidth=1)
        ax.set_xlabel('frequency /Hz')
        ax.set_xlim([0,0.2])
        pass

plt.tight_layout()
plt.show()
```

**Q11 answer**

The transform is $\displaystyle g(k)=\frac{1}{\sqrt{2\pi}}\int_{-t_0}^{t_0}\sin(2\pi t/t_0 +\varphi)e^{-ikt}dt$ which can be evaluated by hand or using SymPy. By hand this is made easier by converting the exponential first and then to sin and cos because we shall want to find the real and imaginary parts.

```python
t0,t,k,phi = symbols('t0 t k phi')

g = 1/(sqrt(2*pi))*integrate( sin(2*pi*t/t0 +phi)*(cos(k*t) +1j*sin(k*t)) ,(t,-t0,t0), conds='none')
simplify(g)
```

```python
re(g)    # sympy does basic conversion to separate real and imaginary parts.
```

**(b)** when $\varphi = n\pi$ and $n = 0, 1, 2, \cdots$ then the cosine term is always unity and the sine zero. Thus the transform is wholly imaginary, $\displaystyle g(k)=\frac{-4\pi t_0\sin(kt_0)}{\sqrt{2\pi}(k^2t_o^2-\pi^2)}$ and the form of the transform is a damped sine wave because the sine is divided by $k^2$.

```python
# real part of transform

freal= lambda k : -4*np.pi*t0*np.sin(k*t0)/(  np.sqrt(2*np.pi)*(k**2*t0**2-4.0*np.pi**2) )  # k is variable
t0 = 1.0
x  = np.linspace(0,40,100 )  # make 100 points 0 to 40

plt.plot(x,freal(x),color='blue')
plt.xlabel( 'k')
plt.title('Re(g(k))')
plt.axhline(0,linewidth=1,color='black')
plt.show()

```

**Q12 answer** **(a)** The Fourier transform of the $\delta$-function is obtained directly by using its selecting property. For instance acting on a function $f$ gives, $\displaystyle \int f(x)\delta(x-a)dx = f(a)$ and no integration is necessary, therefore

$$\displaystyle P_k=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty\delta(x-a)e^{ikx}dx = \frac{1}{\sqrt{2\pi}}e^{ika}$$

The probability density is $P_k^*P_k=1/2\pi$ because the complex conjugate is $\displaystyle  p^* = e^{+ikx}/\sqrt(2\pi)$. Because $P^*P$ is a constant the scattering off an atom is the same in all directions.

**(b)** The Gaussian function has the transform

$$\displaystyle P_k=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty e^{-(x-a)^2/d^2}e^{ikx}dx$$

calculating the integral is helped with substitution $s=(x-a)/d$ and using $\displaystyle \int_{-\infty}^{\infty}e^{-s^2}ds$=1. The result is $\displaystyle P_k=e^{-k^2d^2/4}e^{-ika}/2$.

The transform is itself a Gaussian, but in transform or $k$-space, which is in this case reciprocal distance,
the products $kd$ and $ka$ are dimensionless. The scattering intensity is $\displaystyle P_k^*P_k = e^{-kd^2/2}$. As with the $\delta$  function the scattering is the same in all directions because the power density does not depend on $x$. However, the power density now depends on the width of the Gaussian, the more spread out it is, hence large $d$, the smaller is the scattering. In the limit that the Gaussian becomes very narrow, it approximates the $\delta$-function.

**Q13 Answer**

Using the result from Q12, the transform is $\displaystyle \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}[\delta(x-a)+\delta(x+a)]e^{-ikx} dx = \frac{1}{\sqrt{2\pi}} (e^{-ika}+e^{+ika})$ and because $\displaystyle 2\cos(ka)=e^{-ika}+e^{+ika}$ the transform is $\displaystyle P_k=\frac{2}{\sqrt{2\pi}}\cos(ka)$.

The probability density is simply the square of this because $P$ is real, and shows that the scattering depends on the separation of the centres being maximal when $ka=\pi n$ if $n$ is an integer.

**Q14 answer**

The delta function at  position $a$, is $\delta (x-a)$, hence at position 2$a$ is $\delta (x - 2a)$, and at any position $n$, is $\delta (x − na)$). The comb of atoms is represented by the equation $\displaystyle C(x)=\sum_{n=-\infty}^\infty \delta (x-na)$. The Fourier transform of one atom at position $n$ is (see Q12),

$$\displaystyle P(k)= \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} \delta (x-na)e^{-ikx}dx = \frac{1}{\sqrt{2\pi}}e^{-ikna}$$

The total transfrom $\chi$ is therefore the sum of many of these terms, each term contributing $\displaystyle e^{-ikna}/\sqrt{2\pi}$ to the sum, hence for all positions,

$$\displaystyle \chi (k)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} C(x)e^{-ikx}dx = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} \sum_{n=-\infty}^{\infty}\delta (x-na)e^{-ikx}dx = \frac{1}{\sqrt{2\pi}}\sum_{n=-\infty}^{\infty}e^{ikna}$$

and because of the selecting property of the $\delta$-function it is not necessary to evaluate the integral as $\displaystyle \int f(x)\delta (x-a)=f(a)$.

At this point we need to decide what relative phases the X-rays can have. If this is left undefined then the summation turns out to be zero, because if the waves can have any phase whatsoever when they scatter, then they are on average, out of phase with one another, and add up to zero. To show this using standard summations (Chapter 5, Table 1) the summation can be split into two parts, one with $n$ from $-\infty \to 0$, the other from $0 \to \infty$. The term with $n = 0$ is then subtracted from the total as this is counted twice. The summation, however, comes to zero after one of the extra $n$ = 0 term is  subtracted.

An alternative, and more intuitive, way of performing the summation, is again to start with $\displaystyle \chi= \frac{1}{\sqrt{2\pi}}\sum_{n=-\infty}^{\infty}e^{ikna}$ then take pairs of functions with index values $\pm n$, starting at $\pm 1$, and then add the $n$ = 0 term and evaluate the result using the relationship
$\displaystyle 2 \cos(nx) = e^{inx} + e^{−inx}$. This gives $\displaystyle \chi(k)= \frac{1}{\sqrt{2\pi}}[1+2\sum_{n=1}^\infty \cos(nka)]$.

Next, by defining $ka = 2\pi m$, where $m$ is an integer, means that a wave with wavevector 2$\pi$/a will be exactly in phase with the next atom, and two such waves in phase between the next neighbouring atoms and so forth. The quantity $k = 2\pi m/a$, often called the wavevector, has units of reciprocal distance. The summation becomes

$$\displaystyle  \chi(m)= \frac{1}{\sqrt{2\pi}}[1+2\sum_{n=1}^\infty \cos(2\pi mn)]$$

and is finite for each integer value of $m$ and the cosines are all integer multiples of 2$\pi$ and are therefore unity. This means that the transform is a series of spikes at $m/a$, or at fixed intervals of 1/$a$, in reciprocal space. In practice, the scattering due to a one-dimensional comb of atoms depends on the angle of incidence of the X-rays and the Bragg condition. The scattering forms a cone with its apex on the atoms.

This calculation can be extended to two or three dimensions and to other patterns of gaps between 'atoms'. In three dimensions, a similar calculation illustrates that the process of diffraction, or scattering in X-ray crystallography produces data in reciprocal space.

If we calculate $\chi(m)$, starting with $n = 1 \to 50$ and $m = 0 \to 10$ a series of spikes is produced at $m = 0, 1, 2, \cdots$ and so forth, which become more like $\delta$ functions as $n$ increases. The next figure demonstrates this.

```python

m0   = 10   # maximum m to plot
nmax = 50   # summation max
chi = lambda m: 1.0 + 2.0*sum( [ np.cos(2.0*np.pi*m*n) for n in range(1,nmax)] )
m = np.linspace(0,m0,5000)

plt.plot(m,chi(m))
plt.title(r'$\chi(m) $')
plt.xlabel('m')

plt.show()
```

**Q15 answer**
**(a)** The pulse has an absolute value of $x$, so it is symmetrical about zero. The fwhm is $2\ln$(2) and as the pulse is $\tau= 10$ fs long the scaling is $a = \ln(2)/\tau$ in units of fs$^{-1}$. The Fourier transform can be taken from zero to infinity with $\displaystyle e^{-ax}$, and from $-\infty \cdots 0$ with $\displaystyle e^{ax}$, and the results added and simplified.

The transform is $\displaystyle \frac{1}{\sqrt(2\pi}\int_o^\infty e^{-ax}e^{-ikt}dt = \frac{1}{\sqrt(2\pi}\frac{a}{a+ik}$ and the integration gives $\displaystyle -\frac{1}{\sqrt(2\pi}\frac{a}{a-ik}$.

Adding these results produces $\displaystyle f_k=\frac{1}{\sqrt(2\pi}\frac{2a}{a^2+k^2}$. When plotted this looks something like a bell-shaped curve but is more pointed at the top; it has a Lorentzian shape. The full width at half maximum is $\displaystyle k_{fwhm}=2\left(  \frac{4a}{\sqrt{2\pi}}-a^2 \right) $ which for a $10$ fs pulse is $0.65 \cdot 10^{14}\, s^{-1}$. The central wavelength of the laser is $800$ nm or $12500\,\mathrm{ cm^{-1}}$, the frequency spread then $12500 \pm 0.6505 \cdot 10^{14}/c/2\, \mathrm{cm^{-1}}$ or $14670 \to 10329 \,\mathrm{cm^{-1}}$ which is $681.7 \to 968.1$ nm and covers most of the deep red part of the visible spectrum and spreads into a wavelength region where the eye is insensitive. As a consequence, lasers such as these, titanium sapphire is an example, appear very faint when in fact they could be very intense.

**(b)** The laser cavity is $1.5$ metres long, hence with these wavelengths, approximately $4.4 - 3.1 = 1.3$ million modes (half wavelengths) exist in the cavity and all these have to be in phase to form a pulse.


# SMC@FCC: Influence of Resampling

## Setup

```python
import numpy as np
from numpy.random import randn, choice

%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns

sns.set_style()
```

## Background

The idea is to solve the state filtering problem in the (possibly non-linear) state space model
$$
\begin{align}
    X_t \,\big|\, \left(X_{t - 1} = x_{t - 1}\right) &\sim p\left(x_t \,\big|\, x_{t - 1}\right) \\
    Y_t \,\big|\, \left(X_t = x_t\right) &\sim p\left(y_t \,\big|\, x_t\right)
\end{align}
$$
This means to find $p\left(x_t \,\big|\, y_{1:t}\right)$.

In the construction of e.g. the bootstrap particle filter, the role of resampling is emphasized. These notes are to investigate into the role of resampling in case of the very simple linear Gaussian state space model.

Assume here that
$$
\begin{align}
    X_{t + 1} &= 0.6 X_t + 0.5 E_t \\
    Y_t &= X_t + 0.8 V_t \\
    X_0 &\sim \mathcal{N}\left(0, 1\right)
\end{align}
$$
where $E_t, V_t \sim \mathcal{N}\left(0, 1\right)$. We then get
$$
\begin{align}
    p\left(x_t \,\big|\, x_{t - 1}\right) &= \mathcal{N}\left(0.6 x_{t - 1}, 0.5^2\right) \\
    p\left(y_t \,\big|\, x_t \right) &= \mathcal{N}\left(x_t, 0.8^2\right)
\end{align}
$$

## Simulate from the model

```python
# total amount of time-steps
tmax = 500
xs = np.zeros((tmax + 1,))
ys = np.zeros((tmax,))

xs[0] = randn()
for t in range(tmax):
    xs[t + 1] = 0.6 * xs[t] + 0.5 * randn()
    ys[t] = xs[t + 1] + 0.8 * randn()
```

## Visualise simulated time-series

```python
fig, axs = plt.subplots(2, 1, figsize=(12, 6))

axs[0].plot(range(tmax + 1), xs, 'o-', markersize=4)
axs[0].set_xlabel('Time');
axs[0].set_ylabel('$x_t$');

axs[1].plot(range(1, tmax + 1), ys, 'o-r', markersize=4)
axs[1].set_xlabel('Time');
axs[1].set_ylabel('$y_t$');

fig.tight_layout();
```

```python
fig
```

## Sequential importance sampling

```python
def sis(ys, n=200):
    # Length of data
    tmax = len(ys)

    # Pre-allocate
    xs = np.zeros((tmax + 1, n))
    ws = np.zeros((tmax + 1, n))
    wsnorm = np.zeros((tmax + 1, n))

    # Initialise
    xs[0, :] = randn(n)
    ws[0, :] = 1 / n * np.ones((n,))
    wsnorm[0, :] = ws[0, :]

    for t in range(tmax):
        # Propagate using proposal
        xs[t + 1, :] = 0.6 * xs[t, :] + 0.5 * randn(n)

        # Reweight
        tmp = (ys[t] - xs[t + 1, :]) / 0.8
        ws[t + 1, :] = ws[t, :] * \
            1 / (np.sqrt(2 * np.pi ) * 0.8) * \
            np.exp(-0.5 * tmp * tmp)
        # Normalize weights
        wsnorm[t + 1, :] = ws[t + 1, :] / np.sum(ws[t + 1, :])

    return xs, ws, wsnorm
```

```python
xs, ws, wsnorm = sis(ys, n=30)
```

```python
fig, axs = plt.subplots(1, 3, figsize=(14, 4))

axs[0].hist(wsnorm[2, :], normed=True, bins=30);
axs[1].hist(wsnorm[10, :], normed=True, bins=30);
axs[2].hist(wsnorm[100, :], normed=True, bins=30);
```

## Bootstrap particle filter

```python
def bpf(ys, n=200):
    # Length of data
    tmax = len(ys)

    # Pre-allocate
    xs = np.zeros((tmax + 1, n)); ancs = np.zeros((tmax + 1, n), dtype=np.int32)
    ws = np.zeros((tmax + 1, n)); wsnorm = np.zeros((tmax + 1, n))

    # Initialise
    xs[0, :] = randn(n)
    ws[0, :] = 1 / n * np.ones((n,))
    wsnorm[0, :] = ws[0, :]
    ancs[0, :] = range(n)

    for t in range(tmax):
        # Propagate using proposal
        xs[t + 1, :] = 0.6 * xs[t, ancs[t, :]] + 0.5 * randn(n)

        # Reweight, multiplying with previous weights not necessary since always 1 / n
        tmp = (ys[t] - xs[t + 1, :]) / 0.8
        ws[t + 1, :] = 1 / (np.sqrt(2 * np.pi ) * 0.8) * \
            np.exp(-0.5 * tmp * tmp)
        # Normalize weights
        wsnorm[t + 1, :] = ws[t + 1, :] / np.sum(ws[t + 1, :])

        # Resample
        ancs[t + 1, :] = choice(range(n), size=n, replace=True, p=wsnorm[t + 1, :])

    return xs, ancs, ws, wsnorm
```

```python
xs, ancs, ws, wsnorm = bpf(ys)
```

```python
3fig, axs = plt.subplots(1, 3, figsize=(14, 4))

axs[0].hist(wsnorm[2, :], bins=30);
axs[1].hist(wsnorm[10, :], bins=30);
axs[2].hist(wsnorm[100, :], bins=30);
```


```python
# install the tools you will use laterz
!pip install --upgrade setuptools
!pip install --upgrade pip
!pip install nashpy
```

    Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (57.1.0)
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
    Requirement already satisfied: pip in /usr/local/lib/python3.7/dist-packages (21.1.3)
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
    Requirement already satisfied: nashpy in /usr/local/lib/python3.7/dist-packages (0.0.21)
    Requirement already satisfied: scipy>=0.19.0 in /usr/local/lib/python3.7/dist-packages (from nashpy) (1.4.1)
    Requirement already satisfied: numpy>=1.12.1 in /usr/local/lib/python3.7/dist-packages (from nashpy) (1.19.5)
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

# Abstract the Game

## Game illustration

```python
#@title Table1: Prisoner's Dilemma
# import module
from tabulate import tabulate
table = [['','','Prisoner B',''],
         ['','','confess','deny'],
         ['Prisoner A','confess','-3,-3','0,-6'],
         ['','deny','-6,0','-1,-1']]
print(tabulate(table,tablefmt='grid'))
```

    +------------+---------+------------+-------+
    |            |         | Prisoner B |       |
    +------------+---------+------------+-------+
    |            |         | confess    | deny  |
    +------------+---------+------------+-------+
    | Prisoner A | confess | -3,-3      | 0,-6  |
    +------------+---------+------------+-------+
    |            | deny    | -6,0       | -1,-1 |
    +------------+---------+------------+-------+

```python
#@title Table1: Prisoner's Dilemma
import pandas as pd
import numpy as np
df2 = pd.DataFrame([['-3,-3','0,-6'],
                    ['-6,0','-1,-1']],
                  columns = [['Prisoner B','Prisoner B'],['confess','deny']],
                  index = [['Prisoner A','Prisoner A'],['confess','deny']])
df2
```

      Prisoner B

      confess
      deny

      Prisoner A
      confess
      -3,-3
      0,-6

      deny
      -6,0
      -1,-1

## Define the Game Object

```python
import nashpy as nash
import numpy as np

# Creater the game with the payoff matrix

A = np.array([[-3,0],[-6,-1]]) # A is the row player (in this case, prisoner)
B = np.array([[-3,-6],[0,-1]]) # B is the column player

game1 = nash.Game(A,B)
game1
```

    Bi matrix game with payoff matrices:

    Row player:
    [[-3  0]
     [-6 -1]]

    Column player:
    [[-3 -6]
     [ 0 -1]]

# Solve for Best Responses

```python
pip install quantecon
```

    Collecting quantecon
      Downloading quantecon-0.5.1-py3-none-any.whl (240 kB)
         |████████████████████████████████| 240 kB 9.9 MB/s
    Requirement already satisfied: numba>=0.38 in /usr/local/lib/python3.7/dist-packages (from quantecon) (0.51.2)
    Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from quantecon) (2.23.0)
    Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from quantecon) (1.19.5)
    Requirement already satisfied: sympy in /usr/local/lib/python3.7/dist-packages (from quantecon) (1.7.1)
    Requirement already satisfied: scipy>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from quantecon) (1.4.1)
    Requirement already satisfied: llvmlite<0.35,>=0.34.0.dev0 in /usr/local/lib/python3.7/dist-packages (from numba>=0.38->quantecon) (0.34.0)
    Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from numba>=0.38->quantecon) (57.1.0)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->quantecon) (3.0.4)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->quantecon) (2.10)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->quantecon) (1.24.3)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->quantecon) (2021.5.30)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy->quantecon) (1.2.1)
    Installing collected packages: quantecon
    Successfully installed quantecon-0.5.1
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

```python
# import the package
import quantecon.game_theory as gt
```

    /usr/local/lib/python3.7/dist-packages/numba/np/ufunc/parallel.py:363: NumbaWarning: The TBB threading layer requires TBB version 2019.5 or later i.e., TBB_INTERFACE_VERSION >= 11005. Found TBB_INTERFACE_VERSION = 9107. The TBB threading layer is disabled.
      warnings.warn(problem)

```python
coordination_game_matrix = [[4, 0],
                            [3, 2]]  # square matrix
g_Coo = gt.NormalFormGame(coordination_game_matrix)
```

```python
print(g_Coo)

```

    2-player NormalFormGame with payoff profile array:
    [[[4, 4],  [0, 3]],
     [[3, 0],  [2, 2]]]

```python
g_Coo.players[0].payoff_array  # Player 0's payoff array
```

    array([[4, 0],
           [3, 2]])

```python
g_Coo.players[1].payoff_array  # Player 1's payoff array
```

    array([[4, 0],
           [3, 2]])

```python
g_Coo.players[0].best_response(1)
```

    1

```python
g_Coo.players[0].best_response(0)
```

    0

```python
g_Coo.players[0].best_response([0.5,0.5])
```

    1

# Solve for Nash Equilibrium

## solve for the Nash Equilibrium

```python
#@title Table1: Prisoner's Dilemma
# import module
from tabulate import tabulate
table = [['','','Prisoner B',''],
         ['','','confess','deny'],
         ['Prisoner A','confess','-3,-3','0,-6'],
         ['','deny','-6,0','-1,-1']]
print(tabulate(table,tablefmt='grid'))
```

    +------------+---------+------------+-------+
    |            |         | Prisoner B |       |
    +------------+---------+------------+-------+
    |            |         | confess    | deny  |
    +------------+---------+------------+-------+
    | Prisoner A | confess | -3,-3      | 0,-6  |
    +------------+---------+------------+-------+
    |            | deny    | -6,0       | -1,-1 |
    +------------+---------+------------+-------+

```python
# find the Nash equilibrium with Support Enumeration

equilibria = game1.support_enumeration()
for eq in equilibria:
    print(eq)
```

    (array([1., 0.]), array([1., 0.]))

## calculate utility

```python
# Calculate Utilities

sigma_a = np.array([1,0])
sigma_b = np.array([1,0])
pd = nash.Game(A, B)
pd[sigma_a, sigma_b]
```

    array([-3, -3])

## abstract the game

```python
#@title Table1: Prisoner's Dilemma
# import module
from tabulate import tabulate
table = [['','','Player B',''],
         ['','','Heads','Tails'],
         ['Player A','Heads','1,-1','-1,1'],
         ['','Tails','-1,1','1,-1']]
print(tabulate(table,tablefmt='grid'))
```

    +----------+-------+----------+-------+
    |          |       | Player B |       |
    +----------+-------+----------+-------+
    |          |       | Heads    | Tails |
    +----------+-------+----------+-------+
    | Player A | Heads | 1,-1     | -1,1  |
    +----------+-------+----------+-------+
    |          | Tails | -1,1     | 1,-1  |
    +----------+-------+----------+-------+

```python
#@title Table1: Prisoner's Dilemma
import pandas as pd
import numpy as np
df2 = pd.DataFrame([['1,-1','-1,1'],
                    ['-1,1','1,-1']],
                  columns = [['Player B','Player B'],['Head','Tail']],
                  index = [['Player A','Player A'],['Head','Tail']])
df2
```

      Player B

      Head
      Tail

      Player A
      Head
      1,-1
      -1,1

      Tail
      -1,1
      1,-1

## Define the Game Object

# Example 1: Matching Pennies

```python
import nashpy as nash
import numpy as np

# Creater the game with the payoff matrix

A = np.array([[1,-1],[-1,1]]) # A is the row player (in this case, prisoner)
B = np.array([[-1,1],[1,-1]]) # B is the column player

game2 = nash.Game(A,B)
game2
```

    Zero sum game with payoff matrices:

    Row player:
    [[ 1 -1]
     [-1  1]]

    Column player:
    [[-1  1]
     [ 1 -1]]

## calculating best response

```python
matching_pennies_bimatrix = [[(1, -1), (-1, 1)],
                             [(-1, 1), (1, -1)]]
g_MP = gt.NormalFormGame(matching_pennies_bimatrix)
```

```python
print(g_MP)
```

    2-player NormalFormGame with payoff profile array:
    [[[ 1, -1],  [-1,  1]],
     [[-1,  1],  [ 1, -1]]]

```python
print(g_MP.players[0])  # Player instance for player 0
```

    Player in a 2-player normal form game with payoff array:
    [[ 1, -1],
     [-1,  1]]

```python
print(g_MP.players[1])  # Player instance for player 1
```

    Player in a 2-player normal form game with payoff array:
    [[-1,  1],
     [ 1, -1]]

```python
g_MP.players[0].payoff_array  # Player 0's payoff array
```

    array([[ 1, -1],
           [-1,  1]])

```python
g_MP.players[1].payoff_array  # Player 1's payoff array
```

    array([[-1,  1],
           [ 1, -1]])

```python
g_MP[0, 0]  # payoff profile for action profile (0, 0)
```

    array([ 1, -1])

```python
g_MP.players[0].best_response(1)
```

    1

```python
# By default, returns the best response action with the smallest index
g_MP.players[0].best_response([0.5, 0.5])
```

    0

```python
# With tie_breaking='random', returns randomly one of the best responses
g_MP.players[0].best_response([0.5, 0.5], tie_breaking='random')  # Try several times
```

    1

```python

# With tie_breaking=False, returns an array of all the best responses
g_MP.players[0].best_response([0.5, 0.5], tie_breaking=False)
```

    array([0, 1])

```python
g_MP.is_nash(([0.5, 0.5], [0.5, 0.5]))
```

    True

```python
g_MP.is_nash((0, 0))
```

    False

```python
g_MP.is_nash((0, [0.5, 0.5]))
```

    False

## Solve for Nash Equilibrium

```python
# find the Nash equilibrium with Support Enumeration

equilibria = game2.support_enumeration()
for eq in equilibria:
    print(eq)
```

    (array([0.5, 0.5]), array([0.5, 0.5]))

```python
def print_pure_nash_brute(g):
    """
    Print all pure Nash equilibria of a normal form game found by brute force.

    Parameters
    ----------
    g : NormalFormGame

    """
    NEs = gt.pure_nash_brute(g)
    num_NEs = len(NEs)
    if num_NEs == 0:
        msg = 'no pure Nash equilibrium'
    elif num_NEs == 1:
        msg = '1 pure Nash equilibrium:\n{0}'.format(NEs)
    else:
        msg = '{0} pure Nash equilibria:\n{1}'.format(num_NEs, NEs)

    print('The game has ' + msg)
```

```python
print_pure_nash_brute(g_MP)
```

    The game has no pure Nash equilibrium

## Caculating the Utility

```python
# Calculate Utilities

sigma_a = np.array([0.5,0.5])
sigma_b = np.array([0.5,0.5])
pd = nash.Game(A, B)
pd[sigma_a, sigma_b]
```

    array([0., 0.])

```python

```


```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
x = np.linspace(2, 2*np.pi)
len(x)
print(x[:5])
```

    [ 2.          2.08741195  2.17482389  2.26223584  2.34964778]

```python
x = np.linspace(0, 2*np.pi, 100)
len(x)
```

    100

```python
y = np.sin(x)
len(y)
```

    100

```python
y[:5]
```

    array([ 0.        ,  0.06342392,  0.12659245,  0.18925124,  0.25114799])

```python
plt.figure(figsize=(12,4))
plt.plot(x,y, "x-" , ms=7, label = "$\sin(x)$")
plt.plot(x,np.cos(x), label = "$\cos(x)$", lw = 5)
plt.xlabel("$x$", fontsize = 18)
plt.ylabel("$y$", fontsize = 18)
plt.title("Funciones")
plt.legend()
plt.grid()
plt.plot(x,0*x, lw = 1)
plt.show()

```

```python
import sympy as sym
sym.init_printing(use_latex='mathjax')
```

```python
sym.var('x')
sym.sin(x)*np.sin(3)
```

$$0.141120008059867 \sin{\left (x \right )}$$

```python
sym.var("x", real = True)
```

$$x$$

```python
f= x**2; f
```

$$x^{2}$$

```python
df = sym.diff(f,x,1)
```

```python
# resolver f'(x)=0 y mostrar soluciones
xc = sym.solve(df, x)
xc
```

$$\left [ 0\right ]$$

```python
# convertir f e una función que se pueda evaluar numéricamente

f_num = sym.lambdify([x], f, 'numpy')
x_vec = np.linspace(-5, 5, 100)

# graficar
plt.plot(x_vec, f_num(x_vec), color = "cyan")
plt.grid()
plt.xlabel('$x$', fontsize = 18)
plt.ylabel('$x^2$', fontsize = 18)
plt.show()
```

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
def f(x):
    return x**2
```

```python
def h(x,y):
    return np.sin(x)/np.tan(y)
```

```python
x = np.linspace(-5,5)
plt.plot(x,f(x),"--")
plt.plot(x,h(x,.1), "-.")
plt.plot(x,h(x,.4), "-")
plt.scatter(0,0,marker = "D",s=50,lw=50)
plt.show
```

```python

```


# Characterization of Systems in the Spectral Domain

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Communications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Combination of Systems

The representation of complex systems as combination of simpler systems is often convenient for the analysis or synthesis. This section discusses three of the most common combinations, the series or parallel connection of systems as well as feedback loops.

### Concatenation

When two systems are combined in series by connecting the output of the first system to the input of a second system this is termed as *concatenation* of two systems. Denoting the impulse responses of the two systems as $h_1(t)$ and $h_2(t)$, the output signal $y(t)$ of the second system is given as

\begin{equation}
y(t) =  x(t) * h_1(t) * h_2(t)
\end{equation}

where $x(t)$ denotes the input signal of the first system. Laplace transformation of the respective signals and impulse responses, and repeated application of the convolution theorem yields

\begin{equation}
Y(s) = \underbrace{H_1(s) \cdot H_2(s)}_{H(s)} \cdot X(s)
\end{equation}

It can be concluded that the concatenation of two systems can be regarded as one system with the transfer function $H(s) = H_1(s) \cdot H_2(s)$. Hence, the following structures are equivalent

The extension to a concatenation of $N$ systems is straightforward. The overall transfer function is given by multiplication of all the individual transfer functions $H_n(s)$

\begin{equation}
H(s) = \prod_{n=1}^{N} H_n(s)
\end{equation}

Applications of concatenated systems include for instance the modeling of wireless transmission systems and cascaded filters.

**Example**

Concatenation of LTI systems can be used to construct higher-order filters from lower-order prototypes. Such filters are known as *cascaded filters*. This is illustrated at the [example of the second-order low-pass filter](../laplace_transform/network_analysis.ipynb#Example:-Second-Order-Low-Pass-Filter) introduced before. The transfer function $H_0(s)$ of the low-pass is given as

\begin{equation}
H_0(s) = \frac{1}{LC s^2 + RC s + 1}
\end{equation}

where $R$, $L$ and $C$ denote the values of the resistor, capacitor and inductor. Concatenation of $N$ second-order filters leads to filter with order $2 N$. Its transfer function reads

\begin{equation}
H_N(s) = \left( \frac{1}{LC s^2 + RC s + 1} \right)^N
\end{equation}

The resulting transfer function is illustrated by its [Bode plot](../systems_spectral_domain/transfer_function.ipynb#Bode-Plots) for a varying number of cascaded filters using the normalized values $L = .5$, $R = 1$, $C = .4$. First the transfer function $H_N(s)$ is defined

```python
%matplotlib inline
import sympy as sym
sym.init_printing()

s = sym.symbols('s', complex=True)
w, R, L, C = sym.symbols('omega R L C', real=True)
N = sym.symbols('N', integer=True)

H0 = 1/(L*C*s**2 + R*C*s + 1)
HN = H0**N
HN
```

The bode plot for the transfer function $H_N(j \omega)$ is shown for $N = \{1, 2, 3\}$ (red, green, blue line)

```python
RLC = {R: 1, L: sym.Rational('.5'), C: sym.Rational('.4')}
HNlog = 20*sym.log(sym.Abs(HN.subs(RLC).subs(s, 2*sym.pi*w)), 10)

p1 = sym.plot(HNlog.subs(N, 1), (w, 0.1, 100), xscale='log', xlabel='$\omega$', ylabel='$20 \log_{10} | H_n(j \omega) | in dB$', line_color='r', show=False);
p2 = sym.plot(HNlog.subs(N, 2), (w, 0.1, 100), xscale='log', xlabel='$\omega$', ylabel='$20 \log_{10} | H_n(j \omega) | in dB$', line_color='g', show=False);
p3 = sym.plot(HNlog.subs(N, 3), (w, 0.1, 100), xscale='log', xlabel='$\omega$', ylabel='$20 \log_{10} | H_n(j \omega) | in dB$', line_color='b', show=False);
p1.extend(p2)
p1.extend(p3)
p1.show()
```

**Exercise**

* Derive the magnitude $|H(s)|$ and phase $\varphi(s)$ of a concatenated system.
* Using the result from the first exercise, how will the phase of the cascaded filter develop for an increasing number $N$ of cascaded filters.

### Parallel Connection

A structure where two systems share the same input signal and the output signals are superimposed is called *parallel connection*. The overall output signal $y(t)$ is given as the superposition of the output signals of the individual systems

\begin{equation}
y(t) = h_1(t) * x(t) + h_2(t) * x(t)
\end{equation}

Laplace transformation of the respective signals and impulse responses, and application of the convolution theorem yields

\begin{equation}
Y(s) = \underbrace{\left( H_1(s) + H_2(s)  \right)}_{H(s)} \cdot X(s)
\end{equation}

The overall transfer function $H(s)$ of a parallel connection of two systems is given as the superposition of the transfer functions of the individual systems. Hence, the following structures are equivalent

The extension to a parallel connection of $N$ systems is straightforward. The overall transfer function is given by superposition of all individual transfer functions $H_n(s)$

\begin{equation}
H(s) = \sum_{n=1}^{N} H_n(s)
\end{equation}

A prominent application of a parallel connection of systems are [filter banks](https://en.wikipedia.org/wiki/Filter_bank) as used in signal analysis and many lossy coding schemes.

### Feedback

The connection of two systems, where the input of the second system is connected to the output of the first and the output of the second system is superimposed to the input of the first is called *feedback loop*. This structure is depicted in the following illustration (upper block diagram)

The output signal $y(t)$ is given as

\begin{equation}
y(t) = x(t) * h_1(t) + y(t) * h_2(t) * h_1(t)
\end{equation}

Laplace transform of the signals and impulse responses, application of the convolution theorem and rearrangement of terms yields

\begin{equation}
Y(s) = \frac{H_1(s)}{1 - H_1(s) \cdot H_2(s)} \cdot X(s)
\end{equation}

The overall transfer function $H(s)$ of the feedback loop is then given as

\begin{equation}
H(s) = \frac{H_1(s)}{1 - H_1(s) \cdot H_2(s)}
\end{equation}

This equivalence is depicted by the lower block diagram of above structure.

Applications of feedback loops include [control systems](https://en.wikipedia.org/wiki/Control_system) and the design of amplifiers.

**Copyright**

The notebooks are provided as [Open Educational Resource](https://de.wikipedia.org/wiki/Open_Educational_Resources). Feel free to use the notebooks for your own educational purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Lecture Notes on Signals and Systems* by Sascha Spors.


```python
import numpy as np
import matplotlib.pyplot as plt
plt.style.use('apw-notebook.mplstyle')
%matplotlib inline
np.random.seed(1234)
```

$$
\newcommand{\mat}[1]{\boldsymbol{\rm #1}}
\newcommand{\vec}[1]{\boldsymbol{#1}}
\newcommand{\T}[1]{{#1}^{\mathsf{T}}}
\newcommand{\inv}[1]{{#1}^{-1}}
$$

# Gaussian processes

GPs are a type of very flexible models that are primarily useful for prediction, discovery, and making models robust to unknown error properties. A lot of the GP literature is focused around time-series analysis, but they are useful in other contexts as well in which one variable (the analog to 'time') is very precisely measured or perfectly known. They are pretty slow to compute, but people like Dan Foreman-Mackey take speed very seriously (see [george](http://github.com/dfm/george) and [celerite](http://celerite.readthedocs.io/)).

The default resource for Gaussian processes is [Rasmussen & Williams](http://www.gaussianprocess.org/gpml/).

In previous classes we talked about the problem of estimating the mean flux of a star given noisy measurements. Recall that we assumed that the data were independent, uncorrelated, and Gaussian, and with those assumptions the likelihood over all data was separable into a product over individual (per–datum) Gaussians. For $N$ data points labeled $n$, we wrote:

$$
p(\{f_n\} \,|\, f_0, \sigma_n^2) = \prod_n^N \mathcal{N}(f_n \,|\, f_0, \sigma_n^2) \\
\ln p(\{f_n\} \,|\, f_0, \sigma_n^2) = -\frac{1}{2}\sum_n^N \left[\frac{(f_n - f_0)^2}{\sigma_n^2} + \ln\left(2\pi \,\sigma_n^2\right)\right]
$$

Remember that we can also write this log-likelihood using matrix notation by defining a _residual vector_, $\vec{r} = \vec{f} - f_0$, and a _covariance matrix_, $\mat{C}$:

$$
\begin{align}
\vec{f} &= \begin{bmatrix}
             f_{1} \\
             f_{2} \\
             \vdots \\
             f_{N}
           \end{bmatrix}\\
\mat{C} &= \begin{bmatrix}
             \sigma_1^2 & 0 & \cdots & 0 \\
             0 & \sigma_2^2 & \cdots & 0  \\
             \vdots & \vdots & \ddots & \vdots \\
             0 & 0 & \cdots & \sigma_{N}^2
           \end{bmatrix}\\
\end{align}
$$

With these objects, the log-likelihood is:

$$
\ln p(\vec{f} \,|\, f_0, \mat{C}) = -\frac{1}{2} \T{\vec{r}} \, \inv{\mat{C}} \, \vec{r}
    - \frac{1}{2} \ln\det\mat{C} - \frac{N}{2}\ln 2\pi
$$

Writing it like this you can interpret the data vector $\vec{f}$ as being a single draw from an $N$-dimensional Gaussian with mean $f_0$ and (diagonal) covariance matrix $\mat{C}$. The diagonality of the covariance matrix is a result of our assumption that the data are uncorrelated or independent. But what if the data are correlated in some way? Or, what if there are off-diagonal elements in the covariance matrix? Let's look at both cases:

```python
# for simplicity, set f_0 to 0
f0 = 0.
n_data = 64

times = np.random.uniform(0, 100, n_data)
times.sort()

y_err = np.random.uniform(0.1, 1., n_data)

# first construct a purely diagonal covariance matrix
C1 = np.diag(y_err**2)

# now create a covariance matrix with correlated noise
C2 = 8 * np.exp(-0.5 * (times[:,None]-times[None,:])**2 / 10.**2)
C2[np.diag_indices_from(C2)] += y_err**2
```

```python
fig,axes = plt.subplots(2, 1, figsize=(5,10))
axes[0].imshow(C1, cmap='Greys_r')
axes[1].imshow(C2, cmap='Greys_r')
for ax in axes:
    ax.xaxis.set_visible(False)
    ax.yaxis.set_visible(False)
fig.tight_layout()
```

```python
f1 = np.random.multivariate_normal(np.full(n_data, f0), C1)
f2 = np.random.multivariate_normal(np.full(n_data, f0), C2)

fig,axes = plt.subplots(2, 1, figsize=(8,8), sharex=True, sharey=True)

axes[0].errorbar(times, f1, yerr=y_err, marker='.', linestyle='none')
axes[1].errorbar(times, f2, yerr=y_err, marker='.', linestyle='none')

axes[0].set_ylim(-8,8)
```

If we knew this covariance matrix for the data, then doing least-squares fitting would be as simple as ever:

```python
def solve_linalg(times, flux, Cov):
    # construct design matrix and invert covariance matrix
    X = np.vander(times, N=1)
    Cinv = np.linalg.inv(Cov)

    best_f0 = np.linalg.inv(X.T @ Cinv @ X) @ (X.T @ Cinv @ flux)
    cov_f0 = np.linalg.inv(X.T @ Cinv @ X)

    return best_f0, cov_f0
```

```python
x0 = np.linspace(times.min(), times.max(), 1024)

fig,ax = plt.subplots(1, 1, figsize=(8,6))
ax.errorbar(times, f2, yerr=y_err, marker='.', linestyle='none')

for color, C in zip(['#31a354', '#de2d26'], [C2, C1]):
    best_f0, cov_f0 = solve_linalg(times, f2, C)
    samples = np.random.multivariate_normal(best_f0, cov_f0, 10000)

    lines = np.dot(np.vander(x0, 1), samples.T)
    q = np.percentile(lines, [16, 84], axis=1)

    ax.plot(x0, np.full_like(x0, best_f0), linewidth=2, alpha=0.8, color=color)
    ax.fill_between(x0, q[0], q[1], alpha=0.25, color=color)

ax.set_ylim(-8,8)
```

But what if we don't know the noise structure? For example:

```python
from astropy.io import fits
data = fits.getdata('../data/k2.fits')
```

```python
data = data[np.isfinite(data['PDCSAP_FLUX'])]
raw_time = data['TIME'].astype(np.float64)
raw_flux = data['PDCSAP_FLUX'].astype(np.float64)
raw_flux_err = data['PDCSAP_FLUX_ERR'].astype(np.float64)
```

```python
med = np.median(raw_flux)
robust_std = 1.48 * np.median(np.abs(raw_flux - med))
```

```python
idx = (raw_flux < (med + 5*robust_std)) & (raw_flux > (med - 5*robust_std))

trim = 512
time = raw_time[idx][:trim] # too much data
flux = raw_flux[idx][:trim]
flux_err = raw_flux_err[idx][:trim]

time = time - time.min()
```

```python
plt.figure(figsize=(12,6))
plt.errorbar(time, flux, flux_err, marker='.', linestyle='none')
plt.xlabel('time')
plt.ylabel('flux')
```

We're only given single error-bars for each flux measurement, i.e. just the diagonal terms of the covariance matrix. But there is likely some very correlated noise in this light curve! What do we do? One (very bad) idea would be to add all off-diagonal elements to your model as parameters! For an $N\times N$ symmetric matrix, that amounts to adding $\frac{N\,(N-1)}{2}$ parameters to your model (and scales like the number of data points _squared_!). No one would seriously consider doing that...

Instead: what if we can write a simple parametrized _kernel function_, $k$, that generates the off-diagonal elements as a function of location in the matrix? That is:

$$
C_{ij} = \sigma_i^2 \, \delta_{ij} + k(t_i, t_j)
$$

We now just have to specify this function, $k$. This is where the magic comes in...the function must produce a symmetric, positive, semi-definite function, so there aren't too many options. We'll introduce a few others later, but for now let's use the _Squared Exponential Kernel_:

$$
k(x_i, x_j) = a^2 \, \exp\left(-\frac{(x_i - x_j)^2}{2\,b^2}\right)
$$

(yes it's just another Gaussian...). This kernel says that data points near $x_i$ are correlated with a length scale set by the parameter $b$ and a correlation amplitude set by the parameter $a$.

__Exercise__: Implement this kernel function below

```python
def sq_exp_kernel(x_i, x_j, a, b):
    # TODO: Delete the line below and fill in this function
    pass
```

```python
a = 4.
b = 1.
Cov = sq_exp_kernel(time[:,None], time[None], a, b)

plt.figure(figsize=(6,6))
plt.imshow(Cov, cmap='Greys_r')
plt.gca().xaxis.set_visible(False)
plt.gca().yaxis.set_visible(False)
plt.tight_layout()
```

We'll now implement a log-likelihood function for a Gaussian process with a simple constant mean model (same as we did above for inferring the mean flux):

$$
\ln p(\vec{f} \,|\, f_0, \mat{C}) = -\frac{1}{2} \T{\vec{r}} \, \inv{\mat{C}} \, \vec{r}
    - \frac{1}{2} \ln\det\mat{C} - \frac{N}{2}\ln 2\pi
$$

Replacing $\vec{C}$ with the sum of the diagonal covariance matrix with that generated by the kernel function.

The parameters that come in will be $(f_0, \ln a, \ln b)$. You'll need to invert a large matrix in the likelihood call. Rather than using `numpy.linalg.inv`, I recommend using `numpy.linalg.solve` in the following way. As a part of the likelihood expression, you need to compute the vector $\vec{y}$ in:

$$
\vec{y} = \inv{\mat{C}}\,\vec{r}\\
\vec{r} = \mat{C}\,\vec{y}
$$

Instead of explicitly doing the inversion, you can do:
$$
\vec{y} = {\rm solve}\left(\mat{C}, \vec{r}\right)
$$

Also, instead of taking the log of the determinant, use `numpy.linalg.slogdet` (which is much more numerically stable).

__Exercise__: Implement the likelihood function below

```python
def ln_likelihood(pars, t, flux, flux_err):
    # TODO: Delete the line below and fill in this function
    pass
```

We now need to implement priors for the parameters. We'll assume a uniform prior on $f_0$ and uniform priors on the log hyper-parameters $\ln a$ and $\ln b$. Choose wide and sensible ranges for the parameters.

__Exercise:__ Implement the prior function below

```python
def ln_prior(pars):
    # TODO: Delete the line below and fill in this function
    pass
```

Now we can run MCMC to infer the parameters:

```python
import emcee
```

```python
def ln_posterior(pars, t, flux, flux_err):
    lp = ln_prior(pars)
    if not np.isfinite(lp):
        return -np.inf

    ll = ln_likelihood(pars, t, flux, flux_err)
    if not np.isfinite(ll):
        return -np.inf

    return lp + ll
```

```python
nwalkers = 32
p0 = [np.median(flux), 4., -2.]

sampler = emcee.EnsembleSampler(nwalkers, dim=len(p0), lnpostfn=ln_posterior,
                                args=(time, flux, flux_err))
```

__Warning__: this could take a few minutes...

```python
_p0 = emcee.utils.sample_ball(p0, std=[1E-3]*len(p0), size=nwalkers)

pos,_,_ = sampler.run_mcmc(_p0, 256)
sampler.reset()
_ = sampler.run_mcmc(pos, 256)
```

```python
for k in range(len(p0)):
    plt.figure()
    for walker in sampler.chain[...,k]:
        plt.plot(walker, marker='', drawstyle='steps-mid', color='k', alpha=0.1)
```

We'll use [@dfm](https://github.com/dfm)'s Gaussian process package [George](https://github.com/dfm/george) to plot samples from the GP at each choice of the hyper-parameters $a, b$:

```python
import george
```

```python
def plot_gp(grid, time, flux, flux_err, f0, a, b):
    K = a**2 * george.kernels.ExpSquaredKernel(b)
    gp = george.GP(K, mean=f0)
    gp.compute(time, flux_err)
    return gp.predict(flux, grid, mean_only=True)
```

```python
plt.figure(figsize=(12,6))
plt.errorbar(time, flux, flux_err, marker='.', linestyle='none')

t_grid = np.linspace(time.min(), time.max(), 1024)
flatchain = np.vstack(sampler.chain[:,::32])

for i in np.random.choice(flatchain.shape[0], size=64, replace=False):
    f0,ln_a,ln_b = flatchain[i]
    mean = plot_gp(t_grid, time, flux, flux_err, f0, np.exp(ln_a), np.exp(ln_b))
    plt.plot(t_grid, mean, marker='', zorder=100, color='#2b8cbe', alpha=0.25)

plt.xlabel('time')
plt.ylabel('flux')
```

Now that we know how they work, let's use `George` to compute the likelihood (it's much faster!) and re-do the analysis above:

```python
from george import kernels
```

```python
def ln_likelihood2(pars, t, flux, flux_err):
    f_0 = pars[0]
    a, b = np.exp(pars[1:])

    # residual vector - you could replace with a more complicated mean model
    r = flux - f_0

    K = a**2 * kernels.ExpSquaredKernel(b)
    gp = george.GP(K)
    gp.compute(t, flux_err)
    return gp.lnlikelihood(flux - f_0)
```

```python
def ln_prior2(pars):
    f_0 = pars[0]

    if f_0 < 0 or f_0 > 1E10:
        return -np.inf

    for ln_p in pars[1:]:
        if ln_p < -10 or ln_p > 10:
            return -np.inf

    return 0.
```

```python
def ln_posterior2(pars, t, flux, flux_err):
    lp = ln_prior2(pars)
    if not np.isfinite(lp):
        return -np.inf

    ll = ln_likelihood2(pars, t, flux, flux_err)
    if not np.isfinite(ll):
        return -np.inf

    return lp + ll
```

```python
nwalkers = 32
p0 = [np.median(flux), 4., -2.]

sampler2 = emcee.EnsembleSampler(nwalkers, dim=len(p0), lnpostfn=ln_posterior2,
                                 args=(time, flux, flux_err))
```

```python
_p0 = emcee.utils.sample_ball(p0, std=[1E-3]*len(p0), size=nwalkers)

pos,_,_ = sampler2.run_mcmc(_p0, 256)
sampler2.reset()
_ = sampler2.run_mcmc(pos, 256)
```

```python
for k in range(len(p0)):
    plt.figure()
    for walker in sampler2.chain[...,k]:
        plt.plot(walker, marker='', drawstyle='steps-mid', color='k', alpha=0.1)
```

```python
flatchain = np.vstack(sampler2.chain[:,::32])
```

```python
plt.figure(figsize=(12,6))
plt.errorbar(time, flux, flux_err, marker='.', linestyle='none')

t_grid = np.linspace(time.min(), time.max(), 1024)
flatchain = np.vstack(sampler2.chain[:,::32])

for i in np.random.choice(flatchain.shape[0], size=64, replace=False):
    f0, ln_a, ln_b = flatchain[i]
    a = np.exp(ln_a)
    b = np.exp(ln_b)

    K = a**2 * kernels.ExpSquaredKernel(b)
    gp = george.GP(K, mean=f0)
    gp.compute(time, flux_err)
    mean = gp.predict(flux, t_grid, mean_only=True)
    plt.plot(t_grid, mean, marker='', zorder=100, color='#2b8cbe', alpha=0.25)

plt.xlabel('time')
plt.ylabel('flux')
```

We've been using an exponential-squared kernel function but I never really justified its use. That's a generic problem of GP models: there are a handful of popular kernel functions, but sometimes there's no real motivation to use one over another. In this case, however, the variability in our data (the "noise") clearly has contributions over a large range of frequencies. The kernel we're using has a fixed correlation length-scale, meaning it won't necessarily adapt well when there are sharp features and long-term trends.

__Exercise__: Re-do the analysis (using `George`) we did above but instead with the following kernel, known as the "Matérn 3/2":

$$
k(x_i, x_j) = a^2 \, \left(1 + \frac{\sqrt{3}\,\left|x_i-x_j\right|}{b}\right) \, \exp\left(-\frac{\sqrt{3}\,\left|x_i-x_j\right|}{b}\right)
$$

(see `george.kernels.Matern32Kernel`).

```python
# TODO: fill in here
```

```python

```

notebooks/Gaussian processes - workbook.ipynb