---
title: "Unit test for your Gibbs sampler"
excerpt: "Making your MCMC code ready for production."
---

In [Part 1]({{ site.baseurl }}{% post_url 2017-08-15-productionizing-Gibbs-sampler-integration-test %}), I discussed how to integration test a fast, no-frill Gibbs sampler. In essence, the integration test checks if the posterior estimates are "close enough" to the true values. Unfortunately, such a test can be misleading. Indeed:

- We can pass the test with wrong code. This happens when our model uses regularization or informative prior, causing the posterior estimates to be "close enough" even though the Gibbs sampler is wrong.
- We can fail the test with correct code. If the posterior density has weird geometry, our Gibbs sampler may not converge in time. In this case, the posterior estimates are not "close enough" even though the Gibbs sampler is correct.

Furthermore, the current implementation lacks maintainability:

- The integration test can take hours to derive the posterior samples, discouraging engineers from making incremental improvements to the code.
- The environment within the loop is littered with variables, making it difficult to make small changes without downstream effects.

To solve these problems, [Grosse and Duvenaud (2014)](https://arxiv.org/abs/1412.5218) demonstrates a unit-testing approach for MCMC code. In essence, a unit test requires our code to have an easy-to-check expected behavior. Such a behavior for the full conditional \\(p(\theta \| x)\\) is the following identity:[^1]

$$
\begin{align}
\frac{p(\theta_1 | x)}{p(\theta_2 | x)} &= \frac{p(\theta_1) p(x|\theta_1)}{p(\theta_2)p(x|\theta_2)} \qquad \forall \theta_1, \theta_2
\end{align}
$$

where \\(\theta_1, \theta_2\\) are two arbitrary values of \\(\theta\\).

This identity is perfect for a unit test for three reasons:
- The identity must hold *exactly*, so we don't rely on testing "close enough" behavior.
- The identity must hold for any two arbitrary values of \\(\theta\\), so mocking and setting up is minimal.
- The right hand side contains only terms that are easy to derive. Specifically, \\(p(\theta)\\) is the prior, which is whatever the distribution we choose. \\(p(x \| \theta)\\) is the likelihood, which comes directly from the model with no extra work.

To facilitate unit testing, we modularize the Gibbs sampler as follows. First, we have a `State` class that contains the current state of the parameters.

```python
import numpy as np
import scipy.stats

class State(object):
    def __init__(self, theta, sigma2):
        self.theta = theta
        self.sigma2 = sigma2
```

Second, we have a `Model` class that describes the model.

```python
class Model(object):
    def __init__(self, theta_prior, sigma2_prior, rng=None):
        self.theta_prior = theta_prior
        self.sigma2_prior = sigma2_prior
        self.rng = rng

    # Full Conditional for theta
    def cond_theta(self, state, data):
        n = len(data)
        mean_prior = self.theta_prior.args[0]
        variance_prior = self.theta_prior.args[1] ** 2

        variance_post = 1 / (1 / variance_prior + n / state.sigma2)
        mean_post = variance_post * (mean_prior / variance_prior + n * data.mean() / state.sigma2)
        return scipy.stats.norm(mean_post, np.sqrt(variance_post))

    # Full Conditional for sigma2
    def cond_sigma2(self, state, data):
        n = len(data)
        shape_prior = self.sigma2_prior.kwds['a']
        scale_prior = self.sigma2_prior.kwds['scale']

        shape_post = shape_prior + n / 2
        scale_post = scale_prior + np.sum((data - state.theta) ** 2) / 2
        return scipy.stats.invgamma(shape_post, scale=scale_post)

    # A gibbs step iterate through theta, sigma2 and update them
    def gibbs_step(self, state, data):
        state.theta = self.cond_theta(state, data).rvs(random_state=self.rng)
        state.sigma2 = self.cond_sigma2(state, data).rvs(random_state=self.rng)

    # Joint density, used as the right hand side in the identity to be used in the unit test
    def joint_log_p(self, state, data):
        return self.theta_prior.logpdf(state.theta) + \
               self.sigma2_prior.logpdf(state.sigma2) + \
               (scipy.stats.norm(loc=state.theta, scale=np.sqrt(state.sigma2))
                .logpdf(data).sum())
```

We can then test the identity above as follows. Note the use of `pytest`'s fixture to DRY. We randomize our model, state, and data to be extra sure that our Gibbs sampler is always correct.

```python
import pytest
import copy

@pytest.fixture
def random_model():
    theta_prior = scipy.stats.norm(np.random.uniform(), np.random.uniform())
    sigma2_prior = scipy.stats.invgamma(a=np.random.uniform(),
                                        scale=np.random.uniform())
    return Model(theta_prior=theta_prior,
                 sigma2_prior=sigma2_prior)

@pytest.fixture
def random_state():
    return State(np.random.uniform(), np.random.uniform())

@pytest.fixture
def random_data():
    return np.random.normal(size=10)

def test_cond_theta(random_model, random_state, random_data):
    cond = random_model.cond_theta(random_state, random_data)
    new_state = copy.deepcopy(random_state)
    new_state.theta = np.random.uniform()

    assert np.allclose(cond.logpdf(new_state.theta) - cond.logpdf(random_state.theta),
                       random_model.joint_log_p(new_state, random_data) - \
                       random_model.joint_log_p(random_state, random_data))

def test_cond_sigma2(random_model, random_state, random_data):
    cond = random_model.cond_sigma2(random_state, random_data)
    new_state = copy.deepcopy(random_state)
    new_state.sigma2 = np.random.uniform()

    assert np.allclose(cond.logpdf(new_state.sigma2) - cond.logpdf(random_state.sigma2),
                       random_model.joint_log_p(new_state, random_data) - \
                       random_model.joint_log_p(random_state, random_data))
```

Running `$ pytest` in the terminal from the project root folder will check whether we have implemented the Gibbs sampler correctly. Check out [my Github repo](https://github.com/LaDilettante/productionized-mcmc) to see how I arrange this code with to be tested with `pytest`.

## Check the no-frill Gibbs sampler against the modular Gibbs sampler

The modular Gibbs sampler is very legible and easy to test. However, due to various overhead with creating classes and objects, it can be slower that the no-frill Gibbs sampler that relies exclusively on `numpy`. Therefore, my preferred approach is

1. Optimize for legibility while writing the modular Gibbs
2. Optimize for performance while writing the no-frill Gibbs
3. Check that the no-frill Gibbs produces the same result as the modular Gibbs (by running both using the same seed)
4. Use the no-frill Gibbs in production

This way, we get the best of both worlds. Below I show how to carry out step (3), checking the no-frill Gibbs against the modular Gibbs.

```python
# Generate the data from known parameters
true_theta = 2
true_sigma2 = 3.5
y = np.random.normal(true_theta, np.sqrt(true_sigma2), size = 1000)

# Specify the prior
prior = {'mu_0': 10, 'tau2_0': 10000, 'nu_0': 1, 'sigma2_0': 10}
```

```python
# Run the modular Gibbs and store its result
class Storage(object):
    def __init__(self, S=0):
        self.states = [None] * S

    def add_state(self, state, i):
        self.states[i] = copy.copy(state)

    def __getattr__(self, param):
        return np.array([getattr(state, param) for state in self.states])

# Specify the Model, noticing the seed
my_model = Model(theta_prior=scipy.stats.norm(prior['mu_0'], np.sqrt(prior['tau2_0'])),
                 sigma2_prior=scipy.stats.invgamma(a=prior['nu_0'] / 2,
                                                   scale=(prior['nu_0'] * prior['sigma2_0']) / 2),
                 rng=42)

samples_gibbs_modular = Storage(2)
my_state = State(theta=y.mean(), sigma2=y.var())
samples_gibbs_modular.add_state(my_state, 0)
for i in range(1, 2):
    my_model.gibbs_step(my_state, y)
    samples_gibbs_modular.add_state(my_state, i)
```

```python
# Run the no-frill Gibbs and store its result
def gibbs_simple(S, y, prior, rng=None):
    """
    A no-frill Gibbs sampler

    Parameters
    ----------
    S : int, the number of samples
    y : data vector
    prior : dict, prior parameters
    rng : int, random seed
    """
    n = len(y)
    ybar = y.mean()

    # Initialize storage
    theta_samples = np.empty(S)
    sigma2_samples = np.empty(S)

    # Starting value as the sample variance and mean
    sigma2_samples[0] = y.var()
    theta_samples[0] = ybar

    # Big loop
    for s in range(1, S):
        # Update theta
        tau2_n = 1 / (1 / prior['tau2_0'] + n / sigma2_samples[s - 1])
        mu_n = tau2_n * (prior['mu_0'] / prior['tau2_0'] + n * ybar / sigma2_samples[s - 1])
        theta_samples[s] = scipy.stats.norm(mu_n, np.sqrt(tau2_n)).rvs(random_state=rng)

        # Update sigma2
        nu_n = prior['nu_0'] + n
        nu_sigma2_n = prior['nu_0'] * prior['sigma2_0'] + sum((y - theta_samples[s]) ** 2)
        sigma2_samples[s] = scipy.stats.invgamma(nu_n / 2, scale=nu_sigma2_n / 2).rvs(random_state=rng)

    return {'theta': theta_samples, 'sigma2': sigma2_samples}

# Notice that we set the same seed here as above
samples_gibbs_simple = gibbs_simple(2, y, prior, rng=42)
```

```python
assert np.allclose(samples_gibbs_simple['theta'], samples_gibbs_modular.theta)
assert np.allclose(samples_gibbs_simple['sigma2'], samples_gibbs_modular.sigma2)
```

In a real production environment, all these tests should be put in an automated test framework. (See [my Github repo](https://github.com/LaDilettante/productionized-mcmc) as an example).

## Postcript: Some tips for legible MCMC code

As mentioned above, while writing the modular Gibbs sampler, I optimize for legibility. Below are some tips on doing so.

### 1. Use Python `@property` to compute functions of your parameters

In our MCMC code, we frequently need to compute functions of our parameters. For example, while we parameterize our normal distribution with the variance parameter, `numpy`'s normal distribution may ask for scale, or we may want to extract the precision to monitor. In a simple program it's easy enough to write `sigma = np.sqrt(state.sigma2)` or `tau = 1 / state.sigma2` every time. But in a larger program we want to DRY ("Don't Repeat Yourself") and simply write `state.sigma` or `state.tau`. Besides all [the usual benefits of DRY](https://softwareengineering.stackexchange.com/questions/103233/why-is-dry-important), this also lets our code reads more like its statistical intent.

To create an instance variable (i.e. `sigma`) that is a function of another instance variable (i.e. `sigma2`), we take advantage of Python's `@property` decorator as follows.

```python
class State(object):
    def __init__(self, sigma2):
        self.sigma2 = sigma2

    @property
    def sigma(self):
        return np.sqrt(self.sigma2)

    @property
    def tau(self):
        return 1 / self.sigma2
```

We can set the value of `sigma2` ...

```python
my_state = State(sigma2 = 0.5 ** 2)
print("sigma2: {}".format(my_state.sigma2))
```

    sigma2: 0.25

... and `sigma` and `tau` will be automatically available

```python
print("sigma:  {}".format(my_state.sigma))
print("tau:    {}".format(my_state.tau))
```

    sigma:  0.5
    tau:    4.0

### 2. Wrap `scipy.stats` distributions into classes for flexibility in parameterization

A lot of the times `scipy.stats` distributions don't have the parameterization that I'd like to use. For example, in most Bayesian books, the [Gamma distribution](https://en.wikipedia.org/wiki/Gamma_distribution) uses the shape/rate parameterization, while `scipy.stats` uses the shape/scale parameterization.

In these cases, I like to wrap the `scipy.stats` distributions in a class so that I can pick my own parameterization and naming convention. For example, I wrap the Gamma distribution as follows:

```python
class GammaDist(object):
    def __init__(self, shape, rate, rng=None):
        self.shape = shape
        self.rate = rate
        self.rng = rng

    def rvs(self):
        return scipy.stats.gamma(a=self.shape, scale=1/self.rate).rvs(random_state=self.rng)
```

This way, in my code I can draw a new sample as:

```python
my_gamma = GammaDist(1, 2)
my_gamma.rvs()
```

    0.91282599799158526

Furthermore, I can also easily refer to the Gamma hyperparameters as `my_gamma.shape` and `my_gamma.rate`, which is more legible than `scipy_gamma.kwds['a']` and `1 / scipy_gamma.kwds['scale']`.

[^1]: We obtain this identity by multiplying both the numerator and the denominator with \\( p(x) \\)


# Estadística II

# 1. Parámetros de estimación e hipótesis.

Cuando tenemos una población cuya distribución es conocida pero se desconoce algún parámetro, es posible realizar la estimación de este parámetro si se tiene una muestra representativa.

Un estimador de un parámetro es un valor que puede calcularse a partir de los datos muestrales y que proporciona información sobre el valor del parámetro. Por ejemplo la media muestral es un estimador de la media poblacional, la proporción en la muestra es un estimador de la proporción en la población.

Una estimación es puntual cuando se obtiene un sólo valor para el parámetro.

### 1. Intervalo de confianza para la media.

A partir de una población, se desconoce la media $\mu$, sin embargo, deseamos estimarla a partir de la media de una muestra $\bar{x}$ obtenida en una muestra de tamaño $N$. De esta población se conoce la desviación $\sigma$.

En este caso, se sabe que la distribución muestral de medias es $N(\mu,\sigma/\sqrt{N})$, por lo tanto si se fija una probabilidad 1-$\alpha$ para aceptar lo dicho, se sabe que

\begin{equation}
P\bigg(\bar{x}-z_{\alpha/2}\frac{\sigma}{\sqrt{n}}\leq\mu\leq -\bar{x}+z_{\alpha/2}\frac{\sigma}{\sqrt{N}}\bigg)=1-\alpha
\end{equation}

esto significa que si el $(1-\alpha)$ de las $\bar{x}$ está a una distancia de $\mu$ inferior a $z_{\alpha/2}\frac{\sigma}{\sqrt{N}}$:

- Entonces para un nivel de confianza de $1-\alpha$, $\mu$ pertenece al intervalo $\bigg( \bar{x}-z_{\alpha/2} \frac{\sigma}{\sqrt{N}} , \bar{x}+z_{\alpha/2} \frac{\sigma}{\sqrt{N}}\bigg)$

Si la desviación típica de la población es desconocida, se aproximar a la de la muestra siempre y cuando $N$ sea muy grande

```python
import numpy as np
import matplotlib.pyplot as plt

x=np.random.normal(0,1,1000)
xbarra=np.mean(x)

print("xbarra es=%lf"%xbarra)

intervalo=[xbarra-1.96*1/np.sqrt(1000),xbarra+1.96*1/np.sqrt(1000)]

print("El intervalo de confianza del 0.95 para xbarra es=(%lf, %lf)"%(intervalo[0],intervalo[1]))

```

    xbarra es=-0.026572
    El intervalo de confianza del 0.95 para xbarra es=(-0.088552, 0.035409)

# 2. Pruebas de hipótesis:

Les pruebas de hipótesis son métodos estadísticos utilizados para tomar decisiones basados en datos. En las pruebas de hipótesis, en general, hacemos suposiciones acerca de los parámetros de la población y buscamos buscar conclusiones matemáticas sobre nuestras suposiciones.

Una prueba de hipótesis evalúa dos afirmaciones mutuamente excluyentes acerca de una población para determinar qué afirmación está mejor soportada por los datos de la muestra.

### Parámetros de hipótesis.
##### Hipótesis nula $(H_0)$:
En estadística inferencial, la hipótesis nula es una afirmación general acerca de que no existe relación entre fenómenos medidos. Es la suposición básica basada en el conocimiento del problema. Se prueba la similaridad de esta afirmación de ser verdadera en orden de decidir aceptar o rechazar la hipótesis alternativa.

##### Hipótesis alternativa $(H_\alpha)$:
La hipótesis alternativa es contraria a la hipótesis nula. Se determina si rechazar o no esta afirmación basados en la similaridad (likelihood) de la hipótesis nula de ser verdadera.

#### Nivel de significancia

El nivel de significancia hace alusión al grado de significancia con la cuál se acepta o rechaza la hipótesis nula. El nivel de significancia usualmente se denota con $\alpha$ y es generalmente de $0.05$

## Error tipo I:
Se tiene cuando se rechaza una hipótesis nula, a pesar que la hipótesis sea verdadera. Se denota por $\alpha$. El valor de $\alpha$ se denomina el nivel de la prueba.

## Error tipo II:
Se tiene cuando se acepta una hipótesis nula, pero es falsa. Los errores tipo II son denotados por $\beta$. En pruebas de hipótesis, las curvas normales que muestran la región de aceptancia se llaman regiones $\beta$.

##### Ejemplo:

En una encuesta, n=15 personas fueron muestreadas. Se desea probar la hipótesis $H_0:p=.5$ en contra de la alternativa, $H_a : p<.5$. La estadística de la prueba es $Y$ (binomial), el número de votantes muestreados en favor de una alternativa. Se pide calcular $\alpha$ si la región de rechazo es $={y\leq 2}$.

Por definición:

\begin{equation}
\alpha=P(\textsf{error tipo I})=P(\textsf{rechazar $H_0$ cuando $H_0$ es verdadera})
\end{equation}
\begin{equation}
\alpha=P(\textsf{valor del estadístico de prueba en la región de rechazo cuando H_0 es verdadera})
\end{equation}
\begin{equation}
=P(Y\leq 2, p=.5)
\end{equation}

Como $Y$ es una variable aleatoria binomial con n=15. Si $H_0$ es verdadera, p=.5, se obtiene

\begin{equation}
\alpha=\sum_{y=0}^{2}{(.5)^y(.5)^{15-y}}=0.004.
\end{equation}

Entonces, si decidimos rechazar la región, tenemos un riesgo de $\alpha$ de concluir de manera equivocada.

Ahora, supongamos que $(p=.3)$ y queremos calcular la probabilidad $\beta$ de que una muestra nos lleve erroneamente a concluir que $H_0$ es verdadera.

Por definición,

\begin{equation}
\beta=P(\textsf{error tipo II})=P(\textsf{Aceptar $H_0$ si $H_\alpha$ es verdadera})
\end{equation}

\begin{equation}
=P(\textsf{probabilidad de que el estadístico no esté en la región de rechazo cuando $H_a$ es verdadero})
\end{equation}

Como queremos calcular $\beta$ cuando $p=.3$

\begin{equation}
\beta=P(Y>2 \textsf{ cuando } p=.3)=\sum_{y=3}^{14} {15 \choose y}(.3)^{y}(.7)^{15-y}=.873
\end{equation}

Por lo que, con esta región de rechazo ${y\leq 2}$ nuestra prueba nos llevará eventualmente a concluir la hipótesis previa aún si $p$ es tan bajo como $p=.3$.

## Pruebas de una cola:
Supongamos que queremos probar hipótesis que correspondan a parámetros $\theta$ basados en una muestra $Y_1,Y_2,...Y_n$. Una forma de desarrollar pruebas de hipótesis se basa en un estimador que tiene una distribución muestral normal con media $\theta$ y error estándar $\sigma_{\theta}$. Si $\theta_0$ es un valor específico de $\theta$, y se desea probar $H_0: \theta=\theta_0$ contra $H_\alpha: \theta>\theta_0$. Entonces, se tiene que

\begin{equation}
H_0:\theta=\theta_0
\end{equation}

\begin{equation}
H_\alpha:\theta>\theta_0
\end{equation}

\begin{equation}
\textsf{Estadístico de prueba:} \hat{\theta}
\end{equation}

Con una región de rechazo $RR={\theta>k}$ para algún $k$.

E valor de k en la región de rechazo está determinado al fijar el valor de la probabilidad de error tipo I. Si $H_0$ es verdadera, $\hat{\theta}$ tiene aproximadamente una distribución normal con media $\theta_{0}$ con media $\theta_{0}$ y error estándar $\sigma_{\hat{\theta}}$. Por lo tanto, si se desea una prueba $\alpha$.

\begin{equation}
k=\theta_{0}+z_{\alpha}\sigma_{\hat{\theta}}
\end{equation}
y es la escogencia apropiada para $k$ porque

$RR={\hat{\theta}:\hat{\theta}>\theta_0 + z_{\alpha} \sigma_\hat{\theta}}={\hat{\theta}:\hat{\theta-\theta_0 /  \sigma_{\hat{\theta}}}z_\alpha}$

Si $Z=(\hat{\theta}-\theta_0)/\sigma_{\hat{\theta}}$ es usado como el estadístico de prueba, la región también puede ser escrita como $RR={z>z_\alpha}$. $Z$ mide el número de errores estandar entre el estimador para $\theta$ y $\theta_0$. $H_0$ es rechazado si $Z$ cae lo suficientemente lejos de la cola superior de la distribución normal. La hipótesis alternativa $H_a : \theta>\theta_0$ es llamada alternativa de cola superior, y $RR={z>z_{\alpha}}$ es la región de rechazo de cola superior.

## Pruebas de dos colas:
Una prueba de dos colas es una prueba estadística en la cual el área crítica o región de rechazo de una distrbución es de dos lados. Esto es, continuando con el ejemplo anterior, que para una prueba de dos

En una prueba de dos colas se puede probar que un estimador \theta no corresponda a un valor $\theta_0$. Es decir, $\theta \neq \theta_0$. Para esto, el estadístico de prueba puede ser $Z=\frac{\hat{\theta}-\theta_0}{\sigma_{\hat{\theta}}}$. Con una región de rechazo de ${|z|>z_{\alpha/2}}$.

```python

```

# 3. P-Value

Si $W$ es un estadístico de prueba, el p-value es el mínimo nivel de significancia para el cual los datos observados indican que la hipótesis nula debe ser rechazada. El valor P, o probabilidad calculada, es la probabilidad de encontrar lo observado, resulta cuando la hipótesis nula de un estudio es verdadera. Si el valor P es mayor que la significancia escogida, significa que se debe rechazar la hipótesis nula, es decir, aceptar que la muestra da la suficiente evidencia de apoyar la hipótesis alternativa. Esto no implica que exista realmente una diferencia significativa o importante, por lo que el criterio de la persona que interpreta la prueba de hipótesis es importante para considerar la relevancia del resultado.

Entre más pequeño el p-value se vuelve, más evidente es que la hipótesis nula debe ser rechazada. Si un experimento tiene un valor $\alpha$ en mente, el p-value puede ser utilizado para implementar un test $\alpha$. El p-value es el valor más pequeño para el cual la hipótesis nula puede ser rechazada. Si $\alpha$ es más pequeño que el p-value, entonces la hipótesis nula no puede ser rechazada. El p-value permite estar o no de acuerdo con una hipótesis nula según los requerimientos que uno decida.

# Extra: Bootstraping

Permite hacer uso de datos remuestreados para realizar inferencia estadística. Una réplica bootstrap

Ejemplo:
Es posible crear una función para crear réplicas en bootstrap utilizando np.random.choice(datos, funcion). En esta ocasión realizaremos un bootstrap de la media, tomaremos los datos de michelson_speed_of_light para realizar replicas de bootstrap en 1 dimensión utilizando choice y luego retornaremos una histograma de frecuencias para la media.

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

data_michelson = pd.read_csv('datasets/michelson_speed_of_light.csv')

msl=np.array(data_michelson['velocity of light in air (km/s)'])

def bootstrap_replicate_1d(data, func):

    bs_sample = np.random.choice(data, len(data))
    return func(bs_sample)

bootstrap_replicate_1d(msl, np.mean)

replicas = np.empty(10000)

for i in range(10000):
    replicas[i] = bootstrap_replicate_1d(msl, np.mean)

plt.hist(replicas, bins=30, density=True)
plt.xlabel('Velocidad media de la luz (km/s)')
plt.ylabel('PDF')
plt.show()

```

# Intervalo de confianza

Ahora, para mostrar visualmente la representación de un intervalo de confianza, obtengamos el intervalo de confianza de los datos previamente obtenidos utilizando np.percentile para obtener los valores del intervalo del 95%.

```python
conf_int = np.percentile(replicas, [2.5, 97.5])
print(np.mean(replicas))
print(conf_int)

plt.hist(replicas, bins=30, density=True)
plt.xlabel('Velocidad media de la luz (km/s)')
plt.ylabel('PDF')
plt.fill_between(x=conf_int,y1=[0,0],y2=[0.05,0.05],alpha=0.4,label='P(x)')
plt.show()

```

```python
df = pd.read_csv('datasets/2008_all_states.csv')
df_swing = pd.read_csv('datasets/2008_swing_states.csv')
iris = pd.read_csv('datasets/iris.csv')

sns.jointplot(x='total_votes', y='dem_share', data=df_swing, kind="reg");
```

Se pueden realizar permutaciones con np.random.permutation para reordenar las entradas en un array.

```python
dem_share = df[['dem_share', 'state']]
dem_share_PA = dem_share.loc[dem_share['state'] == 'PA']
dem_share_OH = dem_share.loc[dem_share['state'] == 'OH']

def diff_frac(data_A, data_B):
    frac_A = np.sum(data_A) / len(data_A)
    frac_B = np.sum(data_B) / len(data_B)
    return frac_B - frac_A

dem_share_both = np.concatenate((dem_share_PA, dem_share_OH))
dem_share_perm = np.random.permutation(dem_share_both)

perm_sample_PA = dem_share_perm[:len(dem_share_PA)]
perm_sample_OH = dem_share_perm[len(dem_share_PA):]
```

Ejercicio 1: Realice las CDFs de PA y OH según el dem_share.

Ejercicio 2: Realice el estimador de la media de la diferencia de votos porcentuales entre PA-OH, realice el histograma de la PDF estimada.

# Referencias
- Bertsekas, D. Introduction to probability. (2008)
- Wackerly, D. Mathematical Statistics with Applications. (2008)

```python

```


# Tutorial: advection-diffusion kernels in Parcels
Adding diffusion to simulations is insightful when investigating tracers such as heat, salt or nutrients. The spread of tracers is affected by subgrid-scale processes and unresolved physics ([Van Sebille et al. 2018](https://doi.org/10.1016/j.ocemod.2017.11.008)). Also when individual water parcels are considered, diffusion can be included in ensemble particle simulations to trace the evolution of a probability distribution. This is especially important when coarse velocity fields that are not eddy-resolving are considered, since the absence of eddies can still be parameterized through eddy diffusive fluxes ([Shah et al., 2017](https://doi.org/10.1175/JPO-D-16-0098.1)).

## Stochastic Differential Equations (SDEs) for Particle Trajectories in the Presence of Diffusivity
The stochastic differential equation for a particle trajectory is given by
$$
\begin{aligned}
    d\mathbf{X}(t) &\overset{\text{Îto}}{=} (\mathbf{u} + \nabla \cdot \mathbf{K}) dt + \mathbf{V}(t, \mathbf{X})\cdot d\mathbf{W}(t), \\
    \mathbf{X}(t_0) &= \mathbf{x}_0,
\end{aligned}
$$
where $\mathbf{X}$ is the particle position vector ($\mathbf{x}_0$ being the initial position vector), $\mathbf{v}$ the velocity vector, $\mathbf{K} = \frac{1}{2} \mathbf{V} \cdot \mathbf{V}^T$ the diffusivity tensor, and $d\mathbf{W}(t)$ a Wiener increment (normally distributed with zero mean and variance $dt$). This equation is obtained through writing the advection-diffusion equation as a Fokker-Planck equation. Particle distributions obtained by solving above equation therefore satisfy the advection-diffusion equation ([van Sebille et al., 2018](https://doi.org/10.1016/j.ocemod.2017.11.008); [Shah et al., 2013](https://doi.org/10.1016/j.ocemod.2013.04.001)).

Generally, in three-dimensional ocean models, $\mathbf{K}$ (and therefore $\mathbf{V}$) contains off-diagonal components to account for diffusion along non-flat isopycnal surfaces. Instead, here we will focus on the simpler case of diffusion in a horizontal plane, where diffusivity is specified only in the zonal and meridional direction, i.e.
$$\mathbf{K}(x,y)=\begin{bmatrix}
K_x(x,y) & 0\\
0 & K_y(x,y)
\end{bmatrix}.$$

The above stochastic differential equation then becomes

$$
\begin{align}
    dX(t) &= a_x dt + b_x dW_x(t), \quad &X(t_0) = x_0,\\
    dY(t) &= a_y dt + b_y dW_y(t), \quad &Y(t_0) = y_0,
\end{align}
$$
where $a_i = v_i + \partial_i K_i(x, y)$ is the deterministic drift term and $b_i = \sqrt{2K_i(x, y)}$ a stochastic noise term ($\partial_i$ denotes the partial derivative with respect to $i$).

## Numerical Approximations of SDEs
The simplest numerical approximation of the above SDEs is obtained by replacing $dt$ by a finite time discrete step $\Delta t$ and $dW$ by a discrete increment $\Delta W$, yielding the **Euler-Maruyama (EM) scheme** ([Maruyama, 1955](https://link.springer.com/article/10.1007/BF02846028)):
$$
\begin{equation}
    X_{n+1} = X_n + a_x \Delta t + b_x \Delta W_{n, x},
\end{equation}
$$
with a similar expression for $Y$.

The next higher-order scheme is found by including extra terms from a Taylor expansion on our SDE, yielding the **Milstein scheme of order 1 (M1)**:
$$
\begin{equation}
    X_{n+1} = X_n + a_x \Delta t + b_x \Delta W_x + \frac{1}{2}b_x \partial_x b_x(\Delta W_{n, x}^2 - \Delta t),
\end{equation}
$$
which can be rewritten by explicitly writing $b_x\partial_x b_x$ as $\partial_x K_x(z)$:
$$
\begin{equation}
    X_{n+1} = X_n + v_x \Delta t + \frac{1}{2}\partial_x K_x(\Delta W_{n, x}^2 + \Delta t) + b\Delta W_n.
\end{equation}
$$
The extra term in the M1 scheme provides extra accuracy at negligible computational cost.

The spatial derivatives in the EM and M1 schemes can be approximated by a central difference. Higher order numerical schemes (see [Gräwe et al., 2012](https://doi.org/10.1007/s10236-012-0523-y)) include higher order derivatives. Since Parcels uses bilinear interpolation, these higher order derivatives cannot be computed, meaning that higher order numerical schemes cannot be used.

An overview of numerical approximations for SDEs in a particle tracking setting can be found in [Gräwe (2011)](https://doi.org/10.1016/j.ocemod.2010.10.002).

## The Wiener Increment
The Wiener increment should be a normally distributed random variable with zero mean and a standard deviation of $\sqrt{dt}$. While such random numbers can be generated easily by Parcels (i.e. through `ParcelsRandom.normalvariate(0, particle.dt)`), it is more efficient to draw random numbers from a uniform distribution with the same first and second moments. The central limit theorem ensures that the contribution from consecutive Wiener increments drawn from a uniform distribution quickly converges to a normal distribution, making a substitution with a uniform distribution valid for ensemble distributions. The discrete Wiener increment used by Parcels therefore reads $\Delta W = \sqrt{3\Delta t } R[-1, 1]$. Wiener increments are drawn individually for each direction.

## Using Advection-Diffusion Kernels in Parcels
The EM and M1 advection-diffusion approximations are available as `AdvectionDiffusionEM` and `AdvectionDiffusionM1`, respectively. The `AdvectionDiffusionM1` kernel should be the default choice.

The advection component of these kernels is similar to that of the Explicit Euler advection kernel (`AdvectionEE`). In case the Euler approximation is numerically unstable for the advection part, there are two advection-diffusion kernels available where the advection is approximated through a fourth-order Runge-Kutta algorithm, similar to in `AdvectionRK4`. These are available as `AdvectionRK4DiffusionM1` and `AdvectionRKDiffusionEM`. Since the diffusion part of these kernels is only accurate to first-order, these kernels are only beneficial in case of numerical instability, or if the simulation contains regimes where diffusion vanishes.

In the special case where diffusivity is constant over the entire domain, the diffusion-only kernel `DiffusionUniformKh` can be used in combination with an advection kernel of choice. Since the diffusivity here is space-independent, gradients are not calculated, increasing efficiency. The diffusion-step can in this case be computed after or before advection, thus allowing you to chain kernels using the `+` operator.

To make a central difference approximation for computing the gradient in diffusivity, a resolution for this approximation `dres` is needed. This constant is used by the advection-diffusion kernels and must be chosen and added to the FieldSet by the user (e.g. `fieldset.add_constant('dres', 0.01)`). The size of `dres` should be much smaller than the spatial resolution of the data, but within reasonable limits of machine precision to avoid numerical errors.

## Example: Impermeable Diffusivity Profile

Let's see the `AdvectionDiffusionM1` in action and see why it's preferable over the `AdvectionDiffusionEM` kernel. To do so, we create an idealized profile with diffusivities $K_\text{zonal}$ uniform everywhere ($K_\text{zonal} = \bar{K}=0..25$) and $K_\text{meridional}$ constant in the zonal direction, while having the following profile in the meridional direction:

$$ K_\text{meridional}(y) = \bar{K}\frac{2(1+\alpha)(1+2\alpha)}{\alpha^2H^{1+1/\alpha}} \begin{cases}
y(L-2y)^{1/\alpha},\quad 0 \leq y \leq L/2,\\
(L-y)(2y-1)^{1/a},\quad H/2 \leq y \leq L,
\end{cases}$$
with $L$ being the basin length scale, $\alpha$ as a parameter determining the steepness in the gradient in the profile. This profile is similar to that used by [Gräwe (2011)](https://doi.org/10.1016/j.ocemod.2010.10.002), now used in the meridional direction for illustrative purposes.

Let's plot $K_\text{meridional}(y)$:

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import xarray as xr
from datetime import timedelta
from parcels import ParcelsRandom
from parcels import (FieldSet, ParticleSet, JITParticle,
                     DiffusionUniformKh, AdvectionDiffusionM1, AdvectionDiffusionEM)
```

```python
K_bar = 0.25  # Average diffusivity
alpha = 1.    # Profile steepness
L = 1.        # Basin scale
Ny = 103      # Number of grid cells in y_direction (101 +2, one level above and one below, where fields are set to zero)
dy = 1.03/Ny  # Spatial resolution
y = np.linspace(-0.01, 1.01, 103)  # y-coordinates for grid
y_K = np.linspace(0., 1., 101)     # y-coordinates used for setting diffusivity
beta = np.zeros(y_K.shape)         # Placeholder for fraction term in K(y) formula

for yi in range(len(y_K)):
    if y_K[yi] < L/2:
        beta[yi] = y_K[yi]*np.power(L - 2*y_K[yi], 1/alpha)
    elif y_K[yi] >= L/2:
        beta[yi] = (L - y_K[yi])*np.power(2*y_K[yi] - L, 1/alpha)
Kh_meridional = 0.1*(2*(1+alpha)*(1+2*alpha))/(alpha**2*np.power(L, 1+1/alpha))*beta
Kh_meridional = np.concatenate((np.array([0]), Kh_meridional, np.array([0])))
```

```python
plt.plot(Kh_meridional, y)
plt.ylabel("y")
plt.xlabel(r"$K_{meridional}$")
plt.show()
```

In this profile, diffusivity drops to 0 at $y=0.5$ and at $y=0$ and $y=1$. In the absence of advection, particles starting out in one half of the domain should remain confined to that half as they are unable to cross the points where the diffusivity drops to 0. The line $y=0.5$ should therefore provide an impermeable barrier.

Now we can put this idealized profile into a flat fieldset:

```python
xdim, ydim = (1, Ny)
data = {'U': np.zeros(ydim),
        'V': np.zeros(ydim),
        'Kh_zonal': K_bar*np.ones(ydim),
        'Kh_meridional':  Kh_meridional}
dims = {'lon': 1,
        'lat': np.linspace(-0.01, 1.01, ydim, dtype=np.float32)}
fieldset = FieldSet.from_data(data, dims, mesh='flat', allow_time_extrapolation=True)
fieldset.add_constant('dres', 0.00005)
```

    WARNING: Casting lon data to np.float32
    WARNING: Casting field data to np.float32

We release 100 particles at (0, 0.75).

```python
def get_test_particles():
    return ParticleSet.from_list(fieldset,
                                 pclass=JITParticle,
                                 lon=np.zeros(100),
                                 lat=np.ones(100)*0.75,
                                 time=np.zeros(100),
                                 lonlatdepth_dtype=np.float64)
```

Now we will simulate the advection and diffusion of the particles using the `AdvectionDiffusionM1` kernel. We run the simulation for 0.3 seconds, with a numerical timestep $\Delta t = 0.0001$s. We also write away particle locations at each timestep for plotting. Note that this will hinder a runtime comparison between kernels, since it will cause most time to be spent on I/O.

```python
dt = 0.0001
testParticles = get_test_particles()
output_file = testParticles.ParticleFile(name="M1_out.nc",
                                         outputdt=timedelta(seconds=dt))
ParcelsRandom.seed(1636)  # Random seed for reproducibility
testParticles.execute(AdvectionDiffusionM1,
                      runtime=timedelta(seconds=0.3),
                      dt=timedelta(seconds=dt),
                      output_file=output_file,
                      verbose_progress=True)
output_file.close()  # to write the output to a netCDF file, since `output_file` does not close automatically when using notebooks
```

    INFO: Compiled ParcelsRandom ==> /var/folders/_k/jcmdplbn0yj79g4k3g9f4nxr0000gn/T/parcels-501/parcels_random_90026758-b11d-438d-b43f-c207a0d92a19.so
    INFO: Compiled JITParticleAdvectionDiffusionM1 ==> /var/folders/_k/jcmdplbn0yj79g4k3g9f4nxr0000gn/T/parcels-501/ed3fb56fbd4ce833c3c02d188a729819_0.so
    100% (0.3 of 0.3) |######################| Elapsed Time: 0:00:21 Time:  0:00:21

```python
M1_out = xr.open_dataset("M1_out.nc")
```

We can plot the individual coordinates of the particle trajectories against time ($x$ against $t$ and $y$ against $t$) to investigate how diffusion works along each axis.

```python
fig, ax = plt.subplots(1, 2)
fig.set_figwidth(12)

for data, ai, dim, ystart, ylim in zip([M1_out.lat, M1_out.lon], ax, ('y', 'x'), (0.75, 0), [(0, 1), (-1, 1)]):
    ai.plot(np.arange(0, 0.3002, 0.0001), data.T, alpha=0.2)
    ai.scatter(0, ystart, s=20, c='r', zorder=3)
    ai.set_xlabel("t")
    ai.set_ylabel(dim)
    ai.set_xlim(0, 0.3)
    ai.set_ylim(ylim)

fig.suptitle("`AdvectionDiffusionM1` Simulation: Particle trajectories in the x- and y-directions against time")
plt.show()
```

We see that the along the meridional direction, particles remain confined to the ‘upper’ part of the domain, not crossing the impermeable barrier where the diffusivity drops to zero. In the zonal, particles follow random walks, since all terms involving gradients of the diffusivity are zero.

Now let's execute the simulation with the `AdvectionDiffusionEM` kernel instead.

```python
dt = 0.0001
testParticles = get_test_particles()
output_file = testParticles.ParticleFile(name="EM_out.nc",
                                         outputdt=timedelta(seconds=dt))
ParcelsRandom.seed(1636)  # Random seed for reproducibility
testParticles.execute(AdvectionDiffusionEM,
                      runtime=timedelta(seconds=0.3),
                      dt=timedelta(seconds=dt),
                      output_file=output_file,
                      verbose_progress=True)
output_file.close()  # to write the output to a netCDF file, since `output_file` does not close automatically when using notebooks
```

    INFO: Compiled JITParticleAdvectionDiffusionEM ==> /var/folders/_k/jcmdplbn0yj79g4k3g9f4nxr0000gn/T/parcels-501/c2c1e997a9e46090e2f8d3469b2576ce_0.so
    100% (0.3 of 0.3) |######################| Elapsed Time: 0:00:24 Time:  0:00:24

```python
EM_out = xr.open_dataset("EM_out.nc")
```

```python
fig, ax = plt.subplots(1, 2)
fig.set_figwidth(12)

for data, ai, dim, ystart, ylim in zip([EM_out.lat, EM_out.lon], ax, ('y', 'x'), (0.75, 0), [(0, 1), (-1, 1)]):
    ai.plot(np.arange(0, 0.3002, 0.0001), data.T, alpha=0.2)
    ai.scatter(0, ystart, s=20, c='r', zorder=3)
    ai.set_xlabel("t")
    ai.set_ylabel(dim)
    ai.set_xlim(0, 0.3)
    ai.set_ylim(ylim)

fig.suptitle("`AdvectionDiffusionEM` Simulation: Particle trajectories in the x- and y-directions against time")
plt.show()
```

The Wiener increments for both simulations are equal, as they are fixed through a random seed. As we can see, the Euler-Maruyama algorithm performs worse than the Milstein algorithm, letting particles cross the impermeable barrier at $y=0.5$. In contrast, along the zonal direction, particles follow the same random walk as in the Milstein algorithm, which is expected since the extra terms in the Milstein algorithm are zero in this case..

Lastly, we perform the same simulation with the `DiffusionUniformKh` kernel.

```python
dt = 0.0001
testParticles = get_test_particles()
output_file = testParticles.ParticleFile(name="Uniform_out.nc",
                                         outputdt=timedelta(seconds=dt))
ParcelsRandom.seed(1636)  # Random seed for reproducibility
testParticles.execute(DiffusionUniformKh,
                      runtime=timedelta(seconds=0.3),
                      dt=timedelta(seconds=dt),
                      output_file=output_file,
                      verbose_progress=True)
output_file.close()  # to write the output to a netCDF file, since `output_file` does not close automatically when using notebooks
```

    INFO: Compiled JITParticleDiffusionUniformKh ==> /var/folders/_k/jcmdplbn0yj79g4k3g9f4nxr0000gn/T/parcels-501/ef97ee21f0fdfcd10c9ab50a7d4b2588_0.so
    100% (0.3 of 0.3) |######################| Elapsed Time: 0:00:23 Time:  0:00:23

```python
Uniform_out = xr.open_dataset("Uniform_out.nc")
```

```python
fig, ax = plt.subplots(1, 2)
fig.set_figwidth(12)

for data, ai, dim, ystart, ylim in zip([Uniform_out.lat, Uniform_out.lon], ax, ('y', 'x'), (0.75, 0), [(0, 1), (-1, 1)]):
    ai.plot(np.arange(0, 0.3002, 0.0001), data.T, alpha=0.2)
    ai.scatter(0, ystart, s=20, c='r', zorder=3)
    ai.set_xlabel("t")
    ai.set_ylabel(dim)
    ai.set_xlim(0, 0.3)
    ai.set_ylim(ylim)

fig.suptitle("`DiffusionUniformKh` Simulation: Particle trajectories in the x- and y-directions against time")
plt.show()
```

We clearly see that in the meridional direction, particles are not able to leave the areas with a low diffusivity, accumulating on the impermeable boundaries. In the zonal direction, results are equal to those of the other two kernels. When fields have a uniform diffusivity, this kernel is the most efficient since it does not execute the interpolation steps needed for computing gradients in the diffusivity field.

## References
Gräwe, U. (2011). “Implementation of high-order particle-tracking schemes in a water column model.” *Ocean Modelling*, 36(1), 80–89. https://doi.org/10.1016/j.ocemod.2010.10.002

Gräwe, Deleersnijder, Shah & Heemink (2012). “Why the Euler scheme in particle tracking is not enough: The shallow-sea pycnocline test case.” *Ocean Dynamics*, 62(4), 501–514. https://doi.org/10.1007/s10236-012-0523-y

Maruyama, G. (1955). “Continuous Markov processes and stochastic equations.” *Rendiconti del Circolo Matematico di Palermo*, 4(1), 48.

van Sebille et al. (2018). “Lagrangian ocean analysis: Fundamentals and practices.” *Ocean Modelling*, 121, 49–75. https://doi.org/10.1016/j.ocemod.2017.11.008

Shah, S. H. A. M., Heemink, A. W., Gräwe, U., & Deleersnijder, E. (2013). “Adaptive time stepping algorithm for Lagrangian transport models: Theory and idealised test cases.” *Ocean Modelling*, 68, 9–21. https://doi.org/10.1016/j.ocemod.2013.04.001

Shah, Primeau, Deleersnijder & Heemink (2017). “Tracing the Ventilation Pathways of the Deep North Pacific Ocean Using Lagrangian Particles and Eulerian Tracers.” *Journal of Physical Oceanography*, 47(6), 1261–1280. https://doi.org/10.1175/JPO-D-16-0098.1


```python
"""
Things to do with discrete power law
"""

import numpy as np
import matplotlib.pyplot as plt
from sympy import mpmath
```

```python
# intialize parameters and x data
alpha = 1.4  # -1 - anomalous exponent
xmin = 1  # minimum value drawn from discrete power law
xmax = 1000  # maximum value of x
x = np.linspace(xmin, xmax, (xmax - xmin) + 1)  # discrete values of x
dx = x[1] - x[0]  # space between points, i.e. timestep
```

```python
# plot power law with above parameters
unnormalized = x**-alpha
plt.plot(x, unnormalized)
plt.show()
```

```python
def zeta(x, alpha):
    """ Numerical estimate of Hurwitz zeta function. Used as discrete power law
    distribution normalization constant

    sum from n=0 to infinity of (n + x)^-\alpha

    :param x: point at which to evaluate function
    :param alpha: exponent of power law
    :param upper_limit: number of terms used to calculate Hurwitz zeta function (highest value of n above)

    :type x: float
    :type alpha: float
    :type upper_limit: int

    :return: evaluation of Hurwitz zeta function at x
    :rtype: float
    """

    # Turns out, sympy has an implementation of the Hurwitz zeta function which does a fast evaluation

    if type(alpha) is np.ndarray:
        alpha = alpha[0]

    return float(mpmath.zeta(alpha, a=x))
```

```python
# normalize discrete distribution
normalized = unnormalized / zeta(xmin, alpha)
```

```python
# plot normalized probability distribution
plt.plot(x, normalized)
plt.show()
```

```python
# show that normalized probability converges to an area of 1
print(sum([i*dx for i in normalized]))  # increase xmax to get this closer to 1
```

    0.949217390206

```python
def discrete_powerlaw_CCDF(val, xmin, alpha):
    """ Calculate the complementary cumulative distribution function of a discrete power
    law distribution such that P(x) = Pr(X >= x)

    :param x: value at which to evaluate CDF
    :param xmin: lower bound of power law PDF
    :param alpha: exponent of power law
    :param upper_limit: number of terms used to calculate Hurwitz zeta function

    :type x: float
    :type xmin: float
    :type alpha: float
    :type upper_limit: int

    :return CDF of power law PDF evaluated at x
    :rtype float
    """
    return zeta(val, alpha) / zeta(xmin, alpha)
```

```python
# Plot Complementary Cumulative Distribution Function from xmin to xmax
cdf = [discrete_powerlaw_CCDF(i, xmin, alpha) for i in x]
plt.plot(x, cdf)
plt.show()
```

```python
def sample_discrete_powerlaw(alpha, xmin, size=1):
    """ Exact method for generating random draws from a discrete power law distribution of
    form t**-alpha

    See Appendix D of https://epubs.siam.org/doi/abs/10.1137/070710111.

    This method is slow and the approximation might be more useful

    :param alpha: power law exponent
    :param xmin: lower limit of distribution.
    :param size: number of random draws to perform

    :type: alpha: float
    :type xmin: float
    :type size: int

    :return: array of random power law draws
    """

    r = np.random.uniform(0, 1, size=size)

    t = np.zeros([size])
    for i, val in enumerate(r):
        x2 = xmin
        while discrete_powerlaw_CCDF(x2, xmin, alpha) > (1 - val):
            x1 = x2
            x2 = 2 * x1
        t[i] = x2

    return t
```

```python
def approximate_discrete_powerlaw_samples(alpha, xmin, size=1):
    """ Approximate random draws from a discrete power law distribution of form t**-alpha.
    Much faster than sample_discrete_powerlaw()

    See Appendix D of https://epubs.siam.org/doi/abs/10.1137/070710111

    :param alpha: power law exponent
    :param xmin: lower limit of distribution.
    :param size: number of random draws to perform

    :type: alpha: float
    :type xmin: float
    :type size: int

    :return: array of random power law draws
    """

    r = np.random.uniform(0, 1, size=size)

    t = np.round((xmin - 0.5)*(1 - r)**(-1/(alpha -1)) + 0.5)

    return t
```

```python
# Generate random values from a discrete power law distributio
ndraws = 10000
#draws = sample_discrete_powerlaw(alpha, xmin, size=ndraws, upper_limit=upper_limit)
draws = approximate_discrete_powerlaw_samples(alpha, xmin, size=ndraws)
plt.hist(draws, range=(0, 25), bins=25)
plt.show()
```

```python
def discrete_log_likelihood(alpha, x, xmin,  minimize=False):
    """ Calculate log likelihood for alpha given a set of x values that might come from a
    power law distribution

    :param alpha: power law exponent. Calculates the log-likelihood of this value of alpha
    for the data
    :param x: array of values making up emperical distribution
    :param xmin: lower bound of power law distribution
    :param upper_limit: number of terms used to calculate zeta function

    :type alpha: float
    :type x: np.ndarray
    :type xmin: float
    :type upper_limit: int

    :return log-likelihood of input parameters
    :rtype float
    """

    n = x.size
    z = zeta(xmin, alpha)

    res = -n*np.log(float(z)) - alpha*sum([np.log(i) for i in x])

    if minimize:
        return res * - 1
    else:
        return res
```

```python
# Calculate log-likelihood of data given a range of alphas
ll = []  # list of log-likelihoods for each alpha tested
alphas = np.linspace(1.01, 2, 100)
for i in alphas:
    ll.append(discrete_log_likelihood(i, draws, xmin))

# Plot the log-likelihood as a function of alpha. The maximum appears near the value
# of alpha we provided. It's not perfect, but we only used 1000 data points and the
# approximated discrete value generator
plt.plot(alphas, ll)
plt.ylabel('Log-Likelihood')
plt.xlabel('alpha')
print('Likelihood maximized for alpha = %.2f' % alphas[np.argmax(ll)])
plt.show()
```

```python
# Now actually maximize the likelihood
from scipy.optimize import minimize

args = (draws, xmin, True)
print(sum(draws))
maximum = minimize(discrete_log_likelihood, 1.5, args=args, bounds=[(1.01, 3)])
print('Optimized value of alpha: %.3f' % maximum.x)
```

    912411669.0
    Optimized value of alpha: 1.401

```python

```


# Probabilistic Programming 2: Mixture modeling
## Variational inference

In this notebook, we are exploring a more complex problem set using variational inference, namely mixture modeling. It will show you how to deal with the multivariate case.

### Preliminaries

- Goal
  - Write a variational inference procedure for a 2-dimensional mixture model.
- Materials
  - Mandatory
    - These lecture notes.
  - Optional
    - [Getting started with ForneyLab](https://biaslab.github.io/forneylab/docs/getting-started/).
    - [Demos in ForneyLab](https://github.com/biaslab/ForneyLab.jl/tree/v0.10.0/demo)
    - [Probabilistic Programming notebook](https://github.com/bertdv/BMLIP/tree/master/lessons/notebooks/probprog/Probabilistic-Programming.ipynb)
    - [Cheatsheets: how does Julia differ from Matlab / Python](https://docs.julialang.org/en/v1/manual/noteworthy-differences/index.html).

```julia
# Package managing
using Pkg
Pkg.activate("workspace")
Pkg.instantiate()

include("../scripts/pp-2.jl")

using Logging; disable_logging(LogLevel(0))
using Distributions: MvNormal
using StatsPlots
using LaTeXStrings
using ForneyLab
using Plots
pyplot();
```

## Data generation

We generate samples from three 2-dimensional Gaussian distributions and mix these. These Gaussians are:

$$
\begin{align*}
\mathcal{N}_1 \Big(\begin{bmatrix}-2 \\ -2 \end{bmatrix}, \begin{bmatrix}1 & 0 \\0 & 3 \end{bmatrix}\Big) \, , \quad
\mathcal{N}_2 \Big(\begin{bmatrix}+2 \\ +2 \end{bmatrix}, \begin{bmatrix}3 & 0 \\0 & 1 \end{bmatrix}\Big) \, , \quad
\mathcal{N}_3 \Big(\begin{bmatrix}-2 \\ +2 \end{bmatrix}, \begin{bmatrix}1 & 0 \\0 & 1 \end{bmatrix}\Big) \, .
\end{align*}
$$

We draw $N_1$, $N_2$ and $N_3$ samples from each, respectively.

```julia
# Sample size
N1 = 10
N2 = 20
N3 = 30

# Parameters for each cluster
μ1 = [-2. -2.]
Σ1 = [1.0 0.0; 0.0 3.0]
μ2 = [+2. +2.]
Σ2 = [3.0 0.0; 0.0 1.0]
μ3 = [-2. +2.]
Σ3 = [1.0 0.0; 0.0 1.0]

function generate_data(means::Array{Float64,2},
                       covariances::Array{Float64,3};
                       num_samples::Array{Int64,1}=[10])
    "Generate data from a 2-dimensional mixture model"

    # Extract number of components
    K = length(num_samples)

    # Extract dimensionality
    D = size(covariances)[1]

    # Preallocate vectors
    X = zeros(sum(num_samples), D)
    C = zeros(sum(num_samples),)

    # Create sample index
    ix = cat(0, cumsum(num_samples), dims=1)

    for k = 1:K

        # Current indices
        ix_k = (ix[k]+1:ix[k+1])

        # Sample from cluster component
        X[ix_k, :] = rand(MvNormal(means[k,:], covariances[:,:,k]), num_samples[k])'

        # Store cluster origin
        C[ix_k,] = k*ones(num_samples[k])

    end

    return X, C
end

# Generate data
observations, cluster = generate_data(cat(μ1, μ2, μ3, dims=1), cat(Σ1, Σ2, Σ3, dims=3), num_samples=[N1, N2, N3])

# Total sample size
N = sum([N1, N2, N3])

# Visualization
scatter(observations[cluster .== 1, 1],
        observations[cluster .== 1, 2],
        color="black",
        marker=:circle,
        label="C1")
scatter!(observations[cluster .== 2, 1],
         observations[cluster .== 2, 2],
         color="black",
         marker=:square,
         label="C2")
scatter!(observations[cluster .== 3, 1],
         observations[cluster .== 3, 2],
         marker=:diamond,
         color="black",
         label="C3",
         xlabel=L"$x_1$",
         ylabel=L"$x_2$",
         xlim=[-6, 6],
         ylim=[-6, 6],
         size=(600,400),
         grid=false)
```

## Model specification

Let's quickly recap the $K$-component Gaussian mixture model.

We have a data set of $X$ of $N$ samples by $D$ features. Each component is a Gaussian distribution, $ p_k(x \mid \mu_k, \Sigma_k) \triangleq \mathcal{N}(x \mid \mu_k, \Sigma_k)$. In total, we model $N$ samples:

$$ p(X \mid z, \mu, \Sigma) = \sum_{i=1}^{N} \sum_{k=1}^{K} p_k(x_i \mid \mu_k, \Sigma_k)^{z_i=k} \, .$$

<!-- Note that we summarize parameters as follows: $\phi = (\phi_1, \dots, \phi_K), \mu = (\mu_1, \dots, \mu_K), \Sigma = (\Sigma_1, \dots, \Sigma_K)$.  -->
The variable $z_i$ is the assignment of sample $i$ to class $k$. The probability $p(z_{i} = k)$ is also called the _responsibility_ of component $k$ for the $i$-th sample. All $z_i$ follow a Categorical distribution with mixture weights as parameters:

$$ p(z_i) \sim \text{Cat}(\phi) \, ,$$

Being Bayesians, we put priors on all unknown parameters ($\phi, \mu, \Sigma$):

$$ \begin{align}
p(\phi) \sim&\ \text{Dir}(a_0) \\
p(\mu) \sim&\ \mathcal{N}(m_0, l_0^{-1}\Sigma) \\
p(\Sigma) \sim&\ \mathcal{W}(W_0, v_0) \end{align} \, .$$

The distribution for the mixture weights, $p(\phi)$, follows a Dirichlet distribution because it is a vector that should sum to one. The component parameter priors are a Wishart distribution for the covariance matrix and a Gaussian distribution for the mean, conditioned on the sampled covariance matrix. Together, these two priors form a [Normal-Inverse-Wishart distribution](https://en.wikipedia.org/wiki/Normal-inverse-Wishart_distribution), $p(\mu, \Sigma)$, which is the conjugate prior for the Gaussian components. The full joint distribution is therefore:

$$ \begin{align} p(X, z, \phi, \mu, \Sigma) =&\ p(X \mid z, \phi, \mu, \Sigma)\ p(z \mid \phi)\ p(\mu, \Sigma)\ p(\phi) \\
=&\ \sum_{i=1}^{N} \sum_{k=1}^{K} \mathcal{N}(x_i \mid \mu_k, \Sigma_k)^{z_i=k}\ \text{Cat}(z_i=k \mid \phi_k)\ \mathcal{NIW}(\mu_k, \Sigma_k \mid m_{0k}, l_{0k}, W_{0k}, v_{0k})\ \text{Dir}(\phi_k \mid a_{0k}) \end{align}$$

We are interested in the posterior distribution over $z$, which assigns the current samples to clusters via $\arg \max_{k} p(z_i = k)$. For example, if the posterior distribution for $z_i$ is $[0.32\ 0.21\ 0.47]$, then the $\arg \max_k$ is 3 and we say that $x_i$ belongs to component 3. We are also interested in the posterior distributions for the component parameters, because these allow us to assign future samples to the current clusters. The posterior for $z, \phi, \mu, \Sigma$ has the form:

$$p(z, \phi, \mu, \Sigma \mid X) = \frac{p(X, z, \phi, \mu, \Sigma)}{\int p(X, z, \phi, \mu, \Sigma)\ \mathrm{d}X} \, .$$

We cannot evaluate the integral in the denominator, but because it only serves as a normalization factor we ignore it for now. Note that the numerator is the joint distribution. Unfortunately, it doesn't simplify to a known form. However, in mixture models, it is usually possible to perform a conditional factorization into the two distributions: if we fix the parameters $\phi, \mu, \Sigma$ to certain values, we obtain the factor:

$$p(z_i \mid X, \phi, \mu, \Sigma) = \sum_{i=1}^{N} \sum_{k=1}^{K} \rho_k^{z_i = k} \, ,$$

where $\rho_k = \phi_k \mathcal{N}(x_i \mid \mu_k, \Sigma_k)$ for all $k$. This is a Categorical distribution! The only problem is that $\rho$ is not normalized correctly. But we know that it _should_ be correctly normalized, after division by the denominator $\int p(X, z, \phi, \mu, \Sigma) \ \mathrm{d}X$. We can therefore perform the following re-parameterization:

$$r_k = \frac{\rho_k}{\sum_{k} \rho_k} \, .$$

Now, $r_k$ are valid responsibilities for the assignment variable $z$. In other words, $p(z_i \mid X, \phi, \mu, \Sigma) \sim \text{Cat}(r_k)$.

If we fix $z_i$ to certain values, it acts as a component selector. Take for instance $z_i = 2$;

$$\begin{align} p(\phi_2, \mu_2, \Sigma_2 \mid x_i, z_i) =&\ \mathcal{N}(x_i \mid \mu_1, \Sigma_1)^{0} \ 0\ \ \ \mathcal{NIW}(\mu_1, \Sigma_1 \mid m_{01}, l_{01}, W_{01}, v_{01}) \ \text{Dir}(\phi_1 \mid a_{01}) \\
&+ \mathcal{N}(x_i \mid \mu_2, \Sigma_2)^{1} \ \phi_2\ \mathcal{NIW}(\mu_2, \Sigma_2 \mid m_{02}, l_{02}, W_{02}, v_{02}) \ \text{Dir}(\phi_2 \mid a_{02}) \\
&+ \mathcal{N}(x_i \mid \mu_3, \Sigma_3)^{0} \ 0\ \ \ \mathcal{NIW}(\mu_3, \Sigma_3 \mid m_{03}, l_{03}, W_{03}, v_{03}) \ \text{Dir}(\phi_3 \mid a_{03}) \\
=& \ \mathcal{N}(x_i \mid \mu_2, \Sigma_2) \ \phi_2\ \mathcal{NIW}(\mu_2, \Sigma_2 \mid m_{02}, l_{02}, W_{02}, v_{02}) \ \text{Dir}(\phi_2 \mid a_{02})  \end{align} \, .$$

Fixing all $z_i$ produces $p(\phi, \mu, \Sigma \mid X, z)$. This can be further factorized into $p(\phi \mid z) p(\mu, \Sigma \mid X)$:

$$\begin{align}
p(\phi_k \mid z) =&\ \sum_{i=1}^{N} \text{Cat}(z_i \mid \phi_k)\ \text{Dir}(\phi_k \mid a_{0}) \\
p(\mu_k, \Sigma_k \mid X) =&\ \sum_{i=1}^{N} \mathcal{N}(x_i \mid \mu_k, \Sigma_k)\ \mathcal{NIW}(\mu_k, \Sigma_k \mid m_{0k}, l_{0k}, W_{0k}, v_{0k})
\end{align}$$

which are Dirichlet and a Normal-Inverse-Wishart distributions. So, to summarize:

1. Fix $\phi, \mu, \Sigma$ and compute $p(z \mid X, \phi, \mu, \Sigma)$
2. Fix $z$ and compute both $p(\phi \mid z)$ and $p(\mu, \Sigma \mid X)$.

You will recognize this as being similar to Expectation-Maximization. The difference lies in that we obtain posterior distributions for $\phi, \mu, \Sigma$ instead of point estimates.

Since the posterior factorizes into marginal distributions into known forms (i.e. Categorical, Dirichlet, Normal, Wishart), we can approximate it using an appropriate recognition distribution.

```julia
# Start a graph
g = FactorGraph()

# Mixture weights are drawn from a Beta distribution (1D analog of a Dirichlet)
@RV phi ~ Beta(1.0, 1.0)

# Marginals for first component
@RV W_1 ~ Wishart([1. 0.; 0. 1.], 2.)
@RV m_1 ~ GaussianMeanPrecision([0.0, 0.0], W_1)

# Marginals of second component
@RV W_2 ~ Wishart([1. 0.; 0. 1.], 2.)
@RV m_2 ~ GaussianMeanPrecision([0.0, 0.0], W_2)

z = Vector{Variable}(undef, N)
y = Vector{Variable}(undef, N)
for i = 1:N

    # Assignment variable is drawn from a Bernoulli distribution (1D analog of a Categorical)
    @RV z[i] ~ Bernoulli(phi)

    # ForneyLab has a special node for a Gaussian mixture component
    @RV y[i] ~ GaussianMixture(z[i], m_1, W_1, m_2, W_2)

    # Add data
    placeholder(y[i], :y, dims=(2,), index=i)
end
```

Now, you need to specify a distribution with which you will approximate the posterior distribution that you would normally obtain through exact Bayesian inference.

```julia
# Specify recognition factorization (mean-field)
q = RecognitionFactorization(phi, m_1, W_1, m_2, W_2, z, ids=[:PH, :M1, :W1, :M2, :W2, :Z])

# Generate the algorithm
algo = variationalAlgorithm(q)
algo_F = freeEnergyAlgorithm(q);

# Load algorithms
eval(Meta.parse(algo))
eval(Meta.parse(algo_F));
```

Now, we execute the inference algorithm.

```julia
# Convert data to a format suited to ForneyLab
observed_ = [observations[i,:] for i in 1:N]

# Add to data dictionary
data = Dict(:y => observed_)

# Prepare recognition distributions
marginals = Dict()
marginals[:phi] = ProbabilityDistribution(Beta, a=1.0, b=1.0)
marginals[:m_1] = ProbabilityDistribution(Multivariate, GaussianMeanPrecision, m=[-1.0, -1.0], w=[1. 0.;0. 1.])
marginals[:W_1] = ProbabilityDistribution(MatrixVariate, Wishart, v=[1. 0.;0. 1.], nu=2.)
marginals[:m_2] = ProbabilityDistribution(Multivariate, GaussianMeanPrecision, m=[+1.0, +1.0], w=[1. 0.;0. 1.])
marginals[:W_2] = ProbabilityDistribution(MatrixVariate, Wishart, v=[1. 0.;0. 1.], nu=2.)
for i = 1:N
    marginals[:z_*i] = ProbabilityDistribution(Bernoulli, p=1/2.)
end

# Number of iterations
num_iterations = 20

# Preallocate free energy tracking array
F = Float64[]

# Execute algorithm
for i = 1:num_iterations

    # Update recognition factors
    stepZ!(data, marginals)
    stepPH!(data, marginals)
    stepM1!(data, marginals)
    stepW1!(data, marginals)
    stepM2!(data, marginals)
    stepW2!(data, marginals)

    # Store variational free energy for visualization
    push!(F, freeEnergy(data, marginals))
end
```

```julia
# Plot free energy to check for convergence
plot(1:num_iterations, F, color="black", label="")
xlabel!("VMP iteration")
ylabel!("Variational free energy")
```

Visualize the results.

```julia
# Estimated means (ForneyLab.unsafeMean retrieves parameters directly)
μ1_estimated = ForneyLab.unsafeMean(marginals[:m_1])
μ2_estimated = ForneyLab.unsafeMean(marginals[:m_2])

# Estimated precisions
W1_estimated = ForneyLab.unsafeMean(marginals[:W_1])
W2_estimated = ForneyLab.unsafeMean(marginals[:W_2])

# Invert to covariances
Σ1_estimated = inv(W1_estimated)
Σ2_estimated = inv(W2_estimated)

# Plot data and overlay estimated posterior probabilities
plot_clusters(observations,
              μ=[μ1_estimated, μ2_estimated],
              Σ=[Σ1_estimated, Σ2_estimated],
              x1=range(-10,stop=10),
              x2=range(-10,stop=10),
              size=(600,400))
```

### Effect of initialization

Variational inference is essentially an optimization problem where we try to improve the fit of the approximating distributions $q$ to the generative model $p$ in each iteration. Once we reach a set of parameters for which any change leads to a worse fit, we say that we are done and that the inference algorithm has converged. Mixture models are special in that there are many configurations of clusters (i.e. approximating distributions $q$) that are "locally optimal" (i.e. changing the parameters a bit will always produce a worse fit). As such, it matters how you initialize your approximating distributions: each corner of parameter space might produce a different outcome.

We will change the initialization and see how it affects the resulting clusters.

```julia
# Start a graph
g2 = FactorGraph()

# Mixture weights are drawn from a Beta distribution (1D analog of a Dirichlet)
@RV phi ~ Beta(1.0, 1.0) # => higher concentration parameters

# Marginals for first component
@RV W_1 ~ Wishart(10*[1. 0.; 0. 1.], 2.) # => lower precision
@RV m_1 ~ GaussianMeanPrecision([-1.0, -1.0], W_1) # => start further apart

# Marginals of second component
@RV W_2 ~ Wishart(10*[1. 0.; 0. 1.], 2.) # => lower precision
@RV m_2 ~ GaussianMeanPrecision([+1.0, +1.0], W_2) # => start further apart

z = Vector{Variable}(undef, N)
y = Vector{Variable}(undef, N)
for i = 1:N

    # Assignment variable is drawn from a Bernoulli distribution (1D analog of a Categorical)
    @RV z[i] ~ Bernoulli(phi)

    # ForneyLab has a special node for a Gaussian mixture component
    @RV y[i] ~ GaussianMixture(z[i], m_1, W_1, m_2, W_2)

    # Add data
    placeholder(y[i], :y, dims=(2,), index=i)
end

# Specify recognition factorization (mean-field)
q = RecognitionFactorization(phi, m_1, W_1, m_2, W_2, z,
                             ids=[:PH, :M1, :W1, :M2, :W2, :Z])

# Generate the algorithm
algo = variationalAlgorithm(q)
algo_F = freeEnergyAlgorithm(q);

# Load algorithms
eval(Meta.parse(algo))
eval(Meta.parse(algo_F));

# Prepare recognition distributions
marginals = Dict()
marginals[:phi] = ProbabilityDistribution(Beta, a=100.0, b=100.0)
marginals[:m_1] = ProbabilityDistribution(Multivariate, GaussianMeanPrecision, m=[-0.0, -0.0], w=.1*[1. 0.;0. 1.])
marginals[:W_1] = ProbabilityDistribution(MatrixVariate, Wishart, v=.1*[1. 0.;0. 1.], nu=3.)
marginals[:m_2] = ProbabilityDistribution(Multivariate, GaussianMeanPrecision, m=[+5.0, -5.0], w=10*[1. 0.;0. 1.])
marginals[:W_2] = ProbabilityDistribution(MatrixVariate, Wishart, v=10*[1. 0.;0. 1.], nu=3.)
for i = 1:N
    marginals[:z_*i] = ProbabilityDistribution(Bernoulli, p=0.9)
end

# Preallocate free energy tracking array
F = Float64[]

for i = 1:num_iterations

    # Update recognition factors
    stepZ!(data, marginals)
    stepPH!(data, marginals)
    stepM1!(data, marginals)
    stepW1!(data, marginals)
    stepM2!(data, marginals)
    stepW2!(data, marginals)

    # Store variational free energy for visualization
    push!(F, freeEnergy(data, marginals))
end

# Estimated means
μ1_estimated = ForneyLab.unsafeMean(marginals[:m_1])
μ2_estimated = ForneyLab.unsafeMean(marginals[:m_2])

# Estimated precisions
W1_estimated = ForneyLab.unsafeMean(marginals[:W_1])
W2_estimated = ForneyLab.unsafeMean(marginals[:W_2])

# Enforce positive-definiteness and Hermitian condition and invert to covariances
Σ1_estimated = inv(W1_estimated)
Σ2_estimated = inv(W2_estimated)

# Plot data and overlay estimated posterior probabilities
plot_clusters(observations,
              μ=[μ1_estimated, μ2_estimated],
              Σ=[Σ1_estimated, Σ2_estimated],
              x1=range(-10,stop=10),
              x2=range(-10,stop=10),
              size=(600,400))
```

### Assignment: informativeness of priors

In Bayesian inference, the choice of prior affects the resulting posterior distribution. Only in the limit, when we have infinite data, does the prior lose its influence over the posterior. We saw in the regression & classification notebook that the prior could have a large effect on the final weight estimates. But that is not necessarily the case for other modeling problems.

In this assignment, you will play with other choices of prior distribution parameters in the mixture model setting. How do the means and variances affect the configuration of the clusters?

```julia
### Your code here
```

### Assignment: 3 clusters

Can you change the model to work with 3 clusters instead of 2?

```julia
### Your code here
```


# Convex optimization

```python
# Import Gurobi package
import gurobipy as gb
```

## Exercise

\begin{align}
Min. 98x_1 + 100x_2 + (98x_1 - 100x_2)^2\\
6x_1 + 3x_2 \geq 500 \\
7x_2 \geq 500 \\
2x_1 + 4x_2 \geq 500 \\
x1, x2 \geq 0
\end{align}

```python
# Create model
m = gb.Model()

# Add variables
x1 = m.addVar()
x2 = m.addVar()

# Define objective function
m.setObjective(98 * x1 + 100 * x2 + (98 * x1 - 100 * x2) * (98 * x1 - 100 * x2),
               sense=gb.GRB.MINIMIZE)

# Add constraints
m.addConstr(6 * x1 + 3 * x2 >= 500)
m.addConstr(7 * x2 >= 500)
m.addConstr(2 * x1 + 4 * x2 >= 500)

# Specify Gurobi options
m.setParam('OutputFlag', 0)

# Run optimization
m.optimize()
```

```python
# Get values of decision variables
print('x1', x1.X)
print('x2', x2.X)
print('Objective function value', m.ObjVal)
```

    x1 84.45836376917809
    x2 82.77081811541615
    Objective function value 16554.027757487907

```python

```

Basics of gurobi optimization/02 Convex optimization.ipynb


# Sistema CSTR

Este cuaderno se estudia el sistema discutido en Seborg, Edgar, Melichamp & Doyle "Process Dynamics and Control" (3rd Ed).

```{figure} cstr.png
:name: cstr
:width: 400px
:alt: cstr
:align: center

Reactor continuo de tanque agitado
```

## Modelo

$$
\begin{align}
k &= k_0\exp\left(\frac{-E}{RT}\right)\\
w &=q\rho \\
V\frac{\mathrm{d}c_A}{\mathrm{d}t} &= q(c_{Ai} - c_A) - V k c_A \\
V\rho C\frac{\mathrm{d}T}{\mathrm{d}t} &= wC(T_i - T) + (-\Delta H_R) V k c_A + UA(T_c - T)qc
\end{align}
$$

Balance de Energía:

$$\frac{dE}{dt} = E_{in} - E_{out}$$

Balance de Masa:

$$\frac{dm}{dt} = \dot m_{in} - \dot m_{out}$$

Balance de composición:

$$\frac{dN_A}{dt} = N_{A,in} - N_{A, out} + N_{A, \text{gen}} - N_{A, \text{consumed}}$$

### Descripción

El modelo dinámico del sistema anterior se módifico levenmente para que se manipule el caudal de entrada de refrigerante y no la temperatura como figura en el diagrama.

El objetivo de conrol de esta planta es lograr mantener la temperatura constante de la reacción a pesar de los cambios en la composición de caudal de entrada al reactor.

```python
import control as ctrl
import numpy as np
import matplotlib.pyplot as plt
```

```python
cA0 = 0.5  # mol/L
T0 = 350  # K
```

```python
def derivs_CSTR(t, x, u, params):

    q = params.get('q', 100)   # L/min
    # cA_i = params.get('cAi', 1)  # mol/L
    T_i = params.get('T_i', 350)  # K
    V = params.get('V', 100)  # L
    rho = params.get('rho', 1000) # g/L
    C = params.get('C', 0.239) # J/(g K)
    Hr = params.get('Hr', -5e4)  # J/(g K)
    E_over_R = params.get('E_over_R', 8750)  # K
    k0 = params.get('ko', 7.2e10)   # 1/min
    UA = params.get('UA', 5e2)  # J/(L K)
    Tc = params.get('Tc', 300)  # K

    qc, cA_i= u
    cA, T = x

    k = k0*np.exp(-E_over_R/T)
    w = q*rho
    dcAdt = q*(cA_i - cA)/V - k*cA
    dTdt = 1/(V*rho*C)*(w*C*(T_i - T) - Hr*V*k*cA + UA*(Tc - T)*qc)
    return dcAdt, dTdt

def outputs_CSTR(t, x, u, params):
    return x[0], x[1]

```

```python
CSTR_sys=ctrl.NonlinearIOSystem(derivs_CSTR,
                                outputs_CSTR,
                                inputs=('Tc', 'CA_i'),
                                outputs=('cA', 'T'),
                                states=('dcA', 'dT'))
```

```python
T = np.linspace(0, 10, 3000)
x0 = [cA0, T0]

def Usignal(T):
    U = np.zeros((2, len(T)))
    for i, t in enumerate(T):
        U[0,i] = 105 if t>1 else 100
        U[1,i] = 1 if t >1 else 1

    return U

U=Usignal(T)
```

```python
t,y = ctrl.input_output_response(CSTR_sys, T, U, x0)
```

```python
plt.plot(t,y[0,:])
```

```python

```

```python

```


# Introduction to Bayesian modeling with PyMC3

This post is devoted to give an introduction to Bayesian modeling using [PyMC3](https://pymc-devs.github.io/pymc3/notebooks/getting_started.html), an open source probabilistic programming framework written in Python. Part of this material was presented in the Python Users Berlin (PUB) meet up.

Why PyMC3? As described in the documentation:

- PyMC3’s user-facing features are written in pure Python, it leverages [Theano](http://deeplearning.net/software/theano/) to transparently transcode models to C and compile them to machine code, thereby boosting performance.

- Theano is a library that allows expressions to be defined using generalized vector data structures called tensors, which are tightly integrated with the popular [NumPy](http://www.numpy.org/) ndarray data structure.

In addition, from a practical point of view, PyMC3 syntax is very transpartent from the mathematical point of view.

This post is not aimed to give a full treatment of the ~~mathematical details~~, as there are many good (complete and detailed) references around these topics. Also, we are not going to dive deep into PyMC3 as all the details can be found in the documentation. Instead, we are interested in giving an overview of the basic mathematical consepts combinded with examples (writen in Python code) which should make clear why [Monte Carlo](https://en.wikipedia.org/wiki/Monte_Carlo_method) simulations are useful in Bayesian modeling.

# 1. Mathematical Background

# 1.1 Bayes Theorem

## Frequentist vs Bayesian

*The essential difference between frequentist inference and Bayesian inference is the same as the difference between the two interpretations of what a "probability" means*.

**Frequentist inference** is a method of statistical inference in which conclusions from data is obtained by emphasizing the frequency or proportion of the data.

**Bayesian inference** is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available.

## Conditional Probability

Let \\(A\\) and \\(B\\) be two events, then the *conditional probability* of \\(A\\) given \\(B\\) is defined as the ratio

\begin{equation}
P(A|B):=\frac{P(A\cap B)}{P(B)}
\end{equation}

*Remark:* Formally we have a [probability space](https://en.wikipedia.org/wiki/Probability_space) \\((\Omega, \mathcal{F}, P)\\), where \\(\Omega\\) is the sample space, \\(\mathcal{F}\\) is a \\(\sigma\\)-algebra on \\(\Omega\\) and \\(P\\) is a probability measure. The events \\(A\\), and \\(B\\) are elements of \\(\mathcal{F}\\) and we assume that \\(P(B)\neq 0\\).

Observe in particular

\begin{equation}
P(A|B)P(B)=P(A\cap B)=P(B\cap A) = P(B|A)P(A)
\end{equation}

## Bayes Theorem

From the last formula we obtain the relation

\begin{equation}
P(A|B)=\frac{P(B|A)P(A)}{P(B)}
\end{equation}

which is known as [Bayes Theorem](https://en.wikipedia.org/wiki/Bayes%27_theorem).

**Example:** Suppose you are in the U-Bahn and you see a person with long hair. You want to know the probablity that this person is a woman. Consider the events \\(A=\\) woman \\(B=\\) long hair. You want to compute \\(P(A|B)\\). Suppose that you estimate \\(P(A)=0.5\\), \\(P(B)=0.4\\) and \\(P(B|A)=0.7\\) (the probability that a woman has long hair). Then, given these prior estimated probabilities, Bayes theorem gives

\begin{equation}
P(A|B)=\frac{P(B|A)P(A)}{P(B)} = \frac{0.7\times 0.5}{0.4} = 0.875.
\end{equation}

## Bayesian Approach to Data Analysis

Assume that you have a sample of observations \\(y_1,..., y_n\\) of a random variable \\(Y\sim f(y|\theta)\\), where \\(\theta\\) is a parameter for the distribution. Here we consider \\(\theta\\) as a random variable as well. Following Bayes Theorem (its continuous version) we can write.

\begin{equation}
f(\theta|y)=\frac{f(y|\theta)f(\theta)}{f(y)} =
\displaystyle{\frac{f(y|\theta)f(\theta)}{\int f(y|\theta)f(\theta)d\theta}}
\end{equation}

- The function \\(f(y|\theta)\\) is called the *likelihood*.

- \\(f(\theta)\\) is the *prior* distribution of \\(\theta\\).

Note that \\(f(y)\\) *does not* depend on \\(\theta\\) (just on the data), thus it can be considered as a "normalizing constant". In addition, it is often the case that the integral above is not easy to compute. Nevertheless, it is enough to consider the relation:

\begin{equation}
f(\theta|y)  \propto \text{likelihood} \times \text{prior}.
\end{equation}

(Here \\(\propto\\) denotes the proportionality relation)

## Example: Poisson Data

In order to give a better sense of the relation above we are going to study a concrete example. Consider a \\(n\\) samples of \\(Y\sim Poiss(\lambda)\\). Recall that the [Poisson distribution](https://en.wikipedia.org/wiki/Poisson_distribution) is given by:

$$
\displaystyle{
f(y_i|\lambda)=\frac{e^{-\lambda}\lambda^{y_i}}{y_i!}
}
$$

where \\(\lambda>0\\). It is easy to verify that \\(E(Y)=\lambda\\) and \\(Var(Y)=\lambda\\). Parallel to the formal discussion, we are going to implement a numerical simulation:

```python
import numpy as np
import scipy.stats as ss

# We set a seed so that the results are reproducible.
np.random.seed(5)

# number of samples.
n = 100

# true parameter.
lam_true = 2

# sample array.
y = np.random.poisson(lam=lam_true, size=n)

y
```

    array([2, 4, 1, 0, 2, 2, 2, 2, 1, 1, 3, 2, 0, 1, 3, 3, 4, 2, 0, 0, 3, 6,
           1, 2, 1, 2, 5, 2, 3, 0, 1, 3, 1, 4, 1, 2, 4, 0, 6, 4, 1, 2, 2, 0,
           1, 2, 4, 4, 1, 3, 0, 3, 3, 2, 4, 2, 2, 1, 1, 2, 5, 2, 3, 0, 1, 1,
           1, 3, 4, 1, 3, 4, 2, 1, 2, 4, 2, 2, 1, 0, 2, 2, 3, 0, 3, 3, 4, 2,
           2, 1, 2, 1, 3, 0, 1, 0, 3, 3, 1, 2])

```python
# mean of the sample.
y.mean()
```

    2.06

```python
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()
%matplotlib inline

# Histogram of the sample.
plt.figure(figsize=(8, 6))
plt.hist(y, bins=15)
plt.title('Histogram of Simulated Data');
```

### Prior: Gamma Distribution

Let us consider a [gamma](https://en.wikipedia.org/wiki/Gamma_distribution) prior distribution for the parameter \\(\lambda \sim \Gamma(a,b)\\). Recall that the density function for the gamma distribution is

\begin{equation}
f(\lambda)=\frac{b^a}{\Gamma(a)}\lambda^{a-1} e^{-b\lambda}
\end{equation}

where \\(a>0\\) is the *shape* parameter and \\(b>0\\) is the *rate parameter*.

The *expected value* and *variance* of the gamma distribution is

$$
E(\lambda)=\frac{a}{b}
\quad
\text{and}
\quad
Var(\lambda)=\frac{a}{b^2}
$$

```python
# Parameters of the prior gamma distribution.
a = 3.5 # shape
b = 2 # rate = 1/scale

x = np.linspace(start=0,stop=10, num=100)

plt.figure(figsize=(8, 6))
plt.plot(x, ss.gamma.pdf(x,a=a,scale=1/b), 'r-')
plt.title('Gamma Density Function for a={} and b={}'.format(a,b))

# Define the prior distribution.
prior = lambda x: ss.gamma.pdf(x, a=a, scale=1/b)
```

### Likelihood

As the observations are independent the [likelihood](https://en.wikipedia.org/wiki/Likelihood_function) function is

\begin{align}
f(y|\lambda)=&\prod_{i=1}^{n} \frac{e^{-\lambda}\lambda^{y_i}}{y_i!}
=\frac{e^{-n\lambda}\lambda^{\sum_{i=1}^n y_i}}{\prod_{i=1}^{n}y_i!}
\end{align}

```python
import scipy.special as sp

# Define the likelihood function.
def likelihood(lam,y):

    factorials = np.apply_along_axis(
        lambda x: sp.gamma(x+1),
        axis=0,
        arr=y
    )

    numerator = np.exp(-lam*y.size)*(lam**y.sum())

    denominator = np.multiply.reduce(factorials)

    return numerator/denominator
```

### Posterior distribution for \\(\lambda\\) up to a constant

As we are just interested in the structure of the posterior distribution, up to a constant, we see

\begin{align}
f(\lambda|y)\propto & \text{likelihood} \times \text{prior}\\
\propto & \quad f(y|\lambda)f(\lambda)\\
\propto & \quad e^{-n\lambda}\lambda^{\sum_{i=1}^n y_i} \lambda^{a-1} e^{-b\lambda}\\
\propto & \quad \lambda^{\left(\sum_{i=1}^n y_i+a\right)-1} e^{-(n+b)\lambda}\\
\end{align}

```python
# Define the posterior distribution.
# (up to a constant)
def posterior_up_to_constant(lam,y):
    return likelihood(lam,y)*prior(lam)

# Plot of the prior and (scaled) posterior distribution
# for the parameter lambda.
#
# We multiply the posterior distrubution function
# by the amplitude factor 2.5e74 to make it comparable
# with the prior gamma distribution.
plt.figure(figsize=(8, 6))
plt.plot(x, 2.0e74*posterior_up_to_constant(x,y), label='posterior')
plt.plot(x, ss.gamma.pdf(x,a=a,scale=1/b), 'r-', label='prior')
plt.legend();
```

### True posterior distribution for \\(\lambda\\)

In fact, as  \\(f(\lambda|y) \propto\: \lambda^{\left(\sum_{i=1}^n y_i+a\right)-1} e^{-(n+b)\lambda}\\), one verifies that the posterior distribution is again a gamma

\begin{align}
f(\lambda|y) = \Gamma\left(\sum_{i=1}^n y_i+a, n+b\right)
\end{align}

This means that the gamma and Poisson distribution form a [conjugate pair](https://en.wikipedia.org/wiki/Conjugate_prior).

```python
def posterior(lam,y):

    shape = a + y.sum()
    rate = b + y.size

    return ss.gamma.pdf(lam, shape, scale=1/rate)

plt.figure(figsize=(8, 6))
plt.plot(x, posterior(x,y))
plt.plot(x, ss.gamma.pdf(x,a=a,scale=1/b), 'r-')
plt.title('Prior and Posterior Distributions');
```

We indeed see how the posterior distribution is concentrated around the true parameter \\(\lambda=2\\).

Note that the posterior mean is

\begin{align}
\frac{\sum_{i=1}^n y_i+a}{n+b} = \frac{b}{n+b}\frac{a}{b}+\frac{n}{n+b}\frac{\sum_{i=1}^n y_i}{n}
\end{align}

That is, it is a weighted average of the prior mean \\(a/b\\) and the sample average \\(\bar{y}\\). As \\(n\\) increases,

\begin{align}
\lim_{n\rightarrow +\infty}\frac{b}{n+b}\frac{a}{b}+\frac{n}{n+b}\frac{\sum_{i=1}^n y_i}{n} = \bar{y}.
\end{align}

```python
# Posterior gamma parameters.
shape = a + y.sum()
rate = b + y.size

# Posterior mean.
shape/rate
```

    2.053921568627451

# 1.2 Markov Chain Monte Carlo  (MCMC) Approach

In the last example the posterior distribution was easy to identify. However, in paractice this is not usually the case and therefore, via Bayes Theorem, we would only know the posterior distribution up to a constant. This motivates the idea of using Monte Carlo simulation methods. How can we sample from a distribution that we do not know? The Metropolis–Hastings algorithm, explaned next, is one approach to tackle this problem.

## Metropolis–Hastings algorithm

Let \\(\phi\\) be a function that is proportional to the desired probability distribution \\(f\\).

**Initialization:**

Pick \\(x_{0}\\) to be the first sample, and choose an arbitrary probability density

\begin{equation}
g(x_{n+1}| x_{n})
\end{equation}

that suggests a candidate for the next sample value \\(x_{n+1}\\). Assume \\(g\\) is symmetric.

**For each iteration:**

Generate a candidate \\(x\\) for the next sample by picking from the distribution \\(g(x|x_n)\\). Calculate the *acceptance ratio*

\begin{equation}
\alpha := \frac{f(x)}{f(x_n)} = \frac{\phi(x)}{\phi(x_n)}
\end{equation}

If \\(\alpha \geq 1\\), automatically accept the candidate by setting

\begin{equation}
x_{n+1} = x.
\end{equation}

Otherwise, accept the candidate with probability \\(\alpha \\). If the candidate is rejected, set

\begin{equation}
x_{n+1} = x_{n}.
\end{equation}

Why does this algorithm solve the initial problem? The full explanation is beyond the scope of this post (some references are provided at the end). It relies in the in the following result.

## Ergodic Theorem for Markov Chains

**Theorem (Ergodic Theorem for Markov Chains)** If \\(\{x^{(1)} , x^{(2)} , . . .\}\\) is an *irreducible*, *aperiodic* and *recurrent* [Markov chain](https://en.wikipedia.org/wiki/Markov_chain), then there is a unique probability distribution \\(\pi\\) such that as \\(N\longrightarrow\infty\\),

- \\(P(x^{(N)} ∈ A) \longrightarrow \pi(A)\\).
- \\(\displaystyle{\frac{1}{N}\sum_{n=1}^{N} g(x^{(n)})) \longrightarrow    \int g(x)\pi(x) dx }\\).

*Recall:*

- A Markov chain is said to be **irreducible** if it is possible to get to any state from any state.

- A state \\(n\\) has **period** \\(k\\) if any return to state \\(n\\) must occur in multiples of \\(k\\) time steps.

- If \\(k=1\\), then the state is said to be **aperiodic**.

- A state \\(n\\) is said to be **transient** if, given that we start in state \\(n\\), there is a non-zero probability that we will never return to \\(i\\).

- A state \\(n\\) is **recurrent**  if it is not transient.

# 2. PyMC3 Syntax

Now we perform a MCMC simulation for the data described above. Note how easy is to write the model from the mathematical description.

```python
import pymc3 as pm
import arviz as az

model = pm.Model()

with model:

    # Define the prior of the parameter lambda.
    lam = pm.Gamma('lambda', alpha=a, beta=b)

    # Define the likelihood function.
    y_obs = pm.Poisson('y_obs', mu=lam, observed=y)

    # Consider 2000 draws and 3 chains.
    trace = pm.sample(draws=2000, chains=3)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (3 chains in 4 jobs)
    NUTS: [lambda]
    Sampling 3 chains: 100%|██████████| 7500/7500 [00:01<00:00, 5660.80draws/s]

If we do a trace plot we can see two results:

- We see the simulated posterior distribution for 3 independent Markov Chains (so that, when combined, avoid the dependence on the initial point). The 3 different chains correspond to the color blue, green and orange.

- The sample value of lambda for each iteration.

```python
pm.traceplot(trace);
```

We can also see the mean and quantile information for the posterior distribution.

```python
pm.plot_posterior(trace);
```

# 3. Bayesian Hierarchical Modeling: A Chocolate Cookies Example.

Now we are going to treat a more complicated example which illustrated a hierarchical mdel.
This case of study is taken from the (strongly recomended!) online course:

**Bayesian Statistics: Techniques and Models:**

https://www.coursera.org/learn/mcmc-bayesian-statistics

There, the MCMC simulations are done with [JAGS](http://mcmc-jags.sourceforge.net/) in [R](https://www.r-project.org/). As a matter of fact, this course motivated me to explore an analogous tool for Python.

## 3.1 The data

Assume there is a big factory producing chocolate cookies around the world. The cookies follow a unique recipe, but you want to study the chocolate chips distribution for cookies produced in 5 different locations.

- On the one hand side you would assume that the distribution across the locations is similar, as they all come from a unique recipe. This is why you may not want to model each location separately.

- On the other hand, in reality, as the locations are not exacly the same you might expect some differences between each location.  This is why you may not want to model all locations at once.

To overcome these restrictions, a hierarchical can be a feasible approach.

```python
import pandas as pd

# We begin reading the data into a pandas dataframe.
cookies = pd.read_csv('data/cookies.dat', sep = ' ')

cookies.head()
```

      chips
      location

      0
      12
      1

      1
      12
      1

      2
      6
      1

      3
      13
      1

      4
      12
      1

```python
# Let us verify the number of locations.
cookies.location.unique()
```

    array([1, 2, 3, 4, 5])

Let us start with some visualization of the data.

```python
# Histogram distribution of chocolate chips
# for all cookies.
fig, ax = plt.subplots(figsize=(8, 6))
sns.distplot(cookies['chips'], bins=15, ax=ax);
ax.set(title='Chips Distribution (All Locations)');
```

```python
# Histogram distribution of chocolate chips
# for cookies in each location.
g = sns.FacetGrid(data=cookies, col='location', col_wrap=2, height=3, aspect=2)
g = g.map(sns.distplot, 'chips')
```

```python
# Box plot for different locations.
fig, ax = plt.subplots(figsize=(10,6))
cookies.boxplot(column='chips', by='location', ax=ax);
```

##  3.2 The model: Hierarchical Approach

- Hierarchical Model:

We model the chocolate chip counts by a Poisson distribution with parameter \\(\lambda\\). Motivated by the example above, we choose a gamma prior.

\begin{align}
chips \sim  Poiss(\lambda)
\quad\quad\quad
\lambda \sim  \Gamma(a,b)
\end{align}

- Parametrization:

We parametrize the shape and scale of the gamma prior with the mean \\(\mu\\) and variance \\(\sigma^2\\).

\begin{align}
a=\frac{\mu^2}{\sigma^2}
\quad\quad\quad
b=\frac{\mu}{\sigma^2}
\end{align}

- Prior Distributions:

We further impose prior for these parameters

\begin{align}
\mu  \sim  \Gamma(2,1/5)
\quad\quad\quad
\sigma  \sim  Exp(1)
\end{align}

```python
x = np.linspace(start=0, stop=50, num=100)

fig = plt.figure(figsize=(15, 5))
plt.subplot(1, 2, 1)
plt.plot(x, ss.gamma.pdf(x,a=2,scale=5), 'r-')
plt.title('Prior Distribution for mu \n Gamma Density Function for a={} and b={}'.format(2,1/5))

plt.subplot(1, 2, 2)
x = np.linspace(0,10)
plt.plot(x, ss.expon.pdf(x,1), 'r-')
plt.title('Prior Distribution for sigma2 \n Exponential Density Function')
plt.xlim(1,10)
```

Let us write the model in PyMC3. Note how the syntax mimics the mathematical formulation.

```python
model = pm.Model()

with model:

    # Prior distribution for mu.
    mu = pm.Gamma('mu', alpha=2.0, beta=1.0/5)

    # Prior distribution for sigma2.
    sigma = pm.Exponential('sigma', 1.0)

    # Parametrization for the shape parameter.
    alpha =  mu**2/sigma**2

    # Parametrization for the scale parameter.
    beta = mu/sigma**2

    # Prior distribution for lambda.
    lam = pm.Gamma(
        'lam',
        alpha=alpha,
        beta=beta,
        shape=cookies.location.values.max()
    )

    # Likelihood function for the data.
    chips = [
        pm.Poisson('chips_{}'.format(i),lam[i],
        observed=cookies[cookies.location==i+1].chips.values)
        for i in range(cookies.location.values.max())
    ]

    # Parameters of the simulation:
    # Number of iterations and independent chains.
    n_draws, n_chains = 1000, 3

    n_sim = n_draws*n_chains

    trace = pm.sample(draws=n_draws, chains=n_chains)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (3 chains in 4 jobs)
    NUTS: [lam, sigma, mu]
    Sampling 3 chains: 100%|██████████| 4500/4500 [00:02<00:00, 1815.91draws/s]

## 3.3 Diagnostics

Many [diagnostic](https://pymc-devs.github.io/pymc3/api/diagnostics.html) options are described in the PyMC3 documentation.

```python
pm.traceplot(trace);
```

From the traceplot we see that the chains have converged. We can also have a detailed summary of the posterior distribution for each parameter:

```python
pm.summary(trace)
```

    /opt/anaconda3/lib/python3.7/site-packages/pymc3/stats.py:991: FutureWarning: The join_axes-keyword is deprecated. Use .reindex or .reindex_like on the result to achieve the same functionality.
      axis=1, join_axes=[dforg.index])

      mean
      sd
      mc_error
      hpd_2.5
      hpd_97.5
      n_eff
      Rhat

      mu
      9.104871
      0.952266
      0.019789
      7.285942
      11.142347
      2522.978566
      1.001637

      sigma
      2.075416
      0.716848
      0.013315
      0.988354
      3.541359
      2510.182129
      0.999958

      lam__0
      9.277816
      0.529706
      0.008511
      8.353204
      10.406595
      4050.792217
      0.999807

      lam__1
      6.221088
      0.461659
      0.007158
      5.306228
      7.100886
      3812.117511
      0.999808

      lam__2
      9.520956
      0.544345
      0.008374
      8.516351
      10.607144
      3723.180815
      0.999935

      lam__3
      8.932577
      0.531530
      0.009103
      7.861659
      9.922982
      3353.991802
      0.999881

      lam__4
      11.753710
      0.639027
      0.010974
      10.547972
      13.023027
      3529.810115
      0.999560

We can also see this visually.

```python
pm.plot_posterior(trace);
```

We can verify the convergence of the chains formally using the Gelman Rubin test. Values close to 1.0 mean convergence.

```python
pm.gelman_rubin(trace)
```

    {'mu': 1.0016372262426347,
     'sigma': 0.9999579325854048,
     'lam': array([0.9998075 , 0.99980835, 0.99993469, 0.99988124, 0.99956018])}

We can also test for correlation between samples in the chains. We are aiming for zero auto-correlation to get "random" samples from the posterior distribution.

```python
# Auto-correlation of the parameter sigma for the 3 chains.
pm.autocorrplot(trace, var_names=['sigma'], max_lag=20);
```

```python
# We can also consider all the variables simultaneously.
pm.autocorrplot(trace, max_lag=20);
```

From these plots we see that the auto-correlation is not problematic. Indeed, we can test this through the *effective sample size*, which sould be close to the total sumber of samples `n_sim`.

```python
pm.diagnostics.effective_n(trace)
```

    {'mu': 2522.9785658157243,
     'sigma': 2510.182129406361,
     'lam': array([4050.79221675, 3812.11751115, 3723.18081493, 3353.99180192,
            3529.81011549])}

Finally, we can compute the [Watanabe–Akaike information criterion](https://en.wikipedia.org/wiki/Watanabe–Akaike_information_criterion).

```python
pm.waic(trace, model)
```

    /opt/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:1: FutureWarning: arrays to stack must be passed as a "sequence" type such as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.
      """Entry point for launching an IPython kernel.
    /opt/anaconda3/lib/python3.7/site-packages/pymc3/stats.py:219: UserWarning: For one or more samples the posterior variance of the
            log predictive densities exceeds 0.4. This could be indication of
            WAIC starting to fail see http://arxiv.org/abs/1507.04544 for details

      """)

    WAIC_r(WAIC=789.764634126058, WAIC_se=21.395864846123082, p_WAIC=5.844326163346711, var_warn=1)

## 3.4 Residual analysis

In order to evaluate the model results we analyze the behaviour of the residuals.

```python
# Compute the mean of the simulation.
lambda_mean = np.apply_along_axis(np.mean, 0, trace['lam'])

# Compute for each sample the posterior mean.
cookies['yhat'] = cookies.location.apply(lambda x: lambda_mean[x-1])

# Compute the residuals.
cookies['resid'] = cookies.apply(lambda x: x.chips - x.yhat, axis=1)

cookies.head()
```

      chips
      location
      yhat
      resid

      0
      12
      1
      9.277816
      2.722184

      1
      12
      1
      9.277816
      2.722184

      2
      6
      1
      9.277816
      -3.277816

      3
      13
      1
      9.277816
      3.722184

      4
      12
      1
      9.277816
      2.722184

```python
# Cookies Residuals
fig, ax = plt.subplots(figsize=(8, 6))
cookies.reset_index().plot.scatter(x='index', y='resid', ax=ax)
ax.axhline(y=0.0, color='r', linestyle='--')
ax.set(title='Cookies Residuals', xlabel='Observation');
```

We do not see a particular partern in the scatter plot of the residuals against the observation.

```python
fig, ax = plt.subplots(figsize=(8, 6))
cookies.plot.scatter(x='yhat', y='resid', ax=ax)
ax.axhline(y=0.0, color='red', linestyle='--')
ax.set(title='Cookies Residuals');
```

## 3.5 Predictions

Finally, we are going to illustrate how to use the simulation results to derive predictions.

### 3.5.1 For a known location

Let us consider Location 1. We want, for example, to compute the posterior probability the next cookie in this location has less than 7 chips.

```python
# We generate n_sim samples of a Poisson distribution for
# each value for lam_0 (location 1) simulation..
y_pred_location_1 = np.random.poisson(lam=trace['lam'][:,0] , size=n_sim)

fig, ax = plt.subplots(figsize=(8, 6))
sns.distplot(y_pred_location_1, bins=30, ax=ax)
ax.set(title='Chocolate Chips Distribution for Location 1');
```

```python
# Probability the next cookie in location has less than 7 chips.
(y_pred_location_1 < 7).astype(int).mean()
```

    0.17866666666666667

### 3.5.1 For a new location

Now assume we want to open a new location. First, we want to compute the posterior probability that this new location has \\(\lambda > 15\\).

```python
# Posterior distribution of for a an b
# from the simulated values of mu and sigma2.
post_a = trace['mu']**2/trace['sigma']**2
post_b = trace['mu']/trace['sigma']**2

# We now generate samples of a gamma distribution
# with these generated parameters of a and b.
lambda_pred_dist = np.random.gamma(post_a, 1/post_b, n_sim)

fig, ax = plt.subplots(figsize=(8, 6))
sns.distplot(lambda_pred_dist, bins=30, ax=ax)
ax.set(title='Lambda Predicted Distribution');
```

```python
# Posterior probability a new location has lambda > 15.
(lambda_pred_dist > 15).astype(int).mean()
```

    0.019333333333333334

Now we answer a question at the next level of the hierarchical model. We want to calculate the posterior probability for a cookie produced in a new location to have more than 15 chocolate chips.

```python
# Posterior distribution of the chips.
# Here we use the values of lambda obtained above.
cookies_pred_dist = np.random.poisson(lam=lambda_pred_dist, size=n_sim)

fig, ax = plt.subplots(figsize=(8, 6))
sns.distplot(cookies_pred_dist, bins=30, ax=ax)
ax.set(title='Chocolate Chips Distribution New Location');
```

```python
# Posterior probability that a cookie produced
# in a new location has more than 15 chocolate chips.
(cookies_pred_dist>15).astype(int).mean()
```

    0.054

# 4 References and Further Reading

Here we provide some suggested references used in this post and also to go deeper in the subject.

## 4.1 Bayesian Probability

- [Coursera: Bayesian Statistics: From Concept to Data Analysis](https://www.coursera.org/learn/bayesian-statistics)

- [Coursera: Bayesian Statistics: Techniques and Models](https://www.coursera.org/learn/mcmc-bayesian-statistics)

- [A First Course in Bayesian Statistical Methods, Peter D. Hoff](http://www.springer.com/us/book/9780387922997)

- [An Introduction to Bayesian Analysis: Theory and Methods, Ghosh, Jayanta K., Delampady, Mohan, Samanta, Tapas](http://www.springer.com/la/book/9780387400846)

## 4.2 PyMC3

- [Documentation](https://pymc-devs.github.io/pymc3/index.html)

- [Probabilistic Programming in Python using PyMC, John Salvatier, Thomas Wiecki, Christopher Fonnesbeck](https://arxiv.org/abs/1507.08050)


# Calculate price-equilibrium using simulations

```python
import sys, numpy as np, scipy
from sympy import symbols
from typing import Callable
from log_progress import log_progress
np.random.seed(None)

import matplotlib.pyplot as plt, mpld3
%matplotlib inline
mpld3.enable_notebook()  # to zoom and move in plots

resetSize,r,zmin,zmax,beta,D,L,Supply = symbols('a r  z_{\min} z_{\max} \\beta \\Delta \\ell \\tau', positive=True,finite=True,real=True)
params = {
    L: 10,           # total transfers per pair per day.
    D: 6,            # delta transfers per day (Alice-to-Bob minus Bob-to-Alice) in the asymmetric case.
    beta: 0.01,      # value / transfer-size
    r: 4/100/365,    # interest rate per day
    resetSize: 1.1,  # records per reset tx
    Supply: 288000,  # records per day
    zmin: 0.001,     # min transfer size in bitcoins (for power law distribution)
    zmax: 1,         # max transfer size in bitcoins (for uniform distribution)
}
# NOTE: These are the same params used in the symbolic comnputations (market-equilibrium notebook).
```

```python
if "Simulation" in sys.modules: del sys.modules["Simulation"]
from Simulation import *
sim = PowerlawSymmetricSimulation(params, numOfDays=1000, filenamePrefix="interpolation-tables/powerlaw-symmetric-1000days")
# You can also try the following options:
#sim = PowerlawAsymmetricSimulation(params, numOfDays=1000, filenamePrefix="interpolation-tables/powerlaw-asymmetric-1000days")
#sim = UniformSymmetricSimulation(params, numOfDays=1000, filenamePrefix="interpolation-tables/uniform-symmetric-1000days")
#sim = UniformAsymmetricSimulation(params, numOfDays=1000, filenamePrefix="interpolation-tables/uniform-asymmetric-1000days")

sim.loadTables()
```

    Simulation.py version 1.0

```python
supply = params[Supply]
sim.calculateEquilibriumBlockchainFeeTable(
    numOfDays=1000,
    numsOfUsers=np.linspace(100000,10000000,50),
    supply=supply,
    numOfSamples=50,
    recreateAllSamples=False)
sim.saveTables()
sim.plotEquilibriumBlockchainFeeTable(supply)
```

```python
table=sim.equilibriumBlockchainFeeTables[supply]
xs = table.xValues
ys = table.yValuesAverage

### Log-log regression:
regressionCoeffs = np.polyfit(np.log(xs), np.log(ys), 1)
regressionFunction = lambda x:  regressionCoeffs[0]*x**1 + regressionCoeffs[1]#*x + regressionCoeffs[2]
plt.plot(xs, ys, 'r.')
plt.plot(xs, np.exp(regressionFunction(np.log(xs))), 'g')
```

```python
### Lin-lin regression, higher power:
regressionCoeffs = np.polyfit(xs, ys, 2)
regressionString = "{:.2e} n^2 + {:.2e} n + {:.2e}".format(*regressionCoeffs)
print (regressionString)
regressionFunction = lambda x:  regressionCoeffs[0]*x**2 + regressionCoeffs[1]*x + regressionCoeffs[2]
plt.plot(xs, ys, 'r.')
plt.plot(xs, regressionFunction(xs), 'g')
```

```python
### Lin-lin regression of 1/x:
regressionCoeffs = np.polyfit(1/xs, ys, 1)
print(regressionCoeffs)
regressionFunction = lambda x:  regressionCoeffs[0]*x**1 + regressionCoeffs[1]#*x + regressionCoeffs[2]
plt.plot(xs, ys, 'r.')
plt.plot(xs, regressionFunction(1/xs), 'g')
```

```python

```