# Optimización media-varianza

La **teoría de portafolios** es uno de los avances más importantes en las finanzas modernas e inversiones.
- Apareció por primera vez en un [artículo corto](https://www.math.ust.hk/~maykwok/courses/ma362/07F/markowitz_JF.pdf) llamado "Portfolio Selection" en la edición de Marzo de 1952 de "the Journal of Finance".
- Escrito por un desconocido estudiante de la Universidad de Chicago, llamado Harry Markowitz.
- Escrito corto (sólo 14 páginas), poco texto, fácil de entender, muchas gráficas y unas cuantas referencias.
- No se le prestó mucha atención hasta los 60s.

Finalmente, este trabajo se convirtió en una de las más grandes ideas en finanzas, y le dió a Markowitz el Premio Nobel casi 40 años después.
- Markowitz estaba incidentalmente interesado en los mercados de acciones e inversiones.
- Estaba más bien interesado en entender cómo las personas tomaban sus mejores decisiones cuando se enfrentaban con "trade-offs".
- Principio de conservación de la miseria. O, dirían los instructores de gimnasio: "no pain, no gain".
- Si queremos más de algo, tenemos que perder en algún otro lado.
- El estudio de este fenómeno era el que le atraía a Markowitz.

De manera que nadie se hace rico poniendo todo su dinero en la cuenta de ahorros. La única manera de esperar altos rendimientos es si se toma bastante riesgo. Sin embargo, riesgo significa también la posibilidad de perder, tanto como ganar.

Pero, ¿qué tanto riesgo es necesario?, y ¿hay alguna manera de minimizar el riesgo mientras se maximizan las ganancias?
- Markowitz básicamente cambió la manera en que los inversionistas pensamos acerca de esas preguntas.
- Alteró completamente la práctica de la administración de inversiones.
- Incluso el título de su artículo era innovador. Portafolio: una colección de activos en lugar de tener activos individuales.
- En ese tiempo, un portafolio se refería a una carpeta de piel.
- En el resto de este módulo, nos ocuparemos de la parte analítica de la teoría de portafolios, la cual puede ser resumida en dos frases:
 - No pain, no gain.
 - No ponga todo el blanquillo en una sola bolsa.

**Objetivos:**
- ¿Qué es la línea de asignación de capital?
- ¿Qué es el radio de Sharpe?
- ¿Cómo deberíamos asignar nuestro capital entre un activo riesgoso y un activo libre de riesgo?

*Referencia:*
- Notas del curso "Portfolio Selection and Risk Management", Rice University, disponible en Coursera.
___

## 1. Línea de asignación de capital

### 1.1. Motivación

El proceso de construcción de un portafolio tiene entonces los siguientes dos pasos:
1. Escoger un portafolio de activos riesgosos.
2. Decidir qué tanto de tu riqueza invertirás en el portafolio y qué tanto invertirás en activos libres de riesgo.

Al paso 2 lo llamamos **decisión de asignación de activos**.

Preguntas importantes:
1. ¿Qué es el portafolio óptimo de activos riesgosos?
 - ¿Cuál es el mejor portafolio de activos riesgosos?
 - Es un portafolio eficiente en media-varianza.
2. ¿Qué es la distribución óptima de activos?
 - ¿Cómo deberíamos distribuir nuestra riqueza entre el portafolo riesgoso óptimo y el activo libre de riesgo?
 - Concepto de **línea de asignación de capital**.
 - Concepto de **radio de Sharpe**.

Dos suposiciones importantes:
- Funciones de utilidad media-varianza.
- Inversionista averso al riesgo.

La idea sorprendente que saldrá de este análisis, es que cualquiera que sea la actitud del inversionista de cara al riesgo, el mejor portafolio de activos riesgosos es idéntico para todos los inversionistas.

Lo que nos importará a cada uno de nosotros en particular, es simplemente la desición óptima de asignación de activos.
___

### 1.2. Línea de asignación de capital

Sean:
- $r_s$ el rendimiento del activo/portafolio riesgoso,
- $r_f$ el rendimiento libre de riesgo, y
- $w$ la fracción invertida en el activo riesgoso.

 Realizar deducción de la línea de asignación de capital en el tablero.

**Tres doritos después...**

#### Línea de asignación de capital (LAC):
$E[r_p]$ se relaciona con $\sigma_p$ de manera afín. Es decir, mediante la ecuación de una recta:

$$E[r_p]=r_f+\frac{E[r_s-r_f]}{\sigma_s}\sigma_p.$$

- La pendiente de la LAC es el radio de Sharpe $\frac{E[r_s-r_f]}{\sigma_s}=\frac{E[r_s]-r_f}{\sigma_s}$,
- el cual nos dice qué tanto rendimiento obtenemos por unidad de riesgo asumido en la tenencia del activo (portafolio) riesgoso.

Ahora, la pregunta es, ¿dónde sobre esta línea queremos estar?
___

### 1.3. Resolviendo para la asignación óptima de capital

Recapitulando de la clase pasada, tenemos las curvas de indiferencia: **queremos estar en la curva de indiferencia más alta posible, que sea tangente a la LAC**.

 Ver en el tablero.

Analíticamente, el problema es

$$\max_{w} \quad E[U(r_p)]\equiv\max_{w} \quad E[r_p]-\frac{1}{2}\gamma\sigma_p^2,$$

donde los puntos $(\sigma_p,E[r_p])$ se restringen a estar en la LAC, esto es $E[r_p]=r_f+\frac{E[r_s-r_f]}{\sigma_s}\sigma_p$ y $\sigma_p=w\sigma_s$. Entonces el problema anterior se puede escribir de la siguiente manera:

$$\max_{w} \quad r_f+wE[r_s-r_f]-\frac{1}{2}\gamma w^2\sigma_s^2.$$

 Encontrar la $w$ que maximiza la anterior expresión en el tablero.

**Tres doritos después...**

La solución es entonces:

$$w^\ast=\frac{E[r_s]-r_f}{\gamma\sigma_s^2}.$$

De manera intuitiva:
- $w^\ast\propto E[r_s-r_f]$: a más exceso de rendimiento que se obtenga del activo riesgoso, más querremos invertir en él.
- $w^\ast\propto \frac{1}{\gamma}$: mientras más averso al riesgo seas, menos querrás invertir en el activo riesgoso.
- $w^\ast\propto \frac{1}{\sigma_s^2}$: mientras más riesgoso sea el activo, menos querrás invertir en él.
___

## 2. Ejemplo de asignación óptima de capital: acciones y billetes de EU

Pongamos algunos números con algunos datos, para ilustrar la derivación que acabamos de hacer.

En este caso, consideraremos:
- **Portafolio riesgoso**: mercado de acciones de EU (representados en algún índice de mercado como el S&P500).
- **Activo libre de riesgo**: billetes del departamento de tesorería de EU (T-bills).

Tenemos los siguientes datos:

$$E[r_{US}]=11.9\%,\quad \sigma_{US}=19.15\%, \quad r_f=1\%.$$

Recordamos que podemos escribir la expresión de la LAC como:

\begin{align}
E[r_p]&=r_f+\left[\frac{E[r_{US}-r_f]}{\sigma_{US}}\right]\sigma_p\\
      &=0.01+\text{S.R.}\sigma_p,
\end{align}

donde $\text{S.R}=\frac{0.119-0.01}{0.1915}\approx0.569$ es el radio de Sharpe (¿qué es lo que es esto?).

Grafiquemos la LAC con estos datos reales:

```python
# Importamos librerías que vamos a utilizar
from matplotlib import pyplot as plt
import numpy as np
```

```python
# Datos
Ers = 0.119
ss = 0.1915
rf = 0.01
# Radio de Sharpe para este activo
RS = (Ers - rf) / ss
# Vector de volatilidades del portafolio (sugerido: 0% a 30%)
sp = np.linspace(0, 0.3, 101)
# LAC
lac = RS * sp + rf
```

```python
# Gráfica
plt.figure(figsize=(6, 4))
plt.plot(0, rf, "or", ms=10, label="Activo libre de riesgo")
plt.plot(ss, Ers, "og", ms=10, label="Portafolio de activos riesgosos")
plt.plot(sp, lac, '--k', lw=3, label="Línea de asignación de capital (LAC)")
plt.grid()
plt.legend()
plt.xlabel("Volatilidad $\sigma$")
plt.ylabel("Rendimiento esperado $E[r]$")
```

Bueno, y ¿en qué punto de esta línea querríamos estar?
- Pues ya vimos que depende de tus preferencias.
- En particular, de tu actitud de cara al riesgo, medido por tu coeficiente de aversión al riesgo.

Solución al problema de asignación óptima de capital:

$$\max_{w} \quad E[U(r_p)]$$

$$w^\ast=\frac{E[r_s-r_f]}{\gamma\sigma_s^2}$$

Dado que ya tenemos datos, podemos intentar para varios coeficientes de aversión al riesgo:

```python
# importar pandas
import pandas as pd
```

```python
# Crear un DataFrame con los pesos, rendimiento
# esperado y volatilidad del portafolio óptimo
# entre los activos riesgoso y libre de riesgo
# cuyo índice sean los coeficientes de aversión
# al riesgo del 1 al 10 (enteros)
gamma = np.arange(1, 11)
optimal_alloc = pd.DataFrame(data={"gamma": gamma,
                                   "w_opt": (Ers - rf) / (gamma * ss**2),
                                   "1-w_opt": 1 - (Ers - rf) / (gamma * ss**2)
                                  })
optimal_alloc
```

      gamma
      w_opt
      1-w_opt

      0
      1
      2.972275
      -1.972275

      1
      2
      1.486137
      -0.486137

      2
      3
      0.990758
      0.009242

      3
      4
      0.743069
      0.256931

      4
      5
      0.594455
      0.405545

      5
      6
      0.495379
      0.504621

      6
      7
      0.424611
      0.575389

      7
      8
      0.371534
      0.628466

      8
      9
      0.330253
      0.669747

      9
      10
      0.297227
      0.702773

¿Cómo se interpreta $w^\ast>1$?
- Cuando $0<w^\ast<1$, entonces $0<1-w^\ast<1$. Lo cual implica posiciones largas en el mercado de activos y en el activo libre de riesgo.
- Por el contrario, cuando $w^\ast>1$, tenemos $1-w^\ast<0$. Lo anterior implica una posición corta en el activo libre de riesgo (suponiendo que se puede) y una posición larga (de más del 100%) en el mercado de activos: apalancamiento.

# Anuncios parroquiales.

## 1. Quiz la siguiente clase.

## 2. Pueden consultar sus calificaciones en el siguiente [enlace](https://docs.google.com/spreadsheets/d/1XC4vqiKU8jUoT1Hd3UEuaaLv7EMbUjCc9Ei9YSLstCw/edit?usp=sharing).

Created with Jupyter by Esteban Jiménez Rodríguez.


# Chapter 3

_______

## Opening the black box of MCMC

The previous two chapters hid the inner-mechanics of PyMC, and more generally Markov Chain Monte Carlo (MCMC), from the reader. The reason for including this chapter is three-fold. The first is that any book on Bayesian inference must discuss MCMC. I cannot fight this. Blame the statisticians. Secondly, knowing the process of MCMC gives you insight into whether your algorithm has converged. (Converged to what? We will get to that) Thirdly, we'll understand *why* we are returned thousands of samples from the posterior as a solution, which at first thought can be odd.

### The Bayesian landscape

When we setup a Bayesian inference problem with $N$ unknowns, we are implicitly creating an $N$ dimensional space for the prior distributions to exist in. Associated with the space is an additional dimension, which we can describe as the *surface*, or *curve*, that sits on top of the space, that reflects the *prior probability* of a particular point. The surface on the space is defined by our prior distributions. For example, if we have two unknowns $p_1$ and $p_2$, and priors for both are $\text{Uniform}(0,5)$, the space created is a square of length 5 and the surface is a flat plane that sits on top of the square (representing that every point is equally likely).

```
%matplotlib inline
import scipy.stats as stats
from IPython.core.pylabtools import figsize
import numpy as np
figsize(12.5, 4)

import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

jet = plt.cm.jet
fig = plt.figure()
x = y = np.linspace(0, 5, 100)
X, Y = np.meshgrid(x, y)

plt.subplot(121)
uni_x = stats.uniform.pdf(x, loc=0, scale=5)
uni_y = stats.uniform.pdf(y, loc=0, scale=5)
M = np.dot(uni_x[:, None], uni_y[None, :])
im = plt.imshow(M, interpolation='none', origin='lower',
                cmap=jet, vmax=1, vmin=-.15, extent=(0, 5, 0, 5))

plt.xlim(0, 5)
plt.ylim(0, 5)
plt.title("Landscape formed by Uniform priors.")

ax = fig.add_subplot(122, projection='3d')
ax.plot_surface(X, Y, M, cmap=plt.cm.jet, vmax=1, vmin=-.15)
ax.view_init(azim=390)
plt.title("Uniform prior landscape; alternate view");
```

Alternatively, if the two priors are $\text{Exp}(3)$ and $\text{Exp}(10)$, then the space is all positive numbers on the 2-D plane, and the surface induced by the priors looks like a water fall that starts at the point (0,0) and flows over the positive numbers.

The plots below visualize this. The more dark red the color, the more prior probability is assigned to that location. Conversely, areas with darker blue represent that our priors assign very low probability to that location.

```
figsize(12.5, 5)
fig = plt.figure()
plt.subplot(121)

exp_x = stats.expon.pdf(x, scale=3)
exp_y = stats.expon.pdf(x, scale=10)
M = np.dot(exp_x[:, None], exp_y[None, :])
CS = plt.contour(X, Y, M)
im = plt.imshow(M, interpolation='none', origin='lower',
                cmap=jet, extent=(0, 5, 0, 5))
#plt.xlabel("prior on $p_1$")
#plt.ylabel("prior on $p_2$")
plt.title("$Exp(3), Exp(10)$ prior landscape")

ax = fig.add_subplot(122, projection='3d')
ax.plot_surface(X, Y, M, cmap=jet)
ax.view_init(azim=390)
plt.title("$Exp(3), Exp(10)$ prior landscape; \nalternate view")

```

These are simple examples in 2D space, where our brains can understand surfaces well. In practice, spaces and surfaces generated by our priors can be much higher dimensional.

If these surfaces describe our *prior distributions* on the unknowns, what happens to our space after we incorporate our observed data $X$? The data $X$ does not change the space, but it changes the surface of the space by *pulling and stretching the fabric of the prior surface* to reflect where the true parameters likely live. More data means more pulling and stretching, and our original shape becomes mangled or insignificant compared to the newly formed shape. Less data, and our original shape is more present.  Regardless, the resulting surface describes the *posterior distribution*.

Again I must stress that it is, unfortunately, impossible to visualize this in large dimensions. For two dimensions, the data essentially *pushes up* the original surface to make *tall mountains*. The tendency of the observed data to *push up* the posterior probability in certain areas is checked by the prior probability distribution, so that less prior probability means more resistance. Thus in the double-exponential prior case above, a mountain (or multiple mountains) that might erupt near the (0,0) corner would be much higher than mountains that erupt closer to (5,5), since there is more resistance (low prior probability) near (5,5). The peak reflects the posterior probability of where the true parameters are likely to be found. Importantly, if the prior has assigned a probability of 0, then no posterior probability will be assigned there.

Suppose the priors mentioned above represent different parameters $\lambda$ of two Poisson distributions. We observe a few data points and visualize the new landscape:

```
# create the observed data

# sample size of data we observe, trying varying this (keep it less than 100 ;)
N = 1

# the true parameters, but of course we do not see these values...
lambda_1_true = 1
lambda_2_true = 3

#...we see the data generated, dependent on the above two values.
data = np.concatenate([
    stats.poisson.rvs(lambda_1_true, size=(N, 1)),
    stats.poisson.rvs(lambda_2_true, size=(N, 1))
], axis=1)
print "observed (2-dimensional,sample size = %d):" % N, data

# plotting details.
x = y = np.linspace(.01, 5, 100)
likelihood_x = np.array([stats.poisson.pmf(data[:, 0], _x)
                        for _x in x]).prod(axis=1)
likelihood_y = np.array([stats.poisson.pmf(data[:, 1], _y)
                        for _y in y]).prod(axis=1)
L = np.dot(likelihood_x[:, None], likelihood_y[None, :])

```

    observed (2-dimensional,sample size = 1): [[3 3]]

```
figsize(12.5, 12)
# matplotlib heavy lifting below, beware!
plt.subplot(221)
uni_x = stats.uniform.pdf(x, loc=0, scale=5)
uni_y = stats.uniform.pdf(x, loc=0, scale=5)
M = np.dot(uni_x[:, None], uni_y[None, :])
im = plt.imshow(M, interpolation='none', origin='lower',
                cmap=jet, vmax=1, vmin=-.15, extent=(0, 5, 0, 5))
plt.scatter(lambda_2_true, lambda_1_true, c="k", s=50, edgecolor="none")
plt.xlim(0, 5)
plt.ylim(0, 5)
plt.title("Landscape formed by Uniform priors on $p_1, p_2$.")

plt.subplot(223)
plt.contour(x, y, M * L)
im = plt.imshow(M * L, interpolation='none', origin='lower',
                cmap=jet, extent=(0, 5, 0, 5))
plt.title("Landscape warped by %d data observation;\n Uniform priors on $p_1, p_2$." % N)
plt.scatter(lambda_2_true, lambda_1_true, c="k", s=50, edgecolor="none")
plt.xlim(0, 5)
plt.ylim(0, 5)

plt.subplot(222)
exp_x = stats.expon.pdf(x, loc=0, scale=3)
exp_y = stats.expon.pdf(x, loc=0, scale=10)
M = np.dot(exp_x[:, None], exp_y[None, :])

plt.contour(x, y, M)
im = plt.imshow(M, interpolation='none', origin='lower',
                cmap=jet, extent=(0, 5, 0, 5))
plt.scatter(lambda_2_true, lambda_1_true, c="k", s=50, edgecolor="none")
plt.xlim(0, 5)
plt.ylim(0, 5)
plt.title("Landscape formed by Exponential priors on $p_1, p_2$.")

plt.subplot(224)
# This is the likelihood times prior, that results in the posterior.
plt.contour(x, y, M * L)
im = plt.imshow(M * L, interpolation='none', origin='lower',
                cmap=jet, extent=(0, 5, 0, 5))

plt.scatter(lambda_2_true, lambda_1_true, c="k", s=50, edgecolor="none")
plt.title("Landscape warped by %d data observation;\n Exponential priors on \
$p_1, p_2$." % N)
plt.xlim(0, 5)
plt.ylim(0, 5)

```

The plot on the left is the deformed landscape with the $\text{Uniform}(0,5)$ priors, and the plot on the right is the deformed landscape with the exponential priors. Notice that the posterior landscapes look different from one another, though the data observed is identical in both cases. The reason is as follows. Notice the exponential-prior landscape, bottom right figure, puts very little *posterior* weight on values in the upper right corner of the figure: this is because *the prior does not put much weight there*. On the other hand, the uniform-prior landscape is happy to put posterior weight in the upper-right corner, as the prior puts more weight there.

Notice also the highest-point, corresponding the the darkest red, is biased towards (0,0) in the exponential case, which is the result from the exponential prior putting more prior weight in the (0,0) corner.

The black dot represents the true parameters. Even with 1 sample point, the mountains attempts to contain the true parameter. Of course, inference with a sample size of 1 is incredibly naive, and choosing such a small sample size was only illustrative.

It's a great exercise to try changing the sample size to other values (try 2,5,10,100?...) and observing how our "mountain" posterior changes.

### Exploring the landscape using the MCMC

We should explore the deformed posterior space generated by our prior surface and observed data to find the posterior mountain. However, we cannot naively search the space: any computer scientist will tell you that traversing $N$-dimensional space is exponentially difficult in $N$: the size of the space quickly blows-up as we increase $N$ (see [the curse of dimensionality](http://en.wikipedia.org/wiki/Curse_of_dimensionality)). What hope do we have to find these hidden mountains? The idea behind MCMC is to perform an intelligent search of the space. To say "search" implies we are looking for a particular point, which is perhaps not an accurate as we are really looking for a broad mountain.

Recall that MCMC returns *samples* from the posterior distribution, not the distribution itself. Stretching our mountainous analogy to its limit, MCMC performs a task similar to repeatedly asking  "How likely is this pebble I found to be from the mountain I am searching for?", and completes its task by returning thousands of accepted pebbles in hopes of reconstructing the original mountain. In MCMC and PyMC lingo, the returned sequence of "pebbles" are the samples,  cumulatively called the *traces*.

When I say MCMC intelligently searches, I really am saying MCMC will *hopefully* converge towards the areas of high posterior probability. MCMC does this by exploring nearby positions and moving into areas with higher probability. Again, perhaps "converge" is not an accurate term to describe MCMC's progression. Converging usually implies moving towards a point in space, but MCMC moves towards a *broader area* in the space and randomly walks in that area, picking up samples from that area.

####Why Thousands of Samples?

At first, returning thousands of samples to the user might sound like being an inefficient way to describe the posterior distributions. I would argue that this is extremely efficient. Consider the alternative possibilities:

1. Returning a mathematical formula for the "mountain ranges" would involve describing a N-dimensional surface with arbitrary peaks and valleys.
2. Returning the "peak" of the landscape, while mathematically possible and a sensible thing to do as the highest point corresponds to most probable estimate of the unknowns, ignores the shape of the landscape, which we have previously argued is very important in determining posterior confidence in unknowns.

Besides computational reasons, likely the strongest reason for returning samples is that we can easily use *The Law of Large Numbers* to solve otherwise intractable problems. I postpone this discussion for the next chapter. With the thousands of samples, we can reconstruct the posterior surface by organizing them in a histogram.

### Algorithms to perform MCMC

There is a large family of algorithms that perform MCMC. Most of these algorithms can be expressed at a high level as follows: (Mathematical details can be found in the appendix.)

1. Start at current position.
2. Propose moving to a new position (investigate a pebble near you).
3. Accept/Reject the new position based on the position's adherence to the data and prior distributions (ask if the pebble likely came from the mountain).
4.  1.  If you accept: Move to the new position. Return to Step 1.
    2. Else: Do not move to new position. Return to Step 1.
5. After a large number of iterations, return all accepted positions.

This way we move in the general direction towards the regions where the posterior distributions exist, and collect samples sparingly on the journey. Once we reach the posterior distribution, we can easily collect samples as they likely all belong to the posterior distribution.

If the current position of the MCMC algorithm is in an area of extremely low probability, which is often the case when the algorithm begins (typically at a random location in the space), the algorithm will move in positions *that are likely not from the posterior* but better than everything else nearby. Thus the first moves of the algorithm are not reflective of the posterior.

In the above algorithm's pseudocode, notice that only the current position matters (new positions are investigated only near the current position). We can describe this property as *memorylessness*, i.e. the algorithm does not care *how* it arrived at its current position, only that it is there.

### Other approximation solutions to the posterior
Besides MCMC, there are other procedures available for determining the posterior distributions. A Laplace approximation is an approximation of the posterior using simple functions. A more advanced method is [Variational Bayes](http://en.wikipedia.org/wiki/Variational_Bayesian_methods). All three methods, Laplace Approximations, Variational Bayes, and classical MCMC have their pros and cons. We will only focus on MCMC in this book. That being said, my friend Imri Sofar likes to classify MCMC algorithms as either "they suck", or "they really suck". He classifies the particular flavour of MCMC used by PyMC as just *sucks* ;)

##### Example: Unsupervised Clustering using a Mixture Model

Suppose we are given the following dataset:

```
figsize(12.5, 4)
data = np.loadtxt("data/mixture_data.csv", delimiter=",")

plt.hist(data, bins=20, color="k", histtype="stepfilled", alpha=0.8)
plt.title("Histogram of the dataset")
plt.ylim([0, None])
print data[:10], "..."

```

What does the data suggest? It appears the data has a bimodal form, that is, it appears to have two peaks, one near 120 and the other near 200. Perhaps there are *two clusters* within this dataset.

This dataset is a good example of the data-generation modeling technique from last chapter. We can propose *how* the data might have been created. I suggest the following data generation algorithm:

1. For each data point, choose cluster 1 with probability $p$, else choose cluster 2.
2. Draw a random variate from a Normal distribution with parameters $\mu_i$ and $\sigma_i$ where $i$ was chosen in step 1.
3. Repeat.

This algorithm would create a similar effect as the observed dataset, so we choose this as our model. Of course, we do not know $p$ or the parameters of the Normal distributions. Hence we must infer, or *learn*, these unknowns.

Denote the Normal distributions $\text{Nor}_0$ and $\text{Nor}_1$ (having variables' index start at 0 is just Pythonic). Both currently have unknown mean and standard deviation, denoted $\mu_i$ and $\sigma_i, \; i =0,1$ respectively. A specific data point can be from either $\text{Nor}_0$ or $\text{Nor}_1$, and we assume that the data point is assigned to $\text{Nor}_0$ with probability $p$.

An appropriate way to assign data points to clusters is to use a PyMC `Categorical` stochastic variable. Its parameter is a $k$-length array of probabilities that must sum to one and its `value` attribute is a integer between 0 and $k-1$ randomly chosen according to the crafted array of probabilities. (In our case $k=2$) *A priori*, we do not know what the probability of assignment to cluster 1 is, so we create a uniform variable over 0,1 to model this. Call this `p`. Thus the probability array we enter into the `Categorical` variable is `[p, 1-p]`.

```
import pymc as pm

p = pm.Uniform("p", 0, 1)

assignment = pm.Categorical("assignment", [p, 1 - p], size=data.shape[0])
print "prior assignment, with p = %.2f:" % p.value
print assignment.value[:10], "..."

```

    prior assignment, with p = 0.73:
    [0 0 0 0 1 0 0 1 0 1] ...

Looking at the above dataset, I would guess that the standard deviations of the two Normals are different. To maintain ignorance of what the standard deviations might be, we will initially model them as uniform on 0 to 100. Really we are talking about $\tau$, the *precision* of the Normal distribution, but it is easier to think in terms of standard deviation. Our PyMC code will need to transform our standard deviation into precision by the relation:

$$ \tau = \frac{1}{\sigma^2} $$

In PyMC, we can do this in one step by writing:

    taus = 1.0/pm.Uniform( "stds", 0, 100, size= 2)**2

Notice that we specified `size=2`: we are modeling both $\tau$s as a single PyMC variable. Note that this does not induce a necessary relationship between the two $\tau$s, it is simply for succinctness.

We also need to specify priors on the centers of the clusters. The centers are really the $\mu$ parameters in this Normal distributions. Their priors can be modeled by a Normal distribution. Looking at the data, I have an idea where the two centers might be — I would guess somewhere around 120 and 190 respectively, though I am not very confident in these eyeballed estimates. Hence I will set $\mu_0 = 120, \mu_1 = 190$ and $\sigma_{0,1} = 10$ (recall we enter the $\tau$ parameter, so enter $1/\sigma^2 = 0.01$ in the PyMC variable.)

```
taus = 1.0 / pm.Uniform("stds", 0, 100, size=2) ** 2
centers = pm.Normal("centers", [120, 190], [0.01, 0.01], size=2)

"""
The below deterministic functions map an assignment, in this case 0 or 1,
to a set of parameters, located in the (1,2) arrays `taus` and `centers`.
"""

@pm.deterministic
def center_i(assignment=assignment, centers=centers):
    return centers[assignment]

@pm.deterministic
def tau_i(assignment=assignment, taus=taus):
    return taus[assignment]

print "Random assignments: ", assignment.value[:4], "..."
print "Assigned center: ", center_i.value[:4], "..."
print "Assigned precision: ", tau_i.value[:4], "..."

```

    Random assignments:  [0 0 0 0] ...
    Assigned center:  [ 120.6049531  120.6049531  120.6049531  120.6049531] ...
    Assigned precision:  [ 0.02278271  0.02278271  0.02278271  0.02278271] ...

```
# and to combine it with the observations:
observations = pm.Normal("obs", center_i, tau_i, value=data, observed=True)

# below we create a model class
model = pm.Model([p, assignment, taus, centers])

```

PyMC has an MCMC class, `MCMC` in the main namespace of PyMC, that implements the MCMC exploring algorithm. We initialize it by passing in a `Model` instance:

    mcmc = pm.MCMC( model )

The method for asking the `MCMC` to explore the space is `sample( iterations )`, where `iterations` is the number of steps you wish the algorithm to perform. We try 50000 steps below:

```
mcmc = pm.MCMC(model)
mcmc.sample(50000)

```

    [****************100%******************]  50000 of 50000 complete

Below I plot the paths, or "traces", the unknown parameters (centers, precisions, and $p$) have taken thus far. The traces can be retrieved using the `trace` method in the `MCMC` object created, which accepts the assigned PyMC variable `name`. For example, `mcmc.trace("centers")` will retrieve a `Trace` object that can be indexed (using `[:]` or `.gettrace()` to retrieve all traces, or fancy-indexing like `[1000:]`).

```
figsize(12.5, 9)
plt.subplot(311)
lw = 1
center_trace = mcmc.trace("centers")[:]

# for pretty colors later in the book.
#colors = ["#348ABD", "#A60628"]
#if center_trace[-1, 0] > center_trace[-1, 1] \
#    else ["#A60628", "#348ABD"]

if center_trace[-1, 0] > center_trace[-1, 1]:
    colors = ["#348ABD", "#A60628"]
else:
    colors = ["#A60628", "#348ABD"]

plt.plot(center_trace[:, 0], label="trace of center 0", c=colors[0], lw=lw)
plt.plot(center_trace[:, 1], label="trace of center 1", c=colors[1], lw=lw)
plt.title("Traces of unknown parameters")
leg = plt.legend(loc="upper right")
leg.get_frame().set_alpha(0.7)

plt.subplot(312)
std_trace = mcmc.trace("stds")[:]
plt.plot(std_trace[:, 0], label="trace of standard deviation of cluster 0",
     c=colors[0], lw=lw)
plt.plot(std_trace[:, 1], label="trace of standard deviation of cluster 1",
     c=colors[1], lw=lw)
plt.legend(loc="upper left")

plt.subplot(313)
p_trace = mcmc.trace("p")[:]
plt.plot(p_trace, label="$p$: frequency of assignment to cluster 0",
     color="#467821", lw=lw)
plt.xlabel("Steps")
plt.ylim(0, 1)
plt.legend()

```

Notice the following characteristics:

1. The traces converges, not to a single point, but to a *distribution* of possible points. This is *convergence* in an MCMC algorithm.
2. Inference using the first few thousand points is a bad idea, as they are unrelated to the final distribution we are interested in. Thus is it a good idea to discard those samples before using the samples for inference. We call this period before converge the *burn-in period*.
3. The traces appear as a random "walk" around the space, that is, the paths exhibit correlation with previous positions. This is both good and bad. We will always have correlation between current positions and the previous positions, but too much of it means we are not exploring the space well. This will be detailed in the Diagnostics section  later in this chapter.

To achieve further convergence, we will perform more MCMC steps. Starting the MCMC again after it has already been called does not mean starting the entire algorithm over. In the pseudo-code algorithm of MCMC above, the only position that matters is the current position (new positions are investigated near the current position), implicitly stored in PyMC variables' `value` attribute. Thus it is fine to halt an MCMC algorithm and inspect its progress, with the intention of starting it up again later. The `value` attributes are not overwritten.

We will sample the MCMC one hundred thousand more times and visualize the progress below:

```
mcmc.sample(100000)

```

    [****************100%******************]  100000 of 100000 complete

```
figsize(12.5, 4)
center_trace = mcmc.trace("centers", chain=1)[:]
prev_center_trace = mcmc.trace("centers", chain=0)[:]

x = np.arange(50000)
plt.plot(x, prev_center_trace[:, 0], label="previous trace of center 0",
     lw=lw, alpha=0.4, c=colors[1])
plt.plot(x, prev_center_trace[:, 1], label="previous trace of center 1",
     lw=lw, alpha=0.4, c=colors[0])

x = np.arange(50000, 150000)
plt.plot(x, center_trace[:, 0], label="new trace of center 0", lw=lw, c="#348ABD")
plt.plot(x, center_trace[:, 1], label="new trace of center 1", lw=lw, c="#A60628")

plt.title("Traces of unknown center parameters")
leg = plt.legend(loc="upper right")
leg.get_frame().set_alpha(0.8)
plt.xlabel("Steps")

```

The `trace` method in the `MCMC` instance has a keyword argument `chain`, that indexes which call to `sample` you would like to be returned. (Often we need to call `sample` multiple times, and the ability to retrieve past samples is a useful procedure). The default for `chain` is -1, which will return the samples from the lastest call to `sample`.

#### Cluster Investigation

We have not forgotten our main challenge: identify the clusters. We have determined posterior distributions for our unknowns. We plot the posterior distributions of the center and standard deviation variables below:

```
figsize(11.0, 4)
std_trace = mcmc.trace("stds")[:]

_i = [1, 2, 3, 0]
for i in range(2):
    plt.subplot(2, 2, _i[2 * i])
    plt.title("Posterior of center of cluster %d" % i)
    plt.hist(center_trace[:, i], color=colors[i], bins=30,
             histtype="stepfilled")

    plt.subplot(2, 2, _i[2 * i + 1])
    plt.title("Posterior of standard deviation of cluster %d" % i)
    plt.hist(std_trace[:, i], color=colors[i], bins=30,
             histtype="stepfilled")
    # plt.autoscale(tight=True)

plt.tight_layout()

```

The MCMC algorithm has proposed that the most likely centers of the two clusters are near 120 and 200 respectively. Similar inference can be applied to the standard deviation.

We are also given the posterior distributions for the labels of the data point, which is present in `mcmc.trace("assignment")`. Below is a visualization of this. The y-axis represents a subsample of the posterior labels for each data point. The x-axis are the sorted values of the data points. A red square is an assignment to cluster 1, and a blue square is an assignment to cluster 0.

```
import matplotlib as mpl
figsize(12.5, 4.5)
plt.cmap = mpl.colors.ListedColormap(colors)
plt.imshow(mcmc.trace("assignment")[::400, np.argsort(data)],
       cmap=plt.cmap, aspect=.4, alpha=.9)
plt.xticks(np.arange(0, data.shape[0], 40),
       ["%.2f" % s for s in np.sort(data)[::40]])
plt.ylabel("posterior sample")
plt.xlabel("value of $i$th data point")
plt.title("Posterior labels of data points")

```

Looking at the above plot, it appears that the most uncertainty is between 150 and 170. The above plot slightly misrepresents things, as the x-axis is not a true scale (it displays the value of the $i$th sorted data point.) A more clear diagram is below, where we have estimated the *frequency* of each data point belonging to the labels 0 and 1.

```
cmap = mpl.colors.LinearSegmentedColormap.from_list("BMH", colors)
assign_trace = mcmc.trace("assignment")[:]
plt.scatter(data, 1 - assign_trace.mean(axis=0), cmap=cmap,
        c=assign_trace.mean(axis=0), s=50)
plt.ylim(-0.05, 1.05)
plt.xlim(35, 300)
plt.title("Probability of data point belonging to cluster 0")
plt.ylabel("probability")
plt.xlabel("value of data point")

```

Even though we modeled the clusters using Normal distributions, we didn't get just a single Normal distribution that *best* fits the data (whatever our definition of best is), but a distribution of values for the Normal's parameters. How can we choose just a single pair of values for the mean and variance and determine a *sorta-best-fit* gaussian?

One quick and dirty way (which has nice theoretical properties we will see in Chapter 5), is to use the *mean* of the posterior distributions. Below we overlay the Normal density functions, using the mean of the posterior distributions as the chosen parameters, with our observed data:

```
norm = stats.norm
x = np.linspace(20, 300, 500)
posterior_center_means = center_trace.mean(axis=0)
posterior_std_means = std_trace.mean(axis=0)
posterior_p_mean = mcmc.trace("p")[:].mean()

plt.hist(data, bins=20, histtype="step", normed=True, color="k",
     lw=2, label="histogram of data")
y = posterior_p_mean * norm.pdf(x, loc=posterior_center_means[0],
                                scale=posterior_std_means[0])
plt.plot(x, y, label="Cluster 0 (using posterior-mean parameters)", lw=3)
plt.fill_between(x, y, color=colors[1], alpha=0.3)

y = (1 - posterior_p_mean) * norm.pdf(x, loc=posterior_center_means[1],
                                      scale=posterior_std_means[1])
plt.plot(x, y, label="Cluster 1 (using posterior-mean parameters)", lw=3)
plt.fill_between(x, y, color=colors[0], alpha=0.3)

plt.legend(loc="upper left")
plt.title("Visualizing Clusters using posterior-mean parameters")

```

### Important: Don't mix posterior samples

In the above example, a possible (though less likely) scenario is that cluster 0 has a very large standard deviation, and cluster 1 has a small standard deviation. This would still satisfy the evidence, albeit less so than our original inference. Alternatively, it would be incredibly unlikely for *both* distributions to have a small standard deviation, as the data does not support this hypothesis at all. Thus the two standard deviations are *dependent* on each other: if one is small, the other must be large. In fact, *all* the unknowns are related in a similar manner. For example, if a standard deviation is large, the mean has a wider possible space of realizations. Conversely, a small standard deviation restricts the mean to a small area.

During MCMC, we are returned vectors representing samples from the unknown posteriors. Elements of different vectors cannot be used together, as this would break the above logic: perhaps a sample has returned that cluster 1 has a small standard deviation, hence all the other variables in that sample would incorporate that and be adjusted accordingly. It is easy to avoid this problem though, just make sure you are indexing traces correctly.

Another small example to illustrate the point. Suppose two variables, $x$ and $y$, are related by $x+y=10$. We model $x$ as a Normal random variable with mean 4 and explore 500 samples.

```
import pymc as pm

x = pm.Normal("x", 4, 10)
y = pm.Lambda("y", lambda x=x: 10 - x, trace=True)

ex_mcmc = pm.MCMC(pm.Model([x, y]))
ex_mcmc.sample(500)

plt.plot(ex_mcmc.trace("x")[:])
plt.plot(ex_mcmc.trace("y")[:])
plt.title("Displaying (extreme) case of dependence between unknowns")

```

As you can see, the two variables are not unrelated, and it would be wrong to add the $i$th sample of $x$ to the $j$th sample of $y$, unless $i = j$.

#### Returning to Clustering: Prediction
The above clustering can be generalized to $k$ clusters. Choosing $k=2$ allowed us to visualize the MCMC better, and examine some very interesting plots.

What about prediction? Suppose we observe a new data point, say $x = 175$, and we wish to label it to a cluster. It is foolish to simply assign it to the *closer* cluster center, as this ignores the standard deviation of the clusters, and we have seen from the plots above that this consideration is very important. More formally: we are interested in the *probability* (as we cannot be certain about labels) of assigning $x=175$ to cluster 1. Denote the assignment of $x$ as $L_x$, which is equal to 0 or 1, and we are interested in $P(L_x = 1 \;|\; x = 175 )$.

A naive method to compute this is to re-run the above MCMC with the additional data point appended. The disadvantage with this method is that it will be slow to infer for each novel data point. Alternatively, we can try a *less precise*, but much quicker method.

We will use Bayes Theorem for this. If you recall, Bayes Theorem looks like:

$$ P( A | X ) = \frac{ P( X  | A )P(A) }{P(X) }$$

In our case, $A$ represents $L_x = 1$ and $X$ is the evidence we have: we observe that $x = 175$. For a particular sample set of parameters for our posterior distribution, $( \mu_0, \sigma_0, \mu_1, \sigma_1, p)$, we are interested in asking "Is the probability that $x$ is in cluster 1 *greater* than the probability it is in cluster 0?", where the probability is dependent on the chosen parameters.

\begin{align}
& P(L_x = 1| x = 175 ) \gt P(L_x = 0| x = 175 ) \\\\[5pt]
& \frac{ P( x=175  | L_x = 1  )P( L_x = 1 ) }{P(x = 175) } \gt \frac{ P( x=175  | L_x = 0  )P( L_x = 0 )}{P(x = 175) }
\end{align}

As the denominators are equal, they can be ignored (and good riddance, because computing the quantity $P(x = 175)$ can be difficult).

$$  P( x=175  | L_x = 1  )P( L_x = 1 ) \gt  P( x=175  | L_x = 0  )P( L_x = 0 ) $$

```
norm_pdf = stats.norm.pdf
p_trace = mcmc.trace("p")[:]
x = 175

v = p_trace * norm_pdf(x, loc=center_trace[:, 0], scale=std_trace[:, 0]) > \
    (1 - p_trace) * norm_pdf(x, loc=center_trace[:, 1], scale=std_trace[:, 1])

print "Probability of belonging to cluster 1:", v.mean()

```

    Probability of belonging to cluster 1: 0.00108

Giving us a probability instead of a label is a very useful thing. Instead of the naive

    L = 1 if prob > 0.5 else 0

we can optimize our guesses using a *loss function*, which the entire fifth chapter is devoted to.

### Using `MAP` to improve convergence

If you ran the above example yourself, you may have noticed that our results were not consistent: perhaps your cluster division was more scattered, or perhaps less scattered. The problem is that our traces are a function of the *starting values* of the MCMC algorithm.

It can be mathematically shown that letting the MCMC run long enough, by performing many steps, the algorithm *should forget its initial position*. In fact, this is what it means to say the MCMC converged (in practice though we can never achieve total convergence). Hence if we observe different posterior analysis, it is likely because our MCMC has not fully converged yet, and we should not use samples from it yet (we should use a larger burn-in period ).

In fact, poor starting values can prevent any convergence, or significantly slow it down. Ideally, we would like to have the chain start at the *peak* of our landscape, as this is exactly where the posterior distributions exist. Hence, if we started at the "peak", we could avoid a lengthy burn-in period and incorrect inference. Generally, we call this "peak" the *maximum a posterior* or, more simply, the *MAP*.

Of course, we do not know where the MAP is. PyMC provides an object that will approximate, if not find, the MAP location. In the PyMC main namespace is the `MAP` object that accepts a PyMC `Model` instance. Calling `.fit()` from the `MAP` instance sets the variables in the model to their MAP values.

    map_ = pm.MAP( model )
    map_.fit()

The `MAP.fit()` methods has the flexibility of allowing the user to choose which optimization algorithm to use (after all, this is a optimization problem: we are looking for the values that maximize our landscape), as not all optimization algorithms are created equal. The default optimization algorithm in the call to `fit` is scipy's `fmin` algorithm (which attempts to minimize the *negative of the landscape*). An alternative algorithm that is available is Powell's Method, a favourite of PyMC blogger [Abraham Flaxman](http://healthyalgorithms.com/) [1], by calling `fit(method='fmin_powell')`. From my experience, I use the default, but if my convergence is slow or not guaranteed, I experiment with Powell's method.

The MAP can also be used as a solution to the inference problem, as mathematically it  is the *most likely* value for the unknowns. But as mentioned earlier in this chapter,  this location ignores the uncertainty and doesn't return a distribution.

Typically, it is always a good idea, and rarely a bad idea, to prepend your call to `mcmc` with a call to `MAP(model).fit()`. The intermediate call to `fit` is hardly computationally intensive, and will save you time later due to a shorter burn-in period.

#### Speaking of the burn-in period

It is still a good idea to provide a burn-in period, even if we are using `MAP` prior to calling `MCMC.sample`, just to be safe. We can have PyMC automatically discard the first $n$ samples by specifying the `burn` parameter in the call to `sample`. As one does not know when the chain has fully converged, I like to assign the first *half* of my samples to be discarded, sometimes up to 90% of my samples for longer runs. To continue the clustering example from above, my new code would look something like:

    model = pm.Model( [p, assignment, taus, centers ] )

    map_ = pm.MAP( model )
    map_.fit() #stores the fitted variables' values in foo.value

    mcmc = pm.MCMC( model )
    mcmc.sample( 100000, 50000 )

## Diagnosing Convergence

### Autocorrelation

Autocorrelation is a measure of how related a series of numbers is with itself. A measurement of 1.0 is perfect positive autocorrelation, 0 no autocorrelation, and -1 is perfect negative correlation.  If you are familiar with standard *correlation*, then autocorrelation is just how correlated a series, $x_\tau$, at time $t$ is with the series at time $t-k$:

$$R(k) = Corr( x_t, x_{t-k} ) $$

For example, consider the two series:

$$x_t \sim \text{Normal}(0,1), \;\; x_0 = 0$$
$$y_t \sim \text{Normal}(y_{t-1}, 1 ), \;\; y_0 = 0$$

which have example paths like:

```
figsize(12.5, 4)

import pymc as pm
x_t = pm.rnormal(0, 1, 200)
x_t[0] = 0
y_t = np.zeros(200)
for i in range(1, 200):
    y_t[i] = pm.rnormal(y_t[i - 1], 1)

plt.plot(y_t, label="$y_t$", lw=3)
plt.plot(x_t, label="$x_t$", lw=3)
plt.xlabel("time, $t$")
plt.legend()

```

One way to think of autocorrelation is "If I know the position of the series at time $s$, can it help me know where I am at time $t$?" In the series $x_t$, the answer is No. By construction, $x_t$ are random variables. If I told you that $x_2 = 0.5$, could you give me a better guess about $x_3$? No.

On the other hand, $y_t$ is autocorrelated. By construction, if I knew that $y_2 = 10$, I can be very confident that $y_3$ will not be very far from 10. Similarly, I can even make a (less confident guess) about $y_4$: it will probably not be near 0 or 20, but a value of 5 is not too unlikely. I can make a similar argument about $y_5$, but again, I am less confident. Taking this to it's logical conclusion, we must concede that as $k$, the lag between time points, increases the autocorrelation decreases. We can visualize this:

```
def autocorr(x):
    # from http://tinyurl.com/afz57c4
    result = np.correlate(x, x, mode='full')
    result = result / np.max(result)
    return result[result.size / 2:]

colors = ["#348ABD", "#A60628", "#7A68A6"]

x = np.arange(1, 200)
plt.bar(x, autocorr(y_t)[1:], width=1, label="$y_t$",
        edgecolor=colors[0], color=colors[0])
plt.bar(x, autocorr(x_t)[1:], width=1, label="$x_t$",
        color=colors[1], edgecolor=colors[1])

plt.legend(title="Autocorrelation")
plt.ylabel("measured correlation \nbetween $y_t$ and $y_{t-k}$.")
plt.xlabel("k (lag)")
plt.title("Autocorrelation plot of $y_t$ and $x_t$ for differing $k$ lags.")

```

Notice that as $k$ increases, the autocorrelation of $y_t$ decreases from a very high point. Compare with the autocorrelation of $x_t$ which looks like noise (which it really is), hence we can conclude no autocorrelation exists in this series.

####How does this relate to MCMC convergence?

By the nature of the MCMC algorithm, we will always be returned samples that exhibit autocorrelation (this is because of the step `from your current position, move to a position near you`).

A chain that is [Isn't meandering exploring?] exploring the space well will exhibit very high autocorrelation. Visually, if the trace seems to meander like a river, and not settle down, the chain will have high autocorrelation.

This does not imply that a converged MCMC has low autocorrelation. Hence low autocorrelation is not necessary for convergence, but it is sufficient. PyMC has a built-in autocorrelation plotting function in the `Matplot` module.

###Thinning

Another issue can arise if there is high-autocorrelation between posterior samples. Many post-processing algorithms require samples to be *independent* of each other. This can be solved, or at least reduced, by only returning to the user every $n$th sample, thus removing some autocorrelation. Below we perform an autocorrelation plot for $y_t$ with differing levels of thinning:

```
max_x = 200 / 3 + 1
x = np.arange(1, max_x)

plt.bar(x, autocorr(y_t)[1:max_x], edgecolor=colors[0],
        label="no thinning", color=colors[0], width=1)
plt.bar(x, autocorr(y_t[::2])[1:max_x], edgecolor=colors[1],
        label="keeping every 2nd sample", color=colors[1], width=1)
plt.bar(x, autocorr(y_t[::3])[1:max_x], width=1, edgecolor=colors[2],
        label="keeping every 3rd sample", color=colors[2])

plt.autoscale(tight=True)
plt.legend(title="Autocorrelation plot for $y_t$", loc="lower left")
plt.ylabel("measured correlation \nbetween $y_t$ and $y_{t-k}$.")
plt.xlabel("k (lag)")
plt.title("Autocorrelation of $y_t$ (no thinning vs. thinning) \
at differing $k$ lags.")

```

With more thinning, the autocorrelation drops quicker. There is a tradeoff though: higher thinning requires more MCMC iterations to achieve the same number of returned samples. For example, 10 000 samples unthinned is 100 000 with a thinning of 10 (though the latter has less autocorrelation).

What is a good amount of thinning? The returned samples will always exhibit some autocorrelation, regardless of how much thinning is done. So long as the autocorrelation tends to zero, you are probably ok. Typically thinning of more than 10 is not necessary.

PyMC exposes a `thinning` parameter in the call to `sample`, for example: `sample( 10000, burn = 5000, thinning = 5)`.

### `pymc.Matplot.plot()`

It seems silly to have to manually create histograms, autocorrelation plots and trace plots each time we perform MCMC. The authors of PyMC have included a visualization tool for just this purpose.

As the title suggests, the `pymc.Matplot` module contains a poorly named function `plot`, which I prefer to import as `mcplot` so there is no conflict with other namespaces. `plot`, or `mcplot` as I suggest, accepts an `MCMC` object and will return posterior distributions, traces and auto-correlations for each variable (up to 10 variables).

Below we use the tool to plot the centers of the clusters, after sampling 25 000 more times and `thinning = 10`.

```
from pymc.Matplot import plot as mcplot

mcmc.sample(25000, 0, 10)
mcplot(mcmc.trace("centers", 2), common_scale=False)

```

There are really two figures here, one for each unknown in the `centers` variable. In each figure, the subfigure in the top left corner is the trace of the variable. This is useful for inspecting that possible "meandering" property that is a result of non-convergence.

The largest plot on the right-hand side is the histograms of the samples, plus a few extra features. The thickest vertical line represents the posterior mean, which is a good summary of posterior distribution. The interval between the two  dashed vertical lines in each the posterior distributions represent the *95% credible interval*, not to be confused with a *95% confidence interval*. I won't get into the latter, but the former can be interpreted as "there is a 95% chance the parameter of interest lies in this interval". (Changing default parameters in the call to `mcplot` provides alternatives to 95%.) When communicating your results to others, it is incredibly important to state this interval. One of our purposes for studying Bayesian methods is to have a clear understanding of our uncertainty in unknowns. Combined with the posterior mean, the 95% credible interval provides a reliable interval to communicate the likely location of the unknown (provided by the mean) *and* the uncertainty (represented by the width of the interval).

The plots titled `center_0_acorr` and `center_1_acorr` are the generated autocorrelation plots. They look different than the ones I have displayed above, but the only difference is that 0-lag is centered in the middle of the figure, whereas I have 0 centered to the left.

## Useful tips for MCMC

Bayesian inference would be the *de facto* method if it weren't for MCMC's computational difficulties. In fact, MCMC is what turns most users off practical Bayesian inference. Below I present some good heuristics to help convergence and speed up the MCMC engine:

### Intelligent starting values

It would be great to start the MCMC algorithm off near the posterior distribution, so that it will take little time to start sampling correctly. We can aid the algorithm by telling where we *think* the posterior distribution will be by specifying the `value` parameter in the `Stochastic` variable creation. In many cases we can produce a reasonable guess for the parameter. For example, if we have data from a Normal distribution, and we wish to estimate the $\mu$ parameter, then a good starting value would be the *mean* of the data.

     mu = pm.Uniform( "mu", 0, 100, value = data.mean() )

For most parameters in models, there is a frequentist estimate of it. These estimates are a good starting value for our MCMC algorithms. Of course, this is not always possible for some variables, but including as many appropriate initial values is always a good idea. Even if your guesses are wrong, the MCMC will still converge to the proper distribution, so there is little to lose.

This is what using `MAP` tries to do, by giving good initial values to the MCMC. So why bother specifying user-defined values? Well, even giving `MAP` good values will help it find the maximum a-posterior.

Also important, *bad initial values* are a source of major bugs in PyMC and can hurt convergence.

#### Priors

If the priors are poorly chosen, the MCMC algorithm may not converge, or atleast have difficulty converging. Consider what may happen if the prior chosen does not even contain the true parameter: the prior assigns 0 probability to the unknown, hence the posterior will assign 0 probability as well. This can cause pathological results.

For this reason, it is best to carefully choose the priors. Often, lack of covergence or evidence of samples crowding to boundaries implies something is wrong with the chosen priors (see *Folk Theorem of Statistical Computing* below).

#### Covariance matrices and eliminating parameters

### The Folk Theorem of Statistical Computing

>   *If you are having computational problems, probably your model is wrong.*

## Conclusion

PyMC provides a very strong backend to performing Bayesian inference, mostly because it has abstracted the inner mechanics of MCMC from the user. Despite this, some care must be applied to ensure your inference is not being biased by the iterative nature of MCMC.

### References

1. Flaxman, Abraham. "Powell's Methods for Maximization in PyMC." Healthy Algorithms. N.p., 9 02 2012. Web. 28 Feb 2013. <http://healthyalgorithms.com/2012/02/09/powells-method-for-maximization-in-pymc/>.

```
from IPython.core.display import HTML

def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()

```

<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunss.otf');
    }
    @font-face {
        font-family: "Computer Modern";
        font-weight: bold;
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunsx.otf');
    }
    @font-face {
        font-family: "Computer Modern";
        font-style: oblique;
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunsi.otf');
    }
    @font-face {
        font-family: "Computer Modern";
        font-weight: bold;
        font-style: oblique;
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunso.otf');
    }
    div.cell{
        width:800px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: Helvetica, serif;
    }
    h4{
        margin-top:12px;
        margin-bottom: 3px;
       }
    div.text_cell_render{
        font-family: Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 145%;
        font-size: 130%;
        width:800px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro", source-code-pro,Consolas, monospace;
    }
    .prompt{
        display: None;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 22pt;
        color: #4057A1;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>

```

```


## 1. Maximizando el radio de Sharpe

### ¿Qué pasa si tenemos dos activos riesgosos?

Cuando tenemos dos o más activos riesgosos, tenemos disponibles diferentes LAC. ¿Qué significan sus pendientes?

 Ver en el tablero.

Pregunta:
- ¿Qué es lo que se quiere?

**Conclusión:**
- El mejor portafolio de activos no depende de las preferencias individuales, y por tanto va a ser igual para todos.
- Dicho mejor portafolio maximiza el radio de Sharpe.
- A este portafolio lo llamaremos el portafolio eficiente en media-varianza (EMV)

**Idea principal: el portafolio óptimo de activos riesgosos es independiente de las preferencias del inversionista.**
- El portafolio EMV determina el portafolio óptimo de activos riesgosos.
- Todos tendremos el mismo portafolio de activos riesgosos (EMV), y lo combinaremos con el activo libre de reisgo de acuerdo con las preferencias de cada uno de nosotros (aversión al riesgo).
- La LAC combinando el activo libre de riesgo y el portafolio EMV, se vuelve el conjunto de portafolios eficientes.

Entonces, se deben seguir los siguientes pasos:
1. Crear la frontera media-varianza.
2. Encontrar el portafolio que maximize el radio de Sharpe (portafolio EMV).
3. Construir la frontera eficiente (LAC) del punto $(0,r_f)$ al punto $(\sigma_s,E[r_s])$ del portafolio EMV.
4. Combinar de acuerdo a sus preferencias.
___

## 2. Solución analítica del portafolio EMV: caso con dos activos.

Queremos solucionar el siguiente problema:

\begin{align}
\max_{w_1,w_2} &\quad \frac{E[r_p]-r_f}{\sigma_p}\\
\text{s.a.}    &\quad E[r_p]=w_1E[r_1]+w_2E[r_2]\\
               &\quad \sigma_p=\sqrt{w_1^2\sigma_1^2+w_2^2\sigma_2^2+2w_1w_2\rho_{12}\sigma_1\sigma_2}\\
               &\quad w_1+w_2=1, \quad w_1,w_2\geq0
\end{align}

el cual es equivalente a

\begin{align}
\max_{w_1} &\quad \frac{w_1E[r_1]+(1-w_1)E[r_2]-r_f}{\sqrt{w_1^2\sigma_1^2+(1-w_1)^2\sigma_2^2+2w_1(1-w_1)\rho_{12}\sigma_1\sigma_2}}\\
\text{s.a.}    &\quad 0\leq w_1\leq1
\end{align}

**Actividad.**
El anterior es un problema de maximizar una función de una variable en un dominio cerrado. No debaría representar dificultad.

Encontrar la solución analítica a este problema.

Quien primero lo haga, y salga a explicarlo al tablero, le subo alguna tarea o quiz a 100.

Deben llegar a:

$$w_{1,EMV}=\frac{(E[r_1]-r_f)\sigma_2^2-(E[r_2]-r_f)\sigma_{12}}{(E[r_2]-r_f)\sigma_1^2+(E[r_1]-r_f)\sigma_2^2-((E[r_1]-r_f)+(E[r_2]-r_f))\sigma_{12}}.$$

Si nadie lo ha hecho en 30 min., procederé a hacerlo yo.

**Nota:**
- así como obtuvimos una expresión para el peso del portafolio de mínima varianza con dos activos, obtenemos una expresión para el peso del portafolio Eficiente en Media-Varianza.
- Estas actividades son sin duda un buen ejercicio, y se pueden replicar usando técnicas de varias variables (multiplicadores de Lagrange) cuando se tengan más de dos activos.
- Sin embargo, la complejidad del problema crece considerablemente con el número de variables, y la solución analítica deja de ser viable cuando mencionamos que un portafolio bien diversificado consta aproximadamente de 50-60 activos.
- En esos casos, este problema se soluciona con rutinas numéricas que hagan la optimización por nosotros.
- Por eso, les enseño cómo resolver este problema con optimizadores numéricos, porque son una solución viable y escalable a más variables.

## 3. Ejemplo ilustrativo.

Retomamos el ejemplo de mercados de acciones en los países integrantes del $G5$: EU, RU, Francia, Alemania y Japón.

```python
# Importamos pandas y numpy
import pandas as pd
import numpy as np
```

```python
# Resumen en base anual de rendimientos esperados y volatilidades
annual_ret_summ = pd.DataFrame(columns=['EU', 'RU', 'Francia', 'Alemania', 'Japon'], index=['Media', 'Volatilidad'])
annual_ret_summ.loc['Media'] = np.array([0.1355, 0.1589, 0.1519, 0.1435, 0.1497])
annual_ret_summ.loc['Volatilidad'] = np.array([0.1535, 0.2430, 0.2324, 0.2038, 0.2298])

annual_ret_summ.round(4)
```

      EU
      RU
      Francia
      Alemania
      Japon

      Media
      0.1355
      0.1589
      0.1519
      0.1435
      0.1497

      Volatilidad
      0.1535
      0.243
      0.2324
      0.2038
      0.2298

```python
# Matriz de correlación
corr = pd.DataFrame(data= np.array([[1.0000, 0.5003, 0.4398, 0.3681, 0.2663],
                                    [0.5003, 1.0000, 0.5420, 0.4265, 0.3581],
                                    [0.4398, 0.5420, 1.0000, 0.6032, 0.3923],
                                    [0.3681, 0.4265, 0.6032, 1.0000, 0.3663],
                                    [0.2663, 0.3581, 0.3923, 0.3663, 1.0000]]),
                    columns=annual_ret_summ.columns, index=annual_ret_summ.columns)
corr.round(4)
```

      EU
      RU
      Francia
      Alemania
      Japon

      EU
      1.0000
      0.5003
      0.4398
      0.3681
      0.2663

      RU
      0.5003
      1.0000
      0.5420
      0.4265
      0.3581

      Francia
      0.4398
      0.5420
      1.0000
      0.6032
      0.3923

      Alemania
      0.3681
      0.4265
      0.6032
      1.0000
      0.3663

      Japon
      0.2663
      0.3581
      0.3923
      0.3663
      1.0000

Supondremos, además, que la tasa libre de riesgo es $r_f=5\%$.

```python
# Tasa libre de riesgo
rf = 0.05
```

Entonces, supondremos que tenemos disponibles los activos correspondientes a los mercados de acciones de EU y Japón, y en adición el activo libre de riesgo.

#### 1. Construir la frontera de mínima varianza

```python
# Vector de w variando entre 0 y 1 con n pasos
n = 101
w = np.linspace(0, 1, n)
# Rendimientos esperados individuales
# Activo1: EU, Activo2:Japon
E1 = annual_ret_summ.loc['Media', 'EU']
E2 = annual_ret_summ.loc['Media', 'Japon']
# Volatilidades individuales
s1 = annual_ret_summ.loc['Volatilidad', 'EU']
s2 = annual_ret_summ.loc['Volatilidad', 'Japon']
# Correlacion
r12 = corr.loc['EU', 'Japon']
# Covarianza
s12 = r12 * s1 * s2
```

```python
# DataFrame de portafolios:
# 1. Índice: i
# 2. Columnas 1-2: w, 1-w
# 3. Columnas 3-4: E[r], sigma
# 4. Columna 5: Sharpe ratio
portafolios = pd.DataFrame(data={'w_1': w,
                                 'w_2': 1-w,
                                 'Media': w * E1 + (1 - w) * E2,
                                 'Vol':((w * s1)**2 + ((1 - w)*s2)**2 + 2 * w * (1 - w) * s12)**0.5
                                })
portafolios['RS'] = (portafolios['Media'] - rf)/ portafolios['Vol']
portafolios.head()
#((w * s1)**2 + ((1-w)*s2)**2 + 2 * w * (1 - w) * s12)**0.5
```

      w_1
      w_2
      Media
      Vol
      RS

      0
      0.00
      1.00
      0.149700
      0.229800
      0.433856

      1
      0.01
      0.99
      0.149558
      0.227916
      0.436820

      2
      0.02
      0.98
      0.149416
      0.226041
      0.439814

      3
      0.03
      0.97
      0.149274
      0.224176
      0.442839

      4
      0.04
      0.96
      0.149132
      0.222322
      0.445894

```python
# Importar librerías de gráficos
from matplotlib import pyplot as plt
%matplotlib inline
```

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a RS
plt.figure(figsize=(6, 4))
plt.scatter(portafolios['Vol'], portafolios['Media'], c = portafolios['RS'], cmap='RdYlBu')
plt.grid()
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado $E[r]$')
plt.colorbar()
plt.axis([0.14, 0.20, 0.135, 0.15])
```

#### 2. Encontrar el portafolio que maximiza el radio de Sharpe (EMV)

Primero, encontramos este portafolio con la fórmula que obtuvimos:

$$w_{1,EMV}=\frac{(E[r_1]-r_f)\sigma_2^2-(E[r_2]-r_f)\sigma_{12}}{(E[r_2]-r_f)\sigma_1^2+(E[r_1]-r_f)\sigma_2^2-((E[r_1]-r_f)+(E[r_2]-r_f))\sigma_{12}}.$$

```python
# Fórmula que obtuvimos
w_EMV = ((E1 - rf) * s2**2 - (E2 - rf) * s12) / ((E2 -rf) * s1**2 + (E1 - rf) * s2**2 - (E1 - rf + E2 - rf) * s12)
w_EMV
```

    0.6983139170512034

Ahora sí, con la función scipy.optimize.minimize

```python
# Importar la función minimize del módulo optimize
from scipy.optimize import minimize
```

```python
# Función objetivo (-RS)
def menos_RS(w, E1, E2, s1, s2, s12, rf):
    Erp = w * E1 + (1 - w) * E2
    sp = ((w * s1)**2 + ((1 - w) * s2)**2 + 2 * w * (1 - w) * s12)**0.5
    RS = (Erp - rf) / sp
    return -RS
```

```python
# Dato inicial
w0 = 0.5
# Cotas de las variables
bnd = (0, 1)
```

```python
# Optimización numérica
EMV = minimize(fun=menos_RS,
               x0 = w0,
               args = (E1, E2, s1, s2, s12, rf),
               bounds=(bnd, ))
# Resultado
EMV
```

          fun: array([-0.63087253])
     hess_inv: <1x1 LbfgsInvHessProduct with dtype=float64>
          jac: array([3.39728246e-06])
      message: b'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'
         nfev: 10
          nit: 4
       status: 0
      success: True
            x: array([0.69831582])

```python
w_EMV2 = EMV.x
w_EMV2, w_EMV
```

    (array([0.69831582]), 0.6983139170512034)

Con lo anterior, podemos obtener datos de rendimiento esperado y volatilidad del portafolio EMV

```python
# Rendimiento esperado y volatilidad del portafolio EMV
E_EMV = w_EMV *E1 + (1 - w_EMV) * E2
s_EMV = ((w_EMV * s1)**2 + ((1 - w_EMV) * s2)**2 + 2 * w_EMV * (1 - w_EMV) * s12)**0.5
```

```python
E_EMV, s_EMV
```

    (0.13978394237787292, 0.14231708951606933)

```python
RS_EMV = (E_EMV - rf) / s_EMV
```

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a SR, y portafolio EMV
plt.figure(figsize=(6, 4))
plt.scatter(portafolios['Vol'], portafolios['Media'], c = portafolios['RS'], cmap='RdYlBu', label='Portafolios')
plt.plot(s_EMV, E_EMV, '*y', ms=10, label='Portafolio EMV')
plt.grid()
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado $E[r]$')
plt.legend(loc='best')
plt.colorbar()
plt.axis([0.14, 0.20, 0.135, 0.15])
```

#### 3. Construir LAC

Ahora, dibujamos la LAC, combinando el portafolio EMV con el activo libre de riesgo:

```python
# Vector de wp variando entre 0 y 1.5 con n pasos
sp = np.linspace(0, 0.2)

```

```python
# DataFrame de CAL:
# 1. Índice: i
# 2. Columnas 1-2: wp, wrf
# 3. Columnas 3-4: E[r], sigma
# 4. Columna 5: Sharpe ratio
LAC = pd.DataFrame(data={'Vol': sp,
                         'Media': RS_EMV * sp + rf})
LAC.head()
```

      Vol
      Media

      0
      0.000000
      0.050000

      1
      0.004082
      0.052575

      2
      0.008163
      0.055150

      3
      0.012245
      0.057725

      4
      0.016327
      0.060300

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a SR, portafolio EMV y LAC
plt.figure(figsize=(6, 4))
plt.scatter(portafolios['Vol'], portafolios['Media'], c = portafolios['RS'], cmap='RdYlBu', label='Portafolios')
plt.plot(s_EMV, E_EMV, '*y', ms=10, label='Portafolio EMV')
plt.plot(LAC['Vol'], LAC['Media'], label = 'LAC')
plt.grid()
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado $E[r]$')
plt.legend(loc='best')
plt.colorbar()
#plt.axis([0.14, 0.20, 0.135, 0.15])
```

#### 4. Combinación óptima de acuerdo a preferencias

Con los datos anteriores, y la caracterización de aversión al riesgo, se escoge la combinación óptima entre el portafolio EMV y el activo libre de riesgo de acuerdo a:

$$w^\ast=\frac{E[r_s-r_f]}{\gamma\sigma_s^2}.$$

```python
# Para gamma=7
g = 7
w_opt = (E_EMV - rf) / (g * s_EMV**2)
w_opt
```

    0.6332665142475

```python
w_opt * w_EMV, w_opt * (1 - w_EMV), 1 - w_opt
```

    (0.44221882010153346, 0.1910476941459666, 0.3667334857525)

Created with Jupyter by Esteban Jiménez Rodríguez.


# Replication of Carneiro, Heckman, & Vytlacil's (2011) *Local Instrumental Variables* approach

In this notebook, I reproduce the semiparametric results from

> Carneiro, P., Heckman, J. J., & Vytlacil, E. J. (2011). [Estimating marginal returns to education.](https://pubs.aeaweb.org/doi/pdfplus/10.1257/aer.101.6.2754) *American Economic Review, 101*(6), 2754-81.

The authors analyze the returns to college for white males born between 1957 and 1963 using data from the National Longitudinal Survey of Youth 1979. The authors provide some [replication material]((https://pubs.aeaweb.org/doi/pdfplus/10.1257/aer.101.6.2754)) on their website but do not include geographic identifiers. Therefore, we make use of a mock data merging background characteristics and local data randomly.

In a future update, the semiparametric estimation method will be included in the open-source package *grmpy* for the simulation and estimation of the generalized Roy model in Python. Currently, *grmpy* is limited to the estimation of a parametric normal version of the generalized Roy model.
 For more, see the [online documentation](https://grmpy.readthedocs.io/en/develop/).

## 0) Imports

```python
import numpy as np

from tutorial_semipar_auxiliary import plot_semipar_mte

from grmpy.estimate.estimate import fit

import warnings
warnings.filterwarnings('ignore')
```

## 1) The LIV Framework

The method of Local Instrumental Variables (LIV) is based on the generalized Roy model, which is characterized by the following equations:

\begin{align*}
 &\textbf{Potential Outcomes} & & \textbf{Choice}  &\\
 & Y_1 = \beta_1 X + U_{1} &  & I = Z \gamma - V &\\
 & Y_0 = \beta_0 X + U_{0} &  & D_i = \left\{
\begin{array}{ll}
1 & if \ I > 0 \\
0 & if \ I \leq 0\\
\end{array}
\right.
&&&&\\
& \textbf{Observed Outcome} &&&\\
& Y = D Y_1 + (1-D) Y_0 &&&
\end{align*}

We work with the linear-in-the-parameters version of the generalized Roy model:

\begin{align}
E(Y|X = \overline{x}, P(Z) = p) =  \overline{x} \beta_0 + p \overline{x} (\beta_1 - \beta_0) + K(p),
\end{align}

where $K(p) = E(U_1 - U_0 | D = 1, P(Z) = p)$ is a nonlinear function of $p$ that captures heterogeneity along the unobservable resistance to treatment $u_D$.

In addition, assume that $(X, Z)$ is independent of $\{U_1, U_0, V\}$. Then, the MTE is

1) additively separable in $X$ and $U_D$, which means that the shape of the MTE is independent of $X$, and

2) identified over the common support of $P(Z)$, unconditional on $X$.

The common support, $P(Z)$, plays a crucial role for the identification of the MTE.
It denotes the probability of going to university ($D=1$). Common support is defined as the intersection of the support of $P(Z)$ given $D = 1$ and the support of $P(Z)$ given $D = 0$. i.e., those evaluations of $P(Z)$ for which we obtain positive frequencies in both subsamples. We will plot it below. The larger the common support, the larger the region over which the MTE is identified.

The LIV estimator, $\Delta^{LIV}$, is derived as follows (Heckman and Vytlacil [2001](https://www.aeaweb.org/articles?id=10.1257/aer.91.2.107), [2005](https://www.jstor.org/stable/3598865?seq=1#page_scan_tab_contents)):

\begin{equation}
\begin{split}
\Delta^{LIV} (\overline{x}, u_D) &= \frac{\partial E(Y|X = \overline{x}, P(Z) = p)}{\partial p}  \bigg\rvert_{p = u_D} \\
& \\
&= \overline{x}(\beta_1 - \beta_0) + E(U_1 - U_0 | U_D = u_D) \\
&\\
& = \underbrace{\overline{x}(\beta_1 - \beta_0)}_{\substack{observable \\ component}} + \underbrace{\frac{\partial K}{\partial p} \bigg\rvert_{p = u_D}}_{\substack{k(p): \ unobservable \\ component}} = MTE(\overline{x}, u_D)
\end{split}
%\frac{[E(U_1 - U_0 | U_D \leq p] p}{\partial p}  \bigg\rvert_{p = u_D}
%E(U_1 - U_0 | U_D = u_D)
\end{equation}

Since we do not make any assumption about the functional form of the unobservables, we estimate $k(p)$ non-parametrically. In particualr, $k(p)$ is the first derivative of a locally quadratic kernel regression.

## 3) The Initialization File

For the semiparametric estimation, we need information on the following sections:

* __ESTIMATION__: Specify the dependent (wage) and indicator variable (treatment dummy) of the input data frame.
For the estimation of the propensity score $P(Z)$, we choose a probability model, here logit. Furthermore, we select 30 bins to determine the common support in the treated and untreated subsamples. For the locally quadratic regression, we follow the specification of [Carneiro et al. (2011)](https://pubs.aeaweb.org/doi/pdfplus/10.1257/aer.101.6.2754) and choose a bandwidth of 0.322. The respective gridsize for the locally quadratic regression is set to 500. [Fan and Marron (1994)](https://www.tandfonline.com/doi/abs/10.1080/10618600.1994.10474629) find that a gridsize of 400 is a good default for graphical analysis. Since the data set is large (1785 observations) and the kernel regression function has a very low runtime, we increase the gridsize to 500. Setting it to the default or increasing it even more does not affect the final MTE.

Note that the MTE identified by LIV consists of two components: $\overline{x}(\beta_1 - \beta_0)$ (which does not depend on $P(Z) = p$) and $k(p)$ (which does depend on $p$). The latter is estimated nonparametrically. The key "p_range" in the initialization file specifies the interval over which $k(p)$ is estimated. After the data outside the overlapping support are trimmed, the locally quadratic kernel estimator uses the remaining data to predict $k(p)$ over the entire "p_range" specified by the user. If "p_range" is larger than the common support, *grmpy* extrapolates the values for the MTE outside this region. Technically speaking, interpretations of the MTE are only valid within the common support. Here, we set "p_range" to [0.005, 0.995].

The other parameters in this section are set by default and, normally, do not need to be changed.

* __TREATED, UNTREATED, CHOICE__: In this section, the variables of the outcome equations (treated, untreated) and the college decision (choice) are specified.

* __DIST__: The distribution of the unobservables is not of relevance in the semiparametric apporach and can be ignored.

```python
%%file files/tutorial_semipar.yml
---
ESTIMATION:
    file: data/aer-replication-mock.pkl
    dependent: wage
    indicator: state
    semipar: True
    show_output: True
    logit: True
    nbins: 30
    bandwidth: 0.322
    gridsize: 500
    trim_support: True
    reestimate_p: False
    rbandwidth: 0.05
    derivative: 1
    degree: 2
    ps_range: [0.005, 0.995]
TREATED:
    order:
    - exp
    - expsq
    - lwage5
    - lurate
    - cafqt
    - cafqtsq
    - mhgc
    - mhgcsq
    - numsibs
    - numsibssq
    - urban14
    - lavlocwage17
    - lavlocwage17sq
    - avurate
    - avuratesq
    - d57
    - d58
    - d59
    - d60
    - d61
    - d62
    - d63
UNTREATED:
    order:
    - exp
    - expsq
    - lwage5
    - lurate
    - cafqt
    - cafqtsq
    - mhgc
    - mhgcsq
    - numsibs
    - numsibssq
    - urban14
    - lavlocwage17
    - lavlocwage17sq
    - avurate
    - avuratesq
    - d57
    - d58
    - d59
    - d60
    - d61
    - d62
    - d63
CHOICE:
    params:
    - 1.0
    order:
    - const
    - cafqt
    - cafqtsq
    - mhgc
    - mhgcsq
    - numsibs
    - numsibssq
    - urban14
    - lavlocwage17
    - lavlocwage17sq
    - avurate
    - avuratesq
    - d57
    - d58
    - d59
    - d60
    - d61
    - d62
    - d63
    - lwage5_17numsibs
    - lwage5_17mhgc
    - lwage5_17cafqt
    - lwage5_17
    - lurate_17
    - lurate_17numsibs
    - lurate_17mhgc
    - lurate_17cafqt
    - tuit4c
    - tuit4cnumsibs
    - tuit4cmhgc
    - tuit4ccafqt
    - pub4
    - pub4numsibs
    - pub4mhgc
    - pub4cafqt
DIST:
    params:
    - 0.1
    - 0.0
    - 0.0
    - 0.1
    - 0.0
    - 1.0
```

    Overwriting files/tutorial_semipar.yml

Note that I do not include a constant in the __TREATED, UNTREATED__ section. The reason for this is that in the semiparametric setup, $\beta_1$ and $\beta_0$ are determined by running a Double Residual Regression without an intercept:
	$$ e_Y =e_X \beta_0 \ + \ e_{X \ \times \ p} (\beta_1 - \beta_0) \ + \ \epsilon $$

where $e_X$, $e_{X \ \times \ p}$, and $e_Y$ are the residuals of a local linear regression of $X$, $X$ x $p$, and $Y$ on $\widehat{P}(Z)$.

We now proceed to our replication.

## 3) Estimation

Conduct the estimation based on the initialization file.

```python
rslt = fit('files/tutorial_semipar.yml', semipar=True)
```

The rslt dictionary contains information on the estimated parameters and the final MTE.

```python
list(rslt)
```

    ['quantiles', 'mte', 'mte_x', 'mte_u', 'mte_min', 'mte_max', 'X', 'b1', 'b0']

Before plotting the MTE, let's see what else we can learn.
For instance, we can account for the variation in $X$.

Note that we divide the MTE by 4 to investigate the effect of one additional year of college education.

```python
np.min(rslt['mte_min']) / 4, np.max(rslt['mte_max']) / 4
```

    (-0.5703098725415803, 0.6420576895039005)

Next we plot the MTE based on the estimation results. As shown in the figure below, the replicated MTE gets very close to the original, but its 90 percent confidence bands are wider. This is due to the use of a mock data set which merges basic and local variables randomly. The bootsrap method, which is used to estimate the confidence bands, is sensitive to the discrepancies in the data.

```python
mte, quantiles = plot_semipar_mte(rslt, 'files/tutorial_semipar.yml', nbootstraps=250)
```

People with the highest returns to education (those who have low unobserved resistance $u_D$
) are more likely to go to college. Note that the returns vary considerably with $u_D$
. Low $u_D$ students have returns of up to 40% per year of college, whereas high $u_D$
 people, who would loose from attending college, have returns of approximately -18%.

The magnitude of total heterogeneity is probably even higher, as the MTE depicts the average gain of
college attendance at the mean values of X, i.e. $\bar{x} (\beta_1 - \beta_0)$.
Accounting for variation in $X$, we observe returns as high as 64% and as low as -57%.


# Brianによる神経シミュレーション

Brianは細胞集団レベルでのシミュレーションに向いています。

順番に Shift + Enter を押していくだけで、シミュレーションが可能です。

参考資料

1.公式チュートリアル　https://brian2.readthedocs.io/en/stable/resources/tutorials/index.html

2.Brianで神経活動シミュレーション https://qiita.com/takeajioka/items/11690b9adee612e48b33

## Brianのインストール

```
!pip install brian2
```

    Requirement already satisfied: brian2 in /usr/local/lib/python3.6/dist-packages (2.3.0.2)
    Requirement already satisfied: sympy>=1.2 in /usr/local/lib/python3.6/dist-packages (from brian2) (1.6.2)
    Requirement already satisfied: numpy>=1.10 in /usr/local/lib/python3.6/dist-packages (from brian2) (1.18.5)
    Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python3.6/dist-packages (from brian2) (2.11.2)
    Requirement already satisfied: cython>=0.29 in /usr/local/lib/python3.6/dist-packages (from brian2) (0.29.21)
    Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from brian2) (0.16.0)
    Requirement already satisfied: pyparsing in /usr/local/lib/python3.6/dist-packages (from brian2) (2.4.7)
    Requirement already satisfied: setuptools>=24 in /usr/local/lib/python3.6/dist-packages (from brian2) (49.2.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy>=1.2->brian2) (1.1.0)
    Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from jinja2>=2.7->brian2) (1.1.1)

これで、GoogleColab上でBrianが使えるようになりました。(制限時間12時間)

次からは、デモを行います。

## デモ１　神経細胞集団のラスタープロット

```
import brian2
brian2.start_scope()
```

    /usr/local/lib/python3.6/dist-packages/setuptools/distutils_patch.py:26: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
      "Distutils was imported before Setuptools. This usage is discouraged "

```
# 神経細胞集団の設定
# 100個の細胞にランダムに電位を入力している
N = 100
tau = 10 * brian2.ms #膜時定数
v0 = -50 * brian2.mV #静止膜電位
eqs = '''
dv/dt = (v0-v+I)/tau : volt (unless refractory)
I : volt
'''
G = brian2.NeuronGroup(N, eqs, threshold='v>-40*mV', reset='v=-60*mV', refractory=1*brian2.ms, method='exact')
G.v = -70 * brian2.mV
I_max = 40 * brian2.mV
G.I = 'I_max*rand()'
```

```
# モニタ変数を設定
spikemon = brian2.SpikeMonitor(G)
```

```
# シミュレーション実行
brian2.run(200*brian2.ms)
```

```
# プロット
brian2.plot(spikemon.t/brian2.ms, spikemon.i, '.k')
brian2.xlabel('Time (ms)')
brian2.ylabel('Neuron index');
```

## デモ２　STDPによる可塑性のシミュレーション

```
import brian2
brian2.start_scope()
```

```
# 神経細胞集団の設定
# 100個の細胞にランダムに電位を入力している
N = 100
tau = 10 * brian2.ms #膜時定数
v0 = -50 * brian2.mV #静止膜電位
eqs = '''
dv/dt = (v0-v+I)/tau : volt (unless refractory)
I : volt
'''
G = brian2.NeuronGroup(N, eqs, threshold='v>-40*mV', reset='v=-60*mV', refractory=1*brian2.ms, method='exact')
G.v = -70 * brian2.mV
I_max = 40 * brian2.mV
G.I = 'I_max*rand()'
```

```
# シナプス結合を定義（STDP）
taupre = taupost = 20*brian2.ms
Apre = 0.01
Apost = -Apre*taupre/taupost*1.05
S = brian2.Synapses(G, G,
             '''
             w : 1
             dapre/dt = -apre/taupre : 1 (event-driven)
             dapost/dt = -apost/taupost : 1 (event-driven)
             ''',
             on_pre='''
             apre += Apre
             w = w+apost
             v_post += w*3*mV
             ''',
             on_post='''
             apost += Apost
             w = w+apre
             ''')
S.connect(condition='i!=j')
```

```
# 最初のシナプス結合
brian2.plot(S.w,'o')
brian2.xlabel('Synapse index')
brian2.ylabel('Weight');
```

```
# モニタ変数を設定
spikemon = brian2.SpikeMonitor(G)
```

```
# シミュレーション実行
brian2.run(1*brian2.second)
```

```
# シミュレーション後のシナプス結合
brian2.plot(S.w,'o')
brian2.xlabel('Synapse index')
brian2.ylabel('Weight');
```

```
# 初期のプロット
brian2.plot(spikemon.t/brian2.ms, spikemon.i, '.k')
brian2.xlabel('Time (ms)')
brian2.ylabel('Neuron index')
brian2.xlim([0, 200]);
```

```
# 最後のプロット
brian2.plot(spikemon.t/brian2.ms, spikemon.i, '.k')
brian2.xlabel('Time (ms)')
brian2.ylabel('Neuron index')
brian2.xlim([1000 - 200, 1000]);
```

シミュレーションによってシナプス結合が形成されるようになりました。

神経活動も変化して同期が見られるようになりました。


# 6 - SGD

```python
#@title Run this cell to download the data and helper files. { display-mode: "form" }
!pip install -U wget
!rm -rf data.zip data lib
!mkdir lib

import wget
wget.download('https://github.com/shengpu1126/BDSI2019-ML/raw/master/lib/config.yaml', 'lib/config.yaml')
wget.download('https://github.com/shengpu1126/BDSI2019-ML/raw/master/lib/helper.py', 'lib/helper.py')
wget.download('https://github.com/shengpu1126/BDSI2019-ML/raw/master/data.zip', 'data.zip')

import zipfile
with zipfile.ZipFile("data.zip","r") as zip_ref:
    zip_ref.extractall(".")
```

```python
import numpy as np
import pandas as pd
from tqdm import tqdm
import matplotlib.pyplot as plt
from lib.helper import load_data, config
```

```python
#@title Run this cell to run preprocessing. { display-mode: "form" }

def generate_feature_vector(df):
    """
    Reads a dataframe containing all measurements for a single patient
    within the first 48 hours of the ICU admission, and convert it into
    a feature vector.

    Args:
        df: pd.Dataframe, with columns [Time, Variable, Value]

    Returns:
        a python dictionary of format {feature_name: feature_value}
        for example, {'Age': 32, 'Gender': 0, 'mean_HR': 84, ...}
    """
    static_variables = config['invariant']
    timeseries_variables = config['timeseries']

    # Replace unknow values
    df = df.replace({-1: np.nan})

    # Split time invariant and time series
    static, timeseries = df.iloc[0:5], df.iloc[5:]
    static = static.pivot('Time', 'Variable', 'Value')

    feature_dict = static.iloc[0].to_dict()
    for variable in timeseries_variables:
        measurements = timeseries[timeseries['Variable'] == variable].Value
        feature_dict['mean_' + variable] = np.mean(measurements)

    return feature_dict

def impute_missing_values(X):
    """
    For each feature column, impute missing values  (np.nan) with the
    population mean for that feature.

    Args:
        X: np.array, shape (N, d). X could contain missing values
    Returns:
        X: np.array, shape (N, d). X does not contain any missing values
    """
    from sklearn.impute import SimpleImputer
    return SimpleImputer().fit_transform(X)

def normalize_feature_matrix(X):
    """
    For each feature column, normalize all values to range [0, 1].

    Args:
        X: np.array, shape (N, d).
    Returns:
        X: np.array, shape (N, d). Values are normalized per column.
    """
    from sklearn.preprocessing import MinMaxScaler
    return MinMaxScaler().fit_transform(X)

# Load the dataset
# `raw_data` is a dictionary mapping patient ID to the data associated with that patient
raw_data, df_labels = load_data(N=2500)

# Generate features
features = [generate_feature_vector(df) for _, df in tqdm(sorted(raw_data.items()), desc='Generating feature vectors')]
df_features = pd.DataFrame(features).sort_index(axis=1)
feature_names = df_features.columns.tolist()

# Apply imputation and normalization
X, y = df_features.values, df_labels['In-hospital_death'].values
X = impute_missing_values(X)
X = normalize_feature_matrix(X)

# Split data into train and test
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, stratify=y, random_state=3)
del X, y
```

For a binary classification task, we have feature vectors $\bar{x} = [x_1, x_2, . . . , x_d]^\intercal \in \mathbb{R}^d$ and labels $y \in \{-1,+1\}$. A classifier is a function mapping from feature vectors to labels $h : \mathbb{R}^d \rightarrow \{-1,+1\}$. For linear classification, the classifiers are thresholded linear mappings:

\begin{equation*}
h(\bar{x};\bar\theta) = \mathrm{sign}(\bar\theta \cdot \bar x) = \bigg\{\begin{matrix}
+1 & \bar\theta \cdot \bar x > 0\\
-1 & \bar\theta \cdot \bar x \leq 0
\end{matrix}
\end{equation*}

We have also defined the following:

- Training error
\begin{equation}
    \mathcal{E} (\bar{\theta}) = \frac{1}{N} \sum_{i=1}^N [[y^{(i)} \neq h(\bar{x}^{(i)};\bar{\theta})]]
    \nonumber
\end{equation}

- Empirical risk (wrt some loss function)
\begin{equation}
    R(\bar{\theta}) = \frac{1}{N} \sum_{i=1}^N \mathrm{loss}(y^{(i)} \bar{\theta} \cdot \bar{x}^{(i)})
    \nonumber
\end{equation}

So far we have seen 3 loss functions suitable for a (binary) classification task with label $y\in\{-1,+1\}$:
- $\mathrm{loss}_{0/1}(z) = [[z\leq 0]]$
- $\mathrm{loss}_{h}(z) = \max\{0, 1-z\}$
- $\mathrm{loss}_{\log}(z) = \log_2(1+e^{-z})$

Here $z = y \; (\bar{\theta} \cdot \bar{x})$; interpretation for $z$: "a numerical quantity that indicates whether or not you're right, and to what extent you are right or wrong".

## Review our data...

```python
sel = [0,1,2,3,6,8,55,88]    # Select an easy subset of patients
X = X_train[sel][:, [0,20]]  # Select age and mean_HR
y = y_train[sel]
```

```python
def plot_data(X, y, markersize=25):
    fig = plt.figure(figsize=(5,5))
    for xi, yi in zip(X, y):
        if yi == -1:
            plt.scatter(xi[0], xi[1], c='r', marker='o', s=markersize)
        elif yi == 1:
            plt.scatter(xi[0], xi[1], c='g', marker='x', s=markersize)
    plt.axis('equal')
    plt.xlabel('Age')
    plt.ylabel('mean_HR')
    plt.xlim(0,1)
    plt.ylim(0,1)
    plt.grid(True)
    return fig
```

```python
fig = plot_data(X, y)
plt.show()
```

## (1) SGD with hinge loss

The empirical risk with hinge loss is defined as:
\begin{equation}
    J(\bar{\theta}) = \frac{1}{N} \sum_{i=1}^N \max\{0, 1-y \; (\bar{\theta} \cdot \bar{x}^{(i)})\}
    \nonumber
\end{equation}

Calculate the gradient $\nabla_{\bar\theta}J(\bar{\theta})$:

>$\nabla_{\bar\theta}J(\bar{\theta}) = \dots$

Write down the update rule using SGD upon seeing one example $(\bar{x}^{(i)}, y^{(i)})$:

>$\bar\theta^{(k+1)} = \dots$

```python
# Implement SGD with hinge loss
def SGD_hinge(X, y, lr=1e-3, max_iters=1e4):
    n, d = X.shape
    theta = np.zeros(d)
    losses = []

    n_iter = 0
    while n_iter < max_iters:
        # Loop through the entire dataset
        for xi, yi in zip(X, y):
            n_iter += 1

            # Update theta based on each example (xi, yi)
            theta = ??

            # Track training progress by calculating empirical risk after each update
            losses.append(empirical_risk(X, y, theta))

        # Optionally, shuffle the data points
        X, y = ??shuffle(X, y)??

    return theta, losses
```

```python
X_ = np.concatenate([np.ones((X.shape[0],1)), X], axis=1) # Create a synthetic feature dimension to incorporate the offset term
theta, losses = SGD_hinge(X_, y, 1e-2, 1e4)
print(theta)
```

```python
# Visualize training progress
plt.plot(losses)
plt.xlabel('iterations')
plt.ylabel('Loss')
plt.show()
```

```python
# Visualize the classification boundary in the same plot. Does it match your intuition?
x_clf = np.linspace(0,1,100)
y_clf = (theta_sgd[0] * x_clf + theta_sgd[2]) / -theta_sgd[1]
fig = plot_data(X, y)
plt.plot(x_clf, y_clf, 'k-')
plt.show()
```

```python

```

## (2) SGD with hinge loss and L2-regularization

The empirical risk with hinge loss and L2-regularization is defined as:
\begin{equation}
    J(\bar{\theta}) = \frac{\lambda}{2}\|\bar\theta\|_2^2 + \frac{1}{N} \sum_{i=1}^N \max\{0, 1-y \; (\bar{\theta} \cdot \bar{x}^{(i)})\}
    \nonumber
\end{equation}

Calculate the gradient $\nabla_{\bar\theta}J(\bar{\theta})$:

>$\nabla_{\bar\theta}J(\bar{\theta}) = \dots$

Write down the update rule using SGD upon seeing one example $(\bar{x}^{(i)}, y^{(i)})$:

>$\bar\theta^{(k+1)} = \dots$

```python
def SGD_hinge_L2(X, y, lr=1e-3, lambda_=1):
    theta = None
    losses = []
    return theta, losses
```

```python
X_ = np.concatenate([X, np.ones((X.shape[0],1))], axis=1)
theta, losses = SGD_hinge(X_, y, 1e-2)
```

```python
# Visualizations...
```

## (3) Extensions
- Use a larger training set.
- What should the convergence criteria be?
- Experiment with different learning rates and/or regularization strengths.
- What would the update rule(s) be if we do not regularize the offset parameter?

```python

```

## (4) Extra: SGD with logistic loss

\begin{equation}
J(\bar\theta) = \frac{1}{N} \sum_{i=1}^{N} \ln(1+e^{-y^{(i)}\bar{\theta}\cdot \bar{x}^{(i)}})
\end{equation}

Calculate the gradient $\nabla_{\bar\theta}J(\bar{\theta})$:

>$\nabla_{\bar\theta}J(\bar{\theta}) = \dots$

Write down the update rule using SGD upon seeing one example $(\bar{x}^{(i)}, y^{(i)})$:

>$\bar\theta^{(k+1)} = \dots$

```python
def sigmoid(z):
    return 1 / (1 + np.exp(-z))
```

```python
def SGD_logistic(X, y, lr=1e-3):
    theta = None
    losses = []
    return theta
```

```python

```