#  Neuromatch Academy: Week 1, Day 1, Tutorial 3
# Model Types: "Why" models
__Content creators:__ Matt Laporte, Byron Galbraith, Konrad Kording

__Content reviewers:__ Dalin Guo, Aishwarya Balwani, Madineh Sarvestani, Maryam Vaziri-Pashkam, Michael Waskom

We would like to acknowledge [Steinmetz _et al._ (2019)](https://www.nature.com/articles/s41586-019-1787-x) for sharing their data, a subset of which is used here.

___
# Tutorial Objectives
This is tutorial 3 of a 3-part series on different flavors of models used to understand neural data. In parts 1 and 2 we explored mechanisms that would produce the data. In this tutorial we will explore models and techniques that can potentially explain *why* the spiking data we have observed is produced the way it is.

To understand why different spiking behaviors may be beneficial, we will learn about the concept of entropy. Specifically, we will:

- Write code to compute formula for entropy, a measure of information
- Compute the entropy of a number of toy distributions
- Compute the entropy of spiking activity from the Steinmetz dataset

```python
#@title Video 1: “Why” models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='OOIDEr1e5Gg', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video

```

    Video available at https://youtube.com/watch?v=OOIDEr1e5Gg

# Setup

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```

```python
#@title Figure Settings
import ipywidgets as widgets #interactive display

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")

```

```python
#@title Helper Functions

def plot_pmf(pmf,isi_range):
  """Plot the probability mass function."""
  ymax = max(0.2, 1.05 * np.max(pmf))
  pmf_ = np.insert(pmf, 0, pmf[0])
  plt.plot(bins, pmf_, drawstyle="steps")
  plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
  plt.title(f"Neuron {neuron_idx}")
  plt.xlabel("Inter-spike interval (s)")
  plt.ylabel("Probability mass")
  plt.xlim(isi_range);
  plt.ylim([0, ymax])
```

```python
#@title Download Data
import io
import requests
r = requests.get('https://osf.io/sy5xt/download')
if r.status_code != 200:
  print('Could not download data')
else:
  steinmetz_spikes = np.load(io.BytesIO(r.content), allow_pickle=True)['spike_times']
```

# Section 1: Optimization and Information

Neurons can only fire so often in a fixed period of time, as the act of emitting a spike consumes energy that is depleted and must eventually be replenished. To communicate effectively for downstream computation, the neuron would need to make good use of its limited spiking capability. This becomes an optimization problem:

What is the optimal way for a neuron to fire in order to maximize its ability to communicate information?

In order to explore this question, we first need to have a quantifiable measure for information. Shannon introduced the concept of entropy to do just that, and defined it as

\begin{align}
  H_b(X) &= -\sum_{x\in X} p(x) \log_b p(x)
\end{align}

where $H$ is entropy measured in units of base $b$ and $p(x)$ is the probability of observing the event $x$ from the set of all possible events in $X$. See the Appendix for a more detailed look at how this equation was derived.

The most common base of measuring entropy is $b=2$, so we often talk about *bits* of information, though other bases are used as well e.g. when $b=e$ we call the units *nats*.

First, let's explore how entropy changes between some simple discrete probability distributions. In the rest of this tutorial we will refer to these as probability mass functions (PMF), where $p(x_i)$ equals the $i^{th}$ value in an array, and mass refers to how much of the distribution is contained at that value.

For our first PMF, we will choose one where all of the probability mass is located in the middle of the distribution.

```python
n_bins = 50  # number of points supporting the distribution
x_range = (0, 1)  # will be subdivided evenly into bins corresponding to points

bins = np.linspace(*x_range, n_bins + 1)  # bin edges

pmf = np.zeros(n_bins)
pmf[len(pmf) // 2] = 1.0  # middle point has all the mass
# Since we already have a PMF, rather than un-binned samples, `plt.hist` is not
# suitable. Instead, we directly plot the PMF as a step function to visualize
# the histogram:
pmf_ = np.insert(pmf, 0, pmf[0])  # this is necessary to align plot steps with bin edges
plt.plot(bins, pmf_, drawstyle="steps")
# `fill_between` provides area shading
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range)
plt.ylim(0, 1);
```

If we were to draw a sample from this distribution, we know exactly what we would get every time. Distributions where all the mass is concentrated on a single event are known as *deterministic*.

How much entropy is contained in a deterministic distribution?

## Exercise 1: Computing Entropy

Your first exercise is to implement a method that computes the entropy of a discrete probability distribution, given its mass function. Remember that we are interested in entropy in units of _bits_, so be sure to use the correct log function.

Recall that $\log(0)$ is undefined. When evaluated at $0$, NumPy log functions (such as `np.log2`) return `np.nan` ("Not a Number"). By convention, these undefined terms— which correspond to points in the distribution with zero mass—are excluded from the sum that computes the entropy.

```python
def entropy(pmf):
  """Given a discrete distribution, return the Shannon entropy in bits.

  This is a measure of information in the distribution. For a totally
  deterministic distribution, where samples are always found in the same bin,
  then samples from the distribution give no more information and the entropy
  is 0.

  For now this assumes `pmf` arrives as a well-formed distribution (that is,
  `np.sum(pmf)==1` and `not np.any(pmf < 0)`)

  Args:
    pmf (np.ndarray): The probability mass function for a discrete distribution
      represented as an array of probabilities.
  Returns:
    h (number): The entropy of the distribution in `pmf`.

  """
  ############################################################################
  # Exercise for students: compute the entropy of the provided PMF
  #   1. Exclude the points in the distribution with no mass (where `pmf==0`).
  #      Hint: this is equivalent to including only the points with `pmf>0`.
  #   2. Implement the equation for Shannon entropy (in bits).
  #  When ready to test, comment or remove the next line
#   raise NotImplementedError("Excercise: implement the equation for entropy")
  ############################################################################

  # reduce to non-zero entries to avoid an error from log2(0)
  pmf = pmf[pmf>0]

  # implement the equation for Shannon entropy (in bits)
  h = -np.sum(pmf*np.log2(pmf))

  # return the absolute value (avoids getting a -0 result)
  return h#np.abs(h)

# Uncomment to test your entropy function
print(f"{entropy(pmf):.2f} bits")
```

    -0.00 bits

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D1_ModelTypes/solutions/W1D1_Tutorial3_Solution_55c07dc8.py)

We expect zero surprise from a deterministic distribution. If we had done this calculation by hand, it would simply be

$-1\log_2 1 = -0=0$

Note that changing the location of the peak (i.e. the point and bin on which all the mass rests) doesn't alter the entropy. The entropy is about how predictable a sample is with respect to a distribution. A single peak is deterministic regardless of which point it sits on.

```python
pmf = np.zeros(n_bins)
pmf[2] = 1.0  # arbitrary point has all the mass

pmf_ = np.insert(pmf, 0, pmf[0])
plt.plot(bins, pmf_, drawstyle="steps")
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range)
plt.ylim(0, 1);
entropy(pmf)
```

What about a distribution with mass split equally between two points?

```python
pmf = np.zeros(n_bins)
pmf[len(pmf) // 3] = 0.5
pmf[2 * len(pmf) // 3] = 0.5

pmf_ = np.insert(pmf, 0, pmf[0])
plt.plot(bins, pmf_, drawstyle="steps")
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range)
plt.ylim(0, 1);
print(entropy(pmf))
```

Here, the entropy calculation is

$-(0.5 \log_2 0.5 + 0.5\log_2 0.5)=1$

There is 1 bit of entropy. This means that before we take a random sample, there is 1 bit of uncertainty about which point in the distribution the sample will fall on: it will either be the first peak or the second one.

Likewise, if we make one of the peaks taller (i.e. its point holds more of the probability mass) and the other one shorter, the entropy will decrease because of the increased certainty that the sample will fall on one point and not the other:

$-(0.2 \log_2 0.2 + 0.8\log_2 0.8)\approx 0.72$

Try changing the definition of the number and weighting of peaks, and see how the entropy varies.

If we split the probability mass among even more points, the entropy continues to increase. Let's derive the general form for $N$ points of equal mass, where $p_i=p=1/N$:

\begin{align}
 -\sum_i p_i \log_b p_i&= -\sum_i^N \frac{1}{N} \log_b \frac{1}{N}\\
                       &= -\log_b \frac{1}{N} \\
                       &= \log_b N
\end{align}
$$$$

If we have $N$ discrete points, the _uniform distribution_ (where all points have equal mass) is the distribution with the highest entropy: $\log_b N$. This upper bound on entropy is useful when considering binning strategies, as any estimate of entropy over $N$ discrete points (or bins) must be in the interval $[0, \log_b N]$.

```python
pmf = np.ones(n_bins) / n_bins  # [1/N] * N

pmf_ = np.insert(pmf, 0, pmf[0])
plt.plot(bins, pmf_, drawstyle="steps")
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range);
plt.ylim(0, 1);
print(entropy(pmf))
```

Here, there are 50 points and the entropy of the uniform distribution is $\log_2 50\approx 5.64$. If we construct _any_ discrete distribution $X$ over 50 points (or bins) and calculate an entropy of $H_2(X)>\log_2 50$, something must be wrong with our implementation of the discrete entropy computation.

# Section 2: Information, neurons, and spikes

```python
#@title Video 2: Entropy of different distributions
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='o6nyrx3KH20', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=o6nyrx3KH20

Recall the discussion of spike times and inter-spike intervals (ISIs) from Tutorial 1. What does the information content (or distributional entropy) of these measures say about our theory of nervous systems?

We'll consider three hypothetical neurons that all have the same mean ISI, but with different distributions:

1. Deterministic
2. Uniform
3. Exponential

Fixing the mean of the ISI distribution is equivalent to fixing its inverse: the neuron's mean firing rate. If a neuron has a fixed energy budget and each of its spikes has the same energy cost, then by fixing the mean firing rate, we are normalizing for energy expenditure. This provides a basis for comparing the entropy of different ISI distributions. In other words: if our neuron has a fixed budget, what ISI distribution should it express (all else being equal) to maximize the information content of its outputs?

Let's construct our three distributions and see how their entropies differ.

```python
n_bins = 50
mean_isi = 0.025
isi_range = (0, 0.25)

bins = np.linspace(*isi_range, n_bins + 1)
mean_idx = np.searchsorted(bins, mean_isi)

# 1. all mass concentrated on the ISI mean
pmf_single = np.zeros(n_bins)
pmf_single[mean_idx] = 1.0

# 2. mass uniformly distributed about the ISI mean
pmf_uniform = np.zeros(n_bins)
pmf_uniform[0:2*mean_idx] = 1 / (2 * mean_idx)

# 3. mass exponentially distributed about the ISI mean
pmf_exp = stats.expon.pdf(bins[1:], scale=mean_isi)
pmf_exp /= np.sum(pmf_exp)
```

```python
#@title
#@markdown Run this cell to plot the three PMFs
fig, axes = plt.subplots(ncols=3, figsize=(18, 5))

dists =  [# (subplot title, pmf, ylim)
          ("(1) Deterministic", pmf_single, (0, 1.05)),
          ("(1) Uniform", pmf_uniform, (0, 1.05)),
          ("(1) Exponential", pmf_exp, (0, 1.05))]

for ax, (label, pmf_, ylim) in zip(axes, dists):
  pmf_ = np.insert(pmf_, 0, pmf_[0])
  ax.plot(bins, pmf_, drawstyle="steps")
  ax.fill_between(bins, pmf_, step="pre", alpha=0.4)
  ax.set_title(label)
  ax.set_xlabel("Inter-spike interval (s)")
  ax.set_ylabel("Probability mass")
  ax.set_xlim(isi_range);
  ax.set_ylim(ylim);
```

```python
print(
  f"Deterministic: {entropy(pmf_single):.2f} bits",
  f"Uniform: {entropy(pmf_uniform):.2f} bits",
  f"Exponential: {entropy(pmf_exp):.2f} bits",
  sep="\n",
)
```

    Deterministic: -0.00 bits
    Uniform: 3.32 bits
    Exponential: 3.77 bits

```python
#@title Video 3: Probabilities from histogram
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='e2U_-07O9jo', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=e2U_-07O9jo

In the previous example we created the PMFs by hand to illustrate idealized scenarios. How would we compute them from data recorded from actual neurons?

One way is to convert the ISI histograms we've previously computed into discrete probability distributions using the following equation:

\begin{align}
p_i = \frac{n_i}{\sum\nolimits_{i}n_i}
\end{align}

where $p_i$ is the probability of an ISI falling within a particular interval $i$ and $n_i$ is the count of how many ISIs were observed in that interval.

### Exercise 2: Probabilty Mass Function

Your second exercise is to implement a method that will produce a probability mass function from an array of ISI bin counts.

To verify your solution, we will compute the probability distribution of ISIs from real neural data taken from the Steinmetz dataset.

```python
neuron_idx = 283

isi = np.diff(steinmetz_spikes[neuron_idx])
bins = np.linspace(*isi_range, n_bins + 1)
counts, _ = np.histogram(isi, bins)
```

```python
def pmf_from_counts(counts):
  """Given counts, normalize by the total to estimate probabilities."""
  ###########################################################################
  # Exercise: Compute the PMF. Remove the next line to test your function
#   raise NotImplementedError("Student excercise: compute the PMF from ISI counts")
  ###########################################################################

  pmf = counts/np.sum(counts)

  return pmf

# Uncomment when ready to test your function
pmf = pmf_from_counts(counts)
plot_pmf(pmf,isi_range)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D1_ModelTypes/solutions/W1D1_Tutorial3_Solution_49231923.py)

*Example output:*

# Section 3: Calculate entropy from a PMF

```python
#@title Video 4: Calculating entropy from pmf
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='Xjy-jj-6Oz0', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=Xjy-jj-6Oz0

Now that we have the probability distribution for the actual neuron spiking activity, we can calculate its entropy.

```python
print(f"Entropy for Neuron {neuron_idx}: {entropy(pmf):.2f} bits")
```

    Entropy for Neuron 283: 3.36 bits

## Interactive Demo: Entropy of neurons

We can combine the above distribution plot and entropy calculation with an interactive widget to explore how the different neurons in the dataset vary in spiking activity and relative information. Note that the mean firing rate across neurons is not fixed, so some neurons with a uniform ISI distribution may have higher entropy than neurons with a more exponential distribution.

```python
#@title
#@markdown **Run the cell** to enable the sliders.

def _pmf_from_counts(counts):
  """Given counts, normalize by the total to estimate probabilities."""
  pmf = counts / np.sum(counts)
  return pmf

def _entropy(pmf):
  """Given a discrete distribution, return the Shannon entropy in bits."""
  # remove non-zero entries to avoid an error from log2(0)
  pmf = pmf[pmf > 0]
  h = -np.sum(pmf * np.log2(pmf))
  # absolute value applied to avoid getting a -0 result
  return np.abs(h)

@widgets.interact(neuron=widgets.IntSlider(0, min=0, max=(len(steinmetz_spikes)-1)))
def steinmetz_pmf(neuron):
  """ Given a neuron from the Steinmetz data, compute its PMF and entropy """
  isi = np.diff(steinmetz_spikes[neuron])
  bins = np.linspace(*isi_range, n_bins + 1)
  counts, _ = np.histogram(isi, bins)
  pmf = _pmf_from_counts(counts)

  plot_pmf(pmf,isi_range)
  plt.title(f"Neuron {neuron}: H = {_entropy(pmf):.2f} bits")

```

    interactive(children=(IntSlider(value=0, description='neuron', max=733), Output()), _dom_classes=('widget-inte…

```python
counts = [np.histogram(np.diff(i),bins=50,range=isi_range)[0] for i in steinmetz_spikes]
entropys = [entropy(count/np.sum(count)) for count in counts]
for i in np.argsort(entropys)[::-1][:5]:
    plt.hist(np.diff(steinmetz_spikes[i]),bins=50,range=(0,10),alpha=0.5)
```

---
# Summary

```python
#@title Video 5: Summary of model types
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='X4K2RR5qBK8', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=X4K2RR5qBK8

Congratulations! You've finished your first NMA tutorial. In this 3 part tutorial series, we used different types of models to understand the spiking behavior of neurons recorded in the Steinmetz data set.

 - We used "what" models to discover that the ISI distribution of real neurons is closest to an exponential distribution
 - We used "how" models to discover that balanced excitatory and inbhitiory inputs, coupled with a leaky membrane, can give rise to neuronal spiking with exhibiting such an exponential ISI distribution
 - We used "why" models to discover that exponential ISI distributions contain the most information when the mean spiking is constrained

# Bonus

### The foundations for Entropy

In his foundational [1948 paper](https://en.wikipedia.org/wiki/A_Mathematical_Theory_of_Communication) on information theory, Claude Shannon began with three criteria for a function $H$ defining the entropy of a discrete distribution of probability masses $p_i\in p(X)$ over the points $x_i\in X$:
1. $H$ should be continuous in the $p_i$.
  - That is, $H$ should change smoothly in response to smooth changes to the mass $p_i$ on each point $x_i$.
2. If all the points have equal shares of the probability mass, $p_i=1/N$, $H$ should be a non-decreasing function of $N$.
  - That is, if $X_N$ is the support with $N$ discrete points and $p(x\in X_N)$ assigns constant mass to each point, then $H(X_1) < H(X_2) < H(X_3) < \dots$
3. $H$ should be preserved by (invariant to) the equivalent (de)composition of distributions.
  - For example (from Shannon's paper) if we have a discrete distribution over three points with masses $(\frac{1}{2},\frac{1}{3},\frac{1}{6})$, then their entropy can be represented in terms of a direct choice between the three and calculated $H(\frac{1}{2},\frac{1}{3},\frac{1}{6})$. However, it could also be represented in terms of a series of two choices:
    1. either we sample the point with mass $1/2$ or not (_not_ is the other $1/2$, whose subdivisions are not given in the first choice),
    2. if (with probability $1/2$) we _don't_ sample the first point, we sample one of the two remaining points, masses $1/3$ and $1/6$.

    Thus in this case we require that $H(\frac{1}{2},\frac{1}{3},\frac{1}{6})=H(\frac{1}{2},\frac{1}{2}) + \frac{1}{2}H(\frac{1}{3}, \frac{1}{6})$

There is a unique function (up to a linear scaling factor) which satisfies these 3 requirements:

\begin{align}
  H_b(X) &= -\sum_{x\in X} p(x) \log_b p(x)
\end{align}

Where the base of the logarithm $b>1$ controls the units of entropy. The two most common cases are $b=2$ for units of _bits_, and $b=e$ for _nats_.

We can view this function as the expectation of the self-information over a distribution:

$$H_b(X) = \mathbb{E}_{x\in X} \left[I_b(x)\right]$$

$$I_b(x)=-\log_b p(x)$$

Self-information is just the negative logarithm of probability, and is a measure of how surprising an event sampled from the distribution would be. Events with $p(x)=1$ are certain to occur, and their self-information is zero (as is the entropy of the distribution they compose) meaning they are totally unsurprising. The smaller the probability of an event, the higher its self-information, and the more surprising the event would be to observe.


# Probabilidad II

# 1. Cadenas de Markov

Una cadena de Markov es un proceso aleatorio con la propiedad de Markov. Un proceso aleatorio o estocástico, es un objeto matemático definido como una colección de variables aleatorias. Una cadena de Markov tiene ya sea un espacio de estado discreto (que representaría posibles valores de variables aleatorias) o un índice discreto (usualmente representando tiempo). Usualmente el término "cadena de Markov" se utiliza para describir un proceso con un conjunto discreto de tiempos, es decir una cadena en tiempo discreto de Markov (DTMC).

## Cadena en tiempo discreto de Markov.

Una cadena en tiempo discreto involucra un sistema que cambia aleatoriamente entre pasos. Estos pasos son usualmente vistos como momentos en tiempo (no necesariamente la cantidad física). Una cadena discreta de Markov es una secuencia de variables aleatorias tales que la probabilidad de pasar al siguiente estado sólo depende del estado presente y no de los estados previos.

\begin{equation}
Pr(X_{n+1}=x|X_1=x_1,X_2=x_2,...,X_n=x_n)=P_r(X_{n+1}=x|X_n =x_n)
\end{equation}

```python
import numpy as np
import random as rm

states=["Dormir","Comer","Estudiar"]
nombreTransicion=[["DD","DC","DE"],["CD","CC","CE"],["ED","EC","EE"]]

MT=[[0.2,0.6,0.2],[0.1,0.6,0.3],[0.2,0.7,0.1]]

if sum(MT[0])+sum(MT[1])+sum(MT[2])!=3:
    print("No está bien")
else: print("Todo está bien, adelante!! ;)")
```

    Todo está bien, adelante!! ;)

```python
def prediccion_actividades(days):
    actividadHoy="Dormir"
    print("Estado Inicial: "+actividadHoy)
    listaActividades=[actividadHoy]
    i=0
    prob=1
    while i!=days:
        if actividadHoy!="Dormir":
            change=np.random.choice(nombreTransicion[0],replace=True,p=MT[0])
            if change=="DD":
                prob=prob*0.2
                listaActividades.append("Dormir")
                pass
            elif change=="DE":
                prob=prob*0.6
                actividadHoy="Estudiar"
                listaActividades.append("Estudiar")
            else:
                prob=prob*0.2
                actividadHoy="Comer"
                listaActividades.append("Comer")
        elif actividadHoy!="Estudiar":
            change=np.random.choice(nombreTransicion[2],replace=True,p=MT[2])
            if change=="EE":
                prob=prob*0.5
                listaActividades.append("Estudiar")
                pass
            elif change=="ED":
                prob=prob*0.2
                actividadHoy="Dormir"
                listaActividades.append("Dormir")
            else:
                prob=prob*0.3
                actividadHoy="Comer"
                listaActividades.append("Comer")

        elif actividadHoy!="Comer":
            change=np.random.choice(nombreTransicion[1],replace=True,p=MT[1])
            if change=="CC":
                prob=prob*0.1
                listaActividades.append("Comer")
                pass
            elif change=="CD":
                prob=prob*0.2
                actividadHoy="Dormir"
                listaActividades.append("Dormir")
            else:
                prob=prob*0.7
                actividadHoy="Estudiar"
                listaActividades.append("Estudiar")
        i+=1
    print("Posibles estados: "+ str(listaActividades))
    print("Estado final después de "+str(days)+"dias: "+ actividadHoy)
    print("Probabilidad de secuencia de estados posible: "+str(prob))
prediccion_actividades(2)

```

    Estado Inicial: Dormir
    Posibles estados: ['Dormir', 'Estudiar', 'Comer']
    Estado final después de 2dias: Comer
    Probabilidad de secuencia de estados posible: 0.15

```python
def prediccion_actividades(days):
    # escoge estado inicial
    actividadHoy = "Dormir"
    listaActividades = [actividadHoy]
    i = 0
    prob = 1
    while i != days:
        if actividadHoy == "Dormir":
            change = np.random.choice(nombreTransicion[0],replace=True,p=MT[0])
            if change == "DD":
                prob = prob * 0.2
                listaActividades.append("Dormir")
                pass
            elif change == "DE":
                prob = prob * 0.6
                actividadHoy = "Estudiar"
                listaActividades.append("Estudiar")
            else:
                prob = prob * 0.2
                actividadHoy = "Comer"
                listaActividades.append("Comer")
        elif actividadHoy == "Estudiar":
            change = np.random.choice(nombreTransicion[1],replace=True,p=MT[1])
            if change == "EE":
                prob = prob * 0.5
                listaActividades.append("Estudiar")
                pass
            elif change == "ED":
                prob = prob * 0.2
                actividadHoy = "Dormir"
                listaActividades.append("Dormir")
            else:
                prob = prob * 0.3
                actividadHoy = "Comer"
                listaActividades.append("Comer")
        elif actividadHoy == "Comer":
            change = np.random.choice(nombreTransicion[2],replace=True,p=MT[2])
            if change == "CC":
                prob = prob * 0.1
                listaActividades.append("Comer")
                pass
            elif change == "CD":
                prob = prob * 0.2
                actividadHoy = "Dormir"
                listaActividades.append("Dormir")
            else:
                prob = prob * 0.7
                actividadHoy = "Estudiar"
                listaActividades.append("Estudiar")
        i += 1
    return listaActividades

# para guardar todo listaActividades
lista_actividades = []
count = 0

for iterations in range(1,10000):
        lista_actividades.append(prediccion_actividades(2))

for listita in lista_actividades:
    if(listita[2] == "Estudiar"):
        count += 1

# Calcula la probabilidad de iniciar en:'Dormir' and ending at state:'Estudiar'
porcentaje = (count/10000) * 100
print("La probabilidad del estado inicial :'Dormir' y el estado final :'Estudiar'= " + str(porcentaje) + "%")
```

    La probabilidad del estado inicial :'Dormir' y el estado final :'Estudiar'= 64.99000000000001%

## Metropolis-Hastings

En estadística Bayesiana se busca estimar un distribución posterior, sin embargo, esto es usualmente intratable debido a la integral en muchas dimensiones del denominador (likelihood marginal). Es posible aproximar la distribución posterior si se puede de alguna manera muestrear la distribución posterior. Con el método de MCMC, es posible obtener muestras de una distribución propuesta de manera que cada una depende únicamente del estado previo, tal como sucede con las cadenas de Markov. Bajo ciertas condiciones, la cadena de Markov tendrá una única distribución estacionaria. En adición, no todas las muestras son usadas, en cambio, se utiliza un criterio de aceptación basado en la comparación sucesiva de estados respecto a una distribución blanco para asegurar que la distribución estacionaria es la distribución a posteriori de interés. Lo interesante de esta distribución es que sólo necesita ser proporcional a la distribución posterior, lo que significa que no se necesita evaluar las likelihoods marginales potencialmente intratables, lo cual es sólo una constante de normalización.

Existen diferentes tipos de MCMC, pero la más sencilla es entender  Metropolis-Hastings, que se basa en un algoritmo de caminata aleatoria.

Para esto, se necesita realizar un muestreo a partir de las distribuciones.

- Uniforme estándar
- La distribución propuesta (eg. N(0,$\sigma$))
- La distribución objetivo, que es proporcional a la probabilidad posterior.

Dado un valor inicial adivinado $\theta$ con probabilidad positiva de ser dibujado, Metropolis procede así:

- Escoje un nuevo valor propuesto tal que $\theta_p =\theta+\Delta\theta$ donde $\Delta\theta \propto N(0,\sigma)$

- Calcula la razón
\begin{equation}
\rho=\frac{g(\theta_p | X) p(\theta|\theta_p)}{g(\theta|X)}
\end{equation}

donde $g$ es la probabilidad posterior.

- Si la distribución propuesta no es simétrica, se necesita pesar la probabilidad de ser aceptada para mantener un balance (reversibilidad) de la distribución estacionaria y en su lugar calcular

\begin{equation}
\rho=\frac{p(X|\theta_p) p(\theta_p)}{p(X|\theta)p(\theta)}
\end{equation}

- Si $\rho \geq 1$ entonces $\theta=\theta_p$

- Si $\rho \leq 1$ entonces $\theta=\theta_p$ con probabilidad $\rho$ de lo contrario $\theta=\theta$ (distribución uniforme)

- Repetir.

# Ejercicio:

1. Definir una función para un modelo a*x**2+b*x+c, que reciba (x,lista) donde lista=[a,b,c]
2. Definir una función de likelihood apropiada.
3. Definir una función prior apropiada para los datos
4. Desarrollar MCMC para ajustar con inferencia bayesiana los parámetros que mejor ajustan los datos con el modelo.

```python
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(123)
x=np.random.rand(25)*7
x_=np.linspace(-3,5,25)

##
PARA CREAR DATOS DE JUGUETE
##

a=-2
b=5
c=2

y=a*x_**2+b*x_+c
y+=x
plt.scatter(x_,y)
plt.show()

```

```python
x_obs=x
y_obs=y

def model(x,a,b,c):
    return a*x**2+b*x+c

def loglikelihood(x_obs,y_obs,_a,_b,_c):
    d=y_obs-model(x_obs,_a,_b,_c)
    d=d/1.0
    d=-0.5*np.sum(d**2)
    return d

def logprior(a,b,c):
    p=-np.inf
    if a<10 and a>-10 and b>-10 and b<10 and c>-10 and c<10:
        p=0.0
    return p
N=10000

lista_a=[np.random.random()]
lista_b=[np.random.random()]
lista_c=[np.random.random()]

logposterior = [loglikelihood(x_obs, y_obs, lista_a[0], lista_b[0], lista_c[0]) + logprior(lista_a[0], lista_b[0], lista_c[0])]

sigma_delta_a=0.5
sigma_delta_b=0.5
sigma_delta_c=0.5

for i in range(1,N):
    propuesta_a=lista_a[i-1]+np.random.normal(loc=0.0,scale=sigma_delta_a)
    propuesta_b=lista_b[i-1]+np.random.normal(loc=0.0,scale=sigma_delta_b)
    propuesta_c=lista_c[i-1]+np.random.normal(loc=0.0,scale=sigma_delta_c)

    logposterior_viejo=loglikelihood(x_obs,y_obs,lista_a[i-1],lista_b[i-1],lista_c[i-1])+logprior(lista_a[i-1],lista_b[i-1],lista_c[i-1])
    logposterior_nuevo=loglikelihood(x_obs,y_obs,propuesta_a,propuesta_b,propuesta_c)+logprior(propuesta_a,propuesta_b,propuesta_c)

    r=min(1,np.exp(logposterior_nuevo-logposterior_viejo))
    alpha=np.random.random()

    if(alpha<r):
        lista_a.append(propuesta_a)
        lista_b.append(propuesta_b)
        lista_c.append(propuesta_c)
        logposterior.append(logposterior_viejo)
    else:
        lista_a.append(lista_a[i-1])
        lista_b.append(lista_b[i-1])
        lista_c.append(lista_c[i-1])
        logposterior.append(logposterior_viejo)

lista_a=np.array(lista_a)
lista_b=np.array(lista_b)
lista_c=np.array(lista_c)
logposterior=np.array(logposterior)

plt.hist(lista_a,color="blue")
plt.hist(lista_b,color="red")
plt.hist(lista_c,color="green")

a_=np.mean(lista_a)
b_=np.mean(lista_b)
c_=np.mean(lista_c)

print(a_)
print(b_)
print(a_)

```

# 2. Árboles de Decision.

Los árboles de decisión (DT) son un método de aprendizaje supervisado no paramétrico utilizado para la clasificación y la regresión. El objetivo es crear un modelo que prediga el valor de una variable objetivo mediante el aprendizaje de reglas de decisión simples inferidas de las características de los datos.

Por ejemplo, en el ejemplo a continuación, los árboles de decisión aprenden de los datos para aproximar una curva sinusoidal con un conjunto de reglas de decisión si-entonces-otro. Cuanto más profundo es el árbol, más complejas son las reglas de decisión y más se ajusta el modelo.

Algunas ventajas de los árboles de decisión son:

- Simple de entender e interpretar. Los árboles pueden ser visualizados.
- Requiere poca preparación de datos. Otras técnicas a menudo requieren la normalización de datos, se deben crear variables ficticias y eliminar los valores en blanco. Sin embargo, tenga en cuenta que este módulo no admite valores faltantes.
- El costo de usar el árbol (es decir, predecir datos) es logarítmico en la cantidad de puntos de datos utilizados para entrenar el árbol.
- Capaz de manejar datos numéricos y categóricos. Otras técnicas suelen estar especializadas en el análisis de conjuntos de datos que tienen un solo tipo de variable. Ver algoritmos para más información.
- Capaz de manejar problemas de salida múltiple.
- Utiliza un modelo de caja blanca. Si una situación dada es observable en un modelo, la explicación de la condición se explica fácilmente por la lógica booleana. Por el contrario, en un modelo de caja negra (por ejemplo, en una red neuronal artificial), los resultados pueden ser más difíciles de interpretar.
- Posible validar un modelo utilizando pruebas estadísticas. Eso permite tener en cuenta la fiabilidad del modelo.
- Funciona bien incluso si sus supuestos son violados de alguna manera por el modelo verdadero a partir del cual se generaron los datos.

Las desventajas de los árboles de decisión incluyen:

- Los alumnos de los árboles de decisión pueden crear árboles demasiado complejos que no generalizan bien los datos. Esto se llama sobreajuste. Para evitar este problema, se necesitan mecanismos tales como la poda (actualmente no compatible), establecer el número mínimo de muestras requeridas en un nodo hoja o establecer la profundidad máxima del árbol.
- Los árboles de decisión pueden ser inestables porque pequeñas variaciones en los datos pueden generar un árbol completamente diferente. Este problema se mitiga mediante el uso de árboles de decisión dentro de un conjunto.
- Se sabe que el problema de aprender un árbol de decisión óptimo es NP-completo bajo varios aspectos de la optimización e incluso para conceptos simples. En consecuencia, los algoritmos prácticos de aprendizaje del árbol de decisiones se basan en algoritmos heurísticos como el algoritmo codicioso donde se toman decisiones localmente óptimas en cada nodo. Tales algoritmos no pueden garantizar la devolución del árbol de decisión globalmente óptimo. Esto puede mitigarse entrenando múltiples árboles en un conjunto de alumnos, donde las características y muestras se muestrean aleatoriamente con reemplazo.
- Hay conceptos que son difíciles de aprender porque los árboles de decisión no los expresan fácilmente, como XOR, problemas de paridad o multiplexor.
- Los alumnos del árbol de decisión crean árboles sesgados si algunas clases dominan. Por lo tanto, se recomienda equilibrar el conjunto de datos antes de ajustarlo con el árbol de decisión.

```python
from sklearn.datasets import load_iris
from sklearn import tree
iris = load_iris()
clf = tree.DecisionTreeClassifier()
clf = clf.fit(iris.data, iris.target)
plt.figure(figsize=(15,15))
tree.plot_tree(clf.fit(iris.data, iris.target))
plt.show()
```

Los árboles de decisión se trabajarán un poco más a profundidad después de introducir ML.

### References:
- The Nature of Statistical Learning Theory. Vladimir N. Vapnik -Springer (2000)
- https://people.duke.edu/~ccc14/sta-663/MCMC.html
- https://scikit-learn.org/stable/modules/tree.html
- https://www.datacamp.com/community/tutorials/markov-chains-python-tutorial

```python

```


```python
import numpy as np, pandas as pd, matplotlib as mpl, matplotlib.pyplot as plt#, seaborn as sns
%matplotlib inline
mpl.style.use('classic')
```

Symbolic math

Definitions

$Y=[α\cdot F^{-\rho}+(1-α)\cdot R^{-\rho} ]^{-1/\rho}$ Arrow 1961

MIT: $Y=[α\cdot F^{\rho}+(1-α)\cdot R^{\rho} ]^{1/\rho}$   http://www.sciencedirect.com/science/article/pii/S0140988306000624?np=y Eq. 1

$\sigma = \frac{1}{1+\rho}$

MIT: $\sigma = \frac{1}{1-\rho}$

```python
from sympy import *
init_printing()
```

```python
a = Symbol('alpha')
s = Symbol('sigma')
r = Symbol('rho')
var('Y F R C c_f c_r');
```

```python
eq=Eq(Y,(a*F**-r+(1-a)*R**-r)**(-1/r))
eq  #main CES equation
    #Y = value of function - analogous to energy demand?
    #R = renewables
    #F = fossils
    #alfa = fossil share
    #rho = factor related to elasticity of substituion
```

```python
R_sol=solve(eq,R)[0]
R_sol #expression of R from the main CES equation
```

```python
eq2=Eq(s,1/(1+r))
eq2 #definition of elasticity of substituion
    #rho = elasticity of substitution
```

```python
r_sol=solve(eq2,r)[0]
r_sol #exression of rho in terms of sigma
```

```python
[RR_sol,rr_sol]=solve([eq,eq2],[R,r])[0]
RR_sol #expression of R in terms of sigma
```

```python
Y_sol=solve(eq,Y)[0]
Y_sol #expression of Y in terms of sigma = main expression
```

```python
YR_sol=solve([eq,eq2],[Y,r])[0][0]
YR_sol #expression of Y in terms of sigma
```

Sample policy calculation replicated from page 60 of http://globalchange.mit.edu/files/document/MITJPSPGC_Rpt125.pdf (EPPA).

```python
s_fix=0.4 #fixed elasticity for demand, EPPA pag. 24
s_fix2=0.5 #fixed elasticity for production, EPPA pag. 17
```

```python
step=1 #step size for numerical calculation
years=np.arange(2015,2100+step,step)
years #years to complete transition (fossil phase-out)
```

    array([2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023, 2024, 2025,
           2026, 2027, 2028, 2029, 2030, 2031, 2032, 2033, 2034, 2035, 2036,
           2037, 2038, 2039, 2040, 2041, 2042, 2043, 2044, 2045, 2046, 2047,
           2048, 2049, 2050, 2051, 2052, 2053, 2054, 2055, 2056, 2057, 2058,
           2059, 2060, 2061, 2062, 2063, 2064, 2065, 2066, 2067, 2068, 2069,
           2070, 2071, 2072, 2073, 2074, 2075, 2076, 2077, 2078, 2079, 2080,
           2081, 2082, 2083, 2084, 2085, 2086, 2087, 2088, 2089, 2090, 2091,
           2092, 2093, 2094, 2095, 2096, 2097, 2098, 2099, 2100])

```python
gamma=0.00 #demand increase rate PER YEAR
Y_zero=100 #initial demand
```

```python
init_F=0.95 #initial fossil share, estimation from EPPA pag. 62, fig. 20
F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
F_fix #array of predefined fossil values (in order complete the transition as defined by years)
```

    array([ 95.        ,  93.88235294,  92.76470588,  91.64705882,
            90.52941176,  89.41176471,  88.29411765,  87.17647059,
            86.05882353,  84.94117647,  83.82352941,  82.70588235,
            81.58823529,  80.47058824,  79.35294118,  78.23529412,
            77.11764706,  76.        ,  74.88235294,  73.76470588,
            72.64705882,  71.52941176,  70.41176471,  69.29411765,
            68.17647059,  67.05882353,  65.94117647,  64.82352941,
            63.70588235,  62.58823529,  61.47058824,  60.35294118,
            59.23529412,  58.11764706,  57.        ,  55.88235294,
            54.76470588,  53.64705882,  52.52941176,  51.41176471,
            50.29411765,  49.17647059,  48.05882353,  46.94117647,
            45.82352941,  44.70588235,  43.58823529,  42.47058824,
            41.35294118,  40.23529412,  39.11764706,  38.        ,
            36.88235294,  35.76470588,  34.64705882,  33.52941176,
            32.41176471,  31.29411765,  30.17647059,  29.05882353,
            27.94117647,  26.82352941,  25.70588235,  24.58823529,
            23.47058824,  22.35294118,  21.23529412,  20.11764706,
            19.        ,  17.88235294,  16.76470588,  15.64705882,
            14.52941176,  13.41176471,  12.29411765,  11.17647059,
            10.05882353,   8.94117647,   7.82352941,   6.70588235,
             5.58823529,   4.47058824,   3.35294118,   2.23529412,
             1.11764706,   0.        ])

```python
#calculate Y_fix_zero through resubstituting initial values into CES
Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix2]])
Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
Y_fix #array of predefined demand values
```

    array([50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000], dtype=object)

```python
Y_sol, YR_sol, eq2, R_sol, RR_sol
```

```python
R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix2],[a,init_F]]) for k in range(len(F_fix))]
plt.scatter(R_fix,F_fix)
plt.xlim(0,200);
```

Carbon tax

```python
eq3=Eq(c_r,c_f/((a/(1-a))*(F/R)**(-1-r)))
eq3 #cost of R compared to F
    #c_f fixed cost of F
```

```python
c_r_sol=solve(eq3,c_r)[0]
c_r_sol #cost of R expressed in terms of rho
```

```python
cR_r_sol=solve([eq3,eq2],[c_r,r])[0][0]
cR_r_sol #cost of R expressed in terms of sigma
```

```python
eq4=Eq(C,c_r*((a/(1-a))*(F/R)**(-1-r))-c_f)
eq4 #carbon tax: relative cost of F compared to R - fixed cost of F
```

```python
C_sol=solve(eq4,C)[0]
C_sol #expression of C in terms of sigma, F, R, a, c_f
```

```python
CR_sol=solve([eq4,eq2],[C,r])[0][0]
CR_sol #expression of C in terms of rho, F, R, a, c_f
```

```python
c_f_fix=1 #cost of fossil, taken 1 to normalize
zeta=0.00 #R cost decrease rate PER YEAR
k=0
c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix2],[a,init_F],[c_f,c_f_fix]])
c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
c_r_fix #array of predefined R cost values, based on cost decrease rate zeta
```

    array([19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000], dtype=object)

```python
C_sol
```

```python
C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix2],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(F_fix))]
plt.scatter(years[:-1],C_fix[:-1],color='r')
plt.ylim(0,1000);
```

```python
def ces(init_F=0.75,s_fix=0.5,zeta=0.0,gamma=0.0,step=1):
    years=np.arange(2015,2100+step,step)
    Y_zero=100 #initial demand
    F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
    Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix]])
    Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
    R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix],[a,init_F]]) for k in range(len(F_fix))]

    R_fix=[i for i in R_fix if np.complex(i).imag==0]
    R_fix=[i for i in R_fix if i>0]

    k=0
    c_f_fix=1 #cost of fossil, taken 1 to normalize
    c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix],[a,init_F],[c_f,c_f_fix]])
    c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
    C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(R_fix))]

    return [R_fix[:-1],F_fix[:len(R_fix)-1],C_fix[:len(R_fix)-1]]
```

```python
fig = plt.subplots(1,3,figsize=(14,4))
lw=2

gs = mpl.gridspec.GridSpec(1, 4,width_ratios=[1.5,0.01,1,1])
bg=['#EEEEEE','none','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(wspace = .1)
ax[1].spines['top'].set_color('none')
ax[1].spines['bottom'].set_color('none')
ax[1].spines['left'].set_color('none')
ax[1].spines['right'].set_color('none')

init_F0= [0.85,0.9,0.95]
s_fix0 = [0.5, 2.0, 1.9]
init_F= [init_F0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
s_fix = [s_fix0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ls0=["--","-",":"]
ss0=["","",""]
ls = [ls0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ss = [ss0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
colors0=["#dd1c77","#2c7fb8","#2ca25f"]
colors=[colors0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ax2=ax[2].twinx()
ax2.set_axisbelow(True)
ax[1].xaxis.set_visible(False)
ax[1].yaxis.set_visible(False)
ax[2].set_yticks([])
ax3=ax[3].twinx()
ax3.set_axisbelow(True)
ax[3].set_yticks([])
ax[0].set_xlim(0,200)
ax[2].set_xlim(2015,2100)
ax[2].set_xticks([2025,2050,2075,2100])
ax[3].set_xlim(2015,2100)
ax[3].set_xticks([2025,2050,2075,2100])
ax[0].set_ylim(0,100)
#ax2.set_ylim(0,1000)
#ax3.set_ylim(0,1000)
ax[0].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax2.grid(color='white', linestyle='solid')
ax3.grid(color='white', linestyle='solid')

for i in range(len(init_F)):

    [x,y,c]=ces(init_F[i],s_fix[i])
    [x2,y2,c2]=ces(init_F[i],s_fix[i],0.04)
    ax[0].plot(x,y,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])

    #for legend
    if i in range(len(s_fix0)):
        ax[0].plot([-10],[-10],'#444444',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
        ax3.plot([-10],[-10],'#2ca25f',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
    if i in {2,5,7}:
        ax[0].plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
        ax2.plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
    if i in {1,4,7}:
        ax2.fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)#,zorder=10)
        ax2.plot(years[:len(c)],c2,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
    if i in {1,4,7}:
        ax2.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
    if i in {6,7,8}:
        ax3.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i],zorder=10)
    if i in {7}:
        ax2.annotate('$\\zeta = 0\%$', xy=(years[48], c[48]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))#,zorder=11)
        ax2.annotate('$\\zeta = 4\%$', xy=(years[43], c2[43]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))#,zorder=11)

ax[0].legend(loc=3,framealpha=0.5,fontsize=11)
ax2.legend(loc=2,framealpha=0.5,fontsize=11)
ax3.legend(loc=2,framealpha=0.5,fontsize=11)
ax[0].set_xlabel(u'Renewable energy $R$')
ax[2].set_xlabel(u'Year')
ax[3].set_xlabel(u'Year')
ax2.set_yticklabels(["","","","","",""])
ax[0].set_ylabel(u'Fossil energy $F$')
ax3.set_ylabel(u'Marginal abatement cost $C$')
ax[0].set_title('$F-R$ curve\n $Y=[\\alpha\cdot \
                F^{-\\rho}+(1-\\alpha)\cdot R^{-\\alpha} ]^{-1/\\rho}, \\sigma = 1/(1+\\rho)$',size=12,y=1.02)
ax[2].set_title('Marginal abatement cost $C$\n $C=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot (F/R)^{-\\rho-1} - c_F,$  $\\zeta:$ $yearly$ $decrease$ $in$ $c_R$',size=12,y=1.02,x=1.1)

plt.suptitle(u'Constant Elasticity (CES) Energy Substitution Model at Constant Energy Demand $Y$',fontsize=13,y=1.12)
plt.savefig('ces2.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
fig = plt.subplots(1,3,figsize=(14,4))
lw=2

gs = mpl.gridspec.GridSpec(1, 4,width_ratios=[1.5,0.01,1,1])
bg=['#EEEEEE','none','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(wspace = .1)
ax[1].spines['top'].set_color('none')
ax[1].spines['bottom'].set_color('none')
ax[1].spines['left'].set_color('none')
ax[1].spines['right'].set_color('none')

init_F0= [0.85,0.9,0.95]
s_fix0 = [0.5, 1.5, 2.5]
init_F= [init_F0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
s_fix = [s_fix0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ls0=["--","-",":"]
ss0=["","",""]
ls = [ls0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ss = [ss0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
colors0=["#dd1c77","#2c7fb8","#2ca25f"]
colors=[colors0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ax2=ax[2].twinx()
ax2.set_axisbelow(True)
ax[1].xaxis.set_visible(False)
ax[1].yaxis.set_visible(False)
ax[2].set_yticks([])
ax3=ax[3].twinx()
ax3.set_axisbelow(True)
ax[3].set_yticks([])
ax[0].set_xlim(0,200)
ax[2].set_xlim(2015,2100)
ax[2].set_xticks([2025,2050,2075,2100])
ax[3].set_xlim(2015,2100)
ax[3].set_xticks([2025,2050,2075,2100])
ax[0].set_ylim(0,100)
ax2.set_ylim(0,1000)
ax3.set_ylim(0,1000)

for i in range(len(init_F)):

    [x,y,c]=ces(init_F[i],s_fix[i],0.0,0.03)
    [x2,y2,c2]=ces(init_F[i],s_fix[i],0.01,0.03)
    ax[0].plot(x,y,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])

    #for legend
    if i in range(len(s_fix0)):
        ax[0].plot([-10],[-10],'#444444',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
        ax3.plot([-10],[-10],'#2ca25f',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
    if i in {2,5,7}:
        ax[0].plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
        ax2.plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
    if i in {1,4,7}:
        ax2.fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)#,zorder=10)
        ax2.plot(years[:len(c)],c2,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
    if i in {1,4,7}:
        ax2.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
    if i in {6,7,8}:
        ax3.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i],zorder=10)
    if i in {4}:
        ax2.annotate('$\\zeta = 0\%$', xy=(years[40], c[40]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))#,zorder=11)
        ax2.annotate('$\\zeta = 1\%$', xy=(years[35], c2[35]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))#,zorder=11)

ax[0].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax2.grid(color='white', linestyle='solid')
ax3.grid(color='white', linestyle='solid')
ax[0].legend(loc=3,framealpha=0.5,fontsize=11)
ax2.legend(loc=2,framealpha=0.5,fontsize=11)
ax3.legend(loc=1,framealpha=0.5,fontsize=11)
ax[0].set_xlabel(u'Renewable energy $R$')
ax[2].set_xlabel(u'Year')
ax[3].set_xlabel(u'Year')
ax2.set_yticklabels(["","","","","",""])
ax[0].set_ylabel(u'Fossil energy $F$')
ax3.set_ylabel(u'Carbon tax $C$')
ax[0].set_title('$F-R$ curve\n $Y=[\\alpha\cdot \
                F^{-\\rho}+(1-\\alpha)\cdot R^{-\\alpha} ]^{-1/\\rho}, \\sigma = 1/(1+\\rho)$',size=12,y=1.02)
ax[2].set_title('Carbon tax $C$\n $C=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot (F/R)^{-\\rho-1} - c_F,$  $\\zeta:$ $yearly$ $decrease$ $in$ $c_R$',size=12,y=1.02,x=1.1)

plt.suptitle(u'Constant Elasticity (CES) Energy Substitution Model at Yearly Energy Demand Increase Rate $\\gamma=0.03$',fontsize=13,y=1.12)
plt.savefig('ces3.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
def sces(init_F=0.75,s_fix=0.5,rf=-1,zeta=0.0,gamma=0.0,step=1):
    #dynamically adjust sigma and resolve equation
    years=np.arange(2015,2100+step,step)
    Y_zero=100 #initial demand
    F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
    Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix]])
    Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
    #init R_fix, C_fix
    R_fix=[0 for k in range(len(F_fix))]
    C_fix=[0 for k in range(len(F_fix))]

    r_fix=solve(eq2,r)[0].subs(s,s_fix)
    r_span=-(rf-r_fix)
    s_fix_k=[]
    r_fix_k=[]

    for k in range(len(F_fix)):
        t=1
        s_fix_k.append(solve(eq2,s)[0].subs(r,r_fix)) #append to sigma array
        r_fix_k.append(r_fix) #append to rho array
        #update sigma every t years
        if (k%t==0):
            r_fix=r_fix-(r_span)/len(years) #update rho

    R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F]]) for k in range(len(F_fix))]

    R_fix=[i for i in R_fix if np.complex(i).imag==0]
    R_fix=[i for i in R_fix if i>0]

    k=0
    c_f_fix=1 #cost of fossil, taken 1 to normalize
    c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix]])
    c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
    C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(R_fix))]

    return [R_fix[:-1],F_fix[:len(R_fix)-1],C_fix[:len(R_fix)-1],s_fix_k[:len(R_fix)-1],r_fix_k[:len(R_fix)-1]]
```

```python
[x,y,c,ys,yr]=sces(0.95,1.5,-1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=sces(0.95,1.5,-1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=sces(0.85,1.5,-1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=sces(0.85,1.5,-1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=sces(0.90,1.5,-1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=sces(0.90,1.5,-1,0.01)
plt.plot(years[:len(c)],c,'--')
```

```python
def dces(init_F=0.75,s_fix=0.5,sf=0.01,zeta=0.0,gamma=0.0,step=1):
    #dynamically adjust sigma and resolve equation
    years=np.arange(2015,2100+step,step)
    Y_zero=100 #initial demand
    F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
    Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix]])
    Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
    #init R_fix, C_fix
    R_fix=[0 for k in range(len(F_fix))]
    C_fix=[0 for k in range(len(F_fix))]

    s_fix_k=[]
    r_fix_k=[]

    for k in range(len(F_fix)):
        t=1
        s_fix_k.append(s_fix) #append to sigma array
        r_fix_k.append(solve(eq2,r)[0].subs(s,s_fix)) #append to rho array
        #update sigma every t years
        if (k%t==0):
            s_fix*=1+sf #update rho

    R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F]]) for k in range(len(F_fix))]

    R_fix=[i for i in R_fix if np.complex(i).imag==0]
    R_fix=[i for i in R_fix if i>0]

    k=0
    c_f_fix=1 #cost of fossil, taken 1 to normalize
    c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix]])
    c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
    C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(R_fix))]

    return [R_fix[:-1],F_fix[:len(R_fix)-1],C_fix[:len(R_fix)-1],s_fix_k[:len(R_fix)-1],r_fix_k[:len(R_fix)-1]]
```

```python
[x,y,c,ys,yr]=dces(0.95,1.5,0.1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=dces(0.95,1.5,0.1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=dces(0.85,1.5,0.1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=dces(0.85,1.5,0.1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=dces(0.90,1.5,0.1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=dces(0.90,1.5,0.1,0.01)
plt.plot(years[:len(c)],c,'--')
```

```python
fig = plt.subplots(2,2,figsize=(10,5))
lw=2

gs = mpl.gridspec.GridSpec(2, 2,height_ratios=[1,0.3])
bg=['#EEEEEE','#EEEEEE','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(hspace = .001)

init_F= [0.85,0.9,0.95]
s_fix=1.5
rf=-1
sf=0.1

colors=["#dd1c77","#2c7fb8","#2ca25f"]

for i in range(len(init_F)):

    [x2,y2,c2,ys,yr]=sces(init_F[i],s_fix,rf,0.01)
    [x,y,c,ys,yr]=sces(init_F[i],s_fix,rf)

    ax[0].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[0].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[0].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[2].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[2].tick_params(axis='y', colors='#993404')
        ax[2].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax2=ax[2].twinx()
        ax2.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax2.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[0].annotate('$\\zeta = 0\%$', xy=(years[45], c[45]),  xycoords='data',
                xytext=(30, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[0].annotate('$\\zeta = 1\%$', xy=(years[48], c2[48]),  xycoords='data',
                xytext=(50, 36), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

    [x2,y2,c2,ys,yr]=dces(init_F[i],s_fix,sf,0.01)
    [x,y,c,ys,yr]=dces(init_F[i],s_fix,sf)

    ax[1].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[1].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[1].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[3].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[3].tick_params(axis='y', colors='#993404')
        ax[3].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax3=ax[3].twinx()
        ax3.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax3.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[1].annotate('$\\zeta = 0\%$', xy=(years[64], c[64]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[1].annotate('$\\zeta = 1\%$', xy=(years[33], c2[33]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

ax[0].grid(color='white', linestyle='solid')
ax[1].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax[0].set_xlim(2015,2100)
ax[1].set_xlim(2015,2100)
ax[0].set_xticks([2015,2050,2075,2100])
ax[1].set_xticks([2015,2050,2075,2100])
ax[2].set_xlim(2015,2100)
ax[3].set_xlim(2015,2100)
ax[2].set_xticks([2015,2050,2075,2100])
ax[3].set_xticks([2015,2050,2075,2100])
ax[0].set_xticklabels([])
ax[1].set_xticklabels([])
ax[0].set_ylim(0,40)
ax[1].set_ylim(0,40)

ax[2].set_ylim(-1,-.4)
ax[3].set_ylim(-1,-.4)
ax[2].set_yticks([-.5,-.7,-.9])
ax[3].set_yticks([-.5,-.7,-.9])
ax2.set_ylim(0,60)
ax3.set_ylim(0,4500)
ax2.set_yticks([0,20,40,60])
ax3.set_yticks([0,1500,3000,4500])

ax[1].set_yticklabels([])
ax[2].legend(loc=3,framealpha=0.5,fontsize=11)
ax[3].legend(loc=3,framealpha=0.5,fontsize=11)
ax[0].legend(loc=2,framealpha=0.5,fontsize=11)
ax[1].legend(loc=2,framealpha=0.5,fontsize=11)
ax[2].set_xlabel(u'Year')
ax[3].set_xlabel(u'Year')
ax[0].set_ylabel(u'Carbon tax $C$')
ax[0].text(0.98,0.98,'linearly decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[0].transAxes,fontsize=11)
ax[1].text(0.98,0.98,'exponentially decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[1].transAxes,fontsize=11)

ax[0].set_title('Carbon tax $C$\n $C=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot [F(\\rho(t))/R(\\rho(t))]^{-\\rho(t)-1} - c_F,$  $\\zeta:$ $yearly$ $decrease$ $in$ $c_R$',size=12,y=1.02,x=1.1)

plt.suptitle(u'Dynamic Elasticity (DES) Energy Substitution Model at Constant Energy Demand $Y$',fontsize=13,y=1.07)
plt.savefig('ces4.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
fig = plt.subplots(2,2,figsize=(10,5))
lw=2

gs = mpl.gridspec.GridSpec(2, 2,height_ratios=[1,0.3])
bg=['#EEEEEE','#EEEEEE','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(hspace = .001)

init_F= [0.85,0.9,0.95]
s_fix=1.5
rf=-1
sf=0.1

colors=["#dd1c77","#2c7fb8","#2ca25f"]

for i in range(len(init_F)):

    [x2,y2,c2,ys,yr]=sces(init_F[i],s_fix,rf,0.01,0.03)
    [x,y,c,ys,yr]=sces(init_F[i],s_fix,rf,0.0,0.03)

    ax[0].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[0].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[0].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[2].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[2].tick_params(axis='y', colors='#993404')
        ax[2].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax2=ax[2].twinx()
        ax2.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax2.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[0].annotate('$\\zeta = 0\%$', xy=(years[45], c[45]),  xycoords='data',
                xytext=(30, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[0].annotate('$\\zeta = 1\%$', xy=(years[48], c2[48]),  xycoords='data',
                xytext=(50, 36), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

    [x2,y2,c2,ys,yr]=dces(init_F[i],s_fix,sf,0.01,0.03)
    [x,y,c,ys,yr]=dces(init_F[i],s_fix,sf,0.0,0.03)

    ax[1].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[1].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[1].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[3].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[3].tick_params(axis='y', colors='#993404')
        ax[3].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax3=ax[3].twinx()
        ax3.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax3.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[1].annotate('$\\zeta = 0\%$', xy=(years[64], c[64]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[1].annotate('$\\zeta = 1\%$', xy=(years[33], c2[33]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

ax[0].grid(color='white', linestyle='solid')
ax[1].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax[0].set_xlim(2015,2100)
ax[1].set_xlim(2015,2100)
ax[0].set_xticks([2015,2050,2075,2100])
ax[1].set_xticks([2015,2050,2075,2100])
ax[2].set_xlim(2015,2100)
ax[3].set_xlim(2015,2100)
ax[2].set_xticks([2015,2050,2075,2100])
ax[3].set_xticks([2015,2050,2075,2100])
ax[0].set_xticklabels([])
ax[1].set_xticklabels([])
ax[0].set_ylim(0,80)
ax[1].set_ylim(0,80)

ax[2].set_ylim(-1,-.4)
ax[3].set_ylim(-1,-.4)
ax[2].set_yticks([-.5,-.7,-.9])
ax[3].set_yticks([-.5,-.7,-.9])
ax2.set_ylim(0,60)
ax3.set_ylim(0,4500)
ax2.set_yticks([0,20,40,60])
ax3.set_yticks([0,1500,3000,4500])

ax[1].set_yticklabels([])
ax[2].legend(loc=3,framealpha=0.5,fontsize=11)
ax[3].legend(loc=3,framealpha=0.5,fontsize=11)
ax[0].legend(loc=2,framealpha=0.5,fontsize=11)
ax[1].legend(loc=2,framealpha=0.5,fontsize=11)
ax[2].set_xlabel(u'Year')
ax[3].set_xlabel(u'Year')
ax[0].set_ylabel(u'Carbon tax $C$')
ax[0].text(0.98,0.98,'linearly decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[0].transAxes,fontsize=11)
ax[1].text(0.98,0.98,'exponentially decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[1].transAxes,fontsize=11)

ax[0].set_title('Carbon tax $C$\n $C=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot [F(\\rho(t))/R(\\rho(t))]^{-\\rho(t)-1} - c_F,$  $\\zeta:$ $yearly$ $decrease$ $in$ $c_R$',size=12,y=1.02,x=1.1)

plt.suptitle(u'Dynamic Elasticity (DES) Energy Substitution Model at Yearly Energy Demand Increase Rate $\\gamma=0.03$',fontsize=13,y=1.07)
plt.savefig('ces5.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```


# Stochastic differential equations

Suggested references:

C Gardiner, *Stochastic Methods: A Handbook for the Natural and Social Sciences*, Springer.

D Higham, *An algorithmic introduction to Stochastic Differential Equations*, SIAM Review.

## Kostas Zygalakis

Statement of problem: given function $g \left( X^{(1)}, X^{(2)}, \dots, X^{(n)}, X, s \right) = 0$ defining an $n$-th order differential equation

$$
\begin{equation}
  \frac{dX}{ds} = f \left( X(s) \right)
\end{equation}
$$

where we refer to $f$ as the *drift function* for reasons that will become clear later.

The formal solution to this equation is

$$
\begin{equation}
  X(t) = X_0 + \int_0^t f \left( X(s) \right) \, ds.
\end{equation}
$$

Note a simple example. Given

$$
\begin{equation}
  \frac{dX}{ds} = a X(s) \quad \implies \quad X(s) = X_0 e^{a s} \rightarrow_{s \rightarrow + \infty}
  \begin{cases} \infty & a > 0 \\ 0 & a < 0. \end{cases}
\end{equation}
$$

##### Exercise

$$
\begin{equation}
  \frac{dX}{ds} = f(x), \quad f(x) = \sqrt{x}, \quad X_0 = 0.
\end{equation}
$$

Solve this and think about *how many* solutions may exist.

## Motivating numerical solutions

Think about the *integral form* of the differential equation

$$
\begin{equation}
  X(t + h) - X(t) = \int_t^{t+h} f \left( X(s) \right) \, ds \approx \begin{cases} f \left( X(t) \right) \, h & \\ f \left( X(t+h) \right) \, h & \end{cases}.
\end{equation}
$$

**Picture here**

This gives two numerical methods:

$$
\begin{align}
  X(t+h) - X(t) & \approx f \left( X(t) \right) \, h && \leftrightarrow & X_{n+1} - X_n & = f(X_n) \, h & \text{Explicit Euler}, \\
  X(t+h) - X(t) & \approx f \left( X(t+h) \right) \, h && \leftrightarrow & X_{n+1} - X_n & = f(X_{n+1}) \, h & \text{Implicit Euler}.
\end{align}
$$

Both these methods are *first order* in the sense that, given an *approximate* solution $X^h(t)$ and the *exact* solution $X(t)$, we have that

$$
\begin{equation}
  | X(T) - X^h(T) | \le C(T) h
\end{equation}
$$

where $C(T)$ is a constant depending only on the time at which this is evaluated.

### Key example

Apply the explicit Euler method to the simple problem

$$
\begin{equation}
  \frac{dX}{dt} = -a x, \qquad a \gg 1.
\end{equation}
$$

This is a *stiff* problem.

The algorithm becomes

$$
\begin{align}
  && X_{n+1} - X_n & = - a X_n h \\
  \implies && X_{n+1} & = X_n \left( 1 - a h \right).
\end{align}
$$

This will only *converge* to the correct answer (0!) when  $|1 - a h| < 1$ which implies $h < 2 / a$ (as $h > 0$). When $a \gg 1$ this is a severe restriction.

###### Exercise

Compute the step size restriction for the *implicit* Euler method.

## Modelling with Stochastic Differential Equations.

What are we trying to do? Consider the motivating example

$$
\begin{equation}
  \frac{dX}{ds} = \left( a + \text{"error"} \right) X(s), \quad X(0) = X_0.
\end{equation}
$$

The "error" should average to zero.

More formal definition is

$$
\begin{equation}
  X(t, \omega) = X_0 + \int_0^t f \left( X(s, \omega) \right) \, ds + \underbrace{\sigma B \left( t, \omega \right)}_{= \sigma \int_0^t dB(\omega, s)}.
\end{equation}
$$

Each time I repeat the experiment I get a *different answer*. The first part is "deterministic" and looks like the ODE; the latter part is new.

Now, we list some assumptions of varying strengths.

**Assumption 1**. It should be clear that $B(0, \omega) = 0$ (no "error" accumulation at the initial time).

We can also look at two *close* points in time to see that

$$
\begin{equation}
  X(t+h, \omega) - X(t, \omega) = \int_t^{t+h} f\left( X(s, \omega) \right) \, ds + \sigma \left( B(t+h, \omega) - B(t, \omega) \right).
\end{equation}
$$

**Assumption 2**. Given our discussion we assume

$$
  \mathbb{E} \left( B(t+h, \omega) - B(t, \omega) \right) = 0.
$$

We now drop $\omega$ from the notation to make explicit

**Assumption 3**. An independence assumption:

$$
\begin{equation}
  B(t_2) - B(t_1) \underbrace{\perp}_{\text{independent}} B(s_2) - B(s_1) \quad \text{for all times } s_1 < s_2 < t_1 < t_2.
\end{equation}
$$

**Assumption 4**. Final independence assumption: $B(t+h) - B(t)$ depends only on $|h|$ and not on $t$.

The last two assumptions (assumptions 3 and 4) suggest that this is a "pure" error term - that it has no predictable structure.

## Brownian motion

The error term, or *Brownian motion*, or *Wiener process* we will define satisfies:

1. $B(0) = 0$.
2. $B(t) - B(s) \sim N(0, |t - s|)$.
3. $\text{Cov} \left( B(t), B(s) \right) = \mathbb{E}((B(t) - \mathbb{E}(B(t))) (B(s) - \mathbb{E}(B(s))))) = \min \{ s, t \}$.

## Stochastic integral

We write this as

$$
\begin{equation}
  \int_0^t G(s) \, dB(s).
\end{equation}
$$

This is slightly odd as we're integrating against a random process. What we're aiming for is a way of solving (in the *integral form*) the SDE

$$
\begin{equation}
  \frac{dX}{ds} = f(X(s)) + g(X(s)) \frac{dB}{ds}
\end{equation}
$$

## Generating the Brownian motion

Split the interval $[0, T]$ into segments length $h$. On each segment we want the *increment* to be a normally distributed random variable, $N(0, h)$.

```
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
```

```
T = 1.0
h = 0.01
N = int(T/h)

t = np.linspace(0.0, T, N)
dW = np.sqrt(h) * np.random.randn(N)
W = np.cumsum(dW) - dW[0]
```

```
plt.plot(t, W)
plt.xlabel(r"$t$")
plt.ylabel(r"$B(t)$");
```

Rerun this to see that it changes. Rerun again with a smaller $h$. You'll see it gets "spikier". In the limit as $h \to 0$ the function is *not* differentiable.

#### Generating more paths

```
M = 5
dW = np.sqrt(h) * np.random.randn(N, M)
W = np.cumsum(dW, axis = 0) - dW[0, :]
plt.plot(t, W)
plt.xlabel(r"$t$")
plt.ylabel(r"$B(t)$");
```

## Algorithms for integrals

Try to solve

$$
\begin{equation}
  \int_{0}^T B(s) \, dB(s) \ne \frac{1}{2} \left( B^2(T) - B^2(0) \right)
\end{equation}
$$

which would be the "normal" calculus result.

Instead do the "natural" (or naive) partial sum thing:

$$
\begin{equation}
  S_n = \sum_{i=1}^n B(\tau_i) \left( B(t_i) - B(t_{i-1} \right), \qquad t_{i-1} \le \tau_i \le t_i
\end{equation}
$$

This is a *random variable*, as $B$ is a random variable. So this has a mean, variance, higher moments...

This means that, depending on how you choose your $\tau_i$, you will get **different** answers for the **exact** solution!

$$
\begin{align}
  < S_n > & = < \sum_{i=1}^n B(\tau_i) [ B(t_i) - B(t_{i-1})] > \\
  & = \sum_{i=1}^n \left[ \min(\tau_i, t_i) - \min(\tau_i, t_{i-1}) \right] \\
  & = \sum_{i=1}^n (\tau_i - t_{i-1}) \\
  & = ( t - t_0 ) a,
\end{align}
$$

where $a$ is the relative distance between $t_{i-1}$ and $t_i$ at which $\tau_i$ is located, using the earlier property $< B(t), B(s) > = \min\{t, s\}$.

The Ito integral uses $a = 0$ so that $\tau_i = t_{i-1}$. This implies $< S_n > = 0$.

## Using this to solve SDEs

We are trying to solve

$$
\begin{equation}
  X(t) = X_0 + \int_0^t f(X(s)) \, ds + \int_0^t g(X(s)) \, dB(s).
\end{equation}
$$

The simplest thing to do is to use the Ito integral applied to the final term. By our independence assumptions, as the expectation of the Brownian path is zero, the expectation of the final integral (our "error term") is zero, as we want.

So the equivalent of the explicit Euler method is

$$
\begin{equation}
  X(t+h) - X(t) = h f(X(t)) + g(X(t)) ( B(t+h) - B(t) ).
\end{equation}
$$

The increment in the last part is random, as it's a random variable!

FEEG6016 Simulation and Modelling/2014/SDEs Lecture 1.ipynb


This notebook covers the basics of operations on Series and DataFrames. Again, a quick check on whether you know the material.

1. Can you use numpy ufuncs (e.g. `np.exp`) on DataFrames?
2. What happens when you add two DataFrames with different indicies?

If you know the answers already, you can skip to our first [application](#Application:-Economic-Timeseries). Maybe download some additional datasets from [FRED](https://research.stlouisfed.org) and play with those; try to figure out how to fix the economy. You've got about 5 minutes.

`DataFrame`s and `Series` support all the usual math operations. Additionally,
numpy [`ufuncs`](http://docs.scipy.org/doc/numpy/reference/ufuncs.html) (universial function, e.g. `np.log`) can  be used as expected.

```python
import numpy as np
import pandas as pd

from ph2t import side_by_side  # came with the notebooks
```

# `ufuncs`

`ufunc`s operate on arrays elementwise.
They can be used just like you'd expect on a `Series` or `DataFrame`.

```python
np.random.seed(42)
df = pd.DataFrame(np.random.uniform(0, 10, size=(3, 3)))
df
```

```python
df + 1
```

```python
df ** 2
```

```python
np.log(df)
```

For any decent-sized `DataFrame`, using a `ufunc` will be much faster than applying the function to each row in a for loop or by using `DataFrame.apply`.

# Alignment

When you do binary operations between two `numpy` arrays, like `+`, the arrays are in some sense aligned by position.
It's up to you to ensure that the correct values are in the same position across arrays.

In pandas, we rely on row and column *labels* to do the alignment for us.
Pandas automatically aligns by label when doing operations between `DataFrames` and `Series`.

```python
df1 = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
df2 = pd.DataFrame({'A': [2, 4, 6], 'B': [1, 4, 9]})

side_by_side(df1, df2, "df1", "df2")
```

```python
df1 + df2
```

```python
# Note the index order
df2 = pd.DataFrame({'A': [6, 2, 4], 'B': [9, 1, 4]}, index=[2, 0, 1])
side_by_side(df1, df2, 'df1', 'df2')
```

```python
df1 + df2
```

```python
# Different index entirely
df3 = pd.DataFrame(np.random.randn(5, 3), columns=['A', 'B', 'C'])
side_by_side(df1, df3, 'df1', 'df3')
```

```python
df1 + df3
```

# Recap

- `ufuncs` work just fine
- Operations **align by label** and then are performed

# Application: Economic Timeseries

```python
from itertools import chain

import pandas as pd

import seaborn as sns
import statsmodels.api as sm
from pandas_datareader import data

import matplotlib.pyplot as plt
from toolz import partitionby

%matplotlib inline
pd.options.display.max_rows = 10
sns.set_style('ticks')
```

Pandas recently split off some web-based data reading functionality into its own package `pandas-datareader`. Earlier, I used it to grab data from [FRED](http://research.stlouisfed.org).

```python
gdp = data.DataReader("GDP", data_source='fred',
                      start='1929', end='2014').squeeze()
cpi = data.DataReader("CPIAUCSL", data_source='fred',
                      start='1947-01', end='2015-05').squeeze()
rec = data.DataReader('USREC', data_source='fred',
                      start='1854-12-01',
                      end='2014-08-01').squeeze()
gdp.to_csv('data/gdp.csv', header=True)
cpi.to_csv('data/cpi.csv', header=True)
rec.to_csv('data/rec.csv', header=True)
```

```python
with open('data/gdp.csv') as f:
    print(''.join(f.readlines(100)))
```

```python
pd.read_csv?
```

```python
gdp = pd.read_csv('data/gdp.csv', index_col='DATE', parse_dates=['DATE']).squeeze()
cpi = pd.read_csv('data/cpi.csv', index_col='DATE', parse_dates=['DATE']).squeeze()
rec = pd.read_csv('data/rec.csv', index_col='DATE', parse_dates=['DATE']).squeeze()
```

```python
gdp.head(n=5)
```

```python
gdp.tail()
```

```python
cpi.head()
```

```python
rec.head()
```

## Methods

DataFrames and Series have a bunch of useful aggregation methods, `.mean`, `.max`, `.std`, etc.

```python
gdp.mean()
```

## Plotting

DataFrames and Series can make decent looking plots with a few lines of code.

I wanted to incude bars for recession indicators.
The only pandas-relavent bit is `Series.iteritems`, which you *almost* never want to use since it's relatively slow.

```python
next(rec.iteritems())
```

```python
groups = partitionby(lambda x: x[1] == 1, rec.iteritems())
recessions = filter(lambda x: x[0][1] == 1, groups)
spans = [(months[0][0], months[-1][0]) for months in recessions]
```

```python
def add_rec_bars(ax=None):
    ax = ax or plt.gca()
    ylim = ax.get_ylim()
    for span in spans:
        ax.fill_between(span, *ax.get_ylim(), color='k', alpha=.25)
    ax.set_ylim(*ylim)
    return ax

sns.set(style="white", context="talk")
plt.rcParams['figure.figsize'] = (12, 6)
```

```python
ax = gdp.plot(title='GDP', linewidth=3)
add_rec_bars(ax)
sns.despine()
```

Let's put some of those operations to use.

    EXERCISE: Convert CPI to base 2009

CPI is the Consumer Price Index. The Index part just means that it doesn't really have any units,
just some (arbitrary) time span that is set to 100, and every other observation is relative to that time.
The CPI we have is indexed to the average over 1982-1984.

**Convert the CPI base-2009 by dividing the entire Series `cpi` by the average CPI in 2009.**

Try breaking the problem into pieces:
1. Select just the rows from 2009 (`.loc`)
  + Timeseries (dates in the index) have [speical rules](http://pandas.pydata.org/pandas-docs/stable/timeseries.html#datetimeindex-partial-string-indexing) for slicing. Pass in a string with the subset you want
    + `.loc['2010-01-01']` select all observations from that day
    + `.loc['2010-01']` selects all observations from that month
2. calculate the average on those rows
3. divide `cpi` by 100 * that number (assign that to `cpi09`)

```python
# Your code goes here
# Assign the result to cpi09
```

```python
%load solutions/solutions_operations.py
```

```python
ax = cpi09.plot(title='CPI-2009')
add_rec_bars(ax=ax)
sns.despine();
```

Next, try calculating real GDP, with 2009 as the base year.
This is defined to be `gdp` divided by the base-2009 CPI for that time period.

Call the result `rgdp` and try plotting it.

```python
# Your code here
rgdp = ...
```

If you have trouble with the plotting, look into the `.fillna` and `.dropna` methods.
The next section explains what's going on.

If you finish this exercise early, check out the "Extra Exercises" section at the bottom of the notebook.

# Real GDP: Alignment

Let's say we want real GDP (adjusted for inflation).

\begin{equation}
    rGDP_t = \frac{GDP_t}{CPI_t}
\end{equation}

Problem: our CPI is monthly but GDP is quarterly. Also the two Series have different start and end points. If you didn't have automatic label alignment, you'd have to jump through hoops to select the correct subset of each series.

```python
# this is boolean indexing, we'll see more later
gdp / cpi09[(cpi.index.month % 3 == 1) & (cpi09.index.year <= 2014)]
```

... but that's unneccesary. The operations will automatically align for you.

```python
rgdp = (gdp / cpi09)
rgdp
```

The `NaN`s are missing value indicators. `NaN`s can crop up for many reasons, but in this case it's because the labels didn't overlap perfectly.

Binary operations (addition, multiplication, ...) consists of two steps.

1. align by label
2. operation

```python
l, r = gdp.align(cpi09)
side_by_side(l.head(), r.head())

```

While binary operations *propogate* missing values, aggregations like `sum` or `mean`, will ignore them.
Pandas provides methods for detecting missing data `.isnull`, filling missing data `.fillna`,
or you can dropping it, which is what I'll do here.

```python
rgdp.dropna()
```

Pandas methods are non-mutating by default. This means that even though I called `.dropna()` above, `rgdp` still has missing values.

```python
rgdp
```

To capture the change, assign a name to the result. In this case, I just the same name `rgdp`.

```python
real_gdp = rgdp.dropna()
real_gdp.head()
```

```python
ax = gdp.pct_change().plot(
    figsize=(15, 5), label='GDP', legend=True, linewidth=2
)
real_gdp.pct_change().plot(
    ax=ax, label='real GDP', legend=True, linewidth=2
)
add_rec_bars(ax)
sns.despine()
```

# Recap

DataFrames allow you to rapidly explore data, iterating on ideas.

- `ufunc`s work as expected
- operations between pandas objects automatically align on labels

    Extra Exercises

Explore [FRED](https://fred.stlouisfed.org) for additional datasets.

See if you can produce an updated version of this graph from [*Calculated Risk*](Calculated Risk: BLS: Jobs Openings at 5.0 million in December, Up 28% Year-over-year).

```python
from IPython.display import Image
Image(url='http://1.bp.blogspot.com/-Q60opady2iQ/VNoeGepLMWI/AAAAAAAAiQc/PCA1LfFJFCg/s1600/JOLTSDec2014.PNG')
```

This shows the components of turnover in the labor market. People leave jobs for two reasons: 1. Quits (light blue) or 2. Layoffs, discharges, or other (red).

Companies post job openings (yellow) and fill some number of those (Blue).

The difference between the openings and hires represents the change in the stock of open positions, and measures the slackness of the labor market.

The difference between the Hires and the sum of Quites and Layoffs / Discharges represents the change in the employed labor force (though the unemployment rate is calculated differently).

To get you started, the FRED series codes are

- [JTSJOL](https://fred.stlouisfed.org/series/JTSJOL): Openings
- [JTSQUL](https://fred.stlouisfed.org/series/JTSQUL): Quits
- [JTSHIL](https://fred.stlouisfed.org/series/JTSHIL): Hires
- [JTSLDL](https://fred.stlouisfed.org/series/JTSLDL): Layoffs

Use the `pandas_datareader.data.DataReader` class, which accepts a list of symbols, and a `data_source` ('`fred'`).

As a hint, you can use pandas' `.plot.area()` method for the Quits and Layoffs series. That handles all the stacking for you.

```python
series = ['JTSJOL', 'JTSQUL', 'JTSHIL', 'JTSLDL']
names = ['openings', 'quits', 'hires', 'layoffs']
from pandas_datareader.data import DataReader
```

```python
res = DataReader(series, start='2000-01-01', data_source="fred")
```

```python
res = res.rename(columns=dict(zip(series, names)))
```

```python
ax = res[['quits', 'layoffs']].plot.area()
res[['hires', 'openings']].plot(ax=ax)
```

```python
k
```