 _Por: V. Robles B._

# Descenso por Gradiente: una breve introducción - Parte 1

El descenso por gradiente, es un método que nos permite encontrar el mínimo local de una función dada. Este método parte de la selección de un valor inicial aleatorio y se toma el gradiente de dicha función en ese punto. Luego de elllo, se dan un paso hacia la dirección negativa del gradiente. Se repite el proceso hasta que el algoritmo converge cuando el valor del gradiente es **0** (lo que indica que hemos llegado al mínimo local) [1,2,3].

## Descripción del algoritmo

Los elementos más importantes de este algoritmo son los siguientes [3]:

* ** Dirección y tamaño del paso: ** el primer elemento nos indica en qué dirección deberemos movernos para seguir buscando minimizar la función. El tamaño del paso especifica qué tanto nos movemos en la dirección dada. Con ello, partiremos de un punto arbritario $x^{(0)}$ y entonces en cada paso $k\geq 0$ nos movemos de forma iterativa en la dirección $\Delta x^{(k)}$ con un paso $t_{k}$ al siguiente punto $x^{(k+1)} = x^{(k)}+t_{k} \cdot \Delta x^{(k)}$.  En este método es importante considerar que nos movemos siempre en la dirección negativa (minimizar), por lo que tomamos el valor negativo del gradiente: $\Delta x= - \nabla f(x)$. Con ello, la búsqueda iterativa del descenso por gradiente se puede describir a través de la siguiente fórmula recursiva:

$x^{(k+1)}=x^{k}-t_{k} \nabla f(x^{(k)})$

* ** Tamaño del paso: ** dado que deseamos minimizar la función, debemos escoger un valor que nos permite cumplir con la premisa de minimizar $f(x^{(k+1)})$. Dado que $x^{(k+1)}=x^{(k)}-t \Delta f(x^{(k)})$, el tamaño del paso $t^{*}_{k}$ es una aproximación de:

 $t^{*}_{k} = arg\,\min\limits_{t\geq 0} f(x^{(k)}-t \nabla f(x^{(k)}))$

* **Algoritm:** a continuación se detallan los pasos más importantes del algoritmo y se presenta un presenta un pseudo-código que describe cómo aplicarlo en base a una precisión $\epsilon \geq 0$ deseada[3]:

## Aplicaciones:

Algunas de las diversas aplicaciones del método de descenso por gradiente son las que se señalan a continuación [2]:

1. En el ámbito de la ingeniería, por ejemplo, minimmizar el consumo de combustible en un automóvil.
2. En la economía, un caso concreto es la maximización de del retorno sobre la inversión.
3. Logística en la cadena de suministros, minimizar el tiempo requerido para despachar una orden.

## Ejemplo de aplicación:

En el siguiente ejemplo vamos a ver cómo aplicar el algoritmo de descenso por gradiente para minimizar esta sencilla función:

$y=x^{2}-2x+2$

Para ello, graficamos la función, a fin de tener una idea más clara del lugar donde estará el punto que minimiza nuestra función.

```python
import matplotlib.pyplot as pp
import numpy as np

def fx(x):
    return np.power(x,2)-np.dot(x,2.)+2.

x=np.arange(-7,9.5,0.5)
y=fx(x)

pp.plot(x,y,c='red')
pp.grid(True)
pp.show()
a = 23
```

### Usando cálculo:

Si recordamos **cálculo matemático** (ver este [enlace](http://tutorial.math.lamar.edu/Classes/CalcI/Optimization.aspx) si se desea repasar el concepto), para minimizar la función anterior:

$\hat{f}(x)= \min\limits_{x} x^{2}-2x+2$

Simplemente derivamos dicha función con respecto a $x$ y despejamos $x$:

$\frac{df}{dx} = 2x-2$

$2x=2$

$x=1$

Con ello, podemos observar dónde se ubica dicho valor $x=1$

```python
pp.plot(x,y)
pp.plot([1],[1.-2.+2.],'ro')
pp.grid(True)
pp.show()
```

### Usando el algoritmo de Descenso por Gradiente [3]:

Dado que no conocemos el valor óptimo de $x$, seleccionamos un valor aleatorio: $x=3$ y hacemos $k\leftarrow 0$, como nos indica el algoritmo. Si evaluamos la función original en este punto, observaremos que se obtiene el sigiuente valor: $f(3)=3^{2}-2\cdot 3+2 = 5$

Calculamos la derivada de la función $y=x^{2}-2x+2$ (una sola vez, ya que se usará en los siguientes pasos):

$\frac{df}{dx} = 2x-2$

1. A fin de encontrar el siguiente punto $x^{(k+1)}$ debemos calcular:

$x^{(1)}=x^{(0)}-t^{*}_{0} \nabla f(x^{(0)})$

\_
2. Para encontrar el valor de $t^{*}_{0}$, debemos hallar el mínimo de la función $\theta(t) = f(x^{(0)})-t \nabla f(x^{(0)})$. Para realizar este paso, buscamos en el punto estacionario trabajando con la derivada de la función a minimizar:

  - Como primer paso calculamos el valor de la derivada de la función a minimizar en el punto $x^{(0)}$:

$f^{'}(x^{(0)}) = 2\cdot x^{(0)}-2$

$f^{'}(3) = 2\cdot 3-2$

$f^{'}(3) = 4$

\_
  Este punto representará nuestro valor $\nabla f\left( x^{(0)} \right)$
  - Luego de ello, calculamos el valor de $t^{*}_{k}$ empleando el punto $x^{(0)}$, y la función de $t$ y en base a la derivada de la función a minimizar:

$\theta^{'}(t)=-\nabla f\left( x^{(0)} - t \nabla f(x^{(0)} \right) \nabla f(x^{(0)})$

$=-\nabla f\left( \mathbf(3)-t \nabla f(x^{(0)}) \right) \nabla f(x^{(0)})$

$=-\nabla f\left( 3-t \nabla f(3) \right) \nabla f(x^{(0)})$

$=-\nabla f\left( 3-t\cdot (4) \right) \nabla f(x^{(0)})$

$=-\nabla f\left( 3-4t \right) \nabla f(x^{(0)})$

\_
   En este punto, sustituimos $4-4t$ por $x$ en la función a minimizar que derivamos previamente $f^{'}(x)=2x-2$ y sustituimos $\nabla f(x^{(0)})$ por el valor que calculamos de la derivada $(4):$

$=-\left( 2\cdot(3-4t)-2 \right)\nabla f(x^{(0)})$

$=-\left( 2\cdot(3-4t)-2 \right)\cdot (4)$

$=-\left( 6-8t-2 \right)\cdot (4)$

$=-\left( 4-8t \right)\cdot (4)$

$=-16+32t$

\_
   En este caso, $\theta^{'}(t)=0$ sí y solo sí $t=0.5$. Con ello, $t_{0}=0.5$
3. Dado lo anterior, el siguiente punto $x^{(1)}$ será:

$x^{(1)}=x^{(0)}-t^{*}_{0} \nabla f(x^{(0)})$

$x^{(1)}=3-0.5 (4)$

$x^{(1)}=1$

Si ahora evaluamos la función original con el nuevo punto $x=1$, obtenemos un valor menor, es decir, se ha minimizado la función: $f(1)=1^{2}-2\cdot 1+2 = 1$

Deberemos seguir repitiendo los pasos, hasta que el error obtenido sea menor al planteado en $\epsilon$.

## Ejemplo resuelto usando [sympy](http://www.sympy.org/en/index.html)

En esta sección se presenta un ejemplo de cálculo completo con una nueva función, y para ello se realizarán varias etapas a fin de buscar minimizarla:

$\frac{1}{2.3}x^{3}-3x+x^{2}-12.37$

El valor de punto de partida será $x^{(0)}=5.7$

Graficamos la función que se minimizará y el punto de partida:

```python
def fx2(x2):
    return (1./2.3)*np.power(x2,3.)-3*x2+np.power(x2,2)-12.37

x2=np.arange(-4,6,0.27)
y2=fx2(x2)
pp.plot(x2,y2,'r')
pp.plot(5.7,fx2(5.7),'bo')
pp.grid(True)
pp.show()
```

#### Resolución:

* Calculamos la derivada de la función y evaluamos su valor en el punto $x^{(0)}$:

```python
from sympy import Symbol, Function, diff, solve

x=Symbol('x')
f=Function('f')(x)
x0=5.7

fx=(x**3)/2.3-3*x+x**2-12.37

r=diff(fx)
print("Derivada de la funcion: ",r)
print("Valor en x^(0):",r.subs(x,x0).evalf())

```

    Derivada de la funcion:  1.30434782608696*x**2 + 2*x - 3
    Valor en x^(0): 50.7782608695652

Lo que nos da como resultado lo siguiente:

$f^{'}(x) = 1.3x^{2}+2x-3$

$\nabla f(x^{(0)}) = 50.77$

* A fin de de encontrar el valor de $t^{*}_{0}$ requerido para calcular el siguiente punto $x^{(1)}$, calculamos el valor de la siguiente función:

     $\theta(t) = f(x^{(0)})-t \nabla f(x^{(0)})$.

$\theta^{'}(t)=-\nabla f\left( x^{(0)} - t \nabla f(x^{(0)} \right) \nabla f(x^{(0)})$

$=-\nabla f\left( \mathbf(5.7)-t \nabla f(x^{0}) \right) \nabla f(x^{(0)})$

$=-\nabla f\left( \mathbf(5.7)-50.77\cdot t \right) 50.77$

 \_
   En este punto, sustituimos $(5.7-50.77\cdot t)$ por $x$ en la función a minimizar que derivamos previamente $f^{'}(x) = 1.3x^{2}+2x-3$:

$=-\left( 1.3\cdot(5.7-50.77t)^{2}+2 \cdot (5.7-50.77t)-3 \right)\cdot 50.77$

   Aplicamos factoreo a la ecuación y calculamos las raíces:

```python
# Ingresamos la funcion en sympy y buscamos las raices:
fp=-((1.3*(5.7-50.77*x)**2+2*(5.7-50.77*x)-3))
print(fp.expand())

print("Raices de la ecuacion:" ,solve(fp))
```

    -3350.87077*x**2 + 853.9514*x - 50.637
    Raices de la ecuacion: [0.0938835114298592, 0.160961112136450]

   No realizamos la multiplicación por $50.77$ ya que luego se igualará a $0$, con ello tenemos lo siguiente:

$=-\left( -17023.7x^{2}+43355.1x-2570.84 \right)\cdot 50.77$

   Y las raíces de la ecuación representarán 2 posibles valores para $t^{*}_{0}$:

$\left[ 0.0938, 0.16 \right]$

   Con ello, calculamos los dos posibles puntos $x^{(1)}$ y determinamos con cuál de los 2 se minimiza de mejor forma la función:

   $
       \begin{split}
           x^{(1)}=\begin{cases}
               5.7-0.0938\cdot 50.77~ = & \mathbf{1.003}\\
               5.7-0.16\cdot 50.77~ = & \mathbf{-2.4}
           \end{cases}
       \end{split}
   $

   De igual forma, evaluamos la función original con los dos nuevos puntos $x^{(1)}$ y en base a ello determinamos cuál es el que minimiza de mejor manera su valor:

```python
v1=fx.subs(x,1.0035341247060474).evalf()
v2=fx.subs(x,-2.4019956631675683).evalf()

print("Valores con de la funcion: [%d, %d]" % (v1,v2))
```

    Valores con de la funcion: [-13, -5]

   $
       \begin{split}
           f{(x)}=\begin{cases}
               f(1.003)~ = & \mathbf{-13}\\
               f(-2.4)~ = & \mathbf{-5}
           \end{cases}
       \end{split}
   $

   Con ello, determinamos que la mejor alternativa es la de la segunda raíz para calcular el punto $x^{(1)}=1.003$ y hacemos $k \leftarrow k+1$ .

```python
pp.plot(x2,y2,'r')
pp.plot(1.003,fx2(1.003),'bo')
pp.grid(True)
pp.show()
```

## Boletín de prácticas 1
### Objetivos:
* Familiarizarse con los principales aspectos y etapas del método de descenso por gradiente.
* Conocer y aplicar el método de descenso por gradiente para minimizar funciones sencillas.
* Conocer cómo aplicar el proceso de descenso por gradiente de forma automatizada con soporte de **sympy** y **Jupyter Notebook*.

### Enunciado:

1. Seleccionar una función matemática para realizar el proceso de minimización. Ejecutar al menos **3 pasos** del método de descenso por gradiente.
2. Diseñar y desarrollar un cuaderno en Jupyter Notebook donde se realicen todos los pasos correspondientes al proceso de minimización de la función a través del método de descenso por gradiente.
3. El cuaderno deberá incluir los siguientes puntos:
    1. Gráfica de la función y los puntos que se obtienen a medida que se ejecutan los pasos de cálculo (hasta $k=3$).
    2. Aplicación de las funciones de derivación y evaluación de forma similar a la que se ha detallado en el presente cuaderno.
    3. Incluir un acápite sobre las funciones cóncavas y los puntos estacionarios (incluir gráficos).
    4. Emplear las funcionalidades que proveen los paquetes **matplotlib** y **sympy**.

### Criterios de evaluación:
Los criterios o rúbrica de evaluación del Boletín de Prácticas 1 son los siguientes:
1. Adecuada complejidad de la función seleccionada para el proceso de minimización (procurar no usar funciones cóncavas).
2. Correcta explicación y detalle de cada paso ejecutado con el método de descenso por gradiente.
3. Verificación de la solución encontrada.
4. Elementos extra: incluye recta (flecha) con la dirección del gradiente, animaciones, etc.

### Prerrequisitos:
A fin de poder realizar esta práctica, deberá contar con los siguientes prerrequisitos:
1. Haber leído de forma completa el presente cuaderno.
2. Tener instalados los siguientes paquetes en su computador:
    1. Python 2.7+ (de preferencia 3.6+)
    2. [Sympy](http://www.sympy.org)
    3. [matplotlib](https://matplotlib.org/)

### Detalles de la entrega:
El cuaderno de Jupyter Notebook deberá ser cargado en el AVAC con todos los recursos necesarios (imágenes, enlaces, etc.) en un archivo comprimido, dentro de la tarea **Boletín de Prácticas 1: Descenso por gradiente f(x)**. Los detalles de la fecha y hora de entrega se encuentran especificados en el AVAC.

### Algunos enlaces de utilidad:
A continuación se presentan algunos enlaces que pueden resultar de utilidad:
* Wiki con algunas funciones básicas de la librería [sympy](https://github.com/sympy/sympy/wiki/Quick-examples)
* Un tutorial interactivo con ejemplos de técnicas de [optimización](http://www.benfrederickson.com/numerical-optimization/).

## Referencias
1. [Nedrich Matt, An Introduction to Gradient Descent and Linear Regression, Atomic Object, 2014.](https://spin.atomicobject.com/2014/06/24/gradient-descent-linear-regression/)
2. [Renjie Liao, CSC 411 Tutorial: Optimization for Machine Learning, Torondo University, 2016.](https://www.cs.toronto.edu/~urtasun/courses/CSC411_Fall16/tutorial2.pdf)
3. [Singer Yaron, Advanced Optimization: Lecture 9, Harvard University, 2016](https://people.seas.harvard.edu/~yaron/AM221-S16/lecture_notes/AM221_lecture9.pdf)


# Sources

https://github.com/thiery-lab/data-assimilation

## Model

The model is a [three-dimensional dynamical system developed as a simpified model of atmosphereic convection](https://en.wikipedia.org/wiki/Lorenz_system) by Lorenz [1] which is a classic example of chaotic non-periodic dynamics. A flow map $\varphi : \mathbb{R}_{\geq 0} \to (\mathbb{R}^3 \to \mathbb{R}^3)$ is defined such that $\boldsymbol{x}(\tau) = \varphi(\tau)(\boldsymbol{x}_0)$ where $\boldsymbol{x}(\tau), \tau > 0$ is the solution to the initial value problem
\begin{equation}
  \frac{\mathrm{d} x_0}{\mathrm{d} \tau} = \sigma (x_1 - x_0), \quad
  \frac{\mathrm{d} x_1}{\mathrm{d} \tau} = x_0 (\rho - x_2) - x_1, \quad
  \frac{\mathrm{d} x_2}{\mathrm{d} \tau} = x_0 x_1 - \beta x_2 \quad
  \text{and} \quad \boldsymbol{x}(0) = \boldsymbol{x}_0.
\end{equation}

Here we use the (standard) parameter values $\rho = 28, \sigma=10, \beta=\frac{8}{3}$ which exhibit chaotic dynamics. An implicit mid-point method is used to define an approximate flow map $\tilde{\varphi}(\Delta) \approx \varphi(\Delta)$ with time step $\Delta = 0.005$ and $S=10$ time steps per state update. The state update is $\boldsymbol{x}_t = \bigcirc_{s=1}^S \big(\tilde{\varphi}({\Delta})\big)(\boldsymbol{x}_{t-1}) + \boldsymbol{\xi}_t$ with additive noise with distribution $\boldsymbol{\xi}_t \sim \mathcal{N}(\mathbf{0}_3, 0.1^2 \mathbb{I}_3)$ and initial state distribution $\boldsymbol{x}_0 \sim \mathcal{N}(\mathbf{1}_3, 0.5^2 \mathbb{I}_3)$. The first $x_{0,t}$ component of the state is noisily observed $y_t = x_{0,t} + \epsilon_t$, with additive Gaussian noise $\epsilon_t \sim \mathcal{N}(0, 5^2)$.

References:

1. Lorenz, Edward Norton (1963). Deterministic nonperiodic flow. Journal of the Atmospheric Sciences. 20 (2): 130–141.

## Data Assimilation

```python
import numpy as np
import dapy.filters as filters
from dapy.models import Lorenz1963Model
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
%matplotlib inline
plt.style.use('seaborn-white')
plt.rcParams['figure.dpi'] = 100
```

```python
model_params = {
    'sigma': 10.,
    'rho': 28.,
    'beta': 8. / 3.,
    'initial_state_mean': 1.,
    'initial_state_std': 0.5,
    'state_noise_std': 0.1,
    'observation_function':  lambda x, t: x[..., 0:1],
    'observation_noise_std': 5.,
    'time_step': 0.005,
    'num_integrator_step_per_update': 5,
}
model = Lorenz1963Model(**model_params)
```

### Generate data from model

```python
num_observation_time = 800
```

```python
# TODO 0
# Create an array of indices from 0..(num_observation_time-1)
observation_time_indices = ??
seed = 1234
rng = np.random.default_rng(seed)
state_sequence, observation_sequence = model.sample_state_and_observation_sequences(
    rng, observation_time_indices)
```

```python
fig, ax = plt.subplots(figsize=(12, 4))
ax.plot(observation_time_indices, state_sequence)
ax.plot(observation_time_indices, observation_sequence, '.')
ax.set_xlabel('Time index $t$')
ax.set_ylabel('State')
_ = ax.set_xlim(0, num_observation_time - 1)
ax.legend(['$x_0(S\Delta t)$', '$x_1(S\Delta t)$', '$x_2(S\Delta t)$', '$y_t$'], ncol=4)
fig.tight_layout()
```

```python
fig, ax = plt.subplots(figsize=(8, 8), subplot_kw={'projection': '3d'})
ax.plot(*state_sequence.T, '-', lw=1)
ax.set_xlabel('$x_0$')
ax.set_ylabel('$x_1$')
ax.set_zlabel('$x_2$')
fig.tight_layout()
```

### Infer state from observations

```python
def plot_results(results, observation_time_indices, state_sequence=None,
                 plot_particles=False, plot_region=True,
                 particle_skip=2, trace_alpha=0.5):
    fig, axes = plt.subplots(nrows=3, ncols=1, sharex=True, figsize=(12, 9))
    for i, ax in zip(range(3), axes):
        ax.plot(results['state_mean_sequence'][:, i], 'g-', lw=1, label='Est. mean')
        if plot_region:
            ax.fill_between(
                observation_time_indices,
                results['state_mean_sequence'][:, i] - 3 * results['state_std_sequence'][:, i],
                results['state_mean_sequence'][:, i] + 3 * results['state_std_sequence'][:, i],
                alpha=0.25, color='g', label='Est. mean ± 3 standard deviation'
            )
        if plot_particles:
            lines = ax.plot(
                observation_time_indices, results['state_particles_sequence'][:, ::particle_skip, i],
                'r-', lw=0.25, alpha=trace_alpha)
            lines[0].set_label('Particles')
        if state_sequence is not None:
            ax.plot(observation_time_indices, state_sequence[:, i], 'k--', label='Truth')
        ax.set_ylabel('$x_{0}$'.format(i))
        ax.legend(loc='upper center', ncol=4)
    ax.set_xlabel('Time index $t$')
    fig.tight_layout()
    return fig, axes
```

```python
from IPython.display import Image
```

### Kalman filter

```python
Image(filename='/opt/static/Basic_concept_of_Kalman_filtering.png')
```

References:

1. Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME -- Journal of Basic Engineering, Series D, 82, pp. 35--45.

```python
fkf = filters.FunctionKalmanFilter()
```

```python
results_fkf = fkf.filter(
    model, observation_sequence, observation_time_indices, rng=rng)
```

```python
fig, axes = plot_results(results_fkf, observation_time_indices, state_sequence)
```

### Ensemble Kalman filter (perturbed observations)

Ensemble Kalman filter with perturbed observations.

The filtering distribution at each observation time index is approximated by alternating propagating an ensemble of state particles forward through time under the model dynamics and linearly transforming the ensemble according to a Monte Carlo estimate of the Kalman filter assimilation update due to the observations at the current time index. Here a 'perturbed observation' ensemble Kalman filter assimilation update is used with an observation particle sampled for each state particle from the conditional distribution on the observation given the state, and these observation particles as well as the original state particles used to approximate the covariance and mean statistics used in the Kalman update.

References:

Evensen, G. (1994). Sequential data assimilation with nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. Journal of Geophysical Research, 99 (C5), pp. 143--162

Burgers, G.,van Leeuwen, P. J. and Evensen, G. (1998). Analysis scheme in the ensemble Kalman filter. Monthly Weather Review, (126) pp 1719--1724.

```python
enkf = filters.EnsembleKalmanFilter()
```

```python
# Experiment with different values for num_particle
# How does the result change?
num_particle = ??
```

```python
results_enkf = enkf.filter(
    model, observation_sequence, observation_time_indices,
    num_particle=num_particle, rng=rng, return_particles=True)
```

```python
fig, axes = plot_results(results_enkf, observation_time_indices, state_sequence)
axes[0].set_title('{} particles'.format(num_particle));
```

### Ensemble Transform Kalman filter (deterministic square root)

Ensemble Kalman filter with deterministic matrix square root updates.

The filtering distribution at each observation time index is approximated by alternating propagating an ensemble of state particles forward through time under the model dynamics and linearly transforming the ensemble according to a Monte Carlo estimate of the Kalman filter assimilation update due to the observations at the current time index. Here a 'square-root' ensemble Kalman filter assimilation update is used, which requires that the model has Gaussian observation noise with a known covariance, but compared to the 'perturbed observation' variant avoids the additional variance associated with sampling pseudo-observations.

References:

1. Bishop, C. H. Etherton, B. J. and  Majumdar, S. J. (2001). Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects.Mon. Wea. Rev., 129, 420–436.
2. M. K. Tippett, J. L. Anderson, C. H. Bishop, T. M. Hamill, and J. S. Whitaker, Ensemble square root filters, Monthly Weather Review, 131 (2003), pp. 1485--1490.

```python
etkf = filters.EnsembleTransformKalmanFilter()
```

```python
# Experiment with different values for num_particle
# How does the result change?
num_particle = ??
```

```python
results_etkf = etkf.filter(
    model, observation_sequence, observation_time_indices,
    num_particle=num_particle, rng=rng, return_particles=True)
```

```python
fig, axes = plot_results(results_etkf, observation_time_indices, state_sequence)
axes[0].set_title('{} particles'.format(num_particle));
```

TODO 3

Try different parameters for the model. Does it still exhibit chaotic properties? How do different filters adapt?

## Reservoir computing

Reservoir computing is a framework for computation derived from recurrent neural network theory that maps input signals into higher dimensional computational spaces through the dynamics of a fixed, non-linear system called a reservoir. After the input signal is fed into the reservoir, which is treated as a "black box," a simple readout mechanism is trained to read the state of the reservoir and map it to the desired output. The first key benefit of this framework is that training is performed only at the readout stage, as the reservoir dynamics are fixed. The second is that the computational power of naturally available systems, both classical and quantum mechanical, can be utilized to reduce the effective computational cost.

References:

1. Tanaka, Gouhei; Yamane, Toshiyuki; Héroux, Jean Benoit; Nakane, Ryosho; Kanazawa, Naoki; Takeda, Seiji; Numata, Hidetoshi; Nakano, Daiju; Hirose, Akira (2019). "Recent advances in physical reservoir computing: A review". Neural Networks. 115: 100–123. doi:10.1016/j.neunet.2019.03.005. ISSN 0893-6080. PMID 30981085.
2. Röhm, André; Lüdge, Kathy (2018-08-03). "Multiplexed networks: reservoir computing with virtual and real nodes". Journal of Physics Communications. 2 (8): 085007. Bibcode:2018JPhCo...2h5007R. doi:10.1088/2399-6528/aad56d. ISSN 2399-6528

```python
Image(filename='/opt/static/reservior_computing.jpeg')
```

## Echo state network

The echo state network (ESN) is a type of reservoir computer that uses a recurrent neural network with a sparsely connected hidden layer (with typically 1% connectivity). The connectivity and weights of hidden neurons are fixed and randomly assigned. The weights of output neurons can be learned so that the network can produce or reproduce specific temporal patterns. The main interest of this network is that although its behaviour is non-linear, the only weights that are modified during training are for the synapses that connect the hidden neurons to output neurons. Thus, the error function is quadratic with respect to the parameter vector and can be differentiated easily to a linear system.

### Mackey-Glass

```python
import sys
sys.path.append('/tmp/pyESN')
from pyESN import ESN
data = np.load('/tmp/pyESN/mackey_glass_t17.npy')
```

```python
#TODO: Set the number of output layers
# Experiment with different values for n_reservoir
# How does the result change?
esn = ESN(n_inputs = 1,
          n_outputs = ??,
          n_reservoir = ??,
          spectral_radius = 1.5,
          random_state=42)
```

```python
trainlen = 1000
future = 1000
pred_training = esn.fit(np.ones(trainlen),data[:trainlen])
prediction = esn.predict(np.ones(future))
```

```python
print("test error: \n"+str(np.sqrt(np.mean((prediction.flatten() - data[trainlen:trainlen+future])**2))))
```

```python
plt.figure(figsize=(11,1.5))
plt.plot(range(0,trainlen+future),data[0:trainlen+future],'k',label="target system")
plt.plot(range(trainlen,trainlen+future),prediction,'r', label="free running ESN")
lo,hi = plt.ylim()
plt.plot([trainlen,trainlen],[lo+np.spacing(1),hi-np.spacing(1)],'k:')
plt.legend(loc=(0.61,1.1),fontsize='x-small')
```

### Lorenz

```python
num_observation_time = 4000
observation_time_indices = np.arange(num_observation_time)
seed = 1234
rng = np.random.default_rng(seed)
data, observation_sequence = model.sample_state_and_observation_sequences(
    rng, observation_time_indices)
```

```python
#TODO: Set the number of output layers
# Experiment with different values for n_reservoir
# How does the result change?
esn = ESN(n_inputs = 1,
          n_outputs = ??,
          n_reservoir = ??,
          spectral_radius = 1.5,
          random_state=42)
```

```python
trainlen = 2000
future = 2000
pred_training = esn.fit(np.ones(trainlen),data[:trainlen])
prediction = esn.predict(np.ones(future))
```

```python
fig = plt.figure()
ax = fig.gca(projection='3d')

#TODO: Show prediction result on plot
ax.plot(??,??,??, lw=0.5)
ax.plot(data[future:,0], data[future:,1], data[future:,2], lw=0.5)
ax.set_xlabel("X Axis")
ax.set_ylabel("Y Axis")
ax.set_zlabel("Z Axis")
ax.set_title("Lorenz Attractor")
```

### Bitcoin price forecasting

```python
data = np.genfromtxt('/opt/static/BTC-USD.csv', delimiter=',')
```

```python
#TODO: Set the number of output layers
# Experiment with different values for n_reservoir
# How does the result change?
esn = ESN(n_inputs = 1,
          n_outputs = 1,
          n_reservoir = 400,
          spectral_radius = 1.5,
          random_state=42)
```

```python
# Experiment with different values for trainlen & feature (sum have to be equal 367)
# How does the result change?
trainlen = ??
future = ??
pred_training = esn.fit(np.ones(trainlen),data[:trainlen])
prediction = esn.predict(np.ones(future))
```

```python
plt.plot(range(0,trainlen+future),data[0:trainlen+future],'k',label="target system")
plt.plot(range(trainlen,trainlen+future),prediction,'r', label="free running ESN")
lo,hi = plt.ylim()
plt.plot([trainlen,trainlen],[lo+np.spacing(1),hi-np.spacing(1)],'k:')
plt.legend(loc=(0.61,1.1),fontsize='x-small')
```

```python

```


```python
import numpy as np
import pandas as pd
import linearsolve as ls
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Homework 8

**Instructions:** Complete the notebook below. Download the completed notebook in HTML format. Upload assignment using Canvas.

**Due:** Mar. 4 at **2pm.**

## Exercise: Changing $\beta$ in Prescott's Real Business Cycle Model

Recall that the equilibrium conditions for Prescott's RBC model are:

\begin{align}
\frac{1}{C_t} & = \beta E_t \left[\frac{\alpha A_{t+1}K_{t+1}^{\alpha-1}L_{t+1}^{1-\alpha} +1-\delta }{C_{t+1}}\right]\\
\frac{\varphi}{1-L_t} & = \frac{(1-\alpha)A_tK_t^{\alpha}L_t^{-\alpha}}{C_t} \\
Y_t & = A_t K_t^{\alpha}L_t^{1-\alpha}\\
K_{t+1} & = I_t + (1-\delta) K_t\\
Y_t & = C_t + I_t\\
\log A_{t+1} & = \rho \log A_t + \epsilon_{t+1}
\end{align}

where $\epsilon_{t+1} \sim \mathcal{N}(0,\sigma^2)$.

The objective is use `linearsolve` to simulate impulse responses to a TFP shock for $\beta = 0.96,0.97,0.98,0.99$. Other parameter values are given in the table below:

| $$\sigma$$ | $$\rho$$  | $$\varphi$$ | $$\alpha$$ | $$\delta $$ |
|------------|-----------|-------------|------------|-------------|
| 0.006      | 0.75      | 1.7317      | 0.35       |  0.025      |

## Model Preparation

As usual, we recast the model in the form required for `linearsolve`. Write the model with all variables moved to the left-hand side of the equations and dropping the expecations operator $E_t$ and the exogenous shock $\epsilon_{t+1}$:

\begin{align}
0 & = \beta\left[\frac{\alpha A_{t+1}K_{t+1}^{\alpha-1}L_{t+1}^{1-\alpha} +1-\delta }{C_{t+1}}\right] - \frac{1}{C_t}\\
0 & = \frac{(1-\alpha)A_tK_t^{\alpha}L_t^{-\alpha}}{C_t} - \frac{\varphi}{1-L_t}\\
0 & = A_t K_t^{\alpha}L_t^{1-\alpha} - Y_t\\
0 & = I_t + (1-\delta) K_t - K_{t+1}\\
0 & = C_t + I_t - Y_t\\
0 & = \rho \log A_t - \log A_{t+1}
\end{align}

Remember, capital and TFP are called *state variables* because they're $t+1$ values are predetermined. Output, consumption, and investment are called a *costate* or *control* variables. Note that the model as 5 equations in 5 endogenous variables.

## Initialization, Approximation, and Solution

The next several cells initialize the model in `linearsolve` and then approximate and solve it.

```python
# Create a variable called 'parameters' that stores the model parameter values in a Pandas Series.

# Print the model's parameters

```

```python
# Create variable called 'varNames' that stores the variable names in a list with state variables ordered first.

# Create variable called 'shockNames' that stores an exogenous shock name for each state variable.

```

```python
# Define a function that evaluates the equilibrium conditions of the model solved for zero

    # Parameters

    # Current variables

    # Forward variables

    # Euler equation

    # Labor-labor choise

    # Production function

    # Capital evolution

    # Market clearing

    # Exogenous tfp

    # Stack equilibrium conditions into a numpy array

```

Next, initialize the model using `ls.model` which takes the following required arguments:

* `equations`
* `n_states`
* `var_names`
* `shock_names`
* `parameters`

```python
# Initialize the model into a variable named 'rbc_model'

```

## SImulation and Plotting

Create a $2\times 2$ grid of plots containing the impulse responses of TFP, output, labor, and consumption to a one percent shock to TFP for each of the values for $\beta$: 0.96,0.97,0.98,0.99. Your figure should have a legend that clearly indicates which curves were produced from

Here are the steps that that you should take:

1. Initilize figure and axes for plotting.
2. Iterate over each desired value for $\beta$.
    1. Set `rbc_model.parameters['beta']` equal to current value of $\beta$.
    2. Use `rbc_model.compute_ss()` to compute the steady state with `guess` equal to `[1,4,1,1,1,0.5]`.
    3. Use `rbc_model.approximate_and_solve()` to approximate and solve the model with the current value of $\beta$.
    4. Use `rbc_model.impulse()` to compute the **31** period impulse response to a 0.01 unit shock to TFP in period 5.
    5. Add the computed impulse responses to the axes.

Be sure to add a legend somewhere in your figure so that it's clear which impulse response lines were determined by which value of $\beta$.

**Questions**

1. Describe in your own words how increasing $\beta$ from 0.96 to 0.99 affects the impulse response of consumption to a TFP shock.
2. Explain in your own words the intuition for *why* your observation in the previous question makes sense.
3. Describe in your own words how increasing $\beta$ from 0.96 to 0.99 affects the impulse response of labor to a TFP shock.
4. Explain in your own words the intuition for *why* your observation in the previous question makes sense.

**Answers**

1.

2.

1.

2.

## Exercise: Monetary policy Regimes

In August 6, 1979, Paul Volcker began his first of two terms as chair of the Board of Governors of the Federal Reserve System. In the two decades prior to Volcker's appointment, inflation in the US had been acccelerating rapidly, as the graph produced by the code in the next cell shows:

```python
# Import GDP deflator data from FRED. CELL PROVIDED
deflator = pd.read_csv('https://fred.stlouisfed.org/data/GDPDEF.txt',sep='\s+',skiprows=15,index_col=0,parse_dates=True)

# Set deflator equal to 'VALUE' column of deflator
deflator = deflator['VALUE']

# Compute the inflation rate
inflation = (deflator - deflator.shift(4))/deflator.shift(4)*100

# Plot
inflation.plot(title='US GDP Deflator Inflation',ylabel='Percent',grid=True)
```

The decline and stabilization of inflation in the US starting in the early 1980s is widely attributed to Volcker's leadership. As chair, he pushed the FOMC to aggressively pursue tight monetary policy that led to a sharp contraction in the rate of money growth and an increase in the federal fund rate.

In light of this story, many economists argue that Volcker was the start of a new monetary policy *regime*. Before Volcker, the Fed pursued a looser monetary policy that allowed inflation to accelerate and starting with Volcker, the Fed has pursued a tighter monetary policy that is more aggressive to managing inflation.

Here you will test this proposition by estimating a monetary policy rule for the US on data before Volcker's arrival at the Fed and after. Basically, you will replicate the estimation at the end of class 16 for pre- and post-Volcker data.

### Preliminaries

The next block of code imports federal funds rate, GDP deflator inflation, and output gap data from FRED and returs the data as a DataFrame called `df`.

```python
# Initial an empty DataFrame that will store data. CELL PROVIDED
df = pd.DataFrame()

# Import federal funds rate data from FRED. Use arguments: sep='\s+',skiprows=62,index_col=0,parse_dates=True
fed_funds = pd.read_csv('https://fred.stlouisfed.org/data/FEDFUNDS.txt',sep='\s+',skiprows=62,index_col=0,parse_dates=True)

# Set fed_funds equal to 'VALUE' column of fed_funds
fed_funds = fed_funds['VALUE']

# Use .resample('QS').mean() method of 'fed_funds' to convert the fed funds data from monthdy to quarterly
fed_funds = fed_funds.resample('QS').mean()

# Import GDP deflator data from FRED
deflator = pd.read_csv('https://fred.stlouisfed.org/data/GDPDEF.txt',sep='\s+',skiprows=15,index_col=0,parse_dates=True)

# Set deflator equal to 'VALUE' column of deflator
deflator = deflator['VALUE']

# Compute the inflation rate
inflation = (deflator - deflator.shift(4))/deflator.shift(4)*100

# Import real GDP data from FRED
gdp_actual = pd.read_csv('https://fred.stlouisfed.org/data/GDPC1.txt',sep='\s+',skiprows=17,index_col=0,parse_dates=True)

# Set gdp_actual equal to 'VALUE' column of gdp_actual
gdp_actual = gdp_actual['VALUE']

# Import potential real GDP data from FRED
gdp_potential = pd.read_csv('https://fred.stlouisfed.org/data/GDPPOT.txt',sep='\s+',skiprows=12,index_col=0,parse_dates=True)

# Set gdp_potential equal to 'VALUE' column of gdp_potential
gdp_potential = gdp_potential['VALUE']

# Create variable 'df' that is a DataFrame storing fed funds, inflation, actual and potential real GDP
df = pd.DataFrame({
    'fed_funds':fed_funds,
    'inflation':inflation,
    'gdp_actual':gdp_actual,
    'gdp_potential':gdp_potential
})

# Drop missing values from 'df'
df = df.dropna()
```

### The Output Gap

The output gap is measured as the percent difference of real GDP from the CBO's estimate of the potential real GDP:

\begin{align}
\text{Output gap} & = \left(\frac{\text{Real GDP}-\text{Real potential GDP}}{\text{Real potential GDP}}\right)\cdot 100
\end{align}

Real GDP has FRED series ID `GDPC1` and is available here: https://fred.stlouisfed.org/series/GDPC1. Notice that there are 17 lines of text *before* the line starting with "DATE".

```python
# Construct an output gap column

```

### OLS Regressions

The rule to be estimated is the same considered in Class 16:

\begin{align}
\hat{i}_t & = \bar{\imath} + \phi_{\pi}\pi_t + \phi_{y} y_t + \epsilon_t
\end{align}

where $\pi_t$ is the percent change in the GDP deflator over the previous year and $y_t$ is the output gap measured as the percent difference of real GDP from the CBO's estimate of the potential real GDP. $\phi_{\pi}$ is the weight that the FOMC places on inflation in the rule and $\phi_{y}$ is the weight that the central bank places on the output gap. $\epsilon_t$ is the residual of the regression.

```python
# Import statsmodels.api as sm

```

For a Pandas DataFrame or Series with a DateTime index, you can select a subinterval of the dates using data strings in `.ilo[]`. For example,

    `df.loc[:'1960']`

will return all data through the end of 1960. And:

    df.loc['08-2000':]

will return all data from and after August 2000.

#### Pre-Volcker Era

Estimate the monetary policy rule for dates through September 1979.

```python
# Create variable 'X' with columns inflation, output and a constant

# Create variable 'Y' equal to the federal funds rate

```

```python
# Initialize OLS model

# Fit OLS model

# Print regression results

```

#### Post-Volcker Era

Estimate the monetary policy rule for dates after October 1979.

```python
# Create variable 'X' with columns inflation, output and a constant

# Create variable 'Y' equal to the federal funds rate

```

```python
# Initialize OLS model

# Fit OLS model

# Print regression results

```

**Questions**

1. Compare the results from the two regressions. For which monetary policy era is the estimated coefficient on inflation greater?
2. How do your results help to explain the high and variable inflation of the 1960s and 1970s and the lower and more stable inflation of the 1980s and after?

**Answers**

1.

2.