# Trends and cycles in unemployment

Here we consider three methods for separating a trend and cycle in economic data. Supposing we have a time series $y_t$, the basic idea is to decompose it into these two components:

$$
y_t = \mu_t + \eta_t
$$

where $\mu_t$ represents the trend or level and $\eta_t$ represents the cyclical component. In this case, we consider a *stochastic* trend, so that $\mu_t$ is a random variable and not a deterministic function of time. Two of methods fall under the heading of "unobserved components" models, and the third is the popular Hodrick-Prescott (HP) filter. Consistent with e.g. Harvey and Jaeger (1993), we find that these models all produce similar decompositions.

This notebook demonstrates applying these models to separate trend from cycle in the U.S. unemployment rate.

```python
%matplotlib inline
```

```python
import numpy as np
import pandas as pd
import statsmodels.api as sm
import matplotlib.pyplot as plt
```

```python
from pandas_datareader.data import DataReader
endog = DataReader('UNRATE', 'fred', start='1954-01-01')
```

### Hodrick-Prescott (HP) filter

The first method is the Hodrick-Prescott filter, which can be applied to a data series in a very straightforward method. Here we specify the parameter $\lambda=129600$ because the unemployment rate is observed monthly.

```python
hp_cycle, hp_trend = sm.tsa.filters.hpfilter(endog, lamb=129600)
```

### Unobserved components and ARIMA model (UC-ARIMA)

The next method is an unobserved components model, where the trend is modeled as a random walk and the cycle is modeled with an ARIMA model - in particular, here we use an AR(4) model. The process for the time series can be written as:

$$
\begin{align}
y_t & = \mu_t + \eta_t \\
\mu_{t+1} & = \mu_t + \epsilon_{t+1} \\
\phi(L) \eta_t & = \nu_t
\end{align}
$$

where $\phi(L)$ is the AR(4) lag polynomial and $\epsilon_t$ and $\nu_t$ are white noise.

```python
mod_ucarima = sm.tsa.UnobservedComponents(endog, 'rwalk', autoregressive=4)
# Here the powell method is used, since it achieves a
# higher loglikelihood than the default L-BFGS method
res_ucarima = mod_ucarima.fit(method='powell', disp=False)
print(res_ucarima.summary())
```

    C:\ProgramData\Anaconda3\lib\site-packages\statsmodels\tsa\base\tsa_model.py:165: ValueWarning: No frequency information was provided, so inferred frequency MS will be used.
      % freq, ValueWarning)

                            Unobserved Components Results
    ==============================================================================
    Dep. Variable:                 UNRATE   No. Observations:                  791
    Model:                    random walk   Log Likelihood                 255.062
                                  + AR(4)   AIC                           -498.124
    Date:                Tue, 24 Dec 2019   BIC                           -470.092
    Time:                        15:04:29   HQIC                          -487.349
    Sample:                    01-01-1954
                             - 11-01-2019
    Covariance Type:                  opg
    ================================================================================
                       coef    std err          z      P>|z|      [0.025      0.975]
    --------------------------------------------------------------------------------
    sigma2.level     0.0175      0.003      6.405      0.000       0.012       0.023
    sigma2.ar        0.0106      0.003      3.515      0.000       0.005       0.016
    ar.L1            1.0404      0.066     15.764      0.000       0.911       1.170
    ar.L2            0.4727      0.104      4.525      0.000       0.268       0.677
    ar.L3           -0.3403      0.127     -2.686      0.007      -0.589      -0.092
    ar.L4           -0.1832      0.077     -2.371      0.018      -0.335      -0.032
    ===================================================================================
    Ljung-Box (Q):                       75.97   Jarque-Bera (JB):                44.03
    Prob(Q):                              0.00   Prob(JB):                         0.00
    Heteroskedasticity (H):               0.49   Skew:                             0.25
    Prob(H) (two-sided):                  0.00   Kurtosis:                         4.05
    ===================================================================================

    Warnings:
    [1] Covariance matrix calculated using the outer product of gradients (complex-step).

### Unobserved components with stochastic cycle (UC)

The final method is also an unobserved components model, but where the cycle is modeled explicitly.

$$
\begin{align}
y_t & = \mu_t + \eta_t \\
\mu_{t+1} & = \mu_t + \epsilon_{t+1} \\
\eta_{t+1} & = \eta_t \cos \lambda_\eta + \eta_t^* \sin \lambda_\eta + \tilde \omega_t \qquad & \tilde \omega_t \sim N(0, \sigma_{\tilde \omega}^2) \\
\eta_{t+1}^* & = -\eta_t \sin \lambda_\eta + \eta_t^* \cos \lambda_\eta + \tilde \omega_t^* & \tilde \omega_t^* \sim N(0, \sigma_{\tilde \omega}^2)
\end{align}
$$

```python
mod_uc = sm.tsa.UnobservedComponents(
    endog, 'rwalk',
    cycle=True, stochastic_cycle=True, damped_cycle=True,
)
# Here the powell method gets close to the optimum
res_uc = mod_uc.fit(method='powell', disp=False)
# but to get to the highest loglikelihood we do a
# second round using the L-BFGS method.
res_uc = mod_uc.fit(res_uc.params, disp=False)
print(res_uc.summary())
```

    C:\ProgramData\Anaconda3\lib\site-packages\statsmodels\tsa\base\tsa_model.py:165: ValueWarning: No frequency information was provided, so inferred frequency MS will be used.
      % freq, ValueWarning)

                                Unobserved Components Results
    =====================================================================================
    Dep. Variable:                        UNRATE   No. Observations:                  791
    Model:                           random walk   Log Likelihood                 219.139
                       + damped stochastic cycle   AIC                           -430.277
    Date:                       Tue, 24 Dec 2019   BIC                           -411.599
    Time:                               15:04:32   HQIC                          -423.097
    Sample:                           01-01-1954
                                    - 11-01-2019
    Covariance Type:                         opg
    ===================================================================================
                          coef    std err          z      P>|z|      [0.025      0.975]
    -----------------------------------------------------------------------------------
    sigma2.level        0.0142      0.005      2.823      0.005       0.004       0.024
    sigma2.cycle        0.0172      0.005      3.513      0.000       0.008       0.027
    frequency.cycle     0.0698      0.005     13.603      0.000       0.060       0.080
    damping.cycle       0.9896      0.004    240.586      0.000       0.982       0.998
    ===================================================================================
    Ljung-Box (Q):                      170.72   Jarque-Bera (JB):                85.76
    Prob(Q):                              0.00   Prob(JB):                         0.00
    Heteroskedasticity (H):               0.48   Skew:                             0.47
    Prob(H) (two-sided):                  0.00   Kurtosis:                         4.31
    ===================================================================================

    Warnings:
    [1] Covariance matrix calculated using the outer product of gradients (complex-step).

### Graphical comparison

The output of each of these models is an estimate of the trend component $\mu_t$ and an estimate of the cyclical component $\eta_t$. Qualitatively the estimates of trend and cycle are very similar, although the trend component from the HP filter is somewhat more variable than those from the unobserved components models. This means that relatively mode of the movement in the unemployment rate is attributed to changes in the underlying trend rather than to temporary cyclical movements.

```python
fig, axes = plt.subplots(2, figsize=(13,5));
axes[0].set(title='Level/trend component')
axes[0].plot(endog.index, res_uc.level.smoothed, label='UC')
axes[0].plot(endog.index, res_ucarima.level.smoothed, label='UC-ARIMA(2,0)')
axes[0].plot(hp_trend, label='HP Filter')
axes[0].legend(loc='upper left')
axes[0].grid()

axes[1].set(title='Cycle component')
axes[1].plot(endog.index, res_uc.cycle.smoothed, label='UC')
axes[1].plot(endog.index, res_ucarima.autoregressive.smoothed, label='UC-ARIMA(2,0)')
axes[1].plot(hp_cycle, label='HP Filter')
axes[1].legend(loc='upper left')
axes[1].grid()

fig.tight_layout();
```


# Molecular dynamics

## With OpenMM

```python
import openmm as mm
from openmm import unit
from uibcdf_systems import HarmonicWell

molecular_system = HarmonicWell(n_particles = 1, mass = 32 * unit.amu,
                                k=5.0 * unit.kilocalories_per_mole/unit.nanometers**2)

integrator = mm.LangevinIntegrator(300.0*unit.kelvin, 1.0/unit.picoseconds, 0.02*unit.picoseconds)
platform = Platform.getPlatformByName('CUDA')
simulation = Simulation(molecular_system.topology, molecular_system.system, integrator, platform)

coordinates = np.zeros([1, 3], np.float32) * unit.nanometers
simulation.context.setPositions()

velocities = np.zeros([1, 3], np.float32) * unit.nanometers/unit.picoseconds
simulation.context.setVelocities()

simulation.step(1000)
```

## With this library

## Newtonian dynamics

```python
import numpy as np
import sympy as sy
import matplotlib.pyplot as plt
from openmm import unit
```

```python
from uibcdf_systems import HarmonicWell
from uibcdf_systems.tools import langevin

molecular_system = HarmonicWell(n_particles = 1, mass = 32 * unit.amu,
                                k=5.0 * unit.kilocalories_per_mole/unit.nanometers**2)
```

```python
initial_positions = np.zeros([1, 3], np.float32) * unit.nanometers
initial_positions[0,0] = 1.0 * unit.nanometers

initial_velocities = np.zeros([1, 3], np.float32) * unit.nanometers/unit.picoseconds

molecular_system.set_coordinates(initial_positions)
molecular_system.set_velocities(initial_velocities)
```

```python
traj_dict = langevin(molecular_system,
                     friction=0.0/unit.picoseconds,
                     temperature=0.0*unit.kelvin,
                     time=50.0*unit.picoseconds,
                     saving_timestep=0.1*unit.picoseconds,
                     integration_timestep=0.005*unit.picoseconds)
```

      0%|          | 0/10000 [00:00<?, ?it/s]

We can now plot the trajectory of the x coordinate:

```python
plt.plot(traj_dict['time'], traj_dict['coordinates'][:,0,0])
plt.xlabel('time ({})'.format(traj_dict['time'].unit))
plt.ylabel('X ({})'.format(traj_dict['coordinates'].unit))
plt.show()
```

We can wonder now if the period of these oscillations is in agreement with the value calculated above.

```python
mass = molecular_system.parameters['mass']
k = molecular_system.parameters['k']
T = 2*np.pi*np.sqrt(mass/k)

print('The period of the small oscillations around the minimum is',T)
```

    The period of the small oscillations around the minimum is 7.770948260727904 ps

```python
molecular_system.get_oscillations_time_period()
```

    Quantity(value=7.770948260727904, unit=picosecond)

```python
plt.plot(traj_dict['time'], traj_dict['coordinates'][:,0,0])
plt.axvline(T._value, color='gray', linestyle='--') # Period of the harmonic oscillations
plt.xlabel('time ({})'.format(traj_dict['time'].unit))
plt.ylabel('X ({})'.format(traj_dict['coordinates'].unit))
plt.show()
```

Remember that the integration timestep must be smaller than $\sim T/10.0$ to guarantee that no artifacts are introduced by the timestep size.

The newtonian dynamics can also include damping. This way we can simulate damped oscillations around the minimum.

```python
traj_dict = langevin(molecular_system,
                     friction=0.25/unit.picoseconds,
                     temperature=0.0*unit.kelvin,
                     time=50.0*unit.picoseconds,
                     saving_timestep=0.1*unit.picoseconds,
                     integration_timestep=0.02*unit.picoseconds)
```

      0%|          | 0/2500 [00:00<?, ?it/s]

```python
plt.plot(traj_dict['time'], traj_dict['coordinates'][:,0,0])
plt.xlabel('time ({})'.format(traj_dict['time'].unit))
plt.ylabel('X ({})'.format(traj_dict['coordinates'].unit))
plt.show()
```

What would be the friction value needed to enter in the overdamped regime?

```python
traj_dict = langevin(molecular_system,
                     friction=5.0/unit.picoseconds,
                     temperature=0.0*unit.kelvin,
                     time=50.0*unit.picoseconds,
                     saving_timestep=0.1*unit.picoseconds,
                     integration_timestep=0.02*unit.picoseconds)
```

      0%|          | 0/2500 [00:00<?, ?it/s]

```python
plt.plot(traj_dict['time'], traj_dict['coordinates'][:,0,0])
plt.xlabel('time ({})'.format(traj_dict['time'].unit))
plt.ylabel('X ({})'.format(traj_dict['coordinates'].unit))
plt.show()
```

## Langevin dynamics

```python
traj_dict = langevin(molecular_system,
                     friction=1.0/unit.picoseconds,
                     temperature=300.0*unit.kelvin,
                     time=1.0*unit.nanoseconds,
                     saving_timestep=0.5*unit.picoseconds,
                     integration_timestep=0.02*unit.picoseconds)

```

      0%|          | 0/50000 [00:00<?, ?it/s]

Let us see the time evolution of the coordinate $x$ of our single particle:

```python
plt.plot(traj_dict['time'], traj_dict['coordinates'][:,0,0])
plt.xlabel('time ({})'.format(traj_dict['time'].unit))
plt.ylabel('X ({})'.format(traj_dict['coordinates'].unit))
plt.show()
```

```python
molecular_system.get_standard_deviation(300.0*unit.kelvin)
```

    Quantity(value=0.34530023967331663, unit=nanometer)

```python
np.std(traj_dict['coordinates'][:,0,0])
```

    Quantity(value=0.34393127752734576, unit=nanometer)


# Rosenbrock function
Consider the n-dimensional Rosenbrock function:

\begin{eqnarray}
\sum_{i=1}^{n-1} 100 \cdot (x_{i+1} - x_{i}^2)^2 + (x_{i} - 1)^2 &
\end{eqnarray}

a. What is the i-th position of $\nabla f(x)$, for $1 \le i \le n$?

b. What is the ij-th position of $\nabla^2 f(x)$, for $1 \le i,j \le n$?

```
import numpy as np
import scipy.sparse as spa
import scipy.sparse.linalg as las
import matplotlib.pyplot as plt
import seaborn as sns
from sympy.parsing.sympy_parser import parse_expr
from IPython.display import display, Latex
import sympy as sp
```

    /usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
      import pandas.util.testing as tm

```
def fun_Rosenbrock(n):
  f = parse_expr('+'.join([f'100*(x{i+1}-x{i}**2)**2 + (x{i}-1)**2' for i in range(1,n)]));
  return f

def fun_Gradient(f,x):
  gf = sp.Matrix([sp.diff(f,var) for var in x]);
  return gf;

def fun_Hessian(g,x):
  Hf = sp.Matrix([list(fun_Gradient(fun,x)) for fun in g]);
  return Hf;
```

```
n = 5;
fx = fun_Rosenbrock(n);

xs = sp.Matrix([f'x{i}' for i in range(1,n+1)]);
gx = fun_Gradient(fx,xs);

Hx = fun_Hessian(gx,xs);
```

## Rosenbrock function

```
display(Latex('f({\\bf x}) = '+sp.latex(fx)));
```

f({\bf x}) = \left(x_{1} - 1\right)^{2} + 100 \left(- x_{1}^{2} + x_{2}\right)^{2} + \left(x_{2} - 1\right)^{2} + 100 \left(- x_{2}^{2} + x_{3}\right)^{2} + \left(x_{3} - 1\right)^{2} + 100 \left(- x_{3}^{2} + x_{4}\right)^{2} + \left(x_{4} - 1\right)^{2} + 100 \left(- x_{4}^{2} + x_{5}\right)^{2}

```
def rosenbrock(x,n):
  f = np.sum([100*(x[i+1]-x[i]**2)**2+(x[i]-1)**2 for i in range(0,n-1)]);
  return f
```

## Rosenbrock gradient

```
display(Latex('\\nabla^2 f({\\bf x}) = '+sp.latex(gx)));
```

\nabla^2 f({\bf x}) = \left[\begin{matrix}- 400 x_{1} \left(- x_{1}^{2} + x_{2}\right) + 2 x_{1} - 2\\- 200 x_{1}^{2} - 400 x_{2} \left(- x_{2}^{2} + x_{3}\right) + 202 x_{2} - 2\\- 200 x_{2}^{2} - 400 x_{3} \left(- x_{3}^{2} + x_{4}\right) + 202 x_{3} - 2\\- 200 x_{3}^{2} - 400 x_{4} \left(- x_{4}^{2} + x_{5}\right) + 202 x_{4} - 2\\- 200 x_{4}^{2} + 200 x_{5}\end{matrix}\right]

```
def gradient(x,n):
  g = np.zeros(n);
  g[1:] = -200*np.power(x[:-1],2)
  g[:-1] += -400*x[:-1] * (-np.power(x[:-1],2) + x[1:])
  g[1:-1] += 202*x[1:-1]
  g[0] += 2*x[0]
  g[-1] += 200*x[-1]
  g[:-1] += -2
  return g
```

## Rosenbrock hessian

```
display(Latex('\\nabla f({\\bf x}) = '+sp.latex(Hx)));
```

\nabla f({\bf x}) = \left[\begin{matrix}1200 x_{1}^{2} - 400 x_{2} + 2 & - 400 x_{1} & 0 & 0 & 0\\- 400 x_{1} & 1200 x_{2}^{2} - 400 x_{3} + 202 & - 400 x_{2} & 0 & 0\\0 & - 400 x_{2} & 1200 x_{3}^{2} - 400 x_{4} + 202 & - 400 x_{3} & 0\\0 & 0 & - 400 x_{3} & 1200 x_{4}^{2} - 400 x_{5} + 202 & - 400 x_{4}\\0 & 0 & 0 & - 400 x_{4} & 200\end{matrix}\right]

```
def hessian(x,n):
  ds = -400*x[:-1]
  dp = np.zeros(n);
  dp[:-1] = 1200*np.power(x[:-1],2) - 400*x[1:]
  dp[0] += 2
  dp[1:-1] += 202
  dp[-1] = 200
  ind = np.arange(0,n);
  I = np.concatenate((ind,ind[:-1],ind[:-1]+1));
  J = np.concatenate((ind,ind[:-1]+1,ind[:-1]));
  V = np.concatenate((dp,ds,ds))
  H = spa.coo_matrix((V,(I,J)));
  return H
```

## Newton's method

```
n = 10**4;
xk = 5*np.ones(n);
norm_gradientK = [];
for k in range(0,50):
  Hk = hessian(xk,n);
  gk = gradient(xk,n);
  norm_gradientK.append(np.linalg.norm(gk));
  dxk = las.spsolve(Hk,-gk);
  xk = xk+dxk;
```

    /usr/local/lib/python3.6/dist-packages/scipy/sparse/linalg/dsolve/linsolve.py:138: SparseEfficiencyWarning: spsolve requires A be CSC or CSR matrix format
      SparseEfficiencyWarning)

```
sns.set_style('darkgrid')
plt.plot(np.log10(norm_gradientK));
```

# Rotated Hyper-Ellipsoid function

Consider the n-dimensional rotated hyper-ellipsoid function:

\begin{eqnarray}
\sum_{i=1}^{n} \sum_{j=1}^{i} x^2_j
\end{eqnarray}

a. What is the i-th position of $\nabla f(x)$, for $1 \le i \le n$?

b. What is the ij-th position of $\nabla^2 f(x)$, for $1 \le i,j \le n$?

```
import numpy as np
import scipy.sparse as spa
import scipy.sparse.linalg as las
import matplotlib.pyplot as plt
import seaborn as sns
from sympy.parsing.sympy_parser import parse_expr
from IPython.display import display, Latex
import sympy as sp
```

```
def fun_RHE(n):
  for i in range(1, n):
    f = parse_expr('+'.join([f'x{j}**2' for j in range(1,i+1)]))
  return f

def fun_Gradient(f,x):
  gf = sp.Matrix([sp.diff(f,var) for var in x]);
  return gf;

def fun_Hessian(g,x):
  Hf = sp.Matrix([list(fun_Gradient(fun,x)) for fun in g]);
  return Hf;
```

```
n = 7;
fx = fun_RHE(n);

xs = sp.Matrix([f'x{i}' for i in range(1,n)]);
gx = fun_Gradient(fx,xs);

Hx = fun_Hessian(gx,xs);
```

## Rotated Hyper-Ellipsoid function

```
display(Latex('f({\\bf x}) = '+sp.latex(fx)));
```

f({\bf x}) = x_{1}^{2} + x_{2}^{2} + x_{3}^{2} + x_{4}^{2} + x_{5}^{2} + x_{6}^{2}

```
def RHE(n):
  rhe = np.sum([np.sum([x[j]**2 for j in range(0, i)]) for i in range(0, n)])
  return rhe
```

## RHE gradient

```
display(Latex('\\nabla f({\\bf x}) = '+sp.latex(gx)));
```

\nabla f({\bf x}) = \left[\begin{matrix}2 x_{1}\\2 x_{2}\\2 x_{3}\\2 x_{4}\\2 x_{5}\\2 x_{6}\end{matrix}\right]

```
def gradient(x,n):
  g = np.zeros(n)
  g[0:] = 2*x[0:]
  return g
```

## RHE hessian

```
display(Latex('\\nabla^2 f({\\bf x}) = '+sp.latex(Hx)));
```

\nabla^2 f({\bf x}) = \left[\begin{matrix}2 & 0 & 0 & 0 & 0 & 0\\0 & 2 & 0 & 0 & 0 & 0\\0 & 0 & 2 & 0 & 0 & 0\\0 & 0 & 0 & 2 & 0 & 0\\0 & 0 & 0 & 0 & 2 & 0\\0 & 0 & 0 & 0 & 0 & 2\end{matrix}\right]

```
def hessian(x,n):
  ds = np.zeros(n)
  dp = 2*np.ones(n)
  ind = np.arange(0,n);
  I = np.concatenate((ind,ind[:-1],ind[:-1]+1));
  J = np.concatenate((ind,ind[:-1]+1,ind[:-1]));
  V = np.concatenate((dp,ds,ds))
  H = spa.coo_matrix((V,(I,J)));
  return H
```

# Sum of squares function

Consider the n-dimensional sum of squares function:

\begin{eqnarray}
\sum_{i=1}^{n} i \cdot x^2_i
\end{eqnarray}

a. What is the i-th position of $\nabla f(x)$, for $1 \le i \le n$?

b. What is the ij-th position of $\nabla^2 f(x)$, for $1 \le i,j \le n$?

```
import numpy as np
import scipy.sparse as spa
import scipy.sparse.linalg as las
import matplotlib.pyplot as plt
import seaborn as sns
from sympy.parsing.sympy_parser import parse_expr
from IPython.display import display, Latex
import sympy as sp
```

```
def fun_SOS(n):
  f = parse_expr('+'.join([f'{i}*x{i}**2' for i in range(1,n)]));
  return f

def fun_Gradient(f,x):
  gf = sp.Matrix([sp.diff(f,var) for var in x]);
  return gf;

def fun_Hessian(g,x):
  Hf = sp.Matrix([list(fun_Gradient(fun,x)) for fun in g]);
  return Hf;
```

```
n = 6;
fx = fun_SOS(n);

xs = sp.Matrix([f'x{i}' for i in range(1,n)]);
gx = fun_Gradient(fx,xs);

Hx = fun_Hessian(gx,xs);
```

## Sum of squares function

```
display(Latex('f({\\bf x}) = '+sp.latex(fx)));
```

f({\bf x}) = x_{1}^{2} + 2 x_{2}^{2} + 3 x_{3}^{2} + 4 x_{4}^{2} + 5 x_{5}^{2}

```
def SOS(n):
  sos = np.sum([i*x[i]**2 for i in range(0, n)])
  return sos
```

## Sum of squares gradient

```
display(Latex('\\nabla f({\\bf x}) = '+sp.latex(gx)));
```

\nabla f({\bf x}) = \left[\begin{matrix}2 x_{1}\\4 x_{2}\\6 x_{3}\\8 x_{4}\\10 x_{5}\end{matrix}\right]

```
def SOS_gradient(x, n, multiples_2):
  g = np.zeros(n)
  g[0:] = np.multiply(x[0:],multiples_2)
  return g
```

## Sum of squares hessian

```
display(Latex('\\nabla^2 f({\\bf x}) = '+sp.latex(Hx)));
```

\nabla^2 f({\bf x}) = \left[\begin{matrix}2 & 0 & 0 & 0 & 0\\0 & 4 & 0 & 0 & 0\\0 & 0 & 6 & 0 & 0\\0 & 0 & 0 & 8 & 0\\0 & 0 & 0 & 0 & 10\end{matrix}\right]

```
def SOS_hessian(x, n):
  numbers = np.array([i for i in range(1,n)])
  ds = np.zeros(n)
  dp = 2*numbers
  ind = np.arange(0,n);
  I = np.concatenate((ind,ind[:-1],ind[:-1]+1));
  J = np.concatenate((ind,ind[:-1]+1,ind[:-1]));
  V = np.concatenate((dp,ds,ds))
  H = spa.coo_matrix((V,(I,J)));
  return H
```

# Trid function
Consider the n-dimensional Trid function:

\begin{equation}
\begin{split}
f(x) = \sum_{i=1}^{d} (x_i - 1)^2 - \sum_{i=2}^{d}x_i x_{i-1}
\end{split}
\end{equation}

a. What is the i-th position of $\nabla f(x)$, for $1 \le i \le n$?

b. What is the ij-th position of $\nabla^2 f(x)$, for $1 \le i,j \le n$?

```
import numpy as np
import scipy.sparse as spa
import scipy.sparse.linalg as las
import matplotlib.pyplot as plt
import seaborn as sns
from sympy.parsing.sympy_parser import parse_expr
from IPython.display import display, Latex
import sympy as sp
```

    /usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
      import pandas.util.testing as tm

```
def fun_Trid(n):
  f = parse_expr('+'.join([f'(x{i}-1)**2' for i in range(1,n)])) - parse_expr('+'.join([f'x{i}*x{i-1}' for i in range(2,n)]));
  return f

def fun_Gradient(f,x):
  gf = sp.Matrix([sp.diff(f,var) for var in x]);
  return gf;

def fun_Hessian(g,x):
  Hf = sp.Matrix([list(fun_Gradient(fun,x)) for fun in g]);
  return Hf;
```

```
n = 7;
fx = fun_Trid(n);

xs = sp.Matrix([f'x{i}' for i in range(1,n)]);
gx = fun_Gradient(fx,xs);

Hx = fun_Hessian(gx,xs);
```

## Trid function

```
display(Latex('f({\\bf x}) = '+sp.latex(fx)));
```

f({\bf x}) = - x_{1} x_{2} - x_{2} x_{3} - x_{3} x_{4} - x_{4} x_{5} - x_{5} x_{6} + \left(x_{1} - 1\right)^{2} + \left(x_{2} - 1\right)^{2} + \left(x_{3} - 1\right)^{2} + \left(x_{4} - 1\right)^{2} + \left(x_{5} - 1\right)^{2} + \left(x_{6} - 1\right)^{2}

```
def trid(x,n):
  trid = np.sum([(x[i] - 1)**2 for i in range(1,n)]) - np.sum([x[i]*x[i-1] for i in range(2,n)])
  return trid
```

## Trid gradient

```
display(Latex('\\nabla f({\\bf x}) = '+sp.latex(gx)));
```

\nabla f({\bf x}) = \left[\begin{matrix}2 x_{1} - x_{2} - 2\\- x_{1} + 2 x_{2} - x_{3} - 2\\- x_{2} + 2 x_{3} - x_{4} - 2\\- x_{3} + 2 x_{4} - x_{5} - 2\\- x_{4} + 2 x_{5} - x_{6} - 2\\- x_{5} + 2 x_{6} - 2\end{matrix}\right]

```
def trid_gradient(x, n):
  g = np.zeros(n)
  g[1:] = -x[0:] + 2*x[1:] - x[2:]
  g[-1] = -x[-2] + 2*x[-1] -2
  g[0] = 2*x[0] - x[1] - 2
  g[0:] += -2
  return g
```

## Trid hessian

```
display(Latex('\\nabla^2 f({\\bf x}) = '+sp.latex(Hx)));
```

\nabla^2 f({\bf x}) = \left[\begin{matrix}2 & -1 & 0 & 0 & 0 & 0\\-1 & 2 & -1 & 0 & 0 & 0\\0 & -1 & 2 & -1 & 0 & 0\\0 & 0 & -1 & 2 & -1 & 0\\0 & 0 & 0 & -1 & 2 & -1\\0 & 0 & 0 & 0 & -1 & 2\end{matrix}\right]


# 2021-10-20 Krylov methods

## Last time

* Why iterative solvers
* Stationary iterative methods (Richardson)

## Today

* Limitations of stationary iterative methods
* Preconditioning as a concept
* Krylov methods (GMRES)
* Experiments with preconditioners and diagnostics in PETSc

```julia
using Plots
using LinearAlgebra
using SparseArrays

default(linewidth=4)

function advdiff_matrix(n; kappa=1, wind=[0, 0])
    "Advection-diffusion with Dirichlet boundary conditions eliminated"
    h = 2 / (n + 1)
    rows = Vector{Int64}()
    cols = Vector{Int64}()
    vals = Vector{Float64}()
    idx((i, j),) = (i-1)*n + j
    in_domain((i, j),) = 1 <= i <= n && 1 <= j <= n
    stencil_advect = [-wind[1], -wind[2], 0, wind[1], wind[2]] / h
    stencil_diffuse = [-1, -1, 4, -1, -1] * kappa / h^2
    stencil = stencil_advect + stencil_diffuse
    for i in 1:n
        for j in 1:n
            neighbors = [(i-1, j), (i, j-1), (i, j), (i+1, j), (i, j+1)]
            mask = in_domain.(neighbors)
            append!(rows, idx.(repeat([(i,j)], 5))[mask])
            append!(cols, idx.(neighbors)[mask])
            append!(vals, stencil[mask])
        end
    end
    sparse(rows, cols, vals)
end

function my_spy(A)
    cmax = norm(vec(A), Inf)
    s = max(1, ceil(120 / size(A, 1)))
    spy(A, marker=(:square, s), c=:diverging_rainbow_bgymr_45_85_c67_n256, clims=(-cmax, cmax))
end

function grad_descent(loss, grad, u0; omega=1e-3, tol=1e-5)
    """Minimize loss(c) via gradient descent with initial guess u0
    using learning rate gamma.  Declares convergence when gradient
    is less than tol or after 500 steps.
    """
    u = copy(u0)
    uhist = [copy(u)]
    lhist = [loss(u)]
    for it in 1:500
        g = grad(u)
        u -= omega * g
        push!(uhist, copy(u))
        push!(lhist, loss(u))
        if norm(g) < tol
            break
        end
    end
    (u, hcat(uhist...), lhist)
end
```

    grad_descent (generic function with 1 method)

# Visualize gradient descent

```julia
A = [1 1; 1 16]
loss(u) = .5 * u' * A * u
grad(u) = A * u
u, uhist, lhist = grad_descent(loss, grad, [.9, .9],
    omega=.1)
plot(lhist, yscale=:log10)
```

```julia
plot(uhist[1, :], uhist[2, :], marker=:circle)
x = LinRange(-1, 1, 30)
contour!(x, x, (x,y) -> loss([x, y]))
```

# Richardson iteration

The simplest iterative method is [Richardson's method](https://en.wikipedia.org/wiki/Modified_Richardson_iteration), which solves $A u = b$ by the iteration
$$ u_{k+1} = u_k + \omega (b - A u_k) $$
where $\omega > 0$ is a damping parameter and $u_0$ is an initial guess (possibly the zero vector).

* Algebraically equivalent to gradient descent when $A$ is SPD
* Non-symmetric matrices are harder to visualize

# Richardson convergence

$$ u_{k+1} = u_k + \omega (b - A u_k) $$

If $b = A u_*$, this iteration is equivalent to
\begin{align}
u_{k+1} - u_* &= (u_k - u_*) - \omega A (u_k - u_*) \\
&= (I - \omega A) (u_k - u_*) .
\end{align}
It is convenient for convergence analysis to identify the "error" $e_k = u_k - u_*$, in which this becomes
$$ e_{k+1} = (I - \omega A) e_k $$
or
$$ e_k = (I - \omega A)^k e_0 $$
in terms of the initial error.  Evidently powers of the *iteration matrix* $I - \omega A$ tell the whole story.

Suppose that the eigendecomposition
$$ X \Lambda X^{-1} = I - \omega A $$
exists.  Then
$$ (I - \omega A)^k = (X \Lambda X^{-1})^k = X \Lambda^k X^{-1} $$
and the convergence (or divergence) rate depends only on the largest magnitude eigenvalue.
This analysis isn't great for two reasons:

1. Not all matrices are diagonalizable.
2. The matrix $X$ may be very ill-conditioned.

```julia
A = [1 1e-10; 0 1]
L, X = eigen(A)
#cond(X)
```

    Eigen{Float64, Float64, Matrix{Float64}, Vector{Float64}}
    values:
    2-element Vector{Float64}:
     1.0
     1.0
    vectors:
    2×2 Matrix{Float64}:
     1.0  -1.0
     0.0   2.22045e-6

# Aside: Schur decomposition

We can repair these weaknesses by using the [Schur decomposition](https://en.wikipedia.org/wiki/Schur_decomposition)
$$ Q R Q^* = I - \omega A $$
where $R$ is right-triangular and $Q$ is unitary (i.e., orthogonal if real-valued; $Q^*$ is the Hermitian conjugate of $Q$).
The Schur decomposition always exists and $Q$ has a condition number of 1.

* Where are the eigenvalues in $R$?

Evidently we must find $\omega$ to minimize the maximum eigenvalue of $I - \omega A$.  We can do this if $A$ is well conditioned, but not in general.

# Ill-conditioning

## Question: What is the condition number of the Laplacian on 100 evenly spaced points?

* How does it scale under grid refinement?

```julia
A = advdiff_matrix(20)
cond(Matrix(A))
```

    178.06427461085948

```julia
omega = .008 / 4
ev = eigvals(Matrix(I - omega * A))
scatter(real.(ev), imag.(ev))
```

# Monic polynomials small on the spectrum

Equivalently to finding $\omega$ such that $\lVert I - \omega A \rVert$ is minimized, we may seek a monic polynomial $p(z) = 1 - \omega z$ that minimizes

$$ \max_{\lambda \in \sigma(A)} \lvert p(\lambda) \rvert . $$

This concept can be extended to higher degree polynomials, which is essentially what Krylov methods do (discovering the polynomial adaptively, weighted by the right hand side).

```julia
ev = eigvals(Matrix(A))
scatter(real.(ev), zero.(ev))
plot!(x -> 1 - omega * x)
```

# Preconditioning

Preconditioning is the act of creating an "affordable" operation "$P^{-1}$" such that $P^{-1} A$ (or $A P^{-1}$) is is well-conditoned or otherwise has a "nice" spectrum.  We then solve the system

$$ P^{-1} A x = P^{-1} b \quad \text{or}\quad A P^{-1} \underbrace{(P x)}_y = b $$

in which case the convergence rate depends on the spectrum of the iteration matrix
$$ I - \omega P^{-1} A . $$

* The preconditioner must be applied on each iteration.
* It is *not* merely about finding a good initial guess.

There are two complementary techniques necessary for efficient iterative methods:

* "accelerators" or Krylov methods, which use orthogonality to adaptively converge faster than Richardson
* preconditioners that improve the spectrum of the preconditioned operator

Although there is ongoing research in Krylov methods and they are immensely useful, I would say preconditioning is 90% of the game for practical applications, particularly as a research area.

# Krylov subspaces

All matrix iterations work with approximations in a *Krylov subspace*, which has the form

$$ K_n = \big[ b \big| Ab \big| A^2 b \big| \dotsm \big| A^{n-1} b \big] . $$

This matrix is horribly ill-conditioned and cannot stably be computed as written.  Instead, we seek an orthogonal basis $Q_n$ that spans the same space as $K_n$.  We could write this as a factorization

$$ K_n = Q_n R_n $$

where the first column $q_0 = b / \lVert b \rVert$.  The $R_n$ is unnecessary and hopelessly ill-conditioned, so a slightly different procedure is used.

# Arnoldi iteration

The Arnoldi iteration applies orthogonal similarity transformations to reduce $A$ to [Hessenberg form](https://en.wikipedia.org/wiki/Hessenberg_matrix), starting from a vector $q_0 = b$,

$$ A = Q H Q^* . $$

Let's multiply on the right by $Q$ and examine the first $n$ columns,

$$ A Q_n = Q_{n+1} H_n $$
where $H_n$ is an $(n+1) \times n$ Hessenberg matrix.

# Aside: Conditioning of Arnoldi process
The Arnoldi process is well-conditioned because $Q$ is orthogonal and

$$ \lVert H_n \rVert \le \lVert Q_{n+1}^* \rVert \lVert A \rVert \lVert Q_n \rVert \le \lVert A \rVert $$.

For a lower bound, we have

$$ \sigma_{\min}(A)^2 \le x^* A^* A x $$

for all $x$ of norm 1.  It must also be true for any $x = Q_n y$ where $\lVert y\rVert = 1$, thus

$$ \sigma_{\min}(A)^2 \le y^* Q_n^* A^* A Q_n y = y^* H_n^* Q_{n+1}^* Q_{n+1} H_n y = y^* H_n^* H_n y . $$

# GMRES
$$ A Q_n = Q_{n+1} H_n $$

GMRES (Generalized Minimum Residual) minimizes
$$ \lVert A x - b \rVert $$
over the subspace $Q_n$.  I.e., $x = Q_n y$ for some $y$.  By the Arnoldi recurrence, this is equivalent to
$$ \lVert Q_{n+1} H_n y - b \lVert $$
which can be solved by minimizing
$$ \lVert H_n y - Q_{n+1}^* b \rVert . $$
Since $q_0 = b/\lVert b \lVert$, the least squares problem is to minimize
$$ \Big\lVert H_n y - \lVert b \rVert e_0 \Big\rVert . $$
The solution of this least squares problem is achieved by incrementally updating a $QR$ factorization of $H_n$.

## Notes
* The solution $x_n$ constructed by GMRES at iteration $n$ is not explicitly available.  If a solution is needed, it must be constructed by solving the $(n+1)\times n$ least squares problem and forming the solution as a linear combination of the $n$ vectors $Q_n$.  The leading cost is $2mn$ where $m \gg n$.
* The residual vector $r_n = A x_n - b$ is not explicitly available in GMRES.  To compute it, first build the solution $x_n$ as above.
* GMRES minimizes the 2-norm of the residual $\lVert r_n \rVert$ which is equivalent to the $A^* A$ norm of the error $\lVert x_n - x_* \rVert_{A^* A}$.

# More notes on GMRES

* GMRES needs to store the full $Q_n$, which is unaffordable for large $n$ (many iterations).  The standard solution is to choose a "restart" $k$ and to discard $Q_n$ and start over with an initial guess $x_k$ after each $k$ iterations.  This algorithm is called GMRES(k).  PETSc's default solver is GMRES(30) and the restart can be controlled using the run-time option `-ksp_gmres_restart`.
* Most implementations of GMRES use classical Gram-Schmidt because it is much faster in parallel (one reduction per iteration instead of $n$ reductions per iteration).  The PETSc option `-ksp_gmres_modifiedgramschmidt` can be used when you suspect that classical Gram-Schmidt may be causing instability.
* There is a very similar (and older) algorithm called GCR that maintains $x_n$ and $r_n$.  This is useful, for example, if a convergence tolerance needs to inspect individual entries.  GCR requires $2n$ vectors instead of $n$ vectors, and can tolerate a nonlinear preconditioner.  FGMRES is a newer algorithm with similar properties to GCR.

# Experiments in [PETSc](https://petsc.org)

* PETSc = Portable Extensible Toolkit for Scientific computing
* `./configure`, `make`
  * Depends on BLAS and LAPACK (default system or package manager)
    * or `./configure --download-f2cblaslapack --download-blis`
  * Depends on MPI for parallelism (package manager)
    * or `./configure --download-mpich` (or `--download-openmpi`)
* `docker run -it --rm jedbrown/petsc`
* Lots of examples (mostly C and Fortran, some Python)
* Experimental bindings in Rust, Julia
* We'll use `src/snes/tutorials/ex15.c`
  * "p-Bratu": combines p-Laplacian with Bratu nonlinearity
  $$ -\nabla\cdot\big(\kappa(\nabla u) \nabla u\big) - \lambda e^u = f $$
  * Newton solver with Krylov on each iteration

# Simple preconditioners

## Jacobi `-pc_type jacobi`

$$ P_{\text{Jacobi}}^{-1} = D^{-1} $$
where $D$ is the diagonal of $A$.

## Gauss-Seidel `-pc_type sor`

$$ P_{GS}^{-1} = (L+D)^{-1} $$
where $L$ is the (strictly) lower triangular part of $A$.  The upper triangular part may be used instead, or a symmetric form
$$ P_{SGS}^{-1} = (L+U)^{-1} A \Big( I - (L+D)^{-1} \Big) . $$

### Over-relaxation

`-pc_sor_omega 1.` is default (Gauss-Seidel)

## Run p-Laplacian example
```console
$ cd src/snes/tutorials
$ make ex15
$ ./ex15 -da_refine 2 -dm_view
$ ./ex15 -ksp_monitor -pc_type jacobi
$ ./ex15 -snes_view
```

## Experiments

* How does iteration count vary under grid refinement?
* How sensitive is it to parameters
  * p-Laplacian `-p` $> 1$ and `-epsilon` $> 0$
  * Bratu `-lambda` $< 6.8$
* How sensitive to `-ksp_gmres_restart`?
* `-ksp_monitor_true_residual`

# Symmetric problems

## Lanczos iteration: like GMRES for symmetric problems

If $A$ is symmetric, then $H = Q^T A Q$ is also symmetric.  Since $H$ is Hessenberg, this means $H$ is tridiagonal.  Instead of storing $Q_n$, it is sufficient to store only the last two columns since the iteration satisfies a 3-term recurrence.  The analog of GMRES for the symmetric case is called MINRES and is also useful for solving symmetric indefinite problems.

## Conjugate Gradients: changing the norm

Instead of minimizing the $A^T A$ norm of the error, the Conjugate Gradient method minimizes the $A$ norm of the error.  For $A$ to induce a norm, it must be symmetric positive definite.  [Jeremy Shewchuck's guide to CG](http://www.cs.cmu.edu/%7Equake-papers/painless-conjugate-gradient.pdf) is an excellent resource.


# Overlapping Generations Model (OLG)
### The Usefulness of the Model
The key ingredient of the OLG model is exogenous population turnover. Individuals in the population live for two periods, one in which they work and save, and the second in which they are 'old', do not work, but rather live off their savings.
The model can be used to assess various systems of 'social security'; is there a place for the government in providing for the older generation?
In the following, we describe the basic model setup, and go through how to solve the model analytically. A reader who is familiar with the OLG model may choose to simple skip these next steps and go straight to our coded implementation - our model setup is entirely standard, and as presented e.g. in Macro III at KU.

### Model Setup
Time is discrete and infinite, $t = 0,1,2,...$.
Population grows at a constant rate, n:
    $$ L_t = L_{t-1}(1+n)$$
This implies, that in every period t, there are $L_t$ 'young' (working) individuals, and $L_{t-1}$ 'old' (non-working) individuals.  Agents derive utility from consumption in each of the two periods they are alive:
    $$U_t = u(c_{1t})+\frac{1}{1+\rho} \cdot u(c_{2t+1})$$
Where $c_{1t}$ is consumption of the young at time t, $c_{2t+1}$ is consumption of old at time $t+1$ and $\rho$ is a discount factor.
Let $r_{t+1}$ denote the interest rate between $t$ and $t+1$. Then individuals are budget constrained in each period of their live as follows:
    $$c_{1t} + s_{t} = w_t$$
    $$c_{2t+1} = (1+r_{t+1})\cdot s_t$$
Substituting for s, we get the following life-time constraint:
    $$c_{1+t} + \frac{c_{2t+1}}{1+r_{t+1}} = w_t$$

In every period t, agents born at time t solve the following problem:
    $$\max_{c_{1t},c_{2t+1}} u(c_{1t})+\frac{1}{1+\rho} u(c_{2t+1})$$
Subject to the constraint from above:
    $$c_{1+t} + \frac{c_{2t+1}}{1+r_{t+1}} = w_t$$

Production is assumed to take place with CRS technology, competitive markets and profit-maximizing firms. This yields:
    $$r_t = f'(k_t)$$
    $$w_t = f(k_t)-f'(k_t)k_t$$
Where $k_t = \frac{K_t}{L_t}$. Recall, that $L_t$ refers to the *working population* at time t.

### Solving the Household Problem
The household problem is a standard optimization problem subject to a constraint. We set up the lagrangian:
    $$ L(c_{1t},c_{2t+1},\lambda) = u(c_{1t}) + \frac{1}{1+\rho} u(c_{2t+1}) + \lambda[w_t-c_{1t}-\frac{c_{2t+1}}{1+r_{t+1}}]$$
Differentiate wrt. consumption (in each period):
    $$\frac{\partial L}{\partial c_{1t}} = u'(c_{1t}) -\lambda $$
    $$\frac{\partial L}{\partial c_{2t+1}} = \frac{1}{1+\rho} u'(c_{2t+1}) -\frac{\lambda}{1+r_{t+1}} $$
Equate to zero (first order conditions) and substitute for the lagrange multiplier $\lambda$ to obtain the Euler Equation:
    $$u'(c_{1t}) = \frac{1+r_{t+1}}{1+\rho} u'(c_{2t+1})$$

### Characterizing Optimal Savings
Substitute the budget constrains into the Euler Equation to get:
    $$u'(w_t-s_t) = \frac{1+r_{t+1}}{1+\rho} u'((1+r_{t+1})s_t)$$
This implicitly defines optimal savings as a function of wage and interest rate, i.e. $s(w_t,r_{t+1})$

### Law of Motion for Capital
Knowing the individual decisions allow us to aggregate economy-wide. The aggregate capital stock at time $t+1$ is equal to aggregate savings in time $t$, aggregate dissaving in time t, and un-depreciated capital carried over from time t.
    $$K_{t+1} = S_t -K_t+(1-\delta)K_t$$
    $$\Rightarrow K_{t+1} = S_t - \delta K_t$$
        $$\Rightarrow k_{t+1}(1+n) = s_t-\delta k_t$$

Substituting for savings, substituting for wage and interest rate, and assuming $\delta = 0$:
    $$k_{t+1} (1+n) = s(f(k_t)-k_tf'(k_t),f'(k_{t+1})$$
Which implicitly defines the law of motion for aggregate capital per worker.

```python
import numpy as np
import sympy as sp
import matplotlib.pyplot as plt

from modelproject.model import *
```

```python
sp.init_printing()
```

We solve the OLG model using sympy, thus we begin by setting up symbols for the variables we use

```python
rho = sp.symbols('rho')
delta = sp.symbols('delta')
n = sp.symbols('n')

c1 = sp.symbols('c_{1t}')
c2 = sp.symbols('c_{2t+1}')

r = sp.symbols('r_{t+1}')
w = sp.symbols('w_t')

s = sp.symbols('s_t')
k = sp.symbols('k_t')
kp1 = sp.symbols('k_{t+1}')

lmbda = sp.symbols('lambda')
```

next we define the standard log utility, and a placeholder that keeps the code general for the derivations

```python
def log_u(c):
    return sp.log(c)

u_undetermined = sp.Function('u')

def U(c1, c2, rho, u):
    return u(c1) + 1 / (1 + rho) * u(c2)

U(c1, c2, rho, u = u_undetermined)
```

We define budget constraints and calculate the intertemporal constraint

```python
period_1_budget = sp.Eq(w, c1 + s)
period_2_budget = sp.Eq(c2, (1+r)*s)

period_1_budget, period_2_budget
```

```python
inter_budget = period_1_budget.subs(s, sp.solve(period_2_budget, s)[0])
inter_budget
```

next we set up the lagrangian and calculate the relevant first derivatives

```python
lagrangian = U(c1,c2,rho, u = u_undetermined) - lmbda *  (inter_budget.rhs - w)
lagrangian
```

```python
dc1    = sp.diff(lagrangian, c1)
dc2    = sp.diff(lagrangian, c2)
dlmbda = sp.diff(lagrangian, lmbda)

dc1, dc2, dlmbda
```

Calculate euler equation

Finally we call our `make_euler_equation` function with the derivatives of the lagrangian as inputs. We the nsubstitute in the within period budget constraints to gen an exproession in terms of savings wages and interest rate.

```python
euler_eq = make_euler_equation(dc1, dc2, u = u_undetermined, c1 = c1, lmbda = lmbda)
euler_eq
```

```python
euler_eq = euler_eq.subs(c1, w - s).subs(c2, period_2_budget.rhs)
euler_eq
```

Implicitly the euler equation determines $s$, now we move to determine the evolution of captial in the economy.

This is also a good spot to set the actual functional form of $u$, we use a $log$ function here as it is analytically solveable, and explore using the square root further down.

```python
euler_eq = euler_eq.replace(u_undetermined, log_u).doit()
euler_eq
```

Next setup for firms. We show the aggregate production function but the work with the per capita normalized version to calculate expressions for the equilibrium interest rates and wages.

```python
Y = sp.symbols('Y_t')
K = sp.symbols('K_t')
L = sp.symbols('L_t')

y = sp.symbols('y_t')

alpha = sp.symbols('alpha')

production_function = sp.Eq(
    Y,
    K**alpha * L**(1-alpha)
)
production_function

```

```python
norm_prod_func = sp.Eq(y, k**alpha)
norm_prod_func
```

Define interest rate and wage

```python
interest_rate = sp.Eq(r, sp.Derivative(norm_prod_func.rhs, k)).doit()
wage = sp.Eq(w, norm_prod_func.rhs - k*sp.Derivative(norm_prod_func.rhs, k)).doit()

interest_rate, wage
```

Finally from a standard capital depreciation identity we derive a first expression for the evolution of capital in the economy.

```python
evolution = sp.Eq(kp1, 1/(1+n)* (s - delta*k) )
evolution
```

Using the euler equation and the capital evolution path we can derive the transition path of the economy

```python
savings_of_k = sp.solve(euler_eq, s)[0].subs(w, wage.rhs)

# Only substitute
if r in savings_of_k.atoms():
    savings_of_k = savings_of_k.subs(r, interest_rate.rhs)

transition_eq = evolution.subs(s, savings_of_k)
transition_eq
```

In the following section we show visualizations of the capital accumulation path for various parametrizations, including varying $\alpha$ and $\n$.

```python
_transition_func = sp.lambdify((k, rho, delta, n, alpha, r), transition_eq.rhs)

def transition_func(k, alpha, rho, delta, n, r = 0):
    return _transition_func(k, rho, delta, n , alpha, r)
```

```python
_equilibrium = sp.lambdify(
    (alpha, n, rho, delta),
    sp.solve(transition_eq.subs(kp1, k), k)[0]
            )

def equilibrium(alpha, n, rho, delta):
    return _equilibrium(alpha, n, rho, delta)
```

Visualizing variation in $\alpha$

```python
xr = np.linspace(0,2,1000)
_r = 0.05
_d = 0.02
_n = 0.02

for _a in np.linspace(0.1, 0.5, 5):

    res = [transition_func(k = x, alpha = _a, rho = _r, delta = _d, n = _n) for x in xr]
    kstar = equilibrium(alpha = _a, rho = _r, n = _n, delta = _d)
    plt.plot(xr, res, color = 'blue', alpha = 1)
    plt.scatter([kstar], [kstar], color = 'red')
    plt.annotate(f'$\\alpha$={round(_a,2)}', xy= (kstar + 0.01, kstar - 0.02))

plt.plot(xr, xr, color = 'black')
plt.xlim(0,0.5)
plt.ylim(0,0.5)
plt.xlabel('$k_t$')
plt.ylabel('$k_{t+1}$')
plt.title('Transition curves for varying $\\alpha$')
```

Visualizing variation in $n$

```python
xr = np.linspace(0,2,1000)
_r = 0.05
_d = 0.02
_a = 0.2
#_n = 0.02

for _n in np.linspace(0, 1, 5):

    res = [transition_func(k = x, alpha = _a, rho = _r, delta = _d, n = _n) for x in xr]
    kstar = equilibrium(alpha = _a, rho = _r, n = _n, delta = _d)
    plt.plot(xr, res, color = 'blue', alpha = 1)
    plt.scatter([kstar], [kstar], color = 'red')
    plt.annotate(f'$n$={round(_n,2)}', xy= (kstar + 0.01, kstar - 0.02))

plt.plot(xr, xr, color = 'black')
plt.xlim(0,0.5)
plt.ylim(0,0.5)
plt.xlabel('$k_t$')
plt.ylabel('$k_{t+1}$')
plt.title('Transition curves for varying $n$')
```

Visualizing the convergence to st.st.

```python
k_ = 0.0001
xr = range(10)
xr2 = np.linspace(0,1,1000)

_r = 0.05
_d = 0.05
_a = 0.5
_n = 0

out = list()

for _ in xr:
    k_ = transition_func(k = k_, alpha = _a, rho = _r, delta = _d, n = _n)
    out.append(k_)

```

```python
res = [transition_func(k = x, alpha = _a, rho = _r, delta = _d, n = _n) for x in xr2]

plt.plot(xr, xr, color = 'black')
plt.plot(xr2, res, color = 'blue', alpha = 1)
plt.step(out[:-1], out[1:], where = 'post', color = 'red', linestyle = '--', alpha = 0.8)
plt.scatter(out[:-1], out[1:], color = 'red', alpha = 0.8)

plt.xlim(0,0.1)
plt.ylim(0,0.1)
```

## Part 2 - numerical solution with sqrt utility

In the section below we first show that there is no analytical solution to the OLG model when the within period utility is given by $u(c)=\sqrt(c)$. We then show graphically that even though no analytical solution exists, it can be found numerically, and we iteratively identify the aproximate value of $k^*$ with square root utility.

The first steps are identical to the ones covered above.

```python
def sqrt_u(c):
    return sp.sqrt(c)
```

```python
euler_eq = make_euler_equation(dc1, dc2, u = u_undetermined, c1 = c1, lmbda = lmbda)
euler_eq = euler_eq.subs(c1, w - s).subs(c2, period_2_budget.rhs)
euler_eq = euler_eq.replace(u_undetermined, sqrt_u).doit()
euler_eq
```

```python
savings_of_k = sp.solve(euler_eq, s)[0].subs(w, wage.rhs)
savings_of_k
```

```python
savings_of_k = savings_of_k.subs(r, interest_rate.rhs)
savings_of_k
```

```python
transition_eq = evolution.subs(s, savings_of_k)
transition_eq
```

```python
_tf2 = sp.lambdify((k, alpha, rho, delta, n), transition_eq.rhs)
def tf2(k, alpha, rho, delta, n):
    return _tf2(k, alpha, rho, delta, n)

```

Notice now the analytical solution is empty.

```python
sp.solve(transition_eq.subs(kp1, k), k)
```

Here we compare the steady state of a model with log utility and one with square root utility, notice how the change in utility drastically changes the steady state level of capital.

```python
k_ = 0.0001
k_2 = 0.0001

xr = range(10)
xr2 = np.linspace(0,1,1000)

_r = 0.05
_d = 0.05
_a = 0.5
_n = 0

out = list()
out2 = list()

for _ in xr:
    k_ = tf2(k = k_, alpha = _a, rho = _r, delta = _d, n = _n)
    k_2 = transition_func(k = k_2, alpha = _a, rho = _r, delta = _d, n = _n)

    out.append(k_)
    out2.append(k_2)

res = [tf2(k = x, alpha = _a, rho = _r, delta = _d, n = _n) for x in xr2]
res2 = [transition_func(k = x, alpha = _a, rho = _r, delta = _d, n = _n) for x in xr2]

plt.plot(xr, xr, color = 'black')
plt.plot(xr2, res, color = 'blue', alpha = 1, label = 'sqrt')
plt.plot(xr2, res2, color = 'navy', alpha = 1, label = 'log')

plt.step(out[:-1], out[1:], where = 'post', color = 'red', linestyle = '--', alpha = 0.8)
plt.scatter(out[:-1], out[1:], color = 'red', alpha = 0.8)

plt.step(out2[:-1], out2[1:], where = 'post', color = 'red', linestyle = '--', alpha = 0.8)
plt.scatter(out2[:-1], out2[1:], color = 'red', alpha = 0.8)

plt.xlim(0,0.2)
plt.ylim(0,0.2)

plt.legend()

```

Finally using the simple convergence algorithm shown in the figure we can approximate the steady state level of capital in an economy characterized by square root utility.

```python
k_ = 0.0001
for _ in range(10000):
    k_ = tf2(k = k_, alpha = _a, rho = _r, delta = _d, n = _n)

print(k_)
```

    0.10949660035925014


```python
import numpy as np
import matplotlib.pyplot as plt
```

```python

```

## Referências

- Probabilidade e Variavéis Aleatórias, Magalhães M. EDUSP(2006)
- Grinstead & Snell Introduction's to Probability, Grinstead & Snell. CHANGE PROJECT

# Modelo Uniforme Discreto

Seja $X$ uma variável discreta cujos resultados dos eventos sejam equiprováveis, dizemos que $X$ tem uma distribuição uniforme e usamos a notação $X \tilde U_d$
Ou seja, sejam os eventos
$$
x_1, x_2, \dots, x_n
$$
com função de probabilidade dada por
$$
p(x_i) = \frac{1}{n}, \, i \in 1,2,\dots, n
$$

```python
def distUniform(x):
    n = len(x)
    return [1/n for i in x]
```

```python
x = np.linspace(0,50, 51)
```

```python
y = distUniform(x)
```

```python
plt.figure(figsize= (20,10))
plt.bar(x,y)
plt.title('Distribuição Uniforme para uma variável aleatória $X$ que assume valores entre 0 e 50')
plt.show
```

## Esperança da distribuição
Seja $n$ a quantidade de resultados do experimento, com seus resultados discretamente distribuidos em $[a,b]$, com $a,b$ fazendo parte e sendo respectivamente o menor e maior valor, temos:
$$
\begin{align}
\mathbb{E}(X) &= \sum_{i=1}^{n} x_i p(x_i)\\
&= \sum_{i=1}^{n} x_i \frac{1}{n}\\
&= \frac{1}{n} \sum_{i=1}^{n} x_i\\
&= \frac{1}{n} \frac{n(a+b)}{2}\\
&= \frac{a+b}{2}
\end{align}
$$

```python

```


# Symbolic Mathematics in Python

There are times when you need to solve a difficult problem symbollically or analytically. If you have ever used Wolfram Alpha, then you have already done this. Sympy is a python library that allows you to do symbolic mathematics in python.

```python
import sympy as sym
```

## 1. Introduction

### Example 1.1

If you try to write the follwing in python by itself, you will get an error telling you x is undefined:
$$x-x$$

```python
x-x
```

(The error above is on purpose). Variables in python need to be defined before you can say something specific about them

```python
x=102
x-x
```

If you are trying to show that $x-x=0$ is true for any $x$, the above answer would not be valid. Instead you can use a symbolic expression to show that it is true

**First we define the variable as a symmbolic expression**

```python
x = sym.symbols('x')
```

**Now we can use the variable in a symbolic expression**

```python
x-x
```

### Example 1.2

Sympy can be used to perform algebreic operations (among other things). Consider the following expression: $$(3a-4b)^3$$

We can use symppy to expand the expression algebraically.

**First we need to define the variables as symbolic expressions**

```python
a,b = sym.symbols('a,b')
```

**Side note** Notice that the left hand side of the epression has two variables being defined. Python can define more than one variable at a time:

```python
x1,y1 =10,20
print(x1)
print(y1)
```

    10
    20

**Back to the expression** We can define an expression using the variables $a$ and $b$.

```python
expr = (3*a-4*b)**3
print(expr)
```

    (3*a - 4*b)**3

We can also make it look nicer in our notebook. This doesn't affect the math, but it makes our notebook more readable.

```python
sym.init_printing()
```

```python
expr
```

**Now we expand the function algebreically**

```python
expr.expand()
```

Sympy can also factor the equation

```python
sym.factor(26*a**3-108*a**2*b+144*a*b**2-64*b**3)
```

If you want to copy and paste a result, you print the result.

```python
print(sym.factor(26*a**3-108*a**2*b+144*a*b**2-64*b**3))
```

    2*(a - 2*b)*(13*a**2 - 28*a*b + 16*b**2)

You can also chain together functions

```python
expr.expand().factor()
```

### Exercise 1.1

Show that the following two expressions are true.
$$(2w-3z)(2w+3z)=4w^2-9z^2$$
$$(2w-3z)^2\ne4w^2-9z^2$$

```python
import sympy as sym
w,z = sym.symbols('w,z')
funceq = (2*w - 3*z)*(2*w + 3*z)
funceqe = funceq.expand()
sym.init_printing()
funceqe
```

```python
import sympy as sym
w,z = sym.symbols('w,z')
funcwrong = (2*w - 3*z)**2
funcwronge = funcwrong.expand()
sym.init_printing()
funcwronge
```

## 2. Solving Equations

Sympy can be used to symbolilically solve equations. As before, you need to define which variables are symbols

### Example 2.1

Use sympy to solve the following equation
$$ax^3+bx^2+cx+d=0$$

```python
# Define the variables
a,b,c,d,x = sym.symbols('a,b,c,d,x')
```

```python
# Define the expression
expr=a*x**3+b*x**2+c*x+d
expr
```

We can use the `solvset` function to solve this equation

```python
solutions=sym.solveset(expr,x)
```

```python
print(solutions)
```

    {-(-3*c/a + b**2/a**2)/(3*(sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)) - (sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)/3 - b/(3*a), -(-3*c/a + b**2/a**2)/(3*(-1/2 - sqrt(3)*I/2)*(sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)) - (-1/2 - sqrt(3)*I/2)*(sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)/3 - b/(3*a), -(-3*c/a + b**2/a**2)/(3*(-1/2 + sqrt(3)*I/2)*(sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)) - (-1/2 + sqrt(3)*I/2)*(sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)/3 - b/(3*a)}

```python
solutions
```

What if I need help. You can do this with any python function. `function?`

```python
# Run this command to see a help box
sym.solveset?
```

### Exercise 2.1

Use the `solveset` function to solve the following chemical problem.

Phosgene gas, $\text{COCl}_2$, dissociates at high temperatures according to the following equilibrium:

$$ \text{COCl}_2 \rightleftharpoons  \text{CO} + \text{Cl}_2 $$

At $\text{400 C}$, the equilibrium constant $K_c=8.05$.

If you start with a  $\text{0.250 M}$ phosgene sample at $\text{400 C}$, determine the concentrations of all species at equilibrium.

```python
import sympy as sym
x = sym.symbols('x')
solve = ((x**2)/(0.250 - x)) - 8.05
eq = sym.solveset(solve,x)
eq
```

Why did you pick your answer?

## 3. Calculus

We can use also Sympy to differentiate and integrate. Let us experiment with differentiating the following expression:

$$x ^ 2 - \cos(x)$$

```python
sym.diff(x ** 2 - sym.cos(x), x)
```

Similarly we can integrate:

```python
sym.integrate(x ** 2 - sym.cos(x), x)
```

We can also carry out definite integrals:

```python
sym.integrate(x ** 2 - sym.cos(x), (x, 0, 5))
```

### Exercise 3.1

Use Sympy to calculate the following:

1. $\frac{d(x ^2 + xy - \ln(y))}{dy}$
1. $\int_0^5 e^{2x}\;dx$

```python
import sympy as sym
import numpy as num
import math

x,y = sym.symbols('x,y')

derive = (x**2 + x*y - sym.ln(y))
sym.diff(derive,x)
```

$\displaystyle 2 x + y$

```python
import sympy as sym
import numpy as num
import math
from sympy import exp

x = sym.symbols('x')

integ = exp(2*x)
sym.integrate(integ,(x,0,5))
```

### Exercise 3.2
Solve the following definate integral
$$\int\limits_{ - \infty }^\infty  {\frac{1}{{\sigma \sqrt {2\pi } }}{e^{ - \frac{1}{2}{{\left( {\frac{{x - \mu }}{\sigma }} \right)}^2}}}}$$

Hint, the sympy symbol for infinity is `oo`

```python
import sympy as sym
import numpy as num
import math
from sympy import exp, sqrt

x,b,c = sym.symbols('x,b,c')

g1 = (1/(c*sqrt(2*num.pi)))
g2 = exp((-1/2)*((x - b)/c)**2)
gaussian = g1*g2

sym.integrate(gaussian,(x,'-oo','oo'))

```

$\displaystyle \begin{cases} 0.282094791773878 \sqrt{\pi} \left(2 - \operatorname{erfc}{\left(\frac{0.707106781186548 b}{c} \right)}\right) + 0.282094791773878 \sqrt{\pi} \operatorname{erfc}{\left(\frac{0.707106781186548 b}{c} \right)} & \text{for}\: \left(\left(2 \left|{\arg{\left(c \right)}}\right| \leq \frac{\pi}{2} \wedge \left|{2 \arg{\left(b \right)} - 4 \arg{\left(c \right)}}\right| < \pi\right) \vee \left(\left|{2 \arg{\left(b \right)} - 4 \arg{\left(c \right)}}\right| \leq \pi \wedge 2 \left|{\arg{\left(c \right)}}\right| < \frac{\pi}{2}\right) \vee \left(\left|{2 \arg{\left(b \right)} - 4 \arg{\left(c \right)}}\right| < \pi \wedge 2 \left|{\arg{\left(c \right)}}\right| < \frac{\pi}{2}\right) \vee 2 \left|{\arg{\left(c \right)}}\right| < \frac{\pi}{2}\right) \wedge \left(\left(2 \left|{\arg{\left(c \right)}}\right| \leq \frac{\pi}{2} \wedge \left|{2 \arg{\left(b \right)} - 4 \arg{\left(c \right)} + 2 \pi}\right| < \pi\right) \vee \left(\left|{2 \arg{\left(b \right)} - 4 \arg{\left(c \right)} + 2 \pi}\right| \leq \pi \wedge 2 \left|{\arg{\left(c \right)}}\right| < \frac{\pi}{2}\right) \vee \left(\left|{2 \arg{\left(b \right)} - 4 \arg{\left(c \right)} + 2 \pi}\right| < \pi \wedge 2 \left|{\arg{\left(c \right)}}\right| < \frac{\pi}{2}\right) \vee 2 \left|{\arg{\left(c \right)}}\right| < \frac{\pi}{2}\right) \\\int\limits_{-\infty}^{\infty} \frac{0.398942280401433 e^{- \frac{0.5 \left(- b + x\right)^{2}}{c^{2}}}}{c}\, dx & \text{otherwise} \end{cases}$

Lookup Gaussian functions: https://en.wikipedia.org/wiki/Gaussian_function
Does your answer maake sense?

## 4. Plotting with Sympy

Finally Sympy can be used to plot functions. Note that this makes use of [matplotlib](http://matplotlib.org/).

Let us plot $x^2$:

```python
expr = x ** 2
p = sym.plot(expr)
```

### Exercise 4.1 Plot the following function:

1. $y=x + cos(x)$
1. ${\frac{1}{{ \sqrt {2\pi } }}{e^{ - \frac{x^2}{2}}}}$

```python
import sympy as sym
import numpy as num

x = sym.symbols('x')
e1 = x + sym.cos(x)
p1 = sym.plot(e1)
```

```python
import sympy as sym
import numpy as num
from sympy import exp, sqrt

x = sym.symbols('x')
e2 = (1/sqrt(2*sym.pi)*exp((-x**2)/2))
p2 = sym.plot(e2)
```

# Lecture

## L1. Hydrogen Atom

Sympy has built in modules for the eigenfunctions of the hydrogen atom.

```python
import sympy.physics.hydrogen
import numpy as np
```

You can caluclate the eigenvalues ($E$) in Hartrees

`sym.physics.hydrogen.E_nl(n,Z)`

```python
sym.physics.hydrogen.E_nl(1,1)
```

$\displaystyle - \frac{1}{2}$

We can use a loop to print out many energies

```python
for n in range(1,5):
    print(sym.physics.hydrogen.E_nl(n,1))
```

    -1/2
    -1/8
    -1/18
    -1/32

We can plot the hydrogen radial wavefunction (1s orbital)

```python
r = sympy.symbols('r')
sympy.physics.hydrogen.R_nl(1, 0, r, 1)
```

$\displaystyle 2 e^{- r}$

```python
sym.plot(sympy.physics.hydrogen.R_nl(1, 0, r, 1),(r,0,10.50))
```

And the probablity distribution function

```python
sympy.symbols('r')
prob_1s=sympy.physics.hydrogen.R_nl(1, 0, r, 1)*sympy.physics.hydrogen.R_nl(1, 0, r, 1)
prob_1s
```

$\displaystyle 4 e^{- 2 r}$

```python
sym.plot(prob_1s,(r,0,10))
```

Plot a 2s orbital

```python
sympy.symbols('r')
prob_2s=sympy.physics.hydrogen.R_nl(2, 0, r, 1)*sympy.physics.hydrogen.R_nl(2, 0, r, 1)
prob_2s
```

$\displaystyle \frac{\left(2 - r\right)^{2} e^{- r}}{8}$

```python
sym.plot(prob_2s,(r,0,10))
```

We can change the range to see the node better.

```python
sym.plot(prob_2s,(r,1,8))
```

Notice the node!

### Exercise L1.1

Plot the radial distriubution function for a 2p, 3s, 4s, and 3d orbital.

```python
import sympy.physics.hydrogen
import numpy as np

r = sympy.symbols('r')
sympy.physics.hydrogen.R_nl(2, 1, r, 1)
prob_2p = sympy.physics.hydrogen.R_nl(2, 1, r, 1)*sympy.physics.hydrogen.R_nl(2, 1, r, 1)
prob_2p
sym.plot(prob_2p,(r,0,10))
```

```python
import sympy.physics.hydrogen
import numpy as np

r = sympy.symbols('r')
sympy.physics.hydrogen.R_nl(2, 1, r, 1)
prob_3s = sympy.physics.hydrogen.R_nl(3, 0, r, 1)*sympy.physics.hydrogen.R_nl(3, 0, r, 1)
prob_3s
sym.plot(prob_3s,(r,2,10))
```

```python
import sympy.physics.hydrogen
import numpy as np

r = sympy.symbols('r')
sympy.physics.hydrogen.R_nl(4, 0, r, 1)
prob_4s = sympy.physics.hydrogen.R_nl(4, 0, r, 1)*sympy.physics.hydrogen.R_nl(4, 0, r, 1)
prob_4s
sym.plot(prob_4s,(r,5,16))
```

```python
import sympy.physics.hydrogen
import numpy as np

r = sympy.symbols('r')
sympy.physics.hydrogen.R_nl(3, 2, r, 1)
prob_3d = sympy.physics.hydrogen.R_nl(3, 2, r, 1)*sympy.physics.hydrogen.R_nl(3, 2, r, 1)
prob_3d
sym.plot(prob_3d,(r,0,25))
```


# Relative Entropy in Gaussian networks
Ro Jefferson<br>
Last updated 2021-11-03

This notebook is designed to compute the relative entropy for networks generated by "Gaussian_Feedforward.ipynb". It reads in hdf5 data (weights, biases, and pre-activations) from a pre-trained network, and evaluates the relative entropy or Kullback-Leibler (KL) divergence as a function of depth. **The user must specify** the `PATH_TO_DATA` below.

Take the reference distribution to be that of the pre-activations in layer $\ell$, denoted $p(z^\ell)=p^\ell$ where $z^\ell=\{z_0^\ell,\ldots,z_{n_\ell}^\ell\}$, and the approximate or model distribution to be that for layer $\ell+m$, denoted $q(z^{\ell+m})=q^{\ell+m}$. The KL divergence is then

\begin{equation}
D(p^\ell||q^{\ell+m})=\int\!\mathrm{d} z^\ell\,p(z^\ell)\ln\frac{p(z^\ell)}{q(z^{\ell+m})}
=\langle\ln p(z^\ell)\rangle_p-\langle q(z^{\ell+m})\rangle_p~.
\end{equation}

As detailed in the corresponding paper, this yields three terms:
\begin{equation}
D(p^\ell||q^{\ell+m})=-\frac{1}{2}\ln|2\pi e\Sigma_p|
+\frac{1}{2}\ln|2\pi\Sigma_q|
+\frac{1}{2}\left<Q^{\ell+m}\right>_p
\end{equation}

All of the difficulty stems from the third term, denoted by $Q$, which requires evaluating the expectation values of $z^{\ell+m}$ and $(z^{\ell+m})^2$ with respect to $p(z^\ell)$. Note that in practice, we shall fix the reference layer to $\ell=0$.

```python
# Numpy, scipy, and plotting:
import numpy as np
from scipy.stats import norm         # Gaussian fitting
import scipy.integrate as integrate  # integration
import matplotlib.pyplot as plt      # plotting
import seaborn as sns; sns.set()     # nicer plotting
import pandas as pd                  # dataframe for use with seaborn

# File i/o:
import h5py    # HDF5

# Miscellaneous:
import math
import random     # random number generators
import re         # regular expressions
import gc         # garbage collection
import itertools  # generator slicing
import time       # for timing
import sys        # for printing progress

# symbolic algebra package:
import sympy as sym
from sympy import tanh
```

## Import hdf5 data
Specify the path to the .hdf5 files containing the accuracies, hooks, and parameters (weights & biases), and define functions to load and preprocess (i.e., sort) data:

```python
PATH_TO_DATA = '/full/path/to/HDF5/data/'

# read file of accuracies, return dataset as dictionary:
def read_accuracies(file_name):
    with h5py.File(PATH_TO_DATA + file_name, 'r') as file:
        # cast elements as np.array, else returns closed file datasets:
        acc_dict = {key : np.array(file[key]) for key in file.keys()}

    return acc_dict

# read file of inputs/outputs, return dataset as dictionary:
def read_hooks(file_name):
    with h5py.File(PATH_TO_DATA + file_name, 'r') as file:
        # cast elements as np.array, else returns closed file datasets:
        hook_dict = {key : np.array(file[key]) for key in file.keys()}

    return hook_dict

# given dictionary of hooks, extract list of preactivations:
def extract_preacts(hook_dict):
    keys = hook_dict.keys()

    layers, key_list = [], []
    for key in keys:
        if key.startswith('in'):
            key_list.append(key)
            layers.append(hook_dict[key][-1]) # use only last element

    # sort so that layers appear in correct order (for later analysis):
    layer_indices = [int(x.split('-')[1]) for x in key_list]
    sorted_layers = [lay for lay,_ in sorted(zip(layers, layer_indices), key = lambda x: x[1])]

    return sorted_layers

# read file of weights, biases; return dataset as dictionary:
def read_parameters(file_name):
    with h5py.File(PATH_TO_DATA + file_name, 'r') as file:
        # cast elements as np.array, else returns closed file datasets:
        for key in file.keys():
            para_dict = {key : np.array(file[key]) for key in file.keys()}

    return para_dict

# given dictionary of parameters, extract lists of weights, biases:
def extract_parameters(para_dict):
    keys = para_dict.keys()

    weights, W_keys = [], []
    biases, B_keys = [], []
    for key in keys:
        if key.startswith('W'):
            W_keys.append(key)
            weights.append(para_dict[key])
        elif key.startswith('B'):
            B_keys.append(key)
            biases.append(para_dict[key])

    # sort so that parameters appear in correct order (i.e., layer by layer):
    W_indices = [int(x.split('W')[1]) for x in W_keys]
    sorted_weights = [ww for ww,_ in sorted(zip(weights, W_indices), key = lambda x: x[1])]

    B_indices = [int(x.split('B')[1]) for x in B_keys]
    sorted_biases = [bb for bb,_ in sorted(zip(biases, B_indices), key = lambda x: x[1])]

    return sorted_weights, sorted_biases

# load data, ensuring consistent accuracies and hook files; return as dict:
def load_data(acc_file, hook_file, para_file):
    accuracies = read_accuracies(acc_file)
    hooks = read_hooks(hook_file)
    parameters = read_parameters(para_file)

    # get variances, depths; ensure consistency across files:
    var_w = [accuracies['var_weight'].item(), hooks['var_weight'].item(), parameters['var_weight'].item()]
    var_b = [accuracies['var_bias'].item(), hooks['var_bias'].item(), parameters['var_bias'].item()]
    depth = [accuracies['depth'].item(), hooks['depth'].item(), parameters['depth'].item()]

    # check if all elements match the first:
    if not all([var_w[0] == v for v in var_w]):
        raise Exception('Error: mismatched weight variance in data files.\n')
    if not all([var_b[0] == v for v in var_b]):
        raise Exception('Error: mismatched bias variance in data files.\n')
    if not all([depth[0] == d for d in depth]):
        raise Exception('Error: mismatched depths in data files.\n')

    # extract and sort preactivations, parameters:
    preacts = extract_preacts(hooks)
    weights, biases = extract_parameters(parameters)
    widths = [x.shape[0] for x in preacts]  # get width of each layer

    # encapsulate data in dictionary (for compact return object):
    data = {'depth' : depth[0],
           'var_w' : var_w[0],
           'var_b' : var_b[0],
           'accuracies' : accuracies['accuracies'],
           'preacts' : preacts,
           'weights' : weights,
           'biases' : biases,
           'widths' : widths}

    # print relevant information about dataset and return:
    print('Successfully loaded data for network with depth {},'
          ' weight variance {}, bias variance {}\n'.format(depth[0], var_w[0], var_b[0]))

    return data
```

Note the dimensions of each weight matrix: $|W^{\ell+1}|=(n,m)$ means $n$ rows, $m$ colums (where here $|\cdot|$ means dimension, not determinant)---$n$ rows corresponding to the neurons in the $\ell\!+\!1$ layer, and $m$ columns corresponding to the neurons in the $\ell$ layer. In other words, the *second* index of $W_{ij}$ runs over the neurons in the previous layer that connect to the neuron indexed by $i$. Element $W_{ij}^{\ell+1}$ is therefore the connection between $z_j^{\ell}$ and $z_i^{\ell+1}$. This is consistent with our notation for the connection rule, where we sum over the second index:

\begin{equation}
z_i^{\ell+1}=\sum_jW_{ij}^{\ell+1}\phi(z_j^\ell)+b_i^{\ell+1}~.
\end{equation}

## Recursive MC integral
The procedure implemented below is as follows. Let $m\!=\!0$ denote the reference layer. We begin at $m=1$ by drawing `nMC` Gaussian samples from the distribution with $\mu=\langle z^0\rangle$, $\sigma^2=\langle(z^0)^2\rangle-\langle z^0\rangle^2$, and use these to compute `nMC` samples of activations $\phi(z_i^0)$.

At $m=2$, we do the same, and use these to compute `nMC` samples of the *vector* of activations $\phi(z^1)=\{\phi(z_i^1)\}$.

A key feature underlying the speed of this implementation is that once we have the entire vector, we do *not* recompute these at higher levels. Rather, at $m\!=\!3$, we simply multiply each element of this vector by the appropriate weight, and sum over $j$ to obtain `nMC` samples of $W_{ij}^2\phi(z_j^1)$. After adding the bias $b_i^2$, taking the $\tanh$, and repeating for all $i$, we obtain `nMC` samples of the vector $\phi(z^2)=\{\phi(z_i^2)\}=\{\tanh(W_{ij}^2\phi(z_j^1)+b_i^2)\}$.

At each level, $\phi(z^m)$ can be immediately plugged into the MC integrator, which in this case simply computes the MC average over the samples; that is, for each $i$, $\int\mathcal{D}z_i^0\,\phi(z_i^m)=\frac{1}{N_\mathrm{MC}}\sum_{n=1}^{N_\mathrm{MC}}\phi(z_i^m)$. Similarly, to obtain $\int\mathcal{D}z_i^0\mathcal{D}z_j^0\,\phi(z_i^m)\phi(z_j^m)$, we simply plug-in all possible products over $i,j$, taking care not to mix different samples (because these are drawn from a Gaussian distribution, i.e., $\mathcal{D}z_i^0$ is a Gaussian measure over the reference pre-activations).

```python
# generator of n Gaussian variables:
def gaussian_sampler(mean=0, std=1, n=1):
    while True:
        x = []
        # random.random() returns float in [0.0, 1.0), so rescale & shift:
        [x.append(random.gauss(mean, std)) for _ in range(n)]
        yield x

# return tanh of preactivations:
def tanh_ref(zlist):
    return [np.tanh(z) for z in zlist]

# return samples of phi(z_i^0):
def phi_sampler(m, weights, biases, preacts, num_samples=10**4, ref_layer=0):
    dim = len(preacts[ref_layer])
    mean = np.mean(preacts[ref_layer])
    std = np.std(preacts[ref_layer])

    if m == ref_layer:
        phi = np.array([tanh_ref(zlist) for zlist in
                        itertools.islice(gaussian_sampler(mean, std, dim), num_samples)]).transpose()
    else:
        raise Exception('\nError: sample layer must equal ref_layer.\n')

    return phi

# compute phi(z^m) given phi(z^{m-1}):
def phi_new(m, phi, weights, biases):
    phiT = phi.transpose()  # each element now a single sample of the vector {phi(z_i^m)}

    phi_new = []
    for j in range(weights[m].shape[0]):
        samples = [np.tanh(np.sum(weights[m][j] * p_list) + biases[m][j]) for p_list in phiT]
        phi_new.append(samples)

    return np.array(phi_new)

# given 2d array phi with <width> rows and <nMC> columns,
# compute MC integral of phi(z_i^m) and return list (index i):
def MC_int1(phi):
    nMC = phi.shape[1]
    return [np.sum(p)/nMC for p in phi]

# given 2d array phi with <width> rows and <nMC> columns,
# compute MC integral of phi(z_i^m)*phi(z_j^m) and return list:
def MC_int2(phi):
    width = phi.shape[0]
    nMC = phi.shape[1]

    prod = []
    for j in range(width):
        for i in range(width):
            # element-wise multiplication (does not mix samples):
            prod.append(phi[i]*phi[j])

    return [np.sum(pp)/nMC for pp in prod]
```

## Expectation value of Q
We now have all the pieces required to compute the expectation value of $Q$ (see the corresponding paper for the expression and derivation).

```python
def Q_term(mm, phi, weights, biases, preacts):
    # compute MC integrals:
    fj = MC_int1(phi)
    fjk = MC_int2(phi)

    # ----------------------- linear term: -----------------------
    bshift = biases[mm] - np.mean(preacts[mm])  # vector b^m - mu^m
    WT = weights[mm].transpose()                # invert rows/columns so each element has len(bshift)

    linear_term = []
    for j in range(len(fj)):
        linear_term.append(fj[j] * np.sum(WT[j] * bshift))  # sum over i

    linear_term = 2*np.sum(linear_term)

    # --------------------- quadratic term: ----------------------
    quadratic_term = []
    for j in range(len(fj)):
        for k in range(len(fj)):
            index = j*len(fj) + k

            sum_over_i = np.sum(WT[j] * WT[k])
            quadratic_term.append(fjk[index] * sum_over_i)

    quadratic_term = np.sum(quadratic_term)

    # ---------------------- constant term: ----------------------
    const_term = np.sum(bshift**2)

    return quadratic_term + linear_term + const_term
```

## Entropy functions
Finally, we'll create functions to handle the first two terms, namely entropy $\frac{1}{2}\ln|2\pi e\Sigma_p|$ and "almost-entropy" $\frac{1}{2}\ln|2\pi\Sigma_q|$. In the networks under study, the multivariate Gaussian of each layer factorizes into a product of single-variable Gaussians (one for each neuron), so

\begin{equation}
\frac{1}{2}\ln|2\pi e\Sigma_p|
=\frac{N_p}{2}\ln 2\pi e\sigma_p^2~,
\quad\mathrm{and}\quad
\frac{1}{2}\ln|2\pi\Sigma_q|
=\frac{N_q}{2}\ln 2\pi\sigma_q^2~,
\end{equation}

where $N_p$ arises from the taking the determinant of the $N_p\times N_p$ covariance matrix $\Sigma_p=\mathrm{diag}(\sigma_p^2,\ldots,\sigma_p^2)$, where $\sigma_p$ is the standard deviation of the layer (as per the large-N assumption); note that $N_p$ is the width of the corresponding layer, i.e., $N_p=|z^\ell|$. Similarly for $N_q$.

```python
# Take in layer as array, return entropy:
def entropy(layer):
    var = np.std(layer)**2    # variance
    dim = layer.shape[0]      # dimension = layer width

    return math.log(2*math.pi*math.exp(1)*var)*(dim/2)

# Take in layer as array, return entropy minus N/2
# (i.e., almost entropy, without the factor of e):
def entropy_q(layer):
    return entropy(layer) - layer.shape[0]/2
```

Putting it all together, we'll have a single function that encapsulates the computation of relative entropy between the reference and target layers:

```python
# a little function to monitor progress:
def progress(m,M):
    sys.stdout.write('\r')
    sys.stdout.write('Computing layer {} of {}...'.format(m,M))
    sys.stdout.flush()
```

```python
# compute KL divergence for each layer, relative to ref_layer:
def KL_divergence(weights, biases, preacts, num_samples=10**4, ref_layer=0):
    start = time.time()  # timing

    term1 = -1*entropy(preacts[ref_layer])

    term2, term3 = [], []
    for m in range(ref_layer, len(preacts)):
        progress(m, len(preacts))  # monitor progress

        term2.append(entropy_q(preacts[m]))

        # get samples of phi(z^{m-1}):
        if m == ref_layer:
            term3.append(preacts[m].shape[0]/2)  # p=q implies Q=N/2
        else:
            # m = ref+1, generate MC samples for innermost tanh:
            if m == ref_layer+1:
                phi = phi_sampler(m-1, weights, biases, preacts, num_samples, ref_layer)
            # m > ref+1, use previous layer's tanh instead of recursion:
            else:
                phi = phi_new(m-1, phi, weights, biases)

            # compute Q term, include factor of 1/(2*sig^2):
            cov_q = np.std(preacts[m])**2
            term3.append(Q_term(m, phi, weights, biases, preacts)/(2*cov_q))

    # add contributions at each layer:
    result = [term1+term2+term3 for term2,term3 in zip(term2,term3)]

    # print timing, result:
    end = time.time()
    print('\rResult = {}\n\nComputation took {} seconds\n'.format(result, end-start))

    return result
```

# Runtime
Let's create some functions to write the results, as well as to handle scanning through data files and automatically load & run the KL analysis:

```python
# write file of KL divergences:
def write_KL(file_name, KL_list, var_weight, var_bias):
    with h5py.File(PATH_TO_DATA + file_name, 'w') as file:
        file.create_dataset('var_weight', data=var_weight)
        file.create_dataset('var_bias', data=var_bias)
        file.create_dataset('depth', data=len(KL_list))
        file.create_dataset('KL', data=KL_list)

# corresponding read file, while we're at it:
def read_KL(file_name):
    with h5py.File(PATH_TO_DATA + file_name, 'r') as file:
        # cast elements as np.array, else returns closed file datasets:
        KL_dict = {key : np.array(file[key]) for key in file.keys()}

    return KL_dict

# Files generated by "Gaussian_Feedforward" generally have the format:
#
#     acc-{base}-{dd}.hdf5    for accuracies
#     hooks-{base}-{dd}.hdf5  for hooks (preacts)
#     para-{base}-{dd}.hdf5   for parameters (weights, biases)
#
# where both `base` and `dd` are integers. `base` is usually var_w without the decimal,
# and remains fixed for multiple values of `dd`, which runs over a range of depths.
# (Note that `base` can however be a string, if deviating from standard format).
# Given the base name, and the range of depths, this method will systematically
# read each file in the given range (with the same base), run the KL analysis,
# and write the results to new hdf5 files with the name format 'KL-{base}-{dd}.hdf5'
def load_and_run(base, start, stop, step, num_samples):
    # file names in format acc/hooks/para-{base}-{dd}.hdf5
    file_number = ['{}-{}'.format(base, dd) for dd in range(start, stop, step)]

    print('Loading {} files: {}\n'.format(len(file_number), file_number))

    for i in range(len(file_number)):
        # load data:
        acc_file = 'acc-' + file_number[i] + '.hdf5'
        hook_file = 'hooks-' + file_number[i] + '.hdf5'
        para_file = 'para-' + file_number[i] + '.hdf5'
        data = load_data(acc_file, hook_file, para_file)

        # compute KL divergence:
        result = KL_divergence(data['weights'], data['biases'], data['preacts'], num_samples)

        # write results:
        KL_file = 'KL-' + file_number[i] + '.hdf5'
        write_KL(KL_file, result, data['var_w'], data['var_b'])
```

Standalone example:

```python
nMC = 10**3    # number of MC samples
results = {}   # dict to store results

# pre-training data (e0 = epoch 0):
data = load_data('acc-150-30.hdf5', 'e0-hooks-150-30.hdf5', 'e0-para-150-30.hdf5')
results['pre'] = KL_divergence(data['weights'], data['biases'], data['preacts'], nMC)

write_KL('KL-150-pre.hdf5', results['pre'], data['var_w'], data['var_b'])
```

    Successfully loaded data for network with depth 30, weight variance 1.5, bias variance 0.05

    Result = [0.0, 169.83749922533997, 259.10136352441253, 296.64769694758655, 309.66140958305766, 327.9467634023268, 338.1203797814944, 364.16785675757825, 349.05507595071805, 346.19594938887894, 340.8370061491693, 356.92686848112515, 374.14628766462533, 373.13880635349017, 367.0410710399147, 378.7742440109901, 391.8123058386555, 363.4119047333097, 372.82866356203374, 366.95221015966234, 370.90354841531587, 408.02341724753165, 395.15836550860865, 370.26634067809766, 397.43463702167236, 411.56829635644027, 395.71282920476864, 424.22108979316226, 60.69859128032101, -362.1633642510567]

    Computation took 604.179221868515 seconds

```python
# post-training data (e14 = epoch 14)
data = load_data('acc-150-30.hdf5', 'e14-hooks-150-30.hdf5', 'e14-para-150-30.hdf5')
results['post'] = KL_divergence(data['weights'], data['biases'], data['preacts'], nMC)

write_KL('KL-150-post.hdf5', results['post'], data['var_w'], data['var_b'])
```

    Successfully loaded data for network with depth 30, weight variance 1.5, bias variance 0.05

    Result = [0.0, 187.58238985660154, 271.9379195973325, 303.8697540995652, 323.95030585041417, 352.5775499827908, 371.44320419429977, 408.8312155983141, 396.37055850909155, 390.27873455483183, 396.3798720905895, 416.1955049233876, 424.17227265429557, 423.1581891068858, 414.716510212142, 417.30838661981164, 433.98136769789477, 422.16501270656954, 422.8515555188788, 407.99515033190426, 426.72545001730396, 447.0349002445645, 448.3710699835211, 435.4541297255267, 466.6999994219907, 484.3680341237442, 476.4972395281726, 521.6906994595188, 154.05830257717744, -325.803573941238]

    Computation took 603.8114519119263 seconds

# Plot results
Finally, let's read-in the hdf5 data generated above, and plot the results:

```python
# automatically read and plot KL data; format 'KL-var-base.hdf5',
# comparing pre- vs. post-training; assume same depth, var_b, and var_w across file lists:
def pre_post_compare(base1, base2, start, stop, step, plot=True, write=False):
    # file names in format KL-{var}-{base}.hdf5
    filenames1 = ['KL-{}-{}.hdf5'.format(var, base1) for var in range(start, stop, step)]
    filenames2 = ['KL-{}-{}.hdf5'.format(var, base2) for var in range(start, stop, step)]

    if len(filenames1) != len(filenames2):
        raise Exception('Error: mismatched file lists.\n')

    rel_ent1, rel_ent2, depth, var_w, var_b = [], [], [], [], []
    for i in range(len(filenames1)):
        # load data:
        data1 = read_KL(filenames1[i])
        data2 = read_KL(filenames2[i])

        # truncate & renormalize rel_ent by entropy of reference layer:
        var = data1['var_weight'].item()
        rel_ent_norm1 = data1['KL'][:-2]/(784*np.log(2*np.pi*np.e*var)/2)
        rel_ent_norm2 = data2['KL'][:-2]/(784*np.log(2*np.pi*np.e*var)/2)

        rel_ent1.append(rel_ent_norm1)
        rel_ent2.append(rel_ent_norm2)

        depth.append(data1['depth'].item())
        var_w.append(var)
        var_b.append(data1['var_bias'].item())

    if plot:
        plt.rcParams['figure.figsize'] = [9, 6]   # globally (!) adjust figure size

        # plot first series, labelled by variance:
        list_dict = {r'$\sigma_w^2$ = {}'.format(vv) : pd.Series(rel_ent1[i])
                     for i,vv in enumerate(var_w)}
        df = pd.DataFrame(list_dict)
        KL_plot = df.plot()

        # get colors of lines, so can use same scheme in second plot:
        lines = KL_plot.get_lines()
        colors = [line.get_color() for line in lines]
        # add alpha parameter, to make transluscent:
        colors = [(c[0], c[1], c[2], 0.5) for c in colors]

        # plot second series on top of same graph:
        list_dict2 = {'var_w = {}'.format(vv) : pd.Series(rel_ent2[i])
             for i,vv in enumerate(var_w)}
        styles=['--']*len(filenames2)
        df2 = pd.DataFrame(list_dict2)
        # overlay plots:
        KL_plot = df2.plot(ax=KL_plot, style=styles, color=colors, legend=None)  # use same colors as before!

        # format title, axes:
        KL_plot.set_title('KL divergence vs. depth')
        KL_plot.set_xlabel('layer')
        KL_plot.set_ylabel('KL divergence')

        # optionally save plot as pdf:
        if write:
            plt.savefig(PATH_TO_DATA+'plot-compare.pdf')

    return rel_ent1, rel_ent2, depth, var_w, var_b

```

For example, fig. 7 in the corresponding paper is generated via:

```python
kl1, kl2, dep, var_w, var_b = pre_post_compare('post', 'pre', 150, 351, 50, True, False);
```


# Model of the potassium A-type (transient) current

*equations taken from Sterratt et al. book*

\begin{equation}
C_m\frac{dV}{dt} = -\bar{g}_{Na}m^3h(V-E_{Na}) - \bar{g}n^4_K(V-E_K) - g_{lk}(V-E_{lk}) - I_A
\end{equation}

\begin{equation}
I_A = \bar{g}_Aa^3b(V-E_A);
\end{equation}

\begin{equation}
a_\infty  = \left(\frac{0.0761 \exp (\frac{V+99.22}{31.84}) }{1 + \exp(\frac{V+6.17}{28.93})} \right)^{1/3}\quad
\tau_a = 0.3632 + \frac{1.158}{1 + \exp(\frac{V+60.96}{20.12})}\;
\end{equation}

\begin{equation}
b_\infty = \frac{1}{\left(1 + \exp(\frac{V+58.3}{14.54})\right)^4}\;
\tau_b = 1.24 + \frac{2.678}{1 + \exp(\frac{V-55}{16.027})}
\end{equation}

```python
from __future__ import division
```

```python
from PyDSTool import *
import PyDSTool as dst
```

```python
%pylab inline

style.use('ggplot')
```

    Populating the interactive namespace from numpy and matplotlib

    WARNING: pylab import has clobbered these variables: ['spy', 'gamma', 'six', 'copy', 'who', 'find', 'diff', 'info']
    `%matplotlib` prevents importing * from pylab and numpy

```python
import bokeh
import bokeh.plotting as bkp

from ipywidgets import interact
bkp.output_notebook()
```

    BokehJS successfully loaded.

```python

```

## Setting the model up

```python
Acurr_params = dict(
    Iap = 0,
    Ena = 50,
    Ek = -77,
    Ea = -80,
    Elk = -22,
    gnabar = 120.,# mS/cm^2
    gkbar = 20.0, # mS/cm^2
    gabar = 47.7, # mS/cm^2
    glk = 0.3
)
Vtest = linspace(-100,50, 1000)
```

```python
def ainf(Vm,vshift=5):
    return (0.0761*exp((Vm+vshift+94.22)/31.84)/(1+exp((Vm+vshift+1.17)/28.93)))**(1/3)
def atau(Vm, vshift=5):
    return 0.3632  + 1.158/(1 + exp((Vm+vshift + 55.96)/20.12))
```

```python
def binf(Vm, vshift=5):
    return 1/(1 + exp((Vm+vshift + 53.3)/14.54))**4
def btau(Vm, vshift=5):
    return 1.24 + 2.678/(1 + exp((Vm-vshift+50)/16.027))
```

```python
figure(figsize=(10,5))
subplot(121)
plot(Vtest, clip(ainf(Vtest),0,1), color="orange", label='ainf')
plot(Vtest, binf(Vtest), color="steelblue", label='binf')

legend(loc=0)

subplot(122)
plot(Vtest, atau(Vtest), color="orange", label='atau')
plot(Vtest, btau(Vtest), color="steelblue", label='btau')
legend()
```

```python
q10 = 3.13
Q = q10**((18-6.3)/10) # = 3.8

def vtrap(x,y):
    return where(np.abs(x/y) < 1e-6, y*(1-0.5*x/y), x/(exp(x/y)-1))

def nalpha(Vm, nshift=0.7):
    return Q*0.5*0.01 * vtrap(-(Vm + 50 + nshift), 10)

def nalpha2(Vm, nshift=0.7):
    return Q*0.5*(-0.01) * (Vm+50.7)/(exp(-0.1*(Vm+50.7))-1)

def nbeta (Vm, nshift=0.7):
    return Q*0.5*0.125 * exp(-(Vm + 60 + nshift)/80)

```

```python
plot(Vtest, nalpha(Vtest), Vtest, nbeta(Vtest))#, Vtest, nalpha2(Vtest))
```

```python
mshift = -0.3
hshift = -7

def malpha(Vm):
    return Q* 0.1 * vtrap(-(Vm + 35 + mshift), 10)
def mbeta(Vm):
    return Q* 4 * exp(-(Vm + 60 + mshift)/18)

def halpha(Vm):
    return Q* 0.07*exp(-(Vm + 60 + hshift)/20)
def hbeta(Vm):
    return Q* 1/(exp(-(Vm + 30 + hshift)/10) + 1)

def minf (V):
    return 1 / (1 + mbeta(V)/malpha(V))

def hinf(V):
    return 1/(1 + hbeta(V)/halpha(V))

def ninf(V):
    return 1/(1 + nbeta(V)/nalpha(V))

subplot(121)
plot(Vtest, malpha(Vtest), Vtest,mbeta(Vtest))
title('Activation')

subplot(122)
plot(Vtest, halpha(Vtest), Vtest,hbeta(Vtest))
title('Inactivation')
```

```python

plot(Vtest, minf(Vtest), Vtest,hinf(Vtest))

```

```python

DSargs = dst.args(name="Acurrent",
                  pars = Acurr_params,
                  vars = ['V', 'm', 'h', 'n', 'a', 'b'],
                  tdomain=[0.0,1000.0])

DSargs.ics = dict(V = -75,
                  m=0.1,
                  h=0.1,
                  n = 0.1,
                  a = 0.1,
                  b = 0.1) # Initial conditions

DSargs.fnspecs = dict(ainf = (['V'], 'pow(0.0761*exp((V+99.22)/31.84)/(1 + exp((V+6.17)/28.93)),1./3)'),
                      atau = (['V'], '0.3632 + 1.158/(1 + exp((V+60.96)/20.12))'),
                      #
                      binf = (['V'], '1/(1 + exp((V+58.3)/14.54))**4'),
                      btau = (['V'], '1.24 + 2.678/(1 + exp((V-55)/16.027))'),
                      #
                      malpha = (['V'], '3.8 * (-0.1*(V+34.7))/(exp(-(V + 34.7)/10)-1)'),
                      mbeta = (['V'], '3.8 * 4 * exp(-(V+59.7)/18)'),
                      #
                      halpha =  (['V'], '3.8*0.07*exp(-(V+53)/20)'),
                      hbeta = (['V'], '3.8/(1 + exp(-(V+23)/10))'),
                      #
                      nalpha = (['V'], '0.5*3.8*(-0.01*(V+50.7))/(exp(-(V + 50.7)/10)-1)'),
                      nbeta = (['V'], '0.5*3.8*0.125*exp(-(V+60.7)/80)'),
                      #
                      rpulse = (['tx','width'], '0.5*(1 + tanh(100*(tx)) * tanh(100*(-tx+width)))'),
                     )

DSargs.varspecs = dict(
    V = 'Iap*rpulse((t-100), 800) -\
    (gnabar*m*m*m*h*(V-Ena) + gkbar*n*n*n*n*(V-Ek) + gabar*a*a*a*b*(V-Ea) + glk*(V-Elk))',
    m = 'malpha(V)*(1-m) - mbeta(V)*m',
    h = 'halpha(V)*(1-h) - hbeta(V)*h',
    n = 'nalpha(V)*(1-n) - nbeta(V)*n',
    a = '(ainf(V)-a)/atau(V)',
    b = '(binf(V)-b)/btau(V)')

```

```python

#ode  = dst.Generator.Radau_ODEsystem(DSargs)
#ode  = dst.Generator.Dopri_ODEsystem(DSargs)
ode = dst.Generator.Vode_ODEsystem(DSargs)
traj0 = ode.compute('init')
pts0 = traj0.sample(dt=0.1)
ode.set(ics = pts0[-1])
```

```python

```

    <bokeh.io._CommsHandle at 0x7f12d0ac8e50>

```python
ode.set(pars = dict(Iap=8.3))
#ode.set(algparams=dict(max_pts=50000))
%time traj1 = ode.compute('test')
pts1 = traj1.sample(dt=0.1)
```

    CPU times: user 4.15 s, sys: 16 ms, total: 4.17 s
    Wall time: 4.12 s

```python
plot(pts1['t'], pts1['V'])
```

```python

```


# Controlling a system

**automated irrigation system**

*   input: hydration level of soil
*   desired reference: "kinda damp"
*   disturbances: rain (do nothing until water evaporates); sun (output water); fauna

```python
# "magic" commands, prefaced with "%", changes settings in the notebook

# this ensures plots are embedded in notebook web page
%matplotlib inline

# pdb = Python debugger, so this command turns the debugger OFF
%pdb off

# numpy = numerical Python, implements arrays (/ matrices)
import numpy as np
# limit number of decimal places printed for floating-point numbers
np.set_printoptions(precision=3)

# scipy = scientific Python, implements operations on arrays / matrices
import scipy as sp
# linalg = linear algebra, implements eigenvalues, matrix inverse, etc
from scipy import linalg as la
# optimize = optimization, root finding, etc
from scipy import optimize as op

# produce matlab-style plots
import matplotlib as mpl
# increase font size on plots
mpl.rc('font',**{'size':18})
# use LaTeX to render symbols
mpl.rc('text',usetex=False)
# animation
from matplotlib import animation as ani
# Matlab-style plotting
import matplotlib.pyplot as plt

# symbolic computation, i.e. computer algebra (like Mathematica, Wolfram Alpha)
import sympy as sym
def Jacobian(g,y,d=1e-4):
  """
  approximate derivative via finite-central-differences

  input:
    g - function - g : R^n -> R^m
    y - n array
    (optional)
    d - scalar - finite differences displacement parameter

  output:
    Dg(y) - m x n - approximation of Jacobian of g at y
  """
  # given $g:\mathbb{R}^n\rightarrow\mathbb{R}^m$:
  # $$D_y g(y)e_j \approx \frac{1}{2\delta}(g(y+\delta e_j) - g(y - \delta e_j)),\ \delta\ll 1$$
  e = np.identity(len(y))
  Dyg = []
  for j in range(len(y)):
      Dyg.append((.5/d)*(g(y+d*e[j]) - g(y-d*e[j])))
  return np.array(Dyg).T

def numerical_simulation(f,t,x,t0=0.,dt=1e-4,ut=None,ux=None,utx=None,return_u=False):
  """
  simulate x' = f(x,u)

  input:
    f : R x X x U --> X - vector field
      X - state space (must be vector space)
      U - control input set
    t - scalar - final simulation time
    x - initial condition; element of X

    (optional:)
    t0 - scalar - initial simulation time
    dt - scalar - stepsize parameter
    return_u - bool - whether to return u_

    (only one of:)
    ut : R --> U
    ux : X --> U
    utx : R x X --> U

  output:
    t_ - N array - time trajectory
    x_ - N x X array - state trajectory
    (if return_u:)
    u_ - N x U array - state trajectory
  """
  t_,x_,u_ = [t0],[x],[]

  inputs = sum([1 if u is not None else 0 for u in [ut,ux,utx]])
  assert inputs <= 1, "more than one of ut,ux,utx defined"

  if inputs == 0:
    assert not return_u, "no input supplied"
  else:
    if ut is not None:
      u = lambda t,x : ut(t)
    elif ux is not None:
      u = lambda t,x : ux(x)
    elif utx is not None:
      u = lambda t,x : utx(t,x)

  while t_[-1]+dt < t:
    if inputs == 0:
      _t,_x = t_[-1],x_[-1]
      dx = f(t_[-1],x_[-1]) * dt
    else:
      _t,_x,_u = t_[-1],x_[-1],u(t_[-1],x_[-1])
      dx = f(_t,_x,_u) * dt
      u_.append( _u )

    x_.append( _x + dx )
    t_.append( _t + dt )

  if return_u:
    return np.asarray(t_),np.asarray(x_),np.asarray(u_)
  else:
    return np.asarray(t_),np.asarray(x_)
```

    Automatic pdb calling has been turned OFF

## plotting output from transfer function

$G_{yv}$

```python
b = 1
a = 1
k_i = 1
k_p = 1
x0 = np.array([5,0])

A = np.array([[0,1],[-b*k_i,-a-b*k_p]])
B = np.array([[0],[1]])

def f(t,x):
  u = 1
  xdot = np.squeeze(np.dot(x, A.T) + u * B.T)
  return xdot
```

```python
t = 30. # sec; simulation duration

t_,x_ = numerical_simulation(f,t,x0)

y_,z_ = x_.T

v_ = np.dot(np.array([0, b]),x_)

plt.figure(figsize=(8,8))

ax = plt.subplot(3,1,1)
ax.plot(t_,y_)
ylim = ax.get_ylim()
# for _ in [5., 20.]:
#   ax.plot(_*np.ones(2),ylim,'k--')
# ax.plot(t_,v_r(t_),'k--')
# ax.set_ylim(ylim)
# ax.set_xticklabels([])
# ax.set_ylabel(r'velocity $y$')

# ax = plt.subplot(3,1,2)
# ax.plot(t_,v(t_))
# ylim = ax.get_ylim()
# for _ in [5., 20.]:
#   ax.plot(_*np.ones(2),ylim,'k--')
# ax.set_ylim(ylim)
# ax.set_xticklabels([])
# ax.set_ylabel(r'external force $v$')

# ax = plt.subplot(3,1,3)
# ax.plot(t_,u(t_,y_,z_))
# ylim = ax.get_ylim()
# for _ in [5., 20.]:
#   ax.plot(_*np.ones(2),ylim,'k--')
# ax.set_ylim(ylim)
# ax.set_ylabel(r'input $u$')

# ax.set_xlabel(r'time $t$ (sec)')

plt.tight_layout(); # semicolon suppresses text printout
```

```python
v_ = np.dot(x_,np.array([0, b]))

plt.figure(figsize=(8,8))

ax = plt.subplot(3,1,1)
ax.plot(t_,v_)
ax.set_ylabel('y(t)')
```

```python

```