# US Production Data for RBC Modeling

```python
import pandas as pd
import numpy as np
import fredpy as fp
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
pd.plotting.register_matplotlib_converters()
```

```python
# Load API key
fp.api_key = fp.load_api_key('fred_api_key.txt')

# Download nominal GDP, nominal personal consumption expenditures, nominal
# gross private domestic investment, the GDP deflator, and an index of hours
# worked in the nonfarm business sector produced by the BLS. All data are
# from FRED and are quarterly.
gdp = fp.series('GDP')
cons = fp.series('PCEC')
invest = fp.series('GPDI')
hours = fp.series('HOANBS')
defl = fp.series('GDPDEF')
pcec = fp.series('PCEC')
m2 = fp.series('M2SL')
tb3mo = fp.series('TB3MS')
unemp = fp.series('UNRATE')

# Convert monthly M2, 3-mo T-Bill, and unemployment to quarterly
m2 = m2.as_frequency('Q')
tb3mo = tb3mo.as_frequency('Q')
unemp = unemp.as_frequency('Q')

# Convert unemployment and t-bill data to decimals instead of percents
unemp.data = unemp.data/100
tb3mo.data = tb3mo.data/100

# pcec inflation as pecent change over past year
pcec = pcec.apc()
pcec.data = pcec.data/100

# Make sure that all of the downloaded series have the same data ranges
gdp,cons,invest,hours,defl,pcec,m2,tb3mo,unemp = fp.window_equalize([gdp,cons,invest,hours,defl,pcec,m2,tb3mo,unemp])

# Compute real GDP, real consumption, real investment
gdp.data = gdp.data/defl.data*100
cons.data = cons.data/defl.data*100
invest.data = invest.data/defl.data*100
m2.data = m2.data/defl.data*100

# Print units
print('Hours units:   ',hours.units)
print('Deflator units:',defl.units)
```

    Hours units:    Index 2012=100
    Deflator units: Index 2012=100

Next, compute the quarterly capital stock series for the US using the perpetual inventory method. The discrete-time Solow growth model is given by:

\begin{align}
Y_t & = A_tK_t^{\alpha}L_t^{1-\alpha} \tag{1}\\
C_t & = (1-s)Y_t \tag{2}\\
Y_t & = C_t + I_t \tag{3}\\
K_{t+1} & = I_t + (1-\delta)K_t \tag{4}\\
A_{t+1} & = (1+g)A_t \tag{5}\\
L_{t+1} & = (1+n)L_t \tag{6}.
\end{align}

Here the model is assumed to be quarterly so $n$ is the *quarterly* growth rate of labor hours, $g$ is the *quarterly* growth rate of TFP, and $\delta$ is the *quarterly* rate of depreciation of the capital stock. Given a value of the quarterly depreciation rate $\delta$, an investment series $I_t$, and an initial capital stock $K_0$, the law of motion for the capital stock, Equation (4), can be used to compute an implied capital series. But we don't know $K_0$ or $\delta$ so we'll have to *calibrate* these values using statistics computed from the data that we've already obtained.

Let lowercase letters denote a variable that's been divided by $A_t^{1/(1-\alpha)}L_t$. E.g.,

\begin{align}
y_t = \frac{Y_t}{A_t^{1/(1-\alpha)}L_t}\tag{7}
\end{align}

Then (after substituting consumption from the model), the scaled version of the model can be written as:

\begin{align}
y_t & = k_t^{\alpha} \tag{8}\\
i_t & = sy_t \tag{9}\\
k_{t+1} & = i_t + (1-\delta-n-g')k_t,\tag{10}
\end{align}

where $g' = g/(1-\alpha)$ is the growth rate of $A_t^{1/(1-\alpha)}$. In the steady state:

\begin{align}
k & = \left(\frac{s}{\delta+n+g'}\right)^{\frac{1}{1-\alpha}} \tag{11}
\end{align}

which means that the ratio of capital to output is constant:

\begin{align}
\frac{k}{y} & = \frac{s}{\delta+n+g'} \tag{12}
\end{align}

and therefore the steady state ratio of depreciation to output is:

\begin{align}
\overline{\delta K/ Y} & = \frac{\delta s}{\delta + n + g'} \tag{13}
\end{align}

where $\overline{\delta K/ Y}$ is the long-run average ratio of depreciation to output. We can use Equation (13) to calibrate $\delta$ given $\overline{\delta K/ Y}$, $s$, $n$, and $g'$.

Furthermore, in the steady state, the growth rate of output is constant:

\begin{align}
\frac{\Delta Y}{Y} & = n + g' \tag{14}
\end{align}

1. Assume $\alpha = 0.35$.
2. Calibrate $s$ as the average of ratio of investment to GDP.
3. Calibrate $n$ as the average quarterly growth rate of labor hours.
4. Calibrate $g'$ as the average quarterly growth rate of real GDP minus n.
5. Calculate the average ratio of depreciation to GDP $\overline{\delta K/ Y}$ and use the result to calibrate $\delta$. That is, find the average ratio of Current-Cost Depreciation of Fixed Assets (FRED series ID: M1TTOTL1ES000) to GDP (FRED series ID: GDPA). Then calibrate $\delta$ from the following steady state relationship:
\begin{align}
\delta & = \frac{\left( \overline{\delta K/ Y} \right)\left(n + g' \right)}{s - \left( \overline{\delta K/ Y} \right)} \tag{15}
\end{align}
6. Calibrate $K_0$ by asusming that the capital stock is initially equal to its steady state value:
\begin{align}
K_0 & = \left(\frac{s}{\delta + n + g'}\right) Y_0 \tag{16}
\end{align}

Then, armed with calibrated values for $K_0$ and $\delta$, compute $K_1, K_2, \ldots$ recursively. See Timothy Kehoe's notes for more information on the perpetual inventory method:

http://users.econ.umn.edu/~tkehoe/classes/GrowthAccountingNotes.pdf

```python
# Set the capital share of income
alpha = 0.35

# Average saving rate
s = np.mean(invest.data/gdp.data)

# Average quarterly labor hours growth rate
n = (hours.data[-1]/hours.data[0])**(1/(len(hours.data)-1)) - 1

# Average quarterly real GDP growth rate
g = ((gdp.data[-1]/gdp.data[0])**(1/(len(gdp.data)-1)) - 1) - n

# Compute annual depreciation rate
depA = fp.series('M1TTOTL1ES000')
gdpA = fp.series('gdpa')

gdpA = gdpA.window([gdp.data.index[0],gdp.data.index[-1]])
gdpA,depA = fp.window_equalize([gdpA,depA])

deltaKY = np.mean(depA.data/gdpA.data)
delta = (n+g)*deltaKY/(s-deltaKY)

# print calibrated values:
print('Avg saving rate:        ',round(s,5))
print('Avg annual labor growth:',round(4*n,5))
print('Avg annual gdp growth:  ',round(4*g,5))
print('Avg annual dep rate:    ',round(4*delta,5))

# Construct the capital series. Note that the GPD and investment data are reported on an annualized basis
# so divide by 4 to get quarterly data.
capital = np.zeros(len(gdp.data))
capital[0] = gdp.data[0]/4*s/(n+g+delta)

for t in range(len(gdp.data)-1):
    capital[t+1] = invest.data[t]/4 + (1-delta)*capital[t]

# Save in a fredpy series
capital = fp.to_fred_series(data = capital,dates =gdp.data.index,units = gdp.units,title='Capital stock of the US',frequency='Quarterly')
# plot the computed capital series
plt.plot(capital.data.index,capital.data,'-',lw=3,alpha = 0.7)
plt.ylabel(capital.units)
plt.title(capital.title)
plt.grid()
```

```python
# Compute TFP
tfp = gdp.data/capital.data**alpha/hours.data**(1-alpha)
tfp = fp.to_fred_series(data = tfp,dates =gdp.data.index,units = gdp.units,title='TFP of the US',frequency='Quarterly')

# Plot the computed capital series
plt.plot(tfp.data.index,tfp.data,'-',lw=3,alpha = 0.7)
plt.ylabel(tfp.units)
plt.title(tfp.title)
plt.grid()
```

```python
# Convert each series into per capita using civilian pop 16 and over
gdp = gdp.per_capita(civ_pop=True)
cons = cons.per_capita(civ_pop=True)
invest = invest.per_capita(civ_pop=True)
hours = hours.per_capita(civ_pop=True)
capital = capital.per_capita(civ_pop=True)
m2 = m2.per_capita(civ_pop=True)

# Put GDP, consumption, investment, and M2 in units of thousands of dollars per person
gdp.data = gdp.data*1000
cons.data = cons.data*1000
invest.data = invest.data*1000
capital.data = capital.data*1000
m2.data = m2.data/1000

# Scale hours per person to equal 100 on October (Quarter III) of 2012
hours.data = hours.data/hours.data.loc['2012-10-01']*100

# Compute and plot log real GDP, log consumption, log investment, log hours
gdp_log = gdp.log()
cons_log = cons.log()
invest_log = invest.log()
hours_log = hours.log()
capital_log = capital.log()
tfp_log = tfp.log()
m2_log = m2.log()
m2_log = m2.log()
```

```python
# HP filter to isolate trend and cyclical components
gdp_log_cycle,gdp_log_trend = gdp_log.hp_filter()
cons_log_cycle,cons_log_trend = cons_log.hp_filter()
invest_log_cycle,invest_log_trend = invest_log.hp_filter()
hours_log_cycle,hours_log_trend = hours_log.hp_filter()
capital_log_cycle,capital_log_trend = capital_log.hp_filter()
tfp_log_cycle,tfp_log_trend = tfp_log.hp_filter()
m2_log_cycle,m2_log_trend = m2_log.hp_filter()
tb3mo_cycle,tb3mo_trend = tb3mo.hp_filter()
unemp_cycle,unemp_trend = unemp.hp_filter()
pcec_cycle,pcec_trend = pcec.hp_filter()
```

```python
# Create a DataFrame with actual and trend data
data = pd.DataFrame({
        'gdp':gdp.data,
        'gdp_trend':np.exp(gdp_log_trend.data),
        'gdp_cycle':gdp_log_cycle.data,
        'consumption':cons.data,
        'consumption_trend':np.exp(cons_log_trend.data),
        'consumption_cycle':cons_log_cycle.data,
        'investment':invest.data,
        'investment_trend':np.exp(invest_log_trend.data),
        'investment_cycle':invest_log_cycle.data,
        'hours':hours.data,
        'hours_trend':np.exp(hours_log_trend.data),
        'hours_cycle':hours_log_cycle.data,
        'capital':capital.data,
        'capital_trend':np.exp(capital_log_trend.data),
        'capital_cycle':capital_log_cycle.data,
        'tfp':tfp.data,
        'tfp_trend':np.exp(tfp_log_trend.data),
        'tfp_cycle':tfp_log_cycle.data,
        'real_m2':m2.data,
        'real_m2_trend':np.exp(m2_log_trend.data),
        'real_m2_cycle':m2_log_cycle.data,
        't_bill_3mo':tb3mo.data,
        't_bill_3mo_trend':tb3mo_trend.data,
        't_bill_3mo_cycle':tb3mo_cycle.data,
        'pce_inflation':pcec.data,
        'pce_inflation_trend':pcec_trend.data,
        'pce_inflation_cycle':pcec_cycle.data,
        'unemployment':unemp.data,
        'unemployment_trend':unemp_trend.data,
        'unemployment_cycle':unemp_cycle.data,
    },index = gdp.data.index)

# # RBC Data
# columns_ordered =[]
# names = ['gdp','consumption','investment','hours','capital','tfp']
# for name in names:
#     columns_ordered.append(name)
#     columns_ordered.append(name+'_trend')

# data[columns_ordered].to_csv('../Csv/rbc_data_actual_trend.csv')

# # Create a DataFrame with actual, trend, and cycle data
# columns_ordered =[]
# names = ['gdp','consumption','investment','hours','capital','tfp']
# for name in names:
#     columns_ordered.append(name)
#     columns_ordered.append(name+'_trend')
#     columns_ordered.append(name+'_cycle')

# data[columns_ordered].to_csv('../Csv/rbc_data_actual_trend_cycle.csv')

# Business Cycle Data
columns_ordered =[]
names = ['gdp','consumption','investment','hours','capital','tfp','real_m2','t_bill_3mo','pce_inflation','unemployment']
for name in names:
    columns_ordered.append(name)
    columns_ordered.append(name+'_trend')

data[columns_ordered].to_csv('../Csv/business_cycle_data_actual_trend.csv')

# Create a DataFrame with actual, trend, and cycle data
columns_ordered =[]
names = ['gdp','consumption','investment','hours','capital','tfp','real_m2','t_bill_3mo','pce_inflation','unemployment']
for name in names:
    columns_ordered.append(name)
    columns_ordered.append(name+'_trend')
    columns_ordered.append(name+'_cycle')

data[columns_ordered].to_csv('../Csv/business_cycle_data_actual_trend_cycle.csv')
```


***Distance measures in cosmology by David W. Hogg***

 - formulae for many different cosmological distance measures are provided

```python
import astropy.io.fits as fits
from os import system
import os
import subprocess
import matplotlib.pyplot as plt
import numpy as np
import scipy.special
import scipy.integrate as si
import healpy as hp
from math import sqrt, cos, sin, tan, pi, acos, asin, atan, sinh, pow, log, asinh
import numpy.fft as fft
import time
import numpy.random as ran
from astropy import units, cosmology
from astropy import constants as const
from astropy import units, cosmology
from astropy.cosmology import default_cosmology
from astropy.cosmology import FlatLambdaCDM
#from astropy.cosmology import WMAP9 as cosmo
```

```python
#H0 = 70. #presente day Hubble rate (h.km.s^-1.mpc-1)
#c = 2.99792458E5 # speed of light (km/s)
#G = 6.67428E-11 #gravitational constant (m^3.s^-2.kg^-1)
c = const.c.to('km/s')
print(c)
G = const.G
print(G)
kiloparsec = const.kpc #3.08567758149E19 #kiloparsec (m)
print(kiloparsec)
megaparsec = 3.08567758149E22 #megaparsec (m)
# H0 = 100 * h # (km s^-1 Mpc^-1)
#solar_mass = 2E30 #heliocentric_gravit_const/gravit_const_SI #solar mass (kg)
solar_mass = const.M_sun
print(solar_mass)
```

    299792.458 km / s
      Name   = Gravitational constant
      Value  = 6.6743e-11
      Uncertainty  = 1.5e-15
      Unit  = m3 / (kg s2)
      Reference = CODATA 2018
      Name   = Kiloparsec
      Value  = 3.0856775814913675e+19
      Uncertainty  = 0.0
      Unit  = m
      Reference = Derived from au + IAU 2015 Resolution B 2 note [4]
      Name   = Solar mass
      Value  = 1.988409870698051e+30
      Uncertainty  = 4.468805426856864e+25
      Unit  = kg
      Reference = IAU 2015 Resolution B 3 + CODATA 2018

***Cosmological Parameters***

Hubble constant $H_0$ is the ratio between Recession speed(v) and distance(d) in the expanding Universe. (“0” refers to the present epoch)

\begin{equation}
v = H_0 d
\end{equation}

Hubble time $t_H$ is the inverse of the Hubble constant $H_0$.

\begin{equation}
t_H = 1 / H_0
\end{equation}

Hubble distance $D_H$ is the the speed of light (c) times the Hubble time ($t_H$)

\begin{equation}
D_H = c / H_0
\end{equation}

Dimensionless density parameters:

\begin{equation}
\Omega_{\Lambda} = \frac{\Lambda c^2} {3 H_0^2 }
\end{equation}

\begin{equation}
\Omega_m = \frac{8 \pi G \rho_0}{3 H_0^2}
\end{equation}

A third density parameter $\Omega_k$ measures the “curvature of space”
\begin{equation}
\Omega_m + \Omega_{\Lambda} + \Omega_k = 1
\end{equation}

If $\Omega_{\Lambda}$ = 0, then the deceleration parameter $q_0$ is just half $\Omega_m $, otherwise $q_0$ is not such a useful parameter.

```python
#cosmo = default_cosmology.get()
cosmo = FlatLambdaCDM(H0=70, Om0=0.3, Tcmb0=2.725)
print(cosmo)
Omega_m0 = cosmo.Om0
Omega_Lambda0 = cosmo.Ode0
#print(Omega_Lambda0)
#cosmo.kpc_proper_per_arcmin(3)

Omega_k0 =  0
H0 = cosmo.H(0) #at z = 0
#H0.value, H0.unit
#H0 = 67.7 # km / (Mpc s)
#mpc2km = 3.086E+19
#km2mpc = 3.24078e-20
h = H0/100
D_H = (c/H0).to(units.km) #hubble distante in km
t_H = (1 / H0) #hubble time in seconds
print('hubble distance in Km: ', D_H)
#print('hubble time in seconds: ', t_H.to(units.s))
print('hubble time in years: ', t_H.to(units.yr))

def set_model(model = 'Einstein–de-Sitter'):
    global Omega_m0
    global Omega_Lambda0
    global Omega_k0
    if (model == 'Einstein–de-Sitter'):
        Omega_m0 = 1
        Omega_Lambda0 = 0
        Omega_k0= 1 - Omega_m0 - Omega_Lambda0
    if (model == 'low-density'):
        Omega_m0 = 0.05
        Omega_Lambda0 = 0
        Omega_k0= 1 - Omega_m0 - Omega_Lambda0
    if (model == 'high-lambda'):
        Omega_m0 = 0.2
        Omega_Lambda0 = 0.8
        Omega_k0= 1 - Omega_m0 - Omega_Lambda0

```

    FlatLambdaCDM(H0=70 km / (Mpc s), Om0=0.3, Tcmb0=2.725 K, Neff=3.04, m_nu=[0. 0. 0.] eV, Ob0=None)
    hubble distance in Km:  1.3215183810725603e+23 km
    hubble time in years:  13968460309.72556 yr

***Cosmological constant Lamba ($\Lambda$)***
 - From Equation (6)

\begin{equation}
\Lambda = \frac{3 H_0^2 \Omega_{\Lambda}}{c^2}
\end{equation}

```python
#cosmological constant Lamba
Lambda= 3 * ((H0/c) ** 2) * Omega_Lambda0
print(Lambda)

```

    1.1447774726145685e-07 1 / Mpc2

**Mass Density of University ($\rho_0$)**
 - From Equation (5)

\begin{equation}
\rho_0 = \frac{3 H_0^2 \Omega_m}{8 \pi G}
\end{equation}

```python
#mass density of universe
rho_0 =  3 * (H0**2) * Omega_m0 / (8 * np.pi * G)
#rho_0 = rho_0 * 1.0503e-39  # in kg/m^3
print(rho_0)

```

    2629014649608.4175 kg km2 / (m3 Mpc2)

**Redshift (z)**

Redshift of an object is the ratio fractional doppler shift of its emitted light resulting from radial motion:

\begin{equation}
 z \equiv \frac{v_e}{v_o} - 1 = \frac{\lambda_o}{\lambda_e} - 1
\end{equation}

$v_o$ & $\lambda_o$ are the observed frequency ane wavelength and $v_e$ & $\lambda_e$ are emitted.

In Special relativity:
\begin{equation}
 1+z = \sqrt{\frac{1 + \frac{v}{c}}{1 - \frac{v}{c}}}
\end{equation}

NOTE: redshift is directly observable and radial velocity is not.

The difference between an object’s measured redshift $z_{obs}$ and its cosmological redshift $z_{cos}$ is due to its (radial) peculiar velocity $v_{pec}$.

Peculiar Velocity (derived from above equation by taking derivative):
\begin{equation}
   v_{pec} = c \frac{(z_{obs} - z_{cos})}{(1+z)}
\end{equation}
    assumed $v_{pec}$ << c.

Notes:
1. From now on, z = $z_{cos}$

2. For small v/c, or small distance d, in the expanding Universe, the velocity is linearly proportional to the distance.
\begin{equation}
   z \approx \frac{v}{c} = \frac{d}{D_H}
\end{equation}
   Equation is only true for small redshifts!
3. Many galaxy redshift surveys, when presenting redshifts as radial velocities, always use the non-relativistic approximation v = cz, even when it may not be physically appropriate.

The cosmological redshift is directly related to the scale factor a(t), or the “size” of the Universe.
\begin{equation}
 1+z_{12} = \frac{a(t_1)}{a(t_2)} = \frac{1+z_2}{1+z_1}
\end{equation}

```python

#recession speed
#v = H0 * d #where d is the distance in expanding universe

#c=t_H =D_H =1

#Redshift z & radial velocity v
# 1+z = sqrt((1+ v/c)/(1- v/c))
#Velocity to relativistic redshift
def vel2z(v):
    val = v / c
    doppler_factor = np.sqrt((1 + val) / (1 - val))
    return  (doppler_factor - 1)
# For small v/c, or small distance d, the velocity is linearly proportional to the distance

#value = cosmo.luminosity_distance(z)
#print (value)

#relativistic redshift to a velocity
def z2v(z):
    zp = (1 + z) ** 2
    return (c * (zp - 1) / (zp + 1))

#object’s measured redshift z_obs
#cosmological redshift z_cos
```

**Comoving Distance (Line-of-sight)**

A small comoving distance $\delta DC$ between two nearby objects in the Universe is the distance between them which remains constant with epoch if the two objects are moving with the Hubble flow.

The total line-of-sight comoving distance DC from us to a distant object is computed by integrating the infinitesimal $\delta DC$ contributions between nearby events along the radial ray from z = 0 to the object.

normalized Hubble expansion rate at reshift $z$:
\begin{equation}
E(z) = \sqrt{\Omega_{m,0} (1+z)^3 + \Omega_{\kappa, 0} (1+z)^2 + \Omega_{\Lambda,0}}
\end{equation}

Hubble constant at redshift z:
\begin{equation}
H(z) = H_0 E(z)
\end{equation}

The total line-of-sight comoving distance is
\begin{equation}
DC = DH  \int_{0}^{z} \frac{dz'}{E(z')} \,
\end{equation}

```python
def E_z(z, w=None):
    if w is not None:
        w = w
        #print(w)
    else:
        w = -1.
    zp1 = 1.0+z
    return np.sqrt(Omega_m0*zp1**3.0 + Omega_k0*zp1**2.0 + Omega_Lambda0*(1+z)**(1.+w))

def Ez(z):
    zp1 = 1.0+z
    return np.sqrt(Omega_m0*zp1**3.0 + Omega_k0*zp1**2.0 + Omega_Lambda0) # if w is not in cosmo

def Hubble_Distance(z):
    DH =  (c/(H0*E_z(z))).to(units.km)
    #DH = D_H / E_z(z)
    #print (H_z)
    return DH

#Computes the line of sight comoving distance from redshifts z0 to z
def comovingDistance(z, z0 = 0, w=None):
    if w is not None:
        w = w
        #print(w)
    else:
        w = -1.
        #print (w)
    #DC = D_H * si.quad(lambda z: (1/E_z(z)), z0, z)
    DC = Hubble_Distance(0.) * si.quad(lambda z: (1/E_z(z)), z0, z)
    #print (DC[0])
    return DC[0].to(units.pc)

```

**Traverse Comoving distance**

The comoving distance between two events at the same redshift or distance but separated on the sky by some angle $\delta \theta$ is DM $\delta \theta$ and the transverse comoving distance DM is simply related to the line-of-sight comoving distance DC:

$D_M = \left\{
\begin{array}{ c l }
D_H \frac{1}{\sqrt{\lvert \Omega_{\kappa} \lvert}} sinh \left[\sqrt{\lvert \Omega_{\kappa} \lvert} \frac{D_C}{D_H} \right] & \quad for \quad \Omega_{\kappa}>0,\\
D_C & \quad for \quad \Omega_{\kappa}=0, \\
D_H \frac{1}{\sqrt{\lvert \Omega_{\kappa} \lvert}} sin \left[\sqrt{\lvert \Omega_{\kappa} \lvert} \frac{D_C}{D_H} \right] & \quad for \quad \Omega_{\kappa}<0
\end{array}
\right.$

where the trigonometric functions sinh and sin account for what is called “the curvature of space.

For $\Omega_{\Lambda} = 0$, The solution is:
\begin{equation}
D_M = D_H  \frac{2 [ 2 - \Omega_m (1-z) - (2 - \Omega_m) \sqrt{1 + \Omega_m z} ] }{\Omega_m^2 (1+z)}
\end{equation}

```python
# Computes transverse comoving distance
def traverse_comovingDistance(z):
    #print(Omega_Lambda0)
    DC = comovingDistance(z)
    if(Omega_k0==0.0) :
        return DC
    sqrt_Omk0 = sqrt(np.abs(Omega_k0))
    #angle = sqrt_Omk0 * DC/D_H
    angle = sqrt_Omk0 * DC/Hubble_Distance(0)
    #factor = D_H / sqrt_Omk0
    factor = Hubble_Distance(0) / sqrt_Omk0
    DM = 1.
    if (Omega_k0 > 0.):
        DM = factor * sinh(angle)
    if (Omega_k0 < 0.):
        DM = factor * sin(angle)
    return DM
```

```python
print ("Comoving Distance", comovingDistance(6))
print ("Traverse Comoving Distance", traverse_comovingDistance(6))
```

    Comoving Distance 8247384688.539825 pc
    Traverse Comoving Distance 8247384688.539825 pc

```python
# for Omega_Lambda0 = 0., Computes transverse comoving distance
def proper_traverse_comovingDistance(z):
    sqr_OmM0 = Omega_m0**2
    reg = sqrt(1 + (Omega_m0*z))
    numerator = 2 * (2 - (Omega_m0 * (1-z)) -((2-Omega_m0) * reg))
    denominator = sqr_OmM0 * (1+z)
    return (D_H * numerator/denominator).to(units.pc)

```

```python
print ("Proper Traverse Comoving Distance", proper_traverse_comovingDistance(6))
```

    Proper Traverse Comoving Distance 8910238506.077759 pc

```python
#z = [0, 1, 2, 3, 4, 5]
z = np.linspace(0.,5,6)
set_model(model = 'Einstein–de-Sitter')
DH = Hubble_Distance(0)
#Proper motion distance DM/D_H
PD = []
"""
PD.append(traverse_comovingDistance(0)/D_H)
PD.append(traverse_comovingDistance(1)/D_H)
PD.append(traverse_comovingDistance(2)/D_H)
PD.append(traverse_comovingDistance(3)/D_H)
PD.append(traverse_comovingDistance(4)/D_H)
PD.append(traverse_comovingDistance(5)/D_H)
"""
PD.append(traverse_comovingDistance(0)/DH)
PD.append(traverse_comovingDistance(1)/DH)
PD.append(traverse_comovingDistance(2)/DH)
PD.append(traverse_comovingDistance(3)/DH)
PD.append(traverse_comovingDistance(4)/DH)
PD.append(traverse_comovingDistance(5)/DH)
print ('Dm/Dh: ', PD[5])

set_model(model = 'low-density')

PD1 = []
"""
PD1.append(traverse_comovingDistance(0)/D_H)
PD1.append(traverse_comovingDistance(1)/D_H)
PD1.append(traverse_comovingDistance(2)/D_H)
PD1.append(traverse_comovingDistance(3)/D_H)
PD1.append(traverse_comovingDistance(4)/D_H)
PD1.append(traverse_comovingDistance(5)/D_H)
"""
PD1.append(traverse_comovingDistance(0)/DH)
PD1.append(traverse_comovingDistance(1)/DH)
PD1.append(traverse_comovingDistance(2)/DH)
PD1.append(traverse_comovingDistance(3)/DH)
PD1.append(traverse_comovingDistance(4)/DH)
PD1.append(traverse_comovingDistance(5)/DH)

set_model(model = 'high-lambda')

PD2 = []
PD2.append(traverse_comovingDistance(0)/D_H)
PD2.append(traverse_comovingDistance(1)/D_H)
PD2.append(traverse_comovingDistance(2)/D_H)
PD2.append(traverse_comovingDistance(3)/D_H)
PD2.append(traverse_comovingDistance(4)/D_H)
PD2.append(traverse_comovingDistance(5)/D_H)

PD = np.array(PD)
PD1 = np.array(PD1)
PD2 = np.array(PD2)
z = np.array(z)

#Plotting z and DM
plt.plot(z, PD, 'b-', linewidth=2)
plt.plot(z, PD1, 'b--', linewidth=2)
plt.plot(z, PD2, 'b', linewidth=2, linestyle='dotted')
# naming the x axis
plt.xlabel('Redshift z')
# naming the y axis
plt.ylabel('proper motion distance $D_M$ / $D_H$')
plt.legend(["Einstein–de-Sitter", "low-density", "high-lambda"])
# function to show the plot
plt.show()
```

**Angular Diameter Distance**

The angular diameter distance $D_A$ is defined as the ratio of an object’s physical transverse size to its angular size (in radians).

\begin{equation}
D_A = \frac{D_M}{(1+z)}
\end{equation}

Angular diameter distance $D_{A_{12}}$ between two objects at redshifts $z_1$ and $z_2$, frequently used in gravitational lensing.

It is not found by subtracting the two individual angular diameter distances! The correct formula (not correct for $\Omega_k < 0$), for $\Omega_k ≥ 0$, is
\begin{equation}
D_{A_{12}} = \frac{1}{(1+z_2)} [D_M2 \sqrt{1 + \Omega_k \frac{D_M1^2}{D_H^2}} - D_M1 \sqrt{1 + \Omega_k \frac{D_M2^2}{D_H^2}}]
\end{equation}

where $D_{M1}$ and $D_{M2}$ are the transverse comoving distances to $z_1$ and $z_2$

```python
def angular_diameter_distance(z, z0 = 0):
    dm = traverse_comovingDistance(z)
    if (z0 == 0):
        return dm/(1+z)
    dm_ = traverse_comovingDistance(z0)
    dh = Hubble_Distance(0.)
    #val1 = dm_ * sqrt(1. + Omega_k0 * (dm/D_H)**2.)
    #val2 = dm * sqrt(1. + Omega_k0 * (dm/D_H)**2.)
    val1 = dm_ * sqrt(1. + Omega_k0 * (dm/dh)**2.)
    val2 = dm * sqrt(1. + Omega_k0 * (dm/dh)**2.)
    DA = (val1-val2)/(1+z) # only true for Omega_k >= 0
    return DA
```

```python
set_model(model = 'Einstein–de-Sitter')
DH = Hubble_Distance(0)
#Proper motion distance DM/D_H
PD = []
"""
PD.append(angular_diameter_distance(0)/D_H)
PD.append(angular_diameter_distance(1)/D_H)
PD.append(angular_diameter_distance(2)/D_H)
PD.append(angular_diameter_distance(3)/D_H)
PD.append(angular_diameter_distance(4)/D_H)
PD.append(angular_diameter_distance(5)/D_H)
"""
PD.append(angular_diameter_distance(0)/DH)
PD.append(angular_diameter_distance(1)/DH)
PD.append(angular_diameter_distance(2)/DH)
PD.append(angular_diameter_distance(3)/DH)
PD.append(angular_diameter_distance(4)/DH)
PD.append(angular_diameter_distance(5)/DH)
#print ('DA/Dh: ', PD[5])

set_model(model = 'low-density')

PD1 = []
"""
PD1.append(angular_diameter_distance(0)/D_H)
PD1.append(angular_diameter_distance(1)/D_H)
PD1.append(angular_diameter_distance(2)/D_H)
PD1.append(angular_diameter_distance(3)/D_H)
PD1.append(angular_diameter_distance(4)/D_H)
PD1.append(angular_diameter_distance(5)/D_H)
"""
PD1.append(angular_diameter_distance(0)/DH)
PD1.append(angular_diameter_distance(1)/DH)
PD1.append(angular_diameter_distance(2)/DH)
PD1.append(angular_diameter_distance(3)/DH)
PD1.append(angular_diameter_distance(4)/DH)
PD1.append(angular_diameter_distance(5)/DH)

set_model(model = 'high-lambda')

PD2 = []
PD2.append(angular_diameter_distance(0)/DH)
PD2.append(angular_diameter_distance(1)/DH)
PD2.append(angular_diameter_distance(2)/DH)
PD2.append(angular_diameter_distance(3)/DH)
PD2.append(angular_diameter_distance(4)/DH)
PD2.append(angular_diameter_distance(5)/DH)

PD = np.array(PD)
PD1 = np.array(PD1)
PD2 = np.array(PD2)
z = np.array(z)

#Plotting z and DM
plt.plot(z, PD, 'b-')
plt.plot(z, PD1, 'b--')
plt.plot(z, PD2, 'b-.')
# naming the x axis
plt.xlabel('Redshift z')
# naming the y axis
plt.ylabel('Angular diameter distance $D_A$ / $D_H$')
plt.legend(["Einstein–de-Sitter", "low-density", "high-lambda"])
# function to show the plot
plt.show()
```

**Luminosity Distance**

The luminosity distance DL is defined by the relationship between bolometric (ie, integrated over all frequencies) flux S and bolometric luminosity L.

\begin{equation}
D_L = \sqrt{ \frac{L}{4 \pi S} } = (1+z)D_M = (1+z)^2 D_A
\end{equation}

where $D_M$ is transverse comoving distance and $D_A$ is angular diameter distance.

**Distance Modulus**

\begin{equation}
D_M = 5 log ( \frac{D_L}{10 pc} )
\end{equation}

It is the magnitude difference between an object’s observed bolometric flux and what it would be if it were at 10 pc (this was once thought to be the distance to Vega).

```python
def luminosityDistance(z):
    Da = angular_diameter_distance(z)
    return Da * (1+z)**2

```

```python
set_model(model = 'Einstein–de-Sitter')
DH = Hubble_Distance(0)
#Proper motion distance DM/D_H
PD = []

PD.append(luminosityDistance(0)/DH)
PD.append(luminosityDistance(1)/DH)
PD.append(luminosityDistance(2)/DH)
PD.append(luminosityDistance(3)/DH)
PD.append(luminosityDistance(4)/DH)
PD.append(luminosityDistance(5)/DH)
#print ('DA/Dh: ', PD[5])

set_model(model = 'low-density')

PD1 = []

PD1.append(luminosityDistance(0)/DH)
PD1.append(luminosityDistance(1)/DH)
PD1.append(luminosityDistance(2)/DH)
PD1.append(luminosityDistance(3)/DH)
PD1.append(luminosityDistance(4)/DH)
PD1.append(luminosityDistance(5)/DH)

set_model(model = 'high-lambda')

PD2 = []
PD2.append(luminosityDistance(0)/DH)
PD2.append(luminosityDistance(1)/DH)
PD2.append(luminosityDistance(2)/DH)
PD2.append(luminosityDistance(3)/DH)
PD2.append(luminosityDistance(4)/DH)
PD2.append(luminosityDistance(5)/DH)

PD = np.array(PD)
PD1 = np.array(PD1)
PD2 = np.array(PD2)
z = np.array(z)

#Plotting z and DM
plt.plot(z, PD, 'b-')
plt.plot(z, PD1, 'b--')
plt.plot(z, PD2, 'b-.')
# naming the x axis
plt.xlabel('Redshift z')
# naming the y axis
plt.ylabel('Luminosity distance $D_L$ / $D_H$')
plt.legend(["Einstein–de-Sitter", "low-density", "high-lambda"])
# function to show the plot
plt.show()
```

**Comoving Volume**

The comoving volume $V_C$ is the volume measure in which number densities of non-evolving objects locked into Hubble flow are constant with redshift.

\begin{equation}
dV_C = D_H \frac{(1+z)^2 D_A^2}{E(z)} d\Omega dz
\end{equation}

where $D_A$ is the angular diameter distance at redshift z, $d\Omega$ is solid angle and redshift interval $dz$.

The integral of the comoving volume element from the present to redshift z gives the total comoving volume, all-sky, out to redshift z

$V_C = \left\{
\begin{array}{ c l }
(\frac{4 \pi D_H^3}{2 \Omega_k}) \left [\frac{D_M}{D_H} \sqrt{1 + \Omega_k \frac{D_M^2}{D_H^2}} - \frac{1}{\sqrt{\lvert \Omega_{\kappa} \lvert}} arcsinh (\sqrt{\lvert \Omega_{\kappa} \lvert} \frac{D_M}{D_H}) \right] & \quad for \quad \Omega_{\kappa}>0,\\
\frac{4 \pi }{3} D_M^3 & \quad for \quad \Omega_{\kappa}=0, \\
(\frac{4 \pi D_H^3}{2 \Omega_k}) \left [\frac{D_M}{D_H} \sqrt{1 + \Omega_k \frac{D_M^2}{D_H^2}} - \frac{1}{\sqrt{\lvert \Omega_{\kappa} \lvert}} arcsin (\sqrt{\lvert \Omega_{\kappa} \lvert} \frac{D_M}{D_H}) \right] & \quad for \quad \Omega_{\kappa}<0
\end{array}
\right.$

$D_H^3$ is sometimes called the Hubble volum^.

```python
def diff_comovingVolume(z):
    da = angular_diameter_distance(z)
    ez = Ez(z)
    dh = Hubble_Distance(0)
    return dh* ((1+z)*da)**2 / ez

def comovingVolume(z):
    dm = traverse_comovingDistance(z)
    flatV = 4. * np.pi * dm**3. / 3. # Omega_k0=0
    if(Omega_k0==0.0) :
        return flatV
    sqrt_Omk0 = sqrt(np.abs(Omega_k0))
    angle = sqrt_Omk0 * dm/Hubble_Distance(0)
    reg1 = 4. * np.pi * Hubble_Distance(0)**3. / (2. * Omega_k0)
    reg2 = (dm/Hubble_Distance(0))*np.sqrt(1. + Omega_k0 * (dm/Hubble_Distance(0))**2.)
    Vc= 1.
    if (Omega_k0 > 0.):
        Vc = reg1 * (reg2 - asinh(angle))
    if (Omega_k0 < 0.):
        Vc = reg1 * (reg2 - asin(angle))
    return Vc

#print(comovingVolume(6))

```

```python
set_model(model = 'Einstein–de-Sitter')
dh = Hubble_Distance(0)
dh3 = dh**3
PD = []

PD.append(diff_comovingVolume(0.)/dh3)
PD.append(diff_comovingVolume(1.)/dh3)
PD.append(diff_comovingVolume(2.)/dh3)
PD.append(diff_comovingVolume(3.)/dh3)
PD.append(diff_comovingVolume(4.)/dh3)
PD.append(diff_comovingVolume(5.)/dh3)

set_model(model = 'low-density')
PD1 = []

PD1.append(diff_comovingVolume(0.)/dh3)
PD1.append(diff_comovingVolume(1.)/dh3)
PD1.append(diff_comovingVolume(2.)/dh3)
PD1.append(diff_comovingVolume(3.)/dh3)
PD1.append(diff_comovingVolume(4.)/dh3)
PD1.append(diff_comovingVolume(5.)/dh3)

set_model(model = 'high-lambda')

PD2 = []

PD2.append(diff_comovingVolume(0.)/dh3)
PD2.append(diff_comovingVolume(1.)/dh3)
PD2.append(diff_comovingVolume(2.)/dh3)
PD2.append(diff_comovingVolume(3.)/dh3)
PD2.append(diff_comovingVolume(4.)/dh3)
PD2.append(diff_comovingVolume(5.)/dh3)

PD = np.array(PD)
PD1 = np.array(PD1)
PD2 = np.array(PD2)
z = np.array(z)

#Plotting z and DM
plt.plot(z, PD, 'b-')
plt.plot(z, PD1, 'b--')
plt.plot(z, PD2, 'b-.')
# naming the x axis
plt.xlabel('Redshift z')
# naming the y axis
plt.ylabel('comoving volume element $[ 1 / D_H^3] dV_c/dz/d\Omega $')
plt.legend(["Einstein–de-Sitter", "low-density", "high-lambda"])
# function to show the plot
plt.show()
```

**Lookback Time**

The lookback time $t_L$ to an object is the difference between the age to of the Universe now (at observation) and the age te of the Universe at the time the photons were emitted (according to the object).
It is used to predict properties of high-redshift objects with evolutionary models, such as passive stellar evolution for galaxies.

\begin{equation}
t_l = t_H \int_{0}^{z} \frac{1}{(1+z') E(z')} \,dz'
\end{equation}

```python
def lookbackTime(z, z0=0):
    lt = t_H * si.quad(lambda z: (1/((1+z)*Ez(z))), z0, z)
    #print('Ez: ', Ez(z))
    #print('t_H: ', t_H)
    #print('Look back time: ', lt)
    return lt.to(units.s)

#lookbackTime(1.)
```

```python
set_model(model = 'Einstein–de-Sitter')

PD = []

PD.append(lookbackTime(0.)/t_H.to(units.s))
PD.append(lookbackTime(1.)/t_H.to(units.s))
PD.append(lookbackTime(2.)/t_H.to(units.s))
PD.append(lookbackTime(3.)/t_H.to(units.s))
PD.append(lookbackTime(4.)/t_H.to(units.s))
PD.append(lookbackTime(5.)/t_H.to(units.s))

set_model(model = 'low-density')
PD1 = []

PD1.append(lookbackTime(0.)/t_H.to(units.s))
PD1.append(lookbackTime(1.)/t_H.to(units.s))
PD1.append(lookbackTime(2.)/t_H.to(units.s))
PD1.append(lookbackTime(3.)/t_H.to(units.s))
PD1.append(lookbackTime(4.)/t_H.to(units.s))
PD1.append(lookbackTime(5.)/t_H.to(units.s))

set_model(model = 'high-lambda')

PD2 = []

PD2.append(lookbackTime(0.)/t_H.to(units.s))
PD2.append(lookbackTime(1.)/t_H.to(units.s))
PD2.append(lookbackTime(2.)/t_H.to(units.s))
PD2.append(lookbackTime(3.)/t_H.to(units.s))
PD2.append(lookbackTime(4.)/t_H.to(units.s))
PD2.append(lookbackTime(5.)/t_H.to(units.s))

PD = np.array(PD)
PD1 = np.array(PD1)
PD2 = np.array(PD2)
z = np.array(z)

#Plotting z and DM
plt.plot(z, PD, 'b-')
plt.plot(z, PD1, 'b--')
plt.plot(z, PD2, 'b-.')
# naming the x axis
plt.xlabel('Redshift z')
# naming the y axis
plt.ylabel('lookback time $t_L / t_H$')
plt.legend(["Einstein–de-Sitter", "low-density", "high-lambda"])
# function to show the plot
plt.show()
```

**Probability of intersecting objects**

The incremental probability $dP$ that a line of sight will intersect one of the objects in redshift interval $dz$ at redshift z is

\begin{equation}
dP = n(z) \sigma(z) D_H \frac{(1+z)^2}{E(z)} dz
\end{equation}

```python

```


This notebook is part of https://github.com/AudioSceneDescriptionFormat/splines, see also https://splines.readthedocs.io/.

[back to overview](catmull-rom.ipynb)

# Barry--Goldman Algorithm

The *Barry--Goldman algorithm*
(named after *Phillip Barry* and *Ronald Goldman*)
can be used to calculate values of
[non-uniform Catmull--Rom splines](catmull-rom-non-uniform.ipynb).
We have also applied this algorithm to
[rotation splines](../rotation/barry-goldman.ipynb).

(Catmull and Rom, 1974) describes
"a class of local interpolating splines" and
(Barry and Goldman, 1988) describes
"a recursive evaluation algorithm for a class of Catmull–Rom splines",
by which they mean a sub-class of the original class,
which only contains splines generated from a combination of
[Lagrange interpolation](lagrange.ipynb) and B-spline blending:

> In
particular, they observed that certain choices led to interpolatory
curves. Although Catmull and Rom discussed a more general case,
we will restrict our attention to an important class of Catmull--Rom
splines obtained by combining B-spline basis functions and Lagrange
interpolating polynomials.
> [...]
> They are
piecewise polynomial, have local support, are invariant under affine
transformations, and have certain differentiability and interpolatory
properties.
>
> ---Barry and Goldman (1988), section 1: "Introduction"

The algorithm can be set up to construct curves of arbitrary degree
(given enough vertices and their parameter values),
but here we only take a look at the cubic case
(using four vertices),
which seems to be what most people mean by the term *Catmull--Rom splines*.

The algorithm is a combination of two sub-algorithms:

> The Catmull--Rom evaluation algorithm is constructed by combining the de Boor algorithm for evaluating B-spline curves with Neville's algorithm for evaluating Lagrange
polynomials.
>
> ---Barry and Goldman (1988), abstract

Combining the two will lead to a multi-stage algorithm,
where each stage consists of only linear interpolations (and *extra*polations).

We will use the algorithm here to derive
an expression for the [tangent vectors](#Tangent-Vectors),
which will show that the algorithm indeed generates
[non-uniform Catmull--Rom splines](catmull-rom-non-uniform.ipynb#Tangent-Vectors).

## Triangular Schemes

In (Barry and Goldman, 1988),
the presented algorithms are illustrated
using triangular evaluation patterns,
which we will use here in a very similar form.

As an example, let's look at the most basic building block:
linear interpolation between two given points
(in this case $\boldsymbol{x}_4$ and $\boldsymbol{x}_5$
with corresponding parameter values $t_4$ and $t_5$, respectively):

\begin{equation*}
\begin{array}{ccccc}
&&
\boldsymbol{p}_{4,5}
&&
\\
&
\frac{t_5 - t}{t_5 - t_4}
&&
\frac{t - t_4}{t_5 - t_4}
&
\\
\boldsymbol{x}_4 &&&& \boldsymbol{x}_5
\end{array}
\end{equation*}

The values at the base of the triangle are known,
and the triangular scheme shows
how the value at the apex can be calculated from them.

In this example,
to obtain the *linear* polynomial $\boldsymbol{p}_{4,5}$
one has to add $\boldsymbol{x}_4$,
weighted by the factor shown next to it
($\frac{t_5 - t}{t_5 - t_4}$),
and $\boldsymbol{x}_5$,
weighted by the factor next to it
($\frac{t - t_4}{t_5 - t_4}$).

The parameter $t$ can be chosen arbitrarily,
but in this example we are mostly interested in the range $t_4 \le t \le t_5$.
If the parameter value is outside this range,
the process is more appropriately called *extra*polation
instead of *inter*polation.
Since we will need linear interpolation (and extrapolation) quite a few times,
let's define a helper function:

```python
def lerp(xs, ts, t):
    """Linear interpolation.

    Returns the interpolated value at time *t*,
    given the two values *xs* at times *ts*.

    """
    x_begin, x_end = xs
    t_begin, t_end = ts
    return (x_begin * (t_end - t) + x_end * (t - t_begin)) / (t_end - t_begin)
```

## Neville's Algorithm

We have already seen this algorithm in our
[notebook about Lagrange interpolation](lagrange.ipynb).

In the *quadratic* case, it looks like this:

\begin{equation*}
\begin{array}{ccccccccc}
&&&&
\boldsymbol{p}_{3,4,5}
&&&&
\\
&&&
\frac{t_5 - t}{t_5 - t_3}
&&
\frac{t - t_3}{t_5 - t_3}
&&&
\\
&& \boldsymbol{p}_{3,4} &&&& \boldsymbol{p}_{4,5} &&
\\
& \frac{t_4 - t}{t_4 - t_3} && \frac{t - t_3}{t_4 - t_3} &
& \frac{t_5 - t}{t_5 - t_4} && \frac{t - t_4}{t_5 - t_4} &
\\
\boldsymbol{x}_{3} &&&& \boldsymbol{x}_{4} &&&& \boldsymbol{x}_{5}
\end{array}
\end{equation*}

The *cubic* case is shown in figure 2 of
(Barry and Goldman, 1988).

```python
import matplotlib.pyplot as plt
import numpy as np
```

Let's try to plot this for three points:

```python
points = np.array([
    (0, 0),
    (0.5, 2),
    (3, 0),
])
```

In the following example plots we show the *uniform* case
(with $t_3=3$, $t_4=4$ and $t_5=5$),
but don't worry,
the algorithm works just as well for arbitrary non-uniform time values.

```python
plot_times = np.linspace(4, 5, 30)
```

```python
plt.scatter(*np.array([
    lerp(
        [lerp(points[:2], [3, 4], t), lerp(points[1:], [4, 5], t)],
        [3, 5], t)
    for t in plot_times]).T)
plt.plot(*points.T, 'x:g')
plt.axis('equal');
```

Note that the quadratic curve is defined by three points
but we are only evaluating it between two of them
(for $4 \le t \le 5$).

## De Boor's Algorithm

This algorithm
(named after [Carl de Boor](https://en.wikipedia.org/wiki/Carl_R._de_Boor),
see (de Boor, 1972))
can be used to calculate B-spline basis functions.

The quadratic case looks like this:

\begin{equation*}
\begin{array}{ccccccccc}
&&&&
\boldsymbol{p}_{3,4,5}
&&&&
\\
&&&
\frac{t_5 - t}{t_5 - t_4}
&&
\frac{t - t_4}{t_5 - t_4}
&&&
\\
&& \boldsymbol{p}_{3,4} &&&& \boldsymbol{p}_{4,5} &&
\\
& \frac{t_5 - t}{t_5 - t_3} && \frac{t - t_3}{t_5 - t_3} &
& \frac{t_6 - t}{t_6 - t_4} && \frac{t - t_4}{t_6 - t_4} &
\\
\boldsymbol{x}_{3} &&&& \boldsymbol{x}_{4} &&&& \boldsymbol{x}_{5}
\end{array}
\end{equation*}

The *cubic* case shown in figure 1 of
(Barry and Goldman, 1988).

```python
plt.scatter(*np.array([
    lerp(
        [lerp(points[:2], [3, 5], t), lerp(points[1:], [4, 6], t)],
        [4, 5], t)
    for t in plot_times]).T)
plt.plot(*points.T, 'x:g')
plt.axis('equal');
```

## Combining Both Algorithms

Figure 5 of (Catmull and Rom, 1974)
shows an example where linear interpolation is followed by
quadratic B-spline blending to create a cubic curve.

We can re-create this example with the building blocks from above:

* At the base of the triangle, we put four known vertices.
* Consecutive pairs of these vertices form three linear interpolations
  (and *extra*polations),
  resulting in three interpolated (and *extra*polated) values.
* On top of these three values,
  we arrange a quadratic instance of de Boor's algorithm (as shown above).

This culminates in the final value of the spline
(given an appropriate parameter value $t$)
at the apex of the triangle,
which looks like this:

\begin{equation*}
\def\negspace{\!\!\!\!\!\!}
\begin{array}{ccccccccccccc}
&&&&&&
\boldsymbol{p}_{3,4,5,6}
&&&&&&
\\
&&&&&
\negspace \frac{t_5 - t}{t_5 - t_4} \negspace
&&
\negspace \frac{t - t_4}{t_5 - t_4} \negspace
&&&&&
\\
&&&& \boldsymbol{p}_{3,4,5} &&&& \boldsymbol{p}_{4,5,6} &&&&
\\
&&
& \negspace \frac{t_5 - t}{t_5 - t_3} \negspace && \negspace \frac{t - t_3}{t_5 - t_3} \negspace &
& \negspace \frac{t_6 - t}{t_6 - t_4} \negspace && \negspace \frac{t - t_4}{t_6 - t_4} \negspace &
&&
\\
&& \boldsymbol{p}_{3,4} &&&& \boldsymbol{p}_{4,5} &&&& \boldsymbol{p}_{5,6} &&
\\
& \negspace \frac{t_4 - t}{t_4 - t_3} \negspace && \negspace \frac{t - t_3}{t_4 - t_3} \negspace &
& \negspace \frac{t_5 - t}{t_5 - t_4} \negspace && \negspace \frac{t - t_4}{t_5 - t_4} \negspace &
& \negspace \frac{t_6 - t}{t_6 - t_5} \negspace && \negspace \frac{t - t_5}{t_6 - t_5} \negspace &
\\
\boldsymbol{x}_3 &&&& \boldsymbol{x}_4 &&&& \boldsymbol{x}_5 &&&& \boldsymbol{x}_6
\end{array}
\end{equation*}

Here we are considering the fifth spline segment
$\boldsymbol{p}_{3,4,5,6}(t)$
(represented at the apex of the triangle)
from
$\boldsymbol{x}_4$ to
$\boldsymbol{x}_5$
(to be found at the base of the triangle)
which corresponds to
the parameter range $t_4 \le t \le t_5$.
To calculate the values in this segment,
we also need to know the preceding control point $\boldsymbol{x}_3$
(at the bottom left)
and the following control point $\boldsymbol{x}_6$
(at the bottom right).
But not only their positions are relevant,
we also need the corresponding parameter values
$t_3$ and $t_6$, respectively.

This same triangular scheme is also shown in figure 3 of
(Yuksel et al., 2011),
except that here we shifted the indices by $+3$.

Another way to construct a cubic curve with this algorithm
would be to flip the degrees of interpolation and blending,
in other words:

* Instead of three linear interpolations (and extrapolations),
  apply two overlapping quadratic Lagrange interpolations
  using Neville's algorithm (as shown above) to
  $\boldsymbol{x}_3$, $\boldsymbol{x}_4$, $\boldsymbol{x}_5$ and
  $\boldsymbol{x}_4$, $\boldsymbol{x}_5$, $\boldsymbol{x}_6$, respectively.
  Note that the interpolation of $\boldsymbol{x}_4$ and $\boldsymbol{x}_5$
  appears in both triangles but has to be calculated only once
  (see also figures 3 and 4 in
  (Barry and Goldman, 1988)).
* This will occupy the lower two stages of the triangle,
  yielding two interpolated values.
* Those two values are then linearly blended in the final stage.

Readers of the
[notebook about uniform Catmull--Rom splines](catmull-rom-uniform.ipynb)
may already suspect that,
for others it might be a revelation: both ways
lead to exactly the same triangular scheme
and therefore they are equivalent!

The same scheme, but only for the *uniform* case, is also shown in figure 7 of
(Barry and Goldman, 1988),
which casually mentions the equivalent cases
(with $m$ being the degree of Lagrange interpolation
and $n$ being the degree of the B-spline basis functions):

> Note too from Figure 7 that the case
$n=1$, $m=2$ [...] is identical to the case
$n=2$, $m=1$ [...]
>
> ---Barry and Goldman (1988), section 3: "Examples"

Not an Overhauser Spline

Equally casually, they mention:

> Finally, the particular case here is also an Overhauser spline
> (Overhauser, 1968).
>
> ---Barry and Goldman (1988), section 3: "Examples"

This is not true.
Overhauser splines
-- as described in
(Overhauser, 1968) --
don't provide a choice of parameter values.
The parameter values are determined
by the Euclidean distances between control points,
similar, but not quite identical to
[chordal parameterization](catmull-rom-properties.ipynb#Chordal-Parameterization).
Calculating a value of a Catmull--Rom spline doesn't involve calculating any distances.

For completeness' sake,
there are two more combinations that lead to cubic splines,
but they have their limitations:

* Cubic Lagrange interpolation, followed by no blending at all,
  which leads to a cubic spline that's not $C^1$ continuous (only $C^0$),
  as shown in figure 8 of
  (Barry and Goldman, 1988).
* No interpolation at all, followed by cubic B-spline blending,
  which leads to an approximating spline (instead of an interpolating spline),
  as shown in figure 5 of
  (Barry and Goldman, 1988).

Note

Here we are using the time instances of the Lagrange interpolation
also as B-spline knots.
Equation (9) of
(Barry and Goldman, 1988)
shows a more generic formulation of the algorithm
with separate parameters $s_i$ and $t_i$.

## Step by Step

The triangular figure above looks more complicated than it really is.
It's just a bunch of linear *inter*polations and *extra*polations.

Let's go through the figure above, piece by piece.

```python
import sympy as sp
```

```python
t = sp.symbols('t')
```

```python
x3, x4, x5, x6 = sp.symbols('xbm3:7')
```

```python
t3, t4, t5, t6 = sp.symbols('t3:7')
```

We use some custom SymPy-based tools from [utility.py](utility.py):

```python
from utility import NamedExpression, NamedMatrix
```

### First Stage

In the center of the bottom row,
there is a straightforward linear interpolation
from $\boldsymbol{x}_4$ to $\boldsymbol{x}_5$
within the interval from $t_4$ to $t_5$.

```python
p45 = NamedExpression('pbm_4,5', lerp([x4, x5], [t4, t5], t))
p45
```

Obviously, this starts at:

```python
p45.evaluated_at(t, t4)
```

... and ends at:

```python
p45.evaluated_at(t, t5)
```

The bottom left of the triangle looks very similar,
with a linear interpolation
from $\boldsymbol{x}_3$ to $\boldsymbol{x}_4$
within the interval from $t_3$ to $t_4$.

```python
p34 = NamedExpression('pbm_3,4', lerp([x3, x4], [t3, t4], t))
p34
```

However, that's not the parameter range we are interested in!
We are interested in the range from $t_4$ to $t_5$.
Therefore, this is not actually an *inter*polation between
$\boldsymbol{x}_3$ and $\boldsymbol{x}_4$,
but rather a linear *extra*polation starting at $\boldsymbol{x}_4$ ...

```python
p34.evaluated_at(t, t4)
```

... and ending at some extrapolated point beyond $\boldsymbol{x}_4$:

```python
p34.evaluated_at(t, t5)
```

Similarly, at the bottom right of the triangle
there isn't a linear *inter*polation
from $\boldsymbol{x}_5$ to $\boldsymbol{x}_6$,
but rather a linear *extra*polation that just reaches
$\boldsymbol{x}_5$ at the end of the parameter interval
(i.e. at $t=t_5$).

```python
p56 = NamedExpression('pbm_5,6', lerp([x5, x6], [t5, t6], t))
p56
```

```python
p56.evaluated_at(t, t4)
```

```python
p56.evaluated_at(t, t5)
```

### Second Stage

The second stage of the algorithm
involves linear interpolations of the results of the previous stage.

```python
p345 = NamedExpression('pbm_3,4,5', lerp([p34.name, p45.name], [t3, t5], t))
p345
```

```python
p456 = NamedExpression('pbm_4,5,6', lerp([p45.name, p56.name], [t4, t6], t))
p456
```

Those interpolations are defined over a parameter range
from $t_3$ to $t_5$ and
from $t_4$ to $t_6$, respectively.
In each case, we are only interested in a sub-range,
namely from $t_4$ to $t_5$.

These are the start and end points at $t_4$ and $t_5$:

```python
p345.evaluated_at(t, t4, symbols=[p34, p45])
```

```python
p345.evaluated_at(t, t5, symbols=[p34, p45])
```

```python
p456.evaluated_at(t, t4, symbols=[p45, p56])
```

```python
p456.evaluated_at(t, t5, symbols=[p45, p56])
```

### Third Stage

The last step is quite simple:

```python
p3456 = NamedExpression(
    'pbm_3,4,5,6',
    lerp([p345.name, p456.name], [t4, t5], t))
p3456
```

This time, the interpolation interval is exactly the one we care about.

To get the final result, we just have to combine all the above expressions:

```python
p3456 = p3456.subs_symbols(p345, p456, p34, p45, p56).simplify()
p3456
```

We can make this marginally shorter
if we rewrite the segment durations as
$\Delta_i = t_{i+1} - t_i$:

```python
delta3, delta4, delta5 = sp.symbols('Delta3:6')
deltas = {
    t4 - t3: delta3,
    t5 - t4: delta4,
    t6 - t5: delta5,
    t5 - t3: delta3 + delta4,
    t6 - t4: delta4 + delta5,
    t6 - t3: delta3 + delta4 + delta5,
    # A few special cases that SymPy has a hard time resolving:
    t4 + t4 - t3: t4 + delta3,
    t6 + t6 - t3: t6 + delta3 + delta4 + delta5,
}
```

```python
p3456.subs(deltas)
```

Apart from checking if it's really cubic ...

```python
sp.degree(p3456.expr, t)
```

... and if it's really interpolating ...

```python
p3456.evaluated_at(t, t4).simplify()
```

```python
p3456.evaluated_at(t, t5).simplify()
```

... the only thing left to do is to check its ...

## Tangent Vectors

To get the tangent vectors at the control points,
we just have to take the first derivative ...

```python
pd3456 = p3456.diff(t)
```

... and evaluate it at $t_4$ and $t_5$:

```python
pd3456.evaluated_at(t, t4).simplify().simplify()
```

```python
pd3456.evaluated_at(t, t5).simplify()
```

If all went well,
this should be identical to the result in
[the notebook about non-uniform Catmull--Rom splines](catmull-rom-non-uniform.ipynb#Tangent-Vectors).

## Animation

The linear interpolations (and *extra*polations) of this algorithm
can be shown graphically.
By means of the file [barry_goldman.py](barry_goldman.py),
we can generate an animation of the algorithm:

```python
from barry_goldman import animation
```

```python
from IPython.display import HTML
```

```python
vertices = [
    (1, 0),
    (0.5, 1),
    (6, 2),
    (5, 0),
]
```

```python
times = [
    0,
    1,
    6,
    8,
]
```

```python
ani = animation(vertices, times)
```

```python
HTML(ani.to_jshtml(default_mode='reflect'))
```

If this doesn't look very intuitive to you,
you are not alone.
For a different (and probably more straightforward) point of view,
have a look at the
[notebook about non-uniform Catmull--Rom splines](catmull-rom-non-uniform.ipynb#Animation).


# 7 Solution Methods to Solve the Growth Model with Julia

This notebook is part of a computational appendix that accompanies the paper

> MATLAB, Python, Julia: What to Choose in Economics?
> > Coleman, Lyon, Maliar, and Maliar (2017)

In order to run the codes in this notebook you will need to install and configure a few Julia packages. We recommend following the instructions on [quantecon.org](https://lectures.quantecon.org/jl/getting_started.html).

Once your Julia installation is up and running, there are a few additional packages you will need in order to run the code here. To do this uncomment the lines in the cell below (by deleting the `#` and space at the beginning of each line) and run the cell:

```julia
using Pkg
pkg"add InstantiateFromURL"
```

      Updating registry at `~/.julia/registries/General`
      Updating git-repo `https://github.com/JuliaRegistries/General.git`
     Resolving package versions...
      Updating `~/.julia/environments/v1.1/Project.toml`
     [no changes]
      Updating `~/.julia/environments/v1.1/Manifest.toml`
     [no changes]

```julia
using InstantiateFromURL: activate_github_path
activate_github_path("sglyon/CLMMJuliaPythonMatlab", path="Growth/julia", activate=true, force=true)
```

    ┌ Info: Recompiling stale cache file /Users/sglyon/.julia/compiled/v1.1/InstantiateFromURL/vAXbt.ji for InstantiateFromURL [43edad99-fa64-5e4f-9937-1c09a410b73f]
    └ @ Base loading.jl:1184

      Updating registry at `~/.julia/registries/General`
      Updating git-repo `https://github.com/JuliaRegistries/General.git`
    Precompiling project...

```julia
using Printf, Random, LinearAlgebra

using BasisMatrices, Optim, QuantEcon, Parameters
using BasisMatrices: Degree, Derivative
```

    ┌ Info: Recompiling stale cache file /Users/sglyon/.julia/compiled/v1.1/BasisMatrices/65PSC.ji for BasisMatrices [08854c51-b66b-5062-a90d-8e7ae4547a49]
    └ @ Base loading.jl:1184

## Model

This section gives a short description of the commonly used stochastic Neoclassical growth model.

There is a single infinitely-lived representative agent who consumes and saves using capital. The consumer discounts the future with factor $\beta$ and derives utility from only consumption. Additionally, saved capital will depreciate at $\delta$.

The consumer has access to a Cobb-Douglas technology which uses capital saved from the previous period to produce and is subject to stochastic productivity shocks.

Productivity shocks follow an AR(1) in logs.

The agent's problem can be written recursively using the following Bellman equation

\begin{align}
  V(k_t, z_t) &= \max_{k_{t+1}} u(c_t) + \beta E \left[ V(k_{t+1}, z_{t+1}) \right] \\
  &\text{subject to } \\
  c_t &= z_t f(k_t) + (1 - \delta) k_t - k_{t+1} \\
  \log z_{t+1} &= \rho \log z_t + \sigma \varepsilon
\end{align}

## Julia Code

We begin by defining a type that describes our model. It will hold the three things

1. Parameters of the growth model
2. Grids used for approximating the solution
3. Nodes and weights used to approximate integration

Note the `@with_kw` comes from the `Parameters` package -- It allows one to specify default arguments for the parameters when building a type (for more information refer to their [documentation](http://parametersjl.readthedocs.io/en/latest/)). One of the benefits of using the `Parameters` package is their code allows us to do things like, `@unpack a, b, c = Params` which takes elements from inside the type `Params` and "unpacks" them... i.e. it automates code of the form `a, b, c = Params.a, Params.b, Params.c`

```julia
"""
The stochastic Neoclassical growth model type contains parameters
which define the model

* α: Capital share in output
* β: Discount factor
* δ: Depreciation rate
* γ: Risk aversion
* ρ: Persistence of the log of the productivity level
* σ: Standard deviation of shocks to log productivity level
* A: Coefficient on C-D production function

* kgrid: Grid over capital
* zgrid: Grid over productivity
* grid: Grid of (k, z) pairs
* eps_nodes: Nodes used to integrate
* weights: Weights used to integrate
* z1: A grid of the possible z1s tomorrow given eps_nodes and zgrid
"""
@with_kw struct NeoclassicalGrowth
    # Parameters
    α::Float64 = 0.36
    β::Float64 = 0.99
    δ::Float64 = 0.02
    γ::Float64 = 2.0
    ρ::Float64 = 0.95
    σ::Float64 = 0.01
    A::Float64 = (1.0/β - (1 - δ)) / α

    # Grids
    kgrid::Vector{Float64} = collect(range(0.9, stop=1.1, length=10))
    zgrid::Vector{Float64} = collect(range(0.9, stop=1.1, length=10))
    grid::Matrix{Float64} = gridmake(kgrid, zgrid)
    eps_nodes::Vector{Float64} = qnwnorm(5, 0.0, σ^2)[1]
    weights::Vector{Float64} = qnwnorm(5, 0.0, σ^2)[2]
    z1::Matrix{Float64} = (zgrid.^(ρ))' .* exp.(eps_nodes)
end
```

    NeoclassicalGrowth

We also define some useful functions so that we [don't repeat ourselves](https://lectures.quantecon.org/py/writing_good_code.html#don-t-repeat-yourself) later in the code.

```julia
# Helper functions
f(ncgm::NeoclassicalGrowth, k, z) = @. z * (ncgm.A * k^ncgm.α)
df(ncgm::NeoclassicalGrowth, k, z) = @. ncgm.α * z * (ncgm.A * k^(ncgm.α - 1.0))

u(ncgm::NeoclassicalGrowth, c) = c > 1e-10 ? @.(c^(1-ncgm.γ)-1)/(1-ncgm.γ) : -1e10
du(ncgm::NeoclassicalGrowth, c) = c > 1e-10 ? c.^(-ncgm.γ) : 1e10
duinv(ncgm::NeoclassicalGrowth, u) = u .^ (-1 / ncgm.γ)

expendables_t(ncgm::NeoclassicalGrowth, k, z) = (1-ncgm.δ)*k + f(ncgm, k, z)

```

    expendables_t (generic function with 1 method)

## Solution Methods

In this notebook, we describe the following solution methods:

* Conventional Value Function Iteration
* Envelope Condition Value Function Iteration
* Envelope Condition Derivative Value Function Iteration
* Endogenous Grid Value Function Iteration
* Conventional Policy Function Iteration
* Envelope Condition Policy Function Iteration
* Euler Equation Method

Each of these solution methods will have a very similar structure that follows a few basic steps:

1. Guess a function (either value function or policy function).
2. Using this function, update our guess of both the value and policy functions.
3. Check whether the function we guessed and what it was updated to are similar enough. If so, proceed. If not, return to step 2 using the newly updated functions.
4. Output the policy and value functions.

In order to reduce the amount of repeated code and keep the exposition as clean as possible (the notebook is plenty long as is...), we will define multiple solution types that will have a more general (abstract) type called `SolutionMethod`. A solution can then be characterized by a concrete type `ValueCoeffs` (a special case for each solution method) which consists of an approximation degree, coefficients for the value function, and coefficients for the policy function. The rest of the functions below that are just more helper methods. We will then define a general solve method that applies steps 1, 3, and 4 from the algorithm above. Finally, we will implement a special method to do step 2 for each of the algorithms.

These implementation may seem a bit confusing at first (though hopefully the idea itself feels intuitive) -- The implementation takes advantage of a powerful type system in Julia.

```julia
# Types for solution methods
abstract type SolutionMethod end

struct IterateOnPolicy <: SolutionMethod end
struct VFI_ECM <: SolutionMethod end
struct VFI_EGM <: SolutionMethod end
struct VFI <: SolutionMethod end
struct PFI_ECM <: SolutionMethod end
struct PFI <: SolutionMethod end
struct dVFI_ECM <: SolutionMethod end
struct EulEq <: SolutionMethod end

#
# Type for Approximating Value and Policy
#
mutable struct ValueCoeffs{T <: SolutionMethod,D <: Degree}
    d::D
    v_coeffs::Vector{Float64}
    k_coeffs::Vector{Float64}
end

function ValueCoeffs(::Type{Val{d}}, method::T) where T <: SolutionMethod where d
    # Initialize two vectors of zeros
    deg = Degree{d}()
    n = n_complete(2, deg)
    v_coeffs = zeros(n)
    k_coeffs = zeros(n)

    return ValueCoeffs{T,Degree{d}}(deg, v_coeffs, k_coeffs)
end

function ValueCoeffs(
        ncgm::NeoclassicalGrowth, ::Type{Val{d}}, method::T
    ) where T <: SolutionMethod where d
    # Initialize with vector of zeros
    deg = Degree{d}()
    n = n_complete(2, deg)
    v_coeffs = zeros(n)

    # Policy guesses based on k and z
    k, z = ncgm.grid[:, 1], ncgm.grid[:, 2]
    css = ncgm.A - ncgm.δ
    yss = ncgm.A
    c_pol = f(ncgm, k, z) * (css/yss)

    # Figure out what kp is
    k_pol = expendables_t(ncgm, k, z) - c_pol
    k_coeffs = complete_polynomial(ncgm.grid, d) \ k_pol

    return ValueCoeffs{T,Degree{d}}(deg, v_coeffs, k_coeffs)
end

solutionmethod(::ValueCoeffs{T}) where T <:SolutionMethod = T

# A few copy methods to make life easier
Base.copy(vp::ValueCoeffs{T,D}) where T where D =
    ValueCoeffs{T,D}(vp.d, vp.v_coeffs, vp.k_coeffs)

function Base.copy(vp::ValueCoeffs{T1,D}, ::T2) where T1 where D where T2 <: SolutionMethod
    ValueCoeffs{T2,D}(vp.d, vp.v_coeffs, vp.k_coeffs)
end

function Base.copy(
        ncgm::NeoclassicalGrowth, vp::ValueCoeffs{T}, ::Type{Val{new_degree}}
    ) where T where new_degree
    # Build Value and policy matrix
    deg = Degree{new_degree}()
    V = build_V(ncgm, vp)
    k = build_k(ncgm, vp)

    # Build new Phi
    Phi = complete_polynomial(ncgm.grid, deg)
    v_coeffs = Phi \ V
    k_coeffs = Phi \ k

    return ValueCoeffs{T,Degree{new_degree}}(deg, v_coeffs, k_coeffs)
end
```

We will need to repeatedly update coefficients, build $V$ (or $dV$ depending on the solution method), and be able to compute expected values, so we define some additional helper functions below.

```julia
"""
Updates the coefficients for the value function inplace in `vp`
"""
function update_v!(vp::ValueCoeffs, new_coeffs::Vector{Float64}, dampen::Float64)
    vp.v_coeffs = (1-dampen)*vp.v_coeffs + dampen*new_coeffs
end

"""
Updates the coefficients for the policy function inplace in `vp`
"""
function update_k!(vp::ValueCoeffs, new_coeffs::Vector{Float64}, dampen::Float64)
    vp.k_coeffs = (1-dampen)*vp.k_coeffs + dampen*new_coeffs
end

"""
Builds either V or dV depending on the solution method that is given. If it
is a solution method that iterates on the derivative of the value function
then it will return derivative of the value function, otherwise the
value function itself
"""
build_V_or_dV(ncgm::NeoclassicalGrowth, vp::ValueCoeffs) =
    build_V_or_dV(ncgm, vp, solutionmethod(vp)())

build_V_or_dV(ncgm, vp::ValueCoeffs, ::SolutionMethod) = build_V(ncgm, vp)
build_V_or_dV(ncgm, vp::ValueCoeffs, T::dVFI_ECM) = build_dV(ncgm, vp)

function build_dV(ncgm::NeoclassicalGrowth, vp::ValueCoeffs)
    Φ = complete_polynomial(ncgm.grid, vp.d, Derivative{1}())
    Φ*vp.v_coeffs
end

function build_V(ncgm::NeoclassicalGrowth, vp::ValueCoeffs)
    Φ = complete_polynomial(ncgm.grid, vp.d)
    Φ*vp.v_coeffs
end

function build_k(ncgm::NeoclassicalGrowth, vp::ValueCoeffs)
    Φ = complete_polynomial(ncgm.grid, vp.d)
    Φ*vp.k_coeffs
end

```

    build_k (generic function with 1 method)

Additionally, in order to evaluate the value function, we will need to be able to take expectations.

These functions evaluates expectations by taking the policy $k_{t+1}$ and the current productivity state $z_t$ as inputs. They then integrates over the possible $z_{t+1}$s.

```julia
function compute_EV!(cp_kpzp::Vector{Float64}, ncgm::NeoclassicalGrowth,
                     vp::ValueCoeffs, kp, iz)
    # Pull out information from types
    z1, weightsz = ncgm.z1, ncgm.weights

    # Get number nodes
    nzp = length(weightsz)

    EV = 0.0
    for izp in 1:nzp
        zp = z1[izp, iz]
        complete_polynomial!(cp_kpzp, [kp, zp], vp.d)
        EV += weightsz[izp] * dot(vp.v_coeffs, cp_kpzp)
    end

    return EV
end

function compute_EV(ncgm::NeoclassicalGrowth, vp::ValueCoeffs, kp, iz)
    cp_kpzp = Array{Float64}(undef, n_complete(2, vp.d))

    return compute_EV!(cp_kpzp, ncgm, vp, kp, iz)
end

function compute_EV(ncgm::NeoclassicalGrowth, vp::ValueCoeffs)
    # Get length of k and z grids
    kgrid, zgrid = ncgm.kgrid, ncgm.zgrid
    nk, nz = length(kgrid), length(zgrid)
    temp = Array{Float64}(undef, n_complete(2, vp.d))

    # Allocate space to store EV
    EV = Array{Float64}(undef, nk*nz)

    _inds = LinearIndices((nk, nz))

    for ik in 1:nk, iz in 1:nz
        # Pull out states
        k = kgrid[ik]
        z = zgrid[iz]
        ikiz_index = _inds[ik, iz]

        # Pass to scalar EV
        complete_polynomial!(temp, [k, z], vp.d)
        kp = dot(vp.k_coeffs, temp)
        EV[ikiz_index] = compute_EV!(temp, ncgm, vp, kp, iz)
    end

    return EV
end

function compute_dEV!(cp_dkpzp::Vector,  ncgm::NeoclassicalGrowth,
                      vp::ValueCoeffs, kp, iz)
    # Pull out information from types
    z1, weightsz = ncgm.z1, ncgm.weights

    # Get number nodes
    nzp = length(weightsz)

    dEV = 0.0
    for izp in 1:nzp
        zp = z1[izp, iz]
        complete_polynomial!(cp_dkpzp, [kp, zp], vp.d, Derivative{1}())
        dEV += weightsz[izp] * dot(vp.v_coeffs, cp_dkpzp)
    end

    return dEV
end

function compute_dEV(ncgm::NeoclassicalGrowth, vp::ValueCoeffs, kp, iz)
    compute_dEV!(Array{Float64}(undef, n_complete(2, vp.d)), ncgm, vp, kp, iz)
end

```

    compute_dEV (generic function with 1 method)

###  General Solution Method

As promised, below is some code that "generally" applies the algorithm that we described -- Notice that it is implemented for a type `ValueCoeffs{SolutionMethod}` which is our abstract type. We will define a special version of `update` for each solution method and then we will only need this one `solve` method and won't repeat the more tedious portions of our code.

```julia
function solve(
        ncgm::NeoclassicalGrowth, vp::ValueCoeffs;
        tol::Float64=1e-6, maxiter::Int=5000, dampen::Float64=1.0,
        nskipprint::Int=1, verbose::Bool=true
    )
    # Get number of k and z on grid
    nk, nz = length(ncgm.kgrid), length(ncgm.zgrid)

    # Build basis matrix and value function
    dPhi = complete_polynomial(ncgm.grid, vp.d, Derivative{1}())
    Phi = complete_polynomial(ncgm.grid, vp.d)
    V = build_V_or_dV(ncgm, vp)
    k = build_k(ncgm, vp)
    Vnew = copy(V)
    knew = copy(k)

    # Print column names
    if verbose
        @printf("| Iteration | Distance V | Distance K |\n")
    end

    # Iterate to convergence
    dist, iter = 10.0, 0
    while (tol < dist) & (iter < maxiter)
        # Update the value function using appropriate update methods
        update!(Vnew, knew, ncgm, vp, Phi, dPhi)

        # Compute distance and update all relevant elements
        iter += 1
        dist_v = maximum(abs, 1.0 .- Vnew./V)
        dist_k = maximum(abs, 1.0 .- knew./k)
        copy!(V, Vnew)
        copy!(k, knew)

        # If we are iterating on a policy, use the difference of values
        # otherwise use the distance on policy
        dist = ifelse(solutionmethod(vp) == IterateOnPolicy, dist_v, dist_k)

        # Print status update
        if verbose && (iter%nskipprint == 0)
            @printf("|%-11d|%-12e|%-12e|\n", iter, dist_v, dist_k)
        end
    end

    # Update value and policy functions one last time as long as the
    # solution method isn't IterateOnPolicy
    if ~(solutionmethod(vp) == IterateOnPolicy)
        # Update capital policy after finished
        kp = env_condition_kp(ncgm, vp)
        update_k!(vp, complete_polynomial(ncgm.grid, vp.d) \ kp, 1.0)

        # Update value function according to specified policy
        vp_igp = copy(vp, IterateOnPolicy())
        solve(ncgm, vp_igp; tol=1e-10, maxiter=5000, verbose=false)
        update_v!(vp, vp_igp.v_coeffs, 1.0)

    end

    return vp
end

```

    solve (generic function with 1 method)

### Iterating to Convergence (given policy)

This isn't one of the methods described above, but it is used as an element of a few of our methods (and also as a way to get a first guess at the value function). This method takes an initial policy function, $\bar{k}(k_t, z_t)$, as given, and then, without changing the policy, iterates until the value function has converged.

Thus the "update section" of the algorithm in this instance would be:

* Leave policy function unchanged
* At each point of grid, $(k_t, z_t)$, compute $\hat{V}(k_t, z_t) = u(c(\bar{k}(k_t, z_t))) + \beta E \left[ V(\bar{k}(k_t, z_t), z_{t+1}) \right]$

```julia
function update!(V::Vector{Float64}, kpol::Vector{Float64},
                 ncgm::NeoclassicalGrowth, vp::ValueCoeffs{IterateOnPolicy},
                 Φ::Matrix{Float64}, dΦ::Matrix{Float64})
    # Get sizes and allocate for complete_polynomial
    kgrid = ncgm.kgrid; zgrid = ncgm.zgrid;
    nk, nz = length(kgrid), length(zgrid)

    _inds = LinearIndices((nk, nz))

    # Iterate over all states
    for ik in 1:nk, iz in 1:nz
        # Pull out states
        k = kgrid[ik]
        z = zgrid[iz]

        # Pull out policy and evaluate consumption
        ikiz_index = _inds[ik, iz]
        k1 = kpol[ikiz_index]
        c = expendables_t(ncgm, k, z) - k1

        # New value
        EV = compute_EV(ncgm, vp, k1, iz)
        V[ikiz_index] = u(ncgm, c) + ncgm.β*EV
    end

    # Update coefficients
    update_v!(vp, Φ \ V, 1.0)
    update_k!(vp, Φ \ kpol, 1.0)

    return V
end

```

    update! (generic function with 1 method)

### Conventional Value Function Iteration

This is one of the first solution methods for macroeconomics a graduate student in economics typically learns.

In this solution method, one takes as given a value function, $V(k_t, z_t)$, and then solves for the optimal policy given the value function.

The update section takes the form:

* For each point, $(k_t, z_t)$, numerically solve for $c^*(k_t, z_t)$ to satisfy the first order condition $u'(c^*) = \beta E \left[ V_1((1 - \delta) k_t + z_t f(k_t) - c^*, z_{t+1}) \right]$
* Define $k^*(k_t, z_t) = (1 - \delta) k_t + z_t f(k_t) - c^*(k_t, z_t)$
* Update value function according to $\hat{V}(k_t, z_t) = u(c^*(k_t, z_t)) + \beta E \left[ V(k^*(k_t, z_t), z_{t+1}) \right]$

```julia
function update!(V::Vector{Float64}, kpol::Vector{Float64},
                 ncgm::NeoclassicalGrowth, vp::ValueCoeffs{VFI},
                 Φ::Matrix{Float64}, dΦ::Matrix{Float64})
    # Get sizes and allocate for complete_polynomial
    kgrid = ncgm.kgrid; zgrid = ncgm.zgrid
    nk, nz = length(kgrid), length(zgrid)

    # Iterate over all states
    temp = Array{Float64}(undef, n_complete(2, vp.d))
    _inds = LinearIndices((nk, nz))
    for iz=1:nz, ik=1:nk
        k = kgrid[ik]; z = zgrid[iz]

        # Define an objective function (negative for minimization)
        y = expendables_t(ncgm, k, z)
        solme(kp) = du(ncgm, y - kp) - ncgm.β*compute_dEV!(temp, ncgm, vp, kp, iz)

        # Find sol to foc
        kp = brent(solme, 1e-8, y-1e-8; rtol=1e-12)
        c = expendables_t(ncgm, k, z) - kp

        # New value
        ikiz_index = _inds[ik, iz]
        EV = compute_EV!(temp, ncgm, vp, kp, iz)
        V[ikiz_index] = u(ncgm, c) + ncgm.β*EV
        kpol[ikiz_index] = kp
    end

    # Update coefficients
    update_v!(vp, Φ \ V, 1.0)
    update_k!(vp, Φ \ kpol, 1.0)

    return V
end

```

    update! (generic function with 2 methods)

### Endogenous Grid Value Function Iteration

Method introduced by Chris Carroll. The key to this method is that the grid of points being used to approximate is over $(k_{t+1}, z_{t})$ instead of $(k_t, z_t)$. The insightful piece of this algorithm is that the transformation allows one to write a closed form for the consumption function, $c^*(k_{t+1}, z_t) = u'^{-1} \left( V_1(k_{t+1}, z_{t+1}) \right]$.

Then for a given $(k_{t+1}, z_{t})$ the update section would be

* Define $c^*(k_{t+1}, z_t) = u'^{-1} \left( \beta E \left[ V_1(k_{t+1}, z_{t+1}) \right] \right)$
* Find $k_t$ such that $k_t = (1 - \delta) k_t + z_t f(k_t) - k_{t+1}$
* Update value function according to $\hat{V}(k_t, z_t) = u(c^*(k_{t+1}, z_t)) + \beta E \left[ V(k_{t+1}, z_{t+1}) \right]$

```julia
function update!(V::Vector{Float64}, kpol::Vector{Float64},
                 ncgm::NeoclassicalGrowth, vp::ValueCoeffs{VFI_EGM},
                 Φ::Matrix{Float64}, dΦ::Matrix{Float64})
    # Get sizes and allocate for complete_polynomial
    kgrid = ncgm.kgrid; zgrid = ncgm.zgrid; grid = ncgm.grid;
    nk, nz = length(kgrid), length(zgrid)

    # Iterate
    temp = Array{Float64}(undef, n_complete(2, vp.d))
    _inds = LinearIndices((nk, nz))
    for iz=1:nz, ik=1:nk

        # In EGM we use the grid points as if they were our
        # policy for yesterday and find implied kt
        ikiz_index = _inds[ik, iz]
        k1 = kgrid[ik];z = zgrid[iz];

        # Compute the derivative of expected values
        dEV = compute_dEV!(temp, ncgm, vp, k1, iz)

        # Compute optimal consumption
        c = duinv(ncgm, ncgm.β*dEV)

        # Need to find corresponding kt for optimal c
        obj(kt) = expendables_t(ncgm, kt, z) - c - k1
        kt_star = brent(obj, 0.0, 2.0, xtol=1e-10)

        # New value
        EV = compute_EV!(temp, ncgm, vp, k1, iz)
        V[ikiz_index] = u(ncgm, c) + ncgm.β*EV
        kpol[ikiz_index] = kt_star
    end

    # New Φ (has our new "kt_star" and z points)
    Φ_egm = complete_polynomial([kpol grid[:, 2]], vp.d)

    # Update coefficients
    update_v!(vp, Φ_egm \ V, 1.0)
    update_k!(vp, Φ_egm \ grid[:, 1], 1.0)

    # Update V and kpol to be value and policy corresponding
    # to our grid again
    copy!(V, Φ*vp.v_coeffs)
    copy!(kpol, Φ*vp.k_coeffs)

    return V
end
```

    update! (generic function with 3 methods)

### Envelope Condition Value Function Iteration

Very similar to the previous method. The insight of this algorithm is that since we are already approximating the value function and can evaluate its derivative, we can skip the numerical optimization piece of the update method and compute directly the policy using the envelope condition (hence the name).

The envelope condition says:

$$c^*(k_t, z_t) = u'^{-1} \left( \frac{\partial V(k_t, z_t)}{\partial k_t} (1 - \delta + r)^{-1} \right)$$

so

$$k^*(k_t, z_t) = z_t f(k_t) + (1-\delta)k_t - c^*(k_t, z_t)$$

The functions below compute the policy using the envelope condition.

```julia
function env_condition_kp!(cp_out::Vector{Float64}, ncgm::NeoclassicalGrowth,
                           vp::ValueCoeffs, k::Float64, z::Float64)
    # Compute derivative of VF
    dV = dot(vp.v_coeffs, complete_polynomial!(cp_out, [k, z], vp.d, Derivative{1}()))

    # Consumption is then computed as
    c = duinv(ncgm, dV / (1 - ncgm.δ .+ df(ncgm, k, z)))

    return expendables_t(ncgm, k, z) - c
end

function env_condition_kp(ncgm::NeoclassicalGrowth, vp::ValueCoeffs,
                          k::Float64, z::Float64)
    cp_out = Array{Float64}(undef, n_complete(2, vp.d))
    env_condition_kp!(cp_out, ncgm, vp, k, z)
end

function env_condition_kp(ncgm::NeoclassicalGrowth, vp::ValueCoeffs)
    # Pull out k and z from grid
    k = ncgm.grid[:, 1]
    z = ncgm.grid[:, 2]

    # Create basis matrix for entire grid
    dPhi = complete_polynomial(ncgm.grid, vp.d, Derivative{1}())

    # Compute consumption
    c = duinv(ncgm, (dPhi*vp.v_coeffs) ./ (1-ncgm.δ.+df(ncgm, k, z)))

    return expendables_t(ncgm, k, z) .- c
end
```

    env_condition_kp (generic function with 2 methods)

The update method is then very similar to other value iteration style methods, but avoids the numerical solver.

* For each point, $(k_t, z_t)$ get $c^*(k_t, z_t)$ from the envelope condition
* Define $k^*(k_t, z_t) = (1 - \delta) k_t + z_t f(k_t) - c^*(k_t, z_t)$
* Update value function according to $\hat{V}(k_t, z_t) = u(c^*(k_t, z_t)) + \beta E \left[ V(k^*(k_t, z_t), z_{t+1}) \right]$

```julia
function update!(V::Vector{Float64}, kpol::Vector{Float64},
                 ncgm::NeoclassicalGrowth, vp::ValueCoeffs{VFI_ECM},
                 Φ::Matrix{Float64}, dΦ::Matrix{Float64})
    # Get sizes and allocate for complete_polynomial
    kgrid = ncgm.kgrid; zgrid = ncgm.zgrid;
    nk, nz = length(kgrid), length(zgrid)

    # Iterate over all states
    temp = Array{Float64}(undef, n_complete(2, vp.d))
    _inds = LinearIndices((nk, nz))
    for ik in 1:nk, iz in 1:nz
        ikiz_index = _inds[ik, iz]
        k = kgrid[ik]
        z = zgrid[iz]

        # Policy from envelope condition
        kp = env_condition_kp!(temp, ncgm, vp, k, z)
        c = expendables_t(ncgm, k, z) - kp
        kpol[ikiz_index] = kp

        # New value
        EV = compute_EV!(temp, ncgm, vp, kp, iz)
        V[ikiz_index] = u(ncgm, c) + ncgm.β*EV
    end

    # Update coefficients
    update_v!(vp, Φ \ V, 1.0)
    update_k!(vp, Φ \ kpol, 1.0)

    return V
end

```

    update! (generic function with 4 methods)

### Conventional Policy Function Iteration

Policy function iteration is different than value function iteration in that it starts with a policy function, then updates the value function, and finally finds the new optimal policy function. Given a policy $c(k_t, z_t)$ and for each pair $(k_t, z_t)$

* Define $k(k_t, z_t) = (1 - \delta) k_t + z_t f(k_t) - c(k_t, z_t)$
* Find fixed point of $V(k_t, z_t) = u(c(k_t, z_t)) + \beta E \left[ V(k(k_t, z_t), z_t) \right]$ (Iterate to convergence given policy)
* Given $V(k_t, z_t)$, numerically solve for new policy $c^*(k_t, z_t)$ -- Stop when $c(k_t, z_t) \approx c^*(k_t, z_t)$

```julia
function update!(V::Vector{Float64}, kpol::Vector{Float64},
                 ncgm::NeoclassicalGrowth, vp::ValueCoeffs{PFI},
                 Φ::Matrix{Float64}, dΦ::Matrix{Float64})
    # Get sizes and allocate for complete_polynomial
    kgrid = ncgm.kgrid; zgrid = ncgm.zgrid; grid = ncgm.grid;
    nk, nz = length(kgrid), length(zgrid)

    # Copy valuecoeffs object and use to iterate to
    # convergence given a policy
    vp_igp = copy(vp, IterateOnPolicy())
    solve(ncgm, vp_igp; nskipprint=1000, maxiter=5000, verbose=false)

    # Update the policy and values
    temp = Array{Float64}(undef, n_complete(2, vp.d))
    _inds = LinearIndices((nk, nz))
    for ik in 1:nk, iz in 1:nz
        k = kgrid[ik]; z = zgrid[iz];

        # Define an objective function (negative for minimization)
        y = expendables_t(ncgm, k, z)
        solme(kp) = du(ncgm, y - kp) - ncgm.β*compute_dEV!(temp, ncgm, vp, kp, iz)

        # Find minimum of objective
        kp = brent(solme, 1e-8, y-1e-8; rtol=1e-12)

        # Update policy function
        ikiz_index = _inds[ik, iz]
        kpol[ikiz_index] = kp
    end

    # Get new coeffs
    update_k!(vp, Φ \ kpol, 1.0)
    update_v!(vp, vp_igp.v_coeffs, 1.0)

    # Update all elements of value
    copy!(V, Φ*vp.v_coeffs)

    return V
end

```

    update! (generic function with 5 methods)

### Envelope Condition Policy Function Iteration

Similar to policy function iteration, but, rather than numerically solve for new policies, it uses the envelope condition to directly compute them. Given a starting policy $c(k_t, z_t)$ and for each pair $(k_t, z_t)$

* Define $k(k_t, z_t) = (1 - \delta) k_t + z_t f(k_t) - c(k_t, z_t)$
* Find fixed point of $V(k_t, z_t) = u(c(k_t, z_t)) + \beta E \left[ V(k(k_t, z_t), z_t) \right]$ (Iterate to convergence given policy)
* Given $V(k_t, z_t)$ find $c^*(k_t, z_t)$ using envelope condition -- Stop when $c(k_t, z_t) \approx c^*(k_t, z_t)$

```julia
function update!(V::Vector{Float64}, kpol::Vector{Float64},
                 ncgm::NeoclassicalGrowth, vp::ValueCoeffs{PFI_ECM},
                 Φ::Matrix{Float64}, dΦ::Matrix{Float64})
    # Copy valuecoeffs object and use to iterate to
    # convergence given a policy
    vp_igp = copy(vp, IterateOnPolicy())
    solve(ncgm, vp_igp; nskipprint=1000, maxiter=5000, verbose=false)

    # Update the policy and values
    kp = env_condition_kp(ncgm, vp)
    update_k!(vp, Φ \ kp, 1.0)
    update_v!(vp, vp_igp.v_coeffs, 1.0)

    # Update all elements of value
    copy!(V, Φ*vp.v_coeffs)
    copy!(kpol, kp)

    return V
end

```

    update! (generic function with 6 methods)

### Euler Equation Method

Euler equation methods operate directly on the Euler equation: $u'(c_t) = \beta E \left[ u'(c_{t+1}) (1 - \delta + z_t f'(k_t)) \right]$.

Given an initial policy $c(k_t, z_t)$ for each grid point $(k_t, z_t)$

* Find $k(k_t, z_t) = (1-\delta)k_t + z_t f(k_t) - c(k_t, z_t)$
* Let $c_{t+1} = c(k(k_t, z_t), z_t)$
* Numerically solve for a $c^*$ that satisfies the Euler equation i.e. $u'(c^*) = \beta E \left[ u'(c_{t+1}) (1 - \delta + z_t f'(k_t)) \right]$
* Stop when $c^*(k_t, z_t) \approx c(k_t, z_t)$

```julia
# Conventional Euler equation method
function update!(V::Vector{Float64}, kpol::Vector{Float64},
                 ncgm::NeoclassicalGrowth, vp::ValueCoeffs{EulEq},
                 Φ::Matrix{Float64}, dΦ::Matrix{Float64})
    # Get sizes and allocate for complete_polynomial
    @unpack kgrid, zgrid, weights, z1 = ncgm
    nz1, nz = size(z1)
    nk = length(kgrid)

    # Iterate over all states
    temp = Array{Float64}(undef, n_complete(2, vp.d))
    _inds = LinearIndices((nk, nz))
    for iz in 1:nz, ik in 1:nk
        k = kgrid[ik]; z = zgrid[iz];

        # Create current polynomial
        complete_polynomial!(temp, [k, z], vp.d)

        # Compute what capital will be tomorrow according to policy
        kp = dot(temp, vp.k_coeffs)

        # Compute RHS of EE
        rhs_ee = 0.0
        for iz1 in 1:nz1
            # Possible z in t+1
            zp = z1[iz1, iz]

            # Policy for k_{t+2}
            complete_polynomial!(temp, [kp, zp], vp.d)
            kpp = dot(temp, vp.k_coeffs)

            # Implied t+1 consumption
            cp = expendables_t(ncgm, kp, zp) - kpp

            # Add to running expectation
            rhs_ee += ncgm.β*weights[iz1]*du(ncgm, cp)*(1-ncgm.δ+df(ncgm, kp, zp))
        end

        # The rhs of EE implies consumption and investment in t
        c = duinv(ncgm, rhs_ee)
        kp_star = expendables_t(ncgm, k, z) - c

        # New value
        ikiz_index = _inds[ik, iz]
        EV = compute_EV!(temp, ncgm, vp, kp_star, iz)
        V[ikiz_index] = u(ncgm, c) + ncgm.β*EV
        kpol[ikiz_index] = kp_star
    end

    # Update coefficients
    update_v!(vp, Φ \ V, 1.0)
    update_k!(vp, Φ \ kpol, 1.0)

    return V
end
```

    update! (generic function with 7 methods)

### Envelope Condition Derivative Value Function Iteration

This method uses the same insight of the "Envelope Condition Value Function Iteration," but, rather than iterate directly on the value function, it iterates on the derivative of the value function. The update steps are

* For each point, $(k_t, z_t)$ get $c^*(k_t, z_t)$ from the envelope condition (which only depends on the derivative of the value function!)
* Define $k^*(k_t, z_t) = (1 - \delta) k_t + z_t f(k_t) - c^*(k_t, z_t)$
* Update value function according to $\hat{V}_1(k_t, z_t) = \beta (1 - \delta + z_t f'(k_t)) E \left[ V_1(k^*(k_t, z_t), z_{t+1}) \right]$

Once it has converged, you use the implied policy rule and iterate to convergence using the "iterate to convergence (given policy)" method.

```julia
function update!(dV::Vector{Float64}, kpol::Vector{Float64},
                 ncgm::NeoclassicalGrowth, vp::ValueCoeffs{dVFI_ECM},
                 Φ::Matrix{Float64}, dΦ::Matrix{Float64})
    # Get sizes and allocate for complete_polynomial
    kgrid = ncgm.kgrid; zgrid = ncgm.zgrid; grid = ncgm.grid;
    nk, nz, ns = length(kgrid), length(zgrid), size(grid, 1)

    # Iterate over all states
    temp = Array{Float64}(undef, n_complete(2, vp.d))
    _inds = LinearIndices((nk, nz))
    for iz=1:nz, ik=1:nk
        k = kgrid[ik]; z = zgrid[iz];

        # Envelope condition implies optimal kp
        kp = env_condition_kp!(temp, ncgm, vp, k, z)
        c = expendables_t(ncgm, k, z) - kp

        # New value
        ikiz_index = _inds[ik, iz]
        dEV = compute_dEV!(temp, ncgm, vp, kp, iz)
        dV[ikiz_index] = (1-ncgm.δ+df(ncgm, k, z))*ncgm.β*dEV
        kpol[ikiz_index] = kp
    end

    # Get new coeffs
    update_k!(vp, Φ \ kpol, 1.0)
    update_v!(vp, dΦ \ dV, 1.0)

    return dV
end

```

    update! (generic function with 8 methods)

### Simulation and Euler Error Methods

The following functions to simulate and compute Euler errors are easily defined given our model type and the solution type.

```julia
"""
Simulates the neoclassical growth model for a given set of solution
coefficients. It simulates for `capT` periods and discards first
`nburn` observations.
"""
function simulate(ncgm::NeoclassicalGrowth, vp::ValueCoeffs,
                  shocks::Vector{Float64}; capT::Int=10_000,
                  nburn::Int=200)
    # Unpack parameters
    kp = 0.0  # Policy holder
    temp = Array{Float64}(undef, n_complete(2, vp.d))

    # Allocate space for k and z
    ksim = Array{Float64}(undef, capT+nburn)
    zsim = Array{Float64}(undef, capT+nburn)

    # Initialize both k and z at 1
    ksim[1] = 1.0
    zsim[1] = 1.0

    # Simulate
    temp = Array{Float64}(undef, n_complete(2, vp.d))
    for t in 2:capT+nburn
        # Evaluate k_t given yesterday's (k_{t-1}, z_{t-1})
        kp = env_condition_kp!(temp, ncgm, vp, ksim[t-1], zsim[t-1])

        # Draw new z and update k using policy above
        zsim[t] = zsim[t-1]^ncgm.ρ * exp(ncgm.σ*shocks[t])
        ksim[t] = kp
    end

    return ksim[nburn+1:end], zsim[nburn+1:end]
end

function simulate(ncgm::NeoclassicalGrowth, vp::ValueCoeffs;
                  capT::Int=10_000, nburn::Int=200, seed=42)
    Random.seed!(seed)  # Set specific seed
    shocks = randn(capT + nburn)

    return simulate(ncgm, vp, shocks; capT=capT, nburn=nburn)
end

"""
This function evaluates the Euler Equation residual for a single point (k, z)
"""
function EulerEquation!(out::Vector{Float64}, ncgm::NeoclassicalGrowth,
                        vp::ValueCoeffs, k::Float64, z::Float64,
                        nodes::Vector{Float64}, weights::Vector{Float64})
    # Evaluate consumption today
    k1 = env_condition_kp!(out, ncgm, vp, k, z)
    c = expendables_t(ncgm, k, z) - k1
    LHS = du(ncgm, c)

    # For each of realizations tomorrow, evaluate expectation on RHS
    RHS = 0.0
    for (eps, w) in zip(nodes, weights)
        # Compute ztp1
        z1 = z^ncgm.ρ * exp(eps)

        # Evaluate the ktp2
        ktp2 = env_condition_kp!(out, ncgm, vp, k1, z1)

        # Get c1
        c1 = expendables_t(ncgm, k1, z1) - ktp2

        # Update RHS of equation
        RHS = RHS + w*du(ncgm, c1)*(1 - ncgm.δ + df(ncgm, k1, z1))
    end

    return abs(ncgm.β*RHS/LHS - 1.0)
end

"""
Given simulations for k and z, it computes the euler equation residuals
along the entire simulation. It reports the mean and max values in
log10.
"""
function ee_residuals(ncgm::NeoclassicalGrowth, vp::ValueCoeffs,
                      ksim::Vector{Float64}, zsim::Vector{Float64}; Qn::Int=10)
    # Figure out how many periods we simulated for and make sure k and z
    # are same length
    capT = length(ksim)
    @assert length(zsim) == capT

    # Finer integration nodes
    eps_nodes, weight_nodes = qnwnorm(Qn, 0.0, ncgm.σ^2)
    temp = Array{Float64}(undef, n_complete(2, vp.d))

    # Compute EE for each period
    EE_resid = Array{Float64}(undef, capT)
    for t=1:capT
        # Pull out current state
        k, z = ksim[t], zsim[t]

        # Compute residual of Euler Equation
        EE_resid[t] = EulerEquation!(temp, ncgm, vp, k, z, eps_nodes, weight_nodes)
    end

    return EE_resid
end

function ee_residuals(ncgm::NeoclassicalGrowth, vp::ValueCoeffs; Qn::Int=10)
    # Simulate and then call other ee_residuals method
    ksim, zsim = simulate(ncgm, vp)

    return ee_residuals(ncgm, vp, ksim, zsim; Qn=Qn)
end

```

    ee_residuals (generic function with 2 methods)

## A Horse Race

We can now run a horse race to compare the methods in terms of both accuracy and speed.

```julia
function main(sm::SolutionMethod, nd::Int=5, shocks=randn(capT+nburn);
              capT=10_000, nburn=200, tol=1e-9, maxiter=2500,
              nskipprint=25, verbose=true)
    # Create model
    ncgm = NeoclassicalGrowth()

    # Create initial quadratic guess
    vp = ValueCoeffs(ncgm, Val{2}, IterateOnPolicy())
    solve(ncgm, vp; tol=1e-6, verbose=false)

    # Allocate memory for timings
    times = Array{Float64}(undef, nd-1)
    sols = Array{ValueCoeffs}(undef, nd-1)
    mean_ees = Array{Float64}(undef, nd-1)
    max_ees = Array{Float64}(undef, nd-1)

    # Solve using the solution method for degree 2 to 5
    vp = copy(vp, sm)
    for d in 2:nd
        # Change degree of solution method
        vp = copy(ncgm, vp, Val{d})

        # Time the current method
        start_time = time()
        solve(ncgm, vp; tol=tol, maxiter=maxiter, nskipprint=nskipprint,
              verbose=verbose)
        end_time = time()

        # Save the time and solution
        times[d-1] = end_time - start_time
        sols[d-1] = vp

        # Simulate and compute EE
        ks, zs = simulate(ncgm, vp, shocks; capT=capT, nburn=nburn)
        resids = ee_residuals(ncgm, vp, ks, zs; Qn=10)
        mean_ees[d-1] = log10.(mean(abs.(resids)))
        max_ees[d-1] = log10.(maximum(abs, resids))
    end

    return sols, times, mean_ees, max_ees
end

```

    main (generic function with 3 methods)

```julia
Random.seed!(52)
shocks = randn(10200)

for sol_method in [VFI(), VFI_EGM(), VFI_ECM(), dVFI_ECM(),
                   PFI(), PFI_ECM(), EulEq()]
    # Make sure everything is compiled
    main(sol_method, 5, shocks; maxiter=2, verbose=false)

    # Run for real
    s_sm, t_sm, mean_eem, max_eem = main(sol_method, 5, shocks;
                                         tol=1e-8, verbose=false)

    println("Solution Method: $sol_method")
    for (d, t) in zip([2, 3, 4, 5], t_sm)
        println("\tDegree $d took time $t")
        println("\tMean & Max EE are" *
                "$(round(mean_eem[d-1], digits=3)) & $(round(max_eem[d-1], digits=3))")
    end
end
```

    Solution Method: VFI()
    	Degree 2 took time 0.8547611236572266
    	Mean & Max EE are-3.803 & -2.875
    	Degree 3 took time 0.7150120735168457
    	Mean & Max EE are-4.914 & -3.487
    	Degree 4 took time 0.8598308563232422
    	Mean & Max EE are-5.978 & -4.226
    	Degree 5 took time 0.7123589515686035
    	Mean & Max EE are-6.916 & -4.942
    Solution Method: VFI_EGM()
    	Degree 2 took time 0.3311178684234619
    	Mean & Max EE are-3.803 & -2.876
    	Degree 3 took time 0.24843502044677734
    	Mean & Max EE are-4.914 & -3.487
    	Degree 4 took time 0.30118298530578613
    	Mean & Max EE are-5.978 & -4.226
    	Degree 5 took time 0.18770384788513184
    	Mean & Max EE are-6.916 & -4.942
    Solution Method: VFI_ECM()
    	Degree 2 took time 0.2970089912414551
    	Mean & Max EE are-3.803 & -2.875
    	Degree 3 took time 0.2128901481628418
    	Mean & Max EE are-4.914 & -3.487
    	Degree 4 took time 0.25684499740600586
    	Mean & Max EE are-5.978 & -4.226
    	Degree 5 took time 0.1400759220123291
    	Mean & Max EE are-6.916 & -4.942
    Solution Method: dVFI_ECM()
    	Degree 2 took time 0.517967939376831
    	Mean & Max EE are-3.803 & -2.876
    	Degree 3 took time 0.6294620037078857
    	Mean & Max EE are-4.914 & -3.487
    	Degree 4 took time 0.8079090118408203
    	Mean & Max EE are-5.978 & -4.226
    	Degree 5 took time 0.8993039131164551
    	Mean & Max EE are-6.916 & -4.942
    Solution Method: PFI()
    	Degree 2 took time 0.30332493782043457
    	Mean & Max EE are-3.803 & -2.876
    	Degree 3 took time 0.6854491233825684
    	Mean & Max EE are-4.914 & -3.487
    	Degree 4 took time 0.9637911319732666
    	Mean & Max EE are-5.978 & -4.226
    	Degree 5 took time 0.7570579051971436
    	Mean & Max EE are-6.916 & -4.942
    Solution Method: PFI_ECM()
    	Degree 2 took time 0.24885082244873047
    	Mean & Max EE are-3.803 & -2.876
    	Degree 3 took time 0.19704389572143555
    	Mean & Max EE are-4.914 & -3.487
    	Degree 4 took time 0.23227882385253906
    	Mean & Max EE are-5.978 & -4.226
    	Degree 5 took time 0.1265571117401123
    	Mean & Max EE are-6.916 & -4.942
    Solution Method: EulEq()
    	Degree 2 took time 0.34078478813171387
    	Mean & Max EE are-3.803 & -2.876
    	Degree 3 took time 0.24483489990234375
    	Mean & Max EE are-4.914 & -3.487
    	Degree 4 took time 0.27704310417175293
    	Mean & Max EE are-5.978 & -4.226
    	Degree 5 took time 0.14841604232788086
    	Mean & Max EE are-6.916 & -4.942

```julia

```