```python
import arviz as az
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.simplefilter(action='ignore', category=UserWarning)

import pymc3 as pm
import theano

print('Running on PyMC3 v{}'.format(pm.__version__))
```

    Running on PyMC3 v3.9.3

```python
%config InlineBackend.figure_format = 'retina'
az.style.use('default')
```

# A Tour of Model Checking techniques using PyMC3

Rob Zinkov

## Outline

1.  Introduction
2.  Setting up example problem
3.  Motivate why model checking is necessary
4.  Cross validation and predictive accuracy / LOO
5.  Prior & Posterior Predictive Checks / Population Predictive Checks
6.  BIC / DIC / WAIC
7.  Likelihood on heldout data
8.  Marginal Likelihood
9.  Likelihood ratios, Bayes factors
10. MMD / relative MMD / Kernel Stein Discrepancy / Witness function
11. Posterior dispersion indices
12. Flowchart of when to use different techniques

# Introduction

One of the ongoing challenges when fitting models is figuring out
if what we have is any good.

The issue is there a good guidelines on the model fitting side,
but less so on model evaluation side

I'll go over several methods that exist to give an overview of
how to evaluate and compare different models.

I will cover both Bayesian *and* Frequentist methods. They each
have their places and evaluating models is a place where both can shine

## Model Evaluation vs Model Selection

While I consider these to be two aspects of the same process sometimes it'll help to distinguish

*Model Evaluation* asks is my model of the data any good?

(Sometimes is called Model checking or Model criticism)

*Model Selection* asks which of these models is the best fit for my data

(Sometimes is called Model comparison)

# Assumptions and Cavaets

* Assuming you have run and fit a few models in pymc3
* I won't nor can cover all goodness of fit tests
* Not all tests are or should be natively supported by pymc3

# Running example

Radon is an odorless colorless gas that in high concentrations is known to cause
lung cancer. Levels vary from household to household but also geographically.

The EPA conducted a study and found out that the presence of a basement as well
as general levels of uranium in the level are predictive of radon levels.

## Running example

Gelman et al. (2013) use a hierarchical Bayesian model to fit and predict
radon levels. These and other models have been ported to pymc3 and the radon
data comes included with the library.

So we'll be using these models as a running example of picking between different models

https://docs.pymc.io/notebooks/multilevel_modeling.html
by Chris Fonnesbeck covers the models themselves in more detail

```python

```

```python
# Read data
srrs2 = pd.read_csv(pm.get_data("srrs2.dat"))
srrs2.columns = srrs2.columns.map(str.strip)
srrs_mn = srrs2[srrs2.state == "MN"].copy()

# Add county information
srrs_mn.county = srrs_mn.county.map(str.strip)
mn_counties = srrs_mn.county.unique()
counties = len(mn_counties)
county_lookup = dict(zip(mn_counties, range(counties)))

county = srrs_mn["county_code"] = srrs_mn.county.replace(county_lookup).values
radon = srrs_mn["radon"] = srrs_mn.activity
srrs_mn["log_radon"] = log_radon = np.log(radon + 0.1).values
floor = srrs_mn.floor.values
```

## Radon prediction

The way this works is we will try to predict radon levels based on:

1. What county the house is located in
2. Whether the measurement was taken in the Basement (floor 0) or Ground Level (floor 1)

```python
srrs_mn[['floor', 'county', 'radon']].head(n=10).style.hide_index()
```

        floor	        county	        radon

                                1
                        AITKIN
                        2.200000

                                0
                        AITKIN
                        2.200000

                                0
                        AITKIN
                        2.900000

                                0
                        AITKIN
                        1.000000

                                0
                        ANOKA
                        3.100000

                                0
                        ANOKA
                        2.500000

                                0
                        ANOKA
                        1.500000

                                0
                        ANOKA
                        1.000000

                                0
                        ANOKA
                        0.700000

                                0
                        ANOKA
                        1.200000

```python
srrs_mn.groupby(['county']).size().sort_values().reset_index(name='counts')
```

      county
      counts

      0
      WILKIN
      1

      1
      MAHNOMEN
      1

      2
      MURRAY
      1

      3
      YELLOW MEDICINE
      2

      4
      LAC QUI PARLE
      2

      ...
      ...
      ...

      80
      WASHINGTON
      46

      81
      ANOKA
      52

      82
      DAKOTA
      63

      83
      HENNEPIN
      105

      84
      ST LOUIS
      116

85 rows × 2 columns

## Radon prediction

```python
srrs_mn.log_radon.hist(bins=25, rwidth=0.9);
```

## Radon prediction models

**Complete pooling:**

Treat all counties the same, and estimate a single radon level.

$$ y_i = \alpha +\beta x_i + \epsilon_i $$

**No pooling:**

Model radon in each county independently.

$$ y_i = \alpha_{j[i]} +\beta x_i + \epsilon_i $$

where $j=1,\ldots,85$

## Radon prediction models (continued)

**Partial pooling**

$$ y_i = \alpha_{j[i]} +\beta x_i + \epsilon_i $$

$$ \alpha_{j[i]} \sim \mathcal{N}(\mu_\alpha, \sigma^2_\alpha) $$

where $j=1,\ldots,85$

# Predictive accuracy

The best way to evaluate models is on some downstream task

## Predictive accuracy

One way to evaluate this is to set aside some data and see
if our model can predict the true value from this unseen data.

```python

```

```python
def split_data(df, fraction=0.2):
    out_sample = df.sample(frac = fraction)
    in_sample = df.drop(out_sample.index)
    return in_sample, out_sample
```

```python
# Set aside some of the data
srrs_in_sample, srrs_out_sample = split_data(srrs_mn, fraction=0.1)

county_in = srrs_in_sample.county_code.values
radon_in = srrs_in_sample.log_radon.values
floor_in = srrs_in_sample.floor.values

county_out = srrs_out_sample.county_code.values
radon_out = srrs_out_sample.log_radon.values
floor_out = srrs_out_sample.floor.values
```

```python

```

```python
coords = {
    "Level": ["Basement", "Floor"],
    "obs_id": np.arange(floor_in.size)
}
with pm.Model(coords=coords) as pooled_model:
    floor_idx = pm.Data("floor_idx", floor_in, dims="obs_id")
    b = pm.Normal("b", 0.0, sigma=10.0, dims="Level")

    theta = b[floor_idx]
    sigma = pm.Exponential("sigma", 1.0)

    radon_shared = pm.Data("radon", radon_in, dims="obs_id")
    y = pm.Normal("y", theta, sigma=sigma, observed=radon_shared, dims="obs_id")
    pooled_trace = pm.sample(random_seed=42)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [sigma, b]

  <progress value='4000' class='' max='4000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [4000/4000 00:02<00:00 Sampling 2 chains, 0 divergences]

    Sampling 2 chains for 1_000 tune and 1_000 draw iterations (2_000 + 2_000 draws total) took 3 seconds.

```python
az.summary(pooled_trace)
```

      mean
      sd
      hdi_3%
      hdi_97%
      mcse_mean
      mcse_sd
      ess_mean
      ess_sd
      ess_bulk
      ess_tail
      r_hat

      b[0]
      1.351
      0.030
      1.295
      1.406
      0.001
      0.000
      2366.0
      2366.0
      2358.0
      1301.0
      1.0

      b[1]
      0.776
      0.065
      0.647
      0.890
      0.001
      0.001
      2753.0
      2753.0
      2757.0
      1248.0
      1.0

      sigma
      0.768
      0.020
      0.731
      0.805
      0.000
      0.000
      2511.0
      2511.0
      2502.0
      1430.0
      1.0

```python
coords = {
    "Level": ["Basement", "Floor"],
    "obs_id": np.arange(floor_in.size),
    "County": mn_counties
}
with pm.Model(coords=coords) as unpooled_model:
    floor_idx = pm.Data("floor_idx", floor_in, dims="obs_id")
    county_idx = pm.Data("county_idx", county_in, dims="obs_id")
    a = pm.Normal("a", 0.0, sigma=10.0, dims=("County", "Level"))

    theta = a[county_idx, floor_idx]
    sigma = pm.Exponential("sigma", 1.0)

    radon_shared = pm.Data("radon", radon_in, dims="obs_id")
    y = pm.Normal("y", theta, sigma=sigma, observed=radon_shared, dims="obs_id")
    unpooled_trace = pm.sample(random_seed=42)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [sigma, a]

  <progress value='4000' class='' max='4000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [4000/4000 00:09<00:00 Sampling 2 chains, 0 divergences]

    Sampling 2 chains for 1_000 tune and 1_000 draw iterations (2_000 + 2_000 draws total) took 9 seconds.

```python
az.summary(unpooled_trace)
```

      mean
      sd
      hdi_3%
      hdi_97%
      mcse_mean
      mcse_sd
      ess_mean
      ess_sd
      ess_bulk
      ess_tail
      r_hat

      a[0,0]
      0.667
      0.397
      -0.045
      1.456
      0.006
      0.006
      4175.0
      2421.0
      4194.0
      1313.0
      1.00

      a[0,1]
      0.814
      0.687
      -0.390
      2.177
      0.012
      0.013
      3122.0
      1316.0
      3132.0
      1267.0
      1.00

      a[1,0]
      0.964
      0.102
      0.769
      1.148
      0.002
      0.001
      2488.0
      2469.0
      2487.0
      1529.0
      1.00

      a[1,1]
      -0.130
      0.392
      -0.839
      0.605
      0.007
      0.009
      2953.0
      1044.0
      2988.0
      1490.0
      1.00

      a[2,0]
      1.501
      0.696
      0.214
      2.860
      0.012
      0.010
      3446.0
      2648.0
      3452.0
      1316.0
      1.00

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      a[83,0]
      1.673
      0.198
      1.307
      2.026
      0.004
      0.003
      3134.0
      3120.0
      3126.0
      1368.0
      1.01

      a[83,1]
      -0.263
      9.747
      -18.246
      17.840
      0.179
      0.249
      2970.0
      765.0
      2959.0
      1447.0
      1.00

      a[84,0]
      1.202
      0.480
      0.328
      2.138
      0.008
      0.007
      3192.0
      2722.0
      3214.0
      1468.0
      1.00

      a[84,1]
      0.085
      10.109
      -19.444
      19.265
      0.168
      0.275
      3613.0
      677.0
      3606.0
      1146.0
      1.00

      sigma
      0.693
      0.019
      0.657
      0.727
      0.000
      0.000
      2293.0
      2293.0
      2251.0
      1394.0
      1.00

171 rows × 11 columns

```python
coords = {
    "Level": ["Basement", "Floor"],
    "obs_id": np.arange(floor_in.size),
    "County": mn_counties
}
with pm.Model(coords=coords) as partial_pooling:
    floor_idx = pm.Data("floor_idx", floor_in, dims="obs_id")
    county_idx = pm.Data("county_idx", county_in, dims="obs_id")
    # Hyperpriors:
    a = pm.Normal("a", mu=0.0, sigma=10.0)
    sigma_a = pm.Exponential("sigma_a", 1.0)

    # Varying intercepts:
    a_county = pm.Normal("a_county", mu=a, sigma=sigma_a, dims="County")
    # Common slope:
    b = pm.Normal("b", mu=0.0, sigma=10.0)

    # Expected value per county:
    theta = a_county[county_idx] + b * floor_idx
    # Model error:
    sigma = pm.Exponential("sigma", 1.0)

    radon_shared = pm.Data("radon", radon_in, dims="obs_id")
    y = pm.Normal("y", theta, sigma=sigma, observed=radon_shared, dims="obs_id")
    partial_pooling_trace = pm.sample(random_seed=42)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [sigma, b, a_county, sigma_a, a]

  <progress value='4000' class='' max='4000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [4000/4000 00:06<00:00 Sampling 2 chains, 0 divergences]

    Sampling 2 chains for 1_000 tune and 1_000 draw iterations (2_000 + 2_000 draws total) took 7 seconds.

```python
# Hack since pm.find_MAP diverges

best_ll = -np.infty
best = None

pp_logp = partial_pooling.logp
for p in partial_pooling_trace:
    ll = pp_logp(p)
    if ll > best_ll:
        best = p
        best_ll = ll

partial_pooled_point_estimate = best
```

## Predictive accuracy

For continuous data root mean squared error (rmse) is a popular choice

$$ \text{RMSE}(Y) = \sqrt{\frac{1}{n}\sum_{i=1}^n(Y_i-\mathbb{E}[Y_i \mid \theta])^2}$$

```python
def rmse(test_data, predicted):
    """Calculate root mean squared error.
    Ignoring missing values in the test data.
    """
    I = ~np.isnan(test_data)   # indicator for missing values
    N = I.sum()                # number of non-missing values
    sqerror = abs(test_data - predicted) ** 2  # squared error array
    mse = sqerror[I].sum() / N                 # mean squared error
    return np.sqrt(mse)
```

```python
with pooled_model:
    pooled_point_estimate = pm.find_MAP(model=pooled_model)
    # change the value and shape of the data
    pm.set_data({
        "floor_idx": floor_out,
        # use dummy values with the same shape:
        "radon": radon_out,
    })

    post_pred = pm.sample_posterior_predictive(pooled_trace)
```

  <progress value='14' class='' max='14' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [14/14 00:00<00:00 logp = -960.74, ||grad|| = 8.0476]

  <progress value='2000' class='' max='2000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [2000/2000 00:20<00:00]

```python
preds = post_pred["y"].mean(axis=0)
# lower is better
rmse(preds, radon_out)
```

    0.9721695872119326

```python
with unpooled_model:
    unpooled_point_estimate = pm.find_MAP(model=unpooled_model)
    # change the value and shape of the data
    pm.set_data({
        "floor_idx": floor_out,
        "county_idx": county_out,
        # use dummy values with the same shape:
        "radon": radon_out,
    })

    post_pred = pm.sample_posterior_predictive(unpooled_trace)
```

  <progress value='75' class='' max='75' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [75/75 00:00<00:00 logp = -1,343.4, ||grad|| = 0.041008]

  <progress value='2000' class='' max='2000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [2000/2000 00:20<00:00]

```python
preds = post_pred["y"].mean(axis=0)
# lower is better
rmse(preds, radon_out)
```

    1.0243686247273727

```python
with partial_pooling:
    # change the value and shape of the data
    pm.set_data({
        "floor_idx": floor_out,
        "county_idx": county_out,
        # use dummy values with the same shape:
        "radon": radon_out,
    })

    post_pred = pm.sample_posterior_predictive(partial_pooling_trace)
```

  <progress value='2000' class='' max='2000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [2000/2000 00:19<00:00]

```python
preds = post_pred["y"].mean(axis=0)
# lower is better
rmse(preds, radon_out)
```

    0.9299338549155656

```python

```

# Cross validation

## Cross validation

Predictive accuracy is based on a single split which introduces bias. We can do multiple
splits and average the accuracy across these splits. This is called *cross-validation*

Exercise 1:

Implement cross-validation and report results on radon models

# Likelihood on held-out data

## Held-out likelihood

In the same spirit of predictive accuracy, we can simply ask on unseen data, which of the models is least “surprised”

```python
pooled_point_estimate["y"] = radon_out
unpooled_point_estimate["y"] = radon_out
partial_pooled_point_estimate["y"] = radon_out

print("Pooled model LL:", pooled_model.logp(pooled_point_estimate))
print("Unpooled model LL:", unpooled_model.logp(unpooled_point_estimate))
print("Partially-pooled model LL:", partial_pooling.logp(partial_pooled_point_estimate))
```

    Pooled model LL: -141.3818492829713
    Unpooled model LL: -710.3533170768483
    Partially-pooled model LL: -128.2446075503398

# Marginal Likelihood

Marginal likelihood measures the probability of observing our data under the model family. This is also more Bayesian as we don't use a point estimate of our parameters

```python
coords = {
    "Level": ["Basement", "Floor"],
    "obs_id": np.arange(floor.size)
}
with pm.Model(coords=coords) as pooled_model:
    floor_idx = pm.Data("floor_idx", floor, dims="obs_id")
    b = pm.Normal("b", 0.0, sigma=10.0, dims="Level")

    theta = b[floor_idx]
    sigma = pm.Exponential("sigma", 1.0)

    radon_shared = pm.Data("radon", log_radon, dims="obs_id")
    y = pm.Normal("y", theta, sigma=sigma, observed=radon_shared, dims="obs_id")
    pooled_trace = pm.sample(random_seed=42)

coords = {
    "Level": ["Basement", "Floor"],
    "obs_id": np.arange(floor.size),
    "County": mn_counties
}
with pm.Model(coords=coords) as unpooled_model:
    floor_idx = pm.Data("floor_idx", floor, dims="obs_id")
    county_idx = pm.Data("county_idx", county, dims="obs_id")
    a = pm.Normal("a", 0.0, sigma=10.0, dims=("County", "Level"))

    theta = a[county_idx, floor_idx]
    sigma = pm.Exponential("sigma", 1.0)

    radon_shared = pm.Data("radon", log_radon, dims="obs_id")
    y = pm.Normal("y", theta, sigma=sigma, observed=radon_shared, dims="obs_id")
    unpooled_trace = pm.sample(random_seed=42)

coords = {
    "Level": ["Basement", "Floor"],
    "obs_id": np.arange(floor.size),
    "County": mn_counties
}
with pm.Model(coords=coords) as partial_pooling:
    floor_idx = pm.Data("floor_idx", floor, dims="obs_id")
    county_idx = pm.Data("county_idx", county, dims="obs_id")
    # Hyperpriors:
    a = pm.Normal("a", mu=0.0, sigma=10.0)
    sigma_a = pm.Exponential("sigma_a", 1.0)

    # Varying intercepts:
    a_county = pm.Normal("a_county", mu=a, sigma=sigma_a, dims="County")
    # Common slope:
    b = pm.Normal("b", mu=0.0, sigma=10.0)

    # Expected value per county:
    theta = a_county[county_idx] + b * floor_idx
    # Model error:
    sigma = pm.Exponential("sigma", 1.0)

    radon_shared = pm.Data("radon", log_radon, dims="obs_id")
    y = pm.Normal("y", theta, sigma=sigma, observed=radon_shared, dims="obs_id")
    partial_pooling_trace = pm.sample(random_seed=42)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [sigma, b]

  <progress value='4000' class='' max='4000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [4000/4000 00:02<00:00 Sampling 2 chains, 0 divergences]

    Sampling 2 chains for 1_000 tune and 1_000 draw iterations (2_000 + 2_000 draws total) took 2 seconds.
    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [sigma, a]

  <progress value='4000' class='' max='4000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [4000/4000 00:12<00:00 Sampling 2 chains, 0 divergences]

    Sampling 2 chains for 1_000 tune and 1_000 draw iterations (2_000 + 2_000 draws total) took 12 seconds.
    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [sigma, b, a_county, sigma_a, a]

  <progress value='4000' class='' max='4000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [4000/4000 00:08<00:00 Sampling 2 chains, 0 divergences]

    Sampling 2 chains for 1_000 tune and 1_000 draw iterations (2_000 + 2_000 draws total) took 8 seconds.

```python
with pooled_model:
    pooled_smc_trace = pm.sample_smc(4000)
```

    Initializing SMC sampler...
    Sampling 2 chains in 2 jobs
    Stage:   0 Beta: 0.000
    Stage:   1 Beta: 0.000
    Stage:   2 Beta: 0.001
    Stage:   3 Beta: 0.003
    Stage:   4 Beta: 0.007
    Stage:   5 Beta: 0.015
    Stage:   6 Beta: 0.037
    Stage:   7 Beta: 0.092
    Stage:   8 Beta: 0.238
    Stage:   9 Beta: 0.608
    Stage:  10 Beta: 1.000
    Stage:   0 Beta: 0.000
    Stage:   1 Beta: 0.000
    Stage:   2 Beta: 0.001
    Stage:   3 Beta: 0.003
    Stage:   4 Beta: 0.007
    Stage:   5 Beta: 0.015
    Stage:   6 Beta: 0.036
    Stage:   7 Beta: 0.090
    Stage:   8 Beta: 0.230
    Stage:   9 Beta: 0.588
    Stage:  10 Beta: 1.000

```python
with unpooled_model:
    unpooled_smc_trace = pm.sample_smc(4000)
```

    Initializing SMC sampler...
    Sampling 2 chains in 2 jobs
    Stage:   0 Beta: 0.000
    Stage:   1 Beta: 0.000
    Stage:   2 Beta: 0.000
    Stage:   3 Beta: 0.001
    Stage:   4 Beta: 0.003
    Stage:   5 Beta: 0.006
    Stage:   6 Beta: 0.011
    Stage:   7 Beta: 0.019
    Stage:   8 Beta: 0.028
    Stage:   9 Beta: 0.039
    Stage:  10 Beta: 0.050
    Stage:  11 Beta: 0.063
    Stage:  12 Beta: 0.076
    Stage:  13 Beta: 0.089
    Stage:  14 Beta: 0.104
    Stage:  15 Beta: 0.120
    Stage:  16 Beta: 0.137
    Stage:  17 Beta: 0.154
    Stage:  18 Beta: 0.173
    Stage:  19 Beta: 0.195
    Stage:  20 Beta: 0.216
    Stage:  21 Beta: 0.239
    Stage:  22 Beta: 0.264
    Stage:  23 Beta: 0.292
    Stage:  24 Beta: 0.323
    Stage:  25 Beta: 0.354
    Stage:  26 Beta: 0.390
    Stage:  27 Beta: 0.427
    Stage:  28 Beta: 0.468
    Stage:  29 Beta: 0.513
    Stage:  30 Beta: 0.561
    Stage:  31 Beta: 0.614
    Stage:  32 Beta: 0.673
    Stage:  33 Beta: 0.737
    Stage:  34 Beta: 0.807
    Stage:  35 Beta: 0.882
    Stage:  36 Beta: 0.964
    Stage:  37 Beta: 1.000
    Stage:   0 Beta: 0.000
    Stage:   1 Beta: 0.000
    Stage:   2 Beta: 0.000
    Stage:   3 Beta: 0.001
    Stage:   4 Beta: 0.003
    Stage:   5 Beta: 0.006
    Stage:   6 Beta: 0.011
    Stage:   7 Beta: 0.018
    Stage:   8 Beta: 0.027
    Stage:   9 Beta: 0.038
    Stage:  10 Beta: 0.050
    Stage:  11 Beta: 0.062
    Stage:  12 Beta: 0.075
    Stage:  13 Beta: 0.090
    Stage:  14 Beta: 0.104
    Stage:  15 Beta: 0.119
    Stage:  16 Beta: 0.136
    Stage:  17 Beta: 0.154
    Stage:  18 Beta: 0.173
    Stage:  19 Beta: 0.193
    Stage:  20 Beta: 0.216
    Stage:  21 Beta: 0.239
    Stage:  22 Beta: 0.264
    Stage:  23 Beta: 0.292
    Stage:  24 Beta: 0.322
    Stage:  25 Beta: 0.353
    Stage:  26 Beta: 0.387
    Stage:  27 Beta: 0.424
    Stage:  28 Beta: 0.464
    Stage:  29 Beta: 0.510
    Stage:  30 Beta: 0.557
    Stage:  31 Beta: 0.611
    Stage:  32 Beta: 0.668
    Stage:  33 Beta: 0.731
    Stage:  34 Beta: 0.803
    Stage:  35 Beta: 0.881
    Stage:  36 Beta: 0.965
    Stage:  37 Beta: 1.000

```python
with partial_pooling:
    partial_smc_trace = pm.sample_smc(4000)
```

    Initializing SMC sampler...
    Sampling 2 chains in 2 jobs
    Stage:   0 Beta: 0.000
    Stage:   1 Beta: 0.000
    Stage:   2 Beta: 0.001
    Stage:   3 Beta: 0.003
    Stage:   4 Beta: 0.006
    Stage:   5 Beta: 0.011
    Stage:   6 Beta: 0.029
    Stage:   7 Beta: 0.054
    Stage:   8 Beta: 0.089
    Stage:   9 Beta: 0.139
    Stage:  10 Beta: 0.201
    Stage:  11 Beta: 0.276
    Stage:  12 Beta: 0.363
    Stage:  13 Beta: 0.461
    Stage:  14 Beta: 0.572
    Stage:  15 Beta: 0.696
    Stage:  16 Beta: 0.830
    Stage:  17 Beta: 0.969
    Stage:  18 Beta: 1.000
    Stage:   0 Beta: 0.000
    Stage:   1 Beta: 0.000
    Stage:   2 Beta: 0.001
    Stage:   3 Beta: 0.003
    Stage:   4 Beta: 0.005
    Stage:   5 Beta: 0.011
    Stage:   6 Beta: 0.019
    Stage:   7 Beta: 0.034
    Stage:   8 Beta: 0.063
    Stage:   9 Beta: 0.111
    Stage:  10 Beta: 0.171
    Stage:  11 Beta: 0.243
    Stage:  12 Beta: 0.326
    Stage:  13 Beta: 0.419
    Stage:  14 Beta: 0.522
    Stage:  15 Beta: 0.633
    Stage:  16 Beta: 0.751
    Stage:  17 Beta: 0.875
    Stage:  18 Beta: 1.000

```python
print("pooled marginal_likelihood:", pooled_smc_trace.report.log_marginal_likelihood)
print("unpooled marginal likelihood:", unpooled_smc_trace.report.log_marginal_likelihood)
print("partial_pooling marginal_likelihood:", partial_smc_trace.report.log_marginal_likelihood)
```

    pooled marginal_likelihood: [-1101.22611358 -1101.18021879]
    unpooled marginal likelihood: [-2283.75780614 -2288.24476468]
    partial_pooling marginal_likelihood: [-1075.34181204 -1070.76515456]

## Bayes factors

Marginal likelihoods are nice for evaluating models but they can be used to compare models

## Bayes factors

The intuition is suppose we have two models $p(M_0 \mid y)$ and $p(M_1 \mid y)$ and we want to choose between them

## Bayes factors

Thanks to some algebra we don't need to compute the normalising constants and just need
the likelihoods and the priors for the models.

$$
\begin{align}
BF &= \frac{p(y|M_0)}{p(y|M_1)} \\
&= \frac{\int p(\theta_0|M_0)p(D|\theta_0,M_0)\,d\theta_0}
{\int p(\theta_1|M_1)p(y|\theta_1,M_1)\,d\theta_1} \\
&= \frac{\frac{p(M_0|y)p(y)}{p(M_0)}}{\frac{p(M_1|y)p(y)}{p(M_1)}} \\
&= \frac{p(M_0|y)}{p(M_1|y)}\frac{p(M_1)}{p(M_0)}
\end{align}
$$

We assume each model is equally likely

## Bayes factors

This ratio has some informal intuition over how much you said favour $M_0$ over $M_1$

* 1-3: anecdotal
* 3-10: moderate
* 10-30: strong
* 30-100: very strong
* 100+: extreme

## Bayes factors drawbacks

1. Must be a pair of models
2. Results are sensitive to latent variables
3. Made for discrete set of models

```python
BF_smc = np.exp(partial_smc_trace.report.log_marginal_likelihood -
                pooled_smc_trace.report.log_marginal_likelihood)
print(round(BF_smc[0]))
```

    174344931409

# Prior & Posterior Predictive Checks

## Prior Predictive Checks

Prior Predictive Checks are there to check if your priors are reasonable by comparing them to your data

```python

```

```python
with pooled_model:
    prior_check = pm.sample_prior_predictive(samples=50, random_seed=42)
```

```python
idata = az.from_pymc3(pooled_trace, prior=prior_check)
az.plot_ppc(idata, group='prior');
```

Exercise 2:

Adjust radon models to use a more realistic prior. Does this lead to improved models? Why or why not?

## Posterior Predictive Checks

Posterior Predictive Checks allow us to see if after our model is fitted does it generate replicated data
that resembles our observed data

```python
with pooled_model:
    ppc = pm.sample_posterior_predictive(pooled_trace, random_seed=42)
idata = az.from_pymc3(pooled_trace, posterior_predictive=ppc)
az.plot_ppc(idata);
```

```python
with unpooled_model:
    ppc = pm.sample_posterior_predictive(unpooled_trace, random_seed=42)
idata = az.from_pymc3(unpooled_trace, posterior_predictive=ppc)
az.plot_ppc(idata);
```

```python
with partial_pooling:
    ppc = pm.sample_posterior_predictive(partial_pooling_trace, random_seed=42)

idata = az.from_pymc3(partial_pooling_trace, posterior_predictive=ppc)
az.plot_ppc(idata);
```

## Posterior Predictive Checks

We usually in PyMC3 and Stan we usually plot our observed data and compare to posterior predictive distribution visually.

But any discrepancy measure can be used here.

# Information Criterions

## Information Criterions

Sometimes though, we want a single number to just be able to score our models. These are called *information criterions*

## Akaike information criterion (AIC)

$$ -2\log p(y \mid \hat{\theta}) + 2\text{dim}(\hat{\theta}) $$

where $\hat{\theta} = \text{argmax}_\theta \, p(y \mid \theta)$

Not very Bayesian

## Bayesian information criterion (BIC)

$$ -2\log p(y \mid \hat{\theta}) + \text{dim}(\hat{\theta}) \log n $$

where $\hat{\theta} = \text{argmax}_\theta \, p(y \mid \theta)$

Still not very Bayesian

```python

```

## Deviance information criterion (DIC)

$$ 2\left(\log p(y \mid \mathbb{E}[\theta \mid y]) - \mathbb{E}[\log p(y \mid \theta)]\right) $$

where both expectations are over $p(\theta \mid y)$

Kinda unstable

Exercise 3:

Implement AIC, BIC, and DIC for the radon models?

Hint: `model.ndim` gives the number of parameters

## Watanabe-Akaike or widely available information criterion (WAIC)

This measure is actually Bayesian and more stable than DIC

$$-2\frac{1}{n} \sum_{i=1}^n \log \mu(y_i) + \sigma^2_{log}(y_i)$$

where $\mu(y_i) = \mathbb{E}[p(y_i \mid \theta)]$ and $\sigma^2_{log}(y_i) = \mathbb{V}[\log p(y_i \mid \theta)]$

both $\mu$ and $\sigma^2_{log}$ and wrt to the $p(\theta \mid y)$ distribution

```python
pm.compare({
    'pooled': pooled_trace,
    'unpooled': unpooled_trace,
    'partially_pooled': partial_pooling_trace,
    },
    scale="log",
    ic="waic")
```

      rank
      waic
      p_waic
      d_waic
      weight
      se
      dse
      warning
      waic_scale

      partially_pooled
      0
      -1036.39
      48.5404
      0
      0.999999
      25.8018
      0
      True
      log

      unpooled
      1
      -1080.25
      131.967
      43.8558
      2.10673e-07
      30.7988
      12.12
      True
      log

      pooled
      2
      -1089.91
      3.76284
      53.5134
      4.19402e-07
      28.9869
      10.6924
      False
      log

## Leave one out Cross-Validations (LOO-cv)

The idea behind LOO-cv is that we can repeatedly

1. Set aside one of our data points $y_i$
2. Fit a model on the rest of data $p(\theta \mid y_{-i})$
3. Evaluate $\log \mathbb{E}[y_i \mid \theta]$
4. Repeat for each data point and average the results

```python
pm.compare({
    'pooled': pooled_trace,
    'unpooled': unpooled_trace,
    'partially_pooled': partial_pooling_trace,
    },
    scale="log",
    ic="loo")
```

      rank
      loo
      p_loo
      d_loo
      weight
      se
      dse
      warning
      loo_scale

      partially_pooled
      0
      -1036.77
      48.917
      0
      1
      30.8789
      0
      False
      log

      pooled
      1
      -1089.91
      3.76247
      53.1363
      1.60083e-09
      27.4187
      10.7068
      False
      log

      unpooled
      2
      -1095.06
      146.775
      58.2868
      2.9066e-13
      24.192
      12.5854
      True
      log

While LOO-CV is expensive to run, WAIC and other IC can be seen as fast approximations of it

# Posterior Dispersion Indices

Further reference:

Evaluating Bayesian Models with Posterior Dispersion Indices

Alp Kucukelbir, Yixin Wang, David M. Blei

```python
from arviz.stats.stats_utils import get_log_likelihood as get_log_likelihood
from arviz.stats.stats_utils import logsumexp as logsumexp
from arviz.stats.stats_utils import make_ufunc as make_ufunc
from arviz.stats.stats_utils import stats_variance_2d as svar
from arviz.stats.stats_utils import wrap_xarray_ufunc as wrap_xarray_ufunc

def WAPDI(idata, model=None, varnames=None):
    log_likelihood = get_log_likelihood(idata, 'y')
    log_likelihood = log_likelihood.stack(sample=("chain", "draw"))
    shape = log_likelihood.shape
    n_samples = shape[-1]
    n_data_points = np.product(shape[:-1])

    ufunc_kwargs = {"n_dims": 1, "ravel": False}
    kwargs = {"input_core_dims": [["sample"]]}
    lppd_i = wrap_xarray_ufunc(
        logsumexp,
        log_likelihood,
        func_kwargs={"b_inv": n_samples},
        ufunc_kwargs=ufunc_kwargs,
        **kwargs,
    )

    vars_lpd = log_likelihood.var(dim="sample")
    return vars_lpd / lppd_i

y_pdi = WAPDI(az.from_pymc3(pooled_trace))
```

```python
i = y_pdi.argsort()
#srrs_mn.iloc[int(i)]

data = srrs_mn[['county', 'radon', 'log_radon']]

data.iloc[i[:10]]
```

      county
      radon
      log_radon

      5557
      MCLEOD
      0.0
      -2.302585

      5224
      COTTONWOOD
      0.0
      -2.302585

      5186
      CARVER
      0.0
      -2.302585

      5556
      MCLEOD
      25.4
      3.238678

      5685
      REDWOOD
      20.7
      3.034953

      5716
      SCOTT
      19.3
      2.965273

      5124
      ANOKA
      0.2
      -1.203973

      5318
      FARIBAULT
      0.1
      -1.609438

      5183
      CARVER
      14.7
      2.694627

      5162
      BLUE EARTH
      14.3
      2.667228

```python
data[data.county == "CARVER"]
#data[data.county == "MCLEOD"]
#data[data.county == "COTTONWOOD"]
```

      county
      radon
      log_radon

      5181
      CARVER
      1.6
      0.530628

      5182
      CARVER
      12.9
      2.564949

      5183
      CARVER
      14.7
      2.694627

      5184
      CARVER
      4.7
      1.568616

      5185
      CARVER
      9.6
      2.272126

      5186
      CARVER
      0.0
      -2.302585

# Two-Sample tests

# MMD / Relative MMD / Kernel Stein Discrepancy / Witness function

```python
# Adapted from https://github.com/eugenium/MMD
from sklearn.metrics.pairwise import rbf_kernel
from scipy import stats

def grbf(x1, x2, sigma):
    '''Calculates the Gaussian radial base function kernel'''
    n, nfeatures = x1.shape
    m, mfeatures = x2.shape

    k1 = np.sum((x1*x1), 1)
    q = np.tile(k1, (m, 1)).transpose()
    del k1

    k2 = np.sum((x2*x2), 1)
    r = np.tile(k2.T, (n, 1))
    del k2

    h = q + r
    del q, r

    # The norm
    h = h - 2*np.dot(x1, x2.transpose())
    h = np.array(h, dtype=float)

    return np.exp(-1.*h/(2.*pow(sigma, 2)))

def kernelwidthPair(x1, x2):
    '''Implementation of the median heuristic. See Gretton 2012
       Pick sigma such that the exponent of exp(- ||x-y|| / (2*sigma2)),
       in other words ||x-y|| / (2*sigma2),  equals 1 for the median distance x
       and y of all distances between points from both data sets X and Y.
    '''
    n, nfeatures = x1.shape
    m, mfeatures = x2.shape

    k1 = np.sum((x1*x1), 1)
    q = np.tile(k1, (m, 1)).transpose()
    del k1

    k2 = np.sum((x2*x2), 1)
    r = np.tile(k2, (n, 1))
    del k2

    h = q + r
    del q, r

    # The norm
    h = h - 2*np.dot(x1, x2.transpose())
    h = np.array(h, dtype=float)

    mdist = np.median([i for i in h.flat if i])

    sigma = np.sqrt(mdist/2.0)
    if not sigma:
        sigma = 1

    return sigma

def kernelwidth(Zmed):
    '''Alternative median heuristic when we cant partition the points
    '''
    m = Zmed.shape[0]
    k1 = np.expand_dims(np.sum((Zmed*Zmed), axis=1), 1)
    q = np.kron(np.ones((1, m)), k1)
    r = np.kron(np.ones((m, 1)), k1.T)
    del k1

    h = q + r
    del q, r

    # The norm
    h = h - 2.*Zmed.dot(Zmed.T)
    h = np.array(h, dtype=float)

    mdist = np.median([i for i in h.flat if i])

    sigma = np.sqrt(mdist/2.0)
    if not sigma:
        sigma = 1

    return sigma

def MMD_Test(X, Y, sigma=-1, SelectSigma=2):
    if(sigma < 0):
        # Similar heuristics
        if(SelectSigma > 1):
            siz = np.min((1000, X.shape[0]))
            sigma = kernelwidthPair(X[0:siz], Y[0:siz])
        else:
            siz = np.min((1000, X.shape[0]*3))
            Zem = np.r_[X[0:siz/2], Y[0:siz/2]]
            sigma = kernelwidth(Zem)

    Kyy = grbf(Y, Y, sigma)
    Kxy = grbf(X, Y, sigma)
    Kyynd = Kyy-np.diag(np.diagonal(Kyy))
    m = Kxy.shape[0]
    n = Kyy.shape[0]

    u_yy = np.sum(Kyynd)*(1./(n*(n-1)))
    u_xy = np.sum(Kxy)/(m*n)

    # Compute the test statistic

    Kxx = grbf(X, X, sigma)
    Kxxnd = Kxx-np.diag(np.diagonal(Kxx))
    u_xx = np.sum(Kxxnd)*(1./(m*(m-1)))
    MMDXY = u_xx+u_yy-2.*u_xy
    return MMDXY
```

```python
norm1 = np.random.normal(1., 1., (1000, 2))
norm2 = np.random.normal(0.2, 1., (1000, 2))

n = norm1.shape[0]
tt = MMD_Test(norm1, norm2)
pvalue = stats.norm.sf(tt, scale=(2.0/n)**0.5)
pvalue
```

    3.248337337392221e-05

# Relative MMD

```python
# Adapted from https://github.com/eugenium/MMD
import scipy as sp

def MMD_Diff_Var(Kyy, Kzz, Kxy, Kxz):
    '''
    Compute the variance of the difference statistic MMDXY-MMDXZ
    See http://arxiv.org/pdf/1511.04581.pdf Appendix for derivations
    '''
    m = Kxy.shape[0]
    n = Kyy.shape[0]
    r = Kzz.shape[0]

    Kyynd = Kyy-np.diag(np.diagonal(Kyy))
    Kzznd = Kzz-np.diag(np.diagonal(Kzz))

    u_yy = np.sum(Kyynd)*(1./(n*(n-1)))
    u_zz = np.sum(Kzznd)*(1./(r*(r-1)))
    u_xy = np.sum(Kxy)/(m*n)
    u_xz = np.sum(Kxz)/(m*r)

    # compute zeta1
    t1 = (1./n**3)*np.sum(Kyynd.T.dot(Kyynd))-u_yy**2
    t2 = (1./(n**2*m))*np.sum(Kxy.T.dot(Kxy))-u_xy**2
    t3 = (1./(n*m**2))*np.sum(Kxy.dot(Kxy.T))-u_xy**2
    t4 = (1./r**3)*np.sum(Kzznd.T.dot(Kzznd))-u_zz**2
    t5 = (1./(r*m**2))*np.sum(Kxz.dot(Kxz.T))-u_xz**2
    t6 = (1./(r**2*m))*np.sum(Kxz.T.dot(Kxz))-u_xz**2
    t7 = (1./(n**2*m))*np.sum(Kyynd.dot(Kxy.T))-u_yy*u_xy
    t8 = (1./(n*m*r))*np.sum(Kxy.T.dot(Kxz))-u_xz*u_xy
    t9 = (1./(r**2*m))*np.sum(Kzznd.dot(Kxz.T))-u_zz*u_xz

    zeta1 = (t1+t2+t3+t4+t5+t6-2.*(t7+t8+t9))

    zeta2 = (1/m/(m-1))*np.sum((Kyynd-Kzznd-Kxy.T-Kxy+Kxz+Kxz.T)**2) - \
        (u_yy - 2.*u_xy - (u_zz-2.*u_xz))**2

    data = dict({'t1': t1,
                 't2': t2,
                 't3': t3,
                 't4': t4,
                 't5': t5,
                 't6': t6,
                 't7': t7,
                 't8': t8,
                 't9': t9,
                 'zeta1': zeta1,
                 'zeta2': zeta2,
                 })
    # TODO more precise version for zeta2
    #    xx=(1/m^2)*sum(sum(Kxxnd.*Kxxnd))-u_xx^2;
    # yy=(1/n^2)*sum(sum(Kyynd.*Kyynd))-u_yy^2;
    # xy=(1/(n*m))*sum(sum(Kxy.*Kxy))-u_xy^2;
    # xxy=(1/(n*m^2))*sum(sum(Kxxnd*Kxy))-u_xx*u_xy;
    # yyx=(1/(n^2*m))*sum(sum(Kyynd*Kxy'))-u_yy*u_xy;
    #zeta2=(xx+yy+xy+xy-2*(xxy+xxy +yyx+yyx))

    Var = (4.*(m-2)/(m*(m-1)))*zeta1
    Var_z2 = Var+(2./(m*(m-1)))*zeta2

    return Var, Var_z2, data

def relative_MMD_Test(X, Y, Z, sigma=-1, SelectSigma=2, computeMMDs=False):
    '''Performs the relative MMD test which returns a test statistic for whether Y is closer to X or than Z.
    See http://arxiv.org/pdf/1511.04581.pdf
    The bandwith heuristic is based on the median heuristic (see Smola,Gretton).
    '''
    if(sigma < 0):
        # Similar heuristics
        if(SelectSigma > 1):
            siz = np.min((1000, X.shape[0]))
            sigma1 = kernelwidthPair(X[0:siz], Y[0:siz])
            sigma2 = kernelwidthPair(X[0:siz], Z[0:siz])
            sigma = (sigma1+sigma2)/2.
        else:
            siz = np.min((1000, X.shape[0]*3))
            Zem = np.r_[X[0:siz/3], Y[0:siz/3], Z[0:siz/3]]
            sigma = kernelwidth(Zem)

    Kyy = grbf(Y, Y, sigma)
    Kzz = grbf(Z, Z, sigma)
    Kxy = grbf(X, Y, sigma)
    Kxz = grbf(X, Z, sigma)
    Kyynd = Kyy-np.diag(np.diagonal(Kyy))
    Kzznd = Kzz-np.diag(np.diagonal(Kzz))
    m = Kxy.shape[0]
    n = Kyy.shape[0]
    r = Kzz.shape[0]

    u_yy = np.sum(Kyynd)*(1./(n*(n-1)))
    u_zz = np.sum(Kzznd)*(1./(r*(r-1)))
    u_xy = np.sum(Kxy)/(m*n)
    u_xz = np.sum(Kxz)/(m*r)
    # Compute the test statistic
    t = u_yy - 2.*u_xy - (u_zz-2.*u_xz)
    Diff_Var, Diff_Var_z2, data = MMD_Diff_Var(Kyy, Kzz, Kxy, Kxz)

    pvalue = sp.stats.norm.cdf(-t/np.sqrt((Diff_Var)))
    # pvalue_z2=sp.stats.norm.cdf(-t/np.sqrt((Diff_Var_z2)))
    tstat = t/np.sqrt(Diff_Var)

    if(computeMMDs):
        Kxx = grbf(X, X, sigma)
        Kxxnd = Kxx-np.diag(np.diagonal(Kxx))
        u_xx = np.sum(Kxxnd)*(1./(m*(m-1)))
        MMDXY = u_xx+u_yy-2.*u_xy
        MMDXZ = u_xx+u_zz-2.*u_xz
    else:
        MMDXY = None
        MMDXZ = None
    return pvalue, tstat #, sigma, MMDXY, MMDXZ
```

```python
true_model = np.random.normal(1., 1., (1000, 2))
model1 = np.random.normal(1.3, 1., (1000, 2))
model2 = np.random.normal(1.1, 1., (1000, 2))

pvalue, tstat = relative_MMD_Test(true_model, model1, model2)
pvalue
```

    0.0018383511949117198

## Kernel Stein Discrepancy

```python
class PyMCgof(object):
    def __init__(self, model):
        self.model = model
        self.dlogp = model.dlogp()

    def grad_log(self, x):
        res = self.dlogp({"x": x})
        return res.reshape(x.shape)
```

```python
# Install kgof with pip install git+https://github.com/wittawatj/kernel-gof.git

import kgof.goftest as gof
from kgof import data, kernel, util

# sample true data
mu_r = 0.1
var_r = 1
n = 500

X = np.random.normal(loc=mu_r, scale=var_r, size=(n, 2))

with pm.Model() as model:
    x = pm.Normal("x", [0., 0.1], 1., shape=X.shape)

p = PyMCgof(model)

med = util.meddistance(X, subsample=1000)
k = kernel.KGauss(sigma2=med**2/2.0)
V = util.fit_gaussian_draw(X, 5)
fssd = gof.FSSD(p=p, k=k, V=V) # Finite Set Stein Discrepancy
fssd.perform_test(data.Data(X))
```

    {'alpha': 0.01,
     'pvalue': 0.014666666666666666,
     'test_stat': 0.8537068071404676,
     'h0_rejected': False,
     'n_simulate': 3000,
     'time_secs': 0.008444786071777344}

# Conclusions


```python
# initial setup
try:
    # settings colab:
    import google.colab
except ModuleNotFoundError:
    # settings local:
    %run "../../../common/0_notebooks_base_setup.py"
```

---

#### Distribución Normal

¿Por qué es tan popular esta distribución?

La importancia de la distribución normal proviene principalmente del hecho de que las distribuciones de muchos fenómenos naturales están al menos aproximadamente distribuidos normalmente.

Una de las primeras aplicaciones de la distribución normal fue el análisis de errores de medición realizados en observaciones astronómicas, errores debidos a instrumentos imperfectos y observadores imperfectos.

Galileo en el siglo XVII observó que estos errores eran simétricos y que los errores pequeños ocurrían con mayor frecuencia que los errores grandes.

Esto condujo a varias distribuciones hipotéticas de errores, pero no fue hasta principios del siglo XIX que se descubrió que estos errores seguian una distribución normal.

Independientemente, los matemáticos Adrain en 1808 y Gauss en 1809 desarrollaron la fórmula para la distribución normal y mostraron que los errores se ajustaban bien a esta distribución.

La distribución normal tiene dos parámetros: su media $\mu$ y su varianza $\sigma^2$

Su función de densidad de probabilidad es

\begin{equation}
    f(x) = \frac{1}{\sigma \sqrt{2\pi}}exp^{-\frac{(x-\mu)^2}{2\sigma^2}}
\end{equation}

Los valores posibles de X son los reales entre menos y más infinito.

El gráfico de la densidad para una variable normal tiene forma de campana.

La probabilidad de que una variable aleatoria X con distribución normal y parámetros  $\mu$, $\sigma^2$ sea menor o igual que un valor $a$ fijo es

\begin{equation}
  P(X \le a) = \int^a_{-\infty}\frac{1}{\sigma \sqrt{2\pi}}exp^{-\frac{(x-\mu)^2}{2\sigma^2}}dx
\end{equation}

La distribución normal es simétrica respecto a su media.

En particular cuando la media es cero y el desvío estándar es igual a 1 decimos se se trata de una **normal estándar**.

#### Algunas propiedades de la distribución normal

* Cualquier función lineal de una variable aleatoria normal también es normal.
Si X es una variable aleatoria normal con parámetros $\mu$, $\sigma^2$ podemos definir Y=a+bX  con b distinto de cero (¿Por qué?).
Entonces Y también tendrá distribución normal (¿Con qué parámetros?)

* La variable aleatoria generada por una suma finita de variables aleatorias normales independientes es normal.

* En el caso de que se sumen infinitas variables aleatorias, independientes y tales que tanto la suma de las medias como de las varianzas sean finitas también vale la propiedad.

Función de densidad de probabilidad:

Función de distribución de probabilidad:

Datos con distribución normal:

**Ejemplos**:

* X: longitud de los clavos producidos en una fábrica

* X: altura de los alumnos de DH

* X: tiempo vida de un modelo de lámpara

* X: tiempo que utiliza un empleado en realizar determinada tarea en una empresa.

Prueben ahora generando 100000 datos con distribución normal con distintos valores de media y varianza, usando como modelo el código que vimos en las notebooks anteriores.
Y comparen los gráficos intentando responder cómo varian los gráficos de las distribuciones al modificar estos parámetros (media y varianza).

#### Referencias

https://luminousmen.com/post/data-science-probability-distributions

Gráficos: https://es.wikipedia.org/wiki/Distribuci%C3%B3n_normal


#### [**NICOLAS CACHANOSKY**](http://www.ncachanosky.com) | Department of Economics | Metropolitan State University of Denver | ncachano@msudenver.edu

# A SIMPLE RAMSEY MODEL
---

This note illustrates how to code a simple Ramsey model in Python. The purpose of the note is to walk through Python applications, not to offer a detailed discussion of Ramsey model or to show best coding practices. The note also assumes familiarity with Ramsey model and a beginner experience with Python.

For a more complete and detailed discussion of Python applications see the material in [Quant Econ](https://quantecon.org/).

---

## TABLE OF CONTENTS
1. [Description of the model](#1.-DESCRIPTION-OF-THE-MODEL)
2. [The respresentative household](#2.-THE-REPRESENTATIVE-HOUSEHOLD)
3. [The Representative Firm](#3.-THE-REPRESENTATIVE-FIRM)
4. [Equilibrium](#4.-EQUILIBRIUM)
5. [The code](#5.-THE-CODE)

## 1. DESCRIPTION OF THE MODEL

A known issue with the Solow model is that the savings rate is exogenous (and constant). This means the ratio of consumption to income $Y/C$ is also constant. While the Solow model can be useful to highlight some features of long-run growth, this assumption also limits the applicability of this model. Yet, the Solow model can be interpreted as a specIal case of Ramsey model (or Ramsey-Kass-Koopmans model). Ramsey model can be considered as the building block of macro models with a household that optimizes its utility and, therefore, endogenously sets the savings rate.

This note presents a "simple" Ramsey model (it offers a simplified discussion, does not discuss shocks, and does not add other features). For a more complete and detailed discussion see an advanced macroeconomics textbook.

The model is populated by a representative household that maximizes its utility and by a representative firm that maximizes its profits. The model assumes a competitive market (no monopoly power, agents are price takers, etc.) The household provides labor services to the firm in exchange of a wage. The wage can be used to consume goods or save and accumulate assets. The model assumes a closed economy with no government. Finally, the model is presented in continuous (rather than discrete) time. A "dot" on top of a variable denotes its instantaneous time-change $(\dot x (t) = \frac{\partial x}{\partial t})$

# 2. THE REPRESENTATIVE HOUSEHOLD

Assume a large number of identicial infinitely-lived households. Because these households are identical, we can use  a **representative** household to populate the model. The assumptions that this households are infinitely lived captures a continuum of generations. The representative household has $L(t)$ individuals with a population growth rate of $n$. Therefore, $L(t) = L(0) \cdot e^{nt}$

## 2.1. THE HOUSEHOLD UTILITY

The household utility $(U)$ is the present value of the utility (felicity function) of all household individuals $(u(c))$.

\begin{equation}
    U = \int_{0}^{\infty} u[c(t)] \cdot e^{nt} \cdot \underbrace{e^{-\rho t}}_{L(t)} \cdot dt = \int_{0}^{\infty} u[c(t)] \cdot e^{(n-\rho)t} \cdot dt
\end{equation}

where $\rho > 0$ is the time preference of the household individual and where $n -\rho >0$. Also, $u(c)$ is concave $(u^{'}(c) > 0, u^{''}(c) < 0)$ and satisfies the Inada conditions $(u^{'}(c) \to \infty \text{ as } c \to 0 \text{, and } u^{'}(c) \to 0 \text{ as } c \to \infty$.

The parameter $\rho$ can be interpreted as the time preference of the household or as capturing a discount of the future generation by the present generations. If $\rho = 1$, then present generation assigns equal "value" to the next generatio (i.e. parents with respect to their kids). If $\rho < 0$, then the present generation is altruistic and values the utility of the next generation more than their own.

## 2.2. THE HOUSEHOLD BUDGET CONSTRAINT

The household budget constrained is composed of assets $(\Lambda)$ and wages $(w)$. The non-consumed wage becames an increase in assets. At some period in time, assets can be negative, which means that in that period the household has a net debt. Assets have a rate of return of $r$. Therefore, the stock of assets increase by the return of the assets plus wages, and decrease by the amount of consumption. Since population growths at rate $n$, the change in household's assets can be represented as follows.

\begin{equation}
    \frac{d\Lambda}{dt} = r(t) \cdot \Lambda(t) + w(t)L(t) - C(t)
\end{equation}

Dividing by $L(t)$ we get the change in assets per capita $(a)$ (and dropping the $t$ variable for notation simplicity):

\begin{align}
    \dot{\left(\frac{A}{L} \right)} &= \frac{\dot{A}L - A \dot{L}}{L^2}                \\
    \dot{a} &= \frac{\dot{A}}{L} - \frac{A}{L}n                                        \\
    \dot{a} &= ra + w - c - na                                                         \\
    \dot{a} &= (r - n)a + w - c
\end{align}

To avoid the situation where the household borrows money indifenetely, the model also imposses a "no Ponzi game condition". Intuitively, at the end of the household life $(T)$, assets have to be at least zero $a(T) \geq 0$. A household may have debt at any point in time, but at the end of its lifetime all debts are paid. The present value of the assets have to be asymptotically nonnegative.

\begin{equation}
    \lim\limits_{t \to \infty} a(t) e^{-\tilde{r}(t)} \geq 0
\end{equation}

where $\tilde{r}(t) = \int_{0}^{t} (r(\tau) - n) d\tau$. This expression means that any moment $\tau \in (0, \infty)$, the present value of $a$ cannot be negative. Note that $\tilde{r}$ is taking into consideration the growht rate of the household as well $n$. In a more simple notation, this condition can be written as $\lim\limits_{t \to \infty} a(t) \geq 0$.

## 2.3. THE HOUSEHOLD OPTIMIZATION PROBLEM

The household is  facing a dynamic optimization problem. It needs to set the path of consumption that maximizes the present value of its life utility. The optimization problem can be described with the following equations (where $n - \rho > 0$).

\begin{align}
    \max_{c(t)} U &= \int_{t=0}^{\infty} u[c(t)] \cdot e^{-(n-\rho)t}    \\
    \text{subject to:}                                                   \\
    \dot{a} &= (r(t) - n)a(t) + w(t) - c(t)                              \\
    a(0) &= a_{0}                                                        \\
    \lim\limits_{t \to \infty}a(t) &\geq 0                               \\
    \text{transversality condition:}                                     \\
    \lim\limits_{t \to \infty} \mu(t) \cdot a(t) = 0
\end{align}

The optimization problem is given an initial asset value of $a_0$ and a terminal condition for which assets cannot be negative. Let $\mathscr{H}$ be the present-value Hamiltonian:

\begin{equation}
    \mathscr{H} = e^{-(\rho - n)t} \cdot u[c(t)] + \mu(t) \cdot \left[r(t) - n)a(t) + w(t) - c(t) \right]
\end{equation}

where $\mu(t)$ is the present-value shadow price. Since the household faces an optimization problem at each moment $t$, the shadow price may vary over time as well. The "transversality condition" states that in the limit, either the shadow price or the assets have to be zero.

The first-order-conditions (FOC) of the hamiltonian are:

\begin{align}
    \frac{\partial \mathscr{H}}{\partial c} &= e^{-(\rho - n)t} \cdot u'_{c} - \mu(t) = 0       \\
    \frac{\partial\mathscr{H}}{\partial\mu} &= (r(t) - n)a(t) + w(t) - c(t) = \dot{a}           \\
    \frac{\partial \mathscr{H}}{\partial a} &= - (r(t) - n)\cdot \mu (t) = \dot{\mu}(t)
\end{align}

Now we proceed to use the first equation to get $\dot{\mu}(t)$.

\begin{align}
    \mu(t)       &= e^{-(\rho - n)t} \cdot u'_{c} \\
    \dot{\mu}(t) &= -(\rho - n) \cdot e^{-(\rho - n)t} \cdot  u'_{c} + e^{-(\rho - n)t} \cdot u''_c \cdot \dot{c}
\end{align}

Make now two substitutions: (1) The first FOC in the third FOC, and (2) this last derivative in the third FOC as well.

\begin{align}
    -[r(t) -n] \cdot \mu(t) &= -(\rho - n) \cdot e^{-(\rho - n)t} \cdot u'(c) + e^{-(\rho - n)t} \cdot u''(c) \cdot \dot{c}        \\[10pt]
    -[r(t) -n] \cdot e^{-(\rho - n)t} \cdot u'(c) &= e^{-(\rho - n)t} \left[-(\rho - n) \cdot u'(c) + u''(c) \cdot \dot{c} \right] \\[10pt]
    -[r(t) -n] \cdot u'(c) &= -(\rho - n) \cdot u'(c) + u''(c) \cdot \dot{c}                                                       \\[10pt]
    r(t) &= \rho - \frac{u''(c)\cdot c}{u'(c)} \cdot \frac{\dot{c}}{c}
\end{align}

The last equation is the **Euler equation**, which defines the path for $c$ given the rate of return $r(t)$. Recall that $u''(c)<0$ and therefore $\frac{u''(c)c}{u'(c)}>0$. The right-hand side of the equation can be interpreted as the "rate of return of consumption." Consumers would like to smooth consumption in time. Therefore, the rate of return $(r)$ has to be equal to the time preference $(\rho)$ minus the change of consumption over time. Or, put it differently, if $r = \rho$, then consumption will be flat. There is no opportunity to increase utility by moving consumption over time (extend a loan and postpone consumption or take a debt and advance consumption). Consumers will be willing to postpone consumption if $r$ is sufficiently above $\rho$. The required compensation to postpone consumption is the second term in the righ-hand side of the equation.

\begin{align}
    r &= \rho \to \frac{\dot{c}}{c} = 0 \\
    r &> \rho \to \frac{\dot{c}}{c} > 0 \text{    (saving today: postpone present consumption)} \\
    r &< \rho \to \frac{\dot{c}}{c} < 0 \text{    (consuming today: advance future consumption)}
\end{align}

The Euler equation also includes the elasticity of marginal utility: $\frac{u''(c)\cdot c}{u'(c)}$. For $r$ and $\frac{\dot{c}}{c}$ to be constant, this elasticity of marginal utility has to be constant. A common practice is to assume a CRRA (constant relative risk aversion) utility function that depicts constant intertemporal elasticity of substitution (CIES).

\begin{equation}
    u(c) = \frac{c^{1-\theta}-1}{1-\theta}
\end{equation}

where $\theta > 0$ is the coefficient of relative risk aversion. The $1 - \theta$ denominator assures the function will keep its desired properties of the function $(u'>0, u''<0)$ even if $\theta > 1$ (see below). The inclution of the term $-1/(1-\theta)$ is for convenience; when $\theta \to 1$ then the CRRA function becomes $(u(c) = log(c))$. Consider now the first and second derivative of the CRRA function and the elasticity of substitution $(\sigma)$.

\begin{align}
    u'(c)  &= c^{-\theta} > 0                      \\
    u''(c) &= -\theta \cdot c^{-(\theta + 1)} < 0  \\
    \sigma &= \left[ \frac{u''(c) \cdot c}{u'(c)} \right]^{-1} = \left[\frac{\theta \cdot c^{-(\theta + 1)} \cdot c}{c^{-\theta}} \right]^{-1} = \frac{1}{\theta}
\end{align}

Then, the Euler equation from the household maximization problem becomes

\begin{align}
    r(t) &= \rho - \frac{u''(c)\cdot c}{u'(c)} \cdot \frac{\dot{c}}{c} \\
    r(t) &= \rho - \theta \cdot \frac{\dot{c}}{c}                      \\
\end{align}

We can rewrite the Euler equation to more conveniently depict the consumption path that maximizes the household utility.

\begin{equation}
    \frac{\dot{c}}{c} = \frac{1}{\theta} \cdot \left(r(t) - \rho \right)
\end{equation}

This is one of the two equations that describes Ramsey model. The second equation is derived from the firms profit maximization problem (next section).

---

The following code plots the utility function for different levels of $\theta$. The function also allows for $\theta = 1$, in such case the utility function becomes the log of consumption.

```python
"1|IMPORT PACKAGES"
import numpy as np              # Package for scientific computing with Python
import matplotlib.pyplot as plt # Matplotlib is a 2D plotting library

"2|DEFINE PARAMETERS AND ARRAYS"
# PARAMETERS
size    = 5        # Model domain
steps   = size*100    # Number of "dots" in the domain

# ARRAY
c = np.linspace(0, size, steps) # Create array of consumption

"3|DEFINE UTILITY FUNCTION"
# Production function
def u(x, CRRA):
    if CRRA == 1:
        u = np.log(x)
    else:
        u = ((x)**(1-CRRA)-1)/(1-CRRA)
    return u

"4|CALCULATE UTILITY FUNCTIONS FOR DIFFERENT VALUES OF THETA"
theta = [0.25, 0.75, 1, 1.5, 2]

# Ignore c[0] = 0 to avoid dividing by zero
u1 = u(c[1:], theta[0])
u2 = u(c[1:], theta[1])
u3 = u(c[1:], theta[2])
u4 = u(c[1:], theta[3])
u5 = u(c[1:], theta[4])

"5|PLOT UTILITY FUNCTIONS"
### AXIS RANGE
v = [0, size, -10, 5]
### BUILD PLOT AND POPULIATE WITH LOCI LINES
fig, ax = plt.subplots(figsize=(10, 7))
ax.plot(c[1:], u1, alpha = 0.8, label = r'$\theta=%.2f$' %theta[0])
ax.plot(c[1:], u2, alpha = 0.8, label = r'$\theta=%.2f$' %theta[1])
ax.plot(c[1:], u3, alpha = 0.8, label = r'$\theta=%.2f$' %theta[2])
ax.plot(c[1:], u4, alpha = 0.8, label = r'$\theta=%.2f$' %theta[3])
ax.plot(c[1:], u5, alpha = 0.8, label = r'$\theta=%.2f$' %theta[4])
### AXIS
ax.axhline(0, color = 'k') # Add horizontal axis
ax.axvline(0, color = 'k') # Add vertical axis
ax.yaxis.set_major_locator(plt.NullLocator())                # Hide ticks
ax.xaxis.set_major_locator(plt.NullLocator())                # Hide ticks
### TEXT
plt.text(size-0.10, -0.5, r'$c$', color = 'k') # x-axis label
plt.text(    -0.25,  4.5, r'$u(c)$')           # y-axis label
# SETTINGS
plt.box(False)                                               # Hide axis
plt.legend(loc=0, frameon=False)
plt.axis(v)
plt.show()

```

# 3. THE REPRESENTATIVE FIRM

The representative firm uses a labor-technology augmenting production function $(Y = F(K; AL)$ with constant returns to scale (CRS). The production function depicts the typical properties of diminishing marginal returns and the Inada conditions. Technology $(A)$ grows at rage $g$: $A(t) = A(0)\cdot e^{gt}$.

In terms of effective labor $(AL)$, and  normalizing the price of the good to one $(p = 1)$, the profit of the firm becomes (where the $"\hat{}"$ symbol denotes the variable is in per effective labor units):

\begin{equation}
    \hat{\pi} = f(\hat{k}) - R \hat{k} - \hat{w}
\end{equation}

The rate of return on capital equals the gross rate of return $(R)$ minus the depreciation rate $(\delta)$: $r = R - \delta$. In this model there is no intertermporal optimization. The firm rents capital from the household one period at the time.

The FOC for the firm with respect to capital per effective unit of labor utilization is:

\begin{equation}
    \frac{\partial \pi}{\partial \hat{k}} = f'(\hat{k}) - R = 0
\end{equation}

In equilibrium the firm's profit is zero, therefore:

\begin{align}
    \hat{w} &= f(\hat{w}) - R\hat{k}       \\
    \hat{w} &= f(\hat{w}) - (r + \delta)k
\end{align}

# 4. EQUILIBRIUM

In equilibrium, the assets owned by the household coincide with the capital per unit of labor. Recall that in this model households own the capital that is rented to the firms. Also, note that assets $(a)$ are expressed in per labor terms, while $(k)$ is expressed in per effective labor. Therefore: $a = Ak$. Then:

\begin{align}
    \dot{a}                      &= ra + w - c - na                          \\[5pt]
    \dot{(Ak)}                   &= r \cdot Ak + Aw - Ac - n \cdot Ak        \\[5pt]
    \dot{A}k + A \dot{k}         &= r \cdot Ak + Aw - Ac - n \cdot Ak        \\[5pt]
    \frac{\dot{A}}{A}k + \dot{k} &= rk + w - c - nk                          \\[5pt]
    gk + \dot{k}                 &= rk + w - c - n    k                      \\[5pt]
    \dot{k}                      &= rk + f(k) - (r + \delta)k - c - (n + g)k \\[5pt]
    \dot{k}                      &= f(k) - c - (n + g + \delta)k
\end{align}

Knowing that $R = r + \delta$ and that $f'(k) = R + \delta$, then the two motion equations that model Ramsey model are:

\begin{cases}
    \frac{\dot{c}}{c} &= \frac{1}{\theta} \cdot (f'(k) - \delta - \rho) \\[10pt]
    \dot{k}           &= f(k) - c - (n + g + \delta)
\end{cases}

In equilibrium, $\hat{c} = 0$ and $\hat{k} = 0$. Assuming the equilibrium conditions, we can rewrite the first of Ramsey model equations as an equilibrium **value** of $\hat{k}$, and the second equation can be represented as $\hat{c}$ in terms of $\hat{k}$. First, assume the production following, $Y = A \cdot \left[K^{\alpha} (AL)^{1 - \alpha} \right] (\alpha \in (0, 1))$. In terms of effective labor: $\hat{y} = \hat{k}^{\alpha}$.

\begin{align}
    \hat{k} &= (\delta + \rho)^{1/\alpha}             \\
    \hat{c} &= \hat{k}^{\alpha} - (n + g + \delta) \cdot \hat{k}
\end{align}

From the second equation we can also derive the value of $\hat{k}$ that maximizes $\hat{c}$; the *golden-rule*.

\begin{align}
    \hat{c}                                   &= \hat{k}^{\alpha} - (n + g + \delta) \cdot \hat{k}                   \\[10pt]
    \frac{\partial \hat{c}}{\partial \hat{k}} &= \alpha \cdot \hat{k}^{-(1 - \alpha)} - (n + g + \delta) \hat{k} = 0 \\[8pt]
    \hat{k}^{*}_{g}                           &= \left(\frac{\alpha}{n + g + \delta} \right)^{\frac{1}{1-\alpha}}
\end{align}

The model has one stable saddle path that converges to equilibrium. This saddle-path is unstable, if the model is outside this path by the smallest distance, then the model diverges away from equilibirum.

# 5. THE CODE

The Python code has the following structure. Following the above discussion, after importing the required packages and defining parameters and arrays, the equilibrium, gold values, and _loci_ lines are calculated. The code plots two graphs.

The first one shows the _loci_ lines with five sample paths (in red). Each one of these sample paths depicts the unstable dynamics of the model. The second graph plots the stable-path (in blue).

## 5.1 ESTIMATING THE STABLE-PATH

A common way to estimate the stable-path is with a "shooting" methodology. This approach shoots a number of paths until it finds one that is close enough the actual stable path. Let $\tilde{k}$ and $\tilde{c}$ denote estimated values of the path produced by the "shooting" algorithm.

This note uses a **simple** code and methodology to illustrate this type of numerical method. It is not intended to be the more efficient one. The stable path is estimated in two halves. One estimation for the stable path section to the left of the equilirium point, and another estimation for the stable path to the right of the equilibrium point. These two estimations are coded as two different functions: (1) backward shoot and (2) forward shoot respectively. Alternatively, one simple "shoot" function could include both initial locations.

These type of iterations can be time consuming depending on how many shoots the code is going to perform (which depends on how big a correction is when a shoot misses the steady-state), what the tolerance level is, etc. A more sofisticated code could reduce the number of attempts and save computational time.

Since we know that is very unlikely that our "shoots" will exactly hit the steady state $(k^*, c(k^*)$, we need to define a success criteria. The success criteria is when the shoots hits close enough to the target. We need to define circle around the equilibrium that plays the role of the tolerance level. Once we find a "shoot" that hits inside this circle, the shoots is considered a success and an approximation good enough for the estimation purposes.

The distance of a point to the steady-state is: $distance = \sqrt[2]{(\tilde{k} - k^*)^2 + (\tilde{c} - c^*)^2}$, where $\tilde{k}, \tilde{c}$ are the shoot estimations.

The shooting functions have two inputs. The initial level of $k_0$ and a "step" size. The "step" size is how big the correction of the initial shoot will be after it misses the tolerance level (the target). Given $k_0$, the function estimates a level of $c(k_0)$ (the first shoot). If the shoot misses the target, then the algorithm performs a correction to the initial level of $c_0$ and shoots again. This sequence repeats until the shoot hits the target.

### 5.1.1 "FORWARD" SHOOT
The first shoot of the forward shoot function is located one step below the _loci_ function. We know is very likely that this initial shoot is too high, and therefore the path will go above $c^*$. The loop has two break points to define a succesfull shoot. The first one is if the path produces a point $(k, c)$ inside the tolerance distance from the steady-state (the above discussion). The second one is the path produces a value of $\tilde{k} > k_*$. This break point is in place because the step size (how far the new shoot starting point moves) is a fixed amount without a correction or feedback mechanism. It is possible, then, that after missing the target the next iteration starts at a value of $c$ that is too low. Such path may still be an estimation good enough even if it does not satisfy the tolerance level (for the purposes of this notebook, this is enough).

There is a third break, which is a cap on the number of iterations the loop can perform. When building a code like this, this type of break can help avoid a mistake that sends the code into an infinite loop.

If the path produces a value of $\tilde{c} > c^*$, then the loop resets and starts again from a new initial $c_0$ that is located one "step size" below the previous shoot.

### 5.1.2 "BACKWARD" SHOOT

The backward shoot function follow a similar structure than the forward shoot. Yet, because the starting point is different, the break points have to be modified. However, there is a difference in the starting point. Rather than starting at the _loci_ level given the initial on step above $c_0 = c(k_0)$, the code starts with $c_0 = c(k_0)\cdot 1.5$. This is an example of an easy way to save some computational power. Looking at the first figure (the unstable paths) we know that the level of consumption of the stable path will be several steps above $c(k_0)$. Therefore, we can save computation time by avoiding shoot attempts between $c(k_0)$ and $c(k_0)\cdot 1.5$.

The forward shoot loop has two stop points of success. The first one is if the shoot produces a point inside the tolerance circle. The second one is if the path produces a value of $\tilde{k} < k^*$ (for similar reasons than the forward shoot break). There is also the third break built on the number of iterations the loop will perform. Since the starting point is farther away from the equilibrium point, this break has a larger limit of iterations than the forward shoot loop.

If the shoot produces a value of $\tilde{c} < c^*$, then the code resets and calculates a new shoot with a new $c_0$ one step above the previous shoot.

```python
"1|IMPORT PACKAGES"
import numpy as np              # Package for scientific computing with Python
import matplotlib.pyplot as plt # Matplotlib is a 2D plotting library

"2|DEFINE PARAMETERS AND ARRAYS"
# PARAMETERS
k_size  =  150        # Model domain
steps   =  k_size*100 # Number of "dots" in the domain
alpha   =  0.30       # Output elasticity of capital
delta   =  0.35       # Depreciation rate
rho     =  0.35       # Time preference
n       =  0.05       # Population growth rate
g       =  0.05       # TFP growth rate
theta   =  0.8        # Coefficient or Relative Risk Aversion
# ARRAY
k_array = np.linspace(0, k_size, steps) # Create array of k

"3|DEFINE FUNCTIONS"
# PRODUCTION FUNCTION
def y(k):
    y = k**alpha
    return y

# MARGINAL PRODUCT OF CAPITAL
def y_prime(k):
    y_prime = alpha*k**(alpha-1)
    return y_prime

# CONSUMPTION
def consumption(k):
    c = y(k) - (n + g + delta)*k
    return c

# MOTION FUNCTION OF CONSUMPTION
def cdot(k, c):
    cdot = 1/theta * (y_prime(k) - delta - rho) * c
    return cdot

# MOTION FUNCTION OF CAPITAL
def kdot(k, c):
    kdot = y(k) - c - (n + g + delta)*k
    return kdot

"4|CALCULATE STEADY STATE AND GOLD VALUES"
# STEADY STATE VALUES
k_star = (delta + rho)**(1/alpha)
c_star = consumption(k_star)
y_star = y(k_star)
s_star = (y_star - c_star)/y_star

# GOLD VALUES
k_gold = (alpha/(n+g+delta))**(1/(1-alpha))
c_gold = consumption(k_gold)
y_gold = y(k_gold)
s_gold = (y_gold - c_gold)/y_gold

"5|CALCULATE LOCI FUNCTIONS"
k_loci = k_star
c_loci = consumption(k_array)

"6|CALCULATE SAMPLE PATHS"
k00, k01, k02 = k_star*0.25, k_star*0.50, k_star*2.2
c0 = [consumption(k00)*0.25, consumption(k00), c_star*1.25,
      consumption(k01)*0.40, consumption(k01)*1.50]

def sample_path(k0, c0, n):
    path = np.zeros(shape=(n, 2))
    path[0, 0] = k0
    path[0, 1] = c0

    for j in range(n-1):
        if path[j,0] < 0: # Stop if motion goes off-chart
            break
        else:
            path[j+1, 0] = path[j, 0] + kdot(path[j, 0], path[j, 1])*0.25
            path[j+1, 1] = path[j, 1] + cdot(path[j, 0], path[j, 1])*0.25
    return path

path1 = sample_path(k00, c0[0], 30)
path2 = sample_path(k00, c0[1], 30)
path3 = sample_path(k01, c0[2], 30)
path4 = sample_path(k02, c0[3], 30)
path5 = sample_path(k02, c0[4], 30)

"7|PLOT RAMSEY MODEL WITH SAMPLE PATHS"
# Value of k such that c = 0
k_zero = (1/(n + g + delta))**(1/(1-alpha))
# Axis range
y_max = np.max(c_loci)*1.7
x_max = k_zero*1.2
v = [0, x_max, -0.1, y_max]

### BUILD PLOT AND POPULIATE WITH LOCI LINES
fig, ax = plt.subplots(figsize=(10, 7))
ax.plot(k_array, c_loci, color="K", alpha = 0.8)
ax.axvline(k_loci, color="K", alpha = 0.8)
### SAMPLE PATHS
ax.plot(path1[:,0], path1[:,1], "r--", alpha = 0.7)
ax.plot(path2[:,0], path2[:,1], "r--", alpha = 0.7)
ax.plot(path3[:,0], path3[:,1], "r--", alpha = 0.7)
ax.plot(path4[:,0], path4[:,1], "r--", alpha = 0.7)
ax.plot(path5[:,0], path5[:,1], "r--", alpha = 0.7)
### ADD DOTS AT BEGINNING OF SAMPLE PATHS
plt.plot(k00, c0[0], 'ro', alpha = 0.7)
plt.plot(k00, c0[1], 'ro', alpha = 0.7)
plt.plot(k01, c0[2], 'ro', alpha = 0.7)
plt.plot(k02, c0[3], 'ro', alpha = 0.7)
plt.plot(k02, c0[4], 'ro', alpha = 0.7)
### ADD MOTION ARROWS
plt.arrow(k00, c0[0], 0, 0.10, head_width=0.03,head_length=0.02,fc='r',ec='r')
plt.arrow(k00, c0[0], 0.18, 0, head_width=0.02,head_length=0.03,fc='r',ec='r')
plt.arrow(k01, c0[2], 0, 0.10, head_width=0.03,head_length=0.02,fc='r',ec='r')
plt.arrow(k01, c0[2],-0.08, 0, head_width=0.02,head_length=0.03,fc='r',ec='r')
plt.arrow(k02, c0[3], 0,-0.08, head_width=0.03,head_length=0.02,fc='r',ec='r')
plt.arrow(k02, c0[3], 0.22, 0, head_width=0.02,head_length=0.03,fc='r',ec='r')
plt.arrow(k02, c0[4], 0,-0.12, head_width=0.03,head_length=0.02,fc='r',ec='r')
plt.arrow(k02, c0[4],-0.12, 0, head_width=0.02,head_length=0.03,fc='r',ec='r')
### AXIS
ax.axvline(0, color="k")                                     # y-axis
ax.axhline(0, color="k")                                     # x-axis
ax.yaxis.set_major_locator(plt.NullLocator())                # Hide ticks
ax.xaxis.set_major_locator(plt.NullLocator())                # Hide ticks
### TEXT
plt.text(x_max*0.98, -0.05      , r'$\hat{k}$', color = 'k') # x-axis label
plt.text(-0.15 , y_max*0.95     , r'$\hat{c}$', color = 'k') # y-axis label
plt.text(k_star*1.05, y_max*0.95, r'$\hat{c}=0$', color = "k")
plt.text(k_zero*0.95, 0.07      , r'$\hat{k}=0$', color = "k")
# SETTINGS
plt.axis(v)
plt.box(False)                                               # Hide axis
plt.show()

"8|STABLE-PATH: FORWARD AND BACKWARD SHOOTING"
def forward_shoot(k0, step):
    # Set conditions for initial shoot
    tol   = 1.0e-10
    c0    = consumption(k0) - step
    error = np.abs(((k0 - k_star)**2 + (c0 - c_star)**2)**0.5)
    path      = np.zeros(shape=(1, 2))
    path[0,0] = k0
    path[0,1] = c0

    # Start loop (forward shoot)
    count = 0
    while 1:
        k1 = path[count, 0]
        c1 = path[count, 1]
        k2 = k1 + kdot(k1, c1)*0.1
        c2 = c1 + cdot(k1, c1)*0.1
        error = np.abs(((k2 - k_star)**2 + (c2 - c_star)**2)**0.5)
        path = np.append(path, [[k2, c2]], axis = 0)
        count = count + 1

        # Check if this is the stable-path
        if error < tol:
            break

        # Set up code break and reset the shoot
        if c2 > c_star:
            path = np.zeros(shape=(1, 2))
            c0 = c0 - step
            path[0,0] = k0
            path[0,1] = c0
            count = 0

        # Break code when k < k*
        if k2 > k_star:
            break

        # Put a limit of iterations to the code
        if count > 100:
            break

    return path

def backward_shoot(k0, step):
    # Set conditions for initial shoot
    tol   = 1.0e-10
    c0    = consumption(k_gold) * 1.5
    error = np.abs(((k0 - k_star)**2 + (c0 - c_star)**2)**0.5)
    path      = np.zeros(shape=(1, 2))
    path[0,0] = k0
    path[0,1] = c0

    # Start loop (forward shoot)
    count = 0
    while 1: # Infinite loop with break conditions inside
        k1 = path[count, 0]
        c1 = path[count, 1]
        k2 = k1 + kdot(k1, c1)*0.01
        c2 = c1 + cdot(k1, c1)*0.01
        error = np.abs(((k2 - k_star)**2 + (c2 - c_star)**2)**0.5)
        path = np.append(path, [[k2, c2]], axis = 0)
        count = count + 1

        # Check if this is the stable-path
        if error < tol:
            break

        # Set up code break and reset the shoot
        if c2 < c_star:
            path = np.zeros(shape=(1, 2))
            c0 = c0 + step
            path[0,0] = k0
            path[0,1] = c0
            count = 0

        # Break code when k < k*
        if k2 < k_star:
            break

        # Put a limit of iterations to the code
        if count > 400:
            break

    return path

stable_L = forward_shoot(k00, 0.000001)
stable_R = backward_shoot(k_star*2.00, 0.000001)

"9|PLOT RAMSEY MODEL WITH STABLE PATH"
# Value of k such that c = 0
k_zero = (1/(n + g + delta))**(1/(1-alpha))
# Axis range
y_max = np.max(c_loci)*1.7
x_max = k_zero*1.2
v = [0, x_max, -0.1, y_max]

### BUILD PLOT AND POPULIATE WITH LOCI LINES
fig, ax = plt.subplots(figsize=(10, 7))
ax.plot(k_array, c_loci, color="K", alpha = 0.8)
ax.axvline(k_loci, color="K", alpha = 0.8)
### STABLE PATH
ax.plot(stable_L[:,0], stable_L[:,1], "b--", alpha = 0.7)
ax.plot(stable_R[:,0], stable_R[:,1], "b--", alpha = 0.7)
### AXIS
ax.axvline(0, color="k")                                     # y-axis
ax.axhline(0, color="k")                                     # x-axis
ax.yaxis.set_major_locator(plt.NullLocator())                # Hide ticks
ax.xaxis.set_major_locator(plt.NullLocator())                # Hide ticks
### TEXT
plt.text(x_max*0.98, -0.05      , r'$\hat{k}$', color = 'k') # x-axis label
plt.text(-0.15 , y_max*0.95     , r'$\hat{c}$', color = 'k') # y-axis label
plt.text(k_star*1.05, y_max*0.95, r'$\hat{c}=0$', color = "k")
plt.text(k_zero*0.95, 0.07      , r'$\hat{k}=0$', color = "k")
plt.text(k_star*1.05, -0.05     , r'$k^*$'      , color = 'k')
plt.text(-0.15, c_star          , r'$c^*$'      , color = 'k')
ax.axhline(c_star, 0, k_star/v[1], linestyle=":", color = "grey")
# SETTINGS
plt.axis(v)
plt.box(False)                                               # Hide axis
plt.show()
```


Loading of libraries and initialization

```python
import random

import cirq
from cirq.contrib.svg import SVGCircuit
import matplotlib.pyplot as plt
import numpy as np
import sympy
import tensorflow as tf
import tensorflow_quantum as tfq

if not tf.config.list_physical_devices('GPU'):
    print("Warning: GPU was not found, so simulations can be very slow")
```

```python
qubit = cirq.GridQubit(0, 0)

# Define some circuits.
circuit1 = cirq.Circuit(cirq.X(qubit))
circuit2 = cirq.Circuit(cirq.H(qubit))

# Convert to a tensor.
input_circuit_tensor = tfq.convert_to_tensor([circuit1, circuit2])

# Define a circuit that we want to append
y_circuit = cirq.Circuit(cirq.Y(qubit))

# Instantiate our layer
y_appender = tfq.layers.AddCircuit()

# Run our circuit tensor through the layer and save the output.
output_circuit_tensor = y_appender(input_circuit_tensor, append=y_circuit)

print(tfq.from_tensor(input_circuit_tensor))
print(tfq.from_tensor(output_circuit_tensor))
```

    [cirq.Circuit([
        cirq.Moment(
            cirq.X(cirq.GridQubit(0, 0)),
        ),
    ])
     cirq.Circuit([
        cirq.Moment(
            cirq.H(cirq.GridQubit(0, 0)),
        ),
    ])]
    [cirq.Circuit([
        cirq.Moment(
            cirq.X(cirq.GridQubit(0, 0)),
        ),
        cirq.Moment(
            cirq.Y(cirq.GridQubit(0, 0)),
        ),
    ])
     cirq.Circuit([
        cirq.Moment(
            cirq.H(cirq.GridQubit(0, 0)),
        ),
        cirq.Moment(
            cirq.Y(cirq.GridQubit(0, 0)),
        ),
    ])]