# VAR Model estimation with approximated likelihood ratios

Juan Pavez, Gilles Louppe, Kyle Cranmer, April 2017.

In this notebook we study the usability of the method [Approximating Likelihood Ratios with Calibrated Discriminative Classifiers](http://inspirehep.net/record/1377273) for times series parameter estimation.

In this case an Vector Autoregressive (VAR) model is used. The model is defines as
\begin{equation}
y_t = c + A_1 y_{t-1} + u_t,
\end{equation}
where $y_i \in \mathbb{R}^{Kx1}$, $A_1 \in \mathbb{R}^{KxK}$, $c \in \mathbb{R}^{Kx1}$ and $u_t$ is a white noise process in $K$-dimensions with covariance matrix $\Sigma_u$.
We approximate the coefficients matrix $A_1 = diag(\alpha_0, \alpha_1)$, with $\alpha_0 = 0.5$ and $\alpha_1 = 0.4$
for a 2D VAR(1) process.

We approximate the likelihood of the time series using a recurrent neural network $s(x_{1:T})$ as:

\begin{equation}
\frac{p_{X_{1:T}}(x_{1:T}\vert\theta_0)}{p_{X_{1:T}}(x_{1:T}\vert\theta_1)} = \frac{p_U(u=s(x_{1:T})\vert \theta_0)}{p_U(u=s(x_{1:T})\vert\theta_1)}.
\end{equation}

We also show how to produce the data and train the classifiers in parallel using python multiprocessing.

```python
%load_ext autoreload
%autoreload 2

%matplotlib inline
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib
import theano
import scipy.linalg as L
import pdb
import pickle

import os, sys, inspect

import carl

import multiprocessing

plt.set_cmap('viridis')

np.random.seed(1234)

sys.setrecursionlimit(10000)
```

    <matplotlib.figure.Figure at 0x6810bd0>

```python
import statsmodels.tsa.vector_ar.var_model as vector_ar
```

```python
p_value_ = 1
ndims_ = 2
nparams_ = 2
N_ = 20000
T_ = 50
alpha_value_0 = 0.5
alpha_value_1 = 0.4
sigma_2_value = [[1.,0.],[0.,1.]]
intercept=[0.,0.]
alpha_value_0_bkg = 0.3
alpha_value_1_bkg = 0.3
sigma_2_value_bkg = [[1.,0.],[0.,1.]]

target_position = (alpha_value_0, alpha_value_1)
bkg_position = (alpha_value_0_bkg, alpha_value_1_bkg)
bounds = [(0.3, 0.6), (0.3, 0.6)]
n_points = 5

As = np.linspace(bounds[0][0],bounds[0][1], n_points)
Bs = np.linspace(bounds[1][0],bounds[1][1], n_points)
AA, BB = np.meshgrid(As, Bs)
AABB = np.hstack((AA.reshape(-1, 1),
               BB.reshape(-1, 1)))

```

```python
class VAR:

    def sample(self, coefs, intercept, sigma_2, n_steps, rng=None):
        return vector_ar.util.varsim(coefs, intercept, sigma_2, steps=n_steps)

    def plot(self,steps=1000):
        self.VAR.plotsim(steps)

    def nll(self, X, alphas, intercept, sigma_2):
        """
        Compute the negative log likelihood for a VAR process,
        using formula from Lutkepohl (2005)
        """
        lags = 1
        trend = 'c'
        alpha_0 = alphas[0]
        alpha_1 = alphas[1]
        coefs = np.array([alpha_0,0.,0.,alpha_1]
                        ).reshape(p_value_,ndims_,ndims_)

        VAR = vector_ar.VAR(X)
        VAR.k_trend = k_trend = vector_ar.util.get_trendorder(trend)

        offset = 0
        y = VAR.y[offset:]

        z = vector_ar.util.get_var_endog(y, lags, trend,
                                         has_constant='skip')
        y_sample = y[lags:]
        intercept = intercept
        params = np.vstack((intercept, coefs.reshape((2,2))))
        #params = np.linalg.lstsq(z, y_sample)[0]
        resid = y_sample - np.dot(z, params)
        omega = np.array(sigma_2)

        lastterm = -np.trace(np.dot(np.dot(resid, L.inv(omega)),resid.T))

        varfit = vector_ar.VARResults(y, z, params, omega, lags, names=VAR.endog_names,
                    trend=trend, dates=VAR.data.dates, model=self)
        llf = -varfit.llf
        llf += 0.5*lastterm
        return -llf

    def rvs(self, n_samples, alpha_0, alpha_1, intercept, sigma_2, random_state=1234):
        rng = np.random.RandomState(random_state) if \
                isinstance(random_state,int) else random_state
        coefs = np.array([alpha_0,0.,0.,alpha_1]
                        ).reshape(p_value_,ndims_,ndims_)
        return self.sample(coefs=coefs,intercept=intercept,
                           sigma_2=sigma_2,
                           n_steps=n_samples,rng=rng)
```

```python
p0 = VAR()
p1 = VAR()
rng = np.random.RandomState(1234)

n_true = 1000
X_true = np.array([p0.rvs(T_, alpha_value_0, alpha_value_1, intercept,
                          sigma_2_value, random_state=np.random.randint(0,5000))
                   for i in range(n_true)])
```

```python
from itertools import product
bounds_values = [(np.linspace(bounds[0][0],bounds[0][1], num=5)),
                 (np.linspace(bounds[1][0],bounds[1][1], num=5))]
combinations = list(product(*bounds_values))
```

```python
def produce_data(value):
    return np.array([p0.rvs(T_, value[0], value[1], intercept, sigma_2_value,
                           random_state=np.random.randint(0,5000))
                   for i in range(N_//2)])
```

Producing data from the VAR model in parallel using multiprocessing map function.

```python
Xs = []
ys = []
num_cores = multiprocessing.cpu_count()
pool = multiprocessing.Pool(num_cores)
Xs = pool.map(produce_data, combinations)
pool.close()
X1_ = np.array([p1.rvs(T_, alpha_value_0_bkg, alpha_value_1_bkg,
                       intercept, sigma_2_value,
                       random_state=np.random.randint(0,5000))
               for i in range(N_//2)])
Xs = np.array(Xs)
```

```python
X = Xs[0][0]
```

Plotting a window from the VAR model.

```python
serie1 = X1_[0][:,0]
serie2 = Xs[0][0][:,0]
plt.figure(figsize=(20,10))
plt.subplot(211)
ts = pd.Series(serie1)
ts[0:100].plot()
plt.subplot(212)
ts = pd.Series(serie1)
ts.plot()
```

```python
from keras.models import Sequential
from keras.layers import SimpleRNN, Dense, GRU, LSTM, Dropout
from keras.optimizers import SGD, Adam
from keras.callbacks import ModelCheckpoint, Callback, LearningRateScheduler
from keras.wrappers.scikit_learn import KerasClassifier

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

from carl.ratios import ClassifierRatio
from carl.learning import CalibratedClassifierCV
from carl.learning import as_classifier
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import StratifiedShuffleSplit

```

    Using Theano backend.

Defining the RNN (in this case GRU) classifier to approximate the likelihood ratio.

```python
def make_model_join():
    model = Sequential()
    model.add(GRU(15, input_shape=(T_, ndims_,)))
    model.add(Dense(5, activation='relu'))
    model.add(Dense(1, activation='sigmoid'))
    sgd = Adam(clipnorm=40.)
    model.compile(loss="binary_crossentropy", optimizer='adam')
    return model
```

```python
def make_ratio(num):
    X_num = Xs[num]
    X_den = X1_
    X = np.vstack((X_num, X_den))
    y = np.zeros(len(X_num) + len(X_den), dtype=np.int)
    y[len(X_num):] = 1

    clf = KerasClassifier(make_model_join, nb_epoch=50, verbose=0)

    cv =  StratifiedShuffleSplit(n_iter=1, test_size=0.5, random_state=1)

    ratio = ClassifierRatio(
        base_estimator=CalibratedClassifierCV(clf, cv=cv, bins=20),
        random_state=0)
    ratio.fit(X, y)

    return ratio

```

Training the classifiers at each grid point in parallel.

```python
print('Start training')
num_cores = multiprocessing.cpu_count()
pool = multiprocessing.Pool(num_cores)

clf_ratios = []
n_sample_points = len(combinations)
points = list(range(n_sample_points))
for i, k in enumerate([points[n:n + num_cores] for n in range(0, n_sample_points, num_cores)]):
    print('Iteration {0}'.format(i))
    clf_ratios += pool.map(make_ratio, k)
#pool.close()
print('End training')
```

    Start training
    Iteration 0
    Iteration 1
    Iteration 2
    Iteration 3
    End training

```python
len(clf_ratios)
```

    25

```python
llr = []
zeros = []
distances = []

for i, theta in enumerate(combinations):
    print(i, theta)
    ratio = clf_ratios[i]
    ratios = ratio.predict(X_true, log=True)
    print ratios[np.isinf(ratios)].shape
    zeros.append(ratios[np.isinf(ratios)].shape[0])
    ratios = ratios[np.logical_and(np.isfinite(ratios),~np.isnan(ratios))]
    nllr = -np.mean(ratios)
    llr.append(nllr)
    print(llr[-1])
```

    (0, (0.29999999999999999, 0.29999999999999999))
    (0,)
    0.0054962467896
    (1, (0.29999999999999999, 0.375))
    (0,)
    -0.191834968388
    (2, (0.29999999999999999, 0.44999999999999996))
    (0,)
    -0.237012807811
    (3, (0.29999999999999999, 0.52499999999999991))
    (0,)
    0.124319079978
    (4, (0.29999999999999999, 0.59999999999999998))
    (0,)
    0.762950198252
    (5, (0.375, 0.29999999999999999))
    (0,)
    -0.395275614596
    (6, (0.375, 0.375))
    (0,)
    -0.770527017802
    (7, (0.375, 0.44999999999999996))
    (0,)
    -0.831173722626
    (8, (0.375, 0.52499999999999991))
    (0,)
    -0.565415735516
    (9, (0.375, 0.59999999999999998))
    (0,)
    0.000113244859618
    (10, (0.44999999999999996, 0.29999999999999999))
    (0,)
    -0.938755570645
    (11, (0.44999999999999996, 0.375))
    (0,)
    -1.24060498614
    (12, (0.44999999999999996, 0.44999999999999996))
    (0,)
    -1.24707220876
    (13, (0.44999999999999996, 0.52499999999999991))
    (0,)
    -0.963597355129
    (14, (0.44999999999999996, 0.59999999999999998))
    (0,)
    -0.302930145881
    (15, (0.52499999999999991, 0.29999999999999999))
    (0,)
    -1.02247543468
    (16, (0.52499999999999991, 0.375))
    (0,)
    -1.15986384384
    (17, (0.52499999999999991, 0.44999999999999996))
    (0,)
    -1.16579911981
    (18, (0.52499999999999991, 0.52499999999999991))
    (0,)
    -0.837766913851
    (19, (0.52499999999999991, 0.59999999999999998))
    (0,)
    -0.272463618997
    (20, (0.59999999999999998, 0.29999999999999999))
    (0,)
    -0.752922580642
    (21, (0.59999999999999998, 0.375))
    (0,)
    -0.931205046699
    (22, (0.59999999999999998, 0.44999999999999996))
    (0,)
    -0.887838324554
    (23, (0.59999999999999998, 0.52499999999999991))
    (0,)
    -0.590733282679
    (24, (0.59999999999999998, 0.59999999999999998))
    (0,)
    0.0197703047802

```python
llr = np.array(llr)
llr[np.isnan(llr)] = 0.
```

Computing exact likelihood at same points in order to compare results.

```python
# Computing exact likelihood
exact_contours = np.zeros(len(AABB))
combinations = product(As, Bs)
i = 0
for a in As:
    for b in Bs:
        exact_contours[i] = np.mean([p0.nll(X_true[k], [a,b], intercept, sigma_2_value)
                                     for k in range(100)])
        i += 1
```

```python
#llr, zeros = pickle.load(open('data/ar_results.dat', 'r'))
```

```python
%matplotlib inline
import matplotlib.pyplot as plt
from matplotlib.mlab import griddata
plt.rcParams["figure.figsize"] = (12, 10)

from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import ConstantKernel as C, Matern

# Define a class that forces representation of float to look a certain way
# This remove trailing zero so '1.0' becomes '1'
class nf(float):
    def __repr__(self):
        str = '%.1f' % (self.__float__(),)
        if str[-1] == '0':
            return '%.1f' % self.__float__()
        else:
            return '%.1f' % self.__float__()

# Label levels with specially formatted floats
if plt.rcParams["text.usetex"]:
    fmt = r'%r '
else:
    fmt = '%r '

```

```python
thetas = np.array([v for v in product(As, Bs)])

gp = GaussianProcessRegressor(alpha=0., normalize_y=True,
                              kernel=C(1.0) * Matern(1.0, length_scale_bounds="fixed"))
gp.fit(thetas, exact_contours)

xi = np.linspace(bounds[0][0], bounds[0][1], 50)
yi = np.linspace(bounds[1][0], bounds[1][1], 50)

xx, yy = np.meshgrid(xi, yi)
zz, std = gp.predict(np.c_[xx.ravel(), yy.ravel()], return_std=True)
zi = zz.reshape(xx.shape)

mle = np.unravel_index(zi.argmin(),zi.shape)
zi -= zi[mle]
zi *= 2.

cs = plt.contour(xi, yi, zi, [1.,4.,9.,16.,25.], linewidths=0.5, colors='w')
plt.clabel(cs, cs.levels, inline=1, fontsize=10, fmt = fmt)

cs = plt.contourf(xi, yi, zi, 30,
                  vmax=abs(zi).max(), vmin=0.0)

#plt.colorbar()
plt.scatter(thetas[:,0],thetas[:,1], marker='o', c='w', s=50, lw=0, zorder=10)
plt.scatter([xi[mle[1]]], [yi[mle[0]]], marker='o', c='r', s=50, lw=0, zorder=10)
plt.scatter([alpha_value_0],[alpha_value_1], marker='o', c='g', s=50, lw=0, zorder=10)

plt.xlim(bounds[0][0], bounds[0][1])
plt.ylim(bounds[1][0], bounds[1][1])
plt.tick_params(axis='both', which='major', labelsize=20)
plt.xlabel(r"$\alpha_0$", fontsize=25)
plt.ylabel(r"$\alpha_1$", fontsize=25)
plt.tight_layout()
#plt.savefig('plots/alpha_gp_exact.pdf')
plt.show()
```

```python
thetas = np.array([v for v in product(As, Bs)])

gp = GaussianProcessRegressor(alpha=0., normalize_y=True,
                              kernel=C(1.0) * Matern(1.0, length_scale_bounds="fixed"))
gp.fit(thetas, llr)

xi = np.linspace(bounds[0][0], bounds[0][1], 50)
yi = np.linspace(bounds[1][0], bounds[1][1], 50)

xx, yy = np.meshgrid(xi, yi)
zz, std = gp.predict(np.c_[xx.ravel(), yy.ravel()], return_std=True)
zi = zz.reshape(xx.shape)

mle = np.unravel_index(zi.argmin(),zi.shape)
zi -= zi[mle]
zi *= 2.

cs = plt.contour(xi, yi, zi, [1.,4.,9.,16.,25.], linewidths=0.5, colors='w')
plt.clabel(cs, cs.levels, inline=1, fontsize=10, fmt = fmt)

cs = plt.contourf(xi, yi, zi, 30,
                  vmax=abs(zi).max(), vmin=0.0)

#plt.colorbar()
plt.scatter(thetas[:,0],thetas[:,1], marker='o', c='w', s=50, lw=0, zorder=10)
plt.scatter([xi[mle[1]]], [yi[mle[0]]], marker='o', c='r', s=50, lw=0, zorder=10)
plt.scatter([alpha_value_0],[alpha_value_1], marker='o', c='g', s=50, lw=0, zorder=10)

plt.xlim(bounds[0][0], bounds[0][1])
plt.ylim(bounds[1][0], bounds[1][1])
plt.tick_params(axis='both', which='major', labelsize=20)
plt.xlabel(r"$\alpha_0$", fontsize=25)
plt.ylabel(r"$\alpha_1$", fontsize=25)
plt.tight_layout()
#plt.savefig('plots/alpha_gp_approx.pdf')
plt.show()
```

```python
fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(20,10))
ax = axes.flat[0]

gp = GaussianProcessRegressor(alpha=0.0, normalize_y=True,
                              kernel=C(1.0) * Matern(1.0, length_scale_bounds="fixed"))
gp.fit(thetas, llr)

xi = np.linspace(bounds[0][0], bounds[0][1], 500)
yi = np.linspace(bounds[1][0], bounds[1][1], 500)

xx, yy = np.meshgrid(xi, yi)
zz, std = gp.predict(np.c_[xx.ravel(), yy.ravel()], return_std=True)
zi = zz.reshape(xx.shape)

mle = np.unravel_index(zi.argmin(),zi.shape)
zi -= zi[mle]
zi *= 2.

cs = ax.contour(xi, yi, zi, [1.,4.,9.,16.,25.], linewidths=0.5, colors='w')
cs.levels = [nf(val) for val in cs.levels]
ax.clabel(cs, cs.levels, inline=1, fontsize=10, fmt = fmt)
cs = ax.contourf(xi, yi, zi, 30, cmap="viridis",
                  vmax=abs(zi).max(), vmin=0.0)

#plt.colorbar()
ax.scatter(thetas[:,0],thetas[:,1], marker='o', c='b', s=50, lw=0, zorder=10)
ax.scatter([xi[mle[1]]], [yi[mle[0]]], marker='o', c='r', s=50, lw=0, zorder=10)
ax.scatter([alpha_value_0],[alpha_value_1], marker='o', c='w', s=50, lw=0, zorder=10)

ax.set_xlim(bounds[0][0], bounds[0][1])
ax.set_ylim(bounds[1][0], bounds[1][1])
ax.set_xlabel(r"$\alpha_0$", size=16)
ax.set_ylabel(r"$\alpha_1$", size=16)
ax.set_title("Approx. -2lnL (VAR(2,1))")

ax = axes.flat[1]

gp = GaussianProcessRegressor(alpha=0.0, normalize_y=True,
                              kernel=C(1.0) * Matern(1.0, length_scale_bounds="fixed"))
gp.fit(thetas, exact_contours)

xi = np.linspace(bounds[0][0], bounds[0][1], 500)
yi = np.linspace(bounds[1][0], bounds[1][1], 500)

xx, yy = np.meshgrid(xi, yi)
zz, std = gp.predict(np.c_[xx.ravel(), yy.ravel()], return_std=True)
zi = zz.reshape(xx.shape)

mle = np.unravel_index(zi.argmin(),zi.shape)
zi -= zi[mle]
zi *= 2.

cs2 = ax.contour(xi, yi, zi, [1.,4.,9.,16.,25.], linewidths=0.5, colors='w')
cs2.levels = [nf(val) for val in cs2.levels]
ax.clabel(cs2, cs2.levels, inline=1, fontsize=10, fmt = fmt)
cs2 = ax.contourf(xi, yi, zi, 30, cmap="viridis",
                  vmax=abs(zi).max(), vmin=0.0)

fig.subplots_adjust(right=0.8)
cbar_ax = fig.add_axes([0.81, 0.12, 0.03, 0.775])
fig.colorbar(cs2, cax=cbar_ax)

ax.scatter(thetas[:,0],thetas[:,1], marker='o', c='b', s=50, lw=0, zorder=10)
smle = ax.scatter([xi[mle[1]]], [yi[mle[0]]], marker='o', c='r', s=50, lw=0, zorder=10)
sobs = ax.scatter([alpha_value_0],[alpha_value_1], marker='o', c='w', s=50, lw=0, zorder=10)

lines = [smle, sobs]
labels = ['MLE', 'Observed']

ax.legend(lines, labels, frameon=False, prop={'size':12}, scatterpoints=1)

ax.set_xlim(bounds[0][0], bounds[0][1])
ax.set_ylim(bounds[1][0], bounds[1][1])
ax.set_xlabel(r"$\alpha_0$", size=16)
ax.set_ylabel(r"$\alpha_1$", size=16)
ax.set_title("Exact. -2lnL (VAR(2,1))")

plt.show()
#plt.savefig('ar_2d_approx.pdf')
```

Plot grid of score distributions to check calibration

```python
from carl.ratios.utils import plot_scores
```

```python
combinations = list(combinations)
```

```python
plot_scores(clf_ratios[0:9],
            num_den_labels=[(l,bkg_position) for l in combinations[0:9]])
```

```python

```

examples/Likelihood ratio of VAR time series.ipynb


#  Dynamic systems with limited memory

In this short post we pick up on our previous conversation on [one dimensional convolutions](https://blog.dgsix.com/posts/one_dimensional_convolutions/One_dimensional_convolutions.html), discussing their generalization called *dynamic systems with limited memory*.

You can skip around this document to particular subsections via the hyperlinks below.

-  [Dynamic systems with limited memory](#dynamic-systems-intro)
-  [A range of examples](#examples)

```python
## This code cell will not be shown in the HTML version of this notebook
# imports from custom library for animations
import sys
sys.path.append('../../')
from demo_libraries import dyamic_systems_limited_memory_library as dynalib

# import standard libs
import numpy as np
import pandas as pd
from IPython.display import clear_output

# path to data
datapath = '../../datasets/plain_timeseries/'

# This is needed to compensate for matplotlib notebook's tendancy to blow up images when plotted inline
%matplotlib notebook
from matplotlib import rcParams
rcParams['figure.autolayout'] = True

%load_ext autoreload
%autoreload 2
```

## Dynamic systems with limited memory

A one dimensional convolution consists of a simple linear combination of $D$ consecutive elements of an ordered sequence

\begin{equation}
h_{p} = w_1x_{p-1} + w_2x_{p-2} + \cdots + w_{D}x_{p-D}.
\end{equation}

More generally, instead of a linear combination we can use any general function $f\left(\cdot\right)$ one desires asas

\begin{equation}
h_{p} = f\left(x_{p-1},x_{p-2},\cdots,x_{p-D}\right).
\end{equation}

Such a functional relationship is generally referred to as a *dynamic system with limited memory*.  Breaking this jargon-phrase apart, the "dynamic system" part simply refers to the fact that the function $f\left(\cdot\right)$ takes in elements of a naturally ordered sequence (like a time series).  The "limited memory" part simply refers to the fact that each output $h_p$ is *only* dependent on the previous $D$ elements of an input sequence $(x_{p-1},x_{p-2},\cdots,x_{p-D}$ (that is, it only has "memory" of these inputs).

## A range of examples

Dynamic systems with limited memory - where $f$ is chosen based on particular specifications - are used throughout the sciences, economics, and engineering.  Indeed we have already seen several specific examples in the form of the [moving average](https://blog.dgsix.com/posts/one_dimensional_convolutions/One_dimensional_convolutions.html) and various [convolutions](https://blog.dgsix.com/posts/one_dimensional_convolutions/One_dimensional_convolutions.html#generalizing-moving-average).  Below we briefly illustrate a few additional examples, including those that output is *vector-valued*, that is systems that take the form

\begin{equation}
\mathbf{h}_{p} = f\left(x_{p-1},x_{p-2},\cdots,x_{p-D}\right)
\end{equation}

where $\mathbf{h}$ is some $M$ length vector.  These often arise as some kind of *histogram* that summarizes the input values $x_{p-1},x_{p-2},\cdots,x_{p-D}$ somehow, when using classical signal processing tools like the Discrete Cosine Transform, and when using a multi-layer perceptron.

#### Example 1:   A maximum intake

A simple example (which we have not seen previously) is the *maximum* system defined as

\begin{equation}
h_{p} = \text{maximum}\left(x_{p-1},x_{p-2},\cdots,x_{p-D}\right)
\end{equation}

which simply outputs the maximum value in a $D$ length window of the input.  This simple system can be implemented in `Python` as shown below.  This `model` function takes in the input sequence `x` and a window-length $D$ and returns the desired transformation $h$.

```python
# a simple implementation of the maximum system
def model(x,D):
    # set initial condition
    h = [v for v in x[:D]]

    # loop over remainder of sequence and take maximum of each D length window
    for p in range(len(x) - D):
        # compute maximum of current window and store
        h_p = np.max(x[p:p+D])
        h.append(h_p)
    return np.array(h)
```

Below we show an animation of the maximum system computed over an input sequence shown in black.  The window (whose size here is $D = 5$) is shown moving from left to right, strattled on both sides by vertical blue guide-lines.  The resulting maximum computation is then shown sequentially in orange.

```python
## This code cell will not be shown in the HTML version of this notebook
# generate test series
x = []
for t in range(50):
    x_t = 0.1*t*np.sin(0.5*t)
    x.append(x_t)
x = np.array(x)

# produce moving average time series
D = 5
y = model(x,D)

# run animator
demo = dynalib.dynamic_systems_animators.Visualizer()
demo.animate_system(x,y,D,savepath='videos/animation_1.mp4',fps=5)
clear_output()

# import video into notebook
from IPython.display import HTML
HTML("""

""")
```

Below we show the result of applying the maximum system to this same input series, adjusting the window size $D$ from $D=1$ to $D=19$.  As the value of $D$ increases the corresponding maximum sequence starts to look more and more like a bounding [envelope](https://en.wikipedia.org/wiki/Envelope_(waves)).

```python
## This code cell will not be shown in the HTML version of this notebook
# run animator
demo = dynalib.dynamic_systems_animators.Visualizer()
params = np.arange(1,20)
func = lambda x,D,f=maximum: model(x,D,f)
demo.animate_system_range(x,func,params,savepath='videos/animation_2.mp4',fps=4)
clear_output()

from IPython.display import HTML
HTML("""

""")
```

#### Example 2:   Limited memory normalized histogram

Perhaps the simplest example of a vector-valued dynamic system is a running *histogram*.  That is, a function that takes in $D$ sequential elements of an ordered input sequence and outputs a histogram summarizing the content of this window of values.  A range basic choices need be made when defining a histogram - including the number of bins to use, the value of these bins, lower and upper bounds for the histogram, etc.,  Below we define a simple `histogram_function` in `Python` that takes in the snippet of input values $x_{p-1},x_{p-2},\cdots,x_{p-D}$ and provides a few optional input arguments for controlling these basic histogram parameters.  This histogram is normalized near the end so that it can be treated like a [discrete probability distribution](https://jermwatt.github.io/machine_learning_refined/notes/7_Linear_multiclass_classification/7_5_Categorical.html), and returned along with the bins used.  This is an example of a function $f\left(\cdot\right)$ in the general dynamic system shown in Equation (4).

```python
## This code cell will not be shown in the HTML version of this notebook
def histogram_function(x,**kwargs):
    # define range of histogram and resolution
    lower_bound = -1
    upper_bound = 1
    resolution = 20
    if 'lower_bound' in kwargs:
        lower_bound = kwargs['lower_bound']
    if 'upper_bound' in kwargs:
        upper_bound = kwargs['upper_bound']
    if 'resolution' in kwargs:
        resolution = kwargs['resolution']

    # clip input series based on lower bound / upper bound
    x = np.array(x)
    ind = np.argwhere(x < lower_bound)
    if len(ind) > 0:
        x[ind] = lower_bound
    ind = np.argwhere(x > upper_bound)
    if len(ind) > 0:
        x[ind] = upper_bound

    # make base histogram
    h,bins = np.histogram(x, bins=np.linspace(lower_bound,upper_bound,resolution+1), density=False)
    h = np.array([v/len(x) for v in h])
    return h,bins
```

Below we animate the result of applying this dynamic system on a toy input sequence, shown in black in the top panel of the figure. In each frame of the animation the current window (here $D = 10$) is once again illustrated by two vertical blue guide-lines, and here the points inside each window are highlighted orange.  In the middle panel of the Figure we show the normalized histogram of values in each window in magenta, and in the bottom we illustrate all of the histograms generated up to the current frame via a *heatmap*.  Here each histogram is rotated $90$ degrees, and we view them from above.  The darker orange colors correspond to higher values in each histogram.

```python
## This code cell will not be shown in the HTML version of this notebook
# generate test series
x = []
for t in range(50):
    x_t = np.sin(t) + t*0.2
    x.append(x_t)

# produce moving average time series
D = 10
func = lambda x: histogram_function(x,lower_bound=0,upper_bound=10,resolution=100)

# general linear filter function - which takes in dynamic systems function f
# for the sake of generality in the animation process
def vector_model(x,D,f):
    # initialize
    h = []

    # loop over and create histograms
    for p in range(len(x) - D + 2):
        # make next element
        x_p = x[p:p+D]
        h_p,bins = f(x_p)
        y.append(h_p)
    return y

# run animator
demo = dynalib.dynamic_systems_animators.Visualizer()
demo.animate_vector_histogram(x,D,vector_model,func,savepath='videos/animation_4.mp4',fps=5)
clear_output()
```

```python
## This code cell will not be shown in the HTML version of this notebook
from IPython.display import HTML
HTML("""

""")
```

#### Example 3:   The Discrete Cosine Transform and spectrograms

The *Discrete Cosine Transform* (or DCT for short) is a common dynamic system employed with audio and more generally electrical signals for compression and other various processing tasks.  The gist of a DCT is illustrated in the image below (taken with permission [from this website](https://jermwatt.github.io/machine_learning_refined/notes/9_Feature_engineer_select/9_2_Histogram.html)) for a snippet of signal below.  Here the signal (left panel) is approximated by a weighted sum of cosine waves of increasing frequency (middle panel), with the resulting transformation, also called the signals *frequency* or *spectrum*, consisting of the magnitude of the coefficients of this transformation.

A formulae for this DCT transformation can be found e.g., on [wikipedia](https://en.wikipedia.org/wiki/Discrete_cosine_transform), and can be used as the function $f\left(\cdot\right)$ in Equation (4).  The DCT of a snippet can be computed in `Python` using [`scipy`](https://www.scipy.org/), and imported via the statement below

```
from scipy.fftpack import dct
```

Below we animate the process of generating DCT transformations for a moving window over an input signal.  At each frame the windowed signal (here $D = 128$) is highlighted in magenta.  As with the histogram example given above, here we illustrate each DCT contiguously in the bottom panel as a *heatmap* - viewing the histogram from above vertically.  Here blue denotes small values and red large ones.

 This view of the running DCT transformations is very similar to the concept of a [spectrogram](https://en.wikipedia.org/wiki/Spectrogram).  Also note that in machine learning such a dynamic system is often referred to as a [histogram-based feature transformation](https://jermwatt.github.io/machine_learning_refined/notes/9_Feature_engineer_select/9_2_Histogram.html).

```python
## This code cell will not be shown in the HTML version of this notebook
# general linear filter function
def vector_model(x,D,f):
    # filter input signal
    y = []
    for p in range(len(x) - D + 1):
        # make next element
        x_p = x[p:p+D]
        h = np.abs(f(x_p))
        y.append(h)
    return np.array(y).T
```

```python
## This code cell will not be shown in the HTML version of this notebook
# load in data
csvname =  'data/dog_audio.csv'
data = pd.read_csv(csvname)
x = np.array(data).flatten()

# subsample data
x = x[::40]
x = x[100:]

# normalize
x_mean = np.mean(x)
x_std = np.std(x)
x = (x - x_mean)/x_std

# produce moving average time series
from scipy.fftpack import dct
D = 80
func = dct

# run animator
demo = dynalib.dynamic_systems_animators.Visualizer()
demo.animate_dct_spectrogram(x,D,vector_model,func,savepath='videos/animation_5.mp4',fps=80)
clear_output()
```

```python
## This code cell will not be shown in the HTML version of this notebook
# load animation into notebook
from IPython.display import HTML
HTML("""

""")
```

#### Example 4:   A multi-layer perceptron

A [multi-layer perceptron](https://jermwatt.github.io/machine_learning_refined/notes/13_Multilayer_perceptrons/13_2_Multi_layer_perceptrons.html) can also be used as a function $f\left(\cdot\right)$ in Equation (4).  In the top panel of the image below we show the same input signal used in the previous Example.  In the three panels below it we show the output of a random unit from the first, second, and third hidden layer of a three layer network with ReLU activations, respectively.  This network takes in precisely the windows (of size $D = 128$) of the signal shown in the previous DCT animation.

```python
## This code cell will not be shown in the HTML version of this notebook
# create initial weights for arbitrary feedforward network
def initialize_network_weights(layer_sizes, scale):
    # container for entire weight tensor
    weights = []

    # loop over desired layer sizes and create appropriately sized initial
    # weight matrix for each layer
    for k in range(len(layer_sizes)-1):
        # get layer sizes for current weight matrix
        U_k = layer_sizes[k]
        U_k_plus_1 = layer_sizes[k+1]

        # make weight matrix
        weight = scale*np.random.randn(U_k+1,U_k_plus_1)
        weights.append(weight)

    # re-express weights so that w_init[0] = omega_inner contains all
    # internal weight matrices, and w_init = w contains weights of
    # final linear combination in predict function
    w_init = [weights[:-1],weights[-1]]

    return w_init
```

```python
## This code cell will not be shown in the HTML version of this notebook
# a feature_transforms function for computing
# U_L L layer perceptron units efficiently
def activation(s):
    return np.maximum(0,s)

def feature_transforms(a, w):
    # loop through each layer matrix
    hidden_outputs = []
    for W in w:
        # compute inner product with current layer weights
        a = W[0] + np.dot(a.T, W[1:])

        # output of layer activation
        a = activation(a).T

        # store hidden outputs
        hidden_outputs.append(a)
    return a,hidden_outputs

# an implementation of our model employing a nonlinear feature transformation
def model(x,w):
    # feature transformation
    f,hidden_outputs = feature_transforms(x,w[0])

    # compute linear combination and return
    a = w[1][0] + np.dot(f.T,w[1][1:])
    return a.T

# window input series
def window_series(x,D):
    xwins = []
    for d in range(len(x) - D + 1):
        x_p = x[d:d+D]
        xwins.append(x_p)
    xwins = np.array(xwins)
    return xwins.T
```

```python
## This code cell will not be shown in the HTML version of this notebook
# load in data
csvname =  'data/dog_audio.csv'
data = pd.read_csv(csvname)
x = np.array(data).flatten()

# subsample data
x = x[::40]
x = x[100:]

# normalize
x_mean = np.mean(x)
x_std = np.std(x)
x = (x - x_mean)/x_std

# window series
D = 128
xwins = window_series(x,D)

layer_sizes = [D,10,10,10,1]
w = initialize_network_weights(layer_sizes,scale=0.5)
a,hidden_outputs = feature_transforms(xwins,w[0])
```

```python
## This code cell will not be shown in the HTML version of this notebook
import matplotlib.pyplot as plt
from matplotlib import gridspec

def plot_it(x,hidden_outputs):
    # produce figure
    fig = plt.figure(figsize = (9,8))
    gs = gridspec.GridSpec(len(hidden_outputs)+1, 1)
    ax = plt.subplot(gs[0]);
    layer_colors = ['gold', 'deepskyblue', 'turquoise']

    # plot
    ax.plot(x,c='k')

    # plot hidden layers
    for layer in range(len(hidden_outputs)):
        ax1 = plt.subplot(gs[layer+1]);

        h = hidden_outputs[layer]
        c = layer_colors[layer]

        unit = np.random.permutation(h.shape[0])[0]
        u = 'k'
        h_u = h[unit]
        ax1.plot(h_u,color=u,linewidth=4)
        ax1.plot(h_u,color=c,linewidth=2)
    plt.show()

plot_it(x,hidden_outputs)
```


# Sensibilidad OIS

**Definición:** una curva cupón cero es una colección ordenada de plazos y tasas de interés donde cada una de las tasas es una  tasa adecuada para traer a valor presente un flujo de caja al plazo que corresponde a la tasa.

Consideremos un instrumento financiero cuyo valor depende de una curva cupón cero. Por ejemplo, un swap de tasa de interés. Matemáticamente, esta dependencia se expresa como:

$$
V = V\left(z_1,\ldots,z_n;\alpha\right)
$$

donde $z_1,\ldots,z_N$ son los valores de las tasas de la curva y $\alpha$ representa un vector de parámetros adicionales (como la tasa de cupón en un bono o un tipo de cambio en un swap de monedas).

**Definición:** la cantidad $\Delta_i$ de $V$ respecto a $z_i$ se define como:

$$
\Delta_i=\frac{\partial V}{\partial z_i}
$$

**Definición:** la sensibilidad de $V$ respecto a un cambio $\delta z_i$ (positivo o negativo) en la tasa $z_i$ se define como:

$$
S\left(V,z_i,\delta_i\right)=V\left(z_1,...,z_i+\delta_i,\ldots,z_n\right)-V\left(z_1,\ldots,z_i,\ldots,z_n\right)
$$

Cuando se busca conocer la sensibilidad a un cambio $\delta z_i$  en la tasa $z_i$, donde $\delta z_i$ representa una magnitud sin signo, se utiliza la siguiente definición:

**Definición:** la sensibilidad de $V$ respecto a un cambio de magnitud $\delta z_i$ en la tasa $z_i$ se define como:

$$
\overline{S}\left(V,z_i,\delta_i\right)=\frac{V\left(z_1,\ldots ,z_i+\delta_i,\ldots,z_n\right)-V\left(z_1,\ldots,z_i-\delta_i,...,z_n\right)}{2}
$$

En ocasiones, cuando se busca conocer la sensibilidad a un movimiento de magnitud pequeña, resulta conveniente realizar la siguiente aproximación:

$$
\overline{S}\left(V,z_i,\delta_i\right)\approx\Delta_i \cdot\delta_i
$$

este es el caso, cuando se busca conocer esta sensibilidad para un elevado número de operaciones. Es importante considerar que se puede asegurar que esta aproximación es válida cuando la función $V$ satisface los requerimientos de regularidad necesarios para aplicar el [Teorema de Taylor](https://en.wikipedia.org/wiki/Taylor%27s_theorem).

## Configuración

### Librerías

```python
from finrisk import QC_Financial_3 as Qcf
import modules.auxiliary as aux
from functools import partial
from enum import Enum
import pandas as pd
```

### Variables Globales

```python
class BusCal(Enum):
    NY = 1
    SCL = 2
```

```python
def get_cal(code: BusCal) -> Qcf.BusinessCalendar:
    """
    """
    if code == BusCal.NY:
        cal = Qcf.BusinessCalendar(Qcf.QCDate(1, 1, 2020), 20)
        for agno in range(2020, 2071):
            f = Qcf.QCDate(12, 10, agno)
            if f.week_day() == Qcf.WeekDay.SAT:
                cal.add_holiday(Qcf.QCDate(14, 10, agno))
            elif f.week_day() == Qcf.WeekDay.SUN:
                cal.add_holiday(Qcf.QCDate(13, 10, agno))
            elif f.week_day() == Qcf.WeekDay.MON:
                cal.add_holiday(Qcf.QCDate(12, 10, agno))
            elif f.week_day() == Qcf.WeekDay.TUE:
                cal.add_holiday(Qcf.QCDate(11, 10, agno))
            elif f.week_day() == Qcf.WeekDay.WED:
                cal.add_holiday(Qcf.QCDate(10, 10, agno))
            elif f.week_day() == Qcf.WeekDay.THU:
                cal.add_holiday(Qcf.QCDate(9, 10, agno))
            else:
                cal.add_holiday(Qcf.QCDate(8, 10, agno))
        cal.add_holiday(Qcf.QCDate(15, 2, 2021))

    return cal
```

```python
get_cal(BusCal.NY)
```

    <finrisk.QC_Financial_3.BusinessCalendar at 0x7f63b7087c38>

```python
frmt = {
    'tasa': '{:.6%}',
    'df': '{:.6%}',
    'valor_tasa': '{:.4%}',
    'spread': '{:.4%}',
    'nominal': '{:,.0f}',
    'interes': '{:,.0f}',
    'amortizacion': '{:,.0f}',
    'flujo': '{:,.4f}',
}
```

```python
class TypeOis(Enum):
    SOFR = 1
    ICP = 2
```

```python
type_ois_template = {
    TypeOis.SOFR: {
        'currency': Qcf.QCUSD(),
        'periodicity': Qcf.Tenor('1Y'),
        'stub_period': Qcf.StubPeriod.SHORTFRONT,
        'settlement_lag': 0,
        'calendar': BusCal.NY,
        'bus_adj_rule': Qcf.BusyAdjRules.MODFOLLOW,
        'amort_is_cashflow': True,
        'fixed_rate': Qcf.QCInterestRate(0.0, Qcf.QCAct360(), Qcf.QCLinearWf()),
    },
    TypeOis.ICP: {
        'currency': Qcf.QCCLP(),
        'periodicity': Qcf.Tenor('6M'),
        'stub_period': Qcf.StubPeriod.SHORTFRONT,
        'settlement_lag': 0,
        'calendar': BusCal.SCL,
        'bus_adj_rule': Qcf.BusyAdjRules.MODFOLLOW,
        'amort_is_cashflow': True,
        'fixed_rate': Qcf.QCInterestRate(0.0, Qcf.QCAct360(), Qcf.QCLinearWf()),
    }
}
```

## Construye Curva Cero Cupón

Se importa la data de la curva cupón cero que fue construida en el notebook 5.

```python
df_curva = pd.read_excel('data/20201012_built_sofr_zero.xlsx')
```

```python
def get_curve_from_dataframe(yf: Qcf.QCYearFraction, wf: Qcf.QCWealthFactor,
                             df_curva: pd.DataFrame) -> Qcf.ZeroCouponCurve:
    """
    Retorna un objeto Qcf.ZeroCouponCurve. Esta función requiere que `df_curva` tenga una columna
    de nombre 'plazo' y una columna de nombre 'tasa'. Se usa interpolación lineal en la curva que
    se retorna.
    """
    plazos = Qcf.long_vec()
    tasas = Qcf.double_vec()
    for row in df_curva.itertuples():
        plazos.append(row.plazo)
        tasas.append(row.tasa)
    curva = Qcf.QCCurve(plazos, tasas)
    curva = Qcf.QCLinearInterpolator(curva)
    tipo_tasa = Qcf.QCInterestRate(0.0, yf, wf)
    curva = Qcf.ZeroCouponCurve(curva, tipo_tasa)
    return curva
```

```python
df_curva.head().style.format(frmt)
```

        	        plazo	        tasa	        df

                        0
                        1
                        0.081111%
                        99.999778%

                        1
                        7
                        0.084051%
                        99.998388%

                        2
                        14
                        0.077967%
                        99.997010%

                        3
                        21
                        0.077358%
                        99.995549%

                        4
                        33
                        0.078067%
                        99.992942%

```python
zcc = get_curve_from_dataframe(Qcf.QCAct365(),Qcf.QCCompoundWf(), df_curva)
```

    <finrisk.QC_Financial_3.ZeroCouponCurve at 0x7f63b5e4fbc8>

Algunos métodos del objeto`zcc`.

```python
plazo = 900
print(f"Tasa a {plazo} días es igual a {zcc.get_rate_at(plazo):.4%}")
print(f"Factor de descuento a {plazo} días es igual a {zcc.get_discount_factor_at(plazo):.6%}")
```

    Tasa a 900 días es igual a 0.0652%
    Factor de descuento a 900 días es igual a 99.839384%

## Valorización

```python
def get_ois_using_template(template,
                           type_ois: TypeOis, rp: Qcf.RecPay, notional: float, start_date: Qcf.QCDate,
                           tenor: Qcf.Tenor, fixed_rate_value: float, spread: float, gearing: float):
    """
    """
    template_dict = template[type_ois]
    meses = tenor.get_years() * 12 + tenor.get_months()
    end_date = start_date.add_months(meses)
    template_dict['fixed_rate'].set_value(fixed_rate_value)
    es_bono = False

    # Construye la pata fija
    fixed_rate_leg = Qcf.LegFactory.build_bullet_fixed_rate_leg(
        rp,
        start_date,
        end_date,
        template_dict['bus_adj_rule'],
        template_dict['periodicity'],
        template_dict['stub_period'],
        get_cal(template_dict['calendar']),
        template_dict['settlement_lag'],
        notional,
        template_dict['amort_is_cashflow'],
        template_dict['fixed_rate'],
        template_dict['currency'],
        es_bono)

    # Construye la pata ois
    rp = Qcf.RecPay.PAY if rp == Qcf.RecPay.RECEIVE else Qcf.RecPay.RECEIVE
    icp_clp_leg = Qcf.LegFactory.build_bullet_icp_clp2_leg(
        rp,
        start_date,
        end_date,
        template_dict['bus_adj_rule'],
        template_dict['periodicity'],
        template_dict['stub_period'],
        get_cal(template_dict['calendar']),
        template_dict['settlement_lag'],
        notional,
        template_dict['amort_is_cashflow'],
        spread,
        gearing,
        True
    )

    for i in range(icp_clp_leg.size()):
        cshflw = icp_clp_leg.get_cashflow_at(i)
        cshflw.set_start_date_icp(1.0)
        cshflw.set_end_date_icp(1.0)

    return (fixed_rate_leg, icp_clp_leg)
```

### Operación Ejemplo

```python
op = get_ois_using_template(
    type_ois_template,
    TypeOis.SOFR,
    Qcf.RecPay.RECEIVE,
    10000000,
    Qcf.QCDate(14, 10, 2020),
    Qcf.Tenor('2Y'),
    .01,
    0.0,
    1.0
)
op
```

    (<finrisk.QC_Financial_3.Leg at 0x7f63b5e8b578>,
     <finrisk.QC_Financial_3.Leg at 0x7f63b5e8b1d0>)

#### Digresión: `functools.partial`

Supongamos que estoy en una situación en la que sólo quiero construir OIS de *SOFR*. Me gustaría no tener que repetir los argumentos `type_ois_template` y `TypeOis.SOFR` cada vez que llamo la función `get_ois_using_template`. Puedo definir una nueva función de la siguiente forma:

```python
get_ois_sofr = partial(get_ois_using_template, type_ois_template, TypeOis.SOFR)
```

Con esta nueva función, `get_ois_sofr`, ahora puedo construir la operación `op` de la siguiente forma:

```python
op = get_ois_sofr(
    Qcf.RecPay.RECEIVE,
    10000000,
    Qcf.QCDate(14, 10, 2020),
    Qcf.Tenor('2Y'),
    .01,
    0.0,
    1.0
)
```

#### Continuamos (fin digresión ...)

```python
aux.show_leg(op[0], 'FixedRateCashflow', '').style.format(frmt)
```

        	        fecha_inicial	        fecha_final	        fecha_pago	        nominal	        amortizacion	        interes	        amort_es_flujo	        flujo	        moneda	        valor_tasa	        tipo_tasa

                        0
                        2020-10-14
                        2021-10-14
                        2021-10-14
                        10,000,000
                        0
                        101,389
                        True
                        101,388.8889
                        USD
                        1.0000%
                        LinAct360

                        1
                        2021-10-14
                        2022-10-14
                        2022-10-14
                        10,000,000
                        10,000,000
                        101,389
                        True
                        10,101,388.8889
                        USD
                        1.0000%
                        LinAct360

```python
aux.show_leg(op[1], 'IcpClpCashflow', '').style.format(frmt)
```

        	        fecha_inicial	        fecha_final	        fecha_pago	        nominal	        amortizacion	        amort_es_flujo	        flujo	        moneda	        icp_inicial	        icp_final	        valor_tasa	        interes	        spread	        gearing	        tipo_tasa

                        0
                        2020-10-14
                        2021-10-14
                        2021-10-14
                        -10,000,000
                        0
                        True
                        0.0000
                        CLP
                        1.000000
                        1.000000
                        0.0000%
                        -0
                        0.0000%
                        1.000000
                        LinAct360

                        1
                        2021-10-14
                        2022-10-14
                        2022-10-14
                        -10,000,000
                        -10,000,000
                        True
                        -10,000,000.0000
                        CLP
                        1.000000
                        1.000000
                        0.0000%
                        -0
                        0.0000%
                        1.000000
                        LinAct360

#### Valor Presente Pata Fija

```python
vp = Qcf.PresentValue()
```

```python
fecha_val = Qcf.QCDate(14, 10, 2020)
```

```python
vp_fija = vp.pv(fecha_val, op[0], zcc)
print(f'El valor presente de la pata fija es: USD {vp_fija:,.8f}')
```

    El valor presente de la pata fija es: USD 10,191,249.70919298

**Ejercicio:** Replique el valor de la pata fija utilizando los flujos y los factores de descuento que obtiene de la curva `zcc`.

```python
# Se dan de alta las fechas finales de ambos cupones
fecha1 = Qcf.QCDate(14, 10, 2021)
fecha2 = Qcf.QCDate(14, 10, 2022)

# O de forma más automática:
fecha1 = op[0].get_cashflow_at(0).get_settlement_date()
fecha2 = op[0].get_cashflow_at(1).get_settlement_date()

# Se calcula el número de días entre la fecha de valorización (fecha_val) y las fechas
# finales de ambos cupones.
plazo1 = fecha_val.day_diff(fecha1)
plazo2 = fecha_val.day_diff(fecha2)

# Utilizando la curva zcc se calculan los df a esos plazos
df1 = zcc.get_discount_factor_at(plazo1)
df2 = zcc.get_discount_factor_at(plazo2)

# Se obtienen los flujos totales (interés y amortización) de ambos cupones
flujo1 = op[0].get_cashflow_at(0).amount()
flujo2 = op[0].get_cashflow_at(1).amount()

# Finalmente, se calcula el valor presente como el producto (escalar) entre los df y los flujos.
check_vp = df1 * flujo1 + df2 * flujo2

# Se muestra el resultado.
print(f'El valor presente a mano es: {check_vp:,.8f}')
```

    El valor presente a mano es: 10,191,249.70919298

#### Valor Presente Pata Flotante

```python
fwd = Qcf.ForwardRates()
```

```python
print(f'VP: {vp.pv(fecha_val, op[1], zcc):,.2f}')
```

    VP: -9,988,658.09

```python
print(f'{df2 * 10000000:,.2f}')
```

    9,988,658.09

```python
fwd.set_rates_icp_clp_leg(fecha_val, 1.0, op[1], zcc)
```

```python
aux.show_leg(op[1], 'IcpClpCashflow', '').style.format(frmt)
```

        	        fecha_inicial	        fecha_final	        fecha_pago	        nominal	        amortizacion	        amort_es_flujo	        flujo	        moneda	        icp_inicial	        icp_final	        valor_tasa	        interes	        spread	        gearing	        tipo_tasa

                        0
                        2020-10-14
                        2021-10-14
                        2021-10-14
                        -10,000,000
                        0
                        True
                        -7,023.7778
                        CLP
                        1.000000
                        1.000702
                        0.0700%
                        -7,097
                        0.0000%
                        1.000000
                        LinAct360

                        1
                        2021-10-14
                        2022-10-14
                        2022-10-14
                        -10,000,000
                        -10,000,000
                        True
                        -10,004,327.9717
                        CLP
                        1.000702
                        1.001135
                        0.0400%
                        -4,056
                        0.0000%
                        1.000000
                        LinAct360

```python
vp_flot = vp.pv(fecha_val, op[1], zcc)
print(f'El valor presente de la pata flotante es: USD {vp_flot:,.2f}')
```

    El valor presente de la pata flotante es: USD -10,000,000.00

¿Porqué sabíamos que tenía que dar 10,000,000.00? Hint: la respuesta está en la construcción de la curva cero OIS.

## Sensibilidad

En `QC_Financial_3` al calcular el valor presente, también se calculan las derivadas del valor presente respecto a cada uno de los vértices de la curva.

### Pata Fija

f(g(x)) = y --> dy/dx = f'(g(x)) . g'(x)

```python
vp.pv(fecha_val, op[0], zcc)
der = vp.get_derivatives()
```

Con esas derivadas, se puede calcular la sensibilidad a cada vértice de la curva cupón cero para un movimiento de 1 punto básico.

```python
print(type(der))
```

    <class 'finrisk.QC_Financial_3.double_vec'>

```python
delta = .0001
total = 0
for i, d in enumerate(der):
    total += d * delta
    print(f"Sensibilidad en {i}: {d * delta: 0,.2f}")
print(f"Sensibilidad total: {total:,.2f}")
```

    Sensibilidad en 0:  0.00
    Sensibilidad en 1:  0.00
    Sensibilidad en 2:  0.00
    Sensibilidad en 3:  0.00
    Sensibilidad en 4:  0.00
    Sensibilidad en 5:  0.00
    Sensibilidad en 6:  0.00
    Sensibilidad en 7:  0.00
    Sensibilidad en 8:  0.00
    Sensibilidad en 9:  0.00
    Sensibilidad en 10:  0.00
    Sensibilidad en 11:  0.00
    Sensibilidad en 12:  0.00
    Sensibilidad en 13:  0.00
    Sensibilidad en 14:  0.00
    Sensibilidad en 15: -10.12
    Sensibilidad en 16:  0.00
    Sensibilidad en 17: -2,016.84
    Sensibilidad en 18:  0.00
    Sensibilidad en 19:  0.00
    Sensibilidad en 20:  0.00
    Sensibilidad en 21:  0.00
    Sensibilidad en 22:  0.00
    Sensibilidad en 23:  0.00
    Sensibilidad en 24:  0.00
    Sensibilidad en 25:  0.00
    Sensibilidad en 26:  0.00
    Sensibilidad en 27:  0.00
    Sensibilidad en 28:  0.00
    Sensibilidad en 29:  0.00
    Sensibilidad en 30:  0.00
    Sensibilidad en 31:  0.00
    Sensibilidad en 32:  0.00
    Sensibilidad total: -2,026.97

**Ejercicio:** Verifique por diferencias finitas centrales la sensibilidad en el vértice 17. La aproximación de la derivada por diferencias finitas centrales es:

$$
\begin{equation}
f^{\prime}\left(x\right)\approx\frac{f\left(x+h\right)-f\left(x-h\right)}{2h}
\end{equation}
$$

### Pata Flotante

```python
vp.pv(fecha_val, op[1], zcc)
der = vp.get_derivatives()
```

```python
total = 0
for i, d in enumerate(der):
    total += d * delta
    print(f"Sensibilidad en {i}: {d * delta: 0,.2f}")
print(f"Sensibilidad total: {total:,.2f}")
```

    Sensibilidad en 0:  0.00
    Sensibilidad en 1:  0.00
    Sensibilidad en 2:  0.00
    Sensibilidad en 3:  0.00
    Sensibilidad en 4:  0.00
    Sensibilidad en 5:  0.00
    Sensibilidad en 6:  0.00
    Sensibilidad en 7:  0.00
    Sensibilidad en 8:  0.00
    Sensibilidad en 9:  0.00
    Sensibilidad en 10:  0.00
    Sensibilidad en 11:  0.00
    Sensibilidad en 12:  0.00
    Sensibilidad en 13:  0.00
    Sensibilidad en 14:  0.00
    Sensibilidad en 15:  0.70
    Sensibilidad en 16:  0.00
    Sensibilidad en 17:  1,997.46
    Sensibilidad en 18:  0.00
    Sensibilidad en 19:  0.00
    Sensibilidad en 20:  0.00
    Sensibilidad en 21:  0.00
    Sensibilidad en 22:  0.00
    Sensibilidad en 23:  0.00
    Sensibilidad en 24:  0.00
    Sensibilidad en 25:  0.00
    Sensibilidad en 26:  0.00
    Sensibilidad en 27:  0.00
    Sensibilidad en 28:  0.00
    Sensibilidad en 29:  0.00
    Sensibilidad en 30:  0.00
    Sensibilidad en 31:  0.00
    Sensibilidad en 32:  0.00
    Sensibilidad total: 1,998.16

La estructura es la misma que para una pata fija, lo que indica que se debe también incluir la sensibilidad a la curva de proyección.

```python
import numpy as np
result = []
for i in range(op[1].size()):
    cshflw = op[1].get_cashflow_at(i)
    amt_der = cshflw.get_amount_derivatives()
    df = zcc.get_discount_factor_at(fecha_val.day_diff(cshflw.get_settlement_date()))
    amt_der = [a * delta * df for a in amt_der]
    if len(amt_der) > 0:
        result.append(np.array(amt_der))
total = result[0] * 0

for r in result:
    total += r

for i in range(len(total)):
    print(f"Sensibilidad en {i}: {total[i]:0,.2f}")

print(f"Sensibilidad de proyección: {sum(total):,.2f} USD")
```

    Sensibilidad en 0: -0.00
    Sensibilidad en 1: -0.00
    Sensibilidad en 2: -0.00
    Sensibilidad en 3: -0.00
    Sensibilidad en 4: -0.00
    Sensibilidad en 5: -0.00
    Sensibilidad en 6: -0.00
    Sensibilidad en 7: -0.00
    Sensibilidad en 8: -0.00
    Sensibilidad en 9: -0.00
    Sensibilidad en 10: -0.00
    Sensibilidad en 11: -0.00
    Sensibilidad en 12: -0.00
    Sensibilidad en 13: -0.00
    Sensibilidad en 14: -0.00
    Sensibilidad en 15: -0.70
    Sensibilidad en 16: -0.00
    Sensibilidad en 17: -1,997.46
    Sensibilidad en 18: -0.00
    Sensibilidad en 19: -0.00
    Sensibilidad en 20: -0.00
    Sensibilidad en 21: -0.00
    Sensibilidad en 22: -0.00
    Sensibilidad en 23: -0.00
    Sensibilidad en 24: -0.00
    Sensibilidad en 25: -0.00
    Sensibilidad en 26: -0.00
    Sensibilidad en 27: -0.00
    Sensibilidad en 28: -0.00
    Sensibilidad en 29: -0.00
    Sensibilidad en 30: -0.00
    Sensibilidad en 31: -0.00
    Sensibilidad en 32: -0.00
    Sensibilidad de proyección: -1,998.16 USD

**Ejercicio:** Ambas sensibilidades se cancelan. ¿Porqué?


# Random walks on the spanning cluster

In this notebook we'll explore diffusion on the spanning cluster.

```python
import sys
sys.path.append("..")

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import scipy.ndimage
import tqdm
import sklearn.linear_model

from generate_spanning_cluster import get_spanning_cluster
from percwalk import percwalk, parallel_percwalk
from log_binning import log_bin
```

```python
sns.set(color_codes=True)
```

```python
# Critical percolation probability
p_c = 0.59275
```

## Distance moved on the percolating cluster

We'll start by exploring diffusion on the percolating cluster for $p > p_c$. We'll measure the second moment of the average distance, $\langle R^2 \rangle$, as a function of the number of steps performed. Below we'll demonstrate what the diffusion on the percolating cluster looks like.

```python
spanning_cluster = get_spanning_cluster(100, p_c)
```

Having created a system with a percolating cluster, we start by visualizing the spanning cluster.

```python
plt.figure(figsize=(14, 10))

plt.imshow(spanning_cluster, origin="lower")
plt.show()
```

Here we can see the sites contained in the percolating cluster. Moving on, we'll start a walker on a random site in the percolating cluster.

```python
num_walks = int(1e5)
num_steps = 0
```

```python
while num_steps <= 1:
    walker_map, displacements, num_steps = percwalk(spanning_cluster, num_walks)
```

```python
plt.figure(figsize=(14, 10))

plt.imshow(spanning_cluster, origin="lower")
# walker_map is oriented as row-column (ix, iy)
plt.plot(walker_map[1], walker_map[0])
plt.show()
```

In the above plot we can see how the walker moves in the spanning cluster. A thick line signifies several walks over a site.

## Mean squared distance

Next, we'll look at the mean squared distance, $\langle r^2 \rangle$, as a function of the number of walks, $N$, for varying $p > p_c$.
A free random walker behaves according to
\begin{align}
    \langle r^2 \rangle \propto D t,
\end{align}
as we've seen in the molecular dynamics simulations.
On the percolating cluster we expect that the random walker at times gets stuck in dangling ends or slows down through singly connected bonds.
We introduce the scaling theory
\begin{align}
    \langle r^2 \rangle = t^{2k} f\left[ (p - p_c) t^x \right],
\end{align}
for the mean squared distance on a system with random set sites.
We can categorize the function $f(u)$ for three different situations.
\begin{align}
    f(u) = \begin{cases}
        \text{const} & |u| \ll 1, \\
        u^{\mu} & u \gg 1, \\
        (-u)^{\beta - 2\nu} &  u \ll -1.
    \end{cases}
\end{align}
The first situation arises when $p \approx p_c$ and yields the scaling
\begin{align}
    \langle r^2 \rangle \propto t^{2k},
\end{align}
where $k < 1/2$.
This is known as _subdiffusion_ as opposed to the _superdiffusion_ observed for the initial molecular dynamics simulations.
The second situation $u \gg 1 \implies p > p_c$ and $t > t_0$, where $t_0$ is some threshold time.
This will yield the scaling theory
\begin{align}
    \langle r^2 \rangle \propto (p - p_c)^{\mu} t \propto t^{2k} \left[(p - p_c) t^{x} \right]^{\mu}.
\end{align}
The final situation is found when $p < p_c$ and $t > t_0$ yielding the scaling theory
\begin{align}
    \langle r^2 \rangle \propto (p_c - p)^{\beta - 2\nu} \propto t^{2k} \left[ (p_c - p) t^x \right]^{\beta - 2\nu}.
\end{align}
We will not be exploring the latter situation.
When $p > p_c$ and after a certain time $t_0$ has passed we expect the random walker on the cluster to behave as a normal free random walker.
That is, when $t \gg t_0$ we get
\begin{align}
    \langle r^2 \rangle \propto D t,
\end{align}
when $p > p_c$.
Stated differently, when $\langle r^2 \rangle \gg \xi^2$ the walker behaves as a free random walker.
The interesting fact here is that when $p = p_c$ we know that $\xi \to \infty$ and this condition will never be fullfilled.
This means that the scaling theory described above for the mean squared distance when $p = p_c$ will be true indefinitely.
We can summarize it by the function
\begin{align}
    \langle r^2 \rangle \propto \begin{cases}
        t^{2k} & \langle r^2 \rangle \ll \xi^2, \\
        D t & \langle r^2 \rangle \gg \xi^2.
    \end{cases}
\end{align}

We now simulate a system of size $L$ and compute the mean squared displacement on the spanning cluster several times for every system.

```python
L = 512
num_walks = 1 * int(1e5)
num_walks_arr = np.arange(num_walks)
p_arr = np.linspace(p_c, 0.8, 12)
num_walkers = 100
num_systems = 20

r_squared = np.zeros((len(p_arr), num_walks))
```

It is important to note that the implementation for the percolation walker we use require $2L^2 > N_w$, where $N_w$ is the number of walks (or, time if you like) we let the walkers perform. This is so that the walker won't start backtracking from whence it came thereby flattening the displacement curve.

```python
for i, p in tqdm.tqdm_notebook(enumerate(p_arr), total=len(p_arr)):
    spanning_cluster = get_spanning_cluster(L, p, num_attempts=1000)
    r_squared[i] += parallel_percwalk(spanning_cluster, num_walks, num_walkers, num_systems)
```

    HBox(children=(IntProgress(value=0, max=12), HTML(value='')))

Having computed the mean squared displacement for several different values of $p \geq p_c$ we look at the diffusion behavior.
For $p = p_c$ we know that $\xi \to \infty$ and we know that
\begin{align}
    \langle r^2(t) \rangle \propto t^{2k},
\end{align}
for all values of $t$ as we will never have $\langle r^2(t) \rangle \gg \xi^2$.
We therefore estimate $k$ by taking the logarithm on both sides of the equation and using linear regression.
\begin{gather}
    \log\left( \langle r^2(t) \rangle \right)
    = \log(C) + 2k \log(t),
\end{gather}
where $C$ is a constant serving as the intercept.

```python
plt.figure(figsize=(14, 10))

plt.loglog(num_walks_arr, r_squared[0])
plt.fill_between(num_walks_arr, r_squared[0], alpha=0.2)
plt.xlabel(r"$t$")
plt.ylabel(r"$\langle r^2(t) \rangle$")
plt.title(r"Plot of the mean squared distance as a function of $t$ for $p = p_c$")
plt.show()
```

In this figure we can see how the log-log plot of the mean squared distance as a function of time for $p = p_c$ becomes linear.
We now use linear regression to find an estimate for $k$.

```python
log_walks = np.log(num_walks_arr[1:])
log_r = np.log(r_squared[0, 1:])

clf = sklearn.linear_model.LinearRegression().fit(
    log_walks[:, np.newaxis], log_r[:, np.newaxis]
)

k = clf.coef_[0, 0] / 2
C = clf.intercept_[0]

print(f"k = {k}")
```

    k = 0.3585631489136576

Having found the intercept and the coefficient $k$ we plot $\langle r^2(t) \rangle$ as a function of the number of walks (time, in a sense) and compare it to the theoretical estimate.

```python
plt.figure(figsize=(14, 10))

plt.plot(
    num_walks_arr,
    r_squared[0],
    label=rf"$p = p_c = {p_c}$",
)
plt.fill_between(num_walks_arr, r_squared[0], alpha=0.2)

plt.plot(
    num_walks_arr,
    np.exp(C) * num_walks_arr ** (2 * k),
    "--",
    label=r"$\langle r^2(t) \rangle = \exp(C) t^{2k}$",
)

plt.title(r"Plot of the mean squared displacement as a function of time $t$ for $p = p_c$")
plt.xlabel(r"$t$")
plt.ylabel(r"$\langle r^2(t) \rangle$")
plt.legend(loc="best")
plt.show()
```

In this figure we have plotted the mean squared displacement as a function of "time", i.e., the number of walks, on the spanning cluster for $p = p_c$.
We observe the expected subdiffusion.
We also see how the theoretical estimate of the subdiffusion follows the data closely.
Below we plot the mean squared distance for $p \geq p_c$.

```python
plt.figure(figsize=(14, 10))

plt.loglog(
    num_walks_arr,
    r_squared[0],
    "--",
    label=rf"$p = p_c = {p_c}$",
)
plt.fill_between(num_walks_arr, r_squared[0], alpha=0.2)

for i, p in enumerate(p_arr[1:]):
    plt.loglog(
        num_walks_arr,
        r_squared[i + 1],
        label=rf"$p = {p:.3f}$"
    )

plt.legend(loc="best")
plt.xlabel(r"$t$")
plt.ylabel(r"$\langle r^2(t) \rangle$")
plt.title(r"Plot of the mean squared distance as a function of time for $p \geq p_c$")
plt.show()
```

In the above figure we see how after a time $t_0$ the mean squared distance for $p > p_c$ deviates from the situation when $p = p_c$.
Reformulating the scaling theory for $p > p_c$ to create a data collapse we get
\begin{gather}
    \langle r^2 \rangle \propto t^{2k} (p - p_c) t^{x}
    \implies
    t^{-2k} \langle r^2 \rangle \propto (p - p_c) t^{x}.
\end{gather}
We plot this on a log-log axis when creating the data collapse.
However, the challenge is to find an expression for $x$.
We have previously found that
\begin{align}
    \langle r^2(t) \rangle \propto D(p) t,
\end{align}
where $D(p) = (p - p_c)^{\mu}$ when $p > p_c$ and $r \gg \xi$.
Inserted into the scaling theory we can write
\begin{gather}
    (p - p_c)^{\mu} t \propto t^{2k} \left[(p - p_c) t^{x}\right]^{\mu}
    \implies
    1 = 2k + \mu x
    \implies
    x = \frac{1 - 2k}{\mu}.
\end{gather}
Having already found $k$ we are thus tasked with finding an expression for $\mu$ and we can be on our merry way to plotting a data collapse.

We know that when $t < t_0$ the mean squared distance scales as $t^{2k}$ when $p = p_c$ and $p > p_c$.
However, when $t > t_0$ this scaling continues only for $p = p_c$ whereas for $p > p_c$ the mean squared distance will increase faster in time.
We are interested in locating $t_0$ and we do this by finding the point where
\begin{align}
    \langle r^2(t; p_c) \rangle \leq \frac{1}{2} \langle r^2(t; p > p_c) \rangle,
\end{align}
that is, we find the point where the scaling is more than twice as high.
At $t = t_0$ we know that $\langle r^2(t_0) \rangle = \xi^2$.
We are lucky enough to have a scaling ansatz for the characteristic cluster length stating
\begin{align}
    \xi = \xi_0 (p - p_c)^{-\nu}.
\end{align}

```python
plt.figure(figsize=(14, 10))

plt.loglog(
    num_walks_arr,
    r_squared[0],
    "--",
    label=rf"$p = p_c = {p_c}$",
)
plt.fill_between(num_walks_arr, r_squared[0], alpha=0.2)

xi_squared_list = []
t_list = []
ind_list = []
p_list = []

for i, p in enumerate(p_arr[1:]):
    plt.loglog(
        num_walks_arr,
        r_squared[i + 1],
        label=rf"$p = {p:.3f}$"
    )

    mask = r_squared[0, 1:] <= 0.5 * r_squared[i + 1, 1:]

    if not np.any(mask):
        continue

    index = np.argmax(mask)

    xi_squared_list.append(r_squared[i + 1, index])
    t_list.append(num_walks_arr[index])
    ind_list.append(i + 1)
    p_list.append(p)

    plt.scatter(num_walks_arr[index], r_squared[i + 1, index])

plt.legend(loc="best")
plt.xlabel(r"$t$")
plt.ylabel(r"$\langle r^2(t) \rangle$")
plt.title(r"Plot of the mean squared distance as a function of time for $p \geq p_c$")
plt.show()
```

It is interesting to note that for a larger value of $p > p_c$ the cross-over time $t_0$ becomes smaller.
That is, for a higher percolation probability $p$ the time it takes for the system to behave as a "normal" random walker becomes shorter.
This also makes sense as there are fewer sites not contained in the spanning cluster.

Having found the points $(t_0, \xi^2)$ we now find an expression for $\nu$.
Taking the logarithm on both sides of the scaling ansatz for the characteristic length yields
\begin{align}
    \log(\xi) = \log(\xi_0) - \nu \log(p - p_c).
\end{align}
We use linear regression to find an expression for the intercept $\log(\xi_0)$ and the exponent $\nu$.

```python
p_min_pc_arr = np.abs(np.array(p_list) - p_c)

clf = sklearn.linear_model.LinearRegression().fit(
    np.log(p_min_pc_arr)[:, np.newaxis],
    np.log(np.sqrt(xi_squared_list))[:, np.newaxis],
)

nu = -clf.coef_[0, 0]
log_xi_0 = clf.intercept_[0]

print(f"nu = {nu}")
```

    nu = 1.3927473577662757

The exact value of $\nu$ in two dimensions is $\nu = 4/3$.

```python
nu_exact = 4 / 3
print(f"nu_exact = {nu_exact}")
```

    nu_exact = 1.3333333333333333

We therefore see that we are in ballpark range.
We plot the characteristic length as a function of $p$ and compare it to the theoretical scaling using our estimate of $\nu$.

```python
fig = plt.figure(figsize=(14, 10))

plt.plot(
    p_list,
    np.sqrt(xi_squared_list),
    "-o",
    label=r"$Measured$",
)
plt.plot(
    p_list,
    np.exp(log_xi_0) * p_min_pc_arr ** (-nu),
    "-o",
    label=r"$(p - p_c)^{-%.3f}$" % nu
)
#plt.plot(
#    p_list,
#    np.exp(log_xi_0) * p_min_pc_arr ** (-nu_exact),
#    "-o",
#    label=r"$(p - p_c)^{-%.3f}$" % nu_exact
#)
plt.xlabel(r"$p$")
plt.ylabel(r"$\xi$")
plt.title(r"Plot of the characteristic length as a function of the percolation probability $p$")
plt.legend(loc="best")
plt.show()
```

In this figure we see how our estimate of $\nu$ follows the measured data quite well.
And, more importantly, they follow the theoretical scaling using the exact value of $\nu$ closely.
Looking at the point where the cross-over time occurs, we see that this scales inversely with increasing $p$.
We know that for $p > p_c$ and $t < t_0$ we have the relation
\begin{align}
    \langle r^2(t) \rangle \propto t^{2k},
\end{align}
that is the system experiences subdiffusion.
When $t > t_0$ the relation takes on the form
\begin{align}
    \langle r^2(t) \rangle \propto t^{2k}\left[(p - p_c) t^x \right]^{\mu}.
\end{align}
However, at the cross-over time, that is when $t = t_0$, we expect these to be proportional.
\begin{gather}
    t_0^{2k} \propto
    t_0^{2k}\left[(p - p_c) t_0^{x} \right]^{\mu}
    \implies
    (p - p_c) t_0^{x} \propto 1
    \implies
    t_0 \propto |p - p_c|^{-1/x}.
\end{gather}

```python
neg_inv_x, log_C = np.polyfit(np.log(p_min_pc_arr), np.log(t_list), deg=1)

x = -1 / neg_inv_x
print(f"x = {x}")
```

    x = 0.25973794261093536

```python
fig = plt.figure(figsize=(14, 10))

plt.plot(
    p_list,
    t_list,
    "-o",
    label=r"Measured",
)

plt.plot(
    p_list,
    np.exp(log_C) * p_min_pc_arr ** (-1 / x),
    "-o",
    label=r"$t_0 \propto |p - p_c|^{-1/x}$"
)

plt.xlabel(r"$p$")
plt.ylabel(r"$t_0$")
plt.title(r"Plot of the scaling of the cross-over time as a function for $p$")
plt.legend(loc="best")
plt.show()
```

In order to find an estimate for $x$ we need to find a value for $\mu$.
As we've stated earlier, when $p > p_c$ and $t \gg t_0$ which means that $r \gg \xi$ the random walker moves away from the anomalous diffusion and instead behaves as a free random walker with the relation
\begin{align}
    \langle r^2(t) \rangle = D(p) t.
\end{align}
The Einstein relation for diffusion relates the diffusion constant to the conductance through
\begin{align}
    D(p) \propto \sigma(p) \propto (p - p_c)^{\mu}.
\end{align}

```python
plt.figure(figsize=(14, 10))

diffusion_list = []
intercept_list = []
p_min_pc_diff_list = []

for i, p_i in enumerate(ind_list):
    t_0 = t_list[i]
    p = p_arr[p_i]

    p_min_pc_diff_list.append(p - p_c)

    cross = num_walks_arr > t_0

    t_cross = num_walks_arr[cross]
    r_cross = r_squared[p_i, cross]

    d, intercept = np.polyfit(t_cross, r_cross, deg=1)
    diffusion_list.append(d)
    intercept_list.append(intercept)

    plt.plot(t_cross, r_cross, label=fr"$p = {p:.3f}$")

plt.title(r"Plot of the mean squared distance as a function of $t > t_0$")
plt.xlabel(r"$t$")
plt.ylabel(r"$\langle r^2(t) \rangle$")
plt.legend(loc="best")
plt.show()
```

Ideally we expect the mean squared distance to be completely linear, however, we see that even though $t > t_0$ there are slight deviations to this trend.

Taking the logarithm on both sides of the scaling ansatz we are able to estimate $\mu$.
\begin{align}
    \log(D) \propto \mu \log(p - p_c).
\end{align}
Again we use the least squares method to estimate $\mu$.

```python
mu, intercept = np.polyfit(np.log(p_min_pc_diff_list), np.log(diffusion_list), deg=1)
```

```python
print(mu)
```

    0.819095951447221

From the lecture notes we know that the exact value of $\mu$ is given by $\mu = 1.3$.

```python
mu_exact = 1.3
```

This shows that we miss by quite a lot when it comes to estimating $\mu$.

```python
plt.figure(figsize=(14, 10))

plt.plot(
    p_min_pc_diff_list,
    diffusion_list,
    "-o",
    label=r"Measured",
)
plt.plot(
    p_min_pc_diff_list,
    np.exp(intercept) * np.array(p_min_pc_diff_list) ** (mu),
    "-o",
    label=r"$(p - p_c)^{%.3f}$" % mu,
)
plt.xlabel(r"$p - p_c$")
plt.ylabel(r"$D(p)$")
plt.title(r"Plot of the diffusion constant as a function of $p - p_c$")
plt.legend(loc="best")
plt.show()
```

In this figure we've plotted the diffusion constant as a function of $p - p_c$.
We compare the measured values with the theoretical scaling using our estimate of $\mu$.

```python
x = (1 - 2 * k) / mu
print(f"x = {x}")
```

    x = 0.3453486757844291

```python
p_min_pc = p_arr[1:] - p_c
r_gg_xi = r_squared[1:, 1:]
t_sq = num_walks_arr[1:] ** (-2 * k)
```

```python
plt.figure(figsize=(14, 10))

for i in range(len(p_min_pc)):
    plt.plot(
        p_min_pc[i] * num_walks_arr[1:] ** (x),
        t_sq * r_gg_xi[i],
        label=fr"$p = {p_arr[i + 1]:.3f}$",
    )
    #plt.plot(
    #    p_min_pc[i] * num_walks_arr[1:] ** (x),
    #    np.exp(intercept) * (p_min_pc[i] * num_walks_arr[1:] ** (x)) ** mu,
    #    label=fr"Theoretical: $p = {p_arr[i + 1]:.3f}$",
    #)

plt.legend(loc="best")
plt.xlabel(r"$(p - p_c) t^{x}$")
plt.ylabel(r"$t^{-2k} \langle r^{2} (t)\rangle$")
plt.title(r"Data-collapse plot for mean squared distance")
plt.show()
```

In this figure we can see the data-collapse of the mean squared displacement.
We have compared to the theoretical scaling
\begin{gather}
    \langle r^2 \rangle \propto t^{2k} (p - p_c) t^{x}
    \propto t^{2k}\left[(p - p_c) t^{x}\right]^{\mu},
\end{gather}
where we use the intercept from the diffusion constant as the constant of proportionality.
We note that collapse isn't completely perfect, especially when $p$ is close to $p_c$.
A possible explanation to this is due to the percolation walker being restricted to a finite spanning cluster.
We might be able to alleviate some of this discrepancy if we allow the spanning cluster to have periodic boundaries, however this will require quite a lot of manipulation when labelling the spanning cluster and for the percolation walker and has therefore not been tested.

Finally we are interested in estimating the dimension of the random walk, $d_w$.
We have that
\begin{align}
    d_w = \frac{1}{k}.
\end{align}
We can also estimate $d_w$ from the scaling relation
\begin{align}
    \langle r^2(t) \rangle \propto t^{2k} = t^{2/d_w},
\end{align}
when we $t < t_0$ for $p > p_c$.

```python
d_w_list = []
d_w_intercept_list = []
p_min_pc_d_w_list = []

for i, p_i in enumerate(ind_list):
    t_0 = t_list[i]
    p = p_arr[p_i]

    p_min_pc_d_w_list.append(p - p_c)

    pre_cross = num_walks_arr < t_0

    t_pre_cross = num_walks_arr[pre_cross]
    r_pre_cross = r_squared[p_i, pre_cross]

    d_w_2_inv, d_w_intercept = np.polyfit(t_pre_cross, r_pre_cross, deg=1)
    d_w_list.append(2 / d_w_2_inv)
    d_w_intercept_list.append(d_w_intercept)
```

```python
fig = plt.figure(figsize=(14, 10))

plt.plot(p_list, d_w_list, "-o")
plt.xlabel(r"$p$")
plt.ylabel(r"$d_w$")
plt.title(r"Plot of the dimension of the random walk as a function of $p$")
plt.show()
```