## Table of Contents

1. [Introduction](#scrollTo=uNtcXSBl3uFA)
2. [Setup](#scrollTo=ctqfUWXz3uFE)
3. [Effects of intrinsic properties on neuron responses](#scrollTo=1nv93tcYdWXg)

 - [Interactive Demo 3.1: Parameter Exploration of the model](#scrollTo=1nv93tcYdWXg)

4. [Effects of synaptic properties on neuron responses](#scrollTo=iijHfBSo3uFJ)

 - [Interactive Demo 4.1: Membrane current noise](#scrollTo=5zMDuWNB3uFJ)

 - [Interactive Demo 4.2: Input-Output function](#scrollTo=QfUp9ZVB3uFK)

 - [Interactive Demo 4.3: Spiking Regularity](#scrollTo=2J3b_gmA3uFK)

---

## Introduction

([Return to Table of Contents](#scrollTo=V7gPh3_ZJeRD))

Neurons are able to coordinate movement of sets of muscles because they are synaptically connected to each other into circuits. The way a neural circuit functions depends on two main things:
1. How neurons are connected (types of synapses and identity of connections).
2. Intrinsic physiology of individual neurons.

Neurons are quite diverse. Differences in basic intrinsic properties effect how they are recruited by the circuits in which they are integrated. Intrinsic neuron properties therefore effect the patterned output generated by neural circuits.

We can explore the effects of some basic intrinsic properties on neural recruitment by using neuron "models." In this notebook you will implement the classic and very basic **Leaky Integrate-and-Fire (LIF)** neuron model.
You will study its dynamics in response to various types of inputs. In particular, we are going to be able to:

- simulate the LIF neuron model

- drive the LIF neuron with different types of synaptic input, such as constant and noisy current.

- study how different inputs affect the  LIF neuron's output (firing rate and spike time irregularity)

We will especially emphasize identifying conditions (input statistics) under which a neuron can either: 1. spike at low firing rates and in an irregular manner, or 2. spike at a precise time or in a regular manner. The reason for focusing on this is that different parts of the motor system need to meet different coding requirements. Motor neurons need to fire precicely and at high, regular rates. In most cases, however, neocortical neurons (like in the motor cortex) spike in an irregular manner due to *background synaptic noise*.

### The Leaky Integrate-and-Fire (LIF) model

This supplementary video (https://youtube.com/watch?v=rSExvwCVRYg) introduces the reduction of a biological neuron to a simple leaky-integrate-fire (LIF) neuron model. It is more detailed than you need to know to complete this notebook, but helpful if you are interested in understanding the process more in depth.

Now, it's your turn to implement one of the simplest mathematical model of a neuron: the leaky integrate-and-fire (LIF) model. The basic idea of LIF neuron was proposed in 1907 by Louis Édouard Lapicque, long before we understood the electrophysiology of a neuron (see a translation of [Lapicque's paper](https://pubmed.ncbi.nlm.nih.gov/17968583/) ). More details of the model can be found in the book [**Theoretical neuroscience**](http://www.gatsby.ucl.ac.uk/~dayan/book/) by Peter Dayan and Laurence F. Abbott.

The subthreshold membrane potential dynamics of a LIF neuron is described by

\begin{eqnarray}
C_m\frac{dV}{dt} = -g_L(V-E_L) + I,\quad (1)
\end{eqnarray}

where $V$ is the membrane potential, $g_L$ is the leak conductance ($g_L = 1/R$, the inverse of the leak resistance $R$), $E_L$ is the resting potential, $I$ is the external input current, and $\tau_m$ is membrane time constant. Dividing both sides of the above equation by $g_L$ gives

\begin{align}
\tau_m\frac{dV}{dt} = -(V-E_L) + \frac{I}{g_L}\,,\quad (2)
\end{align}

where the $\tau_m$ is membrane time constant and is defined as $\tau_m=C_m/g_L$.

Note that dividing capacitance by conductance gives units of time!

You will implement a model that uses Eqn.(2) to simulate LIF neuron dynamics. This model implements a couple of shortcuts to simulating spiking activity from membrane potential dynamics:

> If $I$ is sufficiently strong such that $V$ reaches a certain threshold value $V_{\rm th}$, $V$ is reset to a reset potential $V_{\rm reset}< V_{\rm th}$, and voltage is clamped to $V_{\rm reset}$ for $\tau_{\rm ref}$ ms, mimicking the refractoriness of the neuron during an action potential:

The LIF model captures the facts that a neuron:
- performs spatial and temporal integration of synaptic inputs
- generates a spike when the voltage reaches a certain threshold
- goes refractory during the action potential
- has a leaky membrane

However, the LIF model assumes that the spatial and temporal integration of inputs is linear (which is not strictly true in real neurons). Also, membrane potential dynamics close to the spike threshold are much slower in LIF neurons than in real neurons.

## 2. Setup

([Return to Table of Contents](#scrollTo=V7gPh3_ZJeRD))

```python
#@markdown **TASK:** Execute this cell to initialize the notebook
#@markdown and define functions required to simulate an LIF model neuron.

# Imports
import numpy as np
import matplotlib.pyplot as plt
import ipywidgets as widgets  # interactive display
from IPython.display import display
from datetime import datetime,timezone,timedelta
%config InlineBackend.figure_format = 'retina'
# use NMA plot style
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
my_layout = widgets.Layout()

def plot_volt_trace(pars, v, sp):
    """
    Plot trajetory of membrane potential for a single neuron

    Expects:
    pars   : parameter dictionary
    v      : volt trajetory
    sp     : spike train

    Returns:
    figure of the membrane potential trajetory for a single neuron
    """

    V_th = pars['V_th']
    dt, range_t = pars['dt'], pars['range_t']
    if sp.size:
        sp_num = (sp / dt).astype(int) - 1
        v[sp_num] += 40  # draw nicer spikes
    plt.clf()
    plt.plot(pars['range_t'], v, 'b')
    plt.axhline(V_th, 0, 1, color='k', ls='--')
    plt.xlabel('Time (ms)',fontsize=14)
    plt.ylabel('V (mV)',fontsize=14)
    plt.xticks(fontsize=14)
    plt.yticks(fontsize=14)
    plt.legend(['Membrane\npotential', r'Threshold V$_{\mathrm{th}}$'],
             loc=[1.05, 0.75])
    plt.ylim([-90, -20])
    plt.show()

def plot_GWN(pars, I_GWN):
    """
    Args:
    pars  : parameter dictionary
    I_GWN : Gaussian white noise input

    Returns:
    figure of the gaussian white noise input
    """

    plt.figure(figsize=(12, 4))
    plt.subplot(121)
    plt.plot(pars['range_t'][::3], I_GWN[::3], 'b')
    plt.xlabel('Time (ms)',fontsize=14)
    plt.ylabel(r'Current (pA)',fontsize=14)
    plt.xticks(fontsize=14)
    plt.yticks(fontsize=14)
    plt.subplot(122)
    plot_volt_trace(pars, v, sp)
    plt.tight_layout()
    plt.show()

def plot_IF_curve(spk_count, spk_count_dc, I_mean, sig_gwn):
    """
    Plot trajetory of membrane potential for a single neuron

    Expects:
    pars   : parameter dictionary
    v      : volt trajetory
    sp     : spike train

    Returns:
    figure of the membrane potential trajetory for a single neuron
    """
#     hfig,ax = plt.subplots(1)
    plt.clf()
    plt.plot(I_mean, spk_count, 'k',
           label=r'noise=%.2f' % sig_gwn)
    plt.plot(I_mean, spk_count_dc, 'k--', alpha=0.5, lw=4, dashes=(2, 2),
           label='constant current')
    plt.ylabel('Spike count',fontsize=14)
    plt.xlabel('Average injected current (pA)',fontsize=14)
    plt.xticks(fontsize=14)
    plt.yticks(fontsize=14)
    plt.legend(loc='best',fontsize=14)
    plt.show()

def my_hists(isi1, isi2, cv1, cv2, sigma1, sigma2):
    """
    Args:
    isi1 : vector with inter-spike intervals
    isi2 : vector with inter-spike intervals
    cv1  : coefficient of variation for isi1
    cv2  : coefficient of variation for isi2

    Returns:
    figure with two histograms, isi1, isi2

    """
    plt.figure()
    my_bins = np.linspace(10, 30, 20)
    plt.subplot(121)
    plt.hist(isi1, bins=my_bins, color='b', alpha=0.5)
    plt.xlabel('ISI (ms)',fontsize=14)
    plt.ylabel('count',fontsize=14)
    plt.xticks(fontsize=14)
    plt.yticks(fontsize=14)
    plt.title(r'$\sigma_{GWN}=$%.1f, CV$_{\mathrm{isi}}$=%.3f' % (sigma1, cv1),fontsize=14)

    plt.subplot(122)
    plt.hist(isi2, bins=my_bins, color='b', alpha=0.5)
    plt.xlabel('ISI (ms)')
    plt.ylabel('count')
    plt.xticks(fontsize=14)
    plt.yticks(fontsize=14)
    plt.title(r'$\sigma_{GWN}=$%.1f, CV$_{\mathrm{isi}}$=%.3f' % (sigma2, cv2))
    plt.tight_layout()
    plt.show()

def default_pars(**kwargs):
    pars = {}

    # typical neuron parameters#
    pars['V_th'] = -55.     # spike threshold [mV]
    pars['V_reset'] = -75.  # reset potential [mV]
    pars['tau_m'] = 10.     # membrane time constant [ms]
    pars['g_L'] = 10.       # leak conductance [nS]
    pars['V_init'] = -75.   # initial potential [mV]
    pars['E_L'] = -75.      # leak reversal potential [mV]
    pars['tref'] = 2.       # refractory time (ms)

    # simulation parameters #
    pars['T'] = 400.  # Total duration of simulation [ms]
    pars['dt'] = .1   # Simulation time step [ms]

    # external parameters if any #
    for k in kwargs:
        pars[k] = kwargs[k]

    pars['range_t'] = np.arange(0, pars['T'], pars['dt'])  # Vector of discretized time points [ms]

    return pars

def run_LIF(pars, Iinj, stop=False, durstep=100):
    """
    Simulate the LIF dynamics with external input current
    Args:
    pars       : parameter dictionary
    Iinj       : input current [pA]. The injected current here can be a value
                 or an array
    stop       : boolean. If True, use a current pulse
    Returns:
    rec_v      : membrane potential
    rec_sp     : spike times
    """

    # Set parameters
    V_th, V_reset = pars['V_th'], pars['V_reset']
    tau_m, g_L = pars['tau_m'], pars['g_L']
    V_init, E_L = pars['V_init'], pars['E_L']
    dt, range_t = pars['dt'], pars['range_t']
    fs = 1/dt
    Lt = range_t.size
    tref = pars['tref']

    # Initialize voltage
    v = np.zeros(Lt)
    v[0] = V_init

    # Set current time course
    Iinj = Iinj * np.ones(Lt)
    durstep = int((durstep*fs)/2)
    # If current pulse, set beginning and end to 0
    if stop:
        # Iinj[:int(len(Iinj) / 2) - (int(durstep/pars['dt'])/2)] = 0
        # Iinj[int(len(Iinj) / 2) + (int(durstep/pars['dt'])/2):] = 0

        Iinj[:int(len(Iinj) / 2) - durstep] = 0
        Iinj[int(len(Iinj) / 2) + durstep:] = 0

    # Loop over time
    rec_spikes = []  # record spike times
    tr = 0.  # the count for refractory duration

    for it in range(Lt - 1):

        if tr > 0:  # check if in refractory period
            v[it] = V_reset  # set voltage to reset
            tr = tr - 1 # reduce running counter of refractory period

        elif v[it] >= V_th:  # if voltage over threshold
            rec_spikes.append(it)  # record spike event
            v[it] = V_reset  # reset voltage
            tr = tref / dt  # set refractory time

        # Calculate the increment of the membrane potential
        dv = (-(v[it] - E_L) + Iinj[it] / g_L) * (dt / tau_m)

        # Update the membrane potential
        v[it + 1] = v[it] + dv

    # Get spike times in ms
    rec_spikes = np.array(rec_spikes) * dt

    return v, rec_spikes

def my_GWN(pars, mu, sig, myseed=False):
    """
    Function that generates Gaussian white noise input

    Args:
    pars       : parameter dictionary
    mu         : noise baseline (mean)
    sig        : noise amplitute (standard deviation)
    myseed     : random seed. int or boolean
                 the same seed will give the same
                 random number sequence

    Returns:
    I          : Gaussian white noise input
    """

    # Retrieve simulation parameters
    dt, range_t = pars['dt'], pars['range_t']
    Lt = range_t.size

    # Set random seed
    if myseed:
        np.random.seed(seed=myseed)
    else:
        np.random.seed()

    # Generate GWN
    # we divide here by 1000 to convert units to sec.
    I_gwn = mu + sig * np.random.randn(Lt) / np.sqrt(dt / 1000.)

    return I_gwn

# help(my_GWN)

def isi_cv_LIF(spike_times):
  """
  Calculates the inter-spike intervals (isi) and
  the coefficient of variation (cv) for a given spike_train
  Args:
    spike_times : (n, ) vector with the spike times (ndarray)
  Returns:
    isi         : (n-1,) vector with the inter-spike intervals (ms)
    cv          : coefficient of variation of isi (float)
  """

  if len(spike_times) >= 2:
    # Compute isi
    isi = np.diff(spike_times)
    # Compute cv
    cv = isi.std()/isi.mean()
  else:
    isi = np.nan
    cv = np.nan

  return isi, cv

print('Task completed at ' + str(datetime.now(timezone(-timedelta(hours=5)))))
```

## 3. Responses of LIF model neurons with different intrinsic physiology.

([Return to Table of Contents](#scrollTo=V7gPh3_ZJeRD))

### Interactive Demo 3.1: Parameter exploration of constant current input amplitude

Here's an interactive demo that shows how the LIF neuron behavior in response to constant current (DC) input of different amplitudes, changes based on the neuron's intrinsic properties. You can specify specific intrinsic parameter values for the neuron model using the sliders below. Make sure you understand what all of the parameters are. To understand the effects of each parameter, it is helpful to change just one at a time. How many different ways can you change the response of the neuron (whether it spikes or not). How does each parameter effect the response of the neuron to a current injection? You can also specify the duration of the simulation, the amount of current injected, and whether the current injection is continuous or a "step".

```python
#@markdown **TASK:** Run this cell to enable the interactive plot
#@markdown that helps you explore the effect of intrinsic physiology on neuron behavior.

my_layout.width = '700px'
# my_layout.description_width = 'initial'
style = {'description_width': 'initial'}
@widgets.interact(
    current_injection=widgets.FloatSlider(50., min=0., max=1000., step=2.,
                               layout=my_layout,style=style),
    Simulation_Duration=widgets.FloatSlider(400., min=0., max=1000., step=10.,
                               layout=my_layout,style=style),
    Injection_Step=widgets.Checkbox(value=True,
                               description='current step (vs continuous)',
                               layout=my_layout,style=style),
    Current_Step_Duration=widgets.FloatSlider(100., min=10, max=200., step=2.,
                               layout=my_layout,style=style),
    Leak_Reversal=widgets.FloatSlider(-75., min=-90, max=-30., step=2.,
                               layout=my_layout,style=style),
    Leak_Conductance=widgets.FloatSlider(10., min=1, max=50., step=2.,
                               layout=my_layout,style=style),
    AHP_Voltage=widgets.FloatSlider(-80., min=-90., max=-30.,step=2.,
                               layout=my_layout,style=style),
    Spike_Threshold=widgets.FloatSlider(-55., min=-100., max=-30., step=2.,
                               layout=my_layout,style=style),
    Membrane_Tau=widgets.FloatSlider(5., min=1., max=15., step=1.,
                               layout=my_layout,style=style)
)

def diff_DC(current_injection,
            Simulation_Duration,
            Injection_Step,
            Current_Step_Duration,
            Leak_Reversal,
            Leak_Conductance,
            AHP_Voltage,
            Spike_Threshold,
            Membrane_Tau): #I_dc=200., tau_m=10.):
    # pars = default_pars(T=100.)
    # pars['tau_m'] = tau_m
    pars = default_pars(
      E_L = Leak_Reversal,
      g_L = Leak_Conductance,
      V_init = Leak_Reversal,
      V_reset = AHP_Voltage,
      V_th = Spike_Threshold,
      tau_m = Membrane_Tau,
      T=Simulation_Duration)
    # pars=default_pars()
    v, sp = run_LIF(pars, Iinj=current_injection,stop=Injection_Step,durstep=Current_Step_Duration)
    plot_volt_trace(pars, v, sp)

print('Interactive demo initiated at ' + str(datetime.now(timezone(-timedelta(hours=5)))))
```

If you are a bit stuck starting your exploration, here are some ideas to get you started.
Start with "current step (versus continuous)" checked (True/Yes).

- How much DC input current is needed to reach the threshold (*rheobase current*)?

- Change the current amplitude. Once you get the neuron just above spike threshold, what is the effect of changing the:
 - membrane time constant?
 - AHP voltage?
 - leak reversal?

- Change the current amplitude. Find an input current amplitude that drives the neuron to a voltage about halfway between the leak reversal and the spike threshold.
What is the effect of changing the:
 - leak conductance?

## 4. Response of an LIF model to different types of input currents

([Return to Table of Contents](#scrollTo=V7gPh3_ZJeRD))

Given the noisy nature of neuronal activity ___in vivo___, neurons usually receive complex, time-varying inputs. In this section, you will explore the difference between constant membrane current and membrane current with ***noise*** on the response of an LIF neuron.
We will mimic membrane current noise as *gaussian white noise* (random fluctuations with a standard deviation equal to the noise magnitude).

### Interactive Demo 4.1: Membrane current *noise*

([Return to Table of Contents](#scrollTo=V7gPh3_ZJeRD))

The noise describes only the fluctuations of the input received by a neuron. The input current also has a mean amplitude (as it did before). Indeed, when the noise = 0, the input current is just constant as it was before.

So the question arises how does membrane current noise affect the spiking behavior of the neuron. For instance we may want to know:
1.  How does the minimum input current needed to make a neuron spike change with increase in noise? And how does this effect rate coding?
2.  How does the spike regularity change with increase in noise? And how does this effect temporal coding across repetitions of brief (~40msec) excitatory input?

```python
#@markdown **TASK:** Run this cell to make an interactive plot
#@markdown that enables you to explore these questions
#@markdown by changing the amplitude and fluctuation sizes (noise)
#@markdown of the membrane current.
#@markdown >NOTE: With any given set of parameters, you can "repeat stimulation"
#@markdown by hitting that button. Because noise is "random" the spiking can be
#@markdown different on different trials even with the same settings.

my_layout.width = '450px'

@widgets.interact(
    amplitude=widgets.FloatSlider(200., min=0., max=300., step=5.,
                               layout=my_layout),
    noise=widgets.FloatSlider(2.5, min=0., max=5., step=.1,
                                layout=my_layout),
    step=widgets.Checkbox(value=False, description='input a discrete step',
                                layout=my_layout),
    Current_Step_Duration=widgets.FloatSlider(100., min=10, max=200., step=2.,
                               layout=my_layout,style=style),
    repeat_trial=widgets.ToggleButton(value=False,
                                      description='Repeat Simulation')
    )

def diff_GWN_to_LIF(amplitude, noise,step,Current_Step_Duration,repeat_trial):
    pars = default_pars(T=400.)
    I_GWN = my_GWN(pars, mu=amplitude, sig=noise)
#     pars['tau_m'] = tau_m
    v, sp = run_LIF(pars, Iinj=I_GWN,stop=step,durstep=Current_Step_Duration)
    plot_volt_trace(pars, v, sp)

print('Interactive demo initiated at ' + str(datetime.now(timezone(-timedelta(hours=5)))))
```

### Interactive Demo 4.2: Effect of noise on a neuron's input-output function.

([Return to Table of Contents](#scrollTo=V7gPh3_ZJeRD))

As you saw in the [interactive demo 4.2](#scrollTo=qh4PwoIM3uFK), if you increase either the amplitude or the noisiness of the membrane current, the spike count changes.

When we plot the output firing rate as a function of membrane current, it is called the ___input-output transfer function___ of the neuron (so simply F-I curve). In the simulation below, the number of spikes evoked by a 1-second long current step are plotted as a function of current amplitude.

1. Does the membrane current noise always effect the spike count, or does its effects depend on the amplitude of the current?

2. How does the F-I curve of the LIF neuron change as we increase the membrane current noise?

```python
#@markdown **TASK:** Run this cell to make an interactive plot
#@markdown that enables you to explore these questions.
#@markdown <br>There is one slider to control the magnitude of the noise

my_layout.width = '450px'
@widgets.interact(
    noise=widgets.FloatSlider(3.0, min=0., max=6., step=0.5,
                                layout=my_layout)
)

def diff_std_affect_fI(noise):
    pars = default_pars(T=1000.)
    I_mean = np.arange(100., 400., 10.)
    spk_count = np.zeros(len(I_mean))
    spk_count_dc = np.zeros(len(I_mean))

    for idx in range(len(I_mean)):
        I_GWN = my_GWN(pars, mu=I_mean[idx], sig=noise, myseed=2020)
        v, rec_spikes = run_LIF(pars, Iinj=I_GWN)
        v_dc, rec_sp_dc = run_LIF(pars, Iinj=I_mean[idx])
        spk_count[idx] = len(rec_spikes)
        spk_count_dc[idx] = len(rec_sp_dc)

    plot_IF_curve(spk_count, spk_count_dc, I_mean, noise)

print('Interactive demo initiated at ' + str(datetime.now(timezone(-timedelta(hours=5)))))
```

### Interactive Demo 4.3: Effect of noise on a neuron's spiking regularity

([Return to Table of Contents](#scrollTo=V7gPh3_ZJeRD))

Membrane current noise can also change the irregularity of the spiking activity, quantified by the Coefficient of Variation of the inter-spike-interval ($CV_{ISI}$). A neurons's "inter-spike-interval" (ISI) is the time between spikes (and the inverse of its spike rate). The $CV_{ISI}$ is calculated by:

\begin{align}
\text{CV}_{\text{ISI}} = \frac{std(\text{ISI})}{mean(\text{ISI})}
\end{align}

For a clocklike (regular) ISI, $\text{CV}_{\textbf{ISI}} \text{= 0}$ because std(ISI)=0.

1. Is there a difference between the effect of small injected current amplitude and high injected current amplitude and the CV? How does this difference depend on the magnitude of membrane current noise? Why do you think that is?
2. Why does increasing the amplitude of current injection decrease the $CV_{ISI}$?
3.  If you plot spike count (or rate) vs. $CV_{ISI}$, should there be a relationship between the two?

```python
#@markdown **TASK:** Run this cell to make an interactive plot
#@markdown that enables you to explore these questions.
#@markdown <br> There is one slider that lets you control the membrane current noise.

my_layout.width = '450px'
@widgets.interact(
    noise=widgets.FloatSlider(0.0, min=0., max=5.,
                                step=0.1, layout=my_layout)
)

def diff_std_affect_fI(noise):
    pars = default_pars(T=1000.)
    I_mean = np.arange(100., 400., 20)
    spk_count = np.zeros(len(I_mean))
    cv_isi = np.empty(len(I_mean))

    for idx in range(len(I_mean)):
        I_GWN = my_GWN(pars, mu=I_mean[idx], sig=noise)
        v, rec_spikes = run_LIF(pars, Iinj=I_GWN)
        spk_count[idx] = len(rec_spikes)
        if len(rec_spikes) > 3:
            isi = np.diff(rec_spikes)
            cv_isi[idx] = np.std(isi) / np.mean(isi)

    # Plot the F-I curve i.e. Output firing rate as a function of input mean.
    plt.figure()
    plt.clf()
    plt.plot(I_mean[spk_count > 5], cv_isi[spk_count > 5], 'bo', alpha=0.5)
    plt.xlabel('Average injected current (pA)',fontsize=14)
    plt.xticks(fontsize=14)
    plt.ylabel(r'Spike irregularity ($\mathrm{CV}_\mathrm{ISI}$)',fontsize=14)
    plt.yticks(fontsize=14)
    plt.ylim(-0.1, 1.5)
    plt.xlim(50, 400)
    plt.grid(True)
    plt.show()

print('Interactive demo initiated at ' + str(datetime.now(timezone(-timedelta(hours=5)))))
```

---
# Summary

Congratulations! You've just used a classic integrate-and-fire (LIF) neuron model simulation to study the dynamics of neural activity. You drove the LIF neuron with external inputs, such as direct current and Gaussian white noise. You examined how different inputs affect the  LIF neuron's output (firing rate and spike time irregularity). Throughout the course, think about what types of neural coding are required by various neural circuits for motor control. You can now start to relate neural coding properties back to the basic intrinsic physiology of neurons. There are so many other anatomical and physiological properties that effect neural coding that the ones you examined here. This is just scratching the surface on neural complexity in circuits, coding and motor control.

---

Created by Dr. Krista Perks for a Motor Systems course (BIOL358) at Wesleyan University based on 'Neuromatch Academy" Week 2, Day 3: Biological Neuron Models

[Original Notebook Tutorial](https://colab.research.google.com/github/NeuromatchAcademy/course-content/blob/master/tutorials/W2D3_BiologicalNeuronModels/student/W2D3_Tutorial1.ipynb) Information:

__Content creators:__ Qinglong Gu, Songtin Li, John Murray, Richard Naud, Arvind Kumar

__Content reviewers:__  Maryam Vaziri-Pashkam, Ella Batty, Lorenzo Fontolan, Richard Gao, Matthew Krause, Spiros Chavlis, Michael Waskom, Ethan Cheng


# Double pendulum using Lagrange's equation

Defines a LagrangianPendulum class that is used to generate double pendulum plots from solving Lagrange's equations.

## Euler-Lagrange equation

For a simple pendulum, the Lagrangian with generalized coordinate $\phi$ is

$\begin{align}
  \mathcal{L} = \frac12 m L^2 \dot\phi^2 - mgL(1 - \cos\phi)
\end{align}$

The Euler-Lagrange equation is

$\begin{align}
 \frac{d}{dt}\frac{\partial\mathcal{L}}{\partial \dot\phi} = \frac{\partial\mathcal L}{\partial\phi}
 \quad\Longrightarrow\quad
 m L^2 \ddot \phi = -mgL\sin\phi
  \ \mbox{or}\ \ddot\phi = - \omega_0^2\sin\phi = 0
  \;.
\end{align}$

## Hamilton's equations

The generalized momentum corresponding to $\phi$ is

$\begin{align}
   \frac{\partial\mathcal{L}}{\partial \dot\phi} = m L^2 \dot\phi \equiv p_\phi
   \;.
\end{align}$

We can invert this equation to find $\dot\phi = p_\phi / m L^2$.
Constructing the Hamiltonian by Legendre transformation we find

$\begin{align}
  \mathcal{H} &= \dot\phi p_\phi  - \mathcal{L} \\
              &= \frac{p_\phi^2}{m L^2} - \frac12 m L^2 \dot\phi^2 + mgL(1 - \cos\phi) \\
              &= \frac{p_\phi^2}{2 m L^2} + mgL(1 - \cos\phi)
              \;.
\end{align}$

Thus $\mathcal{H}$ is simply $T + V$.  Hamilton's equations are

$\begin{align}
  \dot\phi &= \frac{\partial\mathcal{H}}{\partial p_\phi} = \frac{p_\phi}{m L^2} \\
  \dot p_\phi &= -\frac{\partial\mathcal{H}}{\partial \phi} = -mgL \sin\phi
  \;.
\end{align}$

```python
%matplotlib inline
```

```python
import numpy as np
from scipy.integrate import odeint, solve_ivp

import matplotlib.pyplot as plt
```

```python
# The dpi (dots-per-inch) setting will affect the resolution and how large
#  the plots appear on screen and printed.  So you may want/need to adjust
#  the figsize when creating the figure.
plt.rcParams['figure.dpi'] = 100.    # this is the default for notebook

# Change the common font size (smaller when higher dpi)
font_size = 10
plt.rcParams.update({'font.size': font_size})
```

## Double Pendulum class and utility functions

```python
class DoublePendulum():
    """
    Pendulum class implements the parameters and Lagrange's equations for
     a simple pendulum (no driving or damping).

    Parameters
    ----------
    L1 : float
        length of the 1st pendulum
    L2 : float
        length of the 2nd pendulum
    g : float
        gravitational acceleration at the earth's surface
    m1 : float
        mass of 1st pendulum
    m2 : float
        mass of 2nd pendulum

    Methods
    -------
    dy_dt(t, y)
        Returns the right side of the differential equation in vector y,
        given time t and the corresponding value of y.
    """
    def __init__(self, L1=1., L2=1., mass1=1., mass2=1., g=1.):
        self.L1 = L1
        self.L2 = L2
        self.g = g
        self.m1 = mass1
        self.m2 = mass2

    def dy_dt(self, t, y):
        """
        This function returns the right-hand side of the diffeq:
        [dphi/dt d^2phi/dt^2]

        Parameters
        ----------
        t : float
            time
        y : float
            A 4-component vector with y[0] = theta1 and y[1] = theta1dot
            y[2] = theta2 and y[3] = theta2dot

        Returns
        -------

        """
        theta1, z1, theta2, z2 = y

        c, s = np.cos(theta1 - theta2), np.sin(theta1 - theta2)

        d = (self.m1 + self.m2 * s**2)

        theta1_dot = z1

        z1_dot = ( self.m2 *self.g* np.sin(theta2)*c - self.m2 * s * (self.L1 * z1**2 * c + self.L2 * z2**2 ) - (self.m1 + self.m2) * self.g * np.sin(theta1)) /(self.L1 * d)

        theta2_dot = z2

        z2_dot = ( (self.m1 + self.m2)*(self.L1 * z1**2*s - self.g*np.sin(theta2) + self.g * np.sin(theta1) * c ) + self.m2*self.L2*z2**2*s*c)/(self.L2*d)

        return theta1_dot, z1_dot, theta2_dot, z2_dot

    def solve_ode(self, t_pts, theta1_0, theta1_dot_0, theta2_0, theta2_dot_0,
                  abserr=1.0e-9, relerr=1.0e-9):
        """
        Solve the ODE given initial conditions.
        Specify smaller abserr and relerr to get more precision.
        """
        y = [theta1_0, theta1_dot_0, theta2_0, theta2_dot_0]
        solution = solve_ivp(self.dy_dt, (t_pts[0], t_pts[-1]),
                             y, t_eval=t_pts,
                             atol=abserr, rtol=relerr)
        theta1, theta1_dot, theta2, theta2_dot = solution.y

        return theta1, theta1_dot, theta2, theta2_dot

    def calc_E(y):

        th1, th1d, th2, th2d = y.T
        V = -(self.m1 + self.m2) * self.L1 * self.g * np.cos(th1) - self.m2*self.L2*self.g*np.cos(th2)
        T = 0.5*self.m1*(self.L1*th1d)**2 + 0.5*self.m2*((self.L1*th1d)**2 + (self.L2*th2d)**2 + 2*self.L1*self.L2*th1d*th2d*np.cos(th1-th2))
        return T + V
```

```python
def plot_y_vs_x(x, y, axis_labels=None, label=None, title=None,
                color=None, linestyle=None, semilogy=False, loglog=False,
                ax=None):
    """
    Generic plotting function: return a figure axis with a plot of y vs. x,
    with line color and style, title, axis labels, and line label
    """
    if ax is None:        # if the axis object doesn't exist, make one
        ax = plt.gca()

    if (semilogy):
        line, = ax.semilogy(x, y, label=label,
                            color=color, linestyle=linestyle)
    elif (loglog):
        line, = ax.loglog(x, y, label=label,
                          color=color, linestyle=linestyle)
    else:
        line, = ax.plot(x, y, label=label,
                    color=color, linestyle=linestyle)

    if label is not None:    # if a label if passed, show the legend
        ax.legend()
    if title is not None:    # set a title if one if passed
        ax.set_title(title)
    if axis_labels is not None:  # set x-axis and y-axis labels if passed
        ax.set_xlabel(axis_labels[0])
        ax.set_ylabel(axis_labels[1])

    return ax, line
```

```python
def start_stop_indices(t_pts, plot_start, plot_stop):
    start_index = (np.fabs(t_pts-plot_start)).argmin()  # index in t_pts array
    stop_index = (np.fabs(t_pts-plot_stop)).argmin()  # index in t_pts array
    return start_index, stop_index
```

```python
def start_stop_indices(t_pts, plot_start, plot_stop):
    start_index = (np.fabs(t_pts-plot_start)).argmin()  # index in t_pts array
    stop_index = (np.fabs(t_pts-plot_stop)).argmin()  # index in t_pts array
    return start_index, stop_index
```

```python

```

## Make double pendulum plots

```python
# Labels for individual plot axes
theta_vs_time_labels = (r'$t$', r'$\theta(t)$')

# Common plotting time (generate the full time then use slices)
t_start = 0.
t_end = 50.
delta_t = 0.001

t_pts = np.arange(t_start, t_end+delta_t, delta_t)

L1 = 1.
L2 = 1.
g = 1.
mass1 = 1.
mass2 = 1.

# Instantiate a pendulum
dp1 = DoublePendulum(L1=L1, L2=L2,mass1=mass1, mass2=mass2, g=g)

```

```python
# both plots: same initial conditions
theta1_0 = np.pi/2
theta1_dot_0 = 0.
theta2_0 = np.pi
theta2_dot_0 = 0.

theta1, theta1_dot, theta2, theta2_dot = dp1.solve_ode(t_pts, theta1_0, theta1_dot_0, theta2_0, theta2_dot_0)

# start the plot!
fig = plt.figure(figsize=(10,5))
overall_title = 'Double pendulum from Lagrangian:  ' + \
                rf' $\theta_1(0) = {theta1_0:.2f},$' + \
                rf'  $\dot\theta_1(0) = {theta1_dot_0:.2f},$' + \
                rf' $\theta_2(0) = {theta2_0:.2f}$' + \
                rf'  $\dot\theta_2(0) = {theta2_dot_0:.2f},$'   # \n means a new line (adds some space here)
fig.suptitle(overall_title, va='baseline')

# first plot: phi plot
ax_a = fig.add_subplot(1,1,1)

start, stop = start_stop_indices(t_pts, t_start, t_end)
plot_y_vs_x(t_pts[start : stop], theta1[start : stop],
            axis_labels=theta_vs_time_labels,
            color='blue',
            label=r'$\theta_1(t)$',
            ax=ax_a)

plot_y_vs_x(t_pts[start : stop], theta2[start : stop],
            axis_labels=theta_vs_time_labels,
            color='red',
            label=r'$\theta_2(t)$',
            ax=ax_a)

fig.tight_layout()
fig.savefig('double_pendulum_Lagrange.png', bbox_inches='tight')
```

```python
theta1_0 = np.pi/2
theta1_dot_0 = 0.
theta2_0 = np.pi
theta2_dot_0 = 0.

theta1, theta1_dot, theta2, theta2_dot = dp1.solve_ode(t_pts, theta1_0, theta1_dot_0, theta2_0, theta2_dot_0)

theta1b, theta1b_dot, theta2b, theta2b_dot = dp1.solve_ode(t_pts, theta1_0, theta1_dot_0, theta2_0 - 0.00001, theta2_dot_0)

fig = plt.figure(figsize=(10,5))
overall_title = 'Double pendulum from Lagrangian:  ' + \
                rf' $\theta_1(0) = {theta1_0:.2f},$' + \
                rf'  $\dot\theta_1(0) = {theta1_dot_0:.2f},$' + \
                rf' $\theta_2(0) = {theta2_0:.2f}$' + \
                rf'  $\dot\theta_2(0) = {theta2_dot_0:.2f},$'   # \n means a new line (adds some space here)
fig.suptitle(overall_title, va='baseline')

ax_a = fig.add_subplot(1,1,1)

start, stop = start_stop_indices(t_pts, t_start, t_end)
plot_y_vs_x(t_pts[start : stop], np.abs(theta1[start : stop]- theta1b[start:stop]),
            axis_labels=theta_vs_time_labels,
            color='blue', semilogy = True,
            label=r'$\Delta\theta_1(t)$',
            ax=ax_a)

plot_y_vs_x(t_pts[start : stop], np.abs(theta2[start : stop]- theta2b[start:stop]),
            axis_labels=theta_vs_time_labels,
            color='red', semilogy = True,
            label=r'$\Delta\theta_2(t)$',
            ax=ax_a)

ax_a.set_ylim(1.e-7, 1.e2)

fig.tight_layout()
fig.savefig('double_pendulum_Lagrange.png', bbox_inches='tight')

```

The difference in theta values are going up as time passes. The Liapunov exponent which describes this trend of increase is increasing which shows that the motion of this double pendulum is chaotic.

```python

```


### Tracing test

```python
import numpy as np
import matplotlib.pyplot as plt
from ttim import *
```

Start well with `tsandQ=[(0, -Q), (100, Q), (200, -Q)]`

```python
Q = 100
npor = 0.3
k = 10
H = 10
T = k * H
Ss = 1e-4
S = Ss * H
ml = ModelMaq(kaq=[k], z=[H, 0], Saq=[Ss], tmin=0.01, tmax=1000, M=10)
w = Well(ml, xw=0, yw=0, tsandQ=[(0, -Q), (100, Q), (200, -Q)], rw=0.3)
ml.solve()
trace = timtrace(ml, xstart=0.3, ystart=0, zstart=0,
                 tstartend=[0, 100, 200, 400], tstartoffset=0.01, deltlist=10,
                 nstepmax=120, hstepmax=2, correctionstep=True)
xyzt = trace['trace']
```

    self.neq  1
    solution complete
    reached maximum time tmax
    reached maximum time tmax
    reached maximum time tmax

```python
plt.plot(xyzt[:, 3], xyzt[:, 0])
plt.grid()
```

```python
trace['status'], trace['message']
```

    ([1, 1, 1],
     ['reached maximum time tmax',
      'reached maximum time tmax',
      'reached maximum time tmax'])

#### Terminates at well

```python
ml = ModelMaq(kaq=[k], z=[H, 0], Saq=[Ss], tmin=0.01, tmax=1000, M=10)
w = Well(ml, xw=0, yw=0, tsandQ=[(0, -Q), (100, Q)], rw=0.3, label='testwell')
ml.solve()
trace = timtrace(ml, xstart=0.3, ystart=0, zstart=0,
                 tstartend=[0, 100, 300], tstartoffset=0.01, deltlist=10,
                 nstepmax=120, hstepmax=2, correctionstep=True)
xyzt = trace['trace']
print(trace['status'], trace['message'])
print(xyzt[-3:])
```

    self.neq  1
    solution complete
    reached maximum time tmax
    reached well element with label: testwell
    [1, 2] ['reached maximum time tmax', 'reached well element with label: testwell']
    [[  3.8943918    0.           0.         201.42194579]
     [  1.8943918    0.           0.         202.38286201]
     [  0.           0.           0.         203.05932345]]

#### Compare to numerical integration of velocity from Theis equation
\begin{equation}
Q_r = -T\frac{\partial h}{\partial r} = -T\frac{\text{d} h}{\text{d} u}\frac{\partial u}{\partial r} = -\frac{Q}{2\pi} \frac{1}{r}\exp(-u)
\end{equation}
\begin{equation}
u =  \frac{S r^2}{4T(t-t_0)}
\end{equation}

```python
def vxytheis(t, xy):
    Q = -100
    npor = 0.3
    k = 10
    H = 10
    Ss = 1e-4
    T = k * H
    S = Ss * H
    x, y = xy
    r = np.sqrt(x ** 2 + y ** 2)
    u = S * r ** 2 / (4 * T * t)
    Qr = -Q / (2 * np.pi) / r * np.exp(-u)
    vr = Qr / (H * npor)
    vx = vr * x / r
    vy = vr * y / r
    return np.array([vx, vy])
```

```python
from scipy.integrate import solve_ivp
t = np.linspace(0.01, 100, 100)
path0 = solve_ivp(vxytheis, (0.01, 100), y0=[0.3, 0], t_eval=t)
plt.plot(path0.t, path0.y[0]);
```

#### Velocities in multi-layer system

```python
Q = 100
npor = 0.3
k0 = 10
k1 = 40
H = 10
Hstar = 2
c = 100
T = k * H
Ss = 1e-4
S = Ss * H
ml = ModelMaq(kaq=[k0, k1], z=[2 * H + Hstar, H + Hstar, H, 0], c=[c], Saq=[Ss], tmin=0.01, tmax=10000, M=10)
w = Well(ml, xw=0, yw=0, tsandQ=[(0, -Q)], layers=0, rw=0.3)
ml.solve()
```

    self.neq  1
    solution complete

```python
for z in np.arange(13, 22):
    trace = timtrace(ml, xstart=0.3, ystart=0, zstart=z,
                     tstartend=[0, 1000], tstartoffset=0.01, deltlist=10,
                     nstepmax=500, hstepmax=2, correctionstep=True, silent=True)
    xyzt = trace['trace']
    plt.plot(xyzt[:, 0], xyzt[:, 2])
```

#### Model3D

```python
Q = 100
npor = 0.3
k = 10
H = 10
Hstar = 2
c = 1000
T = k * H
#Ss = 1e-4
Ss = 1e-8
S = Ss * H
ml = Model3D(kaq=10, z=[20, 15, 10, 5, 0], tmin=0.01, tmax=1000)
w = Well(ml, xw=0, yw=0, tsandQ=[(0, -Q), (100, Q)], layers=1, rw=0.1)
ml.solve()
```

    self.neq  1
    solution complete

```python
zstart = np.linspace(10.01, 14.99, 31)
for zs in zstart:
    trace = timtrace(ml, xstart=0.1, ystart=.1, zstart=zs,
                     tstartend=[0, 100], tstartoffset=0.01, deltlist=5,
                     nstepmax=40, hstepmax=2, silent=True)
    xyzt = trace['trace']
    plt.plot(xyzt[:, 0], xyzt[:, 2])
for y in [5, 10, 15]:
    plt.axhline(y, color='k')
plt.axis('scaled');
```

```python
zstart = np.linspace(10.01, 14.99, 31)
for zs in zstart:
    trace = timtrace(ml, xstart=0.1, ystart=0.1, zstart=zs,
                     tstartend=[0, 100, 200], tstartoffset=0.01, deltlist=5,
                     nstepmax=40, hstepmax=2, silent=True, correctionstep=True)
    xyzt = trace['trace']
    plt.plot(xyzt[:, 0], xyzt[:, 2])
for y in [5, 10, 15]:
    plt.axhline(y, color='k')
plt.axis('scaled');
```


```python
# ML_in_Finance-RNNs-HFT
# Author: Matthew Dixon
# Version: 1.1 (27.2.2020)
# License: MIT
# Email: matthew.dixon@iit.edu
# Notes: tested on Mac OS X running Python 3.6.9 with the following packages:
# tensorflow=2.0.0, keras=2.3.1, scikit-learn=0.22.1, numpy=1.18.1, matplotlib=3.1.3, pandas=1.0.3, statsmodels=0.10.1
# Citation: Please cite the following reference if this notebook is used for research purposes:
# Bilokon P., Dixon M.F. and Halperin I., Machine Learning in Finance: From Theory to Practice, Springer Graduate Textbook Series, 2020.
```

## An Introduction to Prediction with RNNs

### Overview
- This notebook provides an example of how Keras can be used to train and test TensorFlow RNNs for time series prediction. The example dataset is for predicting from noisy, non-stationary data.
- Statistical methods used for autoregressive models shall be used to identify the sequence length needed in the RNN and to diagnose the model error.
- Plain RNNs are not suited to non-stationary time series modeling. We can use a GRU or LSTM to model non-stationary data, since these models exhibit dynamic auto-correlation structure.
- Unlike classical time series methods, e.g. ARIMA, there are no parametric assumptions on the distribution of the errors, and non-linear relationships between response and predictors can be captured.

#### Statistician's note
- We choose to build a model which provides strong predictive power, at the expense of reduced explanatory power.
- Our choice to use a recurrent neural network is predicated on each observation in the time series being dependent on previous observations. The ordering of the observations therefore matters and $X$ is not iid.
- Once the input data is appropriately scaled, model building starts with 'feature selection' - identifying the relevant features to include in the model.

- In this notebook, we assume that we've already identifed the relevant set of features (i.e. there is only one time series provided).
- Our primary concern is assessing the extent to which the model is over-fitting, by comparing the in- and out-of-sample MSEs.

#### Implementation notes
- It is important to ensure that `shuffle=False` in the fit function, otherwise the ordering of sequences is not preserved. This is especially important for methods which have memory beyond the current sequence (i.e. all methods except RNNs).
- Time series cross-validation must be used for hyper-parameter tuning because the ordering of the data matters. In particular, the model must never use training data more recent than the forecasting observation date.

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import statsmodels.api as sm
import tensorflow as tf

from sklearn.metrics import mean_squared_error
from sklearn.model_selection import KFold, TimeSeriesSplit, GridSearchCV

import keras.initializers
from keras.layers import Dense, Layer, LSTM, GRU, SimpleRNN, RNN
from keras.models import Sequential
from keras.models import load_model
from keras.regularizers import l1, l2
from keras.callbacks import EarlyStopping
from keras.wrappers.scikit_learn import KerasRegressor

# note that the directory containing these two .py's must be in the path variable:
from alphaRNN import AlphaRNN
from alphatRNN import AlphatRNN

```

    /Users/dez/opt/miniconda3/envs/MLFenv/lib/python3.6/site-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
      import pandas.util.testing as tm
    Using TensorFlow backend.

```python
%matplotlib inline
```

### Example Data
- The example dataset is a chronologically ordered time series. The ordering of the observations matters and each observation is not assumed to be independent (as with cross-sectional classification data).

- For simplicity, since the timestamps are not important for this tutorial, an integer index has been used for the Pandas Dataframe. A real dataset would typically be indexed by a unique timestamp. The observations are not uniform and in general are less than a micro-second apart.

- Each observation $X$ has three variables (a.k.a. features). Feature 3 is the smart price (VWAP) and the label indicates whether the book will up-tick, stay flat or down-tick in the next time interval.

Loading the Pandas Dataframe, viewing the first ten rows and the distribution of the labels:

```python
df = pd.read_csv('../data/HFT.csv')
```

```python
df.head(10)
```

      feature_1
      feature_2
      feature_3
      label

      0
      0.515301
      0.72
      0.710953
      0

      1
      0.515301
      0.72
      0.710953
      0

      2
      0.515301
      0.72
      0.710953
      0

      3
      0.515301
      0.72
      0.710953
      0

      4
      0.515301
      0.72
      0.710953
      0

      5
      0.515301
      0.72
      0.710953
      0

      6
      0.515301
      0.72
      0.710953
      0

      7
      0.515301
      0.72
      0.710953
      0

      8
      0.515301
      0.72
      0.710953
      0

      9
      0.515301
      0.72
      0.710953
      0

# RNN Regression
We consider a univariate prediction problem where the time series is given by 'feature_3' in the data frame, and for each input sequence we predict the value 10 time-steps into the future.

```python
use_features = ['feature_3'] # continuous input
target = ['feature_3'] # continuous output
n_steps_ahead = 10 # forecasting horizon
```

### Stationarity
It is essential to determine whether the time series is "stationary". Informally, stationarity is when the auto-covariance is independent of time. Failure to establish stationarity will almost certainly lead to misinterpretation of model identification and diagnostics tests. Moreover, stationarity is decisive in characterizing the prediction problem and whether to use a more advanced architecture. In particular, we can expect a plain RNN to perform poorly if the data is non-stationary as the RNN exhibits fixed auto-covariance.

We perform an Augmented Dickey-Fuller test to establish stationarity under the assumption that the time series has a constant bias but does not exhibit a time trend. In other words, we assume that the time series is already de-trended.

If the stationarity test fails, even after first de-trending the time series, then one potential recourse is to simply take differences of time series and predict $\Delta y_t$.

The null hypothesis of the Augmented Dickey-Fuller is that there is a unit root, with the alternative that there is no unit root. If the p-value is above $(1-\alpha)$, then we cannot reject that there is a unit root. Note that a subset of the time series is used to reduce the computation time of the test.

```python
sample = df['feature_3'][:200000]
adf, p, usedlag, nobs, cvs, aic = sm.tsa.stattools.adfuller(sample)
```

```python
adf_results_string = 'ADF: {}\np-value: {},\nN: {}, \ncritical values: {}'
print(adf_results_string.format(adf, p, nobs, cvs))
```

    ADF: -1.2215995010265421
    p-value: 0.6642358271469121,
    N: 1995,
    critical values: {'1%': -3.4336320721769433, '5%': -2.862989840784964, '10%': -2.56754183359401}

#### Autoregressive Model Identification: The partial auto-correlation
It is important to determine the number of lags, the sequence length, required in the RNN by statistical analysis. A brute-force approach will in general be too time-consuming.

A partial auto-correlation at lag $h\geq 2$ is a conditional auto-correlation between a variable, $X_t$, and its $h^{th}$ lag, $X_{t-h}$ under the assumption that we control for the values of the intermediate lags, $X_{t-1},\dots, X_{t-h+1}$:

$$\begin{align}\tau_h&:=\tau(X_t, X_{t-h}; X_{t-1},\dots, X_{t-h+1})\\
&:=\frac{\gamma(X_t, X_{t-h}; X_{t-1},\dots, X_{t-h+1})}{\sqrt{\gamma(X_t |X_{t-1},\dots, X_{t-h+1})\gamma(X_{t-h} |X_{t-1},\dots, X_{t-h+1}))}}
,\end{align}$$
where $\gamma_h:=\gamma(X_tX_{t-h})$ is the lag-$h$ autocovariance. The partial autocorrelation function $\tau_h:\mathbb{N} \rightarrow [-1,1]$ is a map $h:\mapsto \tau_h$.

The estimated partial auto-correlation function (PACF) can be used to identify the order of an autoregressive time series model. Values of $|\tau_h|$ greater or equal to $\frac{\Phi^{-1}(\alpha)}{\sqrt{T}}$, where $T$ is the number of observations and $\Phi(z)$ is the standard normal CDF, are significant lag $h$ partial autocorelations at the $\alpha$ confidence level.

We use the stattools package to estimat the PACF. The `nlags` parameter is the maximum number of lags used for PACF estimation.

```python
pacf = sm.tsa.stattools.pacf(df[use_features], nlags=30)
```

Since $\Phi^{-1}(0.99) \simeq 2.58$:

```python
T = len(df[use_features])

sig_test = lambda tau_h: np.abs(tau_h) > 2.58/np.sqrt(T)
```

We find the first lag which isn't significant at the 99% level and automatically determine the number of lags needed in our autoregressive model as one below this value.

```python
for i in range(len(pacf)):
    if sig_test(pacf[i]) == False:
        n_steps = i - 1
        print('n_steps set to', n_steps)
        break
```

    n_steps set to 23

This may lead to a high order model, with more lags than strictly necessary. We could view this value, informally, as an upper bound on the number of lags needed. We can also simply identify the order of the model based on the plot of the PACF. In this case, a minimum of 2 lags appears satisfactory, although more may be needed. Unlike autoregressive models, the advantage of using fewer parameters is purely computational as adding more lags does not increase the number of parameters, only the size of the tensorial representation of the sequence data in TensorFlow.

```python
plt.plot(pacf, label='pacf')
plt.plot([2.58/np.sqrt(T)]*30, label='99% confidence interval (upper)')
plt.plot([-2.58/np.sqrt(T)]*30, label='99% confidence interval (lower)')
plt.xlabel('number of lags')
plt.xticks(np.arange(0, 30, 2))
plt.legend();
```

### Splitting the time series into training and testing sets
Split the training and test set by using the first 80% of the time series and the remaining 20% for the test set. Note that the test set must be in the future of the training set to avoid look-ahead bias. Also, random sampling of the data can not be used as this would eliminate the auto-correlation structure.

```python
train_weight = 0.8
split = int(len(df)*train_weight)

df_train = df[use_features].iloc[:split]
df_test = df[use_features].iloc[split:]
```

### Scaling
Standardization of the data is important to avoid potential scaling difficulties in the fitting of the model. When there is more than one feature (covariate), scaling avoids one feature dominating over another due to disparate scales.

To avoid introducing a look-ahead bias into the prediction, we must re-scale the training data without knowledge of the test set. Hence, we will simply standardize the training set using the mean and standard deviation of the training set and not the whole time series. Additionally, to avoid introducing a systematic bias into test set, we use the identical normalization for the test set - the mean and standard deviation of the training set are used to normalize the test set.

```python
# note that for a multivariate time series, you would need to scale
# each variable by its own mean and standard deviation in the training set
mu = np.float(df_train.mean())
sigma = np.float(df_train.std())

stdize_input = lambda x: (x - mu) / sigma

df_train = df_train.apply(stdize_input)
df_test = df_test.apply(stdize_input)
```

### Data formatting for RNNs

TensorFlow uses tensors to represent data. To perform sequence learning, the time series of variables must be transformed to a series of over-lapping sub-sequences.

For example, consider a univariate time series of increasing integers.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Setting the sequence length to 10, for example, we move the window forward by one observation at a time and construct new sequences:

1 2 3 4 5 6 7 8 9 10

2 3 4 5 6 7 8 9 10 11

3 4 5 6 7 8 9 10 11 12

4 5 6 7 8 9 10 11 12 13

5 6 7 8 9 10 11 12 13 14

6 7 8 9 10 11 12 13 14 15

Let's define the following function for reshaping the data into one-step ahead times series prediction format.

```python
def get_lagged_features(df, n_steps, n_steps_ahead):
    """
    df: pandas DataFrame of time series to be lagged
    n_steps: number of lags, i.e. sequence length
    n_steps_ahead: forecasting horizon
    """
    lag_list = []
    for lag in range(n_steps + n_steps_ahead - 1, n_steps_ahead - 1, -1):
        lag_list.append(df.shift(lag))
    lag_array = np.dstack([i[n_steps+n_steps_ahead-1:] for i in lag_list])
    # We swap the last two dimensions so each slice along the first dimension
    # is the same shape as the corresponding segment of the input time series
    lag_array = np.swapaxes(lag_array, 1, -1)
    return lag_array
```

We shall first transform the training input and output data.

```python
x_train = get_lagged_features(df_train[use_features], n_steps, n_steps_ahead)
y_train =  df_train[target].values[n_steps + n_steps_ahead - 1:]

x_test = get_lagged_features(df_test[use_features], n_steps, n_steps_ahead)
y_test =  df_test[target].values[n_steps + n_steps_ahead - 1:]
```

Print the shapes of each tensor. The first digit is the number of observations. For feature arrays, the second digit is the sequence length (i.e. the number of lags in the model) and the final digit is the dimension of each element in the sequence or output vector respectively. In this case, the example performs univariate time series analysis and so the dimension of the input and output is 1.

```python
print([tensor.shape for tensor in (x_train, y_train, x_test, y_test)])
```

    [(826793, 23, 1), (826793, 1), (206675, 23, 1), (206675, 1)]

### Model Specification

```python
def AlphatRNN_(n_units = 10, l1_reg=0, seed=0):
  model = Sequential()
  model.add(AlphatRNN(n_units, activation='tanh', recurrent_activation='sigmoid', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True))
  model.add(Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model

def AlphaRNN_(n_units = 10, l1_reg=0, seed=0):
  model = Sequential()
  model.add(AlphaRNN(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True))
  model.add(Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model

def SimpleRNN_(n_units = 10, l1_reg=0, seed=0):
  model = Sequential()
  model.add(SimpleRNN(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True, stateful=False))
  model.add(Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model

def GRU_(n_units = 10, l1_reg=0, seed=0):
  model = Sequential()
  model.add(GRU(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True))
  model.add(Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model

def LSTM_(n_units = 10, l1_reg=0, seed=0):
  model = Sequential()
  model.add(LSTM(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True))
  model.add(Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model
```

Use a batch size of 1000 as the dataset is reasonably large and the training time would be too long otherwise. 2000 epochs have been used here, but a better approach would be to use a stopping criteria through a call back.

```python
max_epochs = 2000
batch_size = 1000
```

```python
es = EarlyStopping(monitor='loss', mode='min', verbose=1, patience=50, min_delta=3e-5, restore_best_weights=True)
```

```python
params = {
    'rnn': {
        'model': None, 'function': SimpleRNN_, 'l1_reg': 0.0, 'H': 20,
        'color': 'blue', 'label':'RNN'},
    'alpharnn': {
        'model': None, 'function': AlphaRNN_, 'l1_reg': 0.0, 'H': 10,
        'color': 'green', 'label': '$\\alpha$-RNN' },
    'alphatrnn': {
        'model': None, 'function': AlphatRNN_, 'l1_reg': 0.0, 'H': 5,
        'color': 'cyan', 'label': '$\\alpha_t$-RNN'},
    'gru': {
        'model': None, 'function': GRU_,'l1_reg': 0.0, 'H': 10,
        'color': 'orange', 'label': 'GRU'},
    'lstm': {
        'model': None, 'function': LSTM_, 'l1_reg': 0.0, 'H': 10,
        'color':'red', 'label': 'LSTM'}
}
```

# Optionally, load pre-trained models
Training the models takes several hours. To save time, you may load the already fitted models instead:

```python
do_training = False # Set to False if you wish to train the models
```

```python
if do_training is False:
    custom_objects = {'AlphaRNN': AlphaRNN, 'AlphatRNN': AlphatRNN}
    for key in params.keys():
        params[key]['model'] = load_model('trained-RNNs/RNNs-HFT-' + key + '.hdf5', custom_objects=custom_objects)
```

# Cross-validation

The cell below performs a grid search to optimise the `n_units` and `l1_reg` for each of the models.

The results are cross-validated to avoid over-fitting. Scikit-Learn's `TimeSeriesSplit` function is used to partition the data into 5 pairs of training and testing sets, where the test data is always ahead of the training data in time.

```python
cross_val = False # WARNING: Changing this to True will take many hours to run

if do_training and cross_val:
    n_units = [5, 10, 20]
    l1_reg = [0, 0.001, 0.01, 0.1]

    # A dictionary containing a list of values to be iterated through
    # for each parameter of the model included in the search
    param_grid = {'n_units': n_units, 'l1_reg': l1_reg}

    # In the kth split, TimeSeriesSplit returns first k folds
    # as training set and the (k+1)th fold as test set.
    tscv = TimeSeriesSplit(n_splits = 5)

    # A grid search is performed for each of the models, and the parameter set which
    # performs best over all the cross-validation splits is saved in the `params` dictionary
    for key in params.keys():
        print('Performing cross-validation. Model:', key)
        model = KerasRegressor(build_fn=params[key]['function'], epochs=max_epochs,
                               batch_size=batch_size, verbose=2)
        grid = GridSearchCV(estimator=model, param_grid=param_grid,
                            cv=tscv, n_jobs=1, verbose=1)
        grid_result = grid.fit(x_train, y_train, callbacks=[es])
        print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))

        means = grid_result.cv_results_['mean_test_score']
        stds = grid_result.cv_results_['std_test_score']
        params_ = grid_result.cv_results_['params']
        for mean, stdev, param_ in zip(means, stds, params_):
            print("%f (%f) with %r" % (mean, stdev, param_))

        params[key]['H'] = grid_result.best_params_['n_units']
        params[key]['l1_reg']= grid_result.best_params_['l1_reg']
```

# Train models with selected parameters

If the grid search was performed, the parameters `n_units` and `l1_reg` with the best cross-validated results on the training set are used. If not, the values from the initialisation of `params` above are used.

```python
if do_training is True:
    for key in params.keys():
        tf.random.set_seed(0)
        print('Training', key, 'model')
        model = params[key]['function'](params[key]['H'], params[key]['l1_reg'])
        model.fit(x_train, y_train, epochs=max_epochs, verbose=2,
                  batch_size=batch_size, callbacks=[es], shuffle=False)
        params[key]['model'] = model
```

Optionally save the fitted models:

```python
if do_training is True:
    for key in params.keys():
        params[key]['model'].save('RNNs-HFT-SAVED-' + key + '.hdf5', overwrite=True)  # creates a HDF5 file
```

### Print out the value of $\alpha \in [0,1]$ for the alpha-RNN model

```python
def sigmoid(x):
    return (1 / (1 + np.exp(-x)))
```

```python
model = params['alpharnn']['model']

names = [weight.name for layer in model.layers for weight in layer.weights]

weights = model.get_weights()

for name, weight in zip(names, weights):
    if 'alpha:0' in name:
      print("alpha = " + str(sigmoid(*weight)))
```

    alpha = 0.0632901032196725

### Prediction

```python
for key in params.keys():
    model = params[key]['model']
    model.summary()

    params[key]['pred_train'] = model.predict(x_train, verbose=1)
    params[key]['MSE_train'] = mean_squared_error(y_train, params[key]['pred_train'])

    params[key]['pred_test'] = model.predict(x_test, verbose=1)
    params[key]['MSE_test'] = mean_squared_error(y_test, params[key]['pred_test'])
```

    Model: "sequential_1"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    simple_rnn_1 (SimpleRNN)     (None, 20)                440
    _________________________________________________________________
    dense_1 (Dense)              (None, 1)                 21
    =================================================================
    Total params: 461
    Trainable params: 461
    Non-trainable params: 0
    _________________________________________________________________
    826793/826793 [==============================] - 82s 99us/step
    206675/206675 [==============================] - 22s 108us/step
    Model: "sequential_2"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    alpha_rnn_1 (AlphaRNN)       (None, 10)                121
    _________________________________________________________________
    dense_2 (Dense)              (None, 1)                 11
    =================================================================
    Total params: 132
    Trainable params: 132
    Non-trainable params: 0
    _________________________________________________________________
    826793/826793 [==============================] - 124s 150us/step
    206675/206675 [==============================] - 28s 138us/step
    Model: "sequential_3"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    alphat_rnn_1 (AlphatRNN)     (None, 5)                 80
    _________________________________________________________________
    dense_3 (Dense)              (None, 1)                 6
    =================================================================
    Total params: 86
    Trainable params: 86
    Non-trainable params: 0
    _________________________________________________________________
    826793/826793 [==============================] - 135s 163us/step
    206675/206675 [==============================] - 34s 162us/step
    Model: "sequential_2"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    gru_1 (GRU)                  (None, 10)                360
    _________________________________________________________________
    dense_2 (Dense)              (None, 1)                 11
    =================================================================
    Total params: 371
    Trainable params: 371
    Non-trainable params: 0
    _________________________________________________________________
    826793/826793 [==============================] - 194s 235us/step
    206675/206675 [==============================] - 48s 232us/step
    Model: "sequential_1"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    lstm_1 (LSTM)                (None, 10)                480
    _________________________________________________________________
    dense_1 (Dense)              (None, 1)                 11
    =================================================================
    Total params: 491
    Trainable params: 491
    Non-trainable params: 0
    _________________________________________________________________
    826793/826793 [==============================] - 214s 259us/step
    206675/206675 [==============================] - 53s 258us/step

We shall now apply the fitted RNN model to the training set and the test set, separately. We can then informally assess the extent
of over-fitting.

We observe the in-sample prediction is reliable (upper plots). Most of the out-of-sample prediction (lower plots) is also reliable although the end of the predicted sequence appears to degrade. Such a degradation indicates the prediction horizon might be too long and the model should be periodically retrained to incorporate more recent data.

```python
print('training set:', len(y_train))
print('testing set:', len(y_test))

# Upper limits for indices `l` & `u` in the cells below:
```

    training set: 826793
    testing set: 206675

```python
print(params.keys())

# Set `compare` in the cells below to a list
# containing any subset of these:
```

    dict_keys(['rnn', 'alpharnn', 'alphatrnn', 'gru', 'lstm'])

```python
max_pts = 10**4 # maximum number of  points to plot per series
compare = params.keys() # e.g. ['rnn', 'alpharnn'] or ['lstm']
l, u = (None, None) # lower and upper indices of range to plot
ds = max(1, len(y_train[l:u])//max_pts) # Downsampling ratio for less than max_pts
                                        # per series.  Set `None` to disable
fig = plt.figure(figsize=(15,8))
x_vals = np.arange(len(y_train))[l:u:ds]
for key in compare:
    y_vals = params[key]['pred_train'][l:u:ds]
    label = params[key]['label'] + ' (train MSE: %.2e)' % params[key]['MSE_train']
    plt.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)
plt.plot(x_vals, y_train[l:u:ds], c="black", label="Observed", lw=1)
plt.xlim(x_vals.min(), x_vals.max())
plt.xlabel('Time (ticks)', fontsize=14)
plt.ylabel('$\hat{Y}$', rotation=0, fontsize=14)
plt.legend(loc="best", fontsize=12)
plt.title('Observed vs Model Outputs (Training)', fontsize=16);
```

```python
compare = params.keys() # e.g. ['rnn', 'alpharnn'] or ['lstm']
l, u = (None, None) # lower and upper indices of range to plot - e.g. (None, 10000)
ds = max(1, len(y_train[l:u])//max_pts) # Downsampling to under max_pts per
                                        # series.  Set `None` to disable
fig = plt.figure(figsize=(15,8))
x_vals = np.arange(len(y_train))[l:u:ds]
for key in compare:
    y_vals = params[key]['pred_train'][l:u:ds] - y_train[l:u:ds]
    label = params[key]['label'] + ' (train MSE: %.2e)' % params[key]['MSE_train']
    plt.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)
plt.axhline(0, linewidth=0.8)
plt.xlim(x_vals.min(), x_vals.max())
plt.xlabel('Time (ticks)', fontsize=14)
plt.ylabel('$\hat{Y}-Y$', fontsize=14)
plt.legend(loc="best", fontsize=12)
plt.title('Observed vs Model Error (Training)', fontsize=16);
```

```python
compare = params.keys() # e.g. ['rnn', 'alpharnn'] or ['lstm']
l, u = (None, None) # lower and upper indices of range to plot
ds = max(1, len(y_train[l:u])//max_pts) # Downsampling to under max_pts per
                                        # series.  Set `None` to disable
fig = plt.figure(figsize=(15,8))
x_vals = len(y_train) + np.arange(len(y_test))[l:u:ds]
for key in compare:
    y_vals = params[key]['pred_test'][l:u:ds]
    label = params[key]['label'] + ' (test MSE: %.2e)' % params[key]['MSE_test']
    plt.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)
plt.plot(x_vals, y_test[l:u:ds], c="black", label="Observed", lw=1)
plt.xlim(x_vals.min(), x_vals.max())
plt.xlabel('Time (ticks)', fontsize=14)
plt.ylabel('$\hat{Y}$', rotation=0, fontsize=14)
plt.legend(loc="best", fontsize=12)
plt.title('Observed vs Model Outputs (Testing)', fontsize=16);
```

```python
compare = params.keys() # e.g. ['rnn', 'alpharnn'] or ['lstm']
l, u = (None, None) # e.g. (None, 100000) lower and upper indices of range to plot
ds = max(1, len(y_train[l:u])//max_pts) # Downsampling to under max_pts per
                                        # series.  Set `None` to disable
fig = plt.figure(figsize=(15,8))
x_vals = len(y_train) + np.arange(len(y_test))[l:u:ds]
for key in compare:
    y_vals = params[key]['pred_test'][l:u:ds] - y_test[l:u:ds]
    label = params[key]['label'] + ' (test MSE: %.2e)' % params[key]['MSE_test']
    plt.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)
plt.axhline(0, linewidth=0.8)
plt.xlim(x_vals.min(), x_vals.max())
plt.xlabel('Time (ticks)', fontsize=14)
plt.ylabel('$\hat{Y}-Y$', fontsize=14)
plt.legend(loc="best", fontsize=12)
plt.title('Observed vs Model Error (Testing)', fontsize=16);
```

### Model Diagnostics

A fitted time series model must be examined for underfitting with a white noise test. We analyze the model residuals (i.e. the error $u_t$) to determine whether it is white noise or whether it is auto-correlated. The latter case provides statistical evidence that more lags are needed in the RNN. Box and Pierce propose the Portmanteau statistic:

$$Q^*(m)=T\sum_{l=1}^m\hat{\tau}_l^2,$$
as a test statistic for the null hypothesis $H_0:\tau_1=\dots=\tau_m=0$ against the alternative hypothesis $H_a:\tau_i\neq 0$ for some $i\in\{1,\dots,m\}$, where $T$ is the number of observations, $\hat{\tau}_i$ are the sample autocorrelations of the residual, and $m$ is the maximum lag used in the test. There are several heuristics in the statistics literature to determine the maximum lag such as the Schwert statistic.

The Box-Pierce statistic follows an asymptotically chi-squared distribution with $m$ degrees of freedom.

The Ljung-Box test statistic increases the power of the test in finite samples:
$$Q(m)=T(T+2)\sum_{l=1}^m\frac{\hat{\tau}_l^2}{T-l}$$
This statistic also follows an asymptotically chi-squared distribution with $m$ degrees of freedom. The decision rule is to reject $H_0$ if $Q(m)>\chi_{\alpha}^2$ where $\chi_{\alpha}^2$ denotes the $100(1-\alpha)^{th}$ percentile of a chi-squared distribution with m degrees of freedom and is the significance level for rejecting $H_0$.

The test can be time consuming and we select a subset of the residuals. Here we simply set the maximum lag to 20. In the results below, we find that the p-values are all smaller than 0.01, indicating that we can reject the null at the 99% confidence level for any lag. This is strong evidence that the model is under-fitting and more lags are needed in our model. Unlike an auto-regressive model, increasing the number of lags in the RNN does not increase the number of weights. Thus there is no danger of over-fitting by increasing the lag, although there will be an increase in the training time.

```python
# number of samples to use for computing test statistic
n = 100000
```

```python
params.keys()
```

    dict_keys(['rnn', 'alpharnn', 'alphatrnn', 'gru', 'lstm'])

```python
key = 'alpharnn'
predicted = params[key]['pred_test']
residual = df_test[-n:] - predicted[-n:]
```

```python
lb, p = sm.stats.diagnostic.acorr_ljungbox(residual, lags=20, boxpierce=False)
```

The Box-Ljung test statistics are

```python
lb
```

    array([  92331.09457   ,  177549.28282547,  255852.83614614,
            327900.9610354 ,  394059.01818772,  454614.25222119,
            509887.97764339,  559994.43564881,  605170.23558907,
            645660.61353623,  686312.79112579,  727143.95394009,
            768151.29008262,  809223.87907208,  850262.15092409,
            891277.42521425,  932236.56495928,  973073.62128519,
           1013803.132924  , 1054290.1890087 ])

The p-values are

```python
p
```

    array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
           0., 0., 0.])