[](https://colab.research.google.com/github/MobleyLab/drug-computing/blob/master/uci-pharmsci/assignments/MD/MD.ipynb)

# Molecular Dynamics (MD)

## Objective:
Perform some basic molecular dynamics (MD) simulations on a simple polymer model, perform some initial tests, and then after equilibrating the system, compute the self-diffusion coefficient for several different chain lengths.

__Due date__: As assigned in class

## Overview:
One simple model of a polymer is just a chain of Lennard-Jones atoms. Here we will simulate such chains, interacting according to the potential (in dimensionless form):

\begin{equation}
U^* = \sum \limits_{i<j \mathrm{,\  ij\  not\  bonded}} 4\left( r_{ij}^{-12} - r_{ij}^{-6}\right) + \sum \limits_{i<j\mathrm{,\  ij\  bonded}} \frac{k}{2} \left( r_{ij} - r_0\right)^2
\end{equation}

(Note that, as in the Energy Minimization assignment, we are using the dimensionless form, so that all of the constants are hidden in the units.)

Here, atoms have Lennard-Jones attractions and repulsions, and bonds between atoms along the polymer chain(s) are represented by simple harmonic springs. There are no torsional or angle potentials, and no electrostatic interactions. However, this simple model does share basic elements with the models we still use today for proteins and small molecules -- specifically, our classical MD models today begin with the potential above and add additional terms.
Simple systems like this polymer model have been thoroughly studied as models of polyatomic molecules, and as models of short polymers. It is relatively easy to derive or determine scaling laws for various physical properties as a function of polymer length in such systems. One such study, by Reis et al., [ Fluid Phase Equilibria 221: 25 (2004) ](https://doi.org/10.1016/j.fluid.2004.04.007) evaluated the self-diffusion coefficient for chains of different lengths. (The self-diffusion coefficient measures diffusive motion of something in a solution consisting of itself, for example the self-diffusion coefficient of water in water describes how mobile a water molecule is in pure water.)
Here, you will use some Python and Fortran libraries to set up some initial test simulations and make a plot relating to equilibration. Following that, you will compute the self-diffusion coefficient as directed below, making contact with the data of Reis et al.

Most of the functions you will need have already been written for you and are provided here. Most of this assignment will involve using them to conduct a simulation. In addition to the paper mentioned, you will need `mdlib.f90` and `MD_functions.py`. As in the Energy Minimization assignment you did previously, you will need to compile `mdlib.f90` into a .so file suitable for use within Python.

## Background/settings:
### Introduction of our variables:
Here, the potential energy will be as given above. Again, note that we are working in dimensionless form.

We will simulate a system with a total of N monomers, some of which will be linked to form polymers. Each polymer will consist of M monomers, so that if $N_{poly}$ is the number of polymers, $N = M\times N_{poly}$. That is to say, we have $N_{poly}$ polymers each consisting of $M$ linked monomers in a chain, for a total of $N$ particles.

As usual, our system will have a density, $\rho$, which is N/V. We will work with a particular temperature, $T$, and cutoff distance, $R_c$, beyond which Lennard-Jones interactions will not be included. Additionally, we need to specify a bond strength and equilibrium separation, $k$ and $r_0$, respectively. And we will take timesteps $\Delta t$ using the velocity Verlet integrator.

### Settings to use (unless otherwise noted)

Unless otherwise noted, here you should use the following settings:
* $k = 3000$ (spring constant)
* $r_0 = 1$ (preferred bond length)
* $N = 240$ (number of particles)
* $\rho = N/V = 0.8$ so that $L$, the box size, is $(N/\rho)^{1/3}$
* Use $L$ as the box size in your code
* $\Delta t = 0.001$ (timestep)
* $T = 1.0$ (temperature)
* $R_c = 2.5$ (call this Cut in your code)

Our use of the dimensionless form here includes setting all particle masses to 1. Because of this, forces and accelerations are the same thing. Additionally, units can be dropped, and the Boltzmann constant is equal to 1 in these units.

### What's provided

In this case, mdlib provides almost the same CalcEnergy and CalcEnergyForces routines you used in the previous assignment (for energy minimizations). Additionally, it provides a VVIntegrate function to actually use the integrator (Velocity Verlet) to take a timestep. You should look through the Fortran code to make sure you understand what is being done and see how it connects to what we covered in lecture.

The Python syntax for using VVintegrate looks like:

`Pos, Vel, Accel, KEnergy, PEnergy = mdlib.vvintegrate( Pos, Vel, Accel, M, L, Cut, dt )`

This takes a single timestep (covering time dt) and returns the position, velocity, acceleration, kinetic energy, and potential energy.

Likewise, mdlib provides functions for calculating the potential energy, or the potential energy and forces, as:
`PEnergy = mdlib.calcenergy(Pos, M, L, Cut)`
and
`PE, Forces = mdlib.calcenergyforces(Pos, M, L, Cut, Forces)`

By way of additional background/tips, Scott Shell has some very useful [simulation best practices](https://sites.engineering.ucsb.edu/~shell/che210d/Simulation_best_practices.pdf) tips which can help with thinking through how to code up and conduct effective simulations.

## Your assignment
All, or almost all, of the functions you will need to complete this assignment are described below. But before getting to the description, I want to explain the assignment.

### Part A: Develop a simple molecular dynamics code and examine potential energy versus time for several values of M
Edit the supplied code below (or MD.py if you prefer to work with the plain text; note that I have also provided MD_functions.py which is utilized by this notebook which provides only the functions you need and not a template for the code you need to write, since this is below) to develop a simple molecular dynamics code. Most of the functions you need are already provided (see documentation below, or in MD_functions.py). But, you do need to fill in the core of two functions:

* InitVelocities(N,T): Should take N, a number of particles and a target temperature and return a velocity array (‘Vel’) which is Nx3 with mean velocity corresponding to the correct average temperature. You may wish to do this by assigning random velocities and rescaling (see below).

* RescaleVelocities(Vel, T): Re-center the velocities to zero net momentum to remove any overall translational motion (i.e., subtract off the average velocity from all particles) and then re-scale the velocities to maintain the correct temperature. This can be done by noting that the average kinetic energy of the system is related in a simple way to the effective temperature:

\begin{equation}
\frac{1}{2}\sum \limits_i m_i v_i^2 = \frac{3}{2} N k_B T
\end{equation}

The left-hand term is the kinetic energy, and here can be simplified by noting all of the masses in the system are defined to be 1. The right hand term involves the Boltzmann constant, the number of particles in the system, and the instantaneous temperature.

So, you can compute the effective temperature of your system, and translate this into a scaling factor which you can use to multiply (scale) all velocities in the system to ensure you get the correct average temperature (see http://www.pages.drexel.edu/~cfa22/msim/node33.html). **Specifically, following the Drexel page (eq. 177), compute a (scalar) constant by which you will multiply all of the velocities to ensure that the effective temperature is at the correct target value after rescaling.** To do this calculation you will need to compute the kinetic energy, which involves the sum above.

Remove translational motion, rescale the velocities, and have your function return the  updated velocity array.
Once the above two functions are written, finishing write a simple MD code using the available functions to:
* Initially place atoms on a cubic lattice with the correct box size
* Energy-minimize the initial configuration using the conjugate-gradient energy minimizer; this will help ensure the simulation doesn’t “explode” (a highly technical term meaning “crash”) when you begin MD
* Assign initial velocities and compute forces (accelerations)
* Use the velocity Verlet integrator to perform a molecular dynamics run. Rescale atomic velocities every **RescaleFreq** integration steps to achieve the target temperature T. ( You can test whether you should rescale the velocities using the modulo (remainder) operator, for example $i % RescaleFreq == 0$)
   * You might want to use RescaleFreq = 100 (for extra credit, you can try several values of RescaleFreq and explain the differences in fluctuations in the potential energy versus time that you see)

Use the settings given above for $N$, $\rho$, $T$, the timestep, and the cutoff distance.

Perform simulations for $M = 2, 4, 6, 8, 12,$ and $16$ and store the total energies versus time out to 2,000 timesteps. (Remember, $M$ controls the number of particles per polymer; you are keeping the same total number of particles in the system and changing the size of the polymers). On a single graph, plot the potential energy versus time for each of these cases (each in a different color). Turn in this graph.

Note also you can visualize, if desired, using the Python module for writing to PDB files which you saw in the Energy Minimization exercise.

### Part B: Extend your code to compute the self-diffusion coefficient as a function of chain length

Modify your MD code from above to perform a series of steps that will allow you to compute the self-diffusion coefficient as a function of chain length and determine how diffusion of polymers depends on the size of the polymer. To compute the self-diffusion coefficient, you will simply need to monitor the motion of each polymer in time.

Here, you will first perform two equilibrations at constant temperature using velocity rescaling . The first will allow the system to reach the desired temperature and forget about its initial configuration (remember, it was started on a lattice). The second will allow you to compute the average total energy of the system. Then, you will fix the total energy at this value and perform a production simulation.

Here’s what you should do:
* Following initial preparation (like above), first perform equilibration for NStepsEquil1 using velocity rescaling to target temperature T every RescaleFreq steps, using whatever value of RescaleFreq you used previously (NOT every step!)
* Perform a second equilibration for `NStepsEquil2` timesteps using velocity rescaling at the same frequency, storing energies while you do this.
* Compute the average total energy over this second equilibration and rescale the velocities to start the final phase with the correct total energy. In other words, the total energy at the end of equilibration will be slightly above or below the average; you should find the kinetic energy you need to have to get the correct total energy, and rescale the velocities to get this kinetic energy. After this you will be doing no more velocity rescaling. (Hint: You can do this final rescaling easily by computing a scaling factor, and you will probably not be using the rescaling code you use to maintain the temperature during equilibration.)
* Copy the initial positions of the particles into a reference array for computing the mean squared displacement, for example using `Pos0 = Pos.copy()`
* Perform a production run for `NStepsProd` integration steps with constant energy (NVE) rather than velocity rescaling. Periodically record the time and mean squared displacement of the atoms from their initial positions. (You will need to write a small bit of code to compute mean squared displacements, but it shouldn’t take more than a couple of lines; you may send it to me to check if you are concerned about it). The mean squared displacement is given by
\begin{equation}
\left<\left| \mathbf{r}-\mathbf{r_0} \right|^2 \right>
\end{equation}

  where the $\mathbf{r}$'s are the current and initial positions of the object in question so the mean squared
  displacement measures the square of the distance traveled for the object.

* Compute the self-diffusion coefficient, $D$. The mean squared displacement relates to the self-diffusion coefficient, $D$, in this way:
\begin{equation}
\left< \left| \mathbf{r} - \mathbf{r_0}\right| \right>^2 = 6 D t
\end{equation}

Here $D$ is the self-diffusion coefficient and t is the elapsed time. That is, the expected squared distance traveled (mean squared displacement) grows linearly with the elapsed time.

For settings, use NStepsEquil1 = 10,000 = NStepsEquil2 and NStepsProd = 100,000. (Note: You should probably do a “dry run” first with shorter simulations to ensure everything is working, as 100,000 steps might take an hour or more to run).

* Perform these runs for $M = 2, 4, 6, 8, 12,$ and $16$, storing results for each.
* Plot the mean-squared displacement versus time for each M on the same graph.
* Compute the diffusion coefficient for each $M$ from the slope of each graph and plot these fits on the same graph. You can do a linear least-squares fit easily in Numpy.

`Slope, Intercept = np.polyfit( xvals, yvals, 1)`

Plot the diffusion coefficient versus $M$ and try and see if it follows any obvious scaling law. It should decrease with increasing $M$, but with what power? (You may want to refer to the Reis et al. paper).

### What to turn in:
* Your plot of the potential energy versus time for each $M$ in Part A
* Mean-squared displacement versus time, and fit, for each $M$ in Part B, all on one plot
* The diffusion coefficient versus $M$ in Part B
* Your code for at least Part B
* Any comments you have - do you think you got it right? Why or why not? What was confusing/helpful? What would you do if you had more time?
* Clearly label axes and curves on your plots!

You can send your comments/discussion as an e-mail, and the rest of the items as attachments.

### What’s provided for you:
In this case, most of what you need is provided in the importable module `MD_functions.py` (which you can view with your favorite text editor, like `vi` or Atom), except for the functions for initial velocities and velocity rescaling -- in those cases, the shells are present below and you need to write the core (which will be very brief!).  **However, you will also need to insert the code for the `ConjugateGradient` function in `MD_functions.py`** from your work you did in the Energy Minimization assignment. If you did not do this, or did not get it correct (or if you are not certain if you did), you will need to e-mail David Mobley for solutions.

From the Fortran library `mdlib` (which you will compile as usual via `f2py3 -c -m mdlib mdlib.f90` or similar), the only new function you need is Velocity Verlet. In `MD_functions.py`, the following tools are available (this shows their documentation, not the details of the code, but you should only need to read the documentation in order to be able to use them. NOTE: No modification of these functions is needed; you only need to use them. You will only need to write `InitVelocities` and `RescaleVelocities` as described above, plus provide your previous code for `ConjugateGradient`:

## Installing Packages

***If you are running this on Google Colab, please add the installation blocks from the [getting started notebook](https://github.com/MobleyLab/drug-computing/blob/master/uci-pharmsci/Getting_Started.ipynb) or [condacolab](https://github.com/aakankschit/drug-computing/blob/master/uci-pharmsci/Getting_Started_condacolab.ipynb) here and then execute the code below***

Help on module MD:

NAME

    MD - #MD exercise template for PharmSci 175/275

FUNCTIONS

FUNCTIONS

    ConjugateGradient(Pos, dx, EFracTolLS, EFracTolCG, M, L, Cut)
        Performs a conjugate gradient search.
        Input:
            Pos: starting positions, (N,3) array
            dx: initial step amount
            EFracTolLS: fractional energy tolerance for line search
            EFracTolCG: fractional energy tolerance for conjugate gradient
            M: Monomers per polymer
            L: Box size
            Cut: Cutoff
        Output:
            PEnergy: value of potential energy at minimum
            Pos: minimum energy (N,3) position array

    InitPositions(N, L)
        Returns an array of initial positions of each atom,
        placed on a cubic lattice for convenience.
        Input:
            N: number of atoms
            L: box length
        Output:
            Pos: (N,3) array of positions

    InitVelocities(N, T)
        Returns an initial random velocity set.
        Input:
            N: number of atoms
            T: target temperature
        Output:
            Vel: (N,3) array of atomic velocities

        InstTemp(Vel)
        Returns the instantaneous temperature.
        Input:
            Vel: (N,3) array of atomic velocities
        Output:
            Tinst: float

    InstTemp(Vel):
       Returns the instantaneous temperature.
       Input:
           Vel: (N,3) array of atomic velocities
       Output:
           Tinst: float

    RescaleVelocities(Vel, T)
        Rescales velocities in the system to the target temperature.
        Input:
            Vel: (N,3) array of atomic velocities
            T: target temperature
        Output:
            Vel: same as above

    LineSearch(Pos, Dir, dx, EFracTol, M, L, Cut, Accel=1.5, MaxInc=10.0,
               MaxIter=10000)
        Performs a line search along direction Dir.
        Input:
            Pos: starting positions, (N,3) array
            Dir: (N,3) array of gradient direction
            dx: initial step amount
            EFracTol: fractional energy tolerance
            M: Monomers per polymer
            L: Box size
            Cut: Cutoff
            Accel: acceleration factor
            MaxInc: the maximum increase in energy for bracketing
            MaxIter: maximum number of iteration steps
        Output:
            PEnergy: value of potential energy at minimum along Dir
            Pos: minimum energy (N,3) position array along Dir

## Here, you should actually write your functions:

```python
import mdlib
from MD_functions import *

def InitVelocities(N, T):
    """Returns an initial random velocity set.
Input:
    N: number of atoms
    T: target temperature
Output:
    Vel: (N,3) array of atomic velocities
"""
    #WRITE THIS CODE
    #THEN RETURN THE NEW VELOCITIES
    return Vel

def RescaleVelocities(Vel, T):
    """Rescales velocities in the system to the target temperature.
Input:
    Vel: (N,3) numpy array of atomic velocities
    T: target temperature
Output:
    Vel: same as above
"""
    #WRITE THIS CODE
    #recenter to zero net momentum (assuming all masses same)
    #find the total kinetic energy
    #find velocity scale factor from ratios of kinetic energy
    #Update velocities

    #NOW RETURN THE NEW VELOCITIES
    return Vel
```

## Now use your functions, coupled with those provided, to code up your assignment:

```python
#PART A:

#Define box size and other settings
k=3000
r0=1
N=240
rho=0.8 #Solve to find L
#Set L
dt=0.001
T=1.0
Cut=2.5
RescaleFreq = 100 #See note above - may want to try several values

#Define your M value(s)

#Initially place atoms on a cubic lattice
#Energy-minimize the initial configuration using the conjugate-gradient energy minimizer
#Assign initial velocities and compute forces (accelerations)
#Use the velocity Verlet integrator to perform a molecular dynamics run, rescaling velocities when appropriate
```

```python
#PART B:
#Additionally:
NStepsEquil1 = 10000
NStepsEquil2 = 10000
NStepsProd = 100000

#Set up as in A
#Equilibrate for NStepsEquil1 with velocity rescaling every RescaleFreq steps, discarding energies
#Equilibrate for NStepsEquil2 with velocity rescaling, storing energies
#Stop and average the energy. Rescale the velocities so the current (end-of-equilibration) energy matches the average
#Store the particle positions so you can later compute the mean squared displacement
#Run for NStepsProd at constant energy (NVE) recording time and mean squared displacement periodically

#Compute diffusion coefficient for each M using a fit to the mean squared displacement
#Plot mean squared displacement for each M
#Plot diffusion coefficient as a function of M
```

## Follow-up:
The thermostat used here is not in general recommended, for reasons we will discuss in class. To understand one such reason, please read the “flying ice cube” paper referenced here: J Comp Chem 19:726-740 (1998) <http://onlinelibrary.wiley.com/doi/10.1002/(SICI)1096-987X(199805)19:7%3C726::AID-JCC4%3E3.0.CO;2-S/full>


# Arbitrarily high order accurate explicit time integration methods

 1. Chapter 5: ADER and DeC
    1. [Section 1.1: DeC](#DeC)
    1. [Section 1.2: ADER](#ADER)

## Deferred Correction (Defect correction/ Spectral deferred correction)
Acronyms: DeC, DEC, DC, SDC

References: [Dutt et al. 2000](https://link.springer.com/article/10.1023/A:1022338906936), [Minion (implicit) 2003](https://projecteuclid.org/journals/communications-in-mathematical-sciences/volume-1/issue-3/Semi-implicit-spectral-deferred-correction-methods-for-ordinary-differential-equations/cms/1250880097.full), [Abgrall 2017 (for PDE)](https://hal.archives-ouvertes.fr/hal-01445543v2)

We study Abgrall's version (for notation)

Theory on slides!

```python
# If you do not have numpy, matplotlib, scipy or nodepy, run this cell
!pip install numpy
# This is the basic package in python with all the numerical functions

!pip install scipy
# This package has some functions to deal with polynomials

!pip install matplotlib
# This package allows to  plot

!pip install nodepy
# This package has some interesting features for RK methods
```

```python
# We need a couple of packages in this chapter
import numpy as np
# This is the basic package in python with all the numerical functions

import matplotlib.pyplot as plt
# This package allows to  plot

from nodepy import rk
#This package already implemented some functions for Runge Kutta and multistep methods
```

For the definition of the basis functions in time, we introduce different Lagrange polynomials and point distributions:
1. equispaced
1. Gauss--Legendre--Lobatto (GLB)
1. Gauss--Legendre (not in DeC, because the last point is not $t^{n+1}$)

So, we have the quadrature points $\lbrace t^m \rbrace_{m=0}^M$, the polynomials $\lbrace \varphi_m \rbrace_{m=0}^M$ such that $\varphi_j(t^m)=\delta_{j}^m$, and we are interested in computing

$$
\theta_r^m:=\int_{t^0}^{t^m} \varphi_r(t) dt
$$

To compute the integral we will use exact quadrature rules with Gauss--Lobatto (GLB) points, i.e., given the quadrature nodes and weights $t_q, w_q$ on the interval $[0,1]$ the integral is computed as

$$
\theta_r^m:=\int_{t^0}^{t^m} \varphi_r(t) dt = \sum_q \varphi_r(t^q(t^m-t^0)+t^0) w_q(t^m-t^0)
$$

In practice, at each timestep we have to loop over corrections $(k)$ and over subtimesteps $m$ and compute

$$
y^{m,(k)} = y^{m,(k-1)} - \left(  y^{m,(k-1)} -  y^{0} - \Delta t\sum_{r=0}^M \theta_r^m F(y^{r,(k-1)}) \right)=y^{0} + \Delta t\sum_{r=0}^M \theta_r^m F(y^{r,(k-1)})
$$

```python
from scipy.interpolate import lagrange
from numpy.polynomial.legendre import leggauss

def equispaced(order):
    '''
    Takes input d and returns the vector of d equispaced points in [-1,1]
    And the integral of the basis functions interpolated in those points
    '''
    nodes= np.linspace(-1,1,order)
    w= np.zeros(order)
    for k in range(order):
        yy= np.zeros(order)
        yy[k]=1.
        zz=lagrange(nodes,yy)
        pp=zz.integ()
        w[k]=pp(1)-pp(-1)

    return nodes, w

def lglnodes(n,eps=10**-15):
    '''
    Python translation of lglnodes.m

    Computes the Legendre-Gauss-Lobatto nodes, weights and the LGL Vandermonde
    matrix. The LGL nodes are the zeros of (1-x^2)*P'_N(x). Useful for numerical
    integration and spectral methods.

    Parameters
    ----------
    n : integer, requesting an nth-order Gauss-quadrature rule on [-1, 1]

    Returns
    -------
    (nodes, weights) : tuple, representing the quadrature nodes and weights.
                       Note: (n+1) nodes and weights are returned.

    Example
    -------
    >>> from lglnodes import *
    >>> (nodes, weights) = lglnodes(3)
    >>> print(str(nodes) + "   " + str(weights))
    [-1.        -0.4472136  0.4472136  1.       ]   [0.16666667 0.83333333 0.83333333 0.16666667]

    Notes
    -----

    Reference on LGL nodes and weights:
      C. Canuto, M. Y. Hussaini, A. Quarteroni, T. A. Tang, "Spectral Methods
      in Fluid Dynamics," Section 2.3. Springer-Verlag 1987

    Written by Greg von Winckel - 04/17/2004
        Contact: gregvw@chtm.unm.edu

    Translated and modified into Python by Jacob Schroder - 9/15/2018
    '''

    w = np.zeros((n+1,))
    x = np.zeros((n+1,))
    xold = np.zeros((n+1,))

    # The Legendre Vandermonde Matrix
    P = np.zeros((n+1,n+1))

    epss = eps

    # Use the Chebyshev-Gauss-Lobatto nodes as the first guess
    for i in range(n+1):
        x[i] = -np.cos(np.pi*i / n)

    # Compute P using the recursion relation
    # Compute its first and second derivatives and
    # update x using the Newton-Raphson method.

    xold = 2.0

    for i in range(100):
        xold = x

        P[:,0] = 1.0
        P[:,1] = x

        for k in range(2,n+1):
            P[:,k] = ( (2*k-1)*x*P[:,k-1] - (k-1)*P[:,k-2] ) / k

        x = xold - ( x*P[:,n] - P[:,n-1] )/( (n+1)*P[:,n])

        if (max(abs(x - xold).flatten()) < epss ):
            break

    w = 2.0 / ( (n*(n+1))*(P[:,n]**2))

    return x, w

def lagrange_basis(nodes,x,k):
    y=np.zeros(x.size)
    for ix, xi in enumerate(x):
        tmp=[(xi-nodes[j])/(nodes[k]-nodes[j])  for j in range(len(nodes)) if j!=k]
        y[ix]=np.prod(tmp)
    return y

def get_nodes(order,nodes_type):
    if nodes_type=="equispaced":
        nodes,w = equispaced(order)
    elif nodes_type == "gaussLegendre":
        nodes,w = leggauss(order)
    elif nodes_type == "gaussLobatto":
        nodes, w = lglnodes(order-1,10**-15)
    nodes=nodes*0.5+0.5
    w = w*0.5
    return nodes, w

def compute_theta_DeC(order, nodes_type):
    nodes, w = get_nodes(order,nodes_type)              # These are the subtimesteps nodes
    int_nodes, int_w = get_nodes(order,"gaussLobatto")  # These are the quadrature nodes and weights
    # generate theta and beta coefficients of DeC algorithm
    theta = np.zeros((order,order))
    beta = np.zeros(order)
    for m in range(order):                      # loop over the subtimesteps [t^0,t^m]
        beta[m] = FILL IN  HERE                 # How are the beta defined?
        nodes_m = int_nodes*(nodes[m])          # Rescaling the quadrature nodes on the considered subtimestep
        w_m = int_w*(nodes[m])                  # Weighting the quadrature weights on [t^0, t^m]
        for r in range(order):
            theta[r,m] =  FILL IN HERE          # Compute the integral int_{t^0}^{t^m} \phi_r(t)dt
    return theta, beta

def compute_RK_from_DeC(M_sub,K_corr,nodes_type):
    order=M_sub+1;
    [theta,beta]=compute_theta_DeC(order,nodes_type)
    bar_beta=beta[1:]  # M_sub
    bar_theta=theta[:,1:].transpose() # M_sub x (M_sub +1)
    theta0= bar_theta[:,0]  # M_sub x 1
    bar_theta= bar_theta[:,1:] #M_sub x M_sub
    A=np.zeros((M_sub*(K_corr-1)+1,M_sub*(K_corr-1)+1))  # (M_sub x K_corr +1)^2
    b=np.zeros(M_sub*(K_corr-1)+1)
    c=np.zeros(M_sub*(K_corr-1)+1)

    c[1:M_sub+1]=bar_beta
    A[1:M_sub+1,0]=bar_beta
    for k in range(1,K_corr-1):
        r0=1+M_sub*k
        r1=1+M_sub*(k+1)
        c0=1+M_sub*(k-1)
        c1=1+M_sub*(k)
        c[r0:r1]=bar_beta
        A[r0:r1,0]=theta0
        A[r0:r1,c0:c1]=bar_theta
    b[0]=theta0[-1]
    b[-M_sub:]=bar_theta[M_sub-1,:]
    return A,b,c

```

```python
## Deferred correction algorithm

def dec(func, tspan, y_0, M_sub, K_corr, distribution):
    '''
    Deferred correction algorithm with the formalism of Abgrall 2017
    Input:
    func is the ODE function
    tspan the timesteps
    y_0 is the initial value
    M_sub is the number of subtimesteps [t^0,t^m], m=0,\dots, M_sub (at least order M_sub+1)
    K_corr is the number of iterations of the algorithm (order K_corr)
    '''
    N_time=len(tspan)
    dim=len(y_0)
    U=np.zeros((dim, N_time))              # All solutions
    u_p=np.zeros((dim, M_sub+1))           # u_p is the solution for all subtimesteps at the previous iteration
    u_a=np.zeros((dim, M_sub+1))           # u_a is the solution for all subtimesteps at the current iteration
    rhs= np.zeros((dim,M_sub+1))           # Here we compute all the functions f at u_p
    Theta, beta = compute_theta_DeC(M_sub+1,distribution)      # Dec coefficients
    U[:,0]=y_0
    for it in range(1, N_time):
        delta_t=(tspan[it]-tspan[it-1])
        for m in range(M_sub+1):                     # Initialization of variables
            u_a[:,m]=U[:,it-1]
            u_p[:,m]=U[:,it-1]
        for k in range(  FILL IN THE INDEXES  ):             # Loop over the iterations
            u_p=np.copy(u_a)
            for r in range( FILL IN THE INDEXES  ):          # Loop over the subtimesteps to compute the flux
                rhs[:,r]=func(u_p[:,r])
            for m in range(   FILL IN THE INDEXES  ):    # Loop over the subtimesteps to compute the update values
                u_a[:,m]= U[:,it-1]  FILL IN THE UPDATE FORMULA
        U[:,it]=u_a[:,M_sub]                             # update the new timestep
    return tspan, U
```

```python
import numpy as np

## Linear scalar Dahlquist's equation
def linear_scalar_flux(u,t=0,k_coef=10):
    ff=np.zeros(np.shape(u))
    ff[0]= -k_coef*u[0]
    return ff

def linear_scalar_exact_solution(u0,t,k_coef=10):
    return np.array([np.exp(-k_coef*u0[0]*t)])

def linear_scalar_jacobian(u,t=0,k_coef=10):
    Jf=np.zeros((len(u),len(u)))
    Jf[0,0]=-k_coef
    return Jf

#nonlinear problem y'=-ky|y| +1
def nonlinear_scalar_flux(u,t=0,k_coef=10):
    ff=np.zeros(np.shape(u))
    ff[0]=-k_coef*abs(u[0])*u[0] +1
    return ff

def nonlinear_scalar_exact_solution(u0,t,k_coef = 10):
        sqrtk = np.sqrt(k_coef)
        ustar = 1 / sqrtk
        if u0[0] >= ustar:
            uex=np.array([1./np.tanh(sqrtk * t + np.arctanh(1/sqrtk /u0[0])) / sqrtk])
        elif u0[0] < 0 and t < - np.atan(sqrtk * u0[0]) / sqrtk:
            uex=np.array([np.tan(sqrtk * t + np.arctan(sqrtk * u0[0])) / sqrtk])
        else:
            uex=np.array([np.tanh(sqrtk * t + np.arctanh(sqrtk * u0[0])) / sqrtk])
        return uex

def nonlinear_scalar_jacobian(u,t=0,k_coef=10):
    Jf=np.zeros((len(u),len(u)))
    Jf[0,0]=-k_coef*abs(u[0])
    return Jf

# SYSTEMS

# linear systems
def linear_system2_flux(u,t=0):
    d=np.zeros(len(u))
    d[0]= -5*u[0] + u[1]
    d[1]= 5*u[0] -u[1]
    return d

def linear_system2_exact_solution(u0,t):
    A=np.array([[-5,1],[5,-1]])
    u_e=u0+(1-np.exp(-6*t))/6*np.dot(A,u0)
    return u_e

def linear_system2_jacobian(u,t=0):
    Jf=np.array([[-5,1],[5,-1]])
    return Jf

linear_system2_matrix = np.array([[-5,1],[5,-1]])

def linear_system2_production_destruction(u,t=0):
    p=np.zeros((len(u),len(u)))
    d=np.zeros((len(u),len(u)))
    p[0,1]=u[1]
    d[1,0]=u[1]
    p[1,0]=5*u[0]
    d[0,1]=5*u[0]
    return p,d

#lin system 3 x3

def linear_system3_flux(u,t=0):
    d=np.zeros(len(u))
    d[0]= -u[0] + 3*u[1]
    d[1]= -3*u[1] + 5*u[2]
    d[2]= -5*u[2]
    return d

def linear_system3_exact_solution(u0,t=0):
    u_e = np.zeros(len(u0))
    u_e[0] = 15.0/8.0*u0[2]*(np.exp(-5*t) - 2*np.exp(-3*t)+np.exp(-t))
    u_e[1] = 5.0/2.0*u0[2]*(-np.exp(-5*t) + np.exp(-3*t))
    u_e[2] = u0[2]*np.exp(-5*t)
    return u_e
def linear_system3_jacobian(u,t=0):
    Jf=np.zeros((len(u),len(u)))
    Jf[0,0]=-1.
    Jf[0,1]=3
    Jf[1,1] = -3
    Jf[1,2] = 5
    Jf[2,2] = -5
    return Jf

## Nonlinear 3x3 system production destruction
def nonlinear_system3_flux(u,t=0):
    ff=np.zeros(len(u))
    ff[0]= -u[0]*u[1]/(u[0]+1)
    ff[1]= u[0]*u[1]/(u[0]+1) -0.3*u[1]
    ff[2]= 0.3*u[1]
    return ff

def nonlinear_system3_production_destruction(u,t=0):
    p=np.zeros((len(u),len(u)))
    d=np.zeros((len(u),len(u)))
    p[1,0]=u[0]*u[1]/(u[0]+1)
    d[0,1]=p[1,0]
    p[2,1]=0.3*u[1]
    d[1,2]=p[2,1]
    return p,d

# SIR Model
def SIR_flux(u,t=0,beta=3,gamma=1):
    ff=np.zeros(len(u))
    N=np.sum(u)
    ff[0]=-beta*u[0]*u[1]/N
    ff[1]=+beta*u[0]*u[1]/N - gamma*u[1]
    ff[2]= gamma*u[1]
    return ff

def SIR_jacobian(u,t=0,beta=3,gamma=1):
    Jf=np.zeros((len(u),len(u)))
    N=np.sum(u)
    Jf[0,0]=-beta*u[1]/N
    Jf[0,1]=-beta*u[0]/N
    Jf[1,0]= beta*u[1]/N
    Jf[1,1]= beta*u[0]/N - gamma
    Jf[2,1] = gamma
    return Jf

def SIR_production_destruction(u,t=0,beta=3,gamma=1):
    p=np.zeros((len(u),len(u)))
    d=np.zeros((len(u),len(u)))
    N=np.sum(u)
    p[1,0]=beta*u[0]*u[1]/N
    d[0,1]=p[1,0]
    p[2,1]=gamma*u[1]
    d[1,2]=p[2,1]
    return p,d

# Nonlinear_oscillator
def nonLinearOscillator_flux(u,t=0,alpha=0.):
    ff=np.zeros(np.shape(u))
    n=np.sqrt(np.dot(u,u))
    ff[0]=-u[1]/n-alpha*u[0]/n
    ff[1]=u[0]/n - alpha*u[1]/n
    return ff

def nonLinearOscillator_exact_solution(u0,t):
    u_ex=np.zeros(np.shape(u0))
    n=np.sqrt(np.dot(u0,u0))
    u_ex[0]=np.cos(t/n)*u0[0]-np.sin(t/n)*u0[1]
    u_ex[1]=np.sin(t/n)*u0[0]+np.cos(t/n)*u0[1]
    return u_ex

# Non linear oscillator damped
def nonLinearOscillatorDamped_flux(u,t,alpha=0.01):
    ff=np.zeros(np.shape(u))
    n=np.sqrt(np.dot(u,u))
    ff[0]=-u[1]/n-alpha*u[0]/n
    ff[1]=u[0]/n - alpha*u[1]/n
    return ff

def nonLinearOscillatorDamped_exact_solution(u0,t,alpha=0.01):
    u_ex=np.zeros(np.shape(u0))
    n0=np.sqrt(np.dot(u0,u0))
    n=n0*np.exp(-alpha*t)
    u_ex[0]=n/n0*(np.cos(t/n)*u0[0]-np.sin(t/n)*u0[1])
    u_ex[1]=n/n0*(np.sin(t/n)*u0[0]+np.cos(t/n)*u0[1])
    return u_ex

# pendulum
def pendulum_flux(u,t=0):
    ff=np.zeros(np.shape(u))
    ff[0]=u[1]
    ff[1]=-np.sin(u[0])
    return ff

def pendulum_jacobian(u,t=0):
    Jf=np.zeros((2,2))
    Jf[0,1]=1.
    Jf[1,0]=np.cos(u[0])
    return Jf

def pendulum_entropy(u,t=0):
    return np.array(0.5*u[1]**2.-np.cos(u[0]), dtype=np.float)

def pendulum_entropy_variables(u,t=0):
    v=np.zeros(np.shape(u))
    v[0]=np.sin(u[0])
    v[1]=u[1]
    return v

# Robertson
def Robertson_flux(u,t=0,alpha=10**4,beta=0.04, gamma=3*10**7):
    ff=np.zeros(np.shape(u))
    ff[0] = alpha*u[1]*u[2]-beta*u[0]
    ff[1] = beta*u[0]-alpha*u[1]*u[2] - gamma*u[1]**2
    ff[2] = gamma*u[1]**2
    return ff

def Robertson_jacobian(u,t=0,alpha=10**4,beta=0.04, gamma=3*10**7):
    Jf=np.zeros((3,3))
    Jf[0,0]= -beta
    Jf[0,1]= alpha*u[2]
    Jf[0,2]= alpha*u[1]
    Jf[1,0]= beta
    Jf[1,1]= -alpha*u[2]-2*gamma*u[1]
    Jf[1,2]= -alpha*u[1]
    Jf[2,1] = 2*gamma*u[1]
    return Jf

def Robertson_production_destruction(u,t=0,alpha=10**4,beta=0.04, gamma=3*10**7):
    p=np.zeros((len(u),len(u)))
    d=np.zeros((len(u),len(u)))
    p[0,1]=alpha*u[1]*u[2]
    d[1,0]=p[0,1]
    p[1,0]=beta*u[0]
    d[0,1]=p[1,0]
    p[2,1]=gamma*u[1]**2
    d[1,2]=p[2,1]
    return p,d

# Lotka:
def lotka_flux(u,t=0,alpha=1,beta=0.2,delta=0.5,gamma=0.2):
    ff=np.zeros(np.shape(u))
    ff[0]=alpha*u[0]-beta*u[0]*u[1]
    ff[1]=delta*beta*u[0]*u[1]-gamma*u[1]
    return ff

def lotka_jacobian(u,t=0,alpha=1,beta=0.2,delta=0.5,gamma=0.2):
    Jf=np.zeros((2,2))
    Jf[0,0] = alpha -beta*u[1]
    Jf[0,1] = -beta*u[0]
    Jf[1,0] = delta*beta*u[1]
    Jf[1,1] = delta*beta*u[0] -gamma
    return Jf

#3 bodies problem in 2D: U=(x_1,x_2,v_1,v_2,y_1,y_2,w_1,w_2,z_1,z_2,s_1,s_2)
# where x is the 2D position of body1 and v is speed body1 sun
# y, w are position and velocity body2 earth
# z, s are position and velocity body3 mars

def threeBodies_flux(u,t=0):
    m1=1.98892*10**30
    m2=5.9722*10**24
    m3=6.4185*10**23
    G=6.67*10**(-11)
    f=np.zeros(np.shape(u))
    x=u[0:2]
    v=u[2:4]
    y=u[4:6]
    w=u[6:8]
    z=u[8:10]
    s=u[10:12]
    dxy3=np.linalg.norm(x-y)**3
    dxz3=np.linalg.norm(x-z)**3
    dyz3=np.linalg.norm(y-z)**3
    f[0:2]=v
    f[2:4]=-m2*G/dxy3*(x-y)-m3*G/dxz3*(x-z)
    f[4:6]=w
    f[6:8]=-m1*G/dxy3*(y-x)-m3*G/dyz3*(y-z)
    f[8:10]=s
    f[10:12]=-m1*G/dxz3*(z-x)-m2*G/dyz3*(z-y)
    return f

class ODEproblem:
    def __init__(self,name):
        self.name=name
        if self.name=="linear_scalar":
            self.u0 = np.array([1.])
            self.T_fin= 2.
            self.k_coef=10
            self.matrix=np.array([-self.k_coef])
        elif self.name=="nonlinear_scalar":
            self.k_coef=10
            self.u0 = np.array([1.1/np.sqrt(self.k_coef)])
            self.T_fin= 1.
        elif self.name=="linear_system2":
            self.u0 = np.array([0.9,0.1])
            self.T_fin= 1.
            self.matrix = np.array([[-5,1],[5,-1]])
        elif self.name=="linear_system3":
            self.u0 = np.array([0,0.,10.])
            self.T_fin= 10.
        elif self.name=="nonlinear_system3":
            self.u0 = np.array([9.98,0.01,0.01])
            self.T_fin= 30.
        elif self.name=="SIR":
            self.u0 = np.array([1000.,1,10**-20])
            self.T_fin= 10.
        elif self.name=="nonLinearOscillator":
            self.u0 = np.array([1.,0.])
            self.T_fin= 50
        elif self.name=="nonLinearOscillatorDamped":
            self.u0 = np.array([1.,0.])
            self.T_fin= 50
        elif self.name=="pendulum":
            self.u0 = np.array([2.,0.])
            self.T_fin= 50
        elif self.name=="Robertson":
            self.u0 = np.array([1.,10**-20,10**-20])
            self.T_fin= 10.**10.
        elif self.name=="lotka":
            self.u0 = np.array([1.,2.])
            self.T_fin= 100.
        elif self.name=="threeBodies":
            self.u0 = np.array([0,0,0,0,149*10**9,0,0,30*10**3,-226*10**9,0,0,-24.0*10**3])
            self.T_fin= 10.**8.
        else:
            raise ValueError("Problem not defined")

    def flux(self,u,t=0):
        if self.name=="linear_scalar":
            return linear_scalar_flux(u,t,self.k_coef)
        elif self.name=="nonlinear_scalar":
            return nonlinear_scalar_flux(u,t,self.k_coef)
        elif self.name=="linear_system2":
            return linear_system2_flux(u,t)
        elif self.name=="linear_system3":
            return linear_system3_flux(u,t)
        elif self.name=="nonlinear_system3":
            return nonlinear_system3_flux(u,t)
        elif self.name=="SIR":
            return SIR_flux(u,t)
        elif self.name=="nonLinearOscillator":
            return nonLinearOscillator_flux(u,t)
        elif self.name=="nonLinearOscillatorDamped":
            return nonLinearOscillatorDamped_flux(u,t)
        elif self.name=="pendulum":
            return pendulum_flux(u,t)
        elif self.name=="Robertson":
            return Robertson_flux(u,t)
        elif self.name=="lotka":
            return lotka_flux(u,t)
        elif self.name=="threeBodies":
            return threeBodies_flux(u,t)
        else:
            raise ValueError("Flux not defined for this problem")

    def jacobian(self,u,t=0):
        if self.name=="linear_scalar":
            return linear_scalar_jacobian(u,t,self.k_coef)
        elif self.name=="nonlinear_scalar":
            return nonlinear_scalar_jacobian(u,t,self.k_coef)
        elif self.name=="linear_system2":
            return linear_system2_jacobian(u,t)
        elif self.name=="linear_system3":
            return linear_system3_jacobian(u,t)
        elif self.name=="pendulum":
            return pendulum_jacobian(u,t)
        elif self.name=="SIR":
            return SIR_jacobian(u,t)
        elif self.name=="Robertson":
            return Robertson_jacobian(u,t)
        elif self.name=="lotka":
            return lotka_jacobian(u,t)
        else:
            raise ValueError("Jacobian not defined for this problem")

    def exact(self,u,t):
        if self.name=="linear_scalar":
            return linear_scalar_exact_solution(u,t,self.k_coef)
        elif self.name=="nonlinear_scalar":
            return nonlinear_scalar_exact_solution(u,t,self.k_coef)
        elif self.name=="linear_system2":
            return linear_system2_exact_solution(u,t)
        elif self.name=="linear_system3":
            return linear_system3_exact_solution(u,t)
        elif self.name=="nonLinearOscillator":
            return nonLinearOscillator_exact_solution(u,t)
        elif self.name=="nonLinearOscillatorDamped":
            return nonLinearOscillatorDamped_exact_solution(u,t)
        else:
            raise ValueError("Exact solution not defined for this problem")

    def exact_solution_times(self,u0,tt):
        exact_solution=np.zeros((len(u0),len(tt)))
        for it, t in enumerate(tt):
            exact_solution[:,it]=self.exact(u0,t)
        return exact_solution

    def prod_dest(self,u,t=0):
        if self.name=="linear_system2":
            return linear_system2_production_destruction(u,t)
        if self.name=="nonlinear_system3":
            return nonlinear_system3_production_destruction(u,t)
        elif self.name=="Robertson":
            return Robertson_production_destruction(u,t)
        elif self.name=="SIR":
            return SIR_production_destruction(u,t)
        else:
            raise ValueError("Prod Dest not defined for this problem")

```

```python
#Test the DeC on a three bodies problem (Earth, Mars and Sun)
pr=ODEproblem("threeBodies")
tt=np.linspace(0,pr.T_fin,1000)
tt,U=dec(pr.flux,tt,pr.u0,4,5,"gaussLobatto")

plt.figure()                            # Evolution in time of the planets
plt.plot(U[0,:],U[1,:],'*',label="sun")
plt.plot(U[4,:],U[5,:],label="earth")
plt.plot(U[8,:],U[9,:],label="Mars")
plt.legend()
plt.show()

plt.figure()                                # Plot of the distance from the sun of Earth and Mars
plt.title("Distance from the original position of the sun")
plt.semilogy(tt,U[4,:]**2+U[5,:]**2,label="earth")
plt.semilogy(tt,U[8,:]**2+U[9,:]**2, label="mars")
plt.legend()
plt.show()
```

```python
#Test convergence of DeC for several orders
pr=ODEproblem("linear_system2")

tt=np.linspace(0,pr.T_fin,10)   #Plot the evolution for order 8
tt,uu=dec(pr.flux, tt, pr.u0, 7, 8, "equispaced")
plt.plot(tt,uu[0,:])
plt.plot(tt,uu[1,:])
plt.show()

def compute_integral_error(c,c_exact):  # c is dim x times
    times=np.shape(c)[1]
    error=0.
    for t in range(times):
        error = error + np.linalg.norm(c[:,t]-c_exact[:,t],2)**2.
    error = np.sqrt(error/times)
    return error

NN=5
dts=[pr.T_fin/2.0**k for k in range(3,3+NN)]
errorsDeC=np.zeros(len(dts))

# Compute and plot the errors
for order in range(2,10):
    for k in range(NN):
        dt0=dts[k]
        tt=np.arange(0,pr.T_fin,dt0)
        t2,U2=dec(pr.flux, tt, pr.u0, order-1, order, "gaussLobatto")
        u_exact=pr.exact_solution_times(pr.u0,tt)
        errorsDeC[k]=compute_integral_error(U2,u_exact)

    plt.loglog(dts,errorsDeC,"--",label="DeC%d"%(order))
    plt.loglog(dts,[dt**order*errorsDeC[2]/dts[2]**order for dt in dts],":",label="ref %d"%(order))

plt.title("DeC error convergence")
plt.legend()
#plt.savefig("convergence_DeC.pdf")
plt.show()

```

```python
# DeC as RK and the stability region
for order in range(2,10):
    A,b,c=compute_RK_from_DeC(order-1,order,"equispaced")
    rkDeC = rk.ExplicitRungeKuttaMethod(A,b)
    rkDeC.name="DeC"+str(order)
    rkDeC.plot_stability_region(bounds=[-5,3,-7,7])
```

```python
# Checking the order of DeC as RK
for order in range(2,14):
    A,b,c=compute_RK_from_DeC(order-1,order,"equispaced")
    rkDeC = rk.ExplicitRungeKuttaMethod(A,b)
    rkDeC.name="DeC"+str(order)
    print(rkDeC.name+" has order "+str(rkDeC.order()))
```

```python
# Qualitatively comparison of DeC of order 2 and 8 for Lotka-Volterra problem
pr=ODEproblem("lotka")
tt=np.linspace(0,pr.T_fin,150)
t2,U2=dec(pr.flux, tt, pr.u0, 1, 2, "gaussLobatto")
t8,U8=dec(pr.flux, tt, pr.u0, 7, 8, "gaussLobatto")

tt=np.linspace(0,pr.T_fin,2000)
tref,Uref=dec(pr.flux, tt, pr.u0, 4,5, "gaussLobatto")

plt.figure(figsize=(12,6))
plt.subplot(211)
plt.plot(t2,U2[0,:],label="dec2")
plt.plot(t8,U8[0,:],label="dec8")
plt.plot(tref,Uref[0,:], ":",linewidth=2,label="ref")
plt.legend()
plt.title("Prey")

plt.subplot(212)
plt.plot(t2,U2[1,:],label="dec2")
plt.plot(t8,U8[1,:],label="dec8")
plt.plot(tref,Uref[1,:],":", linewidth=2,label="ref")
plt.legend()
plt.title("Predator")
```

### Pro exercise: implement the implicit DeC presented in the slides
* You need to pass also a function of the Jacobian of the flux in input
* The Jacobian can be evaluated only once per timestep $\partial_y F(y^n)$ and used to build the matrix that must be inverted at each correction
* For every subtimestep the matrix to be inverted changes a bit ($\beta^m \Delta t$ factor in front of the Jacobian)
* One can invert these $M$ matrices only once per time step
* Solve the system at each subtimestep and iteration

$$
y^{m,(k)}-\beta^m \Delta t \partial_y F(y^0)y^{m,(k)} = y^{m,(k-1)}-\beta^m \Delta t \partial_y F(y^0)y^{m,(k-1)} - \left(  y^{m,(k-1)} -  y^{0} - \Delta t\sum_{r=0}^M \theta_r^m F(y^{r,(k-1)}) \right)
$$

defining $M^{m}=I+\beta^m \Delta t \partial_y F(y^0)$, we can simplify it as

$$
y^{m,(k)}=y^{m,(k-1)} - (M^m)^{-1}\left(  y^{m,(k-1)} -  y^{0} - \Delta t\sum_{r=0}^M \theta_r^m F(y^{r,(k-1)}) \right)
$$

```python
def decImplicit(func,jac_stiff, tspan, y_0, M_sub, K_corr, distribution):
    '''
    Deferred correction algorithm with the formalism of Abgrall 2017
    Input:
    func is the ODE function
    jac_still is the jacobian of the stiff part of func in the ODE
    tspan the timesteps
    y_0 is the initial value
    M_sub is the number of subtimesteps [t^0,t^m], m=0,\dots, M_sub (at least order M_sub+1)
    K_corr is the number of iterations of the algorithm (order K_corr)
    '''
    N_time=len(tspan)
    dim=len(y_0)
    U=np.zeros((dim, N_time))               # Solutions at all timesteps
    u_p=np.zeros((dim, M_sub+1))            # Solutions at all subtimesteps at previous iteration
    u_a=np.zeros((dim, M_sub+1))            # Solutions at all subtimesteps at current iteration
    rhs= np.zeros((dim,M_sub+1))            # Fluxes at all subtimesteps at previous iteration
    Theta, beta = compute_theta_DeC(M_sub+1,distribution)   #Coefficients of DeC
    invJac=np.zeros((M_sub+1,dim,dim))      # Inverse of the Jacobian of the system for each subtimestep [t^0,t^m] m=1,...,M_sub
    U[:,0]=y_0
    for it in range(1, N_time):
        delta_t=(tspan[it]-tspan[it-1])
        for m in range(M_sub+1):
            u_a[:,m]=U[:,it-1]
            u_p[:,m]=U[:,it-1]

        FILL IN
        # Compute the Jacobian of the flux at time t^0
        # Compute the Jacobian of the system for all subtimesteps and invert it

        for k in range(1,K_corr+1):
            u_p=np.copy(u_a)
            for r in range(M_sub+1):
                rhs[:,r]=func(u_p[:,r])
            for m in range(1,M_sub+1):

                # Compute the update at each subtimestep with the formula for the implicit DeC

                u_a[:,m]=   FILL IN

        U[:,it]=u_a[:,M_sub]
    return tspan, U
```

```python
# Test on Robertson problem
pr=ODEproblem("Robertson")

Nt=100
tt = np.array([np.exp(k) for k in np.linspace(-14,np.log(pr.T_fin),Nt)])
tt,yy=decImplicit(pr.flux,pr.jacobian, tt, pr.u0, 5,6,"gaussLobatto")

plt.semilogx(tt,yy[0,:])
plt.semilogx(tt,yy[1,:]*10**4)
plt.semilogx(tt,yy[2,:])
```

```python
Nt=1000
tt = np.array([np.exp(k) for k in np.linspace(-14,np.log(pr.T_fin),Nt)])
tt,yy=dec(pr.flux, tt, pr.u0, 5,6,"gaussLobatto")

plt.semilogx(tt,yy[0,:])
plt.semilogx(tt,yy[1,:]*10**4)
plt.semilogx(tt,yy[2,:])
plt.ylim([-0.05,1.05])
```

#### Exercise extra: test convergence

## ADER
Can be interpreted as a finite element method in time solved in an iterative manner.

\begin{align*}
\def\L{\mathcal{L}}
\def\bc{\boldsymbol{c}}
\def\bbc{\underline{\mathbf{c}}}
\def\bphi{\underline{\phi}}
\newcommand{\ww}[1]{\underline{#1}}
\renewcommand{\vec}[1]{\ww{#1}}
\def\M{\underline{\underline{\mathrm{M}}}}
\def\R{\underline{\underline{\mathrm{R}}}}
		\L^2(\bbc ):=& \int_{T^n} \bphi(t) \partial_t \bphi(t)^T \bbc dt + \int_{T^n} \bphi(t)  F(\bphi(t)^T\bbc)  dt =\\
		&\bphi(t^{n+1}) \bphi(t^{n+1})^T \bbc - \bphi(t^{n}) \bc^n -  \int_{T^n} \partial_t \bphi(t) \bphi(t)^T \bbc   - \int_{T^n} \bphi(t)  F(\bphi(t)^T\bbc)  dt \\
&\M = \bphi(t^{n+1}) \bphi(t^{n+1})^T -\int_{T^n} \partial_t \bphi(t) \bphi(t)^T \\
&	\vec{r}(\bbc) =  \bphi(t^{n}) \bc^n + \int_{T^n} \bphi(t)  F(\bphi(t)^T\bbc)  dt\\
&\M \bbc = \vec{r}(\bbc)
\end{align*}

Iterative procedure to solve the problem for each time step

\begin{equation}\label{fix:point}
\bbc^{(k)}=\M^{-1}\vec{r}(\bbc^{(k-1)}),\quad k=1,\dots, K \text{ (convergence)}
\end{equation}

with $\bbc^{(0)}=\bc(t^n)$.

Reconstruction step

\begin{equation*}
	\bc(t^{n+1}) = \bphi(t^{n+1})^T \bbc^{(K)}.
\end{equation*}

### What can be precomputed?
* $\M$
* $$\vec{r}(\bbc) =  \bphi(t^{n}) \bc^n + \int_{T^n} \bphi(t)  F(\bphi(t)^T\bbc)  dt\approx \bphi(t^{n}) \bc^n + \int_{T^n} \bphi(t)\bphi(t)^T dt  F(\bbc) =  \bphi(t^{n}) \bc^n+ \R \bbc$$
$\R$ can be precomputed
* $$ \bc(t^{n+1}) = \bphi(t^{n+1})^T \bbc^{(K)} $$
$\bphi(t^{n+1})^T$ can be precomputed

```python
from scipy.interpolate import lagrange

def lagrange_poly(nodes,k):
    interpVal=np.zeros(np.size(nodes))
    interpVal[k] = 1.
    pp=lagrange(nodes,interpVal)
    return pp

def lagrange_basis(nodes,x,k):
    pp=lagrange_poly(nodes,k)
    return pp(x)

def lagrange_deriv(nodes,x,k):
    pp=lagrange_poly(nodes,k)
    dd=pp.deriv()
    return dd(x)

def get_nodes(order,nodes_type):
    if nodes_type=="equispaced":
        nodes,w = equispaced(order)
    elif nodes_type == "gaussLegendre":
        nodes,w = leggauss(order)
    elif nodes_type == "gaussLobatto":
        nodes, w = lglnodes(order-1,10**-15)
    nodes=nodes*0.5+0.5
    w = w*0.5
    return nodes, w

def getADER_matrix(order, nodes_type):
    '''
    We compute the ADER matrices in this function
    input order of accuracy and the type of node distribution
    M is the matrix to be inverted
    RHSmat is the R matrix in the formula above
    '''
    nodes_poly, w_poly = get_nodes(order,nodes_type)     # Subtimesteps nodes
    if nodes_type=="equispaced":                         # Quadrature nodes
        quad_order=order
        nodes_quad, w = get_nodes(quad_order,"gaussLegendre")
    else:
        quad_order=order
        nodes_quad, w = get_nodes(quad_order,nodes_type)

    # generate mass matrix
    M = np.zeros((order,order))
    for i in range(order):
        for j in range(order):
            M[i,j] = lagrange_basis(nodes_poly,1.0,i) *lagrange_basis(nodes_poly,1.0,j)\
                  -sum([lagrange_deriv(nodes_poly,nodes_quad[q],i)\
                  *lagrange_basis(nodes_poly,nodes_quad[q],j)\
                  *w[q] for q in range(quad_order)])
    # generate RHS matrix
    RHSmat = np.zeros((order,order))
    for i in range(order):
        for j in range(order):
            RHSmat[i,j] = FILL IN THE QUADRATURE FORMULA FOR RHS MATRIX
    return nodes_poly, w_poly, M, RHSmat

def ader(func, tspan, y_0, M_sub, K_corr, distribution):
    '''
    ADER algorithm for ODEs with the formalism of Han Veiga, Oeffner, Torlo 2021
    Input:
    func is the ODE function
    tspan the timesteps
    y_0 is the initial value
    M_sub is the number of subtimesteps [t^0,t^m], m=0,\dots, M_sub (at least order M_sub+1)
    K_corr is the number of iterations of the algorithm (order K_corr)
    '''
    N_time=len(tspan)
    dim=len(y_0)
    U=np.zeros((dim, N_time))           # solutions at all timesteps
    u_p=np.zeros((dim, M_sub+1))        # aolution all subtimesteps at the previous iteration
    u_a=np.zeros((dim, M_sub+1))        # solution all subtimesteps at current iter
    u_tn=np.zeros((dim, M_sub+1))       # vector of dim M_sub+1 with all y^n, practical for update
    rhs= np.zeros((dim,M_sub+1))        # all fluxes at previous iter

    # Precomputing the ADER matrices M=ADER,R=RHS_mat, phi(t^{n+1})=reconstructionCoefficients
    x_poly, w_poly, ADER, RHS_mat = getADER_matrix(M_sub+1, distribution)
    #Inverse of M
    invader = np.linalg.inv(ADER)
    # Computing already M^{-1}R
    evolMatrix=np.matmul(invader,RHS_mat)
    reconstructionCoefficients=np.array([lagrange_basis(x_poly,1.0,i) for i in range(M_sub+1)])

    U[:,0]=y_0

    for it in range(1, N_time):
        delta_t=(tspan[it]-tspan[it-1])
        for m in range(M_sub+1):
            u_a[:,m]=U[:,it-1]
            u_p[:,m]=U[:,it-1]
            u_tn[:,m]=U[:,it-1]
        for k in range(1,K_corr+1):
            u_p=np.copy(u_a)
            for r in range(M_sub+1):
                rhs[:,r]=func(u_p[:,r])
            for d in range(dim):
                u_a[d,:] = u_tn[d,:] + FILL IN WITH THE EVOLUTION # Evolution of the corrections
        U[:,it] = np.matmul(u_a,reconstructionCoefficients)
    return tspan, U
```

```python
#Test ADER ON three bodies problem
pr=ODEproblem("threeBodies")
tt=np.linspace(0,pr.T_fin,1000)
tt,U=ader(pr.flux,tt,pr.u0,4,5,"gaussLegendre")
plt.figure()
plt.plot(U[0,:],U[1,:],'*',label="sun")
plt.plot(U[4,:],U[5,:],label="earth")
plt.plot(U[8,:],U[9,:],label="Mars")
plt.legend()
plt.show()

plt.figure()
plt.title("Distance from the original position of the sun")
plt.semilogy(tt,U[4,:]**2+U[5,:]**2,label="earth")
plt.semilogy(tt,U[8,:]**2+U[9,:]**2, label="mars")
plt.legend()
plt.show()
```

```python
#Test convergence
pr=ODEproblem("linear_system2")

tt=np.linspace(0,pr.T_fin,10)
tt,uu=ader(pr.flux, tt, pr.u0, 7, 8, "equispaced")
plt.plot(tt,uu[0,:])
plt.plot(tt,uu[1,:])
plt.show()

def compute_integral_error(c,c_exact):  # c is dim x times
    times=np.shape(c)[1]
    error=0.
    for t in range(times):
        error = error + np.linalg.norm(c[:,t]-c_exact[:,t],2)**2.
    error = np.sqrt(error/times)
    return error

NN=5
dts=[pr.T_fin/2.0**k for k in range(3,3+NN)]
errorsDeC=np.zeros(len(dts))

for order in range(2,8):
    for k in range(NN):
        dt0=dts[k]
        tt=np.arange(0,pr.T_fin,dt0)
        t2,U2=ader(pr.flux, tt, pr.u0, order-1, order, "gaussLobatto")
        u_exact=pr.exact_solution_times(pr.u0,tt)
        errorsDeC[k]=compute_integral_error(U2,u_exact)

    plt.loglog(dts,errorsDeC,"--",label="ADER%d"%(order))
    plt.loglog(dts,[dt**order*errorsDeC[2]/dts[2]**order for dt in dts],":",label="ref %d"%(order))

plt.title("ADER error convergence")
plt.legend()
plt.show()

```

```python
# Test Lotka-Volterra
pr=ODEproblem("lotka")
tt=np.linspace(0,pr.T_fin,150)
t2,U2=ader(pr.flux, tt, pr.u0, 1, 2, "gaussLobatto")
t8,U8=ader(pr.flux, tt, pr.u0, 7, 8, "gaussLobatto")

tt=np.linspace(0,pr.T_fin,2000)
tref,Uref=dec(pr.flux, tt, pr.u0, 4,5, "gaussLobatto")

plt.figure(figsize=(12,6))
plt.subplot(211)
plt.plot(t2,U2[0,:],label="ADER2")
plt.plot(t8,U8[0,:],label="ADER8")
plt.plot(tref,Uref[0,:], ":",linewidth=2,label="ref")
plt.legend()
plt.title("Prey")

plt.subplot(212)
plt.plot(t2,U2[1,:],label="ADER2")
plt.plot(t8,U8[1,:],label="ADER8")
plt.plot(tref,Uref[1,:],":", linewidth=2,label="ref")
plt.legend()
plt.title("Predator")
```

### Pro exercise: implicit ADER
Using the fact that ADER can be written into DeC, try to make ADER implicit by changing only the definition of $\mathcal{L}^1$

* Write the formulation and the update formula
* Implement it adding (as for the DeC an extra input of the jacobian of the flux)

### Pro exercise: ADER as RK
How can you write the ADER scheme into a RK setting?
At the end we are computing some coefficients in a more elaborated way to use them explicitly, so one should be able to write it down.

### Few notes on the stability
Computing the stability region of the ADER method results, for a fixed order of accuracy, for any point distribution to the same stability region. This coincide with the DeC method stability region for the same order of accuracy.

This can be shown numerically, I put here some plots, but no analytical proof is available yet.

**Stability for ADER and DeC methods with $p$ subtimesteps**

| ADER        | ADER vs DeC |
| ----------- | ----------- |
|   |      |


# Quantum Integer Programming and Quantum Machine Learning 47-779/785, Tepper School of Business.
# Introduction to Quantum Computing 18-819F, Electrical and Computers Engineering.
# Fall 2021, Carnegie Mellon University

This notebook contains material from the Quantum Integer Programming and Quantum Machine Learning Course at CMU Fall 2021 by David Bernal (bernalde at cmu.edu), Sridhar Tayur (stayur at cmu.edu), Elias Towe (towe at cmu.edu), and Davide Venturelli; the content is available on **[Github](https://github.com/bernalde/QuIPML)**.
The text is released under the **[CC-BY-NC-ND-4.0](https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode) license, and code is released under the **[MIT license](https://opensource.org/licenses/MIT).*

This notebook makes simple computations of Graver basis. Because of the complexity of these computation, we suggest that for more complicated problems you install the excellent **[4ti2](https://4ti2.github.io/)** software, an open-source implementation of several routines useful for the study of integer programming through algebraic geometry. It can be used as a stand-alone library and call it from C++ or from Python. In python, there are two ways of accessing it, either through **[Sage](https://www.sagemath.org/)** (which is an open-source mathematics software) or directly compiling it and installing a thing **[Python wrapper](https://github.com/alfsan/Py4ti2)**

Run in **[Google Colab](https://colab.research.google.com/github/bernalde/QuIP/blob/master/notebooks/Notebook%203%20-%20Graver%20basis.ipynb)**

## Introduction to Graver basis computation
In this notebook, we will be covering the computation of Graver basis, which are defined as
$$
\mathcal{G}(\mathbf{A}) = \bigcup_j \mathcal{H}_j(\mathbf{A})
$$
where $\mathcal{H}_j(\mathbf{A})$ are the minimal Hilbert basis of $\mathbf{A}$ in each orthant.

The Graver basis relate to the Groebner basis we saw in the previous lecture in the sense that they contain the Universal Groebner basis (union over all the possible objective functions)
$$
\mathcal{U}(\mathbf{A}) = \bigcup_{c\in\mathbb{Z}^n} \mathcal{B}(\mathbf{A},\mathbf{c})  \subseteq \mathcal{G}(\mathbf{A})
$$

```python
from sympy import *
import matplotlib.pyplot as plt
import numpy as np
from typing import Callable
import itertools
import random
```

Let's first define certain functions that help us write down the Toric ideal of a matrix $\mathbf{A}$.

```python
def to_polynomial(coef,vars):
    '''
    Function to define a single column of the coefficient as a polynomial
    '''
    res1 = 1
    res2 = 1
    for i in range(len(coef)):
        if coef[i] >= 0:
            res1 = res1*vars[i]**coef[i]
        else:
            res2 = res2*vars[i]**(-coef[i])
    res = res1 - res2
    return res
```

```python
def polynomial_ideal(A):
    '''
    Function to define a the polynomial ideal of a matrix A according to Conti and Traverso
    '''
    IA = A.col_insert(0, eye(A.shape[0]))
    # Find nullspace (kernel) of A
    ker = IA.nullspace()

    # Normalize elements of kernel to be integers
    for i in range(len(ker)):
        rationalvector = True
        while rationalvector:
            factor = 1
            for j in ker[i]:
                if j%1 != 0:
                    factor = min(factor,j%1)
            if factor == 1:
                rationalvector = False
            else:
                ker[i]=ker[i] / factor

    # Define symbolic variables zs for each row (index 0 in Python)
    sym_str_z = 'z:' + str(A.shape[0])
    zs = symbols(sym_str_z)

    # Define symbolic variables ws for each column (index 0 in Python)
    sym_str_w = 'w:' + str(A.shape[1])
    ws = symbols(sym_str_w)

    vars = zs + ws

    sys = []
    for k in ker:
        sys.append(to_polynomial(k,vars))

    vars = ws + zs

    return(sys, vars)
```

First example, consider the same matrices as in the previous notebook
$$
4x_1 + 5x_2 + x_3 = 37 \\
2x_1 + 3x_2 + x_4 = 20 \\
\mathbf{x} \in \mathbb{Z}_+^4
$$

```python
# A = np.array([[1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1],
#             [0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1],
#             [0, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1]])
A = np.array([[4,5,1,0], [2,3,0,1]])

A = Matrix(A)

IA, vars = polynomial_ideal(A)
print(IA)
print(vars)
randvars = tuple(random.sample(vars, len(vars)))
result = groebner(IA, vars, order='lex')
print(result)
```

    [w0 - z0**4*z1**2, w1 - z0**5*z1**3, w2 - z0, w3 - z1]
    (w0, w1, w2, w3, z0, z1)
    GroebnerBasis([w0 - z0**4*z1**2, w1 - z0**5*z1**3, w2 - z0, w3 - z1], w0, w1, w2, w3, z0, z1, domain='ZZ', order='lex')

```python
# reduced(vars[-3]*vars[-2]*vars[-1],result)
reduced(vars[-2]**37*vars[-1]**20,result)
```

    ([-w0**8*z0*z1**2 - w0**7*z0**5*z1**4 - w0**6*z0**9*z1**6 - w0**5*z0**13*z1**8 - w0**4*z0**17*z1**10 - w0**3*z0**21*z1**12 - w0**2*z0**25*z1**14 - w0*z0**29*z1**16 - z0**33*z1**18,
      0,
      -w0**9*z1**2,
      -w0**9*w2*w3 - w0**9*w2*z1],
     w0**9*w2*w3**2)

```python
def to_lawrence_polynomial(coef,xs,ys):
    '''
    Function to define a single column of the coefficient as a polynomial
    '''
    res1 = 1
    res2 = 1
    for i in range(len(coef)):
        if coef[i] >= 0:
            res1 = res1*xs[i]**coef[i]
            res2 = res2*ys[i]**coef[i]
        else:
            res2 = res2*xs[i]**(-coef[i])
            res1 = res1*ys[i]**(-coef[i])
    res = res1 - res2
    return res
```

```python
def lawrence_polynomial_ideal(A):
    '''
    Function to define a the polynomial ideal of the Lawrence lifting of matrix A
    '''
    # Find nullspace (kernel) of A
    ker = A.nullspace()

    # Normalize elements of kernel to be integers
    for i in range(len(ker)):
        rationalvector = True
        while rationalvector:
            factor = 1
            for j in ker[i]:
                if j%1 != 0:
                    factor = min(factor,j%1)
            if factor == 1:
                rationalvector = False
            else:
                ker[i]=ker[i] / factor

    # Define symbolic variables zs for each row (index 0 in Python)
    sym_str_x = 'x:' + str(A.shape[1])
    xs = symbols(sym_str_x)

    # Define symbolic variables ws for each column (index 0 in Python)
    sym_str_y = 'y:' + str(A.shape[1])
    ys = symbols(sym_str_y)

    sys = []
    for k in ker:
        sys.append(to_lawrence_polynomial(k,xs,ys))

    return(sys, xs, ys)
```

```python
LA, xs, ys = lawrence_polynomial_ideal(A)
print("Lawrence polynomial ideal")
print(LA)
randvars = tuple(random.sample(xs + ys, len(xs + ys)))
result_lawrence = groebner(LA, randvars, order='lex')
grav = []
for g in result_lawrence:
    for y in ys:
        g = g.subs({(y,1)})
    grav.append(g)

print("Reduced Graver (Groebner since we computed it with the Lawrence ideal) basis")
groebner(grav, xs, order='grevlex')
```

    Lawrence polynomial ideal
    [-x0**3*y1**2*y2**2 + x1**2*x2**2*y0**3, x0**5*x3**2*y1**4 - x1**4*y0**5*y3**2]
    Reduced Graver (Groebner since we computed it with the Lawrence ideal) basis

$\displaystyle GroebnerBasis\left(\left( x_{0}^{5} x_{3}^{2} - x_{1}^{4}, \  - x_{0}^{3} + x_{1}^{2} x_{2}^{2}\right), \left( x_{0}, \  x_{1}, \  x_{2}, \  x_{3}\right)\right)$

## Let's go back to the slides

```python
# If using this on Google collab, if not we can import 4ti2
try:
  import google.colab
  IN_COLAB = True
except:
  IN_COLAB = False

# Let's start with Pyomo
if not IN_COLAB:
    from Py4ti2int32 import *
```

```python
# Define rules to choose augmentation element, either the best one (argmin) or the first one that is found
def argmin(iterable):
    return min(enumerate(iterable), key=lambda x: x[1])

def greedy(iterable):
    for i, val in enumerate(iterable):
        if val[1] != 0:
            return i, val
    else:
        return i, val
```

```python
# Bisection rules for finding best step size
def bisection(g: np.ndarray, fun: Callable, x: np.ndarray, x_lo: np.ndarray = None, x_up: np.ndarray = None, laststep: np.ndarray = None) -> (float, int):
    if np.array_equal(g, laststep):
        return (fun(x), 0)
    if x_lo is None:
        x_lo = np.zeros_like(x)
    if x_up is None:
        x_up = np.ones_like(x)*max(x)*2

    u = max(x_up) - min(x_lo)
    l = -(max(x_up) - min(x_lo))
    for i, gi in enumerate(g):
        if gi >= 1:
            if np.floor((x_up[i] - x[i]) / gi) < u:
                u = int(np.floor((x_up[i] - x[i]) / gi))
            if np.ceil((x_lo[i] - x[i]) / gi) > l:
                l = int(np.ceil((x_lo[i] - x[i]) / gi))
        elif gi <= -1:
            if np.ceil((x_up[i] - x[i]) / gi) > l:
                l = int(np.ceil((x_up[i] - x[i]) / gi))
            if np.floor((x_lo[i] - x[i]) / gi) < u:
                u = int(np.floor((x_lo[i] - x[i]) / gi))
    alpha = u

    while u - l > 1:
        if fun(x + l*g) < fun(x + u*g):
            alpha = l
        else:
            alpha = u
        p1 = int(np.floor((l+u)/2) - 1)
        p2 = int(np.floor((l+u)/2))
        p3 = int(np.floor((l+u)/2) + 1)
        if fun(x + p1*g) < fun(x + p2*g):
            u = int(np.floor((l+u)/2))
        elif fun(x + p3*g) < fun(x + p2*g):
            l = int(np.floor((l+u)/2) + 1)
        else:
            alpha = p2
            break

    if fun(x + l*g) < fun(x + u*g) and fun(x + l*g) < fun(x + alpha*g):
        alpha = l
    elif fun(x + u*g) < fun(x + alpha*g):
        alpha = u

    return (fun(x + alpha*g), alpha)
```

```python
# We can just have a single step move (works well with greedy approach)
def single_move(g: np.ndarray, fun: Callable, x: np.ndarray, x_lo: np.ndarray = None, x_up: np.ndarray = None, laststep: np.ndarray = None) -> (float, int):
    if x_lo is None:
        x_lo = np.zeros_like(x)
    if x_up is None:
        x_up = np.ones_like(x)*max(x)*2

    alpha = 0

    if (x + g <= x_up).all() and (x + g >= x_lo).all():
        if fun(x + g) < fun(x):
            alpha = 1
    elif (x - g <= x_up).all() and (x - g >= x_lo).all():
        if fun(x - g) < fun(x) and fun(x - g) < fun(x + g):
            alpha = -1

    return (fun(x + alpha*g), alpha)
```

```python
EXAMPLE = 3

# Each example defines a problem min f(x)=cx s.t. Ax=b, x_lo <= x <= x_up, x integer

if EXAMPLE == 1:
    # Fist example - Illustrative example Alghassi et al. Graver Bases via Quantum Annealing
    # with application to non-linear integer programs
    A = np.array([[1, 1, 1, 1], [1, 5, 10, 25]])
    b = np.array([[21],[156]])
    # Objective is to minimize distance to 5
    x0 = np.array([1,15,3,2])
    x_lo = np.zeros_like(x0)
    x_up = 15*np.ones_like(x0)
elif EXAMPLE == 2:
    # Second example
    A = np.array([[1, 1, 1, 1], [1, 2, 3, 4]])
    b = np.array([[10], [21]])
    c = np.array([0, 1, 0, 2])
    x0 = np.array([1,8,0,1])
    x_lo = np.zeros_like(x0)
    x_up = 21*np.ones_like(x0)
elif EXAMPLE == 3:
    # Third example
    A = np.array([[1, 1, 1, 1], [0, 1, 2, 3]])
    b = np.array([[10],[15]])
    c = np.array([1, 3, 14, 17])
    x0 = np.array([3,0,6,1])
    x_lo = np.zeros_like(x0)
    x_up = 10*np.ones_like(x0)
elif EXAMPLE == 4:
    # Fourth example
    A = np.array([[1,1,1,1,1,1,1,1,1,1,0,1,0,1,0,1,0,1,1,1,0,1,0,1,0],
    [1,1,1,1,0,1,0,1,0,0,1,0,0,0,1,0,0,1,0,1,1,1,1,1,1],
    [0,1,0,0,0,1,0,1,0,1,1,0,1,1,0,1,1,0,0,1,0,0,1,1,1],
    [0,0,0,0,0,0,0,1,0,1,1,1,0,1,1,1,1,0,0,1,0,0,0,0,0],
    [0,1,1,1,1,1,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,1,0,1,0]])
    b = np.array([[9], [8], [7], [5], [5]])
    # Objective is quadratic expression
    x0 = np.array([1, 1, 1, 1, -1, 1, 1, 1, 1, 1, 1, 1, -2,
               1, 0, -1, 0, 1, -1, 1, -2, -2, 1, 1, 1])
    x_lo = np.zeros_like(x0)
    x_up = np.ones_like(x0)
else:
    print("Example not implemented")

if not IN_COLAB:
    r = graver("mat", A.tolist())
else:
    if EXAMPLE == 1:
        # Taken from paper but the same as computed with 4ti2
        r = [[5, -6, 0, 1], [5, -9, 4, 0], [0, 3, -4, 1], [5, -3, -4, 2], [5, 0, -8, 3]]
    elif EXAMPLE == 2:
        r = [[1, -2, 1, 0], [2, -3, 0, 1], [1, -1, -1, 1], [0, 1, -2, 1], [1, 0, -3, 2]]
    elif EXAMPLE == 3:
        r = [[2, -3, 0, 1], [1, -2, 1, 0], [1, -1, -1, 1], [0, 1, -2, 1], [1, 0, -3, 2]]
    elif EXAMPLE == 4:
        print("The Graver basis has 29789 elements in this case")
    else:
        print("Example not implemented")

r = np.array(r)
```

```python
# Objective function definition
def f(x):
    if EXAMPLE == 1:
        return np.sum(np.abs(x - 5))
    elif EXAMPLE == 4:
        epsilon = 0.01
        mu = np.random.rand(len(x0))
        sigma = np.multiply(np.random.rand(len(x0)),mu)
        return -np.dot(mu, x) + np.sqrt(((1-epsilon)/epsilon)*np.dot(sigma**2, x**2))
    else:
        return np.dot(c,x)

# Constraints definition
def const(x):
    return np.array_equiv(np.dot(A,x),b.T) or np.array_equiv(np.dot(A,x),b)

```

```python
# Let's perform the augmentation
OPTION = 1 # Best augmentation, select using bisection rule
# OPTION = 2 # Greedy augmentation, select using bisection rule
# OPTION = 3 # Greedy augmentation, select using first found

dist = 1
gprev = None
k = 1
print("Initial point:", x0)
while dist != 0:
    if OPTION == 1:
        g1, (obj, dist) = argmin(
            bisection(e, f, x0, laststep=gprev, x_lo=x_lo, x_up=x_up) for e in r)
    elif OPTION == 2:
        g1, (obj, dist) = greedy(
            bisection(e, f, x0, laststep=gprev, x_lo=x_lo, x_up=x_up) for e in r)
    elif OPTION == 3:
        g1, (obj, dist) = greedy(
            single_move(e, f, x0, x_lo=x_lo, x_up=x_up) for e in r)
    else:
        print("Option not implemented")
        break
    x0 = x0 + r[g1]*dist
    gprev = r[g1]
    print("Iteration ", k)
    print(g1, (obj, dist))
    print("Augmentation direction:", gprev)
    print("Distanced moved:", dist)
    print("Step taken:", r[g1]*dist)
    print("Objective function:", obj)
    print("Current point:", x0)
    print("Are constraints satisfied?", const(x0))
    k += 1
```

    Initial point: [3 0 6 1]
    Iteration  1
    1 (77, -3)
    Augmentation direction: [ 1 -2  1  0]
    Distanced moved: -3
    Step taken: [-3  6 -3  0]
    Objective function: 77
    Current point: [0 6 3 1]
    Are constraints satisfied? True
    Iteration  2
    3 (69, 1)
    Augmentation direction: [ 0  1 -2  1]
    Distanced moved: 1
    Step taken: [ 0  1 -2  1]
    Objective function: 69
    Current point: [0 7 1 2]
    Are constraints satisfied? True
    Iteration  3
    0 (69, 0)
    Augmentation direction: [ 2 -3  0  1]
    Distanced moved: 0
    Step taken: [0 0 0 0]
    Objective function: 69
    Current point: [0 7 1 2]
    Are constraints satisfied? True

```python

```

notebooks/Notebook 3 - Graver basis.ipynb


```python
import numpy as np
import pandas as pd
import pickle

from ruspy.estimation.estimation import estimate
from ruspy.simulation.simulation import simulate
from ruspy.model_code.fix_point_alg import calc_fixp
from ruspy.model_code.cost_functions import lin_cost
from ruspy.model_code.cost_functions import calc_obs_costs
from ruspy.estimation.estimation_transitions import create_transition_matrix
from promotion.replication.auxiliary_iskhakov import (process_result, check_simulated_data)
```

---
# Replication of Iskhakov et al. (2016)
---

In this notebook we seek to replicate Iskhakov et al. (2016) with some notable differences in the setup which we will explain in the course of this notebook. Before making your way through this notebook it is advisable to first have a look at the [other replication notebook](https://github.com/OpenSourceEconomics/ruspy/blob/master/promotion/replication/replication.ipynb) to get you started on how to use the ruspy package.

To familiarize you with the set up in the paper let us start with a few key points to understand the remainder. Iskhakov et al. (2016) follow up on Su and Judd (2012) in which they had compared the performance of Mathematical Programming with Equlibrium Constraints (MPEC) and the Nested Fixed Point Algorithm (NFXP) for the canonical structural model of bus engine replacement based on Rust (1987). The aim of Iskhakov et al. is mainly to improve the setup of the NFXP of Su and Judd following the explanations of Rust (2000). Their revised comparison between MPEC and NFXP shows that both approaches perform similarly for a Monte Carlo simulation of the bus engine replacement problem.

We replicate parts of this Monte Carlo simulation below using our ruspy package in which we have implemented MPEC as well as the NFXP in pure Python. This is gives us the flexibility to easily experiment with key assumptions such as tolerance or optimization algorithms used in one common interface.

In the boxes just below we obtain all the key results needed to replicate Table I of Iskhakov et al. For that we simulate the data using the ruspy package and based on the data generating process described in Iskhakov et al. (2016). There they use the setup in which we have 120 time periods and 50 buses. The cost function is assumed to be linear taking the following form $c(x, \theta_1) = 0.001 \theta_{11} x$. The true parameters are the following:

\begin{equation}
RC=11.7257,  \\
\theta_{11}=2.4569, \\
\theta_3=(0.0937, 0.4475, 0.4459, 0.0127, 0.0002).
\end{equation}

The true parameter for $\beta$ is varied: $\beta\in \{0.975, 0.985, 0.995, 0.999, 0.9995, 0.9999\}$. For each $\beta$ 250 data sets are simulated. Those are then estimated using five different starting values for the replacement cost $RC$ and the linear cost parameter $\theta_{11}$. The following are the five different starting values:

\begin{align}
(RC^0, \theta^0_{11}) \in \{(4,1), (5,2), (6,3), (7,4), (8,5)\}
\end{align}

In the case of MPEC also starting values for the discretized expected value function are necessary and they are set to the zero vector every time: $EV^0_1,...,EV^0_{175}=(0,...,0)$. The subscript of 175 also displays the grid size that Iskhakov et al. choose to discretize the continuous variable of mileage. The starting values of the transition probabilities are frequency based in the paper and we follow this approach in our implementation.

All those ingredients can be found below in the code. Further in our code for the NFXP we specify stopping criteria for the fixed point calculation as well as the tolerance at which we switch from contraction steps to Newton-Kantorovich (N-K) steps. Here, there is a first difference in our implementation of the NFXP as there is no switching back from N-K to contraction steps implemented. Furthermore the switching tolerance is solely an absolute one. For the routine to maximize the likelihood function we rely on the scipy L-BFGS-B povided by estimagic which uses a combination of relative and absolute stopping tolerance which also does not exactly match the original paper. Further the authors employ the BHHH algorithm instead of the L-BFGS-B which we will follow as well as soon as the algorithm is available on estimagic.

For MPEC we could not rely on KNITRO as in the paper as it is not freely available. In ruspy it is possible to choose from the library of nlopt or use ipopt for MPEC (please see the References below for links to the respective documentation). For the recreation of the table in Iskhakov et al. we decided to use nlopt here. Again the stopping tolerances cannot exactly be set to those of KNITRO in the paper. Another notable difference is that we only give analytical first order derivatives to nlopt. In the paper, on top of that second order analytical derivatives are provided by using automatic differentiation and also the sparsity patterns of both order derivatives are passed in. Apart from that, though, our setup replicates the paper by using upper and lower bounds as well as a properly recentering the expected value function.

A last difference in our setup that affects both NFXP and MPEC is that we estimate the transition probabilities separately from the cost parameters. In the original paper for MPEC those are estimated jointly and for the NFXP the cost parameters are first estimated partially and then after that together with the transition probabilities.

For the sake of simplicity and comprehensibility, we decide to solely present the output for the discount factor $0.975$ paired with 100 simulation runs and only $(4, 1)$ as starting values for the cost parameter. This setup is selected below. But we also give the option to run the full simulation as done in Iskhakov et al.

```python
discount_factor = [0.975]
number_runs = 100
starting_cost_params = np.array([[4], [1]])
```

In order to run the full simulation as described above, please uncomment the cell below and run it. The simulation then might take quite a while.

```python
# # uncomment and run this cell for a whole replication of Iskhakov et al. (2016)
# discount_factor = [0.975, 0.985, 0.995, 0.999, 0.9995, 0.9999]
# number_runs = 250
# starting_cost_params = np.vstack((np.arange(4,9), np.arange(1,6)))
```

For ruspy to run the estimation using either NFXP or MPEC we only need to pass in a dictionairy plus the data to the function `estimate(init_dict, data)`. Below we create one dictionairy per approach (NFXP or MPEC) and adapt it accordingly within the nested for-loop below depending on which discount factor $\beta$ and which starting values are used in the respective run.

```python
# Initialize the simulation
approach = ["NFXP", "MPEC"]
starting_expected_value_fun = np.zeros(175)
number_buses = 50
number_periods = 120
number_states = 175
number_cost_params = 2
scale = 1e-3

# Initialize the set up for the nested fixed point algorithm
stopping_crit_fixed_point = 1e-13
switch_tolerance_fixed_point = 1e-2

# Initialize the set up for MPEC
lower_bound = np.concatenate((np.full(number_states, -np.inf), np.full(number_cost_params, 0.0)))
upper_bound = np.concatenate((np.full(number_states, 50.0), np.full(number_cost_params, np.inf)))
rel_ipopt_stopping_tolerance = 1e-6

init_dict_nfxp = {
    "model_specifications": {
        "number_states": number_states,
        "maint_cost_func": "linear",
        "cost_scale": scale,
    },
    "optimizer": {
        "approach": "NFXP",
        "algorithm": "scipy_L-BFGS-B",
        # implies that we use analytical first order derivatives as opposed to numerical ones
        "gradient": "Yes",
    },
    "alg_details": {
        "threshold": stopping_crit_fixed_point,
        "switch_tol": switch_tolerance_fixed_point,
    },
}

init_dict_mpec = {
    "model_specifications": {
        "number_states": number_states,
        "maint_cost_func": "linear",
        "cost_scale": scale,
    },
    "optimizer": {
        "approach": "MPEC",
        "algorithm": "LD_SLSQP",
        # implies that we use analytical first order derivatives as opposed to numerical ones
        "gradient": "Yes",
        "set_ftol_rel": 1e-6,
        "set_xtol_rel": 1e-15,
        "set_lower_bounds": lower_bound,
        "set_upper_bounds": upper_bound,
    },
}
```

```python
# Initialize DataFrame to store the results of each run of the Monte Carlo simulation
index = pd.MultiIndex.from_product([discount_factor,
                                    range(number_runs),
                                    range(starting_cost_params.shape[1]),
                                    approach],
                                   names=["Discount Factor", "Run", "Start", "Approach"])

columns=["RC", "theta_11", "theta_30", "theta_31", "theta_32", "theta_33",
         "CPU Time", "Converged", "# of Major Iter.", "# of Func. Eval.",
         "# of Bellm. Iter.", "# of N-K Iter."]

results = pd.DataFrame(index=index, columns=columns)
```

```python
# set seed
np.random.seed(123)

saved_data = {}
# Main loop to calculate the results for each run
for factor in discount_factor:
    saved_data[factor] = {}
    # set up simulation of data
    init_dict_simulation = {
        "simulation": {
            "discount_factor": factor,
            "periods": number_periods,
            "buses": number_buses,
        },
    }
    params = np.array([11.7257, 2.4569])
    trans_probs = np.array([0.0937, 0.4475, 0.4459, 0.0127, 0.0002])

    # Calculate objects necessary for the simulation process. See documentation for details.
    costs = calc_obs_costs(number_states, lin_cost, params, scale)
    trans_mat = create_transition_matrix(number_states, trans_probs)
    ev = calc_fixp(trans_mat, costs, factor)[0]

    for run in range(number_runs):
        # simulate the data
        data = simulate(init_dict_simulation["simulation"], ev, costs, trans_mat)
        saved_data[factor][run] = data
        for start in range(starting_cost_params.shape[1]):
            # Adapt the Initiation Dictionairy of NFXP for this run
            init_dict_nfxp["model_specifications"]["discount_factor"] = factor
            init_dict_nfxp["optimizer"]["params"] = pd.DataFrame(starting_cost_params[:, start], columns=["value"])

            # Run NFXP using ruspy
            transition_result_nfxp, cost_result_nfxp = estimate(init_dict_nfxp, data)

            # store the results of this run
            results.loc[factor, run, start, "NFXP"] = process_result(
                "NFXP", transition_result_nfxp, cost_result_nfxp, number_states)

            # Adapt the Initiation Dictionairy of MPEC for this run
            init_dict_mpec["model_specifications"]["discount_factor"] = factor
            init_dict_mpec["optimizer"]["params"] = np.concatenate((
                starting_expected_value_fun, starting_cost_params[:, start]))

            # Run MPEC using ruspy
            transition_result_mpec, cost_result_mpec = estimate(init_dict_mpec, data)

            # store the results of this run
            results.loc[
                factor, run, start, "MPEC"].loc[
                ~results.columns.isin(["# of Bellm. Iter.", "# of N-K Iter."])] = process_result(
                        "MPEC", transition_result_mpec, cost_result_mpec, number_states)
```

To give you an idea of what the above loop has just produced, let us have a look at the results.

```python
results
```

      RC
      theta_11
      theta_30
      theta_31
      theta_32
      theta_33
      CPU Time
      Converged
      # of Major Iter.
      # of Func. Eval.
      # of Bellm. Iter.
      # of N-K Iter.

      Discount Factor
      Run
      Start
      Approach

      0.975
      0
      0
      NFXP
      15.2277
      3.6888
      0.0913333
      0.4475
      0.446
      0.0146667
      0.62913
      1
      14
      15
      320
      108

      MPEC
      15.2386
      3.69232
      0.0913333
      0.4475
      0.446
      0.0146667
      0.642238
      1
      14
      14
      NaN
      NaN

      1
      0
      NFXP
      11.9636
      2.49321
      0.091
      0.4535
      0.444833
      0.0106667
      0.746165
      1
      14
      17
      355
      120

      MPEC
      11.9649
      2.49367
      0.091
      0.4535
      0.444833
      0.0106667
      0.573166
      1
      13
      13
      NaN
      NaN

      2
      0
      NFXP
      12.2255
      2.35604
      0.0825
      0.460333
      0.444833
      0.0121667
      0.807471
      1
      16
      18
      352
      124

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      97
      0
      MPEC
      12.3231
      2.66631
      0.0918333
      0.447
      0.448833
      0.0123333
      0.753175
      1
      13
      13
      NaN
      NaN

      98
      0
      NFXP
      10.9552
      2.19758
      0.0913333
      0.4465
      0.447333
      0.0145
      0.86645
      1
      14
      17
      357
      112

      MPEC
      10.9566
      2.19815
      0.0913333
      0.4465
      0.447333
      0.0145
      0.684407
      1
      12
      12
      NaN
      NaN

      99
      0
      NFXP
      11.4834
      2.33454
      0.0941667
      0.4485
      0.444667
      0.0125
      0.861002
      1
      14
      16
      331
      111

      MPEC
      11.4843
      2.33482
      0.0941667
      0.4485
      0.444667
      0.0125
      0.755911
      1
      13
      13
      NaN
      NaN

200 rows × 12 columns

As we have actually relied on our own simulated data and do not use the one provided by Iskhakov et al., we calculate some some key statistics of theirs and ours to validate our simulated data and ensure that our results are actually comparrable to theirs. Some selected statistics are presented below for our simulated data sets.

```python
check_simulated_data(saved_data, discount_factor, number_runs)
```

      Average State at Replacement
      Average of all States
      Average Replacement

      Discount Factor

      0.975
      125.362
      60.1843
      0.00715833

This is followed by the same statistics for the simulated data of Iskhakov et al.

```python
pd.read_pickle("check_simulated_data_iskhakov.pickle").loc[discount_factor, :]
```

      Average State at Replacement
      Average of all States
      Average Replacement

      Discount Factor

      0.975
      125.012
      60.0888
      0.00714533

---
### Table I from Su & Judd (2012)
---

Before we proceed to Table I in Iskhakov et al., we give out the means and standard deviations of our parameter estimations like it was done by Su and Judd (2012) in Table I.

```python
# Create Table I from Su & Judd (2012) with the simulated values from Iskahkov et al. (2016)
columns_table_1 = ["RC", "theta_11", "theta_30", "theta_31", "theta_32", "theta_33"]
table_1_temp = results.loc[results["Converged"] == 1, columns_table_1].astype(float).groupby(
    level=["Discount Factor", "Approach"])

statistic = ["Mean", "Standard Deviation"]
index = pd.MultiIndex.from_product([discount_factor, approach, statistic],
                                   names=["Discount Factor", "Approach", "Statistic"])
table_1 = pd.DataFrame(index=index, columns=columns_table_1)
table_1.loc(axis=0)[:,:,"Mean"] = table_1_temp.mean()
table_1.loc(axis=0)[:,:,"Standard Deviation"] = table_1_temp.std()
```

```python
table_1.astype(float).round(3)
```

      RC
      theta_11
      theta_30
      theta_31
      theta_32
      theta_33

      Discount Factor
      Approach
      Statistic

      0.975
      NFXP
      Mean
      11.983
      2.525
      0.094
      0.447
      0.446
      0.013

      Standard Deviation
      1.479
      0.434
      0.004
      0.006
      0.006
      0.002

      MPEC
      Mean
      11.986
      2.526
      0.094
      0.447
      0.446
      0.013

      Standard Deviation
      1.481
      0.434
      0.004
      0.006
      0.006
      0.002

---
### Corresponding (reduced) results from Iskhakov et al. (2016)
---

To get an idea of the quality of our estimates, we compare this now to the results obtained by Iskhakov et al. using their original matlab code for the implementation of their NFXP. Those results were not published in the paper.

```python
# Create a Table with the results
index = pd.MultiIndex.from_product([discount_factor, statistic],
                                   names=["Discount Factor", "Statistic"])
NFXP_Iskhakov = pd.DataFrame(index=index, columns=["RC", "theta_11"])
# load stored results
results_iskhakov = pickle.load(open("results_iskhakov.pickle", "rb"))
for factor in discount_factor:
    NFXP_Iskhakov_temp = results_iskhakov[factor]
    NFXP_Iskhakov.loc[factor, "Mean"] = NFXP_Iskhakov_temp.mean(axis=0)
    NFXP_Iskhakov.loc[factor, "Standard Deviation"] = NFXP_Iskhakov_temp.std(axis=0)
```

```python
NFXP_Iskhakov.astype(float).round(3)
```

      RC
      theta_11

      Discount Factor
      Statistic

      0.975
      Mean
      11.914
      2.508

      Standard Deviation
      1.517
      0.468

---
### Table I from Iskhakov et al. (2016)
---

Lastly, below we create our results of Table I of Iskhakov et al. (2016). Please be aware that both our implementations are following the original setup closely but not perfectly. Especially notable is that both our implementation of MPEC as well as NFXP are somewhat inferior to those of the paper as we do not provided the analytical Hessian plus sparsity patterns for MPEC and in the case of the NFXP no back-and-forth-switching for the fixed point calculation.

```python
# Create Table I from Iskhakov et al. (2016)
columns_table_2 = ["CPU Time", "Converged", "# of Major Iter.", "# of Func. Eval.", "# of Bellm. Iter.", "# of N-K Iter."]
table_2 = results[columns_table_2].astype(float).groupby(["Discount Factor", "Approach"]).mean()
table_2["Converged"] = (table_2["Converged"]*number_runs*starting_cost_params.shape[1]).astype(int)
```

```python
table_2.astype(float).round(3)
```

      CPU Time
      Converged
      # of Major Iter.
      # of Func. Eval.
      # of Bellm. Iter.
      # of N-K Iter.

      Discount Factor
      Approach

      0.975
      MPEC
      0.645
      100.0
      12.75
      12.75
      NaN
      NaN

      NFXP
      0.771
      100.0
      13.69
      15.36
      321.58
      105.68

### References

Gabler, J., "A Python Tool for the Estimation of (Structural) Econometric Models.", unpublished (2019), https://github.com/OpenSourceEconomics/estimagic

Iskhakov, F., Lee, J., Rust, J., Schjerning, B. and Seo, K. (2016), Comment on “Constrained Optimization Approaches to Estimation of Structural Models”. Econometrica, 84: 365-370. doi:10.3982/ECTA12605

Johnson, Steven G., The NLopt nonlinear-optimization package, http://github.com/stevengj/nlopt

Rust, John. "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher." Econometrica 55, no. 5 (1987): 999-1033. Accessed June 7, 2020. doi:10.2307/1911259.

Rust, John. "Nested fixed point algorithm documentation manual." Unpublished Manuscript (6) (2000): 1-43.

Su, C.‐L. and Judd, K.L. (2012), Constrained Optimization Approaches to Estimation of Structural Models. Econometrica, 80: 2213-2230. doi:10.3982/ECTA7925

Wächter, A. and Biegler, L. T., On the Implementation of a Primal-Dual Interior Point Filter Line Search Algorithm for Large-Scale Nonlinear Programming, Mathematical Programming 106(1), pp. 25-57, 2006 (preprint), https://github.com/coin-or/Ipopt