# Numerical Methods in Scientific Computing
# Assignment 2

# Q1.

We know that for cubic spline interpolation:

\begin{equation}
    g_i\ (x)=\frac{g_i^{\prime \prime}(x_i)}{6}(\frac{(x_{i+1}-x)^3}{h}-h(x_{i+1}-x))+\frac{g_{i+1}^{\prime \prime} (x_{i+1})}{6}(\frac{(x-x_i)^3}{h}-h(x-x_i\ ))+f_i(\frac{x_{i+1}-x}{h})+f_{i+1}(\frac{x-x_i}{h})
\end{equation}

So for the interpolating polynomial in the first interval and with parabolic run-out condition $g^{\prime \prime}(x_0)=g^{\prime \prime}(x_1)$ we have,

\begin{equation}
    g_0\ (x)=\frac{g^{\prime \prime}(x_0)}{6}(\frac{(x_{1}-x)^3}{h}-h(x_{1}-x))+\frac{g^{\prime \prime} (x_{0})}{6}(\frac{(x-x_0)^3}{h}-h(x-x_0\ ))+f_0(\frac{x_{1}-x}{h})+f_{1}(\frac{x-x_0}{h})
\end{equation}

$\Rightarrow$ The coefficients of $x^3$ in the above equation $-\frac{g^{\prime \prime}(x_0)}{6h}$ and $\frac{g^{\prime \prime}(x_0)}{6h}$ cancel out each other and we have a parabola(quadratic)

Similarly for the interpolating polynomial in the last interval and with parabolic run-out condition $g^{\prime \prime}(x_{n-1})=g^{\prime \prime}(x_n)$ we have,

\begin{equation}
    g_{n-1}\ (x)=\frac{g^{\prime \prime}(x_{n-1})}{6}(\frac{(x_{n}-x)^3}{h}-h(x_{n}-x))+\frac{g^{\prime \prime}(x_{n-1})}{6}(\frac{(x-x_{n-1})^3}{h}-h(x-x_{n-1}\ ))+f_{n-1}(\frac{x_{n}-x}{h})+f_{n}(\frac{x-x_{n-1}}{h})
\end{equation}

$\Rightarrow$ The coefficients of $x^3$ in the above equation $-\frac{g^{\prime \prime}(x_{n-1})}{6h}$ and $\frac{g^{\prime \prime}(x_{n-1})}{6h}$ cancel out each other and we have a parabola(quadratic)

Hence interpolating polynomials are parabolas at the first and last interval for cubic spline interpolation with parabolic run-out cinditions

# Q2.

Quadratic Spline Interpolation:

Say, for given set of points $\{x_i, f_i\}_{i=0}^n$ we have piecewise quadratic polynomials as
\begin{equation}
  g(x) =
\begin{cases}
      g_0(x) & , x \in [x_0,x_1] \\
      g_1(x) & , x \in [x_1,x_2] \\
      ... \\
      g_{n-1}(x) & , x \in [x_{n-1},x_n] \\
\end{cases}
\end{equation}

- Suitable joint conditions for quadratic spline

\begin{equation}
    g_i^\prime(x_{i+1})=g_{i+1}^\prime(x_{i+1}) \quad , \quad \forall x \in \{0,1,...,n-2\}
\end{equation}
\begin{equation}
    g_i(x_{i})=f_i \quad , \quad \forall x \in \{0,1,...,n-1\}
\end{equation}
\begin{equation}
    g_i(x_{i+1})=f_{i+1} \quad , \quad \forall x \in \{0,1,...,n-1\}
\end{equation}

In total there are (n-1) + (n) + (n) equations.

- Obtaining coefficients of the spline

The first derivative of the piecewise polynomial for Quadratic spline interpolation are linear. Hence, we have

\begin{equation}
    g_i^\prime(x)=g_i^\prime(x_i)\frac{(x_{i+1}-x)}{h}+g_i^\prime(x_{i+1})\frac{(x-x_{i})}{h}
\end{equation}

Integrating the above equation,

\begin{equation}
    g_i(x)=-g_i^\prime(x_i)\frac{(x_{i+1}-x)^2}{2h}+g_i^\prime(x_{i+1})\frac{(x-x_{i})^2}{2h}+C
\end{equation}

Using the conditions $g_i(x_{i})=f_i$ and $g_i(x_{i+1})=f_{i+1}$ to obtain two equations and subtracting one from the other to eliminate C we have over the entire range,

\begin{equation}
    (g^\prime(x_i) + g^\prime(x_{i+1})) = \frac{2}{h}(f_{i+1}-f_i) \quad \forall i \in \{0,1,...,n-1\}
\end{equation}

This form is similar to what was obtained in Cubic spline interpolation. In total there are n piecewise quadratic polynomials with 3n unknowns and we have already got 3n-1 unknowns. With another extra constraint we can solve the set of linear equations as a Linear system to obtain the first derivatives which are substituted back to get $g(x)$.

Say if the initial condition is given as $g_i^\prime(x_0) = 0$, then the form of equation reduces to

\begin{equation}
    g^\prime(x_{i+1}) = \frac{2}{h}(f_{i+1}-f_i)-g^\prime(x_i) \quad \forall i \in \{0,1,...,n-1\}
\end{equation}

With all the first derivatives computed at $\{x_i\}_{i=0}^n$ the piecewise quadratic equations are given by,

\begin{equation}
    g_i(x)=g^\prime(x_i)(-\frac{(x_{i+1}-x)^2}{2h}+\frac{h}{2})+g^\prime(x_{i+1})\frac{(x-x_{i})^2}{2h}+f_i \quad \forall i \in \{0,1,...,n-1\}
\end{equation}

- Suitable end Conditions

We require an additional constraint explicitly mentioned to solve the above system of equations. It could be one of the following:

\begin{equation}
    g^\prime(x_0)=0
\end{equation}

\begin{equation}
    g^\prime(x_n)=0
\end{equation}

\begin{equation}
    g^\prime(x_0)=g^\prime(x_1)
\end{equation}

\begin{equation}
    g^\prime(x_n)=g^\prime(x_{n-1})
\end{equation}

- Computational Efforts

    * In the case of Cubic we are required to solve the Linear system for second derivatives by computing inverse of the tridiagonal matrix whose computational cost is given by O(n) and then obtaining the piecewise cubic polynomial for the entire range is O(n). So, totally O(n)

    * In the case of Quadratic splines we are required to solve the Linear system for second derivatives by computing directly from n-1 equations without requirement of calculating inverse of a matrix although still the computational cost is given by O(n) and then obtaining the piecewise quadratic polynomial for the entire range is O(n). So, totally O(n)

```python
# Import the necessary libraries
import numpy as np
import matplotlib.pyplot as plt
from ipywidgets import interact
```

```python
# Function to compute the piecewise Quadratic splines
def  quadratic_spline(a, b, n):
    # Initializing required variables
    h = (b-a)/n
    x_runge = np.linspace(a,b,n+1)
    f_runge = 1/(1+25*x_runge**2)
    g_prime = np.zeros((n+1))

    # Initial condition
    g_prime[0] = 0

    # Calculating g_prime values at all nodes
    for i in range(0,n):
        g_prime[i+1] = (2/h)*(f_runge[i+1]-f_runge[i]) - g_prime[i]

    # Compute piecewise values for each of g(Interpolant), f(Runge Function), x(values to plot)
    g_piecewise = [0 for _ in range(n)]
    f_piecewise = [0 for _ in range(n)]
    x_piecewise = [0 for _ in range(n)]
    for i in range(0,n):
        x = np.linspace(a+i*h, a+(i+1)*h, 10)
        g_piecewise[i] = g_prime[i]*( ((-(x_runge[i+1]-x)**2)/(2*h)) + (h/2) ) + g_prime[i+1]*( (((x-x_runge[i])**2)/(2*h)) ) + f_runge[i]
        f_piecewise[i] = 1/(1+25*x**2)
        x_piecewise[i] = x

    return g_piecewise, f_piecewise, x_piecewise

# Function to convert peicewise interpolant to global quadratic spline interpolant
def piecewise_to_global(g_piecewise, f_piecewise, x_piecewise):
    n_pieces = np.shape(g_piecewise)[0]
    n_plot = np.shape(g_piecewise)[1]
    g = []
    f = []
    x = []

    for i in range(n_pieces):
        for j in range(n_plot):
            if j == n_plot-1 and i != n_pieces-1: continue
            g.append(g_piecewise[i][j])
            f.append(f_piecewise[i][j])
            x.append(x_piecewise[i][j])

    return g, f, x
```

```python
# Set Interval boundaries and No. of pieces in the overall interval
a = 0
b = 1
n = 8

# Call the functions defined in previous steps
g_piecewise, f_piecewise, x_piecewise = quadratic_spline(a, b, n)
g, f, x = piecewise_to_global(g_piecewise, f_piecewise, x_piecewise)

# Next few lines to model the plot of obtained results
plt.plot(x,f,color='r', label="Runge Function", linestyle='dashdot')
plt.plot(x,g,color='b', label="Interpolant")
plt.legend()
xcoords = np.linspace(a,b,n+1)
xcoords = xcoords[1:-1]
for xc in xcoords:
    plt.axvline(x=xc, linestyle='dashed')
plt.xlabel('x', fontsize=15)
plt.ylabel('f(x)/g(x)', fontsize=15)
plt.title('Quadratic Spline Interpolation - Runge Function \n (Note: Vertical Lines indicate each interval) \nn=%i' %n, fontsize=15)
plt.xlim(a, b)
plt.rcParams["figure.figsize"] = [10,10]
plt.show()
```

```python
# Interactive Version
# [Need to run this cell again when opening the jupyter notebook each time]

@interact
def inter(a=(-1,1,0.1),b=(-1,1,0.1), n=(1,20,1)):
    if b<=a : return print('Set b greater than a !')
    g_piecewise, f_piecewise, x_piecewise = quadratic_spline(a, b, n)
    g, f, x = piecewise_to_global(g_piecewise, f_piecewise, x_piecewise)
    plt.plot(x,f,color='r', label="Runge Function", linestyle='dashdot')
    plt.plot(x,g,color='b', label="Interpolant")
    plt.legend()
    xcoords = np.linspace(a,b,n+1)
    xcoords = xcoords[1:-1]
    for xc in xcoords:
        plt.axvline(x=xc, linestyle='dashed')
    plt.xlabel('x')
    plt.ylabel('y')
    plt.title('Quadratic Spline Interpolation - Runge Function\n a =%1.2f' % a + ', b =%1.2f' % b +', n =%i' % n, fontsize=15)
    plt.xlim(a, b)
    plt.rcParams["figure.figsize"] = [10,10]
    plt.show()
```

    interactive(children=(FloatSlider(value=0.0, description='a', max=1.0, min=-1.0), FloatSlider(value=0.0, descr…

# Q3.

- fourth order Pade approximation for the second derivative at the ith node

Say,

\begin{equation}
    y_i^{\prime\prime} = a_{-1}y_{i-1} + a_0y_{i} + a_1y_{i+1} + b_{-1}y_{i-1}^{\prime\prime} + b_{1}y_{i+1}^{\prime\prime}
\end{equation}

By taylor series expansion we have,

\begin{equation}
    y_{i\pm1} = y_i \pm hy_i^{\prime} + \frac{h^2}{2!}y_i^{\prime\prime} \pm \frac{h^3}{3!}y_i^{\prime\prime\prime}+\frac{h^4}{4!}y_i^{\prime\prime\prime\prime} + ...
\end{equation}

\begin{equation}
    y_{i\pm1}^{\prime\prime} = y_i^{\prime\prime} \pm hy_i^{\prime\prime\prime} + \frac{h^2}{2!}y_i^{\prime\prime\prime\prime} \pm \frac{h^3}{3!}y_i^{\prime\prime\prime\prime\prime} + ...
\end{equation}

Substituting back in the first equation and segregating by derivatives of y we have,
\begin{equation}
    y_i^{\prime\prime} = y_i(a_{1}+a_0+a_{-1}) + y_i^{\prime}(ha_{1}-ha_{-1}) + y_i^{\prime\prime}(\frac{h^2}{2!}a_{1}+\frac{h^2}{2!}a_{-1} + b_{1}+b_{-1}) + y_i^{\prime\prime\prime}(\frac{h^3}{3!}a_{1}-\frac{h^3}{3!}a_{-1} + hb_{1}-hb_{-1})+ y_i^{\prime\prime\prime\prime}(\frac{h^4}{4!}a_{1}+\frac{h^4}{4!}a_{-1} + \frac{h^2}{2!}b_{1}+\frac{h^2}{2!}b_{-1})+...
\end{equation}

Equating the coefficients of the derivatives of y from the above equation with the first equation. From the coefficients of $y_i^{\prime}$ and $y_i^{\prime\prime\prime}$ we can see that $a_1=a_{-1}$ and $b_1=b_{-1}$. The equations now reduce to,
\begin{equation}
    2a_1+a_0=0
\end{equation}

\begin{equation}
    h^2a_1+2b_1=1
\end{equation}

\begin{equation}
    \frac{2h^2}{4!}a_1+b_1=0
\end{equation}

Solving them and obtaining all the coefficients yields the following equation,

\begin{equation}
    f_{i-1}^{\prime\prime} + 10f_{i}^{\prime\prime} + f_{i+1}^{\prime\prime} = 12\frac{f_{i+1}-2f_{i}+f_{i-1}}{h^2} \quad \forall i \in \{2,...,N-1\}
\end{equation}

\begin{equation}
    \Rightarrow \frac{f_{i-1}^{\prime\prime}}{10} + f_{i}^{\prime\prime} + \frac{f_{i+1}^{\prime\prime}}{10} = \frac{12}{10}\frac{f_{i+1}-2f_{i}+f_{i-1}}{h^2} \quad \forall i \in \{2,...,N-1\}
\end{equation}

The nodes are,
\begin{equation}
    x_i = (i-0.5)h, \quad i \in \{1,2,...,N\}, \quad h=\frac{1-0}{N}
\end{equation}

The order of trailing terms is given by
\begin{equation}
    y_i^{\prime\prime\prime\prime\prime}(\frac{h^5}{5!}(a_1-a_{-1}) + \frac{h^3}{3!}(b_1-b_{-1})) + y_i^{\prime\prime\prime\prime\prime\prime}(\frac{h^6}{6!}(a_1+a_{-1}) + \frac{h^4}{4!}(b_1+b_{-1})) + ... = O(\frac{h^6}{h^2}) = O(h^4)
\end{equation}
\begin{equation}
    \Rightarrow y_i^{\prime\prime\prime\prime\prime\prime}(\frac{h^6}{6!}(a_1+a_{-1}) + \frac{h^4}{4!}(b_1+b_{-1})) + ... = O(\frac{h^6}{h^2}) = O(h^4)
\end{equation}

- For the left boundary
\begin{equation}
    y_1^{\prime\prime}+b_2y_2^{\prime\prime} = a_1y_1 + a_2y_2 + a_3y_3 + a_4y^{\prime}(0)+O(h^3)
\end{equation}

Writing as Taylor series expansions

\begin{equation}
    y_{2} = y_1 + hy_1^{\prime} + \frac{h^2}{2!}y_1^{\prime\prime} + \frac{h^3}{3!}y_1^{\prime\prime\prime}+\frac{h^4}{4!}y_1^{\prime\prime\prime\prime} + \frac{h^5}{5!}y_1^{\prime\prime\prime\prime\prime} + ...
\end{equation}

\begin{equation}
    y_{3} = y_1 + 2hy_1^{\prime} + \frac{(2h)^2}{2!}y_1^{\prime\prime} + \frac{(2h)^3}{3!}y_1^{\prime\prime\prime}+\frac{(2h)^4}{4!}y_1^{\prime\prime\prime\prime} + \frac{(2h)^5}{5!}y_1^{\prime\prime\prime\prime\prime} + ...
\end{equation}

\begin{equation}
    y^{\prime}(0) = y_1^{\prime} - \frac{h}{2}y_1^{\prime\prime} + \frac{(\frac{h}{2})^2}{2!}y_1^{\prime\prime\prime} - \frac{(\frac{h}{2})^3}{3!}y_1^{\prime\prime\prime\prime}+\frac{(\frac{h}{2})^4}{4!}y_1^{\prime\prime\prime\prime\prime} - \frac{(\frac{h}{2})^5}{5!}y_1^{\prime\prime\prime\prime\prime\prime} + ...
\end{equation}

\begin{equation}
    y_{2}^{\prime\prime} = y_1^{\prime\prime} + hy_1^{\prime\prime\prime} + \frac{h^2}{2!}y_1^{\prime\prime\prime\prime} + \frac{h^3}{3!}y_1^{\prime\prime\prime\prime\prime} + ...
\end{equation}

Substituting back and comparing the coefficients of the derivatives yield the following values for the coefficients,
\begin{equation}
    b_2 = \frac{-11}{23}
\end{equation}

\begin{equation}
    a_1 = \frac{-36}{23h^2}
\end{equation}

\begin{equation}
    a_2 = \frac{48}{23h^2}
\end{equation}

\begin{equation}
    a_3 = \frac{-12}{23h^2}
\end{equation}

\begin{equation}
    a_4 = \frac{-24}{23h}
\end{equation}

- Similarly for the right boundary.
\begin{equation}
    y_n^{\prime\prime}+b_2y_{n-1}^{\prime\prime} = a_1y_n + a_2y_{n-1} + a_3y_{n-2} + a_4y^{\prime}(1)+O(h^3)
\end{equation}

Writing as Taylor series expansions

\begin{equation}
    y_{n-1} = y_n - hy_n^{\prime} + \frac{h^2}{2!}y_n^{\prime\prime} - \frac{h^3}{3!}y_n^{\prime\prime\prime}+\frac{h^4}{4!}y_n^{\prime\prime\prime\prime} - \frac{h^5}{5!}y_n^{\prime\prime\prime\prime\prime} + ...
\end{equation}

\begin{equation}
    y_{n-2} = y_n - 2hy_1^{\prime} + \frac{(2h)^2}{2!}y_n^{\prime\prime} - \frac{(2h)^3}{3!}y_n^{\prime\prime\prime}+\frac{(2h)^4}{4!}y_n^{\prime\prime\prime\prime} - \frac{(2h)^5}{5!}y_n^{\prime\prime\prime\prime\prime} + ...
\end{equation}

\begin{equation}
    y^{\prime}(1) = y_n^{\prime} + \frac{h}{2}y_n^{\prime\prime} + \frac{(\frac{h}{2})^2}{2!}y_n^{\prime\prime\prime} + \frac{(\frac{h}{2})^3}{3!}y_n^{\prime\prime\prime\prime}+\frac{(\frac{h}{2})^4}{4!}y_n^{\prime\prime\prime\prime\prime} + \frac{(\frac{h}{2})^5}{5!}y_n^{\prime\prime\prime\prime\prime\prime} + ...
\end{equation}

\begin{equation}
    y_{n-1}^{\prime\prime} = y_n^{\prime\prime} - hy_n^{\prime\prime\prime} + \frac{h^2}{2!}y_n^{\prime\prime\prime\prime} - \frac{h^3}{3!}y_n^{\prime\prime\prime\prime\prime} + ...
\end{equation}

By Observation we can see that the above 5 equations can be conoverted to the 5 equations presented previously for left boundary condition by changing h with -h,
\begin{equation}
    b_2 = \frac{-11}{23}
\end{equation}

\begin{equation}
    a_1 = \frac{-36}{23h^2}
\end{equation}

\begin{equation}
    a_2 = \frac{48}{23h^2}
\end{equation}

\begin{equation}
    a_3 = \frac{-12}{23h^2}
\end{equation}

\begin{equation}
    a_4 = \frac{24}{23h}
\end{equation}

\begin{equation}
\end{equation}

- Linear System

Formulating in the form Ax=b,

$$ A= \left(\begin{matrix}1&\frac{-11}{23}&0&.&.&.&0\\\frac{1}{10}&1&\frac{1}{10}&0&.&.&0\\0&\frac{1}{10}&1&\frac{1}{10}&0&.&0\\.&.&.&.&.&.&.\\.&.&.&.&.&.&.\\.&.&.&.&\frac{1}{10}&1&\frac{1}{10}\\0&.&.&.&0&\frac{-11}{23}&1\end{matrix}\right) $$

$$ x= \left(\begin{matrix}y_1^{\prime\prime}\\y_2^{\prime\prime}\\.\\.\\y_{n-1}^{\prime\prime}\\y_{n}^{\prime\prime}\end{matrix}\right) $$

$$ b= \left(\begin{matrix}\frac{(-36y_1+48y_2-12y_3-24hy^\prime(0))}{23h^2}\\\frac{12(y_1-2y_2+y_3)}{10h^2}\\\frac{12(y_2-2y_3+y_4)}{10h^2}\\.\\.\\\frac{12(y_{n-2}-2y_{n-1}+y_n)}{10h^2}\\\frac{(-36y_n+48y_{n-1}-12y_{n-2}+24hy^\prime(1))}{23h^2}\end{matrix}\right) $$

\begin{equation}
    \Rightarrow \left(\begin{matrix}1&\frac{-11}{23}&0&.&.&.&0\\\frac{1}{10}&1&\frac{1}{10}&0&.&.&0\\0&\frac{1}{10}&1&\frac{1}{10}&0&.&0\\.&.&.&.&.&.&.\\.&.&.&.&.&.&.\\.&.&.&.&\frac{1}{10}&1&\frac{1}{10}\\0&.&.&.&0&\frac{-11}{23}&1\end{matrix}\right) \left(\begin{matrix}y_1^{\prime\prime}\\y_2^{\prime\prime}\\.\\.\\y_{n-1}^{\prime\prime}\\y_{n}^{\prime\prime}\end{matrix}\right) = \left(\begin{matrix}\frac{(-36y_1+48y_2-12y_3-24hy^\prime(0))}{23h^2}\\\frac{12(y_1-2y_2+y_3)}{10h^2}\\\frac{12(y_2-2y_3+y_4)}{10h^2}\\.\\.\\\frac{12(y_{n-2}-2y_{n-1}+y_n)}{10h^2}\\\frac{(-36y_n+48y_{n-1}-12y_{n-2}+24hy^\prime(1))}{23h^2}\end{matrix}\right)
\end{equation}

\begin{equation}
    \Rightarrow \left(\begin{matrix}1&\frac{-11}{23}&0&.&.&.&0\\\frac{1}{10}&1&\frac{1}{10}&0&.&.&0\\0&\frac{1}{10}&1&\frac{1}{10}&0&.&0\\.&.&.&.&.&.&.\\.&.&.&.&.&.&.\\.&.&.&.&\frac{1}{10}&1&\frac{1}{10}\\0&.&.&.&0&\frac{-11}{23}&1\end{matrix}\right) \left(\begin{matrix}y_1^{\prime\prime}\\y_2^{\prime\prime}\\.\\.\\y_{n-1}^{\prime\prime}\\y_{n}^{\prime\prime}\end{matrix}\right) = \left(\begin{matrix}\frac{-36}{23h^2}&\frac{48}{23h^2}&\frac{-12}{23h^2}&0&.&.&0\\\frac{12}{10h^2}&\frac{-24}{10h^2}&\frac{12}{10h^2}&0&.&.&0\\0&\frac{12}{10h^2}&\frac{-24}{10h^2}&\frac{12}{10h^2}&0&.&0\\.&.&.&.&.&.&.\\.&.&.&\frac{12}{10h^2}&\frac{-24}{10h^2}&\frac{12}{10h^2}&0\\0&.&.&0&\frac{12}{10h^2}&\frac{-24}{10h^2}&\frac{12}{10h^2}\\0&.&.&0&\frac{-12}{23h^2}&\frac{48}{23h^2}&\frac{-36}{23h^2}\end{matrix}\right)\left(\begin{matrix}y_1\\y_2\\.\\.\\.\\y_{n-1}\\y_n\end{matrix}\right)
\end{equation}

\begin{equation}
    Ay^{\prime\prime}=By
\end{equation}

\begin{equation}
    y^{\prime\prime}=x^3-y
\end{equation}

Substituting $y^{\prime\prime}$ as a function of $y$ we can solve the ODE with,
\begin{equation}
    y = (A+B)^{-1}Ax^3
\end{equation}

- Exact Solution

For the ODE given by,
\begin{equation}
    y^{\prime\prime}+y=x^3
\end{equation}

The solution is given by the equation
\begin{equation}
    y = c_1\cos{x}+c_2\sin{x}+x^3-6x
\end{equation}

Using the boundary conditions,
\begin{equation}
    y^{\prime}(0)=y^{\prime}(1)=0
\end{equation}

The coefficients $c_1$ and $c_2$ are,
\begin{equation}
    c_2=6 ; \quad c_1=\frac{6\cos{1}-3}{\sin{1}}=0.2873
\end{equation}
\begin{equation}
    \Rightarrow y = 0.2873\cos{x}+6\sin{x}+x^3-6x
\end{equation}

```python
# Function to find the numerical solution to ODE
def numerical_ODE(n_nodes):
    # Initialize the required parameters
    n =n_nodes
    h = 1/n
    x = np.array([(i-0.5)*h for i in range(1,n+1)])
    y_exact = 0.2873*np.cos(x)+6*np.sin(x)+x**3-6*x

    # Formulate the linear system derived earlier
    A = np.eye(n)
    A[0][1] = -11/23
    A[n-1][n-2] = -11/23
    for i in range(1,n-1):
        A[i][i+1] = 1/10
        A[i][i-1] = 1/10

    B = np.eye(n)
    B[0][0] = -36/(23*h**2)
    B[0][1] = 48/(23*h**2)
    B[0][2] = -12/(23*h**2)
    B[n-1][n-1] = -36/(23*h**2)
    B[n-1][n-2] = 48/(23*h**2)
    B[n-1][n-3] = -12/(23*h**2)
    for i in range(1,n-1):
        B[i][i+1] = 12/(10*h**2)
        B[i][i-1] = 12/(10*h**2)
        B[i][i] = -24/(10*h**2)

    # Solve the linear system to obtain second derivatives
    y_num = np.dot(np.dot(np.linalg.inv(A+B), A), np.power(x, 3))

    return y_num, y_exact, x
```

```python
# Plot the results to compare between Numerical and Exact solutions to the ODE for different values of n
fig = plt.figure(figsize=(20,20))

N = [10, 20, 50, 100, 200, 500, 1000]
for i, n in enumerate(N):
    plt.subplot(3, 3, i+1)
    y_num, y_exact, x = numerical_ODE(n)
    plt.plot(x,y_exact, marker='o',color='r', label="Exact Value", linestyle='dashdot')
    plt.plot(x,y_num,color='b', label="Numerical Value")
    plt.legend()
    plt.xlabel('x')
    plt.ylabel('y')
    plt.title('Numerical vs Exact solution for the ODE\n n =%i' %n)
    plt.xlim(0, 1)
```

The numerical and exact solutions are similar as seen in the plots for all values of n which indicates and verifies the derivation for numerical approximation of the ODE.

```python
# Plot for the max norm error in numerical solution as a function of n in log-log plot.
N = [10, 20, 50, 100, 200, 500, 1000]
error = []
for i, n in enumerate(N):
    y_num, y_exact, x = numerical_ODE(n)
    error.append(max((np.abs(y_exact-y_num))))
plt.figure(figsize=(5,5))
plt.loglog(N, error, marker='o',color='r', label="Max norm error", linestyle='dashed')
plt.xlabel('n (no. of nodes)', fontsize=12)
plt.ylabel('max norm Error', fontsize=12)
plt.title('Error in computation as a function of number of nodes\n n=[10, 20, 50, 100, 200, 500, 1000]', fontsize=15)
plt.legend()
plt.show()

# Plot for the max norm error in numerical solution as a function of n in log-log plot.
N = [50, 100, 200, 500, 1000, 1250, 1500, 1750]
error = []
for i, n in enumerate(N):
    y_num, y_exact, x = numerical_ODE(n)
    error.append(max((np.abs(y_exact-y_num))))
plt.figure(figsize=(5,5))
plt.loglog(N, error, marker='o',color='r', label="Max norm error", linestyle='dashed')
plt.xlabel('n (no. of nodes)', fontsize=12)
plt.ylabel('max norm Error', fontsize=12)
plt.title('Error in computation as a function of number of nodes\n n=[50, 100, 200, 500, 1000, 1250, 1500, 1750]', fontsize=15)
plt.legend()
plt.show()
```

There is an initial decrease in the error in log-log plot and then followed by a slight increase as number of nodes increases.
Plotting again with a different set of number of nodes we see a kind of quadratic increase in error.

- Neumann Boundary conditions

The Neumann Boundary conditions are enforced into the discretized boundary value problem when we used a third order Pade scheme to approximate $y_1^{\prime\prime}$ and $y_n^{\prime\prime}$ where we modeled each as a function of other parameters including $y^{\prime}(0)$ and $y^{\prime}(1)$ respectively.


## Closed-system Examples
This notebook will get you started with HOQST by introducing you to the functionality for solving closed-system equations.

### Define the Hamiltonian
The first step is to define a Hamiltonian. In this tutorial, we focus on a 2-level system with the following Hamiltonian:

$$H(s) = - \sigma_z$$

where $s= t/t_f$ is the dimensionless time and $t_f$ is the total evolution time. We use a constant Hamiltonian so that the simulation results can be trivially confirmed. The syntax is the same for time-dependent Hamiltonians. Let's first define the Hamiltonian by:

```julia
using OpenQuantumTools, OrdinaryDiffEq, Plots
# define the Hamiltonian
H = DenseHamiltonian([(s)->1.0], [-σz], unit=:ħ)
```

In this example, we use the `DenseHamiltonian` [type](https://docs.julialang.org/en/v1/manual/types/#man-types). It means that the underlying data structure of this Hamiltonian type is [Julia array](https://docs.julialang.org/en/v1/manual/arrays/). There exists a different Hamiltonian type named `SparseHamiltonian` that relies on [sparse array](https://docs.julialang.org/en/v1/stdlib/SparseArrays/) as its internal data structure. Sparsity can only provide performance improvement when the system size is large. So, as a rule of thumb, users should only consider using `SparseHamiltonian` when the system size is larger than 10 qubits.

The closed-system evolution is completely specified by the Hamiltonian and the initial state. We can combine them into a single `Annealing` object by:

```julia
# define the initial state by PauliVec[k][j],
# which is the jth eigenvector of the
# Pauli matrix σₖ
u0 = PauliVec[1][1]
# define total evolution time in (ns)
tf = 10
# combine H and u0 into an Annealing object
annealing = Annealing(H, u0)
```

The initial state in the above code block is
$$\lvert \phi(0) \rangle = \lvert + \rangle \ .$$

We will consider three variants of the closed-system equations in this tutorial.

### Schrodinger equation
We start with the Schrodinger equation
\begin{equation}
    \lvert \dot{\phi} \rangle = -i t_f H(s) \lvert \phi \rangle \ .
\end{equation}

To solve this differential equation, we need to choose a proper algorithm. HOQST relies on `OrdinaryDiffEq.jl` as the low-level solver, which supports a large collection of [algorithms](https://docs.sciml.ai/latest/solvers/ode_solve/). We do not guarantee compatibilities to every solver in this list. Users can try specific algorithms if they are interested. We recommend a list of algorithms we tested as follows:

1. The default Tsitouras 5/4 Runge-Kutta method(Tsit5()).

   This is the default method in `OrdinaryDiffEq` and works well in most cases.

2. A second-order A-B-L-S-stable one-step ESDIRK method(TRBDF2()).

   This is the method widely used in large scale classical circuit simulations. Because this is a second-order method, we recommended using a smaller error tolerance than with other higher-order methods.

3. A simple linear exponential method(LinearExponential()).

   This method discretizes the Hamiltonian and does the matrix exponential for each interval.

4. Adaptive exponential Rosenbrock methods(Exprb32()/Exprb43()).

   This method belongs to the adaptive exponential Runge-Kutta method family.

It is important to note that methods 3 and 4 are exponential methods that are supposed to preserve the state vectors' norm. To solve the Schrodinger equation, we use the function `solve_schrodinger`:

```julia
sol_tsit = solve_schrodinger(annealing, tf, alg=Tsit5(), abstol=1e-6, reltol=1e-6);
sol_trbdf = solve_schrodinger(annealing, tf, alg=TRBDF2(), abstol=1e-6, reltol=1e-6);
# LinearExponential is a fixed-stepsize method, the user needs to specify the time steps using the keyword argument `tstops`.
sol_linexp = solve_schrodinger(annealing, tf, alg=LinearExponential(), abstol=1e-6, reltol=1e-6, tstops=range(0,tf,length=100));
# Even though Exprb is an adaptive method, it tends to skip a lot of middle points. So if you want an accurate solution in the middle,
# it is better to manually add more points.
sol_exprb32 = solve_schrodinger(annealing, tf, alg=Exprb32(), tstops=range(0,tf,length=100));
```

In the above code block, the keyword arguments `abstol` and `reltol` are the absolute and relative error tolerances for [stepsize control](https://diffeq.sciml.ai/stable/basics/common_solver_opts/) in adaptive stepsize ODE algorithms. They are usually chosen by trial-and-error in real applications.

We plot the observable $\langle X \rangle$ during the evolution.

```julia
# this code block shows how to plot the expectation value of X
t_list = range(0,tf,length=100)
tsit = []
trbdf = []
linexp = []
exprb32 = []
for s in t_list
    # sol_tsit(s)'*σx*sol_tsit(s) calculates the
    # expectation value <ψ(s)|X|ψ(s)>
    push!(tsit, real(sol_tsit(s)'*σx*sol_tsit(s)))
    push!(trbdf, real(sol_trbdf(s)'*σx*sol_trbdf(s)))
    push!(linexp, real(sol_linexp(s)'*σx*sol_linexp(s)))
    push!(exprb32, real(sol_exprb32(s)'*σx*sol_exprb32(s)))
end
scatter(t_list[1:3:end], tsit[1:3:end], label="Tsit", marker=:+, markersize=8)
scatter!(t_list[2:3:end], trbdf[2:3:end], label="TRBDF")
scatter!(t_list[3:3:end], linexp[3:3:end], label="LinExp", marker=:d)
plot!(t_list, exprb32, label="Exprb", linestyle=:dash)
xlabel!("t (ns)")
ylabel!("<X>")
title!("Free Evolution")
```

### Other closed-system equations
The package also contains several other closed-system solvers.
#### Von Neumann equation
The Von Neumann equation is the "Schrodinger" equation for density matrices:

$$\dot{\rho} = -it_f[H(s), \rho] \ .$$

Even though the Von Neumann equation is equivalent to the Schrodinger equation, it is sometimes numerically more stable than the Schrodinger equation. Users are encouraged to try to solve it using different algorithms.

```julia
annealing = Annealing(H, u0)
sol_tsit = solve_von_neumann(annealing, tf, alg=Tsit5(), abstol=1e-6, reltol=1e-6)
```

As shown below, the solution given by the solver is the density matrix instead of the state vector:

```julia
sol_tsit(0.5)
```

##### Recommended algorithm
Only explicit methods are supported for solving equations w.r.t. density matrices. [Vectorization] (https://en.wikipedia.org/wiki/Vectorization_(mathematics)) is needed for implicit methods. This can be done by setting the `vectorize` keyword argument to be true. For example, in the following code block, we solve the Von Neumann equation with the TRBDF2 method:

```julia
sol_bdf = solve_von_neumann(annealing, tf, alg=TRBDF2(), reltol=1e-6, vectorize=true)
sol_bdf(0.5 * tf)
```

As shown above, the solution given by the solver becomes a vectorized version of the density matrix.

Side note: `TRBDF2` can actually work without vectorizing the Von Neumann equation. However, this is not generally true for other algorithms/solvers. For example, we will run into errors for `LinearExponential` method if no vectorization is performed:

```julia
sol_linexp = solve_von_neumann(annealing, tf, alg=LinearExponential(), tstops=range(0,tf,length=100));
sol_linexp = solve_von_neumann(annealing, tf, alg=LinearExponential(), tstops=range(0,tf,length=100), vectorize=true);
```

We can again plot the $\langle X \rangle$ for different methods:

```julia
sol_tsit = solve_von_neumann(annealing, tf, alg=Tsit5(), reltol=1e-6);
sol_trbdf = solve_von_neumann(annealing, tf, alg=TRBDF2(), reltol=1e-6);

sol_linexp = solve_von_neumann(annealing, tf, alg=LinearExponential(), tstops=range(0,tf,length=100), vectorize=true);

sol_exprb32 = solve_von_neumann(annealing, tf, alg=Exprb32(), tstops=range(0,tf,length=100), vectorize=true);

t_list = range(0,tf,length=100)
tsit = []
trbdf = []
linexp = []
exprb32 = []
for s in t_list
    push!(tsit, real(tr(sol_tsit(s)*σx)))
    push!(trbdf, real(tr(sol_trbdf(s)*σx)))
    push!(linexp, real(tr(σx*reshape(sol_linexp(s),2,2))))
    push!(exprb32, real(tr(σx*reshape(sol_exprb32(s),2,2))))
end
scatter(t_list[1:3:end], tsit[1:3:end], label="Tsit", marker=:+, markersize=8)
scatter!(t_list[2:3:end], trbdf[2:3:end], label="TRBDF")
scatter!(t_list[3:3:end], linexp[3:3:end], label="LinExp", marker=:d)
plot!(t_list, exprb32, label="Exprb", linestyle=:dash)
xlabel!("t (ns)")
ylabel!("<X>")
title!("Free Evolution")
```

#### Unitary
Lastly, we can also solve the unitary

$$U(s) = T_+ \exp\bigg\{ -i t_f \int_0^s H(s') \mathrm{d}s' \bigg\}$$

using `solve_unitary`. The ODE form of the problem is

$$\dot{U} = -i t_f H(s) U \ .$$

Although this is in principle equivalent to the Schrodinger/Von Neumann equation, the unitary becomes handy in certain cases, e.g., when solving the Redfield equation.

```julia
annealing = Annealing(H, u0)
sol_tsit = solve_unitary(annealing, tf, alg=Tsit5(),abstol=1e-6, reltol=1e-6)
sol_tsit(0.5 * tf)
```

Again we plot the $\langle X \rangle$ obtained by multiplying the unitary by the initial state.

```julia
sol_tsit = solve_unitary(annealing, tf, alg=Tsit5(), reltol=1e-6);
sol_trbdf = solve_unitary(annealing, tf, alg=TRBDF2(), reltol=1e-6, vectorize=true);
# LinearExponential is a fixed step size method, users need to specify the time steps using the keyword argument `tstops`.
sol_linexp = solve_unitary(annealing, tf, alg=LinearExponential(), tstops=range(0,tf,length=100), vectorize=true);
# Even though Exprb method is an adaptive method, it tends to skip a lot of middle points. So if you want an accurate evolution in the middle,
# it is better to manually add more points for the algorithm.
sol_exprb32 = solve_unitary(annealing, tf, alg=Exprb32(), tstops=range(0,tf,length=100), vectorize=true);

t_list = range(0,tf,length=100)
tsit = []
trbdf = []
linexp = []
exprb32 = []
for s in t_list
    state_tsit = sol_tsit(s) * u0
    state_trbdf = reshape(sol_trbdf(s), 2, 2) * u0
    state_linexp = reshape(sol_linexp(s), 2, 2) * u0
    state_exprb32 = reshape(sol_exprb32(s), 2, 2) * u0
    push!(tsit, real(state_tsit' * σx * state_tsit))
    push!(trbdf, real(state_trbdf' * σx * state_trbdf))
    push!(linexp, real(state_linexp' * σx * state_linexp))
    push!(exprb32, real(state_exprb32' * σx * state_exprb32))
end
scatter(t_list[1:3:end], tsit[1:3:end], label="Tsit", marker=:+, markersize=8)
scatter!(t_list[2:3:end], trbdf[2:3:end], label="TRBDF")
scatter!(t_list[3:3:end], linexp[3:3:end], label="LinExp", marker=:d)
plot!(t_list, exprb32, label="Exprb", linestyle=:dash)
xlabel!("t (ns)")
ylabel!("<X>")
title!("Free Evolution")
```

```julia
using HOQSTTutorials
HOQSTTutorials.tutorial_footer(WEAVE_ARGS[:folder],WEAVE_ARGS[:file])
```


# Diagrammatic Differentiation

**Quantum Group Workshop**, _26th January 2021_

Alexis Toumi, joint work with Richie Yeung and Giovanni de Felice

## 1) Parametrised matrices

Fix a rig $(\mathbb{S}, +, \times, 0, 1)$.

**Definition:** A matrix $f \in \mathbf{Mat}_\mathbb{S}(m, n)$ is a function $f : [m] \times [n] \to \mathbb{S}$ where $[n] = \{0, \dots, n - 1\}$ for $n \in \mathbb{N}$.

**Definition:** A parametrised matrix is a function $f : \mathbb{S} \to \mathbf{Mat}_\mathbb{S}(m, n)$, or equivalently a function-valued matrix $f \in \mathbf{Mat}_{\mathbb{S} \to \mathbb{S}}(m, n)$.

**Example:**

```python
from discopy import *
from sympy.abc import phi

Rz(phi).array
```

    array([[exp(-1.0*I*pi*phi), 0],
           [0, exp(1.0*I*pi*phi)]], dtype=object)

```python
(lambda phi: Rz(phi).array)(0.25)
```

    array([[0.70710678-0.70710678j, 0.        +0.j        ],
           [0.        +0.j        , 0.70710678+0.70710678j]])

## 2) Parametrised diagrams

Fix a monoidal signature $(\Sigma_0, \Sigma_1, \text{dom}, \text{cod} : \Sigma_1 \to \Sigma_0^\star)$
for $X^\star = \coprod_{n \in \mathbb{N}} X^n$ the free monoid.

**Definition:** An _abstract_ diagram $d \in \mathbf{C}_\Sigma(s, t)$ is defined by a list of
$\text{layers}(d) = (\text{left}_0, \text{box}_0, \text{right}_0), \dots, (\text{left}_n, \text{box}_n, \text{right}_n) \in \Sigma_0^\star \times \Sigma_1 \times \Sigma_0^\star$.

**Definition:** A _concrete_ diagram is an abstract diagram with a monoidal functor $F : \mathbf{C}_\Sigma \to \mathbf{Mat}_\mathbb{S}$.

**Definition:** A _parametrised_ diagram is a function $d : \mathbb{S} \to \mathbf{C}_\Sigma(s, t)$, or equivalently, a diagram with a monoidal functor $F : \mathbf{C}_\Sigma \to \mathbf{Mat}_{\mathbb{S} \to \mathbb{S}}$.

**Example:**

```python
from discopy import *

x, y, z = Dim(1), Dim(2), Dim(2)
f = tensor.Box('f', x, y, [phi + 1, -phi * 2])
g = tensor.Box('g', y @ y, z, [1, 0, 0, 0, 0, 0, 0, phi ** 2 + 1])

d = f @ f >> g
print(d)
d.draw(figsize=(2, 2))

d.eval().array
```

```python
d.subs(phi, 0.25).eval().array
```

    array([1.56250000000000, 0.265625000000000], dtype=object)

## 3) Product rule

We define the gradient of a parametrised matrix $f \in \mathbf{Mat}_{\mathbb{S} \to \mathbb{S}}(m, n)$ elementwise:
$$
\frac{\partial f}{\partial x}(i, j)
= \frac{\partial}{\partial x} f(i, j)
$$

Given a parametrised diagram $d, F$ we want a new diagram $\frac{\partial d}{\partial x}$ such that:
$$
F\big(\frac{\partial d}{\partial x}\big)
= \frac{\partial F(d)}{\partial x}
$$

We can do this by defining gradients as _formal sums of diagrams_ and using the product rule:
$$
\frac{\partial}{\partial x} (d \otimes d')
= \frac{\partial d}{\partial x} \otimes d'
+ d \otimes \frac{\partial d'}{\partial x}
$$
$$
\frac{\partial}{\partial x} (d \circ d')
= \frac{\partial d}{\partial x} \circ d'
+ d \circ \frac{\partial d'}{\partial x}
$$

**Example:**

```python
d.grad(phi).draw(figsize=(8, 3))
```

## 4) Chain rule

Given an arbitrary function $f : \mathbb{S} \to \mathbb{S}$,
we lift it to matrices by applying it elementwise.
Diagrammatically, we represent this as a _bubble_ around a subdiagram.

Gradients of bubbles are then given by the chain rule:

$$
\frac{\partial}{\partial x} f(d)
= \frac{\partial f}{\partial x} (d)
\times \frac{\partial d}{\partial x}
$$

where the elementwise product can be encoded as pre- and post-composition with spiders.

```python
g = tensor.Box('g', Dim(2), Dim(2), [2 * phi, 0, 0, phi + 1])
f = lambda d: d.bubble(func=lambda x: x ** 2, drawing_name="f")
lhs, rhs = tensor.Box.grad(f(g), phi), f(g).grad(phi)
drawing.equation(lhs, rhs, draw_type_labels=False)
```

## 5) Applications

* Gradients of quantum circuits using the parameter shift rule.
* Gradients of neural nets and classical post-processing with bubbles.
* Gradients of circuit functors for QNLP.


# Expectation Propagation

ReactiveMP comes with support for expectation propagation (EP). In this demo we illustrate EP in the context of state-estimation in a linear state-space model that combines a Gaussian state-evolution model with a discrete observation model. Here, the probit function links continuous variable $x_t$ with the discrete variable $y_t$. The model is defined as:

\begin{align}
    u &= 0.1 \\
    x_0 &\sim \mathcal{N}(0, 100) \\
    x_t &\sim \mathcal{N}(x_{t-1}+ u, 0.01) \\
    y_t &\sim \mathrm{Ber}(\Phi(x_t))
\end{align}

## Import packages

```julia
using Pkg
Pkg.activate("..")

using StatsFuns: normcdf
using Random
using PyPlot
using Revise

using ReactiveMP
using GraphPPL
using Rocket
```

      Activating environment at `~/.julia/dev/ReactiveMP/Project.toml`
    ┌ Info: Precompiling PyPlot [d330b81b-6aea-500a-939a-2ce795aea3ee]
    └ @ Base loading.jl:1342

##  Data generation

```julia
function generate_data(nr_samples::Int64)

    # hyper parameters
    u = 0.1

    # allocate space for data
    data_x = zeros(nr_samples + 1)
    data_y = zeros(nr_samples)

    # initialize data
    data_x[1] = -2

    # generate data
    for k = 2:nr_samples + 1

        # calculate new x
        data_x[k] = data_x[k-1] + u + sqrt(0.01)*randn()

        # calculate y
        data_y[k-1] = normcdf(data_x[k]) > rand()

    end

    # return data
    return data_x, data_y

end;
```

```julia
n = 40
```

    40

```julia
data_x, data_y = generate_data(n);
```

```julia
plt.figure()
plt.scatter(1:n, data_y, label="y")
plt.plot(data_x, label="x")
plt.xlabel("t")
plt.ylabel("x, y")
plt.legend()
plt.grid();
```

## Model specification

```julia
@model function probit_model(nr_samples::Int64)

    # allocate space for variables
    x = randomvar(nr_samples + 1)
    y = datavar(Float64, nr_samples)

    # specify uninformative prior
    x[1] ~ NormalMeanPrecision(0.0, 0.01)

    # create model
    for k = 2:nr_samples + 1
        x[k] ~ NormalMeanPrecision(x[k - 1] + 0.1, 100)
        y[k - 1] ~ Probit(x[k]) where {
            # Probit node by default uses RequireInbound pipeline with vague(NormalMeanPrecision) message as initial value for `in` edge
            # To change initial value use may specify it manually, like
            # pipeline = RequireInbound(in = NormalMeanPrecision(0, 1.0))
        }
    end

    # return parameters
    return x, y

end;
```

## Inference

```julia
function inference(data_y; nr_iterations::Int64=10)

    # fetch number of samples
    nr_samples = length(data_y)

    # create model
    model, (x, y) = probit_model(nr_samples)

    # allocate marginals and free energy
    mx = buffer(Marginal, nr_samples + 1)
    fe = []

    # subscribe to marginals
    x_sub = subscribe!(getmarginals(x), mx)

    # update data (i.e. run inference)
    for it = 1:nr_iterations
        update!(y, data_y)
    end

    # unsubscribe
    unsubscribe!(x_sub)

    # return marginals
    return mx

end;
```

```julia
mx = inference(data_y);
```

## Results

```julia
plt.figure()
plt.scatter(1:n, data_y, label="y", color="blue")
plt.plot(data_x, label="x (true)", color="blue")
plt.plot(mean.(mx), label="x (inferred mean)", color="orange")
plt.fill_between(0:n, mean.(mx) .- std.(mx), mean.(mx) + std.(mx), label="x (inferred ±σ)", alpha=0.3, color="orange")
plt.xlabel("t")
plt.ylabel("x, y")
plt.legend()
plt.grid();
```

```julia

```

```julia

```


# Finite Difference Approximation

```python
from sympy import *
init_printing()
```

```python
x, y = symbols('x, y')
sin(x).series(x,y,8)
```

```python
x = symbols('x_0:10')
```

```python
sin(x[1]).series(x[1], x[0], 8)
```

```python
f = Function("f")
```

```python
f(x[1]).series(x[1], x[0], 8)
```

```python
x_star = symbols('x_*')
```

```python
x_star
```

```python
M = zeros(2, 3)
```

```python
M[0,1]
```

```python
for i in range(3):
    M[0,i] = (x[1] - x[0])**i
```

```python
M
```

```python
m = 5
```

```python
m = symbols('m')
```

```python
x = symbols('x_0:8.')
```

```python
x
```

```python
a = 5
```

```python
x = symbols('x_0:a')
```

```python
x
```

```python
print (a)
```

    5

```python
def m():
    return m
```

```python
m
```

    <function __main__.m>

```python
def value(m):
    return m
```

```python
a
```

```python
value(a)
```

```python
x = symbols('x_0:a')
```

```python
x
```

```python
x = symbols('x_0:value(a)')
```

```python
x
```

```python
x = symbols('x_0:10')
```

```python
x
```

```python
x = symbols('x0:%d'%a)
```

```python
x
```

```python
M = zeros(5,5)
factorial(a)
```

```python
M = zeros(int('%d'%(a+1)), int('%d'%(a+1)))
M[5,5]
```

```python
x = symbols('x_0:%d'%(a+1))
x
```

```python
for i in range(int('%d'%(a+1))):
    for j in range(int('%d'%(a+1))):
        M[i,j] = Rational(1,factorial(j))*(x[i] - x[0])**j
display(M)
```

```python
b = zeros(int('%d'%(a+1)),1)
b[1] = 1
#M.inv()
```

```python
def FDA_coeff_incl(number, derivative):
    x = symbols('x_0:%d'%(number))
    M = zeros(int('%d'%(number)), int('%d'%(number)))
    b = zeros(int('%d'%(number)))
```

```python

```


## Cory Baird
# Problem set 1

# Question 1
- Two-period consumption-savings problem
- For this problem, I want you to redo the two-period consumption-savings problem under certainty presented in class assuming that the utility function is of the exponential family:
$$U(c) = \frac{-1}{a}e^{-ac}, a>0$$

## 1.A  Draw a picture of this utility function over positive c, and verify that this utility function is increasing and concave.
- Note that there is nothing wrong with having utility be negative for all positive c--remember that the purpose of a utility function is ordinal, not cardinal.

```python
# Import libraries
import os
import sys
import numpy as np
import sympy as sy
from sympy.abc import a,c,b
import matplotlib.pyplot as plt

```

### 1.A. Solution: Plot

```python
# Import user created function to plot exponential function
from ps1_functions import consum_function_plot

#Show the exponential function
u_exp = (-1/a)*sy.exp(-a*c)
u_exp

#Plot the exponential function
consum_function_plot(u_exp)
plt.title('Exponential example: CARA');
```

## 1.B  Set up the maximization problem, derive the exact form of the Euler equation with this utility function, and derive a closed-form solution for optimal consumption at time 1.
- Assume that capital markets are perfect.

### 1.B Solution: Using the intertemporal budget constraint

# Question 2

This problem asks you to solve a general T-period consumption-savings problem under certainty with perfect capital markets.

Agents preferences

$$\sum_{t=0}^T\beta^t U(c_t)\tag{2.1}$$

Where

$$U(c) = \frac{\sigma}{\sigma-1} c^{\frac{\sigma-1}{\sigma}}\tag{2.2}$$

The agent's dynamic budget constraint (DBC) each period is given by:

$$a_{t+1}=(1+r)(a_t+y_t-c_t)\tag{2.3}$$

It is not optimal for agent to die with positive wealth $A_{T+1}=0$

## 2.A. Show that DBC (eq 2.3) combined with solvency requirement implies the following IBC

$$\sum_{t=1}(\frac{1}{1+r})^t c_t = a_0+\sum_{t=0}^T(\frac{1}{1+r})^t y_t\tag{2.4}$$

### 2.A: solution

## 2.B.  Set up the agent's maximization problem, derive the Euler Equation, and find a closed form solution for c0.

### 2.B.1 Agent's maximization problem

### 2.B.2 Derive Euler equation

### 2.B.3 Closed form solution for $c_0$

## 2.C. Now suppose that there is a liquidity constraint, of the form that at must be greater than or equal to zero each period.

- Rewrite the problem adding this set of constraints, and re-derive the Euler equation.

- Prove that if β(1+r) = 1, then optimal consumption must be a non-decreasing function of time while the agent is alive.

### 2.C.1 Rewrite the problem adding this set of constraints, and re-derive the Euler equation.

#### 2.C.1.1 Lagrangian with constraints

#### 2.C.1.2

#### 2.C.2 Prove that if β(1+r) = 1, then optimal consumption must be a non-decreasing function of time while the agent is alive.

## 2.D.
- Suppose someone lives for 3 periods.
- Their incomes satisfy y(3) > y(1) > y(2).  Assume β(1+ r) is equal to 1.
- Draw pictures graphing what income and consumption will look like over time for this person under three scenarios:
    1. the life cycle model with no liquidity constraints
    - Keynesian/Myopic behavior
    - liquidity constraints.

### 2.D.1 Life cycle model with no liquitidy constraints

```python
def life_cycle():

    #Assume income is present value
    income = np.array([20,10,40])
    consumption = np.repeat(np.mean(np.array([20,10,40])),3)

    plt.plot(income)
    plt.plot(consumption)
    plt.legend(['Income', "Consumption"])
    plt.xlabel('Time period')

life_cycle()
```

### 2.D.2 Keynesian

```python
def keynesian():

    #Assume income is present value
    income = np.array([20,10,40])
    mpc=.9
    consumption = mpc*income
    plt.plot(income)
    plt.plot(consumption)
    plt.legend(['Income', "Consumption"])
    plt.xlabel('Time period')

keynesian()
```

### 2.D.3 Liquidity constraints

```python
def liquidity():

    #Assume income is present value
    income = np.array([20,10,40])
    consumption = np.array([.75, 1.5, 1])*income
    plt.plot(income)
    plt.plot(consumption)
    plt.legend(['Income', "Consumption"])
    plt.xlabel('Time period')

liquidity()
```

```python

```