# 3. Hands-on experiment:  Computing a geometric embedding for the Sparsest Cut Problem via Semidefinite Programming (20 pts)

The Uniform Sparsest Cut problem (USC) aims to find a bipartition $(S, \bar{S})$ of the nodes of a graph $G = (V, E)$, $\vert V \vert = p$, which minimizes the quantity
\begin{equation*}
\label{sceq}
       \frac{E(S, \bar{S})}{\lvert S \rvert \; \lvert \bar{S} \rvert},
\end{equation*}
where $E(S, \bar{S})$ is the number of edges connecting $S$ and $\bar{S}$, and $\lvert S \rvert $ is the number of nodes in $S$.

Computing such a bipartition is NP-hard and intense research has gone into designing efficient approximation algorithms for this problem. In the seminal work of Arora, Rao and Vazirani (2009), an $\mathcal{O}(\sqrt{\log p})$ approximation algorithm is proposed for solving USC, which relies on finding a well-spread $\ell_2^2$ geometric representation of $G$ where each node $i\in V$ is mapped to a vector $\mathbf{v}_i$ in $\mathbb{R}^p$. In this experimental section we focus on solving the SDP that computes this geometric embedding.

### SDP formulation
The canonical formulation of the SDP is
\begin{align}
\mathbf{X}^\star \in \arg \min_{ \mathbf{X} } \bigg\{ \langle \boldsymbol{C}, \mathbf{X} \rangle : &~p \; \mathrm{Tr}(\mathbf{X}) -  \mathrm{Tr}(\mathbf{1}_{p\times p}\mathbf{X}) = \frac{p^2}{2}, \\\\
& \; \mathbf{X}_{i,j} + \mathbf{X}_{j,k} - \mathbf{X}_{i, k} - \mathbf{X}_{j,j} \leq 0, \; \forall\ i \neq j \neq k \neq i\in V, \\\\
&~\underbrace{\mathrm{Tr}(\mathbf{X}) \leq p, ~\mathbf{X} \in \mathbb{R}^{p\times p} ,~\mathbf{X}\succeq 0}_{\mathcal{X}} \bigg\},
\end{align}
where $\boldsymbol{C}$ represents the Laplacian of graph $G$ and $\mathbf{X}_{i,j} = \langle \mathbf{v}_{i}, \mathbf{v}_j\rangle$ gives the geometric embedding of the nodes. We can rewrite the above SDP as

\begin{equation}
    \label{eq:reform-sdp}
    \min_{\mathbf{X} \in \mathcal{X}} f(\mathbf{X}) + g(A(\mathbf{X})) ~~~~~~~~\text{ subject to } ~~~~B_{i,j,k}(\mathbf{X}) \in \mathcal{K}, \;\forall\ i \neq j \neq k \neq i\in V,
\end{equation}
where $f(\mathbf{X}) = \langle \boldsymbol{C}, \mathbf{X} \rangle$ and  $g(\cdot) = \delta_{\left\{\frac{p^2}{2}\right\}}(\cdot)$ is the indicator function of singleton $\left\{\frac{p^2}{2}\right\}$.

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy.sparse.linalg import eigsh, svds, eigs
from math import sqrt
import scipy.io as sio
import random
import numpy.matlib
from helpers import *
import time
from scipy.sparse import isspmatrix
import itertools
from datetime import datetime

# fix the seed
random.seed( 3 )
```

```python
# Load data
dataset = 'data/102n-insecta-ant-colony4-day10.mat'
# dataset = 'data/55n-insecta-ant-colony1-day37.mat'
# dataset = 'data/25mammalia-primate-association-13.mat'

Problem = sio.loadmat(dataset)
C = np.double(Problem['LAPL']) # euclidean distance matrix
p = np.int(Problem['d']) # number of data points
opt_val = Problem['opt_val'] # optimum value
X_true = Problem['X_true']
nchoosek_inds = list(itertools.combinations(range(p), 3))
```

#### 3.(c) *(10 points)*

We will now observe the behavior of HCGM on three graphs from the Network Repository dataset \cite{nr}:
* **G1 (25 nodes)**: *mammalia-primate-association-13*,
* **G2 (55 nodes)**: *55n-insecta-ant-colony1-day37*
* **G3 (102 nodes)**: *insecta-ant-colony4-day10*

$\triangleright$ Fill in the missing code and run the algorithm.

$\triangleright$ Add resulting plots to your report.

$\triangleright$ What do you notice about the running times of the algorithm for the three problem instances?

$\triangleright$ What are the potential bottlenecks to applying this method to large graphs?

```python
def grad_F(X, beta):
    # Add the equality constraint
    [grad_val, displacement] = get_equality_constr_grad(X, beta)
    feas_eq = abs(displacement)

    # Add the triangle constraints. This needs to be scaled by d to match
    # the order of the equality constr
    n = len(nchoosek_inds)
    feas_ineq = 0
    for l in range(n):
        index_triplet = nchoosek_inds[l]
        [update_vector, row_idxs, col_idxs, part_feas_normsq] = get_triangle_constr_grad(X, beta, index_triplet)

        grad_val[row_idxs, col_idxs] = grad_val[row_idxs, col_idxs] + update_vector
        grad_val[col_idxs, row_idxs] = grad_val[col_idxs, row_idxs] + update_vector

        feas_ineq = feas_ineq + part_feas_normsq

    # Total gradient
    grad_val = grad_val + C
    feas_ineq = sqrt(feas_ineq)

    return (grad_val, feas_eq, feas_ineq)

def get_equality_constr_grad(X, beta):
    # X is a p x p matrix
    displacement = p * np.trace(X) - np.sum(X) - p**2/2
    grad_val = (p/beta) * (p * np.eye(p) - np.ones((p, p))) * displacement

    return (grad_val, displacement)

def get_triangle_constr_grad(X, beta, index_triplet):
    # X needs to be in a matrix shape, not vectorized
    #print(index_triplet)
    i = index_triplet[0]
    j = index_triplet[1]
    k = index_triplet[2]
    t1 = max(X[i, j] + X[j, k] - X[i, k] - X[j, j], 0.)
    t2 = max(X[i, k] + X[i, j] - X[j, k] - X[i, i], 0.)
    t3 = max(X[i, k] + X[j, k] - X[i, j] - X[k, k], 0.)

    grad_update = (1. / beta) * np.array([t1 + t2 - t3, # for Xij
                                -t1 + t2 + t3, # for Xik
                                t1 - t2 + t3, # for Xjk
                                -t2/2., # for Xii - ./2 because we update these twice when we symmetrize
                                -t1/2., # for Xjj - ./2 because we update these twice when we symmetrize
                                -t3/2.]) # for Xkk - ./2 because we update these twice when we symmetrize

    update_row_idxs = [i, i, j, i, j, k]
    update_col_idxs = [j, k, k, i, j, k]
    part_feas_normsq = 2. * (t1**2 + t2**2 + t3**2)

    return (grad_update, update_row_idxs, update_col_idxs, part_feas_normsq)
```

```python
# Plotting function
def plot_func(cur_iter, feasibility1,feasibility2, objective, X, X_true):
    date = datetime.now().strftime("%y%m%d_%H%M%S")

    plt.figure(figsize=(12, 8))
    plt.subplot(221)
    plt.loglog(cur_iter, feasibility1)#, 'go--', linewidth=2, markersize=12))
    plt.xlabel('iteration',fontsize=15)
    plt.ylabel('$\|A(X) - b\|$',fontsize=15)
    plt.grid(True)

    plt.subplot(222)
    plt.loglog(cur_iter, feasibility2)
    plt.xlabel('iteration',fontsize=15)
    plt.ylabel('dist$(B(X), (-\infty, 0])$',fontsize=15)
    plt.grid(True)
    plt.savefig(f'./results1_{date}.pdf', bbox_inches='tight')
    plt.show()

    #plt.subplot(223)
    obj_res = np.reshape(np.abs(objective - opt_val)/opt_val, (len(objective),))
    plt.figure(figsize=(12, 8))
    plt.loglog((cur_iter), (obj_res))
    plt.xlabel('iteration',fontsize=15)
    plt.ylabel('$(f(X) - f^*)/f^*$',fontsize=15)
    plt.title('Relative objective residual',fontsize=15)
    plt.grid(True)
    plt.savefig(f'./results2_{date}.pdf', bbox_inches='tight')
    plt.show()

    plt.figure(figsize=(12, 8))
    plt.subplot(221)
    plt.imshow(X)
    plt.title('HCGM solution',fontsize=15)
    plt.colorbar()

    plt.subplot(222)
    plt.imshow(X_true)
    plt.title('True solution',fontsize=15)
    plt.colorbar()
    plt.savefig(f'./results3_{date}.pdf', bbox_inches='tight')
    plt.show()
```

```python
def HCGM(kappa=p, maxit=np.int(1e3), beta0=1e2):
    # Initialize
    X = np.zeros((p,p))

    feasibility1 = [] # norm(A1(X)-b1)/norm(b1)
    feasibility2 = [] # dist(X, \mathcal{K})
    objective    = [] # f(x)
    cur_iter    = []
    t    = []

    #u = np.zeros((N,1))
    iter_track = np.unique(np.ceil(np.power(2, np.linspace(0,20,50))))

    start = time.time()

    for iteration in range(1, maxit+1):

        # Update Step Size
        gamma = 2 / (iteration + 1)

        # Update beta
        beta_ = beta0 / np.sqrt(iteration + 1)

        # Write down the vk to use in the lmo (eigenvalue routine)
        (vk, feas_eq, feas_ineq) = grad_F(X, beta_)
        # the above needs to be symmetric

        # Linear minimization oracle
        q, u = eigsh(vk, k=1, tol=1e-16, which='SA')
        if q >= 0:
            X_bar = 0
        else:
            u = sqrt(kappa)*u
            X_bar = np.outer(u,u)

        # Update X
        X = (1.0 - gamma) * X + gamma * (X_bar)

        if any(iteration == iter_track) or iteration==maxit:
            feasibility1.append(feas_eq)
            feasibility2.append(feas_ineq)
            objective.append(np.sum(C.flatten()*X.flatten()))
            cur_iter.append(iteration)
            t.append(time.time()-start)
            print('{:03d} | {:.4e}| {:.4e}| {:.4e}|'.format(iteration, feasibility1[-1], feasibility2[-1],objective[-1]))

    return X, feasibility1, feasibility2, objective, cur_iter, t
```

```python
print("\nDataset = {}. \n".format(dataset))
print("\n f_opt = {}. \n".format(opt_val))

X_HCGM, f1_HCGM, f2_HCGM, obj_HCGM, iter_HCGM, time_HCGM = HCGM(kappa=p, maxit=np.int(1e3), beta0=5e1)

print("\nHCGM finished. Running time: {} seconds.".format(time_HCGM[-1]))
```

    Dataset = data/102n-insecta-ant-colony4-day10.mat.

     f_opt = [[669.56459553]].

    001 | 5.2020e+03| 0.0000e+00| 1.3078e+03|
    002 | 5.2020e+03| 1.4711e+02| 4.3593e+02|
    003 | 1.7340e+03| 4.9038e+01| 1.2256e+03|
    004 | 1.7340e+03| 3.1488e+02| 7.3536e+02|
    005 | 1.0404e+03| 1.8893e+02| 1.1127e+03|
    006 | 1.0404e+03| 7.3615e+01| 7.9480e+02|
    008 | 7.4314e+02| 9.7239e+01| 8.1120e+02|
    010 | 5.7800e+02| 7.3494e+01| 9.4545e+02|
    013 | 4.0015e+02| 2.6638e+01| 1.0002e+03|
    017 | 3.0600e+02| 1.5553e+01| 8.8891e+02|
    023 | 2.2617e+02| 1.2735e+01| 8.2143e+02|
    030 | 1.7938e+02| 6.9966e+00| 7.2529e+02|
    040 | 1.3338e+02| 4.0708e+00| 7.0069e+02|
    053 | 9.8151e+01| 3.6254e+00| 7.1116e+02|
    070 | 7.5391e+01| 3.2214e+00| 6.7731e+02|
    093 | 5.5935e+01| 2.8352e+00| 6.8735e+02|
    123 | 4.2293e+01| 2.0105e+00| 6.8107e+02|
    163 | 3.1914e+01| 1.7190e+00| 6.7766e+02|
    217 | 2.3972e+01| 1.6077e+00| 6.7498e+02|
    287 | 1.8125e+01| 1.3839e+00| 6.7301e+02|
    381 | 1.3654e+01| 1.2024e+00| 6.7176e+02|
    505 | 1.0301e+01| 1.0463e+00| 6.7097e+02|
    670 | 7.7758e+00| 9.1208e-01| 6.6846e+02|
    889 | 5.8515e+00| 7.9142e-01| 6.7013e+02|
    1000 | 5.2072e+00| 7.4784e-01| 6.6869e+02|

    HCGM finished. Running time: 5032.813604831696 seconds.

```python
plot_func(iter_HCGM, f1_HCGM, f2_HCGM, obj_HCGM, X_HCGM, X_true)
```


# Lecture 6

Using partial differentiation, we showed in the lecture that for the implicit representation of a line

$$
f(x, y) = c,
$$

where $f$ is and function and $c$ is constant, the gradient $\nabla f$  on the line is perpendicular to the line. We demonstrate this here for an ellipse.

## Equation of an ellipse

The implicit equation of an ellipse is

$$
\frac{x^{2}}{a^{2}} + \frac{y^{2}}{b^{2}} = 1
$$

where $a \ne 0$ and $b \ne 0$. The parametric representation of a ellipse is

$$
\begin{align}
    x(s) &= a \cos s
    \\
    y(s) &= b \sin s
\end{align}
$$

We can plot the parametric function with SymPy:

```
from sympy import *
init_printing()
from IPython.display import display
from sympy.plotting import plot_parametric

# This command makes plots appear inside the browser window
%matplotlib inline

# Create independent variable and function for x and y
s = Symbol('s')
xs = Function('xs')
ys = Function('ys')

# Pick concrete values for a and b
a = 5
b = 3

# Parametric representation of ellipse
xs = a*cos(s)
ys = b*sin(s)

# Plot parametric line
plot_parametric(xs, ys)
```

## Tangent to the line

The tangent vector $\boldsymbol{t}$ to the surface is given by

$$
\boldsymbol{t} = \frac{d x}{d s} \boldsymbol{e}_{1}
 + \frac{d y}{d s} \boldsymbol{e}_{2}
$$

We can compute the tangent vector via differentiation, and plot the tangent at a given point.

```
t = (diff(xs, s), diff(ys, s))
```

The normal direction to the surface is given by $\nabla f$

$$
\nabla f = \frac{\partial f}{\partial x} \boldsymbol{e}_{1} + \frac{\partial f}{\partial y} \boldsymbol{e}_{2}
$$

evaluated at a point on the surface.

```
x, y = symbols('x y')
f = Eq(x**2/a**2 + y**2/b**2, 1)
display(f)
grad_f = (diff(f.args[0], x), diff(f.args[0], y))
display(grad_f)
```

We will now plot the ellipse, and the tangent vector $\boldsymbol{t}$ and the normal $\boldsymbol{n} = \nabla f$ to the surface at a point $(x(s_{0}), y(s_{0}))$. For later visualisaton, we will scale the vectors dependent on the size of the the ellipse.

```
# Pick point to evaluate tangent and normal vector at
s0 = 0.4
x0, y0 = xs.subs(s, s0), ys.subs(s, s0)

# Evaluate tangent and normal vector, and scale according to ellipse major axis length
t0 = (t[0].subs(s, s0), t[1].subs(s, s0))
l = sqrt(t0[0]**2 + t0[1]**2)/max(a, b)
t0 = (t0[0]/l, t0[1]/l)

n0 = (grad_f[0].subs(x, x0).subs(y, y0), grad_f[1].subs(x, x0).subs(y, y0))
l = sqrt(n0[0]**2 + n0[1]**2)/max(a, b)
n0 = (n0[0]/l, n0[1]/l)
```

We will now plot the ellipse, as well as the tangent and normal vectors at the point $(x(s_{0}), y(s_{0}))$.

```
from sympy.utilities.lambdify import lambdify
import numpy as np
import matplotlib.pyplot as plt

# Set plot limits (based on ellipse size)
plt.xlim(-(a + 1), a + 1)
plt.ylim(-(b + 1), b + 1)

# Make plot aspect ratio equal -> normal lines appear normal
ax = plt.axes()
ax.set_aspect('equal')

# Prepare the symbolix expression for x(s) and y(s) for plotting
xs = lambdify(s, xs, modules=['numpy'])
ys = lambdify(s, ys, modules=['numpy'])

# Plot ellipse
s = np.linspace(0, 7, 300)
plt.plot(xs(s), ys(s))

# Add tangent vector to plot
ax.arrow(x0, y0,  float(t0[0]),  float(t0[1]), label='tangent', color='g')
ax.arrow(x0, y0, -float(t0[0]), -float(t0[1]), color='g')

# Add normal vector to splot
ax.arrow(x0, y0,  float(n0[0]),  float(n0[1]), label='normal', color='r')

# Show plot
plt.show()
```

It can be seen from the plot that the gradient (red line) is normal to the ellipse.


# Cauchy Product

The Cauchy Product is the discrete equivalent to the convolution operation. It is important in combinatorics (e.g. generating functions) and will cross your path because convolution gives the probability of a sum of independent random variables.

```python
import numpy as np
from time import time

def timer(func):
    def wrapperfunc(*args):
        t0 = time()
        res = func(*args)
        print("this took %1.3fs" % (time()-t0))
        return res
    return wrapperfunc

def memoize(func):
    """
    memoizing function wrapper
    """
    S = {}
    def wrapperfunction(*args):
        if args not in S:
            S[args] = func(*args)
        return S[args]
    return wrapperfunction

def constant(const = 1, min_n = 0, max_n=np.inf):
    """
    return function that gives series coefficent for the ith term

    c = const if min_n< = n<=max_n

    c = 0 else
    """
    def coeff(i):
        if i >= min_n and i <= max_n:
            return const
        else:
            return 0

    return coeff

def cauchy_2(a,b):
    """
    returns function that gives coefficient of the cauchy product of
    two series with coefficients a(n) and b(n)

    (it's just the discrete convolution)
    """

    @memoize
    def coeff(i):
        return np.sum([a(h)*b(i-h) for h in range(i+1)])

    return coeff

@memoize
def cauchy(*args):
    """
    returns function that gives coefficient of the cauchy product of
    many series with coefficients a(n) and b(n)

    (it's just the discrete convolution)

    Maximum recursion depth will be reached for products of too many series.

    Evaluation will also become slow.
    """
    if len(args) == 1:
        return args[0]

    else:
        return cauchy_2(args[0],cauchy(*args[1:]))
```

```python
a = constant()
b = constant()

c = cauchy(a,b)

d = cauchy(*tuple(500*[a]))

for i in range(10):
    print(i,a(i),b(i),c(i),d(i))
```

    0 1 1 1 1
    1 1 1 2 500
    2 1 1 3 125250
    3 1 1 4 20958500
    4 1 1 5 2635531375
    5 1 1 6 265661562600
    6 1 1 7 22359848185500
    7 1 1 8 1616297597409000
    8 1 1 9 102432860235795375
    9 1 1 10 5781765888864894500

# Dice

Using the Cauchy Product to calculate the probability distribution of the sum of $n$ 6-sided dice.

```python
import matplotlib.pyplot as plt

d6 = constant(const = 1/6, min_n=1,max_n=6)

N = 150

t0=time()
plt.figure(figsize=(12,10))
ss = list(range(-10,int(3.5*N)))
for n in range(1,N):

    d = cauchy(*tuple(n*[d6]))

    p = [d(s) for s in ss]

    plt.plot(ss,p)
    plt.text(x=3.5*n,y=max(p),s=n)

    print(n,end='\r')

print(time()-t0)
```

# Integer Partitons

## NEEDS REVISION -- NOT CORRECT

The famed partition funcions P(k,n), which give the number of ways in which an integer k can divided across n subsets.

Previously, the coefficients of the product of the generating functions was:

\begin{equation}
c_k = \sum_{l=0}^{k} a_l b_{k-l}
\end{equation}

This counts the number of multisets with $k$ elements. I.e., it counts $[a_k,b_0]$ as well as $[a_0,b_k]$. In order to only count the partitions, the unordered pair $[a_k,b_0]$ should only show up once. This is accomplished by modifying the limits of the generating function:

\begin{equation}
\sum_{i=0}^k a_i x^i \sum_{j=0}^k b_j x^j \rightarrow \sum_{i=0}^k a_i x^i \sum_{j=0}^i b_j x^j
\end{equation}

```python
"""
The problem here is that the partition_n operation can't be built up from partition_2 operations the way I am trying here.
"""

def partition_2(a,b):
    """
    returns function that gives coefficient of the cauchy product of
    two series with coefficients a(n) and b(n) modified so that the second coefficient is always
    zero whenever its subscript is larger than the subscript of the first coefficient
    """

    f = lambda i,h: 1 if i-h <= h else 0

    @memoize
    def coeff(i):
        return np.sum([a(h)*b(i-h)*f(i,h) for h in range(i+1)])

    return coeff

@memoize
def partition_n(*args):
    """
    returns function that gives coefficient of the cauchy product of
    many series with coefficients a(n) and b(n)

    (it's just the discrete convolution)

    Maximum recursion depth will be reached for products of too many series.

    Evaluation will also become slow.
    """
    if len(args) == 1:
        return args[0]

    else:
        return partition_2(args[0],partition_n(*args[1:]))

@memoize
def partition(k,n):
    if n == 0:
        return 0
    else:
        d = partition_n(*tuple(n*[constant()]))
        return d(k)
```

```python
N = 16
print(''.join(['\t']+['k=%i\t' % n for n in range(1,N)]))
for n in range(1,N):
    print('\t'.join(['n=%i' % n]+['%i' % partition(k,n) for k in range(1,N)]))
```

    	k=1	k=2	k=3	k=4	k=5	k=6	k=7	k=8	k=9	k=10	k=11	k=12	k=13	k=14	k=15
    n=1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1
    n=2	1	2	2	3	3	4	4	5	5	6	6	7	7	8	8
    n=3	1	2	2	4	4	6	6	9	9	12	12	16	16	20	20
    n=4	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=5	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=6	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=7	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=8	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=9	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=10	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=11	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=12	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=13	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=14	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26
    n=15	1	2	2	4	4	6	6	10	10	14	14	20	20	26	26

```python
N = 50
P = np.zeros([int(N/2),N])

for i in range(int(N/2)):
    n = i
    for j in range(N):
        k = j

        P[i,j] = partition(k,n)
```

```python
plt.figure(figsize=(18,9))
plt.pcolor(P,cmap='gist_rainbow')
plt.xlabel('k')
plt.ylabel('n')

for i in range(int(N/2)):
    for j in range(N):
        plt.text(x=j+0.125,y=i+0.25,s=str(int(P[i,j])))
```

```python
#@memoize
def partitions(k,n,i_max=None):
    """
    number partitions
    """

    if not i_max:
        i_max = k

    if k == 0:
        return 1
    elif n == 1:
        return 1
    elif n == 0:
        return 0
    else:
        i_min = int(np.ceil(k/n))
        return int(np.sum([partitions(k-i,n-1,i_max=i) for i in range(i_min,i_max+1)]))
```

```python
for i in range(10):
    print(partitions(5,i))
```

    0
    1
    3
    9
    25
    69
    172
    414
    951
    2115


### Without delays

##### Description:
Trading problem:
We will trade a item on infinite time horizon. The item only has 5 possible prices: 3,4,5,6,7, the following table gives transition matrix, the element $p_{ij}$ represents the probability from current price $i$ to next price $j$. The possible inventory positions are -1, 0, 1 where -1 represents shorting one item and 1 represents longing one item. At each time step, we observe the price and change our position immediately, then we will hold the position till the next time step. Profit is given by the P&L at next time step, plus a transaction fee with a rate $\gamma$ per item.

| i\\j | 3   | 4   | 5   | 6   | 7
| ---  | --- | --- | --- |--- |--- |
| 3    | 0.0 | 1.0 | 0.0 | 0.0 | 0.0
| 4    | 0.25| 0.0 | 0.75| 0.0 | 0.0
| 5    | 0.0   | 0.5 | 0.0 | 0.5 | 0.0
| 6    | 0.0   | 0.0 | 0.75| 0.0 | 0.25
| 7    | 0.0   | 0.0 | 0.0 | 1.0 | 0.0

##### Formulation:
* Stages: $i = 1,2,\cdots$.
* State Space: $\mathcal{X} = \{-1, 0, 1\}\times\{3,4,5,6,7\}$, $x_t = (q_t, p_t)$ where $q_t$ is the number of items on hand, $p_t$ is the current price of this item.
* Control Space: $\mathcal{U} = \{-1,0,1\}$, holding position from current time step to next time step. Position will be immediately changed and be kept till the next time step.
* Feasible control mapping $\mathcal{X}\rightarrow\mathcal{U}: \mathcal{U}(x)=\{-1,0,1\}$
* Transition mapping, given $x_i, u_i$, $\mathcal{X}\rightarrow [0,1]$, for $i\in \{1,2,\cdots\}$. It is not difficult to see that $q_{t+1}=u_t$ by the definition of $u$. Price process is Markovian and it changes independently from $q_t$, and it follows above table.

For $q_t$:
$$P(q_{i+1} \mid q_i, u_i) =
\begin{cases}
1,\quad &q_{i+1}=u_i\\
0, \quad & o.w.
\end{cases}
 $$

For $p_t$:
 $$P(p_{i+1} \mid p_i, u_i) =
\begin{cases}
0.25\cdot (p_i-3),\quad &p_{i+1}=p_i-1\\
1-0.25\cdot (p_i-3), \quad &p_{i+1}=p_i+1\\
0, \quad & o.w.
\end{cases}
 $$
* Reward function $\mathcal{X}\times\mathcal{U}\rightarrow\mathbb{R}$:
$$r(x_t, u_t)=r((q_t, p_t), u_t)=\mathbb{E}[p_{t+1}-p_t\mid p_t]\cdot u_t-|u_t-q_t|\cdot \gamma$$
The first term is profit & loss and the last term is transaction cost.
* Let $v^*(x)$ be the optimal value function for state $x$, so it satisfies the following equation:

\begin{align}
    v^*(x_i) = &\max_{u\in \mathcal{U}(x)}\{r(x_i, u)+\alpha\mathbb{E}[v^*(x_{i+1})]\}\\
\end{align}

And it can be solved by value iteration:

\begin{align}
    v^{k+1}(x_i) = \mathcal{D}(v)(x_i) = &\max_{u\in \mathcal{U}(x)}\{r(x_i, u)+\alpha\mathbb{E}[v^k(x_{i+1})]\}\\
\end{align}
with arbitrary $v^0$, since $\mathcal{D}$ is a contraction mapping.

```python

```

```python
from itertools import product
import numpy as np
import pandas as pd
from copy import deepcopy
import random
```

```python
state_space = list(product([-1,0,1], [3,4,5,6,7]))
control_space = [-1,0,1]
gamma = 0.2
alpha = 0.95
def trans_prob(next_x, x, u):
    q, p = x
    next_q, next_p = next_x
    if next_q != u:
        return 0.0
    else:
        if next_p == p - 1:
            return 0.25*(p-3)
        elif next_p == p + 1:
            return 1-0.25*(p-3)
        else:
            return 0.0
def reward(x, u):
    q, p = x
    # E[next_p-p]*u = (1-0.5*(p-3))*u
    return (1-0.5*(p-3))*u-abs(u-q)*gamma
```

```python
v = pd.DataFrame({x:[0.0]*3 for x in [3,4,5,6,7]}, index = [-1,0,1])
pi = pd.DataFrame({x:[0]*3 for x in [3,4,5,6,7]}, index = [-1,0,1])
```

```python
def value_iteration(state_space, control_space, trans_prob, reward, alpha, v, pi, MAX_ITER = 1000000, threshold = 1e-6):
    for _ in range(MAX_ITER):
        next_v = deepcopy(v)
        for x in state_space:
            tmp = [reward(x, u)+alpha*
                   np.sum([v.loc[next_x[0], next_x[1]]*trans_prob(next_x, x, u) for next_x in state_space])
                   for u in control_space]
            next_v.loc[x[0], x[1]] = np.max(tmp)
            pi.loc[x[0], x[1]] = np.argmax(tmp)-1
        if abs(next_v-v).values.max()<1e-6:
            break
        else:
            v = next_v

    return v, pi
```

```python
v, pi = value_iteration(state_space, control_space, trans_prob, reward, alpha, v, pi)
```

```python
v
```

      3
      4
      5
      6
      7

      -1
      6.466842
      5.775624
      5.676842
      6.175624
      6.866842

      0
      6.666842
      5.975624
      5.676842
      5.975624
      6.666842

      1
      6.866842
      6.175624
      5.676842
      5.775624
      6.466842

```python
pi
```

      3
      4
      5
      6
      7

      -1
      1
      1
      -1
      -1
      -1

      0
      1
      1
      0
      -1
      -1

      1
      1
      1
      1
      -1
      -1

### With observation delays $o=1$

* Stages: $i = 1,2,\cdots$.
* State Space: $\mathcal{I} = \mathcal{X}\times\mathcal{U}= \{-1, 0, 1\}\times\{3,4,5,6,7\}\times\{-1, 0, 1\}$, $I_t = (x_{t-1}, u_{t-1}) = ((q_{t-1}, p_{t-1}), u_{t-1})$ where $q_{t-1}$ is the number of stocks on hand, $p_{t-1}$ is the price process at time step $t-1$.
* Control Space: $\mathcal{U} = \{-1,0,1\}$
* Feasible control mapping $\mathcal{I}\rightarrow\mathcal{U}: \mathcal{U}(x)=\{-1,0,1\}$
* Transition mapping， given $I_k, u$, $\mathcal{X}\rightarrow [0,1]$, for $i\in \{1,2,\cdots\}$

$$P'(I_{k+1} \mid I_k, u) = P'((x_{k}, u_{k}) \mid (x_{k-1}, u_{k-1}), u)=
\begin{cases}
P(x_k\mid x_{k-1}, u_{k-1}), &\quad u_k=u\\
0, &\quad o.w.\\
\end{cases}
$$

* Reward function $\mathcal{I}\times\mathcal{U}\rightarrow\mathbb{R}$:
$$r^{(1)}(I_k, u_k) = \mathbb{E}[r(x_k,u_k)\mid I_k]$$
$$r^{(2)}(I_k, u_k)=r(x_{k-1}, u_{k-1})$$

```python
state_space_d = list(product([-1,0,1], [3,4,5,6,7], [-1, 0, 1]))
control_space = [-1,0,1]
gamma = 0.2
alpha = 0.95
def trans_prob_d(next_x, x, u):
    q, p , a = x
    next_q, next_p, next_a = next_x
    if u == next_a:
        return trans_prob((next_q,next_p), (q, p), a)
    else:
        return 0.0
def reward_1(x, u):
    q, p, a = x
    # E[r(x_k, u_k)|(x_{k-1}, u_{k-1})]
    return np.sum([trans_prob(x_k, (q, p), a)*reward(x_k, u) for x_k in state_space])
    # E[next_p-p]*u = (1-0.5*(p-3))*u

def reward_2(x, u):
    q, p, a = x
    # E[next_p-p]*u = (1-0.5*(p-3))*u
    return reward((q, p), a)
```

```python
v_1 = np.array([0.0 for _ in state_space_d])
pi_1 = np.array([0 for _ in state_space_d])
```

```python
def value_iteration(state_space, control_space, trans_prob, reward, alpha, v, pi, MAX_ITER = 1000000, threshold = 1e-6):
    for _ in range(MAX_ITER):
        next_v = deepcopy(v)
        for x_idx, x in enumerate(state_space):
            tmp = [reward(x, u)+alpha*
                   np.sum([v[next_x_idx]*trans_prob(next_x, x, u)
                           for next_x_idx, next_x in enumerate(state_space)])
                   for u in control_space]
            next_v[x_idx] = np.max(tmp)
            pi[x_idx] = control_space[int(np.argmax(tmp))]
#         print(next_v)
        if abs(next_v-v).max()<threshold:
            break
        else:
            v = next_v

    return v, pi
```

```python
v_1, pi_1 = value_iteration(state_space_d, control_space, trans_prob_d, reward_1, alpha, v_1, pi_1)
```

```python
v_1
```

    array([2.37389249, 2.57389249, 2.77389249, 1.99357187, 2.19357187,
           2.39357187, 2.08389249, 2.08389249, 2.08389249, 2.39357187,
           2.19357187, 1.99357187, 2.77389249, 2.57389249, 2.37389249,
           2.37389249, 2.57389249, 2.77389249, 1.99357187, 2.19357187,
           2.39357187, 2.08389249, 2.08389249, 2.08389249, 2.39357187,
           2.19357187, 1.99357187, 2.77389249, 2.57389249, 2.37389249,
           2.37389249, 2.57389249, 2.77389249, 1.99357187, 2.19357187,
           2.39357187, 2.08389249, 2.08389249, 2.08389249, 2.39357187,
           2.19357187, 1.99357187, 2.77389249, 2.57389249, 2.37389249])

```python
pi_1
```

    array([ 1,  1,  1,  1,  1,  1, -1,  0,  1, -1, -1, -1, -1, -1, -1,  1,  1,
            1,  1,  1,  1, -1,  0,  1, -1, -1, -1, -1, -1, -1,  1,  1,  1,  1,
            1,  1, -1,  0,  1, -1, -1, -1, -1, -1, -1])

```python
v_2 = np.array([0.0 for _ in state_space_d])
pi_2 = np.array([0 for _ in state_space_d])
```

```python
v_2, pi_2 = value_iteration(state_space_d, control_space, trans_prob_d, reward_2, alpha, v_2, pi_2)
```

```python
v_2
```

    array([1.25519687, 2.24519687, 3.23519687, 1.39389249, 1.88389249,
           2.37389249, 1.97969687, 1.77969687, 1.57969687, 2.77389249,
           1.88389249, 0.99389249, 3.63519687, 2.24519687, 0.85519687,
           1.05519687, 2.44519687, 3.43519687, 1.19389249, 2.08389249,
           2.57389249, 1.77969687, 1.97969687, 1.77969687, 2.57389249,
           2.08389249, 1.19389249, 3.43519687, 2.44519687, 1.05519687,
           0.85519687, 2.24519687, 3.63519687, 0.99389249, 1.88389249,
           2.77389249, 1.57969687, 1.77969687, 1.97969687, 2.37389249,
           1.88389249, 1.39389249, 3.23519687, 2.24519687, 1.25519687])

```python
pi_2==pi_1
```

    array([ True,  True,  True,  True,  True,  True,  True,  True,  True,
            True,  True,  True,  True,  True,  True,  True,  True,  True,
            True,  True,  True,  True,  True,  True,  True,  True,  True,
            True,  True,  True,  True,  True,  True,  True,  True,  True,
            True,  True,  True,  True,  True,  True,  True,  True,  True])

### With action delays $a=1$

* Stages: $i = 1,2,\cdots$.
* State Space: $\mathcal{I} = \mathcal{X}\times\mathcal{U}= \{-1, 0, 1\}\times\{3,4,5,6,7\}\times\{-1, 0, 1\}$, $I_t = (x_{t}, u_{t-1}) = ((q_{t}, p_{t}), u_{t-1})$ where $q_t$ is the number of stocks on hand, $p_t$ is the price process at time step $t$.
* Control Space: $\mathcal{U} = \{-1,0,1\}$
* Feasible control mapping $\mathcal{I}\rightarrow\mathcal{U}: \mathcal{U}(x)=\{-1,0,1\}$
* Transition mapping， given $I_k, u$, $\mathcal{X}\rightarrow [0,1]$, for $i\in \{1,2,\cdots\}$

$$P'(I_{k+1} \mid I_k, u) = P'((x_{k+1}, u_{k}) \mid (x_{k}, u_{k-1}), u)=
\begin{cases}
P(x_{k+1}\mid x_{k}, u_{k-1}), &\quad u_k=u\\
0, &\quad o.w.\\
\end{cases}
$$

* Reward function $\mathcal{I}\times\mathcal{U}\rightarrow\mathbb{R}$:
$$r^{(1)}(I_k, u_k) = \mathbb{E}[r(x_{k+1},u_k)\mid I_k]$$
$$r^{(2)}(I_k, u_k)=r(x_{k}, u_{k-1})$$

```python

```


# An introduction to Gaussian Processes

## The Gaussian Distribution

In this notebook, we'll go over the very basics of Gaussian Processes (GPs) and how to construct and draw samples from them. But first, let's review some stuff about the Gausssian distribution $-$ the familiar bell curve $-$ itself.

We're probably all familiar with ``numpy``'s built-in ``np.random.randn()`` function, which draws a sample from the standard normal $\mathcal{N}(0, 1)$, i.e., a Gaussian with zero mean and unit variance:

```python
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1)
u = np.random.randn(100000)
plt.hist(u, bins=50, density=1);
plt.axvline(0, color="C1", ls="--")
y = np.exp(-0.5) / (np.sqrt(2 * np.pi))
plt.plot([-1, 1], [y, y], "C1-")
plt.text(0.1, 0.05, r"$\mu$", color="C1", fontsize=20);
plt.text(0.3, 0.26, r"$\sigma$", color="C1", fontsize=20);
plt.text(-0.5, 0.26, r"$\sigma$", color="C1", fontsize=20);
```

To draw from a Gaussian distribution with a different mean $\mu$, we simply *add* $\mu$ to $u$, and to draw from a distribution with different variance $\sigma^2$, we simply *multiply* $u$ by $\sigma$. We can therefore draw from a Gaussian with mean (say) $10.0$ and variance $5.0^2$ by running

```python
np.random.seed(1)
u = np.random.randn(100000)
plt.hist(10.0 + 5.0 * u, bins=50, density=1);
```

**Easy.** But what about Gaussians in multiple dimensions, where there can be correlations (covariances) between the various dimensions? Say we want to draw from the normal distribution $\mathcal{N}(0, \mathbf{\Sigma})$, where $\mathbf{\Sigma}$ is the *covariance matrix*:

$$
\mathbf{\Sigma} =
\begin{pmatrix}
 \mathrm{Var(x_0)} & \mathrm{Cov(x_0, x_1)} & \cdots & \mathrm{Cov(x_0, x_N)} \\ \\
 \mathrm{Cov(x_1, x_0)} & \mathrm{Var(x_1)} & \cdots & \mathrm{Cov(x_1, x_N)} \\ \\
 \vdots & \vdots & \ddots & \vdots \\ \\
 \mathrm{Cov(x_N, x_0)} & \mathrm{Cov(x_N, x_1)} & \cdots & \mathrm{Var(x_N)}
\end{pmatrix}
$$

Here, $\mathrm{Var(x_i)}$ is the variance, or square of the standard deviation, along the $i^\mathrm{th}$ dimension, and $\mathrm{Cov(x_i, x_j)}$ is the covariance $-$ a measure of how two variables vary jointly $-$ between the $i^\mathrm{th}$ and $j^\mathrm{th}$ dimensions.

It's hard to picture a multivariate Gaussian, but it helps to imagine what the contours of equal probability (or equipotential surfaces) look like. For a two dimensional standard normal with no covariance ($\mathbf{\Sigma} = \mathbf{I}$), these are just circles. As I add to the off-diagonal terms of the matrix, I introduce a preferred direction in the space, since now the dimensions are correlated. The circle becomes an ellipse whose elongation increases with the covariance. In the general case of a multidimensional Gaussian, the contours form a multidimensional ellipsoid, elongated by different amounts along different dimensions.

**So.** How do we draw samples from a multivariate Gaussian given $\mathbf{\Sigma}$? It turns out there's a simple trick. Take $u$ to be a variable drawn from the standard normal (as we did above with ``u = np.random.randn()``) in $N$ dimensions. If we transform $u$ by a yet-to-be-determined operator (matrix) $\mathbf{L}$,

$$
\begin{align}
x = \mathbf{L}u
\end{align}
$$

we now have a random variable $x$ whose variance is

$$
\begin{align}
\mathrm{Var}(x) &\equiv \left< x x^\top \right> \\
                &= \left< \left(\mathbf{L}u\right) \left(\mathbf{L}u \right)^\top \right> \\
                &= \left< \mathbf{L} uu^\top \mathbf{L}^\top \right> \\
                &= \mathbf{L} \left<uu^\top\right> \mathbf{L}^\top \\
                &= \mathbf{L} \mathrm{Var}(u) \mathbf{L}^\top \\
                &= \mathbf{L}\mathbf{L}^\top.
\end{align}
$$

In the expressions above, I used the fact that the variance of a random variable is defined as the expectation value (denoted by $\left<\,\right>$) of the squared deviation from the mean (zero, in this case). The expectation value of $\mathbf{L}$ is just $\mathbf{L}$ (since it is constant), and the variance of $u$ is unity (by construction).

Now, all we have to do is find a matrix $\mathbf{L}$ such that

$$
\begin{align}
\mathbf{\Sigma} &= \mathbf{L} \mathbf{L}^\top
\end{align}
$$

and we're done!

**But how do we do that?** As it happens, what we're trying to do has a name: it's the **Cholesky decomposition** of $\mathbf{\Sigma}$. Formally, all the Cholesky decomposition does is to find a factorization of a matrix $\mathbf{\Sigma}$ in the form given by the expression above, where $\mathbf{L}$ is a lower triangular matrix. You can (informally) think of this decomposition as finding the square root of $\mathbf{\Sigma}$, and it is *enormously* useful for GPs (and linear algebra in general). Note that the Cholesky decomposition only works for Hermitian (symmetric), positive-definite matrices, but those are the only kinds of matrices we'll be dealing with here anyways.

To perform the Cholesky decomposition in Python, use ``scipy``:

```python
from scipy.linalg import cho_factor
L, _ = cho_factor(S, lower=True)  # Compute the lower triangular matrix L
L = np.tril(L)                    # Ensure the upper triangle of L is zeroed out
```

Exercise 1a

Define a function ``draw_from_gaussian(mu, S, ndraws=1)`` that returns ``ndraws`` samples from a multivariate Gaussian with mean ``mu`` and covariance matrix ``S``. The shape of the output should be ``(ndraws, ndim)`` where ``ndim`` is the dimension of the Gaussian. Note that as ``S`` gets large, roundoff error can often accumulate in the Cholesky decomposition, potentially causing weird ``LinearAlgebraError``s. For this reason, it's good practice to always add a tiny number to the diagonal of a matrix (something like ``1e-12``) whenever decomposing it, factorizing it, inverting it, etc. You probably want to bake that into your code to compute ``L``.

```python
from scipy.linalg import cho_factor
L, _ = cho_factor(S, lower=True)  # Compute the lower triangular matrix L
L = np.tril(L)

def draw_from_gaussian(mu,S,ndraws=1):
    return np.random.multivariate_normal(mu,S,(nseaw))
```

Exercise 1b

Draw ``10,000`` samples from a zero-mean Gaussian with covariance

$$
\mathbf{\Sigma} =
\begin{pmatrix}
 1 & 0.5 \\
 0.5 & 1
\end{pmatrix}
$$

Plot the "corner" plot for your ``samples`` using the ``corner`` package (``!pip install corner``):

```python
from corner import corner
fig = corner(samples);
```

Vary the terms in the covariance matrix (recalling that it must be symmetric!) to get a sense of how they affect the joint distribution of samples.

## Parametrizing the covariance

As the number of dimensions increases, and in particular as it becomes *infinite* (which it will, when we get to actual Gaussian Processes), it's no longer convenient to describe the covariance in terms of every single entry in the covariance matrix. Instead, it's useful to introduce the notion of a *kernel*, a function of just a few parameters that describes the overall structure of the covariance matrix.

In general, the covariance matrix can have *any* structure, but quite often in timeseries analysis *data points close to each other in time are strongly correlated*. This is true whether your timeseries contains photometric variability from a rotating star, the rise and fall of a supernova light curve, a gravitational wave signal, or PSF shape changes due to temperature fluctuations on the detector. This is because each of these processes have characteristic timescales over which they operate, and on timescales shorter than that, all measurements you make are likely to be close to relatively each other. But if you wait long enough, what your system will do down the line will be pretty decoupled from what it is doing right now.

So we can imagine defining a kernel that looks something like this:

```python
%show_kernel
```

where the covariance peaks at zero time lag and drops smoothly to zero as the time lag increases. This particular kernel is extremely useful and has a name: the *Squared Exponential Kernel*, defined as

$$
\begin{align}
    k(t_i, t_j) = A^2 \exp\left(-\frac{\left(t_i - t_j\right)^2}{2l^2}\right)
\end{align}
$$

Exercise 2a

Define a function ``ExpSquaredKernel(t, A=1.0, l=1.0)`` that returns the covariance matrix described by the squared exponential kernel for a timeseries ``t``.

Exercise 2c

For ``t = np.linspace(0, 2, 11)``, draw ``10,000`` samples from the guassian described by the covariance matrix and plot the corresponding corner plot, labeling each subplot with the time point it represents. Comment on the structure of the plot and the correlations among the different dimensions.

## From Gaussians to Gaussian Processes

So far we've been taking about Gaussians *distributions* and how they can jointly model a few random variables at a time. But what happens as the dimensionality of the Gaussian increases and eventually becomes infinite? The Gaussian no longer represents a collection of random variables, but instead the behaviour of a continuous function. And that's exactly what a Gaussian Process is: a distribution over functions with infinitely many points. We can't technically model a function with infinitely many points on a computer, but we can investigate what happens as the number of points becomes very large.

Exercise 3a

Construct a ``1000 x 1000`` covariance matrix using the squared exponential kernel for a timeseries spanning 10 time units, and plot an image of it using ``plt.imshow``. Comment on the structure of the matrix.

Exercise 3b

Draw ``30`` samples from the zero-mean Gaussian described by the covariance matrix above. Each sample should be a vector of length ``1,000``. Plot each sample against time on the same plot. Also plot the mean function and use ``plt.fill_between`` to shade the 1- and 2-$\sigma$ levels of the Gaussian Process. Recall that the variance $\sigma^2$ at each point in time is given by the corresponding entry on the diagonal of $\mathbf{\Sigma}$.

Next, vary the hyperparameters of the GP, $A$ and $l$, and comment on how they change the behavior of the GP.

## Conditioning on data

If you completed **Exercise 3b**, you have actually implemented your own Gaussian Process! But it's still pretty simple, because so far we are drawing only from our *prior*: we know the Gaussian has a certain covariance and a certain mean (zero, in the case above), but nothing else about how it's supposed to behave. To apply this to a real dataset, we need to *condition* the GP on observations.

Say I make two perfectly noise-free observations:

$$
\begin{align}
    y(t = 2.5) &= 1.0 \\
    y(t = 7.5) &= -1.0
\end{align}
$$

Again, $y$ could be anything: the flux received from a star, the strain due to a gravitational wave, etc. I want to draw new samples from my GP, but not just *any* samples: I explicitly want to draw functions that go through those two data points. One (very inefficient) way to do this is to draw a *ton* of samples and discard any samples that don't agree with the data. (In this particular case, that would be literally impossible, since I know $y(t)$ at those two points *exactly*: I will never draw at random a function whose value is exactly what I want it to be. But note that *rejection sampling* can actually be useful in other cases. Anyways.)

A *better* way to do this is to consider the *joint distribution* of the observed data $y(t)$ (usually called the "training set") and the points where I'm trying to predict the value of the function $y_\star(t_\star)$ (the "test set"):

$$
\begin{align}
\begin{pmatrix}
 y \\
 y_\star
\end{pmatrix}
&=
\mathcal{N}
\left[
\mathbf{0},
\begin{pmatrix}
 \mathbf{\Sigma}(t, t) & \mathbf{\Sigma}(t, t_\star)\\
 \mathbf{\Sigma}(t_\star, t) & \mathbf{\Sigma}(t_\star, t_\star)
\end{pmatrix}
\right]
\end{align}
$$

where $\mathbf{\Sigma}(t, t)$ is the $N_\mathrm{train} \times N_\mathrm{train}$ covariance matrix evaluated at all pairs of training points, $\mathbf{\Sigma}(t_\star, t_\star)$ is the $N_\mathrm{test} \times N_\mathrm{test}$ covariance matrix evaluated at all pairs of test points, and the remaining two entries are the (rectangular) covariance matrices evaluated at all pairs of (test, training) points.

Given this joint distribution, we can compute the *conditional* distribution of $y_\star$ given $y$ with some linear algebra (for a derivation, see Appendix A.2 of [Rasmussen & Williams (2006)](http://www.gaussianprocess.org/gpml/chapters/RW.pdf) and references therein):

$$
\begin{align}
y_\star | y \sim \mathcal{N}\left(
    \mathbf{\Sigma}(t_\star, t) \mathbf{\Sigma}(t, t)^{-1}y,
    \mathbf{\Sigma}(t_\star, t_\star) -
        \mathbf{\Sigma}(t_\star, t) \mathbf{\Sigma}(t, t)^{-1} \mathbf{\Sigma}(t, t_\star)
\right)
\end{align}
$$

In other words, given $y$, the distribution for $y_\star$ is *still Gaussian*, but this time with mean equal to
$$
\begin{align}
    \mathbf{\Sigma}(t_\star, t) \mathbf{\Sigma}(t, t)^{-1}y
\end{align}
$$
and covariance
$$
\begin{align}
    \mathbf{\Sigma}(t_\star, t_\star) - \mathbf{\Sigma}(t_\star, t) \mathbf{\Sigma}(t, t)^{-1} \mathbf{\Sigma}(t, t_\star)
\end{align}
$$

Exercise 4

Show that when $t_\star = t$, the mean and covariance of the GP for $y_\star$ give you what you'd expect. When $t_\star \neq t$, what can you say about the variance of the GP after versus before data was collected?

Exercise 5a

Define a function ``compute_gp(t_train, y_train, t_test, **kwargs)`` that returns the mean and covariance of the Gaussian process for ``y_test(t_test)`` conditioned on ``y_train(t_train)``. The ``**kwargs`` are for additional optional keyword arguments passed directly to the kernel (in this simple case, ``A`` and ``l``). You will need to re-define the ``ExpSquaredKernel`` to take *two* timeseries as inputs so you can compute the covariance between $t$ and $t_\star$.

Exercise 5b

Given

```python
t_train = np.array([2.5, 7.5])
y_train = np.array([1.0, -1.0])
t_test = np.linspace(0, 10, 1000)
```

compute the mean and covariance of the GP. Draw ``30`` samples from it as before. Plot the mean function and shade the 1- and 2-$\sigma$ levels. Comment on the behavior of the GP. Does the GP correctly predict the training points? How does the variance behave close to and far from the training points? How does the GP behave very far from the training points?

Exercise 6

The last thing we'll do regarding drawing from a GP is to account for noise in the observations. If the measurement error on the data points is Gaussian and uncorrelated (white), as we usually assume it to be, we simply add a term to the joint distribution of the training and test data:

$$
\begin{align}
\begin{pmatrix}
 y \\
 y_\star
\end{pmatrix}
&=
\mathcal{N}
\left[
\mathbf{0},
\begin{pmatrix}
 \mathbf{\Sigma}(t, t) + \sigma_n^2 \mathbf{I} & \mathbf{\Sigma}(t, t_\star)\\
 \mathbf{\Sigma}(t_\star, t) & \mathbf{\Sigma}(t_\star, t_\star)
\end{pmatrix}
\right]
\end{align}
$$

where $\sigma_n$ is the standard deviation for the $n^\mathrm{th}$ data point in the training set and $\mathbf{I}$ is the identiy matrix. Figure out how this changes the expression for the mean and variance of the GP and re-plot the figure from **Exercise 5b**, this time given an observational uncertainty $\sigma = 0.25$ at both training points. Comment on the new behavior of the GP.


[Climbing Stairs](https://leetcode.com/problems/climbing-stairs/)。有$n$个台阶，每次只能往上走一阶或两阶，问爬上顶层有多少种方法。[矩形填充](https://www.nowcoder.com/questionTerminal/72a5a919508a4251859fb2cfb987a0e6)。用$n$个$2\times{1}$的矩形去填充$2\times{n}$的区域，共有几种方法。

思路：第一题，每级阶梯只能由前一级或者前两级走过来，所以$dp[i]=dp[i-1]+dp[i-2]$。第二题，对于一个$2\times{i}$的区域，可以竖直填充一个$2\times{1}$变成一个$2\times{i-1}$的区域，或者横着填充两个$2\times{1}$的矩形变成$2\times{i-2}$的区域，所以$dp[i]=dp[i-1]+dp[i-2]$。

```python
def climbStairs(n: int) -> int:
    if n < 3:
        return n

    dp = [0]*n
    dp[0], dp[1] = 1, 2

    for i in range(2, n):
        dp[i] = dp[i-1]+dp[i-2]

    return dp[-1]
```

[Min Cost Climbing Stairs](https://leetcode.com/problems/min-cost-climbing-stairs/)。一个数组表示$n$个台阶，每个数字表示跨越当前台阶的花费，每次跨越台阶可以选择跨一阶或跨两阶，求到达最高层的最小化费。起点可以在第一阶与第二阶中任选。

思路：易得每一层的状态只能有两种状态转换而来，前一层跨一层，前两层跨两层。所以设dp数组，其中$dp[i]$表示到达$i$层的最小花费，状态转移方程为：
$$
dp[i]=min(dp[i-1]+cost[i-1],dp[i-2]+cost[i-2])
$$

```python
def minCostClimbingStairs(cost) -> int:
    n = len(cost)
    dp = [0]*(n+1)

    for i in range(2, n+1):
        dp[i] = min(dp[i-1]+cost[i-1], dp[i-2]+cost[i-2])

    return dp[-1]
```

    6

[Word Break](https://leetcode.com/problems/word-break/)，**2019阿里、流利说手撕代码题**。给一个非空字符串$s$与一个字串集合$wordDict$，判断$s$能否拆分成若干个字典中的字串。

思路：设立dp数组，其中$dp[i]$表示以$i$位置结尾的字串是否可拆分。增加一个字符后，需要考察所有可能的切分点$j$，只有当切分点左边的字串可拆分并且切分点右边的字串在字典中时，切分点$j$才是合法的。

```python
def wordBreak(s: str, wordDict) -> bool:
    n = len(s)
    dp = [False]*(n+1)
    dp[0] = True

    for i in range(1, n+1):    # i比字串索引要大1
        for j in range(i):    # 切分点
            if dp[j] and s[j:i] in wordDict:
                dp[i] = True
                break

    return dp[-1]
```

[Coin Change](https://leetcode.com/problems/coin-change/)。给定若干面额的硬币，再给一个数值$num$，求组成$num$所需最少的硬币数。

思路：动态规划。设立dp数组，其中$dp[i]$表示对于数值$i$最少需要的硬币数，初始状态$dp[0]=0$，其他位置初始化为最大值。对于数值$i$与面值$denomination$，状态转移方程为：
$$
dp[i]=min(dp[i],dp[i-denomination]+1)
$$

```python
def coinChange(coins, amount: int) -> int:
    dp = [0x7FFFFFFF]*(amount+1)
    dp[0] = 0

    for i in range(1, amount+1):
        for val in coins:
            if val <= i:
                dp[i] = min(dp[i], dp[i-val]+1)

    return dp[-1] if dp[-1] != 0x7FFFFFFF else -1
```

    3

[Coin Change 2](https://leetcode.com/problems/coin-change-2/)。给定若干面额的硬币，再给一个数值，求组成$num$有几种方式。

思路：设立dp数组，其中$dp[i]$表示组成$i$有多少种方法。该题难点就在于怎么避免计算重复的答案，如$1+2=3$和$2+1=3$是重复答案。假设初始状态只有一种面额的硬币，那么状态转移方程就为：
$$
dp[val]=dp[val-denomination]+dp[val-2*denomination]+...
$$
即只有$val$能被$denomination$整除时才有解。然后再加入一种面额，状态转移方程不变。

```python
def change(amount: int, coins) -> int:
    n = len(coins)
    dp = [0]*(amount+1)
    dp[0] = 1

    for denomination in coins:    # 以增加面额的方式来考虑问题
        for val in range(denomination, amount+1):
            dp[val] += dp[val-denomination]

    return dp[-1]
```

    4

[Unique Paths](https://leetcode.com/problems/unique-paths/)。从一个矩形的左上方走到右下方有多少种不同的走法？只能往右或往下走。

思路：设立dp矩阵，其中$dp[row][col]$表示从左上方走到$(row,col)$的可能数。初始状态，首行跟首列的位置只能是$1$，状态转移方程为：
$$
dp[row][col]=dp[row-1][col]+dp[row][col-1]
$$

```python
def uniquePaths(m: int, n: int) -> int:
    dp = [[1]*n for _ in range(m)]
    for row in range(1, m):
        for col in range(1, n):
            dp[row][col] = dp[row-1][col]+dp[row][col-1]

    return dp[-1][-1]
```

[Longest Increasing Subsequence](https://leetcode.com/problems/longest-increasing-subsequence/)。给一数组，求出其最长递增子序列的长度。

思路：设立一个dp数组，其中$dp[i]$表示以索引$i$结尾的字串最长子序列的长度，初始状态为$dp[i]=1$，因为任一单字符都可构成一个子序列。那么状态转移方程为：
$$
dp[i]=max(dp[i],dp[j_{1}]+1,dp[j_{2}]+1,...,dp[j_{k}]+1)
$$
其中$j$特指$nums[j]<nums[i]$的索引。注意该题的状态转移并不由之前的单个状态决定，而是需要遍历。

```python
def lengthOfLIS(self, nums: List[int]) -> int:
    if not nums:
        return 0
    n = len(nums)
    dp = [1]*n

    for i in range(1, n):
        for pre in range(i):
            if nums[pre] < nums[i]:
                dp[i] = max(dp[i], dp[pre]+1)

    return max(dp)
```

    4

[LCS](https://www.nowcoder.com/practice/c996bbb77dd447d681ec6907ccfb488a?tpId=49&&tqId=29348&rp=2&ru=/activity/oj&qru=/ta/2016test/question-ranking)。求最长公共子序列的长度。

思路：设立二维$dp$数组，其中$dp[i][j]$表示$s1$的前$i$位与$s2$的前$j$位LCS的长度。易得$dp$的初始状态为首行首列都为$0$，状态转移方程为：
$$
\begin{align}
dp[i][j]=
\begin{cases}
dp[i-1][j-1]+1 & ,s1[i]=s2[j] \\
max(dp[i-1][j],dp[i][j-1]) & ,s1[i]\ne{s2[j]} \\
\end{cases}
\end{align}
$$

```python
def LCS(a, b):
    len_a, len_b = len(a), len(b)
    dp = [[0]*(len_b+1) for _ in range(len_a+1)]

    for i in range(1, len_a+1):
        for j in range(1, len_b+1):
            if a[i-1] == b[j-1]:
                dp[i][j] = dp[i-1][j-1]+1
            else:
                dp[i][j] = max(dp[i-1][j], dp[i][j-1])

    return dp[-1][-1]
```

    4

[Jump Game II](https://leetcode.com/problems/jump-game-ii/)。给一数组，每个数字表示在当前位置可以前进的最大距离。现在保证从起点出发一定可以到达终点，求最小的步数。

思路：设立dp数组，其中$dp[i]$表示到达$i$位置的最小步数。那么可以根据$dp[i]$与$nums[i]$的值去更新后面dp的值。思路应该没错，但是超时了，时间复杂度为$O(n^{2})$。

```python
def jump(nums) -> int:
    n = len(nums)
    dp = [0x7FFFFFFF]*n
    dp[0] = 0

    for idx, num in enumerate(nums):
        for offset in range(1, num+1):
            pos = idx+offset if idx+offset < n else n-1
            dp[pos] = min(dp[pos], dp[idx]+1)

    return dp[-1]
```

[Best Time to Buy and Sell Stock III](https://leetcode.com/problems/best-time-to-buy-and-sell-stock-iii/)。给一数组表示一只股票的历史价格，允许买卖两次，求最大利润。

思路：设立dp数组，其中$dp[i][j]$表示在允许交易$i$次的条件下第$j$天的最大利润。同时对每一个不同$i$的配置，维护一个余额变量```balance```，表示持股状态下的最大余额。易得初始状态为$dp[0][j]=0$，$balance=-prices[0]$，两者的状态转移方程为：
$$
\left\{
\begin{align}
dp[i][j]&=max(dp[i][j-1],balance+prices[j]) \\
balance&=max(balance,dp[i-1][j-1]-prices[j]) \\
\end{align}
\right.
$$

```python
def maxProfit(prices) -> int:
    if len(prices) < 2:
        return 0

    n = len(prices)
    dp = [[0]*n for _ in range(2+1)]

    for i in range(1, 3):
        balance = -prices[0]
        for j in range(1, n):
            dp[i][j] = max(dp[i][j-1], balance+prices[j])    # 不操作或者卖掉
            balance = max(balance, dp[i-1][j-1]-prices[j])    # 不操作或者买进

    return dp[-1][-1]
```

[Best Time to Buy and Sell Stock IV](https://leetcode.com/problems/best-time-to-buy-and-sell-stock-iv/)。给一数组表示一只股票的历史价格，允许至多买卖$k$次，求最大利润。

思路：同上题。但是测试用例中有$k$特别大的情况。当$k$大于天数的一半时，相当于不限买卖次数，等同于[Best Time to Buy and Sell Stock II](https://leetcode.com/problems/best-time-to-buy-and-sell-stock-ii/)。

```python
def maxProfit(k: int, prices) -> int:
    n = len(prices)
    if n < 2:
        return 0

    if k > (n//2):    # 同等于不限买卖次数的条件
        res = 0
        for idx in range(n-1):
            if prices[idx+1] > prices[idx]:
                res += prices[idx+1]-prices[idx]
        return res

    n = len(prices)
    dp = [[0]*n for _ in range(k+1)]

    for i in range(1, k+1):
        balance = -prices[0]
        for j in range(1, n):
            dp[i][j] = max(dp[i][j-1], balance+prices[j])
            balance = max(balance, dp[i-1][j-1]-prices[j])

    return dp[-1][-1]
```

    7

[Best Time to Buy and Sell Stock with Cooldown](https://leetcode.com/problems/best-time-to-buy-and-sell-stock-with-cooldown/)。给一数组表示一只股票的历史价格，不限定买卖次数，但是抛售之后需要隔一天才能再买，求最大利润。

思路：设两个dp数组，```hold```与```unhold```分别表示持股与不持股的最大利润，需要返回的是```unhold[-1]```。首先考虑初始状态，对于```hold```数组而言，```hold[0]=-price[0]```，即买入亏钱；```hold[1]=max(-prices[0],-prices[1])```，前两天选择价格最低的买入以获得最大利润或最小亏损；```unhold[0]=0```，因为第一天无法卖出；```unhold[1]=max(unhold[0],hold[0]+prices[1])```，不操作或者卖出第一天买入的股票。

状态转移方程为：
$$
\begin{align}
hold[i]&=max(hold[i-1],unhold[i-2]-prices[i]) \\
unhold[i]&=max(unhold[i-1],hold[i-1]+prices[i]) \\
\end{align}
$$
状态方程简而言之就是两种可能：无操作或者买入(卖出)，注意卖出之后买入需要隔天。

```python
def maxProfit(prices) -> int:
    n = len(prices)
    if n < 2:
        return 0

    hold, unhold = [0]*n, [0]*n

    hold[0] = -prices[0]
    hold[1] = max(-prices[0], -prices[1])
    unhold[1] = max(unhold[0], hold[0]+prices[1])

    for i in range(2, n):
        hold[i] = max(hold[i-1], unhold[i-2]-prices[i])
        unhold[i] = max(unhold[i-1], hold[i-1]+prices[i])

    return unhold[-1]
```

[Best Time to Buy and Sell Stock with Transaction Fee](https://leetcode.com/problems/best-time-to-buy-and-sell-stock-with-transaction-fee/)。

思路：同样设立两个dp数组，```hold```与```unhold```分别表示持股与不持股下的最大利润，最后返回```unhold[-1]```。初始状态，```hold[0]=-prices[0]```，```unhold[0]=0```，状态转移方程：
$$
\begin{align}
hold[i]&=max(hold[i-1],unhold[i-1]-prices[i]) \\
unhold[i]&=max(unhold[i-1],hold[i-1]+prices[i]-fee) \\
\end{align}
$$

```python
def maxProfit(prices, fee: int) -> int:
    n = len(prices)
    if n < 2:
        return 0

    hold, unhold = [0]*n, [0]*n
    hold[0] = -prices[0]

    for i in range(1, n):
        hold[i] = max(hold[i-1], unhold[i-1]-prices[i])
        unhold[i] = max(unhold[i-1], hold[i-1]+prices[i]-fee)

    return unhold[-1]
```

    8

[Maximum Product Subarray](https://leetcode.com/problems/maximum-product-subarray/)。给一整形数组，找一连续子区间，返回该子区间的最大乘积。

思路：因为数组包含负数，所以要考虑负负得正的情况。维护两个$dp$数组```max_dp```和```min_dp```，其中$dp[i]$表示以$i$结尾且包含$i$的最大(小)乘积。状态转移方程为：
$$
\begin{align}
max\_dp[i]&=max(nums[i],min\_dp[i-1]*nums[i],max\_dp[i-1]*nums[i]) \\
min\_dp[i]&=min(nums[i],min\_dp[i-1]*nums[i],max\_dp[i-1]*nums[i]) \\
\end{align}
$$
因为最大乘积不一定包含最后一个数，所以返回的不是```dp[-1]```，而是需要随时记录最大乘积。注意到状态转移只取决于上一个位置，所以不需要设置数组，只需要设置变量即可。

```python
def maxProduct(nums) -> int:
    if not nums:
        return 0
    n = len(nums)
    if n == 1:
        return nums[0]

    res = min_pre = max_pre = nums[0]

    for i in range(1, n):
        min_cur = min(nums[i], min_pre*nums[i], max_pre*nums[i])
        max_cur = max(nums[i], min_pre*nums[i], max_pre*nums[i])
        res = max(res, max_cur)
        min_pre, max_pre = min_cur, max_cur

    return res
```

    6

[01背包问题](https://www.acwing.com/problem/content/2/)。给出若干件物品的价值$v_{i}$跟重量$w_{i}$，现有一个负重量为$W$的背包，求能得到的最大价值。

思路：经典DP问题。设立一个二维$dp$矩阵，其中$dp[i][j]$表示在有$j$负重量的条件下考虑前$i$件物品能得到的最大利润。易得初始状态为$dp[0][0]=0$，然后推状态转移方程。新增一个物品时，有两种决定：拿或不拿。不拿的情况下，dp值不变，$dp[i][j]=dp[i-1][j]$；拿的话，势必要给物品先腾地方，$dp[i][j]=dp[i-1][j-w[i]]+v[i]$。那么状态转移方程就为：
$$
dp[i][j]=max(dp[i-1][j],dp[i-1][j-w[i]]+v[i])
$$

```python
def Knapsack(v, w, W):
    v.insert(0, 0)    # 保持idx一致，为第0件物品添加体积跟价值
    w.insert(0, 0)
    dp = [[0]*(W+1) for _ in range(N+1)]

    for i in range(1, N+1):    # 第i件物品
        for j in range(W+1):
            dp[i][j] = dp[i-1][j]    # 不拿

            if j >= w[i]:    # 负重量允许的情况下才能拿
                dp[i][j] = max(dp[i][j],
                               dp[i-1][j-w[i]]+v[i])

    return dp[-1][-1]
```

[完全背包问题](https://www.acwing.com/problem/content/3/)。现有不限量的$N$种物品，给出每种物品的价值$v$与重量$w$，还有一个负重量为$W$的背包，求能带走的最大价值。

思路：虽然题意指出每种物品不限量，但因为背包负重量是有限的，所以实际上所能拿的每种物品也是有限的。设$j$为递推时的背包负重量，那第$i$件物品的最大数量上限为$j//w[i]$。所以该题只需要在01背包问题上面加一重循环即可。

```python
def Knapsack(v, w, W):
    v.insert(0, 0)
    w.insert(0, 0)
    dp = [[0]*(W+1) for _ in range(N+1)]

    for i in range(1, N+1):
        for j in range(W+1):
            dp[i][j] = dp[i-1][j]

            max_cnt = j//w[i]
            for k in range(max_cnt+1):
                dp[i][j] = max(dp[i][j],
                               dp[i-1][j-k*w[i]]+k*v[i])

    return dp[-1][-1]
```

[Minimum Path Sum](https://leetcode.com/problems/minimum-path-sum/)。给一个$m\times{n}$的矩阵，以左上角为起点，每次只能向下或向右移动，求最小路径和。

思路：典型的动态规划。设立dp数组，其中$dp[i][j]$表示走到$(i,j)$位置时的最小路径和。每一个位置只能由两个状态变换而来，上边和左边，所以状态转移方程为：

$$
dp[i][j]=min(dp[i-1][j],dp[i][j-1])+nums[i][j]
$$

```python
def minPathSum(grid) -> int:
    rows, cols = len(grid), len(grid[0])
    dp = [[0]*cols for _ in range(rows)]

    # 首行首列需特殊处理
    dp[0][0] = grid[0][0]
    for col in range(1, cols):
        dp[0][col] = dp[0][col-1]+grid[0][col]
    for row in range(1, rows):
        dp[row][0] = dp[row-1][0]+grid[row][0]

    for row in range(1, rows):
        for col in range(1, cols):
            dp[row][col] = min(dp[row-1][col],
                               dp[row][col-1])+grid[row][col]

    return dp[-1][-1]
```

    7

[Combination Sum IV](https://leetcode.com/problems/combination-sum-iv/)。给一无重复元素的正整数数组$nums$与一个目标值$target$，数组中的数字可用无限次，求有多少种组合方式使得取出的数字求和等于$target$。

思路：设dp数组，其中$dp[i]$表示求和等于$i$的方式有多少种，那么状态转移方程为：
$$
\begin{align}
dp[i]&=dp[i-nums[0]]+dp[i-nums[1]]+\cdots+dp[i-nums[n]] \\
&=\sum_{j=1}^{n}{dp[i-nums[j]]} \\
\end{align}
$$
当然，$i-nums[j]$必须大于$0$才合法。

```python
def combinationSum4(nums, target: int) -> int:
    dp = [0]*(target+1)
    dp[0] = 1

    for sub_target in range(1, target+1):
        for num in nums:
            if sub_target >= num:
                dp[sub_target] += dp[sub_target-num]

    return dp[-1]
```

    7

[House Robber](https://leetcode.com/problems/house-robber/)。一数组表示各房屋内的金钱，不能同时搜刮相邻的两个房子，问最多能偷到多少钱。

思路：设```dp[i]```为有$i$个房子时最多能偷到的钱。增加一个房子时，因为不能偷相邻的房子，所以如果要偷最后的房子，则倒数第二个房子就不能偷；如果投了倒数第二个房子，则最后一个房子就不能偷。状态转移方程为：
$$
dp[i]=max(dp[i-2]+house[i],dp[i-1])
$$

```python
def rob(nums) -> int:
    n = len(nums)
    if n == 0:
        return 0
    if n < 3:
        return max(nums)

    dp = [0]*n
    dp[0] = nums[0]
    dp[1] = max(nums[:2])

    for i in range(2, n):
        dp[i] = max(dp[i-2]+nums[i], dp[i-1])

    return dp[-1]
```

[Ugly Number II](https://leetcode.com/problems/ugly-number-ii/)。丑数的定义如下，因子只有$2$、$3$和$5$的正数，第一个丑数定义为$1$。求第$n$个丑数。

思路：丑数的因子只含$2$、$3$和$5$且第一个丑数为$1$，那么从$1$开始乘上这些因子即可得到丑数序列。关键在于避免得到重复丑数，设置三个指针$p_{2}$、$p_{3}$和$p_{5}$，分别代表$\times{2}$、$\times{3}$和$\times{5}$的位置，计算丑数的递推式为：
$$
dp[i]=\min(dp[p2]*2,dp[p3]*3,dp[p5]*5)
$$
$dp[i]$肯定是由三种运算中的一种得来的，要避免重复计算，还需要判断另外两种运算是否会得到一样的结果，如果是的话需要做处理。

```python
def nthUglyNumber(n: int) -> int:
    dp = [1]*n
    m2 = m3 = m5 = 0

    for i in range(1, n):
        cur_num = min(dp[m2]*2, dp[m3]*3, dp[m5]*5)
        dp[i] = cur_num

        if cur_num == dp[m2]*2:
            m2 += 1
        if cur_num == dp[m3]*3:
            m3 += 1
        if cur_num == dp[m5]*5:
            m5 += 1

    return dp[-1]
```

[Maximum Length of Repeated Subarray](https://leetcode.com/problems/maximum-length-of-repeated-subarray/)。给两个整数数组，求公共连续子数组的最大长度。

思路：设置dp矩阵，其中$dp[i][j]$表示以$i$结尾且包含$A[i]$的子数组与以$j$结尾且包含$B[j]$的子数组的公共子数组长度。易得状态转移方程为：
$$
\begin{align}
dp[i][j]=
\begin{cases}
0, & A[i]\ne{B[j]}\\
dp[i-1][j-1]+1, & A[i]=B[j]\\
\end{cases}
\end{align}
$$

```python
def findLength(A, B) -> int:
    len_a, len_b = len(A), len(B)
    dp = [[0]*len_b for _ in range(len_a)]
    res = 0

    for j in range(len_b):
        dp[0][j] = 1 if A[0] == B[j] else 0
        res = max(res, dp[0][j])
    for i in range(len_a):
        dp[i][0] = 1 if B[0] == A[i] else 0
        res = max(res, dp[i][0])

    for i in range(1, len_a):
        for j in range(1, len_b):
            if A[i] != B[j]:
                dp[i][j] == 0
            else:
                dp[i][j] = dp[i-1][j-1]+1
                res = max(res, dp[i][j])

    return res
```

    3

[Pascal's Triangle](https://leetcode.com/problems/pascals-triangle/)。构建Pascal(杨辉)三角。

思路：每行的元素数等于行标号，除了边界元素为$1$，其他元素全由上行相加得出。

```python
def generate(numRows: int):
    if numRows==0:
        return list()

    res = [[1]]
    for row in range(2, numRows+1):
        cur_row = [1]*row
        for col in range(1, row-1):
            cur_row[col] = res[-1][col-1]+res[-1][col]

        res.append(cur_row)

    return res
```

[Pascal's Triangle II](https://leetcode.com/problems/pascals-triangle-ii/)。给一行索引，返回杨辉三角的特定一行，行索引从$0$开始。

思路：类似动态规划，只保存上一行即可。

```python
def getRow(rowIndex: int):
    if rowIndex==0:
        return [1]

    pre_row=[1]
    for row in range(2,rowIndex+2):
        cur_row=[1]*row
        for col in range(1,row-1):
            cur_row[col]=pre_row[col-1]+pre_row[col]

        pre_row=cur_row

    return pre_row
```

    [1, 2, 1]

```python

```


#Classiﬁcation and logistic regression
The linear regression can be modified to classification problem, by reduce y to discrete values. For now, we focus on **binary classification**, and it it y is either $0$ nor $1$.

## Logistic regression
Simply use linear regression to predit binary classification ($y \in \{0,1\}$) performs poorly. To fix this, we change our $h_\theta(x)$:

\begin{gather}
h_\theta(x) = g(\theta^Tx) = \frac{1}{1 + e^{-\theta^Tx}}
\end{gather}

where $g(z) = \frac{1}{1+e^{-z}}$ , which is called **logistic function** or **sigmoid function**. $g(z)$ tends toward to 1 as $z \rightarrow \infty$, and tends toward to 0 as $z \rightarrow -\infty$.

Keeping $x_0 = 1$ as in linear regression, $\theta^Tx = \theta_0 + \sum_{j=1}^n \theta_jx_j$.

One useful property of sigmoid function is:
\begin{align}
g'(z) &= \frac{d}{dz}\frac{1}{1 + e^{-z}}\\
&= \frac{1}{(1+e^{-z})^2}(e^{-z})\\
&= \frac{1}{(1+e^{-z})}(1- \frac{1}{(1+e^{-z})})\\
&= g(z)(1-g(z))
\end{align}

To estimate $\theta$, we do maximum likelihood estimator under a set of asumptions (similar to least squares regression). Assume that
\begin{gather}
P(y=1|x;\theta) = h_\theta(x)
\end{gather}

\begin{gather}
P(y=0|x;\theta) = 1-h_\theta(x)
\end{gather}

This can be written as

\begin{gather}
p(y|x;\theta) = (h_\theta(x))^y(1-h_\theta(x))^{1-y}
\end{gather}

Assuming we have $m$ training examples which were generated independently, we write down the likelihood of the parameters as
\begin{align}
L(\theta) &= p(\vec y| X;\theta)\\
&= \prod_{i=1}^m p(y^{(i)}|x^{(i)};\theta)\\
&= \prod_{i=1}^m (h_\theta(x^{(i)}))^{y^{(i)}} (1-h_\theta(x^{(i)}))^{1-y^{(i)}}
\end{align}

\begin{align}
l(\theta) &= \log L(\theta)\\
&= \sum_{i=1}^m y^{(i)}\log h(x^{(i)}) + (1- y^{(i)})\log(1-h(x^{(i)}))
\end{align}

Again, similar to the derivation in the case of linear regression, we can use gradient ascent to maximize the likelihood. $\theta := \theta + \alpha \nabla_\theta l(\theta)$ (plus is because we are maximizing the function).
With one training example and the usful property, we have:

\begin{align}
\frac{\partial}{\partial \theta_j}l(\theta) &= (y \frac{1}{g(\theta^T x) - (1-y)\frac{1}{1-g(\theta^Tx)}})\frac{\partial}{\partial\theta_j}g(\theta_j x)\\
&= (y \frac{1}{g(\theta^T x) - (1-y)\frac{1}{1-g(\theta^Tx)}})g(\theta^Tx)(1-g(\theta^Tx)\frac{\partial}{\partial\theta_j}\theta^Tx\\
&= (y(1-g(\theta^Tx))-(1-y)g(\theta^Tx))x_j\\
&= (y-h(\theta^Tx))x_j
\end{align}

Therefore, it give us the stochastic gradient ascent rule:
\begin{gather}
\theta_j := \theta_j + \alpha(y^{(i)}-h_\theta(x^{(i)}))x_j^{(i)}
\end{gather}

Of course, we can also change it to a batch gradient ascent:
\begin{gather}
\theta_j := \theta_j + \alpha\sum_{i=1}^m(y^{(i)}-h_\theta(x^{(i)}))x_j^{(i)}
\end{gather}

Here, the cost function $J(\theta)$ becomes $l(\theta)$ written above.

```
import pandas as pd
import seaborn as sns
import numpy as np
```

```
df = pd.read_csv("challenger.csv")
df.info()
df.head()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 23 entries, 0 to 22
    Data columns (total 2 columns):
     #   Column       Non-Null Count  Dtype
    ---  ------       --------------  -----
     0   failure      23 non-null     bool
     1   temperature  23 non-null     int64
    dtypes: bool(1), int64(1)
    memory usage: 335.0 bytes

      failure
      temperature

      0
      False
      66

      1
      True
      70

      2
      False
      69

      3
      False
      68

      4
      False
      67

```
df.loc[df['failure'] == False, ["failure"]] = 0
df.loc[df['failure'] == True, ["failure"]] = 1
df.info()
df.head()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 23 entries, 0 to 22
    Data columns (total 2 columns):
     #   Column       Non-Null Count  Dtype
    ---  ------       --------------  -----
     0   failure      23 non-null     object
     1   temperature  23 non-null     int64
    dtypes: int64(1), object(1)
    memory usage: 496.0+ bytes

      failure
      temperature

      0
      0
      66

      1
      1
      70

      2
      0
      69

      3
      0
      68

      4
      0
      67

```
sns.scatterplot(data=df, x="temperature", y="failure")
```

```
def logistic_regression(x, y, *args, **kwargs):
  """
  Implementation for logistic regression. It is a batch logistic regression.

  Input: x training data, column vector or matrix, each column for one feature
         y training data, ground truth, column vector
         alpha learning rate optional
         tol tolorance optional

  Output: theta estimated parameters
          h estimations
  """

  alpha = 10**-3
  tol = 10**-5

  if len(kwargs) > 2:
    raise Exception("Too many inputs.")
  elif len(kwargs) > 0:
    if "alpha" in kwargs.keys():
      alpha = kwargs["alpha"]
    if "tol" in kwargs.keys():
      tol = kwargs["tol"]

  num_feature = np.shape(x)[1]
  num_example = np.shape(x)[0]
  theta = np.zeros((num_feature + 1, 1))
  new_col = np.ones(num_example)
  x = np.insert(x, 0, new_col, axis=1)
  pj = 0
  num_loop = 0

  # start gradient ascent
  while num_loop < 10000:
    for i in range(num_example):
      h = 1 / (1 + np.exp(-np.dot(x, theta).astype(float)))
      tmp = sum((y - h) * x)
      theta = theta + alpha * tmp.reshape((num_feature + 1, 1)) # theta update
      j = sum(y*np.log(h) + (1-y)*np.log(1-h)) # cost function
      if abs(j - pj) <= tol:
        return theta, h
      else:
        pj = j
    num_loop = num_loop + 1

  print("gradient descent does not convergence after maximum interation.")
  return theta, h
```

```
x = df["temperature"].values
y = df['failure'].values
n = np.size(y)
x = x.reshape((n, 1))
y = y.reshape((n, 1))
theta1, h1 = logistic_regression(x, y)
```

```
transparent = h1.reshape(n)

for idx, val in enumerate(df['failure']):
  if val == 1:
    sns.scatterplot(x=[df["temperature"][idx]], y=val, alpha=transparent[idx], color="green")
  else:
    sns.scatterplot(x=[df["temperature"][idx]], y=val, alpha=1-transparent[idx], color="red")
sns.lineplot(data=df, x="temperature", y=h1.reshape(np.size(h1)), color='blue')
```

One advantage of logistic regression is that it gives the probability of each instance belongs to a class. It makes a big difference knowing a instance has a 99% probability for a class compared to 51%. The transparency in the plot above inicates the probability of the instance for the class. Deeper color stands for higher probability(some points have deeper color than it should have, it because multiple points overlap to each other, e.g. temperature=67).

If we set any point with probebiliy larger than 0.5 as green, and others as red, we get the following plot. The transparency is again the the probability of the instance for the class.

```
for idx, val in enumerate(df['failure']):
  if h1[idx] > 0.5:
    sns.scatterplot(x=[df["temperature"][idx]], y=val, alpha=transparent[idx], color="green")
  else:
    sns.scatterplot(x=[df["temperature"][idx]], y=val, alpha=1-transparent[idx], color="red")
sns.lineplot(data=df, x="temperature", y=h1.reshape(np.size(h1)), color='blue')
```


# Symbolic Manipulation

Most computer mathematical work directly manipulates numbers.  Symbolic manipulation or computer algebra is using a computer to manipulate algebraic equations.

You can do many algebraic and calculus operations.

You can learn more [here](http://docs.sympy.org/dev/tutorial/).

```python
import sympy

# this makes for mathematical typesetting
sympy.init_printing()

# defines symbols
x = sympy.symbols('x')
y = sympy.symbols('y')
```

We can define a polynomial.

```python
expr = x**2 + 2*x + 1
expr
```

We can do some algebraic manipulations.

```python
expr.factor()
```

Here is an example of integration

```python
expr.integrate(x)
```

Here is a substitution

```python
expr.subs(x, y**2)
```

You can also make numerical substitutions for a variable.

```python
expr.subs(x, 1)
```

```python

```


# Animating a simple wave

We'll plot at various times a wave $u(x,t)$ that starts as a triangular shape as in Taylor Example 16.1, and then animate it.  We can imagine this as simulating a wave on a taut string.  Here $u$ is the transverse displacement (i.e., $y$ in our two-dimensional plots).  We are not solving the wave equation as a differential equation here, but starting with $u(x,0) \equiv u_0(x)$ and plotting the solution at time $t$:

$\begin{align}
  u(x,t) = \frac12 u_0(x - ct) + \frac12 u_0(x + ct)
  \;,
\end{align}$

which *is* the solution to the wave equation starting with $u_0(x)$ at time $t=0$.

We have various choices for animation in a Jupyter notebook.  We will consider two possibilities, which both use `FuncAnimation` from `matplotlib.animation`.
1. Make a javascript movie that is then displayed inline with a movie-playing widget using `HTML`.  We use
`%matplotlib inline` for this option and use `%%capture` to prevent the figure from displaying prematurely.
2. Update the figure in real time (so to speak) by using `%matplotlib notebook`, which creates active figures that we can modify after they are displayed.

We'll do the first option here. We should define at least one class for the animation.

v1: Created 25-Mar-2019. Last revised 27-Mar-2019 by Dick Furnstahl (furnstahl.1@osu.edu).

```python
%matplotlib inline
```

To use option 2: uncomment `%matplotlib notebook` here and `fig.show()` just after we define `anim`.  Comment  out `%%capture` and `HTML(anim.to_jshtml())` below.

```python
#%matplotlib notebook
```

```python
import numpy as np
import matplotlib.pyplot as plt

from matplotlib import animation, rc
from IPython.display import HTML
```

First define functions for the $t=0$ wave function form (here a triangle) and for the subsequent shape at any time $t$ based on the wave speed `c_wave`.

```python
def u_0_triangle(x_pts, height=1., width=1.):
    """Returns a triangular wave of amplitude height and width 2*width.
    """
    y_pts = np.zeros(len(x_pts))  # set the y array to all zeros
    for i, x in enumerate(x_pts):
        if x < width and x >= 0.:
            y_pts[i] = -(height/width) * x + height
        elif x < 0 and x >= -width:
            y_pts[i] = (height/width) * x + height
        else:
            pass   # do nothing (everything else is zero already)
    return y_pts

```

```python
def u_triangle(x_pts, t, c_wave = 1., height=1., width=1.):
    """Returns the wave at time t resulting from a triangular wave of
       amplitude height and width 2*width at time t=0.  It is the
       superposition of two traveling waves moving to the left and the right.
    """
    y_pts = u_0_triangle(x_pts - c_wave * t) / 2. + \
            u_0_triangle(x_pts + c_wave * t) / 2.
    return y_pts
```

```python
# Set up the array of x points (whatever looks good)
x_min = -5.
x_max = +5.
delta_x = 0.01
x_pts = np.arange(x_min, x_max, delta_x)

```

First look at the initial ($t=0$) wave form.

```python
# Define the initial (t=0) wave form and the wave speed.
height = 1.
width = 1.
c_wave = 1.

# Make a figure showing the initial wave.
t_now = 0.

fig = plt.figure(figsize=(6,2), num='Triangular wave')
ax = fig.add_subplot(1,1,1)
ax.set_xlim(x_min, x_max)
gap = 0.1
ax.set_ylim(-gap, height + gap)
ax.set_xlabel(r'$x$')
ax.set_ylabel(r'$u_0(x)$')
ax.set_title(rf'$t = {t_now:.1f}$')

line, = ax.plot(x_pts,
                u_triangle(x_pts, t_now, c_wave, height, width),
                color='blue', lw=2)

fig.tight_layout()

```

Next make some plots at an array of time points.

```python
t_array = np.array([0., 1,  2.,  3,  4.,  -2.])

fig_array = plt.figure(figsize=(12,6), num='Triangular wave')

for i, t_now in enumerate(t_array):
    ax_array = fig_array.add_subplot(3, 2, i+1)
    ax_array.set_xlim(x_min, x_max)
    gap = 0.1
    ax_array.set_ylim(-gap, height + gap)
    ax_array.set_xlabel(r'$x$')
    ax_array.set_ylabel(r'$u_0(x)$')
    ax_array.set_title(rf'$t = {t_now:.1f}$')

    ax_array.plot(x_pts,
                  u_triangle(x_pts, t_now, c_wave, height, width),
                  color='blue', lw=2)

fig_array.tight_layout()
fig_array.savefig('Taylor_Example_16p1_triangle_waves.png',
                   bbox_inches='tight')
```

Now it is time to animate!

```python
# Set up the t mesh for the animation.  The maximum value of t shown in
#  the movie will be t_min + delta_t * frame_number
t_min = 0.   # You can make this negative to see what happens before t=0!
t_max = 15.
delta_t = 0.1
t_pts = np.arange(t_min, t_max, delta_t)
```

We use the cell "magic" `%%capture` to keep the figure from being shown here.  If we didn't the animated version below would be blank.

```python
%%capture

fig_anim = plt.figure(figsize=(6,2), num='Triangular wave')
ax_anim = fig_anim.add_subplot(1,1,1)
ax_anim.set_xlim(x_min, x_max)
gap = 0.1
ax_anim.set_ylim(-gap, height + gap)

# By assigning the first return from plot to line_anim, we can later change
#  the values in the line.
line_anim, = ax_anim.plot(x_pts,
                          u_triangle(x_pts, t_min, c_wave, height, width),
                          color='blue', lw=2)

fig_anim.tight_layout()
```

```python
def animate_wave(i):
    """This is the function called by FuncAnimation to create each frame,
        numbered by i.  So each i corresponds to a point in the t_pts
        array, with index i.
    """
    t = t_pts[i]
    y_pts = u_triangle(x_pts, t, c_wave, height, width)

    line_anim.set_data(x_pts, y_pts)  # overwrite line_anim with new points
    return (line_anim,)   # this is needed for blit=True to work
```

```python
frame_interval = 40.  # time between frames
frame_number = 100    # number of frames to include (index of t_pts)
anim = animation.FuncAnimation(fig_anim,
                               animate_wave,
                               init_func=None,
                               frames=frame_number,
                               interval=frame_interval,
                               blit=True,
                               repeat=False)

#fig.show()
```

```python
HTML(anim.to_jshtml())

```

```python

```


# Direct Inversion via LU Decomposition

- toc: true
- hide: false
- badges: true
- comments: true
- categories: [machine learning]

# Motivation

- this tutorial is also available as a [colab notebook](https://colab.research.google.com/drive/1SJtGSP_1XcjLh0pyF_RwmakUMdDuNcxt?usp=sharing)

Continuing with the Vandermonde interpolation example this tutorial goes step by step on how computers typically implement matrix inversions for the purpose of solving least squares problems.

This typically takes the form of a matrix factorization of the matrix to be inverted followed by forward and back substitution. The matrix factorization demoed here is LU decomposition. We compare our hand-coded solvers with Numpy library solvers to compare results.

```
cd 'drive/My Drive/Colab Notebooks/Solving Least Squares Problems'
```

    [Errno 2] No such file or directory: 'drive/My Drive/Colab Notebooks/Solving Least Squares Problems'
    /content

# Mechanics of solving a least squares problem
## Introduction to LU decomposition

Our goal is to explore how to solve systems of linear equations following Trefethan and fastai's numerical linear algebrea course: https://www.youtube.com/watch?v=O2x5KPJr5ag&list=PLtmWHNX-gukIc92m1K0P6bIOnZb-mg0hY&index=5.

# First perform LU decomposition
https://en.wikipedia.org/wiki/LU_decomposition

It is also based on Trefethen ch. 20 on LU decompositions and ch. 17 on Backsubstitution.
https://www.amazon.com/Numerical-Linear-Algebra-Lloyd-Trefethen/dp/0898713617

LU decomposition is essentially Gaussian elimination to find the upper matrix while recording your operations in the lower matrix

```
import numpy as np
def LU(A):
    U = np.copy(A)
    m, n = A.shape
    L = np.eye(n)
    for k in range(n-1):
        for j in range(k+1,n):
            L[j,k] = U[j,k]/U[k,k]
            U[j,k:n] -= L[j,k] * U[k,k:n]
    return L, U
```

```
A = np.array([[1,2,-3],[2,-1,1],[1,4,-2]]).astype(np.float)
print(A)
```

    [[ 1.  2. -3.]
     [ 2. -1.  1.]
     [ 1.  4. -2.]]
    [[ 1.  2. -3.]
     [ 2. -1.  1.]
     [ 1.  4. -2.]]

```
L, U = LU(A)
print("L = ")
print(L)
print("U = ")
print(U)
```

    L =
    [[ 1.   0.   0. ]
     [ 2.   1.   0. ]
     [ 1.  -0.4  1. ]]
    U =
    [[ 1.   2.  -3. ]
     [ 0.  -5.   7. ]
     [ 0.   0.   3.8]]
    L =
    [[ 1.   0.   0. ]
     [ 2.   1.   0. ]
     [ 1.  -0.4  1. ]]
    U =
    [[ 1.   2.  -3. ]
     [ 0.  -5.   7. ]
     [ 0.   0.   3.8]]

```
print("LU = ")
print(L@U)
```

    LU =
    [[ 1.  2. -3.]
     [ 2. -1.  1.]
     [ 1.  4. -2.]]
    LU =
    [[ 1.  2. -3.]
     [ 2. -1.  1.]
     [ 1.  4. -2.]]

```
b = np.array([[1],[1],[9]])
print(b)
```

    [[1]
     [1]
     [9]]
    [[1]
     [1]
     [9]]

Here L and U must be square. Once we have our matrix factorizations of $A$, then we can solve for $x$ in a 2 step process:

1. solve for $y$ in $Ly = b$
2. then solve for $x$ after solving for $y$ in $Ux = y$

```
def forwardsub(L, b):
    y = np.zeros(len(L))
    y[0] = b[0]
    for i in range(1,len(y)):
        y[i] = b[i] - np.dot(L[i,:],y)
    return y
```

```
def backsub(U, y):
    N = len(U)
    x = np.zeros([N,1])
    x[-1] = y[-1]/U[-1,-1]
    for i in range(N-2, -1, -1):
        x[i] = (y[i] - np.dot(U[i,:],x))/float(U[i,i])
    return x
```

```
y = forwardsub(L, b)
x = backsub(U, y)
print(x)
```

    [[1.]
     [3.]
     [2.]]
    [[1.]
     [3.]
     [2.]]

```
bhat = (A@x)
```

```
np.allclose(b,bhat)
```

    True

    True

Yay we got back the same solution!
 ## Put into function and compare against Numpy
We can combine all 3 of these functions in a least squares solver..

```
def LU_solve(A, b):
    L, U = LU(A)
    y = forwardsub(L, b)
    x = backsub(U, y)
    return x
```

```
x = LU_solve(A, b)
print(x)
```

    [[1.]
     [3.]
     [2.]]
    [[1.]
     [3.]
     [2.]]

```
A@x
```

    array([[1.],
           [1.],
           [9.]])

    array([[1.],
           [1.],
           [9.]])

Finally let's check against numpy's built in least squares solver...

```
print(np.linalg.lstsq(A, b, rcond = None)[0])
```

    [[1.]
     [3.]
     [2.]]
    [[1.]
     [3.]
     [2.]]

Yay we get the same answer as numpy's least squares solver!

# Now let's applied what we've learned to an interpolation example

Our goal is to interpolate the function below

```
import matplotlib.pyplot as plt
nSamples = 10
t = np.linspace(0, 10, nSamples)
y = np.exp(-5*(t-3)**2)+0.1*np.random.randn(nSamples)

plt.plot(t,y,'*')
plt.show()
```

```
deg=6
A = np.vander(y, N=deg, increasing=True)
```

Here we solve the interpolation problem by solving for $x$ in the given equation $y = Ax$, found by taking the pseudo inverse:

\begin{equation}
(A^*A)^{-1}A^*y = x
\end{equation}

This works by left multiply $y=Ax$ by $A^*$ to get $A^*y = A^*Ax$ and then noticing that $A^*A$ is a square matrix, which thus can be inverted using the LU decomposition solve we introduced above

(If $[A] = [m \times n]$ then $[A*A] = [n \times m] \times [m \times n] = [n \times n]$)

Above is how things work mathematically, to actually do this in code we need to look at the problem like this...

Our goal is to estimate $y$ by using the Vandermonde matrix and an appropriate $x$ vector such that

\begin{equation}
y \approx \hat{y} = Ax
\end{equation}

however to find the appropriate $x$ we must perform least squares fitting....

\begin{equation}
x = \min ||y-\hat{y}||_2^2 = \min || y - Ax ||_2^2
\end{equation}

Anyway to solve equation 1 we actually back up a step before finding the inverse

\begin{equation}
A^*y = A^*A x
\end{equation}

which we can rewrite as $Y = A^*y$ and $A^*A = M$ to get

\begin{equation}
Mx = Y
\end{equation}

and invert to find x exactly like we did above

```
Astar = np.conjugate(A.transpose())
print(np.shape(Astar))
```

    (6, 10)
    (6, 10)

```
print(np.shape(Astar@A))
```

    (6, 6)
    (6, 6)

```
x = LU_solve(Astar@A, Astar@y)
plt.plot(t,y,'*',t,A@x)
plt.show()
```

# Compare results
Finally we put it all into a function a show that this is a solution for Least squares of *rectangular* matrices

```
def LUsolveRect(A, y):
    Astar = np.conjugate(A.transpose())
    x = LU_solve(Astar@A, Astar@y)
    return x
```

```
x = LUsolveRect(A, y)
print(x)
```

    [[ 6.49036380e-14]
     [ 1.00000000e+00]
     [-2.24586093e-11]
     [ 1.26177226e-10]
     [ 1.10151372e-09]
     [-2.25442647e-09]]
    [[ 4.70179451e-15]
     [ 1.00000000e+00]
     [-3.83457512e-12]
     [ 1.17952022e-11]
     [ 2.43384392e-10]
     [-7.20062159e-10]]

```
plt.plot(t,y,'*',t,A@x)
plt.show()
```

```
x = np.linalg.lstsq(A, y, rcond=None)[0]
print(x)
```

    [ 4.30450636e-17  1.00000000e+00 -6.05071548e-15  3.39173134e-14
      2.12982409e-13 -4.50577076e-13]
    [-2.38197120e-17  1.00000000e+00 -1.18793864e-14  5.71764858e-15
      8.34748937e-13 -2.22377498e-12]

```
plt.plot(t,y,'*',t,A@x)
plt.show()
```

Move on to the next notebook on applying Vandermonde interpolation to a beam hardening problem in CT imaging: https://colab.research.google.com/drive/1AMKoOjs7y6l18d70RcbHMLU9ZFxkzRpO

```

```


### Example 2: Nonlinear convection in 2D

Following the initial convection tutorial with a single state variable $u$, we will now look at non-linear convection (step 6 in the original). This brings one new crucial challenge: computing a pair of coupled equations and thus updating two time-dependent variables $u$ and $v$.

The full set of coupled equations is now

\begin{aligned}
\frac{\partial u}{\partial t} + u \frac{\partial u}{\partial x} + v \frac{\partial u}{\partial y} = 0 \\
\\
\frac{\partial v}{\partial t} + u \frac{\partial v}{\partial x} + v \frac{\partial v}{\partial y} = 0\\
\end{aligned}

and rearranging the discretised version gives us an expression for the update of both variables

\begin{aligned}
u_{i,j}^{n+1} &= u_{i,j}^n - u_{i,j} \frac{\Delta t}{\Delta x} (u_{i,j}^n-u_{i-1,j}^n) - v_{i,j}^n \frac{\Delta t}{\Delta y} (u_{i,j}^n-u_{i,j-1}^n) \\
\\
v_{i,j}^{n+1} &= v_{i,j}^n - u_{i,j} \frac{\Delta t}{\Delta x} (v_{i,j}^n-v_{i-1,j}^n) - v_{i,j}^n \frac{\Delta t}{\Delta y} (v_{i,j}^n-v_{i,j-1}^n)
\end{aligned}

So, for starters we will re-create the original example run in pure NumPy array notation, before demonstrating
the Devito version. Let's start again with some utilities and parameters:

```python
from examples.cfd import plot_field, init_hat
import numpy as np
%matplotlib inline

# Some variable declarations
nx = 101
ny = 101
nt = 80
c = 1.
dx = 2. / (nx - 1)
dy = 2. / (ny - 1)
sigma = .2
dt = sigma * dx
```

Let's re-create the initial setup with a 2D "hat function", but this time for two state variables.

```python
#NBVAL_IGNORE_OUTPUT

# Allocate fields and assign initial conditions
u = np.empty((nx, ny))
v = np.empty((nx, ny))

init_hat(field=u, dx=dx, dy=dy, value=2.)
init_hat(field=v, dx=dx, dy=dy, value=2.)

plot_field(u)
```

Now we can create the two stencil expression for our two coupled equations according to the discretised equation above. We again use some simple Dirichlet boundary conditions to keep the values on all sides constant.

```python
#NBVAL_IGNORE_OUTPUT
for n in range(nt + 1): ##loop across number of time steps
    un = u.copy()
    vn = v.copy()
    u[1:, 1:] = (un[1:, 1:] -
                 (un[1:, 1:] * c * dt / dx * (un[1:, 1:] - un[1:, :-1])) -
                  vn[1:, 1:] * c * dt / dy * (un[1:, 1:] - un[:-1, 1:]))
    v[1:, 1:] = (vn[1:, 1:] -
                 (un[1:, 1:] * c * dt / dx * (vn[1:, 1:] - vn[1:, :-1])) -
                 vn[1:, 1:] * c * dt / dy * (vn[1:, 1:] - vn[:-1, 1:]))

    u[0, :] = 1
    u[-1, :] = 1
    u[:, 0] = 1
    u[:, -1] = 1

    v[0, :] = 1
    v[-1, :] = 1
    v[:, 0] = 1
    v[:, -1] = 1

plot_field(u)
```

Excellent, we gain get a wave that resembles the one from the oiginal examples.

Now we can set up our coupled problem in Devito. Let's start by creating two initial state variables $u$ and $v$, as before, and intialising them with our "hat function.

```python
#NBVAL_IGNORE_OUTPUT
from devito import Grid, TimeFunction

# First we need two time-dependent data fields, both initialised with the hat function
grid = Grid(shape=(nx, ny), extent=(2., 2.))
u = TimeFunction(name='u', grid=grid)
init_hat(field=u.data[0], dx=dx, dy=dy, value=2.)

v = TimeFunction(name='v', grid=grid)
init_hat(field=v.data[0], dx=dx, dy=dy, value=2.)

plot_field(u.data[0])
```

Using the two `TimeFunction` objects we can again derive our dicretized equation, rearrange for the forward stencil point in time and define our variable update expression - only we have to do everything twice now! We again use foward differences for time via `u.dt` and backward differnces in space via `u.dxl` and `u.dyl` to match the original tutorial.

```python
from sympy import Eq, solve
from devito import Operator

eq_u = Eq(u.dt + u*u.dxl + v*u.dyl)
eq_v = Eq(v.dt + u*v.dxl + v*v.dyl)

# We can use the same SymPy trick to generate two
# stencil expressions, one for each field update.
stencil_u = solve(eq_u, u.forward)[0]
stencil_v = solve(eq_v, v.forward)[0]
update_u = Eq(u.forward, stencil_u)
update_v = Eq(v.forward, stencil_v)

print("U update:\n%s\n" % update_u)
print("V update:\n%s\n" % update_v)
```

    U update:
    Eq(u(t + dt, x, y), (dt*h_x*(-u(t, x, y) + u(t, x, y - h_y))*v(t, x, y) + dt*h_y*(-u(t, x, y) + u(t, x - h_x, y))*u(t, x, y) + h_x*h_y*u(t, x, y))/(h_x*h_y))

    V update:
    Eq(v(t + dt, x, y), (dt*h_x*(-v(t, x, y) + v(t, x, y - h_y))*v(t, x, y) + dt*h_y*(-v(t, x, y) + v(t, x - h_x, y))*u(t, x, y) + h_x*h_y*v(t, x, y))/(h_x*h_y))

We then set Dirichlet boundary conditions at all sides of the domain to $1$.

```python
x, y = grid.dimensions
t = grid.stepping_dim
bc_u = [Eq(u.indexed[t+1, 0, y], 1.)]  # left
bc_u += [Eq(u.indexed[t+1, nx-1, y], 1.)]  # right
bc_u += [Eq(u.indexed[t+1, x, ny-1], 1.)]  # top
bc_u += [Eq(u.indexed[t+1, x, 0], 1.)]  # bottom
bc_v = [Eq(v.indexed[t+1, 0, y], 1.)]  # left
bc_v += [Eq(v.indexed[t+1, nx-1, y], 1.)]  # right
bc_v += [Eq(v.indexed[t+1, x, ny-1], 1.)]  # top
bc_v += [Eq(v.indexed[t+1, x, 0], 1.)]  # bottom
```

And finally we can put it all together to build an operator and solve our coupled problem.

```python
#NBVAL_IGNORE_OUTPUT
from devito import Operator

# Reset our data field and ICs
init_hat(field=u.data[0], dx=dx, dy=dy, value=2.)
init_hat(field=v.data[0], dx=dx, dy=dy, value=2.)

op = Operator([update_u, update_v] + bc_u + bc_v)
op(time=nt+1, dt=dt)

plot_field(u.data[0])
```