# Convergence & Stability, Discretized Pressure Field, Nonlinear BVP
### Nov. 2019

# Problem 1
## 1a)

To show this numerical scheme is second-order accurate, we Taylor expand to rewrite the scheme. We can see there are 5 terms to Taylor expand: $U_j^n, U_j^{n+1}, U_j^{n-1}, U_{j+1}^n, U_{j-1}^n$

Given that $U_j^n = u(j\Delta x, n \Delta t)$, we Taylor expand around $x=j$ or $t=n$:

First, because it represents the point at the current time and location (so $\Delta t = \Delta x = 0$), we have:

\begin{equation}
U_j^n = u(j,n) = u
\end{equation}

Around $t=n$ we have:

\begin{equation}
U_j^{n+1} = u(j, n+1) = u + \Delta t u_t + \frac{\Delta t^2}{2}u_{tt} + \frac{\Delta t^3}{6}u_{ttt} + \frac{\Delta t^4}{24}u_{tttt} + h.o.t.
\end{equation}

\begin{equation}
U_j^{n-1} = u(j, n-1) = u - \Delta t u_t + \frac{\Delta t^2}{2}u_{tt} - \frac{\Delta t^3}{6}u_{ttt} + \frac{\Delta t^4}{24}u_{tttt} + h.o.t.
\end{equation}

Around $x=j$ we have:

\begin{equation}
U_{j+1}^n = u(j+1, n) = u + \Delta x u_x + \frac{\Delta x^2}{2}u_{xx} + \frac{\Delta x^3}{6}u_{xxx} + \frac{\Delta x^4}{24}u_{xxxx} + h.o.t.
\end{equation}

\begin{equation}
U_{j-1}^n = u(j-1, n) = u - \Delta x u_x + \frac{\Delta x^2}{2}u_{xx} - \frac{\Delta x^3}{6}u_{xxx} + \frac{\Delta x^4}{24}u_{xxxx} + h.o.t.
\end{equation}

Plugging in these approximations in the term to the left produces:

\begin{equation}
\frac{U_j^{n+1} - 2U_j^n + U_j^{n-1}}{\Delta t^2} = \frac{u + \Delta t u_t + \frac{\Delta t^2}{2}u_{tt} + \frac{\Delta t^3}{6}u_{ttt} + \frac{\Delta t^4}{24}u_{tttt} + h.o.t.}{\Delta t^2} - \frac{2u}{\Delta t^2} + \frac{u - \Delta t u_t + \frac{\Delta t^2}{2}u_{tt} - \frac{\Delta t^3}{6}u_{ttt} + \frac{\Delta t^4}{24}u_{tttt} + h.o.t.}{\Delta t^2}
\end{equation}

which simplifies to:

\begin{equation}
\frac{U_j^{n+1} - 2U_j^n + U_j^{n-1}}{\Delta t^2} = \frac{\Delta t^2 u_{tt} + \frac{\Delta t^4}{12} u_{tttt}}{\Delta t^2}
\end{equation}

By a similar pattern the term to the right becomes:

\begin{equation}
-c^2 \frac{U_{j+1}^n - 2U_j^n + U_{j-1}^n}{\Delta x^2} = -c^2 \frac{\Delta x^2 u_{xx} + \frac{\Delta x^4}{12}u_{xxxx}}{\Delta x^2}
\end{equation}

Combining these and simplifying them further allows us to rewrite the numerical scheme as:

\begin{equation}
u_{tt} + \frac{\Delta t^2}{12}u_{tttt} - c^2 \left(u_{xx} + \frac{\Delta x^2}{12} u_{xxxx}\right)
\end{equation}

Recall that hte order of accuracy is the largest $p$ such that $T_j^n = O((\Delta x)^p + (\Delta t)^p)$. So we can see that for our scheme, $p=2$, and thus this numerical scheme is indeed second-order accurate.

## 1b)

We can modify the ansatz $U_j^n = \lambda(k)^n e ^{ijk\Delta x}$ for the $n \pm 1$ and $j \pm 1$ terms to get the following:

\begin{equation}
U_j^{n+1} = \lambda(k)^{n+1} e^{ijk\Delta x} = \lambda(k)^n \lambda(k) e^{ijk\Delta x}
\end{equation}

\begin{equation}
U_j^{n-1} = \lambda(k)^{n-1} e^{ijk\Delta x} = \lambda(k)^n \lambda(k)^{-1} e^{ijk\Delta x}
\end{equation}

\begin{equation}
U_{j+1}^{n} = \lambda(k)^{n} e^{i(j+1)k\Delta x} = \lambda(k)^n e^{ijk\Delta x} e^{ik\Delta x}
\end{equation}

\begin{equation}
U_{j-1}^{n} = \lambda(k)^{n} e^{i(j-1)k\Delta x} = \lambda(k)^n e^{ijk\Delta x} e^{-ik\Delta x}
\end{equation}

We can begin by modifying the term to the left by substituting from the above:

\begin{equation}
\frac{U_j^{n+1} - 2U_j^n + U_j^{n-1}}{\Delta t^2} = \frac{\lambda(k)^n \lambda(k) e^{ijk\Delta x} - 2\lambda(k)^n e ^{ijk\Delta x} + \lambda(k)^n \lambda(k)^{-1} e^{ijk\Delta x}}{\Delta t^2}
\end{equation}

We rearrange to produce:

\begin{equation}
\frac{U_j^{n+1} - 2U_j^n + U_j^{n-1}}{\Delta t^2} = \frac{\lambda(k)^n e ^{ijk\Delta x} \left( \lambda(k) - 2 + \lambda(k)^{-1} \right)}{\Delta t^2}
\end{equation}

Now we substitute from above into the term to the right:

\begin{equation}
-c^2 \frac{U_{j+1}^n - 2U_j^n + U_{j-1}^n}{\Delta x^2} = \frac{c^2 \left(\lambda(k)^n e^{ijk\Delta x} e^{ik\Delta x} - 2\lambda(k)^n e ^{ijk\Delta x} + \lambda(k)^n e^{ijk\Delta x} e^{-ik\Delta x}\right)}{\Delta x^2}
\end{equation}

Rearrange to produce:
\begin{equation}
-c^2 \frac{U_{j+1}^n - 2U_j^n + U_{j-1}^n}{\Delta x^2} = \frac{-c^2 \lambda(k)^n e ^{ijk\Delta x} \left(e^{ik\Delta x} - 2 + e^{-ik\Delta x} \right)}{\Delta x^2}
\end{equation}

Now, combining the two pieces back together, we have:

\begin{equation}
\frac{\lambda(k)^n e ^{ijk\Delta x} \left( \lambda(k) - 2 + \lambda(k)^{-1} \right)}{\Delta t^2} - \frac{c^2 \lambda(k)^n e ^{ijk\Delta x} \left(e^{ik\Delta x} - 2 + e^{-ik\Delta x} \right)}{\Delta x^2} = 0
\end{equation}

Pull the one term to the other side to produce:
\begin{equation}
\frac{\lambda(k)^n e ^{ijk\Delta x} \left( \lambda(k) - 2 + \lambda(k)^{-1} \right)}{\Delta t^2} = \frac{c^2 \lambda(k)^n e ^{ijk\Delta x} \left(e^{ik\Delta x} - 2 + e^{-ik\Delta x} \right)}{\Delta x^2}
\end{equation}

The $\lambda(k)^n e ^{ijk\Delta x}$ terms cancel, producing:
\begin{equation}
\frac{\lambda(k) - 2 + \lambda(k)^{-1}}{\Delta t^2} = \frac{c^2 \left(e^{ik\Delta x} - 2 + e^{-ik\Delta x} \right)}{\Delta x^2}
\end{equation}

Rearrange and recall that $\nu = \frac{c \Delta t}{\Delta x}$. This gives us:

\begin{equation}
\lambda(k) + \lambda(k)^{-1} = 2 + \nu^2 (-2 + e^{ik\Delta x} + e^{-ik\Delta x})
\end{equation}

Multiply both sides by $\lambda(k)$ and rearrnage to produce the quadratic:

\begin{equation}
0 = \lambda(k)^2 - \lambda(k) \left(2 + \nu^2(-2 + e^{ik\Delta x} + e^{-ik\Delta x}) \right) + 1
\end{equation}

Using Euler's formula, we have:

\begin{equation}
0 = \lambda(k)^2 - \lambda(k) \left(2 + \nu^2(-2 + cos(k\Delta x) + isin(k\Delta x) + cos(-k\Delta x) + isin(-k\Delta x)) \right) + 1
\end{equation}

Because $cos(x) = cos(-x)$ and $sin(-x) = -sin(x)$, the above equation reduces to:

\begin{equation}
0 = \lambda(k)^2 - \lambda(k) \left(2 + \nu^2(-2 + 2cos(k\Delta x)\right) + 1
\end{equation}

Factor a $2$ and this becomes:
\begin{equation}
0 = \lambda(k)^2 - \lambda(k) \left(2 \left( 1 + \nu^2(-1 + cos(k\Delta x)\right)\right) + 1
\end{equation}

Using the quadratic formula produces two solutions:

\begin{equation}
\lambda (k) = \left(1 + \nu^2(-1 + cos(k\Delta x))\right) \pm \sqrt{\left( 1 + \nu^2(-1 + cos(k\Delta x)\right)^2 -1}
\end{equation}

Since $-1 \leq cos(k \Delta x) \leq 1$, the parenthetical term $(-1 + cos(k \Delta x)) \in [-2,0]$, and we know $\nu \in (0, 1]$ to ensure CFL is satisfied. So the value under the square root is less than or equal to 0.

So we can factor out a $\sqrt{-1} = i$, which gives us:

\begin{equation}
\lambda (k) = \left(1 + \nu^2(-1 + cos(k\Delta x))\right) \pm i\sqrt{-\left( 1 + \nu^2(-1 + cos(k\Delta x)\right)^2 +1}
\end{equation}

For stability we know $|\lambda(k)| \leq 1$. In addition, we can recognize the above solution can be represented as a vector on a real-imaginary plane, where the real and imaginary components of the vector correspond to those we see in the solution.

Thus, the magnitude of the $\lambda(k)$ vector can be found using the Pythagorean theorem and the two components. And the two solutions will have the same magnitude, so we can collapse them into one situation.

This looks like

\begin{equation}
\left|\lambda (k)\right|^2 = \left|\left(1 + \nu^2(-1 + cos(k\Delta x))\right)\right|^2 + \left|\sqrt{-\left( 1 + \nu^2(-1 + cos(k\Delta x)\right)^2 +1}\right|^2
\end{equation}

and becomes

\begin{equation}
\left|\lambda (k)\right|^2 = \left|\left(1 + \nu^2(-1 + cos(k\Delta x))\right)\right|^2 + \left(-\left( 1 + \nu^2(-1 + cos(k\Delta x)\right)^2 +1\right)
\end{equation}

Everything but the $1$ cancels, leaving:

$\left|\lambda (k)\right|^2 = 1$

This satisfies the stability condition $|\lambda(k)| \leq 1$. And thus we can see that the numerical scheme is stable.

# Problem 2
## 2a)

Rewriting the two-dimensiona discretization allows us to put the $P^{n+1}$ term on the left-hand side, with the $P^n$ and $P^{n-1}$ terms on the right-hand side:

\begin{equation}
P_{j,k}^{n+1} = -P_{j,k}^{n-1} + 2P_{j,k}^n + \frac{c^2 \Delta t^2}{h^2}(P_{j+1,k}^n + P_{j,k+1}^n - 4P_{j,k}^n + P_{j-1,k}^n + P_{j,k-1}^n
\end{equation}

Substituting $\nu = \frac{c^2 \Delta t^2}{h^2}$ gives us:

\begin{equation}
P_{j,k}^{n+1} = -P_{j,k}^{n-1} + 2P_{j,k}^n + \nu (P_{j+1,k}^n + P_{j,k+1}^n - 4P_{j,k}^n + P_{j-1,k}^n + P_{j,k-1}^n
\end{equation}

Thus we have a clear path to get from $P^n$ and $P^{n-1}$ to $P^{n+1}$. Doing this recursively allows us to step "forward" in the $P^n$ values.

Using this as the core (and applying boundary conditions as necessary), we can write the program below, which solves for the pressure field inside the building:

```python
import numpy as np
pierce = np.loadtxt("./pierce.txt")

```

```python
# This builds the 20,000 x 20,000 matrix that "maps" from P_n and P_{n-1} to P_{n+1}
def n_matrix(edge_v, center_v, Pierce_map):
    m = 100
    n = 200

    A = np.zeros((m*n, m*n))
    np.fill_diagonal(A, center_v)
    np.fill_diagonal(A[1:], edge_v)
    np.fill_diagonal(A[:,1:], edge_v)
    np.fill_diagonal(A[n:], edge_v)
    np.fill_diagonal(A[:,n:], edge_v)

    for j in range(m):
        for k in range(n):
            if j < m-1 and Pierce_map[j+1, k] == 1:
                A[n*j + k, n*(j+1) + k] = 0
                A[n*j + k, n*j + k] += edge_v
            if j > 0 and Pierce_map[j-1, k] == 1:
                A[n*j + k, n*(j-1) + k] = 0
                A[n*j + k, n*j + k] += edge_v
            if k < n-1 and Pierce_map[j, k+1] == 1:
                A[n*j + k, n*j + (k+1)] = 0
                A[n*j + k, n*j + k] += edge_v
            if k > 0 and Pierce_map[j, k-1] == 1:
                A[n*j + k, n*j + (k-1)] = 0
                A[n*j + k, n*j + k] += edge_v

    return A
```

```python
# Plug in constants (and make sure they are all in SI units)
c = 343
h = 0.366
dt = h/(2*c)
v = ((c**2)*(dt**2))/(h**2)
center_val = 2 - 4*v

A = n_matrix(v, center_val, pierce)
```

```python
# This function corresponds to the speaker's ongoing sound
def P_vec_with_sin(P_vec, p_0, w, t):
    P_vec = np.reshape(P_vec, (100,200))
    P_vec[57:61, 15:19] = p_0*np.sin(w*t)
    P_vec = np.reshape(P_vec, (20000,1))
    return P_vec
```

```python
# Putting it all together produces the following.

Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 400
p_0 = 10
w = 100*np.pi

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone # set values for next iteration

    # This statement allows us to break when we reach our desired threshold
    if time_now >= 0.015:
        break

# The print statement below allows us to verify that the program terminated because
# it hit the "if time_now >=" break, and not because it simply ran out of time_iters
print(time_now)
```

    0.015472303206997082

## 2b)

```python
# This code will allow us to plot our pressure fields

import numpy as np
from math import *
import matplotlib.pyplot as plt
import sys

# Returns a scaled value of a function in the range 0 to 1, truncating it if
# necessary
def fscale(v,vmin,vsca):
    vs=(v-vmin)*vsca
    if vs<0: vs=0
    if vs>1: vs=1
    return vs

# Returns a red->white->blue color scheme
def palette1(v):
    v*=2
    if(v>1):
        v=2-v
        return (v,v,min(1,0.5+3*v))
    else:
        return (min(1,0.5+3*v),v,v)

# Outputs a 2D image from a field using the palette1 color scheme
# fn: the filename to save to
# p: the 2D field to plot
# wa: the matrix describing walls to overlay
# (vmin,vmax): the field range
# ups: the upsampling factor to use (so each field point is convert into an ups
#      by ups square)
def plot1(fn,p,wa,vmin,vmax,ups):

    # Check matrix dimensions are the same
    (m,n)=p.shape
    if (m,n)!=wa.shape:
        print("Matrix dimension mismatch")

    # Set up output array and scaling constant
    o=np.zeros((m*ups,n*ups,3))
    vsca=1.0/(vmax-vmin)

    # Assemble the output array
    for i in range(m):
        iu=i*ups
        for j in range(n):
            ju=j*ups
            if wa[i,j]==1:
                o[iu:iu+ups,ju:ju+ups,0]=0
                o[iu:iu+ups,ju:ju+ups,1]=0
                o[iu:iu+ups,ju:ju+ups,2]=0
            else:
                (re,gr,bl)=palette1(fscale(p[i,j],vmin,vsca))
                o[iu:iu+ups,ju:ju+ups,0]=re
                o[iu:iu+ups,ju:ju+ups,1]=gr
                o[iu:iu+ups,ju:ju+ups,2]=bl

    # Save the image
    plt.imsave(fn,o)

# Returns a black->red->yellow color scheme (matching Gnuplot's PM3D scheme)
def palette2(v):
    if v>0.5:
        vs=0
    else:
        vs=sin(2*pi*v)
    return (sqrt(v),v*v*v,vs)

# Outputs a 2D image from a field using the palette2 color scheme
# fn: the filename to save to
# p: the 2D field to plot
# wa: the matrix describing walls to overlay
# (vmin,vmax): the field range
# ups: the upsampling factor to use (so each field point is convert into an ups
#      by ups square)
def plot2(fn,p,wa,vmin,vmax,ups):

    # Check matrix dimensions are the same
    (m,n)=p.shape
    if (m,n)!=wa.shape:
        print("Matrix dimension mismatch")

    # Set up output array and scaling constant
    o=np.zeros((m*ups,n*ups,3))
    vsca=1.0/(vmax-vmin)

    # Assemble the output array
    for i in range(m):
        iu=i*ups
        for j in range(n):
            ju=j*ups
            if wa[i,j]==1:
                o[iu:iu+ups,ju:ju+ups,0]=1
                o[iu:iu+ups,ju:ju+ups,1]=1
                o[iu:iu+ups,ju:ju+ups,2]=1
            else:
                (re,gr,bl)=palette2(fscale(p[i,j],vmin,vsca))
                o[iu:iu+ups,ju:ju+ups,0]=re
                o[iu:iu+ups,ju:ju+ups,1]=gr
                o[iu:iu+ups,ju:ju+ups,2]=bl

    # Save the image
    plt.imsave(fn,o)

```

```python
# Putting it all together produces the following.

Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 400
p_0 = 10
w = 100*np.pi

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone # set values for next iteration

    # This statement allows us to break when we reach our desired threshold
    if time_now >= 0.015:
        break

# The print statement below allows us to verify that the program terminated because
# it hit the "if time_now >=" break, and not because it simply ran out of time_iters
print(time_now)
```

    0.015472303206997082

```python
P_nplusone_for_plotting = np.reshape(P_nplusone, (100,200))
plot1("./Plot_1.png", P_nplusone_for_plotting, pierce, -1.1, 1.1, 3)
plot2("./Plot_2.png", P_nplusone_for_plotting, pierce, -1.1, 1.1, 3)
```

So our plots of the pressure field in the building at t = 0.015s are:

Plot_1.png

Plot_2.png

```python
Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 400
p_0 = 10
w = 100*np.pi

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone

    if time_now >= 0.105:
        break

```

```python
P_nplusone_for_plotting = np.reshape(P_nplusone, (100,200))
plot1("./Plot_3.png", P_nplusone_for_plotting, pierce, -1.1, 1.1, 3)
plot2("./Plot_4.png", P_nplusone_for_plotting, pierce, -1.1, 1.1, 3)
```

Our plots of the pressure field in the building at t = 0.105s are:

Plot_3.png

Plot_4.png

```python
Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 1000
p_0 = 10
w = 100*np.pi

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone

    if time_now >= 0.505:
        break

print(time_now)
```

    0.5052507288629737

```python
P_nplusone_for_plotting = np.reshape(P_nplusone, (100,200))
plot1("./Plot_5.png", P_nplusone_for_plotting, pierce, -1.1, 1.1, 3)
plot2("./Plot_6.png", P_nplusone_for_plotting, pierce, -1.1, 1.1, 3)
```

Our plots of the pressure field in the building at t = 0.505s are:

Plot_5.png

Plot_6.png

```python
Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 2000
p_0 = 10
w = 100*np.pi

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone

    if time_now >= 1.005:
        break

print(time_now)
```

    1.0051661807580174

```python
P_nplusone_for_plotting = np.reshape(P_nplusone, (100,200))
plot1("./Plot_7.png", P_nplusone_for_plotting, pierce, -1.1, 1.1, 3)
plot2("./Plot_8.png", P_nplusone_for_plotting, pierce, -1.1, 1.1, 3)
```

Our plots of the pressure field in the building at t = 1.005s are:

Plot_7.png

Plot_8.png

## 2c)

Use similar code as before, but set the new terminating condition.

```python
Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 1000
p_0 = 10
w = 100*np.pi

person1_p_list = []

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone

    P_reshaped = np.reshape(P_nplusone, (100,200))
    P_abs_val = abs(P_reshaped[35, 73])

    person1_p_list.append(P_abs_val)

    if P_abs_val > 10**(-3):
        break

```

```python
print("Statements to check termination condition:")
print(P_abs_val)
print(person1_p_list[-1])
print(person1_p_list[-2])
print("\n", "Time when person first hears the sound:")
print(time_now)
```

    Statements to check termination condition:
    0.00142746642970674
    0.00142746642970674
    0.0009096378034446683

     Time when person first hears the sound:
    0.07309329446064139

So Person C first hears the sound at 0.073 seconds.

```python
Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 1000
p_0 = 10
w = 100*np.pi

person2_p_list = []

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone

    P_reshaped = np.reshape(P_nplusone, (100,200))
    P_abs_val = abs(P_reshaped[61, 109])

    person2_p_list.append(P_abs_val)

    if P_abs_val > 10**(-3):
        break
```

```python
print("Statements to check termination condition:")
print(P_abs_val)
print(person2_p_list[-1])
print(person2_p_list[-2])
print("\n", "Time when person first hears the sound:")
print(time_now)
```

    Statements to check termination condition:
    0.001262251537415511
    0.001262251537415511
    0.0008633498112028842

     Time when person first hears the sound:
    0.10830612244897958

So Person G first hears the sound at 0.108 seconds.

```python
Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 1000
p_0 = 10
w = 100*np.pi

time_list = []
person3_p_list = []

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone

    P_reshaped = np.reshape(P_nplusone, (100,200))
    P_abs_val = abs(P_reshaped[91, 188])

    person3_p_list.append(P_abs_val)

    time_list.append(time_now)

    if P_abs_val > 10**(-3):
        break
```

```python
print("Statements to check termination condition:")
print(P_abs_val)
print(person3_p_list[-1])
print(person3_p_list[-2])
print("\n", "Time when person first hears the sound:")
print(time_now)
```

    Statements to check termination condition:
    0.0010012132239861087
    0.0010012132239861087
    0.0009889676987950844

     Time when person first hears the sound:
    0.23048396501457724

So Person M first hears the sound at 0.230 seconds.

These resutls are reasonable. The order in which they hear the sound corresponds to the order of their closeness to the speaker (e.g., Person C is closest to the speaker and hears the sound first). The plots above show this visually -- the wave propogates outward from the speaker as time progresses, and it reaches Person C before it reaches Person G before it reaches Person M.

## 2d)

```python
Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 2000
p_0 = 10
w = 100*np.pi

personC_p_abs_val_list = []
personC_p_list = []
time_list = []

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone

    P_reshaped = np.reshape(P_nplusone, (100,200))
    P_abs_val = abs(P_reshaped[35, 73])
    P_val = P_reshaped[35,73]

    personC_p_abs_val_list.append(P_abs_val)
    personC_p_list.append(P_val)
    time_list.append(time_now)

    if time_now >= 1.000:
        break
print(time_now)
```

    1.0003644314868803

```python
Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 2000
p_0 = 10
w = 100*np.pi

personG_p_abs_val_list = []
personG_p_list = []
time_list = []

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone

    P_reshaped = np.reshape(P_nplusone, (100,200))
    P_abs_val = abs(P_reshaped[61, 109])
    P_val = P_reshaped[61,109]

    personG_p_abs_val_list.append(P_abs_val)
    personG_p_list.append(P_val)
    time_list.append(time_now)

    if time_now >= 1.000:
        break
print(time_now)
```

    1.0003644314868803

```python
Pn_minus_1 = np.zeros((20000,1))
Pn = np.zeros((20000,1))

time_iters = 2000
p_0 = 10
w = 100*np.pi

personM_p_abs_val_list = []
personM_p_list = []
time_list = []

for interval in range(time_iters):

    time_now = interval*dt # Update time for sin function
    Pn = P_vec_with_sin(Pn, p_0, w, time_now) # Update P_n vector with speaker sin wave

    P_nplusone = np.dot(A, Pn) - Pn_minus_1 # Produce new P at n+1
    Pn_minus_1 = Pn # set values for next iteration
    Pn = P_nplusone

    P_reshaped = np.reshape(P_nplusone, (100,200))
    P_abs_val = abs(P_reshaped[91, 188])
    P_val = P_reshaped[91,188]

    personM_p_abs_val_list.append(P_abs_val)
    personM_p_list.append(P_val)
    time_list.append(time_now)

    if time_now >= 1.000:
        break
print(time_now)
```

    1.0003644314868803

```python
import matplotlib.pyplot as plt

plt.plot(time_list, personC_p_list, label="$p(t)$ at Person C's location", color='g')
plt.plot(time_list, personG_p_list, label="$p(t)$ at Person G's location", color='y')
plt.plot(time_list, personM_p_list, label="$p(t)$ at Person M's location", color='b')
x1, x2, y1, y2 = plt.axis()
plt.axis((0,1,y1,y2))
plt.xlabel("Time (seconds)")
plt.ylabel("$p(t)$")
plt.legend()
plt.show()
```

We can see that Person G is most likely to be disturbed by the speaker. At first, this is surprising, since Person C is closer to the speaker. But due to the way the floor plan interacts with the sound wave, the sound is lounder in Person G's office; we can see this in the plots above (the color is more intense in Person G's office than Person C's). So this conclusion aligns with our plots above. The fact that even Person M ends up hearing more than Person C indicates that the sound waves interfere destructively in Person C's office more than at the locations of Person G and Person M (or else, the waves interfere constructively at the G and M locations in comparison to their interference at C).

# Problem 3

## 3a)

To derive the Jacobian, let's first write out the $F(U)$ vector:

\begin{equation*}
    F(U) =
    \begin{bmatrix}
    \frac{U_2 - 2U_1}{h^2} - e^{U_1} \\
    \frac{U_3 - 2U_2 + U_1}{h^2} - e^{U_2} \\
    \frac{U_4 - 2U_3 + U_2}{h^2} - e^{U_3} \\
    \frac{U_5 - 2U_4 + U_3}{h^2} - e^{U_4} \\
    \vdots \\
    \frac{-2U_{n-2} + U_{n-3}}{h^2} - e^{U_{n-2}} \\
    \end{bmatrix}
\end{equation*}

We can therefore see the form of the Jacobian $J_F$ will be:

\begin{equation*}
    J_F =
    \begin{bmatrix}
    \frac{\delta F_1(U)}{\delta U_1} & \frac{\delta F_1(U)}{\delta U_2} & 0 & 0 & 0 & 0 & \ldots & 0 \\
    \frac{\delta F_i(U)}{\delta U_{i-1}} & \frac{\delta F_i(U)}{\delta U_i} & \frac{\delta F_i(U)}{\delta U_{i+1}} & 0 & 0 & 0 & \ldots & 0 \\
    0 & \ddots & \ddots & \ddots & 0 & 0 & \ldots & 0 \\
    0 & 0 & \ddots & \ddots & \ddots & 0 & \ldots & 0 \\
    0 & \ldots & \ldots & \ddots & \ddots & \ddots & \ldots & 0 \\
    \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\
    0 & 0 & \ldots & \ldots & \ldots & \ldots & \frac{\delta F_{n-2}(U)}{\delta U_{n-3}} & \frac{\delta F_{n-2}(U)}{\delta U_{n-2}} \\
    \end{bmatrix}
\end{equation*}

This gives us the derivatives we need to find, which are:

(For the first row)

$\frac{\delta F_1(U)}{\delta U_1} = \frac{-2}{h^2} - e^{U_1}$

$\frac{\delta F_1(U)}{\delta U_2} = \frac{1}{h^2}$

(For the middle rows)

$\frac{\delta F_i(U)}{\delta U_{i-1}} = \frac{1}{h^2}$

$\frac{\delta F_i(U)}{\delta U_i} = \frac{-2}{h^2} - e^{U_i}$

$\frac{\delta F_i(U)}{\delta U_{i+1}} = \frac{1}{h^2}$

(For the final row)

$\frac{\delta F_{n-2}(U)}{\delta U_{n-3}} = \frac{1}{h^2}$

$\frac{\delta F_{n-2}(U)}{\delta U_{n-2}} = \frac{-2}{h^2} - e^{U_{n-2}}$

So now we can write the complete Jacobian by substituting the above:

\begin{equation*}
    J_F =
    \begin{bmatrix}
    \frac{-2}{h^2} - e^{U_1} & \frac{1}{h^2} & 0 & 0 & 0 & 0 & \ldots & 0 \\
    \frac{1}{h^2} & \frac{-2}{h^2} - e^{U_i} & \frac{1}{h^2} & 0 & 0 & 0 & \ldots & 0 \\
    0 & \frac{1}{h^2} & \frac{-2}{h^2} - e^{U_i} & \frac{1}{h^2} & 0 & 0 & \ldots & 0 \\
    0 & 0 & \ddots & \ddots & \ddots & 0 & \ldots & 0 \\
    0 & \ldots & \ldots & \ddots & \ddots & \ddots & \ldots & 0 \\
    \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\
    0 & 0 & \ldots & \ldots & \ldots & \ldots & \frac{1}{h^2} & \frac{-2}{h^2} - e^{U_{n-2}} \\
    \end{bmatrix}
\end{equation*}

In the case $n = 101$:

\begin{equation*}
    J_F =
    \begin{bmatrix}
    \frac{-2}{h^2} - e^{U_1} & \frac{1}{h^2} & 0 & 0 & 0 & 0 & \ldots & 0 \\
    \frac{1}{h^2} & \frac{-2}{h^2} - e^{U_2} & \frac{1}{h^2} & 0 & 0 & 0 & \ldots & 0 \\
    0 & \frac{1}{h^2} & \frac{-2}{h^2} - e^{U_3} & \frac{1}{h^2} & 0 & 0 & \ldots & 0 \\
    0 & 0 & \ddots & \ddots & \ddots & 0 & \ldots & 0 \\
    0 & \ldots & \ldots & \ddots & \ddots & \ddots & \ldots & 0 \\
    \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\
    0 & 0 & 0 & \ldots & 0 & \frac{1}{h^2} & \frac{-2}{h^2} - e^{U_{98}} & \frac{1}{h^2} \\
    0 & 0 & \ldots & \ldots & \ldots & \ldots & \frac{1}{h^2} & \frac{-2}{h^2} - e^{U_{99}} \\
    \end{bmatrix}
\end{equation*}

So the sparsity pattern of $J_F$ is a square matrix with $ n-2 = 99$ rows and columns, with zeros everywhere except for 3 spots in each row: the value that represents the matrix diagonal for that row, as well as the value immediately to the left and right of that diagonal value. If you step back, it looks like a 0 matrix with three adjacent diagonal lines running from the top left to the bottom right.

## 3b)

```python
n = 101
h = 2/(n-1)

def Jacobian(U_vector):

    J = np.zeros((99, 99))
    np.fill_diagonal(J, (-2/(h**2) - np.exp(U_vector)))
    np.fill_diagonal(J[1:], 1/(h**2))
    np.fill_diagonal(J[:,1:], 1/(h**2))

    return J

def f(U):
    f = []
    f.append((U[1] - 2*U[0])/h**2 - np.exp(U[0])) # first elem
    for i in range(1,U.shape[0]-1): # middle elems
        f.append((1/h**2)*(U[i+1] - 2*U[i] + U[i-1]) - np.exp(U[i]))

    f.append((1/h**2)*(-2*U[-1] + U[-2]) - np.exp(U[-1])) # last elem
    return np.array(f)
```

```python
U = np.zeros((n-2, 1))

def newtons_vec_method(U, n):

    while 1 == 1:

        F = f(U)
        J = Jacobian(U)

        delta_U = np.linalg.solve(J, F)
        Unew = U - delta_U

        delta_U_norm = np.linalg.norm(delta_U)
        U_norm = np.linalg.norm(U)

        if U_norm != 0:
            if delta_U_norm/U_norm < 10**(-10):
                return U
        U = Unew

    return U

```

```python
U_array = newtons_vec_method(U, n)

U_list = list(U_array)
ODE_soln = [0.0] + U_list + [0.0]
```

```python
x_vals = []
for i in range(n):
    x_vals.append(-1 + i*h)

#Find index value where x = 0, to find u(0)
index = list(x_vals).index(0)

#Grab u(0), the solutiion value at index i
print("Approximation of u(0) is", ODE_soln[index])

plt.plot(x_vals, ODE_soln)
plt.title("Approximate solution for the given ODE BVP")
plt.xlabel("x")
plt.ylabel("$u(x)$")
plt.show()
```

So our approximation for $u(0)$ is -0.36804822

For this problem set I spoke with Kaela Nelson and Matthieu Meeus.

```python

```

Convergence & Stability, Discretized Pressure Field, Nonlinear BVP.ipynb


```python
from sympy import symbols, fourier_transform, sinc, pi,\
                  DiracDelta, Heaviside, exp, sin, cos

w, t = symbols('omega t')
a, b, W, T = symbols('a b W T', positive=True)

fourier_transform(DiracDelta(t), t, w)
```

$\displaystyle 1$

```python
fourier_transform(Heaviside(t + T/2) - Heaviside(t - T/2), t, w)
```

$\displaystyle \frac{\sin{\left(\pi T \omega \right)}}{\pi \omega}$

```python
fourier_transform(sinc(2*pi*W*t), t, w)
```

$\displaystyle \begin{cases} \frac{1}{2 W} & \text{for}\: \frac{W^{2}}{\left|{\omega^{2}}\right|} > 1 \\0 & \text{otherwise} \end{cases}$

```python
fourier_transform(exp(-a**2*t**2), t, w)
```

$\displaystyle \frac{\sqrt{\pi} e^{- \frac{\pi^{2} \omega^{2}}{a^{2}}}}{a}$


# SymPy試してみる

[SymPy](https://docs.sympy.org/latest/index.html)ライブラリを試してみる。

まずは[tutorial](https://docs.sympy.org/latest/tutorial/intro.html)を参考にして、、、

### 微分積分

微分積分してみる。

```python
import sympy as sp

x=sp.symbols('x')
y1=sp.sin(x)
y2=sp.Integral(y1,x)
y3=sp.Integral(y1,(x, 0, 2*sp.pi))

```

不定積分

```python
eq1=sp.Eq(y2,y2.doit())
eq1
```

$\displaystyle \int \sin{\left(x \right)}\, dx = - \cos{\left(x \right)}$

定積分

```python
eq2=sp.Eq(y3,y3.doit())
eq2
```

$\displaystyle \int\limits_{0}^{2 \pi} \sin{\left(x \right)}\, dx = 0$

プロット

```python
sp.plot(y1,y2.doit(),(x,0,2*sp.pi),legend = True)
```

ハイパボリック

```python
b=sp.sinh(x)
c=sp.Integral(b,x)
sp.Eq(c,c.doit())
```

$\displaystyle \int \sinh{\left(x \right)}\, dx = \cosh{\left(x \right)}$

微分

$(e^{-x}\sinh{(x^2+1)})'=$　の形で出力する方法がまだわからいない

```python
a=sp.exp(-x)*sp.sinh(x**2+1)
a

```

$\displaystyle e^{- x} \sinh{\left(x^{2} + 1 \right)}$

```python
sp.diff(a,x)

```

$\displaystyle 2 x e^{- x} \cosh{\left(x^{2} + 1 \right)} - e^{- x} \sinh{\left(x^{2} + 1 \right)}$

使った機能をとりあえず以下にメモ(まだ中身はよく理解してない)

・[symbol.symbols()メソッド](https://docs.sympy.org/latest/modules/core.html#symbols)
・[Integralクラス ](https://docs.sympy.org/latest/modules/integrals/integrals.html#sympy.integrals.integrals.Integral)
・[Integral.doit()メソッド](https://docs.sympy.org/latest/modules/core.html#sympy.core.basic.Basic.doit)
・[diff()メソッド ](https://docs.sympy.org/latest/modules/core.html#sympy.core.function.diff)
・[Eq/Equality()クラス](https://docs.sympy.org/latest/modules/core.html#sympy.core.relational.Equality)

###　微分方程式

微分方程式解いてみる

```python
f = sp.symbols('f', cls=sp.Function)
diffeq = sp.Eq(f(x).diff(x, x) - 2*f(x).diff(x) + f(x), sp.sin(x))
diffeq
```

$\displaystyle f{\left(x \right)} - 2 \frac{d}{d x} f{\left(x \right)} + \frac{d^{2}}{d x^{2}} f{\left(x \right)} = \sin{\left(x \right)}$

```python
sp.dsolve(diffeq,f(x))
```

$\displaystyle f{\left(x \right)} = \left(C_{1} + C_{2} x\right) e^{x} + \frac{\cos{\left(x \right)}}{2}$

使った機能メモ

[Functionクラス](https://docs.sympy.org/latest/modules/core.html?highlight=function#sympy.core.function.Function)
[dsolve()メソッド](https://docs.sympy.org/latest/modules/solvers/ode.html?highlight=dsolve#sympy.solvers.ode.dsolve)

### 因数分解

多項式の因数分解

```python
poly=x**3 - x**2 + x - 1
sp.Eq(poly,sp.factor(poly))
```

$\displaystyle x^{3} - x^{2} + x - 1 = \left(x - 1\right) \left(x^{2} + 1\right)$

使った機能メモ

[Factorメソッド](https://docs.sympy.org/latest/modules/polys/reference.html?highlight=factor#sympy.polys.polytools.factor)

### ここまでを詳しく調べてみる。

```python
type(x)
```

```python
dir(x)
```

```python
type(y1)
```

    sin

```python
dir(y1)
```

```python
type(eq1)
```

```python
dir(eq1)
```


```python
from sympy import *

a, q0 = var("a, q0")

Ah, Av, Bh, Bv, Gh, Gv = var("Ah, Av, Bh, Bv, Gh, Gv")

x = var("x")
q1 = q0/3   * x/a
q2 = 2*q0/3 * x/(2*a)
q3 = q0/3

# Resultants:
R1 = integrate( q1, (x, 0,   a) )
R2 = integrate( q2, (x, 0, 2*a) )
R3 = integrate( q3, (x, 0, 2*a) )

x1 = integrate( x*q1, (x, 0,   a) ) / R1
x2 = integrate( x*q2, (x, 0, 2*a) ) / R2
x3 = integrate( x*q3, (x, 0, 2*a) ) / R3

# Print results:
pprint("\nx1, R1:")
pprint(x1)
pprint(R1)
pprint("\nx2, R2:")
pprint(x2)
pprint(R2)
pprint("\nx3, R3:")
pprint(x3)
pprint(R3)

eq1 = Eq(  0, Ah + Gh             )
eq2 = Eq(  0, R1 + Gv - Av        )
eq3 = Eq(  0, 2*R1/3 + Gv + Gh    )
eq4 = Eq(  0, Bh - Gh             )
eq5 = Eq(  0, R2 + R3 - Gv - Bv   )
eq6 = Eq(  0, 2*Gv - R3 - 2*R2/3  )

eqns = [eq1, eq2, eq3, eq4, eq5, eq6]
unks = [Ah, Av, Bh, Bv, Gh, Gv]

sol = solve( eqns, unks)
pprint(sol)
pprint(latex(sol))

# x1, R1:
# 2⋅a
# ───
#  3
# a⋅q₀
# ────
#  6
#
# x2, R2:
# 4⋅a
# ───
#  3
# 2⋅a⋅q₀
# ──────
#   3
#
# x3, R3:
# a
# 2⋅a⋅q₀
# ──────
#   3
# ⎧    2⋅a⋅q₀      13⋅a⋅q₀      -2⋅a⋅q₀       7⋅a⋅q₀      -2⋅a⋅q₀       5⋅a⋅q₀⎫
# ⎨Ah: ──────, Av: ───────, Bh: ────────, Bv: ──────, Gh: ────────, Gv: ──────⎬
# ⎩      3            18           3            9            3            9   ⎭

```


 $$\left\{ \begin{array}{lcc}
            \dot{x}_{1}=2x_{2}+x_{1}(x_{1}^{2}+2x_{2}^{4}) \\
             \\ \dot{x}_{2}=-2x_{1}+x_{2}(x_{1}^{2}+x_{2}^{4})
             \end{array}
\right.$$

```python
import sympy as sym
```

```python
#Con esto las salidas van a ser en LaTeX
sym.init_printing(use_latex=True)
```

```python
x_1, x_2 ,theta = sym.symbols('x_1 x_2 theta')
```

```python
X = sym.Matrix([x_1, x_2])
X
```

```python
f_1 = 2*x_2 + x_1*(x_1**2 + 2*x_2**4)
f_1
```

```python
f_2 = -2*x_1 + x_2*(x_1**2 + x_2**4)
f_2
```

```python
F = sym.Matrix([f_1,f_2])
F
```

```python
# puntos de equilibrio del sistema
pes = sym.solve([f_1,f_2])
pes
```

```python
A = F.jacobian(X)
A
```

```python
sym.latex(A)
```

    '\\left[\\begin{matrix}3 x_{1}^{2} + 2 x_{2}^{4} & 8 x_{1} x_{2}^{3} + 2\\\\2 x_{1} x_{2} - 2 & x_{1}^{2} + 5 x_{2}^{4}\\end{matrix}\\right]'

```python
A_1 = A.subs({x_1:0,x_2:0})
A_1
```

```python
A_1.eigenvals()
```

```python
expr = 2*x_1 *F[0] + 2*x_2*F[1]
expr = expr.simplify()
```

```python
sym.latex(expr)
```

    '2 x_{1}^{4} + 4 x_{1}^{2} x_{2}^{4} + 2 x_{1}^{2} x_{2}^{2} + 2 x_{2}^{6}'

```python
expr
```

```python

```


# Examples on Using MOIRA
---
__Mokbel Karam, James C. Sutherland and Tony Saad__

__Department of Chemical Engineering, University of Utah__

Below are the necessary libraries used in the examples below.

```python
# %config InlineBackend.figure_format = 'svg'
import matplotlib.pyplot as plt
# plt.rcParams['figure.figsize'] = [3.5, 4]
plt.rc("font", family='serif')
# plt.rc('text', usetex=True)
import numpy as np
from sympy import symbols, pretty_print
from IPython.display import display, Math, clear_output
from matplotlib import cm, animation
```

##  Example Using FTUS on Advection PDE (1D)
---

Consider the following PDE
\begin{equation}
u_t=-a u_x ; a>0
\end{equation}
to be solved using forward Euler discretization in time and upwind discretization in space(FTUS). The difference equation is
\begin{equation}
\frac{u_{i}^{n+1}-u_{i}^{n}}{\Delta t}=-a\left(\frac{u_{i}^{n}-u_{i-1}^{n}}{\Delta x}\right).
\end{equation}

```python
from src.MOIRA import DifferentialEquation, i, n # import the differential equation and the indicies we need to use

a= symbols('a') # define positive a constant for the advection velocity

DE1 = DifferentialEquation(dependentVar='u',independentVars=['x']) # construct a one D differential equation with 'x' and 'u' as independent and dependent variables, respectively

# method I of constructing the rhs:
advectionTerm1 = DE1.expr(points=[1, 0], direction='x', order=1, time=n+1) # construct the first upwind derivative of the dependent variabe 'u'
                                                                        # with respect to the independent variable 'x' using the stencil -1, and 0 evaluated at time tn = n

# set the rhs of the differential equation
DE1.set_rhs(- a * advectionTerm1 )

# compute the modified equation up to two terms
DE1.modified_equation(nterms=3)

# display the modified equation in latex form
display(Math(DE1.latex()))

```

$$\frac{\partial}{\partial t} u{\left (x,t \right )} =  - a \frac{\partial}{\partial x} u{\left (x,t \right )} + \frac{a}{2} \left(- \Delta x + \Delta{t} a\right) \frac{\partial^{2}}{\partial x^{2}}  u{\left (x,t \right )} + \frac{a}{6} \left(- \Delta x^{2} + 3 \Delta x \Delta{t} a - 2 \Delta{t}^{2} a^{2}\right) \frac{\partial^{3}}{\partial x^{3}}  u{\left (x,t \right )}$$

```python
from src.MOIRA import DifferentialEquation, j, n # import the differential equation and the indicies we need to use

b = symbols('b') # define positive a constant for the advection velocity

DE2 = DifferentialEquation(dependentVar='f',independentVars=['y'], indices=[j])# construct the first upwind derivative of the dependent variabe 'f'
                                                                        # with respect to the independent variable 'y' using the stencil -1, and 0 evaluated at time tn = n

# method II of constructing the rhs:
advectionTerm2 = (DE2.f(time=n, y=j) - DE2.f(time=n, y = j-1)) / DE2.dy

DE2.set_rhs(- b * advectionTerm2 ) # set the rhs of the Differential equation DE

DE2.modified_equation(nterms=2)  # compute the modified equation up to 2 terms

display(Math(DE2.latex())) # display the latex form of the modified equation
```

$$\frac{\partial}{\partial t} f{\left (y,t \right )} =  - b \frac{\partial}{\partial y} f{\left (y,t \right )} + \frac{b}{2} \left(\Delta y - \Delta{t} b\right) \frac{\partial^{2}}{\partial y^{2}}  f{\left (y,t \right )}$$

## Example Using BTCS on Advection-Diffusion PDEs (1D)
---

Consider the following PDE
\begin{equation}
u_t=-cu_x+\gamma u_{xx}
\end{equation}
to be solved using backward Euler discretization in time and central discretization in space(BTCS). The difference equation is
\begin{equation}
\frac{u_{i}^{n+1}-u_{i}^{n}}{\Delta t}=-c\left(\frac{u_{i+1}^{n+1}-u_{i-1}^{n+1}}{2\Delta x}\right)+ \gamma \left(\frac{u_{i+1}^{n+1}-2u_i^{n+1}+u_{i-1}^{n+1}}{\Delta x^2}\right).
\end{equation}

```python
from src.MOIRA import DifferentialEquation, i, n # import the differential equation and the indicies we need to use

c, g = symbols('c gamma') # define positive a constant for the advection velocity

DE3 = DifferentialEquation(dependentVar='u',independentVars=['x'], indices=[i])# construct the first upwind derivative of the dependent variabe 'f'
                                                                        # with respect to the independent variable 'y' using the stencil -1, and 0 evaluated at time tn = n

# method II of constructing the rhs:
advectionTerm3 = (DE3.u(time=n+1, x=i+1) - DE3.u(time=n+1, x=i-1)) / DE3.dx /2

diffusionTerm3 = (DE3.u(time=n+1, x=i+1) - 2* DE3.u(time=n+1, x=i) + DE3.u(time=n+1, x=i-1)) / DE3.dx / DE3.dx

DE3.set_rhs(- c * advectionTerm3 + g*diffusionTerm3 ) # set the rhs of the Differential equation DE

DE3.modified_equation(nterms=3)  # compute the modified equation up to 2 terms

display(Math(DE3.latex())) # display the latex form of the modified equation
```

$$\frac{\partial}{\partial t} u{\left (x,t \right )} =  - c \frac{\partial}{\partial x} u{\left (x,t \right )} + \left(\frac{\Delta{t} c^{2}}{2} + \gamma\right) \frac{\partial^{2}}{\partial x^{2}}  u{\left (x,t \right )} - c \left(\frac{\Delta x^{2}}{6} + \frac{\Delta{t}^{2} c^{2}}{3} + \Delta{t} \gamma\right) \frac{\partial^{3}}{\partial x^{3}}  u{\left (x,t \right )}$$

## Example Using FTUS on Advection PDEs (2D)
---

Consider the advection equation in a two dimensional space
\begin{equation}
u_{t}=-bu_{x}-cu_{y};\quad(b,c)\in\mathbb{R}^{+}
\end{equation}
to be discretized using a forward Euler method in time and UPWIND in space (FTUS)
\begin{equation}
\frac{u_{i,j}^{n+1}-u_{i,j}^{n}}{\Delta t}=-b\frac{u_{i,j}^{n}-u_{i-1,j}^{n}}{\Delta x}-c\frac{u_{i,j}^{n}-u_{i,j-1}^{n}}{\Delta y},
\end{equation}

with a modified equation

```python
from src.MOIRA import DifferentialEquation, i, n # import the differential equation and the indicies we need to use

b, c= symbols('b c') # define positive a constant for the advection velocity

DE1 = DifferentialEquation(dependentVar='u',independentVars=['x','y']) # construct a one D differential equation with 'x' and 'u' as independent and dependent variables, respectively

# method I of constructing the rhs:
advectionTerm1 = - b * DE1.expr(points=[-1, 0], direction='x', order=1, time=n)  - c * DE1.expr(points=[-1, 0], direction='y', order=1, time=n)

# set the rhs of the differential equation
DE1.set_rhs( advectionTerm1 )

# compute the modified equation up to two terms
DE1.modified_equation(nterms=2)

# display the modified equation in latex form
display(Math(DE1.latex()))
```

$$\frac{\partial}{\partial t} u{\left (x,y,t \right )} =  - c \frac{\partial}{\partial y} u{\left (x,y,t \right )} - b \frac{\partial}{\partial x} u{\left (x,y,t \right )} + \frac{c}{2} \left(\Delta y - \Delta{t} c\right) \frac{\partial^{2}}{\partial y^{2}}  u{\left (x,y,t \right )} - \Delta{t} b c \frac{\partial^{2}}{\partial x\partial y}  u{\left (x,y,t \right )} + \frac{b}{2} \left(\Delta x - \Delta{t} b\right) \frac{\partial^{2}}{\partial x^{2}}  u{\left (x,y,t \right )}$$

### Numerical Solution

```python
nx = 64
ny = 64
Lx = 1.0
Ly = 1.0
x = np.linspace(0,Lx,nx)
y = np.linspace(0,Ly,ny)
dx = Lx/(nx-1)
dy = Ly/(ny-1)
# create a grid of coordinates
xx,yy = np.meshgrid(x,y)

u0 = lambda x,y : np.exp(-(x-0.2)**2/0.01)*np.exp(-(y-0.2)**2/0.01)

cx = 1.0
cy = 1.0

dt = 1/128
tend = 0.5 #s
t = 0

cflx = cx * dt/dx
cfly = cy * dt/dy

# setup the initial condition
sol = []
u = np.zeros([ny+2,nx+2]) # we will ghost cells to simplify the implementation of periodic BCs
u[1:-1, 1:-1] = u0(xx,yy)
# set periodic boundaries
u[:,0] = u[:,-3] #x-minus face
u[:,-1] = u[:,2] #x-plus face
u[0,:] = u[-3,:] #y-minus face
u[-1,:] = u[2,:] #y-plus face
sol.append(u)
```

```python
while t < tend:
    un = sol[-1]
    unew = un.copy()
    unew[1:-1,1:-1] = un[1:-1,1:-1] - cflx * (un[1:-1,1:-1] - un[1:-1,:-2]) - cfly * (un[1:-1,1:-1] - un[:-2,1:-1])
    # set periodic boundaries
    unew[:,0] = unew[:,-3] #x-minus face
    unew[:,-1] = unew[:,2] #x-plus face
    unew[0,:] = unew[-3,:] #y-minus face
    unew[-1,:] = unew[2,:] #y-plus face

    sol.append(unew)
    t += dt
```

```python
levs = np.linspace(-1,1,20)
plt.figure(figsize=(10,8))
plt.contourf(xx,yy,sol[-1][1:-1,1:-1]+sol[0][1:-1,1:-1],cmap=cm.jet,levels=levs,vmax=1.0,vmin=-1.0)
plt.text(0.1, 0.4, 'time=0.0s', fontsize=12)
plt.text(0.6, 0.92,'time={}s'.format(tend), fontsize=12)
plt.xlabel('x', fontsize=14)
plt.ylabel('y', fontsize=14)
cbar = plt.colorbar()
plt.clim(-1,1)
cbar.set_ticks(np.linspace(-1,1,5))
# plt.savefig('./advection_2d.pdf',transparent=True)
plt.show()
```

## Numerical Solver
---

```python
%matplotlib notebook
from numerical.layout import *
plty=PlotStyling()
```

    Tab(children=(VBox(children=(HBox(children=(FloatText(value=0.01, description='dt:'), FloatText(value=10.0, de…


# Introduction to variational inference

## Reminder on bayesian statistics
We are given a set of observations $x \in \mathbb{R}^{n \times d}$ that we try interpret using a model parametrized by a vector $\theta \in \mathbb{R}^m$.

In the case of coin toss, an observation  $x_i \in x$ is the outcome (either heads or tails), and we can parametrize the experiment by a Bernoulli law with parameter $p$ (p=0.5 for fair coin). In that case, $\theta = \lbrace p \rbrace$.

The objective in that case is finding the true parameter of the coin, to decide if it is fair or not for instance.
Using Bayes theorem, we can write
$$ p(\theta | x) = \frac{p(x | \theta) p(\theta)}{p(x)} $$
or $$posterior =\frac{likelihood \times prior}{evidence} $$

**refine following paragraph**
In simple settings such as this one, computing the posterior distribution can be done in closed form using [conjugate priors](https://en.wikipedia.org/wiki/Conjugate_prior).
The other complicated task in bayesian inference is estimating the evidence -or marginal likelihood- as it requires computing an integral that is often intractable: $p(x)= \int p(X | \theta) p(\theta) d\theta$.

The solution to work around this issue is to fall back on aproximating the true posterior. The most popular methods are Markov Chain Monte Carlo ([MCMC](https://github.com/johnhw/mcmc_demo_2019.git) and Variational Inference (see next section for details). MCMC are sampling-based methods that take advantage of the law of large numbers for computing integrals using $\mathbb{E}_p \lbrack f(X) \rbrack = \int f(x) p(x) dx $.

## Variational Inference

For consistency with litterature, let us denote the latent variables $z \in \mathbb{R}^m$ (instead of $\theta$).
Our objective is to find a good approximation of the quantity $$p(z | x) =\frac{p(x | z) p(z)}{p(x)} $$ given that the quantity $p(x)$ is not tractable.

VI consists in choosing a distribution $q^*$ among a set $\mathcal{Q}$ that is "close" to the true posterior. That distance between probability distributions can be measured using the [Kullback-Leibler divergence](https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence).

Mathematically, we can write this as an optimization problem:
$$q^*(z) = \underset{q(z) \in \mathcal{Q}}{\text{arg min}}~ \mathrm{KL}(q(z) || p(z|x) $$
The trick consists in optimizing a surrogate of this objective function, since the presence of the marginal likelihood makes it intractable

$
\begin{equation}
\begin{split}
\mathrm{KL}(q(z) || p(z|x)) & =  \mathbb{E}_q \lbrack log \frac{q(z)}{p(z | x)} \rbrack \\
    &= \mathbb{E}_q \lbrack log(q(z)) - log(p(z, x)) \rbrack + \mathbb{E}_q \lbrack p(x) \rbrack
\end{split}
\end{equation}
$

As $p(x)$ is independant of $q(z)$, minimizing the KL divergence between $q$ and $p$ is equivalent to minimizing the first term.

We call evidence of lower bound the expectancy that we want to maximize.

$
\begin{equation}
\begin{split}
\mathrm{ELBO}(q(z)) &= \mathbb{E}_q \lbrack log~p(z,x) \rbrack - \mathbb{E}_q \lbrack log~q(z) \rbrack  \\
    &= \mathbb{E}_q \lbrack log~p(z, x) \rbrack - \mathrm{KL}(q(z) || p(z))
\end{split}
\end{equation}
$

The first represents the likelihood of the data whereas the second one represents the distance between the variational distribution and the prior.

Our objective can therefore be written as
$$ q^*(z) = \underset{q(z) \in \mathcal{Q}}{\text{arg max}}~ \mathbb{E}_q \lbrack log~p(z, x) \rbrack - \mathrm{KL}(q(z) || p(z)) $$

### Coordinate Ascent Variational Inference
The CAVI consists in iteratively modifying latent variables to reach an optimum of the ELBO. At each step, we keep all the variables constant, and update one of them in the direction that increases the objective.

Formally, we update the coordinate $j$ following
$$q_j^*(z_j)= exp\lbrace \mathbb{E}_{-j} \lbrack \log~ p(z_j, z_{-j},x) \rbrack  \rbrace $$

## A practical example: Gaussian Mixture Model

Let's consider a dataset of $n$ points.

The gaussian mixture model assumes the samples are drawn from a mixture of K gaussian distributions with mean $\mu_1 \dots \mu_K$ and variances $\sigma_1 \dots \sigma_K$.

Therefore, the latent variables of that model are the K means and variances as well as $n$ one-hot-vectors $c_i$ that indicate from which distribution the sample $x_i$ has been drawn from.

The variational density can then be written $q(\mu, c) = \displaystyle\prod_{k=1}^{K} q(\mu_k, m_k, s_k^2) \displaystyle\prod_{i=1}^{n}q(c_i, \phi_i)$

```python
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import seaborn as sns

from sklearn.mixture import GaussianMixture, BayesianGaussianMixture

sns.set()
```

```python
%matplotlib inline
```

```python
a = np.ones((5, 2))
a * [2, 3]
```

    array([[2., 3.],
           [2., 3.],
           [2., 3.],
           [2., 3.],
           [2., 3.]])

```python
def generate_data(n_points=1500, ratio=(0.5, 0.25, 0.25), mu_1=[3, 1], sigma1=[1, 0.5],
                 mu_2=[-2, 2], sigma2=[0.2, 0.2], mu_3=[1, 2], sigma3=[0.8, 2]):
    n1 = int(n_points * ratio[0])
    n2 = int(n_points * ratio[1])
    n3 = int(n_points * ratio[2])

    mu1 = np.array([mu_1]*n1)
    mu2 = np.array([mu_2]*n2)
    mu3 = np.array([mu_3]*n3)

    x1 = np.random.randn(n1, 2) * sigma1 + mu1
    y1 = np.ones((n1, 1))
    x2 = np.random.randn(n2, 2) * sigma2 + mu2
    y2 = 2 * np.ones((n2, 1))
    x3 = np.random.randn(n3, 2) * sigma3 + mu3
    y3 = 3 * np.ones((n3, 1))

    x1 = np.hstack((x1, y1))
    x2 = np.hstack((x2, y2))
    x3 = np.hstack((x3, y3))
    x = np.vstack((x1, x2, x3))
    np.random.shuffle(x)
    return x

x = generate_data()
print(x.shape)

```

    (1500, 3)

```python
plt.figure(figsize=(10, 10))
sns.scatterplot(x=x[:, 0], y=x[:, 1], hue=x[:, 2], palette='viridis')
```

```python
COLORS = ['magenta', 'darkorange', 'turquoise', 'green', 'red', 'indigo', 'aqua']

def make_ellipses(gmm, ax, n_components, colors=COLORS):
    colors = colors[:n_components]
    for n, color in enumerate(colors):
        if gmm.covariance_type == 'full':
            covariances = gmm.covariances_[n][:2, :2]
        elif gmm.covariance_type == 'tied':
            covariances = gmm.covariances_[:2, :2]
        elif gmm.covariance_type == 'diag':
            covariances = np.diag(gmm.covariances_[n][:2])
        elif gmm.covariance_type == 'spherical':
            covariances = np.eye(gmm.means_.shape[1]) * gmm.covariances_[n]
        v, w = np.linalg.eigh(covariances)
        u = w[0] / np.linalg.norm(w[0])
        angle = np.arctan2(u[1], u[0])
        angle = 180 * angle / np.pi  # convert to degrees
        v = 2. * np.sqrt(2.) * np.sqrt(v)
        ell = mpl.patches.Ellipse(gmm.means_[n, :2], v[0], v[1],
                                  180 + angle, color=color)
        ell.set_clip_box(ax.bbox)
        ell.set_alpha(0.25)
        ax.add_artist(ell)
        ax.set_aspect('equal', 'datalim')
```

```python
x = generate_data(n_points=1500, ratio=(0.5, 0.1, 0.4), mu_1=[5, 1], sigma1=[1, 0.5],
                 mu_2=[-3, 2], sigma2=[0.7, 0.2], mu_3=[1, 5], sigma3=[1, 2])

n_iter = np.linspace(1, 50, num=6, dtype=int)
COMPONENTS = 6
weight_prior = 1e-5
estimators = [
    BayesianGaussianMixture(n_components=COMPONENTS, tol=1e-7, max_iter=i, init_params='random',
                            weight_concentration_prior=weight_prior,
                            random_state=0) for i in n_iter]
plt.figure(figsize=(18, 20))
for i, estim in enumerate(estimators):
    estim.fit(x)
    ax = plt.subplot(5, 2, 1 + i)
    make_ellipses(estim, ax, n_components=estim.n_components)
    scatter = ax.scatter(x=x[:, 0], y=x[:, 1], c=x[:, 2], s=10, cmap='plasma', alpha=0.7)
    legend = ax.legend(*scatter.legend_elements(prop='colors'))
    plt.title('Confidence regions for {} iteration'.format(n_iter[i]))
```

## Stochastic Variational Inference

The main drawback of CAVi algorithm is that each iteration requires computing the gradient of the loss, which does not scale well for large datasets.

To fix this, stochastic VI aims at computing cheap unbiased estimates of the gradients at each iteration (same as in numerical optimization).
$$ \mathcal{L}_{SVI}(q) = \displaystyle\sum_{i=1}^{b} \mathbb{E}_q \lbrack \log p(z_ ...) \rbrack $$


# Gauss integration of Finite Elements

```python
from __future__ import division
from sympy.utilities.codegen import codegen
from sympy import *
from sympy import init_printing
from IPython.display import Image
init_printing()
```

```python
r, s, t, x, y, z = symbols('r s t x y z')
k, m, n = symbols('k m n', integer=True)
rho, nu, E = symbols('rho, nu, E')
```

## Predefinition

The constitutive model tensor in Voigt notation (plane strain) is
$$C = \frac{(1 - \nu) E}{(1 - 2\nu) (1 + \nu) }
\begin{pmatrix}
1 & \frac{\nu}{1-\nu} & 0\\
\frac{\nu}{1-\nu} & 1 & 0\\
0 & 0 & \frac{1 - 2\nu}{2(1 - \nu)}
\end{pmatrix}$$

But for simplicity we are going to use
$$\hat{C} = \frac{C (1 - 2\nu) (1 + \nu)}{E} =
\begin{pmatrix}
1-\nu & \nu & 0\\
\nu & 1-\nu & 0\\
0 & 0 & \frac{1 - 2\nu}{2}
\end{pmatrix} \enspace ,$$
since we can always multiply by that factor afterwards to obtain the correct stiffness matrices.

```python
C = Matrix([[1 - nu, nu, 0],
           [nu, 1 - nu, 0],
           [0, 0, (1 - 2*nu)/2]])
C_factor = E/(1-2*nu)/(1 + nu)
C
```

## Interpolation functions

The enumeration that we are using for the elements is shown below

```python
Image(filename='../img_src/4node_element_enumeration.png', width=300)
```

What leads to the following shape functions

```python
N = S(1)/4*Matrix([(1 + r)*(1 + s),
         (1 - r)*(1 + s),
         (1 - r)*(1 - s),
         (1 + r)*(1 - s)])
N
```

Thus, the interpolation matrix renders

```python
H = zeros(2,8)
for i in range(4):
    H[0, 2*i] = N[i]
    H[1, 2*i + 1] = N[i]

H.T  # Transpose of the interpolation matrix
```

And the mass matrix integrand is
$$M_\text{int}=H^TH$$

```python
M_int = H.T*H
```

## Derivatives interpolation matrix

```python
dHdr = zeros(2,4)
for i in range(4):
    dHdr[0,i] = diff(N[i],r)
    dHdr[1,i] = diff(N[i],s)

jaco = eye(2)  # Jacobian matrix, identity for now
dHdx = jaco*dHdr

B = zeros(3,8)
for i in range(4):
    B[0, 2*i] = dHdx[0, i]
    B[1, 2*i+1] = dHdx[1, i]
    B[2, 2*i] = dHdx[1, i]
    B[2, 2*i+1] = dHdx[0, i]

B
```

Being the stiffness matrix integrand
$$K_\text{int} = B^T C B$$

```python
K_int = B.T*C*B
```

## Analytic integration

The mass matrix is obtained integrating the product of the interpolator matrix with itself, i.e.
$$\begin{align*}
M &= \int\limits_{-1}^{1}\int\limits_{-1}^{1} M_\text{int} dr\, ds\\
 &= \int\limits_{-1}^{1}\int\limits_{-1}^{1} H^T H\, dr\, ds \enspace .
\end{align*}$$

```python
M = zeros(8,8)
for i in range(8):
    for j in range(8):
        M[i,j] = rho*integrate(M_int[i,j],(r,-1,1), (s,-1,1))

M
```

The stiffness matrix is obtained integrating the product of the interpolator-derivatives (displacement-to-strains) matrix with the constitutive tensor and itself, i.e.
$$\begin{align*}
K &= \int\limits_{-1}^{1}\int\limits_{-1}^{1} K_\text{int} dr\, ds\\
 &= \int\limits_{-1}^{1}\int\limits_{-1}^{1} B^T C\, B\, dr\, ds \enspace .
\end{align*}$$

```python
K = zeros(8,8)
for i in range(8):
    for j in range(8):
        K[i,j] = integrate(K_int[i,j], (r,-1,1), (s,-1,1))

K
```

We can generate automatically code for `Fortran`, `C` or `Octave/Matlab`, although it will be useful just for non-distorted elements.

```python
K_local = MatrixSymbol('K_local', 8, 8)
code = codegen(("local_stiff", Eq(K_local, simplify(K))), "f95")
print code[0][1]
```

    !******************************************************************************
    !*                      Code generated with sympy 0.7.6                       *
    !*                                                                            *
    !*              See http://www.sympy.org/ for more information.               *
    !*                                                                            *
    !*                       This file is part of 'project'                       *
    !******************************************************************************

    subroutine local_stiff(nu, K_local)
    implicit none
    REAL*8, intent(in) :: nu
    REAL*8, intent(out), dimension(1:8, 1:8) :: K_local

    K_local(1, 1) = -2.0d0/3.0d0*nu + 1.0d0/2.0d0
    K_local(2, 1) = 1.0d0/8.0d0
    K_local(3, 1) = (1.0d0/6.0d0)*nu - 1.0d0/4.0d0
    K_local(4, 1) = (1.0d0/2.0d0)*nu - 1.0d0/8.0d0
    K_local(5, 1) = (1.0d0/3.0d0)*nu - 1.0d0/4.0d0
    K_local(6, 1) = -1.0d0/8.0d0
    K_local(7, 1) = (1.0d0/6.0d0)*nu
    K_local(8, 1) = -1.0d0/2.0d0*nu + 1.0d0/8.0d0
    K_local(1, 2) = 1.0d0/8.0d0
    K_local(2, 2) = -2.0d0/3.0d0*nu + 1.0d0/2.0d0
    K_local(3, 2) = -1.0d0/2.0d0*nu + 1.0d0/8.0d0
    K_local(4, 2) = (1.0d0/6.0d0)*nu
    K_local(5, 2) = -1.0d0/8.0d0
    K_local(6, 2) = (1.0d0/3.0d0)*nu - 1.0d0/4.0d0
    K_local(7, 2) = (1.0d0/2.0d0)*nu - 1.0d0/8.0d0
    K_local(8, 2) = (1.0d0/6.0d0)*nu - 1.0d0/4.0d0
    K_local(1, 3) = (1.0d0/6.0d0)*nu - 1.0d0/4.0d0
    K_local(2, 3) = -1.0d0/2.0d0*nu + 1.0d0/8.0d0
    K_local(3, 3) = -2.0d0/3.0d0*nu + 1.0d0/2.0d0
    K_local(4, 3) = -1.0d0/8.0d0
    K_local(5, 3) = (1.0d0/6.0d0)*nu
    K_local(6, 3) = (1.0d0/2.0d0)*nu - 1.0d0/8.0d0
    K_local(7, 3) = (1.0d0/3.0d0)*nu - 1.0d0/4.0d0
    K_local(8, 3) = 1.0d0/8.0d0
    K_local(1, 4) = (1.0d0/2.0d0)*nu - 1.0d0/8.0d0
    K_local(2, 4) = (1.0d0/6.0d0)*nu
    K_local(3, 4) = -1.0d0/8.0d0
    K_local(4, 4) = -2.0d0/3.0d0*nu + 1.0d0/2.0d0
    K_local(5, 4) = -1.0d0/2.0d0*nu + 1.0d0/8.0d0
    K_local(6, 4) = (1.0d0/6.0d0)*nu - 1.0d0/4.0d0
    K_local(7, 4) = 1.0d0/8.0d0
    K_local(8, 4) = (1.0d0/3.0d0)*nu - 1.0d0/4.0d0
    K_local(1, 5) = (1.0d0/3.0d0)*nu - 1.0d0/4.0d0
    K_local(2, 5) = -1.0d0/8.0d0
    K_local(3, 5) = (1.0d0/6.0d0)*nu
    K_local(4, 5) = -1.0d0/2.0d0*nu + 1.0d0/8.0d0
    K_local(5, 5) = -2.0d0/3.0d0*nu + 1.0d0/2.0d0
    K_local(6, 5) = 1.0d0/8.0d0
    K_local(7, 5) = (1.0d0/6.0d0)*nu - 1.0d0/4.0d0
    K_local(8, 5) = (1.0d0/2.0d0)*nu - 1.0d0/8.0d0
    K_local(1, 6) = -1.0d0/8.0d0
    K_local(2, 6) = (1.0d0/3.0d0)*nu - 1.0d0/4.0d0
    K_local(3, 6) = (1.0d0/2.0d0)*nu - 1.0d0/8.0d0
    K_local(4, 6) = (1.0d0/6.0d0)*nu - 1.0d0/4.0d0
    K_local(5, 6) = 1.0d0/8.0d0
    K_local(6, 6) = -2.0d0/3.0d0*nu + 1.0d0/2.0d0
    K_local(7, 6) = -1.0d0/2.0d0*nu + 1.0d0/8.0d0
    K_local(8, 6) = (1.0d0/6.0d0)*nu
    K_local(1, 7) = (1.0d0/6.0d0)*nu
    K_local(2, 7) = (1.0d0/2.0d0)*nu - 1.0d0/8.0d0
    K_local(3, 7) = (1.0d0/3.0d0)*nu - 1.0d0/4.0d0
    K_local(4, 7) = 1.0d0/8.0d0
    K_local(5, 7) = (1.0d0/6.0d0)*nu - 1.0d0/4.0d0
    K_local(6, 7) = -1.0d0/2.0d0*nu + 1.0d0/8.0d0
    K_local(7, 7) = -2.0d0/3.0d0*nu + 1.0d0/2.0d0
    K_local(8, 7) = -1.0d0/8.0d0
    K_local(1, 8) = -1.0d0/2.0d0*nu + 1.0d0/8.0d0
    K_local(2, 8) = (1.0d0/6.0d0)*nu - 1.0d0/4.0d0
    K_local(3, 8) = 1.0d0/8.0d0
    K_local(4, 8) = (1.0d0/3.0d0)*nu - 1.0d0/4.0d0
    K_local(5, 8) = (1.0d0/2.0d0)*nu - 1.0d0/8.0d0
    K_local(6, 8) = (1.0d0/6.0d0)*nu
    K_local(7, 8) = -1.0d0/8.0d0
    K_local(8, 8) = -2.0d0/3.0d0*nu + 1.0d0/2.0d0

    end subroutine

```python
code = codegen(("local_stiff", Eq(K_local, simplify(K))), "C")
print code[0][1]
```

    /******************************************************************************
     *                      Code generated with sympy 0.7.6                       *
     *                                                                            *
     *              See http://www.sympy.org/ for more information.               *
     *                                                                            *
     *                       This file is part of 'project'                       *
     ******************************************************************************/
    #include "local_stiff.h"
    #include <math.h>

    void local_stiff(double nu, double *K_local) {

       K_local[0] = -2.0L/3.0L*nu + 1.0L/2.0L;
       K_local[1] = 1.0L/8.0L;
       K_local[2] = (1.0L/6.0L)*nu - 1.0L/4.0L;
       K_local[3] = (1.0L/2.0L)*nu - 1.0L/8.0L;
       K_local[4] = (1.0L/3.0L)*nu - 1.0L/4.0L;
       K_local[5] = -1.0L/8.0L;
       K_local[6] = (1.0L/6.0L)*nu;
       K_local[7] = -1.0L/2.0L*nu + 1.0L/8.0L;
       K_local[8] = 1.0L/8.0L;
       K_local[9] = -2.0L/3.0L*nu + 1.0L/2.0L;
       K_local[10] = -1.0L/2.0L*nu + 1.0L/8.0L;
       K_local[11] = (1.0L/6.0L)*nu;
       K_local[12] = -1.0L/8.0L;
       K_local[13] = (1.0L/3.0L)*nu - 1.0L/4.0L;
       K_local[14] = (1.0L/2.0L)*nu - 1.0L/8.0L;
       K_local[15] = (1.0L/6.0L)*nu - 1.0L/4.0L;
       K_local[16] = (1.0L/6.0L)*nu - 1.0L/4.0L;
       K_local[17] = -1.0L/2.0L*nu + 1.0L/8.0L;
       K_local[18] = -2.0L/3.0L*nu + 1.0L/2.0L;
       K_local[19] = -1.0L/8.0L;
       K_local[20] = (1.0L/6.0L)*nu;
       K_local[21] = (1.0L/2.0L)*nu - 1.0L/8.0L;
       K_local[22] = (1.0L/3.0L)*nu - 1.0L/4.0L;
       K_local[23] = 1.0L/8.0L;
       K_local[24] = (1.0L/2.0L)*nu - 1.0L/8.0L;
       K_local[25] = (1.0L/6.0L)*nu;
       K_local[26] = -1.0L/8.0L;
       K_local[27] = -2.0L/3.0L*nu + 1.0L/2.0L;
       K_local[28] = -1.0L/2.0L*nu + 1.0L/8.0L;
       K_local[29] = (1.0L/6.0L)*nu - 1.0L/4.0L;
       K_local[30] = 1.0L/8.0L;
       K_local[31] = (1.0L/3.0L)*nu - 1.0L/4.0L;
       K_local[32] = (1.0L/3.0L)*nu - 1.0L/4.0L;
       K_local[33] = -1.0L/8.0L;
       K_local[34] = (1.0L/6.0L)*nu;
       K_local[35] = -1.0L/2.0L*nu + 1.0L/8.0L;
       K_local[36] = -2.0L/3.0L*nu + 1.0L/2.0L;
       K_local[37] = 1.0L/8.0L;
       K_local[38] = (1.0L/6.0L)*nu - 1.0L/4.0L;
       K_local[39] = (1.0L/2.0L)*nu - 1.0L/8.0L;
       K_local[40] = -1.0L/8.0L;
       K_local[41] = (1.0L/3.0L)*nu - 1.0L/4.0L;
       K_local[42] = (1.0L/2.0L)*nu - 1.0L/8.0L;
       K_local[43] = (1.0L/6.0L)*nu - 1.0L/4.0L;
       K_local[44] = 1.0L/8.0L;
       K_local[45] = -2.0L/3.0L*nu + 1.0L/2.0L;
       K_local[46] = -1.0L/2.0L*nu + 1.0L/8.0L;
       K_local[47] = (1.0L/6.0L)*nu;
       K_local[48] = (1.0L/6.0L)*nu;
       K_local[49] = (1.0L/2.0L)*nu - 1.0L/8.0L;
       K_local[50] = (1.0L/3.0L)*nu - 1.0L/4.0L;
       K_local[51] = 1.0L/8.0L;
       K_local[52] = (1.0L/6.0L)*nu - 1.0L/4.0L;
       K_local[53] = -1.0L/2.0L*nu + 1.0L/8.0L;
       K_local[54] = -2.0L/3.0L*nu + 1.0L/2.0L;
       K_local[55] = -1.0L/8.0L;
       K_local[56] = -1.0L/2.0L*nu + 1.0L/8.0L;
       K_local[57] = (1.0L/6.0L)*nu - 1.0L/4.0L;
       K_local[58] = 1.0L/8.0L;
       K_local[59] = (1.0L/3.0L)*nu - 1.0L/4.0L;
       K_local[60] = (1.0L/2.0L)*nu - 1.0L/8.0L;
       K_local[61] = (1.0L/6.0L)*nu;
       K_local[62] = -1.0L/8.0L;
       K_local[63] = -2.0L/3.0L*nu + 1.0L/2.0L;

    }

```python
code = codegen(("local_stiff", Eq(K_local, simplify(K))), "Octave")
print code[0][1]
```

    function K_local = local_stiff(nu)
      %LOCAL_STIFF  Autogenerated by sympy
      %   Code generated with sympy 0.7.6
      %
      %   See http://www.sympy.org/ for more information.
      %
      %   This file is part of 'project'

      K_local = [-2*nu/3 + 1/2           1/8    nu/6 - 1/4    nu/2 - 1/8    nu/3 - 1/4          -1/8          nu/6   -nu/2 + 1/8;
      1/8 -2*nu/3 + 1/2   -nu/2 + 1/8          nu/6          -1/8    nu/3 - 1/4    nu/2 - 1/8    nu/6 - 1/4;
      nu/6 - 1/4   -nu/2 + 1/8 -2*nu/3 + 1/2          -1/8          nu/6    nu/2 - 1/8    nu/3 - 1/4           1/8;
      nu/2 - 1/8          nu/6          -1/8 -2*nu/3 + 1/2   -nu/2 + 1/8    nu/6 - 1/4           1/8    nu/3 - 1/4;
      nu/3 - 1/4          -1/8          nu/6   -nu/2 + 1/8 -2*nu/3 + 1/2           1/8    nu/6 - 1/4    nu/2 - 1/8;
      -1/8    nu/3 - 1/4    nu/2 - 1/8    nu/6 - 1/4           1/8 -2*nu/3 + 1/2   -nu/2 + 1/8          nu/6;
      nu/6    nu/2 - 1/8    nu/3 - 1/4           1/8    nu/6 - 1/4   -nu/2 + 1/8 -2*nu/3 + 1/2          -1/8;
      -nu/2 + 1/8    nu/6 - 1/4           1/8    nu/3 - 1/4    nu/2 - 1/8          nu/6          -1/8 -2*nu/3 + 1/2];

    end

We can check some numerical vales for $E=8/3$ Pa, $\nu=1/3$ and $\rho=1$ kg\m$^3$, where we can multiply by the factor that we took away from the stiffness tensor

```python
(C_factor*K).subs([(E, S(8)/3), (nu, S(1)/3)])
```

```python
M.subs(rho, 1)
```

## Gauss integration

As stated before, the analytic expressions for the mass and stiffness matrices is useful for non-distorted elements. In the general case, a mapping between distorted elements and these _canonical_ elements is used to simplify the integration domain. When this transformation is done, the functions to be integrated are more convoluted and we should use numerical integration like _Gauss-Legendre quadrature_.

The Gauss-Legendre quadrature approximates the integral:
$$     \int_{-1}^1 f(x)\,dx \approx \sum_{i=1}^n w_i f(x_i)$$

The nodes $x_i$ of an order $n$ quadrature rule are the roots of $P_n$
and the weights $w_i$ are given by:
$$w_i = \frac{2}{\left(1-x_i^2\right) \left(P'_n(x_i)\right)^2}$$

For the first four orders, the weights and nodes are

```python
wts = [[2], [1,1], [S(5)/9, S(8)/9, S(5)/9],
       [(18+sqrt(30))/36,(18+sqrt(30))/36, (18-sqrt(30))/36, (18-sqrt(30))/36]
       ]

pts = [[0], [-sqrt(S(1)/3), sqrt(S(1)/3)],
       [-sqrt(S(3)/5), 0, sqrt(S(3)/5)],
[-sqrt(S(3)/7 - S(2)/7*sqrt(S(6)/5)), sqrt(S(3)/7 - S(2)/7*sqrt(S(6)/5)),
-sqrt(S(3)/7 + S(2)/7*sqrt(S(6)/5)), sqrt(S(3)/7 + S(2)/7*sqrt(S(6)/5))]]
```

And the numerical integral is computed as

```python
def stiff_num(n):
    """Compute the stiffness matrix using Gauss quadrature

    Parameters
    ----------
    n : int
        Order of the polynomial.

    """
    if n>4:
        raise Exception("Number of points not valid")
    K_num = zeros(8,8)
    for x_i, w_i in zip(pts[n-1], wts[n-1]):
        for y_j, w_j in zip(pts[n-1], wts[n-1]):
            K_num = K_num + w_i*w_j*K_int.subs([(r,x_i), (s,y_j)])

    return simplify(K_num)

```

```python
K_num = stiff_num(3)
K_num - K
```

### Best approach

A best approach is to use Python built-in functions for computing the Gauss-Legendre nodes and weights

```python
from sympy.integrals.quadrature import gauss_legendre
```

```python
x, w = gauss_legendre(5, 15)
print x
print w
```

    [-0.906179845938664, -0.538469310105683, 0, 0.538469310105683, 0.906179845938664]
    [0.236926885056189, 0.478628670499366, 0.568888888888889, 0.478628670499366, 0.236926885056189]

## References

[1] http://en.wikipedia.org/wiki/Gaussian_quadrature

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('../styles/custom_barba.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>
/* Based on Lorena Barba template available at: https://github.com/barbagroup/AeroPython/blob/master/styles/custom.css*/
@font-face {
font-family: "Computer Modern";
src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}
div.cell{
width:800px;
margin-left:16% !important;
margin-right:auto;
}
h1 {
font-family: 'Alegreya Sans', sans-serif;
}
h2 {
font-family: 'Fenix', serif;
}
h3{
font-family: 'Fenix', serif;
margin-top:12px;
margin-bottom: 3px;
}
h4{
font-family: 'Fenix', serif;
}
h5 {
font-family: 'Alegreya Sans', sans-serif;
}
div.text_cell_render{
font-family: 'Alegreya Sans',Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
line-height: 135%;
font-size: 120%;
width:600px;
margin-left:auto;
margin-right:auto;
}
.CodeMirror{
font-family: "Source Code Pro";
font-size: 90%;
}
/* .prompt{
display: None;
}*/
.text_cell_render h1 {
font-weight: 200;
font-size: 50pt;
line-height: 100%;
color:#CD2305;
margin-bottom: 0.5em;
margin-top: 0.5em;
display: block;
}
.text_cell_render h5 {
font-weight: 300;
font-size: 16pt;
color: #CD2305;
font-style: italic;
margin-bottom: .5em;
margin-top: 0.5em;
display: block;
}
.warning{
color: rgb( 240, 20, 20 )
}
</style>


# Messtechnik HS2021 - Tutorial 4

## Aufgabe 1: Diskrete Fourier Transformation und Datenverarbeitung
---------
Ein Kohlenstoff NMR Signal ($^{13}{\rm C}$) wurde aufgenommen und mit einer Abtastrate von 15 kHz digitalisiert. Das komplexwertige FID wurde durch ein MATLAB Program digitalisiert, exportiert und ist in `fid.mat` zu finden.

---------

### 1a)
Generieren Sie aus dem komplexwertigen FID das dazugehörige Spektrum und plotten Sie dessen Realteil mit der korrekten Frequenzachse in kHz.

*Hinweis:* Um MATLAB Dateien in Python zu importieren, kann man die ``scipy.io.loadmat`` Funktion brauchen:

````
from scipy.io import loadmat
data = loadmat('fid.mat')
fid = data['fid']
fid = np.squeeze(fid)
````

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy.io import loadmat
from numpy.fft import fft, fftshift, fftfreq

# Importieren der MATLAB Daten
data = loadmat('fid.mat')
fid = data['fid']
fid = np.squeeze(fid)

# Abtastrate und Parameter
fs = 15   # kHz
dt = 1/fs # ms
N = len(fid)

# Erstellen der Zeitachse
tmax = N*dt
t = np.linspace(0,tmax,N) #ms

# Frequenzachse
frq = fftfreq(N,d=dt) # kHz
frq = fftshift(frq)   # kHz

# Spektrum
spc = fftshift(fft(fid))

# Plot
plt.figure(figsize=(16,6))
plt.subplot(121)
plt.plot(t,np.real(fid),'k')
plt.xlim((0,np.amax(t)))
plt.xlabel('T [ms]',fontsize=13)
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.title('FID',fontsize=13)

plt.subplot(122)
plt.plot(frq,np.real(spc),'k')
plt.xlim((-1,2))
plt.xlabel('$\\nu$ [kHz]',fontsize=13)
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.title('FFT',fontsize=13);
```

### 1b)
Durch einen Matched Filter lässt sich das Signal-zu-Rausch-Verhältnis (SNR) nachträglich verbessern. Das Signal kann beschrieben werden als
$$ y(t) = \exp(-i\omega_0 t) \exp(-\frac{t}{T_2}) $$
wobei $\omega_0$ die Larmorfrequenz ist und $T_2$ die transversale Relaxationszeit. Bei einem Matched Filter wird das Zeitsignal $ y(t) $ mit der Zerfallsfunktion $ y_e(t) = \exp(-\frac{t}{T_2}) $ multipliziert.
Zeigen Sie die Auswirkungen auf das Spektrum bei der Anwendung von Matched Filtern mit Zerfallszeiten $T_2 = [1,2,5,10,50,100]$.

```python
import warnings
warnings.filterwarnings("ignore")

T2       = [[],100,50,10,5,2,1]
colors   = ['black','indigo','purple','blue','dodgerblue','mediumturquoise','forestgreen']
leg_text = ['exp. data','100 ms','50 ms','10 ms','5 ms','2 ms','1 ms']

plt.figure(figsize=(16,6))

for k in range(len(T2)):

    # Matched Filter
    if k == 0:
        envelope = np.ones_like(fid)
    else:
        envelope = np.exp(-(t/T2[k]))

    # Multiplikation mit Filter
    currsig = fid*envelope
    currspc = fftshift(fft(currsig))

    # Plot
    plt.subplot(121)
    if k == 0:
        plt.plot(t,np.real(currsig)/np.amax(np.real(currsig)),color=colors[k])
    else:
        plt.plot(t,envelope,color=colors[k])
    plt.xlim((0,np.amax(t)))
    plt.xlabel('T [ms]',fontsize=13)
    plt.ylabel('Intensität [a.u.]',fontsize=13)
    plt.title('FID',fontsize=13)

    plt.subplot(122)
    plt.plot(frq,np.real(currspc),color=colors[k])
    plt.xlim((-1,2))
    plt.xlabel('$\\nu$ [kHz]',fontsize=13)
    plt.ylabel('Intensität [a.u.]',fontsize=13)
    plt.legend(leg_text,fontsize=13)
    plt.title('FFT',fontsize=13);
```

### 1c)
Finden Sie eine passende Zerfallszeit für den Matched Filter, sodass Sie eine gute Balance zwischen Auflösung und SNR erhalten.

```python
T2 = 25 #ms

envelope = np.exp(-(t/T2))
matchedsig = fid*envelope
matchedspc = fftshift(fft(matchedsig))

plt.figure(figsize=(16,6))
plt.subplot(121)
plt.plot(t,np.real(fid)/np.amax(np.real(fid)),'k')
plt.plot(t,envelope,color='dodgerblue')
plt.xlim((0,np.amax(t)))
plt.xlabel('T [ms]',fontsize=13)
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.title('FID',fontsize=13)

plt.subplot(122)
plt.plot(frq,np.real(spc),'k')
plt.plot(frq,np.real(matchedspc),color='dodgerblue')
plt.xlim((-2,2))
plt.xlabel('$\\nu$ [kHz]',fontsize=13)
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.title('FFT',fontsize=13);
```

### 1d)
Nehmen Sie nun an die Aufnahmezeit des FID's ist beschränkt auf 20 ms. Erstellen Sie aus `fid.mat` das entsprechende abgeschnittene Signal, hängen Sie dann Nullen an (Zerofilling), sodass Sie am Ende ein Signal mit insgesamt 2048 Punkte erhalten. Was fällt Ihnen auf, wenn Sie die Spektren des abgeschnittenen und des originalen Signal mit dem vollständigen vergleichen?

```python
# Truncation
end    = 20 # ms
fidcut = fid[t<=end]
tcut   = t[t<=end]

# Zerofilling
Nzf   = 2048
zf    = Nzf-len(fidcut)
tzf   = t[0:Nzf]
fidzf = np.pad(fidcut,(0,zf))

# Spektrum
frqzf = fftshift(fftfreq(len(tzf),d=dt))
spczf = fftshift(fft(fidzf))

plt.figure(figsize=(16,6))
plt.subplot(121)
plt.plot(t,np.real(fid),'k')
plt.plot(tcut,np.real(fidcut),color='dodgerblue')
plt.xlim((0,np.amax(t)))
plt.xlabel('T [ms]',fontsize=13)
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.title('FID',fontsize=13)

plt.subplot(122)
plt.plot(frq,np.real(spc),'k')
plt.plot(frqzf,np.real(spczf),color='dodgerblue')
plt.xlim((-1,2))
plt.xlabel('$\\nu$ [kHz]',fontsize=13)
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.title('FFT',fontsize=13)
plt.legend(['exp. data','truncated,zerofilled'],fontsize=13);

```

### 1e)
Programmieren Sie ein Hamming Apodisierungsfenster

$$ w_{\text{hamming}}(t) = 0.54 + 0.46 \cos(\frac{\pi t}{t_{\text{max}}})$$

für das bei 60 ms abgeschnittene FID. Wenden Sie das Hamming Window auf das abgeschnittene Signal an, zerofillen Sie auf 2048 Punkte und vergleichen Sie die Spektren mit und ohne Hamming Window.

```python
import math as m

tmax    = tcut[len(tcut)-1]
hamming = 0.54 + 0.46*np.cos(m.pi*tcut/tmax)

# Apodisierung
fidhamming = fidcut*hamming

# Spektrum
frqhamming = fftshift(fftfreq(Nzf,d=dt))
spchamming = fftshift(fft(fidhamming,Nzf))

# Plot
plt.figure(figsize=(16,6))
plt.subplot(121)
plt.plot(tcut,np.real(fidcut)/np.amax(np.real(fidcut)),color='black')
plt.plot(tcut,hamming,'--',color='dodgerblue')
plt.xlim((0,np.amax(tcut)))
plt.xlabel('T [ms]',fontsize=13)
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.title('FID',fontsize=13)

plt.subplot(122)
plt.plot(frqzf,np.real(spczf),color='black')
plt.plot(frqhamming,np.real(spchamming),color='dodgerblue')
plt.xlim((-2,2))
plt.xlabel('$\\nu$ [kHz]',fontsize=13)
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.legend(['truncated','truncated,Hamming'],fontsize=13)
plt.title('FFT',fontsize=13);
```

### 1f)
Zeigen Sie, dass ein Hamming Fenster die Auflösung im Frequenzbereich im Vergleich zu einem "hart" abgeschnittenen FID verschlechtert, jedoch die *ripples* besser unterdrückt. Verwenden Sie Zerofilling au 2048 Punkte. Vergleichen Sie dazu die entsprechende Faltungsfunktion im Frequenzbereich.

```python
# Hamming Fenster
tmax    = tcut[len(tcut)-1]
hamming = 0.54 + 0.46*np.cos(m.pi*tcut/tmax)

# Rechteckiges Fenster
rect    = np.ones_like(tcut)

# Spektrum
spchamming = fftshift(fft(hamming,Nzf))
spcrect    = fftshift(fft(rect,Nzf))

plt.figure(figsize=[8,6])
plt.plot(frqzf,np.real(spcrect)/max(np.real(spcrect)),'k');
plt.plot(frqzf,np.real(spchamming)/max(np.real(spchamming)),'r')
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.xlabel('$\\nu$ [kHz]',fontsize=13)
plt.xlim([-0.25, 0.25])
plt.legend(['Rectangular','Hamming'],fontsize=13)
```

## Aufgabe 2: Digitalisierung eines Zeitsignals
---------
Ein zeitlich abfallendes oszillierendes Signal hat die Form:
$\begin{equation}
	x(t) =
	\begin{cases}
		 \exp \left( - \frac{t}{T_2} \right) \left\{ \cos(2\pi \nu_1 t) + 3\cos(2\pi \nu_2 t) \right\} & t\geq 0 \\
	 	0 & t < 0
	 \end{cases}
\end{equation}$
mit den beiden Frequenzkomponenten $ \nu_1 =10\,{\rm Hz}$ und $\nu_2 = 150\,{\rm Hz}$ sowie der Zerfallszeit $T_2  = 0.5\,{\rm s}$. Das Signal wird total durch 2048 Punkte aufgenommen und digitalisiert.

---------

### 2a)
Welche Abtastrate schlagen Sie für das Digitalisieren dieses Signals vor? Begründen Sie Ihre Antwort mit Hilfe des Abtasttheorems.
Berechnen Sie anhand der vorgeschlagenen Abtastrate das Zeitsignal und das dazugehörige Spektrum.

<u>Lösung:</u>

Das Nyquist-Theorem besagt, dass die Abtastrate mehr als zwei mal so gross wie die höchste Frequenzkomponente des zu abtastenden Signals. Die höchste Frequenzkomponente des Zeitsignals is $\nu_2 =  150\,{\rm Hz}$, was bedeutet:
$$ f_s > 300\,{\rm Hz} $$
Der exponentielle Zerfall mit der Zerfallzeit $T_2 = 0.5\,{\rm s}$ führt allerdings zusätzlich zu einer Linienverbreiterung von $2\,{\rm Hz}$. Für die korrekte Abtastrate muss das auch miteinberechnet werden und wir verwenden:
$$ f_s > 2 (150 + 5\cdot 2)\,{\rm Hz} = 320\,{\rm Hz}$$

```python
# Parameter
nu1 = 10
nu2 = 150
T2  = 0.5

# Abtastrate
fs   = 320
dt   = 1/fs

# Zeitachse und Signal
N      = 2048
t      = np.linspace(0,(N-1)*dt,N)
signal = (np.cos(2*m.pi*nu1*t)+3*np.cos(2*m.pi*nu2*t))*np.exp(-t/T2)

# Frequenzachse und Spektrum
frq = fftshift(fftfreq(N,d=dt))
spc = fftshift(fft(signal))

# Plot
plt.figure(figsize=(16,6))
plt.subplot(121)
plt.plot(t,np.real(signal),color='black')
plt.xlim((0,5))
plt.xlabel('T [s]',fontsize=13)
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.title('FID',fontsize=13)

plt.subplot(122)
plt.plot(frq,np.real(spc),color='black')
plt.xlim((0,np.amax(frq)))
plt.xlabel('$\\nu$ [Hz]',fontsize=13)
plt.ylabel('Intensität [a.u.]',fontsize=13)
plt.title('FFT',fontsize=13);
```

### 2b)
Vergleichen Sie unterschiedlichen Abtastraten $f_s = 200,250,320,1000\,{\rm Hz}$ für das obengenannte Signal . Was für Erkenntnisse können Sie aus den entsprechenden Spektren gewinnen?

```python
# Parameter
nu1 = 10
nu2 = 150
T2  = 0.5

# Abtastrate
fs   = [200,250,320,1000]

# Plot-Parameter
plt.figure(figsize=(8,6))
color = ['indigo','blue','dodgerblue','forestgreen']
legtex= ['$f_s = 200$ Hz','$f_s = 250$ Hz','$f_s = 320$ Hz','$f_s = 1000$ Hz']

for k in range(len(fs)):
    dt   = 1/fs[k]

    # Zeitachse und Signal
    N      = 2048
    t      = np.linspace(0,(N-1)*dt,N)
    signal = (np.cos(2*m.pi*nu1*t)+3*np.cos(2*m.pi*nu2*t))*np.exp(-t/T2)

    # Frequenzachse und Spektrum
    frq = fftshift(fftfreq(N,d=dt))
    spc = fftshift(fft(signal))

    # Plot
    plt.plot(frq,np.real(spc),color=color[k])
    plt.xlim((0,160))
    plt.xlabel('$\\nu$ [Hz]',fontsize=13)
    plt.ylabel('Intensität [a.u.]',fontsize=13)
    plt.legend(legtex,fontsize=13)
    plt.title('FFT',fontsize=13);
```


# Statistical Decision Making
## Task 1

Consider the two-dimensional, discrete random variable $X = [X_1\ X_2]^\top$ subjected to the joint probability density $p_X$ as described in the following table.

$\begin{array}{c|cc} p_X(X_1, X_2)  & X_2 = 0 & X_2 = 1 \\ \hline\hline X_1 = 0 & 0.4 & 0.3 \\ X_1 = 1 & 0.2 & 0.1\end{array}$

a) Compute the marginal probability densities $p_{X1}, p_{X2}$ and the conditional probability $P(X_2 = 0|X_1 = 0)$ as well as the expected value $\mathbb{E}[X]$ and the covariance matrix $\mathbb{E}[(X - \mathbb{E}[X])(X - \mathbb{E}[X])^\top]$ by hand.

```python
import numpy as np

# joint probability table
p_table = np.array([[0.4, 0.3], [0.2, 0.1]])

# each column of X contains a possible event. 1st row corresponds to x1,
# 2nd row corresponds to x2
#X = [[0 1 0 1],
#     [0 0 1 1]]
X = np.array([[0, 1, 0, 1],
              [0, 0, 1, 1]])

# p_table.ravel('F') = [0.4  0.2  0.3  0.1] are the joint probability values that
# correspond to these columns

# marginal probabilities: sum accross rows
p_x1 = np.sum(p_table, axis=1)
p_x2 = np.sum(p_table, axis=0)

print('Marginal probability: [p(x1=0), p(x1=1)] = {}'.format(p_x1))
print('Marginal probability: [p(x2=0), p(x2=1)] = {}'.format(p_x2))

# conditional  p(x2 = 0 | x1 = 0)
# p(A|B) = p(A \intersect B) / p(B)
p_x2equals0condx1equals0 = p_table[0,0] / p_x1[0]
print('p(x2=0 | x1=0) = {}'.format(p_x2equals0condx1equals0))

# expected value
E_X = np.dot(X, p_table.ravel('F')) # ravel('F') while keeping column order
print('Expected value: {}'.format(E_X))

# covariance matrix
X_centered = X - np.expand_dims(E_X, axis=1) # expand_dims is needed that numpy is able to subtract the vector from the matrix
CovX = np.dot(np.dot(X_centered, np.diag(p_table.ravel('F'))), X_centered.T)
print('Covariance matrix:\n {}'.format(CovX))
```

    Marginal probability: [p(x1=0), p(x1=1)] = [ 0.7  0.3]
    Marginal probability: [p(x2=0), p(x2=1)] = [ 0.6  0.4]
    p(x2=0 | x1=0) = 0.5714285714285715
    Expected value: [ 0.3  0.4]
    Covariance matrix:
     [[ 0.21 -0.02]
     [-0.02  0.24]]

b) Write a PYTHON function `toyrnd` that expects the positive integer parameter `n` as its input and returns a matrix `X` of size `[2,n]`, containing `n` samples drawn independently from the distribution $p_X$, as its output.

```python
def toyrnd(n):
    X_out = np.zeros((2,n))
    Q = np.zeros((n,))
    T = np.random.rand(n)

    # Interpreting [x1, x2] as binary number and Q as its decimal representation
    Q[T>0.4] = 1
    Q[T>0.7] = 2
    Q[T>0.9] = 3

    X_out[0] = Q // 2 # floor division
    X_out[1] = Q % 2 # modulus division

    return X_out
```

c) Verify your results in a) by generating `10000` samples with `toyrnd` and computing the respective empirical values.

```python
N = 100000
X_observed = toyrnd(N)
# marginal probabilities
p_X1equals0_empirical = np.float(np.sum(X_observed[0,:]==0))/N
p_X1equals1_empirical = np.float(np.sum(X_observed[0,:]==1))/N
p_X2equals0_empirical = np.float(np.sum(X_observed[1,:]==0))/N
p_X2equals1_empirical = np.float(np.sum(X_observed[1,:]==1))/N

p_x1_empirical = np.array([p_X1equals0_empirical, p_X1equals1_empirical])
p_x2_empirical = np.array([p_X2equals0_empirical, p_X2equals1_empirical])

print('Empirical marginal probability: [p(x1=0), p(x1=1)] = {}'.format(p_x1_empirical))
print('Empirical marginal probability: [p(x2=0), p(x2=1)] = {}'.format(p_x2_empirical))

# conditional probability
X2condX1 = X_observed[1, X_observed[0,:]==0]
P_X2equals0condX1eqzals0_empirical = np.float(np.sum(X2condX1 == 0)) / len(X2condX1)
print('Empirical conditional probability P(x2=0|x1=0):', P_X2equals0condX1eqzals0_empirical)
# expected value
E_X_empirical = np.sum(X_observed, axis=1)/N
print('Empirical expected value: {}'.format(E_X_empirical))

# covariance matrix
CovX_empirical = np.dot(X_observed - np.expand_dims(E_X_empirical, axis=1), (X_observed - np.expand_dims(E_X_empirical, axis=1)).T) / N
print('Empirical covariance matrix:\n {}'.format(CovX_empirical))
```

    Empirical marginal probability: [p(x1=0), p(x1=1)] = [ 0.70282  0.29718]
    Empirical marginal probability: [p(x2=0), p(x2=1)] = [ 0.60093  0.39907]
    Empirical conditional probability P(x2=0|x1=0): 0.5719103042030677
    Empirical expected value: [ 0.29718  0.39907]
    Empirical covariance matrix:
     [[ 0.20886405 -0.02039562]
     [-0.02039562  0.23981314]]

## Task 2

The MNIST training set consists of handwritten digits from 0 to 9, stored as PNG files of size $28 \times 28$ and indexed by label. Download the provided ZIP file from Moodle and make yourself familiar with the directory structure.

a) Grayscale images are typically described as matrices of `uint8` values. For numerical calculations, it is more sensible to work with floating point numbers. Load two (abitrary) images from the database and convert them to matrices `I1` and `I2` of `float64` values in the interval $[0, 1]$.

```python
from scipy import misc

# define to image paths which to import
data_folder = './mnist/'

impath1 = data_folder + 'd2/d2_0075.png'
impath2 = data_folder + 'd3/d3_0013.png'

# import and convert to numpy array
I1 = np.array(misc.imread(impath1)).astype(np.float64)
I2 = np.array(misc.imread(impath2)).astype(np.float64)

# check values
print('First image min/max value: {}/{}'.format(np.min(I1), np.max(I1)))

# normalize values to [0,1]
I1 = I1 / 255.
I2 = I2 / 255.
```

    First image min/max value: 0.0/255.0

b) The matrix equivalent of the euclidean norm $\|\cdot\|_2$ is the Frobenius norm. For any matrix $\mathbf{A} \in \mathbb{R}^{m \times n}$, it is defined as
			\begin{equation}
			    \|\mathbf{A}\|_F = \sqrt{\mathrm{tr}(\mathbf{A}^\top \mathbf{A})},
			\end{equation}
			where $\mathrm{tr}$ denotes the trace of a matrix. Compute the distance $\|\mathbf{I}_1 - \mathbf{I}_2\|_F$ between the images `I1` and `I2` by using three different procedures in PYTHON:
-  Running the `numpy.linalg.norm` function with the `'fro'` parameter
-  Directly applying the above equation
-  Computing the euclidean norm between the vectorized images

```python
# using frobenius norm
frob1 = np.linalg.norm(I1 - I2, 'fro')

print('Numpy Frobenius norm: {}'.format(frob1))

# using formula
frob2 = np.sqrt(np.trace(np.dot(I1 - I2, (I1 - I2).T)))

print('Implemented Frobenius norm: {}'.format(frob2))

# using euclidean norm of vectorized images
frob3 = np.sqrt(np.dot((I1 - I2).ravel(), (I1 - I2).ravel()))
print('Euclidean norm of vectorized images: {}'.format(frob3))
```

    Numpy Frobenius norm: 11.71589679029833
    Implemented Frobenius norm: 11.715896790298329
    Euclidean norm of vectorized images: 11.71589679029833

c) In the following, we want to solve a simple classification problem by applying *$k$-Nearest Neighbours*. To this end, choose two digit classes, e.g. $0$ and $1$, and load `n_train = 500` images from each class to the workspace. Convert them according to subtask a) and store them in vectorized form in the matrix `X_train` of size `[784, 2*n_train]`. Provide an indicator vector `Y_train` of length `2*n_train` that assigns the respective digit class label to each column of `X_train`.

From each of the two classes, choose another set of `n_test=10` images and create the according matrices `X_test` and `Y_test`. Now, for each sample in the test set, determine the `k = 20` training samples with the smallest Frobenius distance to it and store their indices in the `2*n_test, k` matrix `NN`. Generate a vector `Y_kNN` containing the respective estimated class labels by performing a majority vote on `NN`. Compare the result with `Y_test`.

```python
# chose which numbers to load:
d_id1 = 2
d_id2 = 3
d = [d_id1, d_id2]

# define # of training and testing samples
n_train = 500
n_test = 10
k = 20

# initialize data matrices
X_train = np.zeros((784, 2*n_train))
X_test = np.zeros((784, 2*n_test))

print('Loading training data...')
for j in range(0,n_train):
    impath = data_folder + 'd' + str(d[0]) + '/d' + str(d[0]) + '_' + str(j+1).zfill(4) + '.png'
    X_train[:,0*n_train+j] = np.array(misc.imread(impath)).astype(np.float).ravel()/255
    impath = data_folder + 'd' + str(d[1]) + '/d' + str(d[1]) + '_' + str(j+1).zfill(4) + '.png'
    X_train[:,1*n_train+j] = np.array(misc.imread(impath)).astype(np.float).ravel()/255

Y_train = np.concatenate((np.zeros(n_train), np.ones(n_train)))

print('Loading test data...')
for j in range(n_test):
    impath = data_folder + 'd' + str(d[0]) + '/d' + str(d[0]) + '_' + str(n_train+j+1).zfill(4) + '.png'
    X_test[:,0*n_test+j] = np.array(misc.imread(impath)).astype(np.float).ravel()/255
    impath = data_folder + 'd' + str(d[1]) + '/d' + str(d[1]) + '_' + str(n_train+j+1).zfill(4) + '.png'
    X_test[:,1*n_test+j] = np.array(misc.imread(impath)).astype(np.float).ravel()/255

Y_test = np.concatenate((np.zeros(n_test), np.ones(n_test)))

print('Computing Frobenius distances...')

D = np.zeros((2*n_test, 2*n_train))
for i in range(2*n_test):
    # compute norm of distance of test sample i to all training samples
    D[i,:] = np.sqrt(np.sum((np.expand_dims(X_test[:,i], axis=1) - X_train) ** 2, axis=0))

print('Determining nearest neighbors...')
# np.argsort outputs indices required for sorting
NN = np.argsort(D, axis = 1)
# we only need the k closest neighbors, hence, we cut off after k columns
NN = NN[:,0:k]

print('Ground truth label data:')
print(Y_test.astype(np.float))

# compute nearest neighbor labelling
# sum over labels of k nearest training examples
# and divide by k
# if the resulting number is smaller than 0.5, we assign label 0
# if the resulting number is greater, we assign label 1
kNN_mask = np.sum(Y_train[NN], axis=1)/k >= 0.5
# convert boolean to integer
Y_kNN= kNN_mask.astype(np.float)

print('Labels determined by kNN:')
print(Y_kNN)
```

    Loading training data...
    Loading test data...
    Computing Frobenius distances...
    Determining nearest neighbors...
    Ground truth label data:
    [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  1.  1.  1.  1.  1.  1.  1.
      1.  1.]
    Labels determined by kNN:
    [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  1.  1.  1.  1.  1.  1.  1.
      1.  1.]