# Implementing FIR filters

In real-time filtering applications, filters are implemented by using some variation or other of their constant-coefficient difference equation (CCDE), so that one new output sample is generated for each new input sample. If all input data is available in advance, as in non-real-time (aka "offline") applications, then the CCDE-based algorithm is iteratively applied to all samples in the buffer.

In the case of FIR filters, the CCDE coefficients correspond to the impulse response and implementing the CCDE is equivalent to performing a convolution sum. In this notebook we will look at different ways to implement FIR filters.

```python
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
```

## Online implementation

The classic way to implement a filter is the one-in one-out approach. We will need to implement a persistent delay line. In Python we can either define a class or use function attributes; classes are tidier and reusable:

```python
class FIR_loop():
    def __init__(self, h):
        self.h = h
        self.ix = 0
        self.M = len(h)
        self.buf = np.zeros(self.M)

    def filter(self, x):
        y = 0
        self.buf[self.ix] = x
        for n in range(0, self.M):
            y += self.h[n] * self.buf[(self.ix+self.M-n) % self.M]
        self.ix = (self.ix + 1) % self.M
        return y
```

```python
# simple moving average:
h = np.ones(5)/5

f = FIR_loop(h)
for n in range(0, 10):
    print(round(f.filter(n), 3),  end='  ')
```

    0.0  0.2  0.6  1.2  2.0  3.0  4.0  5.0  6.0  7.0

While there's nothing wrong with the above implementation, when the data to be filtered is known in advance, it makes no sense to explicitly iterate over its element and it's better to use higher-level commands to perform the convolution. In Numpy, the command is `convolve`; before we use it, though, we need to take border effects into consideration.

## Offline implementations: border effects

When filtering a finite-length data vector with a finite-length impulse response, we need to decide what to do with the "invalid" shifts appearing in the terms of the convolution sum. Remember that, in the infinite-length case, the output is defined as

$$
    y[n] = \sum_{k=-\infty}^{\infty} h[k]x[n-k]
$$

Let's say that the impulse response is $M$ points long, so that $h[n]$ is nonzero only between $0$ and $M-1$; this means that the sum is reduced to

$$
    y[n] = \sum_{k=0}^{M-1} h[k]x[n-k]
$$

Now assume that $x[n]$ is a length-$N$ signal, so it is defined only for $0 \leq n \le N$ (we can safely consider $N > M$, otherwise exchange the roles of $x$ and $h$). In this case, the above sum is properly defined only for $M - 1 \le n \le N-1$; for any other value of $n$, the sum will contain an element $x[n-k]$ outside of the valid range of indices for the input.

So, if we start with an $N$-point input, we can only formally compute $N-M+1$ output samples. While this may not be a problem in some applications, it certainly is troublesome if repeated filtering operations end up "chipping away" at the signal little by little.

The solution is to "embed" the finite-length input data signal into an infinite-length sequence and, as always, the result will depend on the method we choose: finite support or periodization. (Note that the impulse response is already an infinite sequence since it's the response of the filter to the infinite sequence $\delta[n]$).

However, the embedding will create "artificial" data points that are dependent on the chosen embedding: these data points are said to suffer from **border effects**.

Let's build a simple signal and a simple FIR filter:

```python
# let's use a simple moving average:
M = 5
h = np.ones(M)/float(M)

# let's build a signal with a ramp and a plateau
x = np.concatenate((np.arange(1, 9), np.ones(5) * 8, np.arange(8,0,-1)))
plt.stem(x, use_line_collection=True);

print(f'signal length: {len(x)}')
```

### 1) No border effects

We may choose to accept the loss of data points and use only the $N-M+1$ output samples that correspond to a full overlap between the input data and the impulse response. This can be achieved by selecting `mode='valid'` in `correlate`:

```python
y = np.convolve(x, h, mode='valid')
print(f'signal length: {len(y)}')
plt.stem(y, use_line_collection=True);
```

### 2) finite-support extension

By embedding the input into a finite-support signal, the convolution sum is now well defined for all values of $n$, which now creates a new problem: the output will be nonzero for all values of $n$ for which $x[n-k]$ is nonzero, that is for $0 \le n \le N+M-1$: we end up with a *longer* support for the output sequence. This is the default in `correlate`  and corresponds to `mode='full'`:

```python
y = np.convolve(x, h, mode='full')
print(f'signal length: {len(y)}')
plt.stem(y, use_line_collection=True);
```

If we want to preserve the same length for input and output, we need to truncate the result. You can keep the *first* $N$ samples and discard the tail; this corresponds to the online implementation of the FIR filter. Alternatively, you can discard half the extra samples from the beginning and half from the end of the output and distribute the border effect evenly; this is achieved in `correlate`  by setting `mode='same'`:

```python
y = np.convolve(x, h, mode='same')
print(f'signal length: {len(y)}')
plt.stem(y, use_line_collection=True);
```

### 3) Periodic extension

As we know, the other way of embedding a finite-length signal is to build a periodic extension. The convolution in this case will return an $N$-periodic output:

$$
    \tilde{y}[n] = \sum_{k=0}^{M-1} h[k]\tilde{x}[n-k]
$$

We can easily implement a circular convolution using `convolve` like so: since the overlap between time-reversed impulse response and input is already good for the last $N-M$ points in the output, we just need to consider two periods of the input to compute the first $M$:

```python
def cconv(x, h):
    # as before, we assume len(h) < len(x)
    L = len(x)
    xp = np.concatenate((x,x))
    # full convolution
    y = np.convolve(xp, h)
    return y[L:2*L]
```

```python
y = cconv(x, h)
print(f'signal length: {len(y)}')
plt.stem(y, use_line_collection=True);
```

OK, clearly the result is not necessarily what we expected; note however that in both circular and "normal" convolution, you still have $M-1$ output samples "touched" by border effects, it's just that the border effects act differently in the two cases.

Interestingly, you can still obtain a "normal" convolution using a circular convolution if you zero-pad the input signal with $M-1$ zeros:

```python
y = cconv(np.concatenate((x, np.zeros(M-1))), h)
print(f'signal length: {len(y)}')
plt.stem(y, use_line_collection=True);

# plot in red the *difference* with the standard conv
plt.stem(y - np.convolve(x, h, mode='full'), markerfmt='ro', use_line_collection=True);
```

Why is this interesting? Because of the DFT....

## Offline implementations using the DFT

The convolution theorem states that, for infinite sequences,

$$
    (x\ast y)[n] = \mbox{IDTFT}\{X(e^{j\omega})Y(e^{j\omega})\}[n]
$$

Can we apply this result to the finite-length case? In other words, what is the inverse DFT of the product of two DFTs? Let's see:

\begin{align}
    \sum_{k=0}^{N-1}X[k]Y[k]e^{j\frac{2\pi}{N}nk} &= \sum_{k=0}^{N-1}\sum_{p=0}^{N-1}x[p]e^{-j\frac{2\pi}{N}pk}\sum_{q=0}^{N-1}y[q]e^{-j\frac{2\pi}{N}qk} \,e^{j\frac{2\pi}{N}nk} \\
    &= \sum_{p=0}^{N-1}\sum_{q=0}^{N-1}x[p]y[q]\sum_{k=0}^{N-1}e^{j\frac{2\pi}{N}(n-p-q)k} \\
    &= N\sum_{p=0}^{N-1}x[p]y[(n-p) \mod N]
\end{align}

The results follows from the fact that $\sum_{k=0}^{N-1}e^{j\frac{2\pi}{N}(n-p-q)k}$ is nonzero only for $n-p-q$ multiple of $N$; as $p$ varies from $0$ to $N-1$, the corresponding value of $q$ between $0$ and $N$ that makes $n-p-q$ multiple of $N$ is $(n-p) \mod N$.

So the fundamental result is: **the inverse DFT of the product of two DFTs is the circular convolution of the underlying time-domain sequences!**

To apply this result to FIR filtering, the first step is to choose the space for the DFTs. In our case we have a finite-length data vector of length $N$ and a finite-support impulse response of length $M$ with $M<N$ so let's operate in $\mathbb{C}^N$ by zero-padding the impulse response to size $N$. Also, we most likely want the normal convolution, so let's zero-pad both signals by an additional $M-1$ samples

```python
def DFTconv(x, h, mode='full'):
    # we want the compute the full convolution
    N = len(x)
    M = len(h)
    X = np.fft.fft(x, n=N+M-1)
    H = np.fft.fft(h, n=N+M-1)
    # we're using real-valued signals, so drop the imaginary part
    y = np.real(np.fft.ifft(X * H))
    if mode == 'valid':
        # only N-M+1 points, starting at M-1
        return y[M-1:N]
    elif mode == 'same':
        return y[int((M-1)/2):int((M-1)/2+N)]
    else:
        return y
```

Let's verify that the results are the same

```python
y = np.convolve(x, h, mode='valid')
print(f'signal length: {len(y)}')
plt.stem(y, use_line_collection=True);

y = DFTconv(x, h, mode='valid')
print(f'signal length: {len(y)}')
plt.stem(y, markerfmt='ro', use_line_collection=True);
```

```python
y = np.convolve(x, h, mode='same')
print(f'signal length: {len(y)}')
plt.stem(y, use_line_collection=True);

y = DFTconv(x, h, mode='same')
print(f'signal length: {len(y)}')
plt.stem(y, markerfmt='ro', use_line_collection=True);
```

Of course the question at this point is: why go through the trouble of taking DFTs if all we want is the standard convolution? The answer is: **computational efficiency.**

If you look at the convolution sum, each output sample requires $M$ multiplications (and $M-1$ additions but let's just consider multiplications). In order to filter an $N$-point signal we will need $NM$ multiplications. Assume $N \approx M$ and you can see that the computational requirements are on the order of $M^2$. If we go the DFT route using an efficient FFT implementation we have approximately:

 * $M\log_2 M$ multiplication to compute $H[k]$
 * $M\log_2 M$ multiplication to compute $X[k]$
 * $M\log_2 M$ multiplication to compute $X[k]H[k]$
 * $M\log_2 M$ multiplication to compute the inverse DFT

Even considering that we now have to use complex multiplications (which will cost twice as much), we can estimate the cost of the DFT based convolution at around $8M\log_2M$, which is smaller than $M^2$ as soon as $M>44$.

In practice, the data vector is much longer than the impulse response so that filtering via standard convolution requires on the order of $MN$ operations. Two techniques, called [Overlap Add](https://en.wikipedia.org/wiki/Overlap%E2%80%93add_method) and [Overlap Save](https://en.wikipedia.org/wiki/Overlap%E2%80%93save_method)
can be used to divide the convolution into $N/M$ independent convolutions between $h[n]$ and an $M$-sized piece of $x[n]$; FFT-based convolution can then be used on each piece. While the exact cost per sample of each technique is a bit complicated to estimate, as a rule of thumb **as soon as the impulse response is longer than 50 samples, it's more convenient to use DFT-based filtering.**

```python

```


---
# The gaussian probability distribution function (PDF)
---

In this script, we display several examples of 2-D gaussian PDF.

In its matrix-form, the equation for the N-D gausian PDF reads like this:

<blockquote>  $P(\bf{x}) = \frac{1}{(2\pi)^{0.5N} |\Sigma|^{0.5}} \exp{[-\frac{1}{2}(\bf{x}-\bf{\mu})^\top \Sigma^{-1} (\bf{x}-\bf{\mu})]}$  </blockquote>

where

<blockquote>
$
\begin{align}
\bf{x} &= [x_{1} x_{2} \cdots  x_{N}]^\top \\
\bf{\mu} &= [\mu_{1} \mu_{2} \cdots \mu_{N}]^\top  \\
\Sigma &= \begin{pmatrix} \sigma_{1}^2 & \sigma_{1,2} & \cdots & \sigma_{1,N} \\
\sigma_{1,2} & \sigma_{2}^2 & \cdots & \sigma_{2,N} \\
\vdots & \vdots & \ddots & \vdots \\
\sigma_{1,N} & \sigma_{2,N} & \cdots & \sigma_{N}^2 \end{pmatrix}
\end{align}
$
</blockquote>

In the 2-D case, the covariance matrix $\Sigma$ simplifies to
<blockquote>
$
\begin{align}
\Sigma & = \begin{pmatrix} \sigma_{1}^2 & \sigma_{1,2} \\ \sigma_{1,2} & \sigma_{2}^2 \end{pmatrix} \\
&= \begin{pmatrix} \sigma_{x_{1}}^2 & \rho\sigma_{x_{1}}\sigma_{x_{2}} \\ \rho\sigma_{x_{1}}\sigma_{x_{2}} & \sigma_{x_{2}}^2 \end{pmatrix}
\end{align}
$
</blockquote>
where $\rho$ is the correlation factor between the $x_{1}$ and $x_{2}$ data.

It is common to use $x$ and $y$ instead of $x_{1}$ and $x_{2}$ but we will stick to the more general notation.

```python
print(__doc__)

# Author: Pierre Gravel <pierre.gravel@iid.ulaval.ca>
# License: BSD

%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm

from scipy import stats
from scipy.stats import multivariate_normal

import seaborn as sns
sns.set(color_codes=True)

# Used for reproductibility of the results
np.random.seed(43)
```

    Automatically created module for IPython interactive environment

Let us first define the printing function to show the results.

```python
def show_PDF(X):

    # Generate a spatial grid where the PDF will be evaluated locally
    x_min, x_max = 0., 4.
    y_min, y_max = 0., 4.
    h = 0.01
    x, y = np.mgrid[x_min:x_max:h, y_min:y_max:h]
    pos = np.dstack((x, y))

    # Generator for the 2-D PDF
    rv = multivariate_normal(Mu,Sigma)

    # Remove datapoints outside figure limits
    X = X[(X[:,0]>x_min) & (X[:,0]<x_max) & (X[:,1]>y_min) & (X[:,1]<y_max)]

    fig = plt.figure(figsize = (10,10))
    cm = plt.get_cmap('viridis')

    ax = fig.add_subplot(111)
    ax.contourf(x, y, rv.pdf(pos),cmap=cm)

    ax.scatter(X[:, 0], X[:, 1], c="w", s=30, linewidth=1, marker="o")

    # Origin of the PDF
    plt.text(Mu[0], Mu[1], '$\mu$', horizontalalignment='center', verticalalignment='center', fontsize=20, color='red')

    ax.set_xlim([x_min,x_max])
    ax.set_ylim([y_min,y_max])
    ax.axis('equal')

    ax.set_xlabel('$x_{1}$',fontsize=18)
    ax.set_ylabel('$x_{2}$',rotation=0,fontsize=18)
    ax.yaxis.set_label_coords(-0.1,0.5)
    ax.xaxis.set_label_coords(0.5, -0.05)
    fig.tight_layout()

    plt.savefig(figName + '.png')
    plt.savefig(figName + '.pdf')

    plt.show()
```

## Example I: Isotropic distribution without correlation

The standard deviations $\sigma_{x1}$ and $\sigma_{x2}$ are identical with no correlation
between $x_{1}$ and $x_{2}$.

```python
# Parameters
Mu = [2., 2.]

sigma_x1 = 1.0
sigma_x2 = 1.0
rho = 0.
Sigma = [(sigma_x1**2, rho*sigma_x1*sigma_x2), (rho*sigma_x1*sigma_x2, sigma_x2**2)]

# Create a dataset of N random points
n = 100
X = np.random.multivariate_normal(Mu, Sigma, n)

figName = "Example_of_isotropic_2D_gaussian_PDF_without_correlation"

show_PDF(X)
```

## Example II: Anisotropic distribution without correlation

The standard deviations $\sigma_{x1}$ and $\sigma_{x2}$ are different with no correlation
between $x_{1}$ and $x_{2}$.

```python
# Parameters
Mu = [2., 2.]

sigma_x1 = 1.
sigma_x2 = 0.7
rho = 0.
Sigma = [(sigma_x1**2, rho*sigma_x1*sigma_x2), (rho*sigma_x1*sigma_x2, sigma_x2**2)]

# Create a dataset of N random points
n = 100
X = np.random.multivariate_normal(Mu, Sigma, n)

figName = "Example_of_anisotropic_2D_gaussian_PDF_without_correlation"

show_PDF(X)
```

## Example III: More general case

The standard deviations $\sigma_{x1}$ and $\sigma_{x2}$ are different with a 90% correlation
between $x_{1}$ and $x_{2}$.

```python
# Parameters
Mu = [2., 2.]

sigma_x1 = 0.5
sigma_x2 = 1.0
rho = 0.9
Sigma = [(sigma_x1**2, rho*sigma_x1*sigma_x2), (rho*sigma_x1*sigma_x2, sigma_x2**2)]

# Create a dataset of N random points
n = 100
X = np.random.multivariate_normal(Mu, Sigma, n)

figName = "Example_of_correlated_2D_gaussian_PDF"

show_PDF(X)
```

```python

```


```python
import sympy as sp

(x, r, H, m, pt, p_raise, t, S, ins, C,
 P_cap) = sp.symbols('x r H m prop_t prop_raise fed_t S ins C P_cap')
N, T = sp.symbols('N T', integer=True)
```

* {{x}} : Downpayment, what we want to optimize.
* {{r}} : Mortgage monthly rate.
* {{H}} : House price
* {{m}} : Monthly mortgage payment.
* {{pt}} : Yearly property tax.
* {{p_raise}} : Yearly property value increase, by county.
* {{t}} : Marginal yearly tax rate.
* {{S}} : Standard deduction (taxes).
* {{ins}}: Monthly insurance.
* {{T}} : Mortgage term in years.
* {{N}} : Sale date in years.
* {{C}} : Monthly capital available
* {{P_cap}} : Cap on principal interest deduction.

```python
def sp_max(var1, var2):
    return sp.Piecewise((var1, var1 >= var2), (var2, var1 < var2))

def sp_min(var1, var2):
    return sp.Piecewise((var1, var1 <= var2), (var2, var1 > var2))

k = sp.Symbol('k', integer=True)
P_k = sp.Symbol('P_k')
P_k_sub = (1 + r)**k * (H - x) - (m / r) * ((1 + r)**k - 1)

I_k = sp.Symbol('I_k')
I_k_sub = r * sp_min(P_k, P_cap)

year_num = sp.Symbol('Y', integer=True)
yearly_property_tax = pt * (1 + p_raise)**year_num * H
deductible_year = yearly_property_tax + sp.Sum(I_k, (k, 12 * year_num, 12 *
                                                     (year_num + 1) - 1))

deduction_year = sp_max(deductible_year - S, 0) * t

year_k = sp.floor(k / 12)

deduction_k = sp.Rational(1, 12) * deduction_year.subs(year_num, year_k)
property_tax_k = sp.Rational(1, 12) * yearly_property_tax.subs(
    year_num, year_k)
```

```python
# Creating equations for display:
(disp_deductible_year, disp_deduction_year,
 disp_deduction_k) = sp.symbols('deductible_Y deduction_Y deduction_k')

disp_eq1 = sp.Eq(P_k, P_k_sub)
disp_eq2 = sp.Eq(I_k, I_k_sub)
disp_eq3 = sp.Eq(disp_deductible_year, deductible_year)
disp_eq4 = sp.Eq(disp_deduction_year, deduction_year)
disp_eq5 = sp.Eq(disp_deduction_k, deduction_k)
```

Remaining principal for the {{k}}th month:

{{disp_eq1}}

Interest for the {{k}}th month:

{{disp_eq2}}

Deductible portion for year {{year_num}}:

{{disp_eq3}}

Deductions for year {{year_num}}:

{{disp_eq4}}

Deductions for the {{k}}th month:

{{disp_eq5}}

```python
substitution_dict = {
    N: 7,
    T: 30,
    r: (3.125 / 100) / 12,
    pt: 1.3 / 100,
    p_raise: 1.5 / 100,
    t: 32 / 100,
    H: 1.4e6,
    C: 5900,
    S: 24400,
    ins: 100,
    P_cap: 750000
}
```

```python
k_subs = {I_k: I_k_sub, P_k: P_k_sub}

deduction_N = deduction_k.subs(k_subs).subs(k, 12 * N)
property_tax_N = property_tax_k.subs(k_subs).subs(k, 12 * N)
eq1 = sp.Eq(m + property_tax_N - deduction_N + ins, C).doit()

P_T = P_k_sub.subs(k, 12 * T)
eq2 = sp.Eq(P_T, 0).doit()
```

```python
eq1 = eq1.subs(substitution_dict)
eq2 = eq2.subs(substitution_dict)
vals = sp.nsolve([eq1, eq2], [m, x], [1, 1])
```

```python
m_star = vals[0]
x_star = vals[1]

all_substitutions = {**substitution_dict, **{m: m_star, x: x_star}}
monthly_tax = property_tax_N.subs(all_substitutions)
monthy_deductions = deduction_N.subs(all_substitutions)
```

Monthly mortgage payment ({{m}}): {{'{:,.0f}'.format(m_star)}}

Downpayment ({{x}}): {{'{:,.0f}'.format(x_star)}}

Monthly tax : {{'{:,.0f}'.format(monthly_tax)}}

Monthly deduction ({{disp_deduction_k}}): {{'{:,.0f}'.format(float(monthy_deductions))}}

## Total gains in the buyers case

```python
h, s, espp, closing_costs = sp.symbols('h s espp closing-costs')

appreciation_substitutions = {
    **all_substitutions,
    **{
        h: 2 / 100,
        s: 8 / 100,
        espp: 1900 * 6,
        closing_costs: 30_000
    }
}
appreciated_house_price = (1 + h)**N * H
gains = appreciated_house_price - P_k_sub.subs(k, 12 * N) - closing_costs
```

House price after {{appreciation_substitutions[N]}} years: {{'{:,.0f}'.format(appreciated_house_price.subs(appreciation_substitutions).doit())}}

Total gains with house appreciation of {{appreciation_substitutions[h]*100}}%: {{'{:,.0f}'.format(gains.subs(appreciation_substitutions).doit())}}

#### Total gains in the renters case

```python
appreciated_downpayment = (1 + s)**N * appreciation_substitutions[x]
```

Downpayment after {{appreciation_substitutions[N]}} years:
{{'{:,.0f}'.format(appreciated_downpayment.subs(appreciation_substitutions).doit())}}

```python
s_12 = sp.sqrt(1 + s) - 1

espp_gains = sp.Sum((1 + s_12)**k * espp, (k, 0, 2 * N))
```

ESPP after {{appreciation_substitutions[N]}} years: {{'{:,.0f}'.format(espp_gains.subs(appreciation_substitutions).doit())}}

Total gains with stock appreciation of {{appreciation_substitutions[s]*100}}%:
{{'{:,.0f}'.format((appreciated_downpayment + espp_gains).subs(appreciation_substitutions).doit())}}

## More Downpayment on mortgage

```python
appreciation_substitutions[x] = 2 * appreciation_substitutions[x]
```

Downpayment:
{{'{:,.0f}'.format(appreciation_substitutions[x])}}

House price after {{appreciation_substitutions[N]}} years: {{'{:,.0f}'.format(appreciated_house_price.subs(appreciation_substitutions).doit())}}

Total gains with house appreciation of {{appreciation_substitutions[h]*100}}%: {{'{:,.0f}'.format(gains.subs(appreciation_substitutions).doit())}}

### Compared to investing that money

```python
appreciation_substitutions[x] = x_star
```

Downpayment:
{{'{:,.0f}'.format(appreciation_substitutions[x])}}

House price after {{appreciation_substitutions[N]}} years: {{'{:,.0f}'.format(appreciated_house_price.subs(appreciation_substitutions).doit())}}

Total gains with house appreciation of {{appreciation_substitutions[h]*100}}%: {{'{:,.0f}'.format(gains.subs(appreciation_substitutions).doit())}}

Total gains with stock appreciation of {{appreciation_substitutions[s]*100}}%:
{{'{:,.0f}'.format((appreciated_downpayment + espp_gains).subs(appreciation_substitutions).doit())}}

Sum:
{{'{:,.0f}'.format((gains + appreciated_downpayment + espp_gains).subs(appreciation_substitutions).doit())}}


# Fitzhugh-Nagumo Equation
> Simulating the Fitzhugh-Nagumo equations using a spectral method

- toc: true
- badges: true
- comments: false
- categories: [jupyter]

> youtube: https://youtu.be/5au-G5FuI_A

[https://youtu.be/5au-G5FuI_A](https://youtu.be/5au-G5FuI_A)

## Introduction

We simulate the Fitzhugh-Nagumo equations
$$
u_t = u - u^3 - v + \nabla^2 u\\
v_t = \epsilon(u - a_1 v - a_0) + \delta \nabla^2 v,
$$
using the semi-spectral time integration method.

This simultation was heavily inspired by Aric Hagberg's simulation in ["From Labyrinthine Patterns to Spiral Turbulence", PRL 1994](http://in.bgu.ac.il/en/bidr/SIDEER/DSEEP/Ehud_Meron/Documents/Papers/HaMe94prl.pdf).

The code below provides 3 initial conditions, "squiggle, blocks, and random". For time integration, besides the spectral method, we also provide the Euler method. Details about the semi-spectral method can be found after the code.

Parameters: $\epsilon=0.3$, $\delta=2.0$, $a_1=1.4$, and $a_0=0$.

Other simulations and Python examples can be found on my website: [yairmau.com](yairmau.com).

## The code

### import packages

```python
import numpy as np
import matplotlib.pyplot as plt
import os
from matplotlib import rcParams
rcParams['font.family'] = 'monospace'
```

### define class with all the methods

```python
class FitzHughNagumo(object):
    def __init__(self, epsilon=0.3, delta=2.0, a1=1.4, a0=0.0,
                 n=(256, 256), l=(400, 400),
                 start=0.0, step=1.0, finish=2000.0, dt=0.1,
                 integration_type="spectral"):
        self.epsilon = epsilon
        self.delta = delta
        self.a1 = a1
        self.a0 = a0
        self.n = n
        self.l = l
        self.start = start
        self.step = step
        self.finish = finish
        self.dt = dt
        self.integration_type = integration_type
        self.rhs_a = np.zeros((2, self.n[0], self.n[1]))

    def spectral_multiplier(self):
        dx = float(self.l[0]) / self.n[0]
        dy = float(self.l[1]) / self.n[1]
        # wave numbers
        fx = 2.0 * np.pi * np.fft.fftfreq(self.n[0], dx)
        fy = 2.0 * np.pi * np.fft.fftfreq(self.n[1], dy)
        kx = np.outer(fx, np.ones(self.n[0]))
        ky = np.outer(np.ones(self.n[1]), fy)
        # multiplier
        mult_a = np.zeros((2, self.n[0], self.n[1]))
        mult_a[0] = np.exp(-(kx ** 2 + ky ** 2) * self.dt)  # u
        mult_a[1] = np.exp(-self.delta * (kx ** 2 + ky ** 2) * self.dt)  # v
        return mult_a

    def rhs_reaction(self, a):
        u = a[0]  # alias
        v = a[1]  # alias
        # FHN right hand side
        self.rhs_a[0] = u - u ** 3 - v
        self.rhs_a[1] = self.epsilon * (u - self.a1 * v - self.a0)
        return self.rhs_a

    def rhs_euler(self, a):
        # boundary conditions in laplacian
        laplacian = self.periodic_laplacian
        u = a[0]  # alias
        v = a[1]  # alias
        dx = float(self.l[0]) / self.n[0]
        # FHN right hand side
        self.rhs_a[0] = u - u ** 3 - v + laplacian(u, dx=dx)
        self.rhs_a[1] = self.epsilon * (u - self.a1 * v - self.a0) + \
                        self.delta * laplacian(v, dx=dx)
        return self.rhs_a

    def draw(self, a, t):
        u = a[0]
        self.im = plt.imshow(u.real, cmap="Greys_r", origin='lower',
                             vmin=-0.534522, vmax=0.534522,
                             interpolation="gaussian")
        self.title = plt.title('time = {:>4.0f}'.format(0))
        plt.xticks([])
        plt.yticks([])
        self.im.figure.canvas.draw()

    def draw_update(self, a, t):
        u = a[0]
        self.title.set_text('time = {:>4.0f}'.format(t))
        self.im.set_data(u.real)
        self.im.figure.canvas.draw()

    def save_frame(self, i):
        fname = "_tmp{:05d}.png".format(i)
        self.frame_names.append(fname)
        self.fig.savefig(fname, bbox_inches='tight', dpi=300)

    def periodic_laplacian(self, u, dx=1):
        """Return finite difference Laplacian approximation of 2d array.
        Uses periodic boundary conditions and a 2nd order approximation."""
        laplacian = (np.roll(u, -1, axis=0) +
                     np.roll(u, +1, axis=0) +
                     np.roll(u, -1, axis=1) +
                     np.roll(u, +1, axis=1) - 4.0 * u) / (dx ** 2)
        return laplacian

    def random_ic(self):
        return 0.5 * (np.random.random((2, self.n[0], self.n[1])) - 0.5)

    def blocks_ic(self):
        a = np.ones((2, self.n[0], self.n[1]))
        a[0] = 0.534522
        a[1] = 0.381802
        n = self.n
        p = n[0] / 8
        a[0][3 * p - 4:3 * p + 4, 5 * p - 4:5 * p + 4] = -0.534522
        a[0][6 * p - 4:6 * p + 4, 3 * p - 4:3 * p + 4] = -0.534522
        return a

    def squiggle_ic(self):
        a = np.ones((2, self.n[0], self.n[1]))
        l = self.l
        uplus = 0.534522
        vplus = 0.381802
        uminus = -uplus
        X, Y = np.meshgrid(np.linspace(0, self.l[0], self.n[0]),
                           np.linspace(0, self.l[0], self.n[0]))
        cos_term = 0.05 * l[0] * np.sin(10 * (2 * np.pi) * Y / l[1] + np.pi * 0.3)
        exp_term = np.exp(-((Y - l[1] / 2) / (0.1 * l[1])) ** 2)
        width = 0.05 * l[0]
        Z = np.exp(-((X - l[0] / 2 + cos_term * exp_term) / width) ** 2)
        a[0] = uplus
        a[1] = vplus
        a[0][Z > 0.8] = uminus
        return a
```

### run simulation, save snapshots

```python
plt.ion()
plt.clf()
foo = FitzHughNagumo()
foo.fig = plt.figure(1)
ax = foo.fig.add_subplot(111)
a = foo.squiggle_ic()
mult_a = foo.spectral_multiplier()
fft_a = np.fft.fftn(a, axes=(1, 2))
t = foo.start
foo.draw(a, t)
foo.frame_names = []
foo.save_frame(0)
for i, tout in enumerate(np.arange(foo.start + foo.step,
                                   foo.finish + foo.step,
                                   foo.step)):
    while t < tout:
        if foo.integration_type == "spectral":
            rhs_a = foo.rhs_reaction(a)
            fft_a = mult_a * (fft_a + foo.dt * np.fft.fftn(rhs_a, axes=(1, 2)))
            a = np.fft.ifftn(fft_a, axes=(1, 2))
        if foo.integration_type == "euler":
            a = a + foo.dt * foo.rhs_euler(a)
        t += foo.dt
    foo.draw_update(a, t)
    foo.save_frame(i + 1)
```

### make movie, delete snapshots

```python
fps = 24
frames = "_tmp%5d.png"
movie_command = "ffmpeg -y -r {:} -i {:} fhn.mp4".format(fps, frames)
os.system(movie_command)
for fname in foo.frame_names:
    os.remove(fname)
```

## The semi-spectral method

The explanation below was taken from my thesis: ["Pattern Formation in Spatially Forced Systems: Application to Vegetation Restoration"](http://aranne5.bgu.ac.il/others/MauYair3.pdf).

The semi-spectral method is extremely useful when working with reaction-diffusion systems, and with parabolic PDEs in general.
This was the method used to run all the simulations of the Swift-Hohenberg model in this thesis, and it proved to be reliable and fast.
The explanation below is a summary of ["Spectral algorithms for reaction-diffusion equations"](https://pdfs.semanticscholar.org/6bb0/b794a07b0c9b19e7c7ab03a9a523eac2141d.pdf), by Richard V. Craster and Roberto Sassi, with a step by step recipe, so the reader can easily apply the method to any suitable problem.

### the method
The semi-spectral transform method is very useful when we have to integrate a system that evolves really slowly. Let us say we have a (parabolic) system of the form:
$$
\begin{equation*}
   u_t=\epsilon u + f(u)+D\nabla^2u,
   \label{eq:1} \tag{1}
\end{equation*}
$$

where $f(u)$ is a nonlinear function. First, we compute the Fourier transform of \eqref{eq:1}:
$$
\begin{equation*}
   \hat{u}_t=\epsilon \hat{u} + \hat{f}(u)-k^2D\hat{u},
   \label{eq:2} \tag{2}
\end{equation*}
$$
where the hat denotes the Fourier transform.

We rearrange \eqref{eq:2} in the following way:
$$
\begin{equation*}
   \hat{u}_t+a\hat{u}=\hat{f}(u),
   \label{eq:3} \tag{3}
\end{equation*}
$$
where $a=-\epsilon +k^2D$,
and now we make a variable substitution
$$
\begin{align*}
   \hat{v}(k,t)&=\;\hat{u}(k,t)\,e^{at}   \label{eq:4a} \tag{4a}\\
   \hat{v}_t&=\;\hat{u}_te^{at}+a\hat{u}\,e^{at}.  \label{eq:4b} \tag{4b}
\end{align*}
$$

We multiply \eqref{eq:3} by $e^{at}$ and we finally get
$$
\begin{equation*}
   \hat{v}_t=e^{at}\hat{f}(u).
   \label{eq:5} \tag{5}
\end{equation*}
$$

We can now advance $\hat{v}$ in time using a simple Euler step
$$
\begin{equation*}
   \hat{v}^{t_{n+1}}=\hat{v}^{t_n}+\Delta t\left( e^{at_n}\hat{f}(u) \right).
   \label{eq:6} \tag{6}
\end{equation*}
$$

What we really want is $\hat{u}$, which, according to \eqref{eq:4a}, is given by

$$
\begin{align*}
\displaystyle\hat{u}^{t_{n+1}}=&\;\hat{v}^{t_{n+1}}e^{-at_{n+1}} \label{eq:7a} \tag{7a}\\
   =&\;\hat{v}^{t_{n+1}}e^{-at_{n}}e^{-a\Delta t} \label{eq:7b} \tag{7b}\\
   =&\;\left(\hat{v}^{t_n}+\Delta t\; e^{a t_n}\hat{f}(u)\right)e^{-at_{n}}e^{-a\Delta t} \label{eq:7c} \tag{7c}\\
   =&\;\left(\hat{v}^{t_n}e^{-at_{n}}+\Delta t \;{e^{a t_n}}\hat{f}(u) {e^{-at_{n}}}\right)e^{-a\Delta t} \label{eq:7d} \tag{7d}\\
   =&\;\left( \hat{u}^{t_n}+\Delta t \hat{f}(u) \right)e^{-a\Delta t} \label{eq:7e} \tag{7e}.
\end{align*}
$$

There is actually no need to use the variable substitution in \eqref{eq:4a}. We now have an expression for $\hat{u}^{t_{n+1}}$:
$$
\begin{equation*}
   \hat{u}^{t_{n+1}}=\left( \hat{u}^{t_n}+\Delta t \hat{f}(u) \right)e^{-a\Delta t}.
   \label{eq:8} \tag{8}
\end{equation*}
$$

Now it is time to go back from the Fourier space to the real space, and for that we use an inverse Fourier transform
$$
   u^{t_{n+1}}=\mathcal{F}^{-1}[\hat{u}^{t_{n+1}}].
   \label{eq:9} \tag{9}
$$

### step by step
To implement this technique, one just has to follow the steps below:

* Calculate the Fourier transform of $u$: $\hat{u}=\mathcal{F}[u]$.
* Have $f(u)$ calculated and then take its Fourier transform: $\hat{f}(u)=\mathcal{F}[f(u)]$.
* For a given lattice with $N$ points, and $\delta x$ being the distance between them, make the frequency bin vector (matrix) $k$ for your one (two) dimensional system. In python the command would be
```python3
numpy.fft.fftfreq(N, dx).
```
The frequency bin vector $k$ looks like:

$$
\begin{align}
   k&=2\pi\cdot\left[ 0,1,\cdots,\tfrac{N}{2}-1,-\tfrac{N}{2},\cdots,-1 \right]/(N\, \delta x), \qquad \mbox{if N is even;} \label{eq:10a} \tag{10a}\\
   k&=2\pi\cdot\left[ 0,1,\cdots,\tfrac{N-1}{2},-\tfrac{N-1}{2},\cdots,-1 \right]/(N\, \delta x), \qquad \mbox{if N is odd.} \label{eq:10b} \tag{10b}
\end{align}
$$

Remember that the domain size is given by $L=N\, \delta x$, which means that the denominator in the expressions above can be written simply as $L$. It is clear from that fact that $\delta k$, the tiniest slice of the Fourier space is $\delta k=2\pi/L$. Corollary: if you want to divide the Fourier space into very many parts, simply have a huge domain.
If the system is two-dimensional, then have $k_x$ and $k_y$ calculated separately. The domain might not be square ($L_x\neq L_y$), and you might want to divide the domain into a different number of points ($N_x\neq N_y$). Anyway, prepare one-dimensional arrays of $k_x$ and $k_y$ as explained above, and then make an *outer product* of these arrays with a *ones* array of length $N$, as following:

$$
 k_{x,2d} = \begin{pmatrix}
             1 \\ 1 \\ \vdots \\ 1
            \end{pmatrix}
            \begin{pmatrix}
             k_{x1} & k_{x2} & ... & k_{xN}
            \end{pmatrix}
            =
            \begin{pmatrix}
             k_{x1} & k_{x2} & ... & k_{xN}\\
             k_{x1} & k_{x2} & ... & k_{xN}\\
                    & \vdots &        &       \\
             k_{x1} & k_{x2} & ... & k_{xN}
            \end{pmatrix}
 \label{eq:11} \tag{11}
$$

and

$$
 k_{y,2d} = \begin{pmatrix}
             k_{x1} \\ k_{x2} \\ \vdots \\ k_{xN}
            \end{pmatrix}
            \begin{pmatrix}
             1 & 1 & ... & 1
            \end{pmatrix}
            =
            \begin{pmatrix}
             k_{y1} & k_{y1} &        & k_{y1}\\
             k_{y2} & k_{y2} & ... & k_{y2}\\
             \vdots & \vdots &        & \vdots\\
             k_{yN} & k_{yN} &        & k_{yN}
            \end{pmatrix}.
  \label{eq:12} \tag{12}
$$

Then factor $e^{-a\Delta t}$ equals

$$
   e^{-a\Delta t}= e^{\left[\epsilon-D(k_x^2+k_y^2)\right]\Delta t},
   \label{eq:13} \tag{13}
$$

where $k_x^2$ is the element-wise exponentiation of the 2d array $k_{x,2d}$.

* Now that we have all the factors we need, we simply calculate
$$
 \hat{u}^{t_{n+1}}=\left( \hat{u}^{t_n}+\Delta t \hat{f}(u) \right)e^{\left[\epsilon-D(k_x^2+k_y^2)\right]\Delta t}.
 \label{eq:14} \tag{14}
$$
* We finally go back to the real space by applying the inverse Fourier transform: $u^{t_{n+1}}=\mathcal{F}^{-1}[\hat{u}^{t_{n+1}}]$.

### example
For the parametrically forced Swift-Hohenberg equation
$$
\frac{\partial u}{\partial t} = [\epsilon + \gamma \cos(k_f x)]u - u^3 -(\nabla^2+k_0^2)^2 u,
 \label{eq:15} \tag{15}
$$
we have
$$
 f(u)= -u^3 + \gamma u \cos(k_f x),\qquad a = \epsilon - \left(k_0- k_x^2 - k_y^2 \right)^2.
  \label{eq:16} \tag{16}
$$

```python

```


# Introduction to the Harmonic Oscillator

*Note:* Much of this is adapted/copied from https://flothesof.github.io/harmonic-oscillator-three-methods-solution.html

This week week we are going to begin studying molecular dynamics, which uses classical mechanics to study molecular systems. Our "hydrogen atom" in this section will be the 1D harmomic oscillator.

The harmonic oscillator is a system that, when displaced from its equilibrium position, experiences a restoring force F proportional to the displacement x:

$$F=-kx$$

The potential energy of this system is

$$V = {1 \over 2}k{x^2}$$

These are sometime rewritten as

$$ F=- \omega_0^2 m x, \text{ }  V(x) = {1 \over 2}  \omega_0^2 m {x^2}$$

Where $\omega_0  = \sqrt {{k \over m}} $

In classical mechanics, our goal is to determine the equations of motion, $x(t),y(t)$, that describe our system.

In this notebook we will use sympy to solve an second order, ordinary differential equation.

## 1. Solving differential equations with sympy

Soliving differential equations can be tough, and there is not always a set plan on how to proceed. Luckily for us, the harmonic osscillator is the classic second order diffferential eqations.

Consider the following second order differential equation

$$ay(t)''+by(t)'=c$$

where $y(t)'' = {{{d^2}y} \over {dt^2}}$, and $y(t)' = {{{d}y} \over {dt}}$

We can rewrite this as a homogeneous linear differential equations

$$ay(t)''+by(t)'-c=0$$

The goal here is to find $y(t)$, similar to our classical mechanics problems. Lets use sympy to solve this equation

### Second order ordinary differential equation

First we import the sympy library

```python
import sympy as sym
```

Next we initialize pretty printing

```python
sym.init_printing()
```

Next we will set our symbols

```python
t,a,b,c=sym.symbols("t,a,b,c")
```

Now for somehting new. We can define functions using `sym.Function("f")`

```python
y=sym.Function("y")
y(t)
```

Now, If I want to define a first or second derivative, I can use `sym.diff`

```python
sym.diff(y(t),(t,1)),sym.diff(y(t),(t,2))
```

My differential equation can be written as follows

```python
dfeq=a*sym.diff(y(t),(t,2))+b*sym.diff(y(t),(t,1))-c
dfeq
```

```python
sol = sym.dsolve(dfeq)
sol
```

The two constants $C_1$ and $C_2$ can be determined by setting boundry conditions.
First, we can set the condition $y(t=0)=y_0$

The next intial condition we will set is $y'(t=0)=v_0$

To setup the equality we want to solve, we are using `sym.Eq`. This function sets up an equaility between a lhs aand rhs of an equation

```python
# sym.Eq example
alpha,beta=sym.symbols("alpha,beta")
sym.Eq(alpha+2,beta)
```

Back to the actual problem

```python
y0,v0=sym.symbols("y_0,v_0")
ics=[sym.Eq(sol.args[1].subs(t, 0), y0),
     sym.Eq(sol.args[1].diff(t).subs(t, 0), v0)]
ics
```

We can use this result to first solve for $C_2$ and then solve for $C_1$.
Or we can use sympy to solve this for us.

```python
solved_ics=sym.solve(ics)
solved_ics
```

Substitute the result back into $y(t)$

```python
full_sol = sol.subs(solved_ics[0])
full_sol
```

We can plot this result too. Assume that $a,b,c=1$ and that the starting conditions are $y_0=0,v_0=0$

We will use two sample problems:

* case 1 : initial position is nonzero and initial velocity is zero
* case 2 : initial position is zero and initialvelocity is nonzero

```python
# Print plots
%matplotlib inline
```

#### Initial velocity set to zero

```python
case1 = sym.simplify(full_sol.subs({y0:0, v0:0, a:1, b:1, c:1}))
case1
```

```python
sym.plot(case1.rhs)
sym.plot(case1.rhs,(t,-2,2))
```

#### Initial velocity set to one

```python
case2 = sym.simplify(full_sol.subs({y0:0, v0:1, a:1, b:1, c:1}))
case2
```

```python
sym.plot(case2.lhs,(t,-2,2))
```

## Calculate the phase space

As we will see in lecture, the state of our classical systems are defined as points in phase space, a hyperspace defined by ${{\bf{r}}^N},{{\bf{p}}^N}$. We will convert our sympy expression into a numerical function so that we can plot the path of $y(t)$ in phase space $y,y'$.

```python
case1
```

```python
# Import numpy library
import numpy as np

# Make numerical functions out of symbolic expressions
yfunc=sym.lambdify(t,case1.rhs,'numpy')
vfunc=sym.lambdify(t,case1.rhs.diff(t),'numpy')

# Make list of numbers
tlst=np.linspace(-2,2,100)

# Import pyplot
import matplotlib
import matplotlib.pyplot as plt
# Make plot
plt.plot(yfunc(tlst),vfunc(tlst))
plt.xlabel('$y$')
plt.ylabel("$y'$")
plt.show()
```

### Exercise 1.1

Change the initial starting conditions and see how that changes the plots. Make three different plots with different starting conditions

```python
# Import numpy library
import numpy as np

#new cases
case2 = sym.simplify(full_sol.subs({y0:2, v0:10, a:1, b:1, c:1}))
case3 = sym.simplify(full_sol.subs({y0:3, v0:2, a:1, b:1, c:1}))
case4 = sym.simplify(full_sol.subs({y0:1, v0:5, a:1, b:1, c:1}))

# Make numerical functions out of symbolic expressions
yfunc2=sym.lambdify(t,case2.rhs,'numpy')
vfunc2=sym.lambdify(t,case2.rhs.diff(t),'numpy')
yfunc3=sym.lambdify(t,case3.rhs,'numpy')
vfunc3=sym.lambdify(t,case3.rhs.diff(t),'numpy')
yfunc4=sym.lambdify(t,case4.rhs,'numpy')
vfunc4=sym.lambdify(t,case4.rhs.diff(t),'numpy')

# Make list of numbers
tlst=np.linspace(-2,2,100)

# Import pyplot
import matplotlib
import matplotlib.pyplot as plt
# Make plot
plt.plot(yfunc2(tlst),vfunc2(tlst))
plt.plot(yfunc3(tlst),vfunc3(tlst))
plt.plot(yfunc4(tlst),vfunc4(tlst))
plt.xlabel('$y$')
plt.ylabel("$y'$")
plt.show()
```

## 2. Harmonic oscillator

Applying the harmonic oscillator force to Newton's second law leads to the following second order differential equation

$$ F = m a $$

$$ F= - \omega_0^2 m x $$

$$ a = - \omega_0^2 x $$

$$ x(t)'' = - \omega_0^2 x $$

The final expression can be rearranged into a second order homogenous differential equation, and can be solved using the methods we used above

Your goal is determine and plot the equations of motion of a 1D harmomnic oscillator

### Exercise 2.1

1. Use the methodology above to determine the equations of motion $x(t), v(t)$ for a harmonic ocillator
1. Solve for any constants by using the following initial conditions: $x(0)=x_0, v(0)=v_0$
1. Show expressions for and plot the equations of motion for the following cases:
    1. $x(0)=0, v(0)=0$
    1. $x(0)=0, v(0)>0$
    1. $x(0)>0, v(0)=0$
    1. $x(0)<0, v(0)=0$
1. Plot the phasespace diagram for the harmonic oscillator

```python
import sympy as sym

sym.init_printing()
F,t,a,m,omega0 = sym.symbols("F,t,a,m,omega_0")
x = sym.Function("x")
a = sym.diff(x(t),(t,2))+ omega0**2*x(t)
sola = sym.dsolve(a)
sola

x0,v0 = sym.symbols("x_0,v_0")
con = [sym.Eq(sola.args[1].subs(t, 0), x0),
       sym.Eq(sola.args[1].diff(t).subs(t, 0), v0)]
con
consolve = sym.solve(con)
consolve

full_sol = sola.subs(consolve[0])
full_sol
```

```python
import numpy as np
import sympy as sym
import matplotlib
import matplotlib.pyplot as plt

case1 = sym.simplify(full_sol.subs({x0:0, v0:0, omega0:1}))
case2 = sym.simplify(full_sol.subs({x0:0, v0:1, omega0:1}))
case3 = sym.simplify(full_sol.subs({x0:1, v0:0, omega0:1}))
case4 = sym.simplify(full_sol.subs({x0:-1, v0:0, omega0:1}))

sym.plot(case1.rhs,(t,-10,10))
sym.plot(case2.rhs,(t,-10,10))
sym.plot(case3.rhs,(t,-10,10))
sym.plot(case4.rhs,(t,-10,10))
```

```python
import numpy as np
import sympy as sym
import matplotlib
import matplotlib.pyplot as plt

case1 = sym.simplify(full_sol.subs({x0:0, v0:0, omega0:1}))
case2 = sym.simplify(full_sol.subs({x0:0, v0:1, omega0:1}))
case3 = sym.simplify(full_sol.subs({x0:1, v0:0, omega0:1}))
case4 = sym.simplify(full_sol.subs({x0:-1, v0:0, omega0:1}))

xfunc1 = sym.lambdify(t,case1.rhs,'numpy')
vfunc1 = sym.lambdify(t,case1.rhs.diff(t),'numpy')
xfunc2 = sym.lambdify(t,case2.rhs,'numpy')
vfunc2 = sym.lambdify(t,case2.rhs.diff(t),'numpy')
xfunc3 = sym.lambdify(t,case3.rhs,'numpy')
vfunc3 = sym.lambdify(t,case3.rhs.diff(t),'numpy')
xfunc4 = sym.lambdify(t,case4.rhs,'numpy')
vfunc4 = sym.lambdify(t,case4.rhs.diff(t),'numpy')

plt.plot(xfunc1(tlst),vfunc1(tlst))
plt.xlabel('$x$')
plt.ylabel("$x'$")
plt.show()
plt.plot(xfunc2(tlst),vfunc2(tlst))
plt.xlabel('$x$')
plt.ylabel("$x'$")
plt.show()
plt.plot(xfunc3(tlst),vfunc3(tlst))
plt.xlabel('$x$')
plt.ylabel("$x'$")
plt.show()
plt.plot(xfunc4(tlst),vfunc4(tlst))
plt.xlabel('$x$')
plt.ylabel("$x'$")
plt.show()
```


# 3/ Problem solutions

```python
from sympy import *
init_printing()
```

```python

```

## P3.1

```python
AUG = Matrix([
    [1, 5,   25],
    [2, 1,   32]])
AUG
```

```python
AUG.rref()
```

```python

```

## P3.2

In the above solution we showed how to build `AUG` matrix directly.
This time, we'll build `AUG` by row-joining (`row_join`) a matrix of coefficients and a vector of constants.

```python
A = Matrix([
    [3,       3],
    [2,  S(3)/2]])
A
```

```python
b = Matrix([6,5])  # b is a column vector
b
```

```python
# row-join A and b to obtain the augmented matrix
AUG = A.row_join(b)
AUG
```

### a) Alice

Let's obtain the matrix `AUGA` which is the result after Alice's row operation.

```python
AUGA = AUG.copy()  # make a copy of AUG
AUGA[0,:] = AUGA[0,:]/3
AUGA
```

```python
AUGA[1,:] = AUGA[1,:] - 2*AUGA[0,:]
AUGA
```

```python
AUGA[1,:] = -2*AUGA[1,:]
AUGA
```

```python
AUGA[0,:] = AUGA[0,:] - AUGA[1,:]
AUGA
```

### b) Bob

```python
AUGB = AUG.copy()
AUGB[0,:] = AUGB[0,:] - AUGB[1,:]
AUGB
```

```python
AUGB[1,:] = AUGB[1,:] - 2*AUGB[0,:]
AUGB
```

```python
AUGB[1,:] = -1*S(2)/3*AUGB[1,:]
AUGB
```

```python
AUGB[0,:] = AUGB[0,:] - S(3)/2*AUGB[1,:]
AUGB
```

```python

```

### c) Charlotte

```python
AUGC = AUG.copy()
AUGC[0,:], AUGC[1,:] = AUGC[1,:], AUGC[0,:]
AUGC
```

```python
AUGC[0,:] = AUGC[0,:]/2
AUGC
```

```python
AUGC[1,:] = AUGC[1,:] - 3*AUGC[0,:]
AUGC
```

```python
AUGC[1,:] = S(4)/3*AUGC[1,:]
AUGC
```

```python
AUGC[0,:] = AUGC[0,:] - S(3)/4*AUGC[1,:]
AUGC
```

```python

```

## P3.3

```python
# define agmented matrices for three systems of eqns. with unique sol'ns
AUGA = Matrix([
        [ -1, -2, -2],
        [  3, 3, 0]])

AUGB = Matrix([
        [ 1, -1, -2,  1],
        [-2,  3,  3, -1],
        [-1,  0,  1,  2]])

AUGC = Matrix([
        [ 2, -2,  3, 2],
        [ 1, -2, -1, 0],
        [-2,  2,  2, 1]])
```

```python
AUGA
```

```python
AUGA.rref()
```

```python
AUGB
```

```python
AUGB.rref()
```

```python
AUGC
```

```python
AUGC.rref()
```

```python

```

## P3.4

These three systems of equations have infinitely many solutions.

### P3.4 a)

```python
AUGA = Matrix([
    [ -1, -2,  -2],
    [  3,  6,   6]])
AUGA
```

```python
AUGA.rref()
```

```python
AUGA[0:2,0:2].nullspace()
```

```python
# the solutions to the sytem of equations represented by AUGA
# is of the form    point + nullspace
point = AUGA.rref()[0][:,2]
nullspace = AUGA[0:2,0:2].nullspace()
```

```python
# the point is also called he particular solution
point
```

```python
# if the augmented matrix AUGA is [A|b], then the point satisfies A*point = b
print( AUGA[0:2,0:2]*point == AUGA[:,2] )
AUGA[0:2,0:2]*point
```

#### Finding the null space

```python
# the nullspace of A is one dimensional and spanned by
n = nullspace[0]
n
# every vector n in the nullspace of A satisfies  A*n=0
```

```python
# so solution to A*x=b is any (point+s*n) where s is any real number
# since  A*(point+s*n) = A*point + sA*n = A*point + 0 = b.
# Let's verify claim for values of s in the range -5,-4,-3,-2,-1,0,1,2,3,4,5
for s in range(-5,6):
    print( AUGA[0:2,0:2]*(point + s*n),
           AUGA[0:2,0:2]*(point + s*n) == AUGA[:,2] )
```

    Matrix([[-2], [6]]) True
    Matrix([[-2], [6]]) True
    Matrix([[-2], [6]]) True
    Matrix([[-2], [6]]) True
    Matrix([[-2], [6]]) True
    Matrix([[-2], [6]]) True
    Matrix([[-2], [6]]) True
    Matrix([[-2], [6]]) True
    Matrix([[-2], [6]]) True
    Matrix([[-2], [6]]) True
    Matrix([[-2], [6]]) True

### P3.4 b)

```python
AUGB = Matrix([
        [ 1, -1, -2,   1],
        [-2,  3,  3,  -1],
        [-1,  2,  1,   0]])
AUGB
```

```python
AUGB.rref()
```

```python
point_B = AUGB.rref()[0][:,3]
nullspace_B = AUGB[0:3,0:3].nullspace()[0]
s = symbols('s')
point_B + s*nullspace_B
```

### P3.4 c)

```python
AUGC = Matrix([
        [ 2, -2, 3,  2],
        [ 0,  0, 5,  3],
        [-2,  2, 2,  1]])
AUGC
```

```python
AUGC.rref()
```

```python
constants = AUGC.rref()[0][:,3]

# construct point_C by placing the constants into the location of the pivots
pivots = AUGC.rref()[1]
point_C = zeros(3,1)
for idx, pivot in enumerate(pivots):
    point_C[pivot] = constants[idx]

nullspace_C = AUGC[0:3,0:3].nullspace()[0]
s = symbols('s')
point_C + s*nullspace_C
```

```python

```


# Calculus of variations: Ritz method

The [Ritz method](http://en.wikipedia.org/wiki/Ritz_method) is a direct variational method to find an approximate solution for boundary value problems. The method is named after [Walter Ritz](http://en.wikipedia.org/wiki/Walther_Ritz).

## Example 1: Ordinary differential equation

Let us consider the differential equation
$$\frac{d^2u}{dx^2} + u + x = 0 \enspace , $$
with boundary conditions $u(0)=u(1)=0$.

The variational form that is equivalent to the differential equation is
$$J[u] = \int\limits_{0}^{1} F(u, u')dx = \int\limits_{0}^{1} \left[\frac{1}{2}\left( \frac{du}{dx}\right)^2 - \frac{1}{2}u^2 - ux \right] dx \enspace , $$
we can verify that computing the [Euler-Lagrange equations](http://en.wikipedia.org/wiki/Euler%E2%80%93Lagrange_equation), i.e.,
$$F_u - \frac{d}{dx}F_{u'} = 0 \enspace ,$$
where the subindices refer to derivatives with respect to $u$ and $u'$. In this case
$$\begin{align*}
F_u =& -u - x\\
F_{u'} =& u' \enspace ,
\end{align*}$$
hence
$$ -u'' - u - x = 0 \enspace ,$$
or
$$\frac{d^2 u}{dx^2} + u + x = 0 \enspace .$$

The approximating functions need to satisfy the boundary conditions, then, we propose solutions of the form
$$u_n(x) = x (1-x) \sum_{i=0}^n c_i x^i$$

```
from __future__ import division
from sympy import *
from sympy import init_printing
from sympy.solvers import solve
import matplotlib.pyplot as plt, mpld3
import numpy as np
init_printing()
```

```
x = symbols('x')
```

```
def approx_sol(num):
    """Compute the trial function using polynomials

    Parameters
    ----------
    num : int
        Number of terms in the expansion.

    Returns
    -------
    u_n : Sympy expression
        Trial function for the given basis.
    c : (num) Sympy symbols list
        List of coefficients.

    """
    c = symbols('c0:%d'%num)
    u_n = x*(1 - x)*sum([c[k]*x**k for k in range(num)])
    return u_n, c
```

Let us compute the functional for this function

```
def funct_J(u, x):
    """Functional for a given trial function

    Parameters
    ----------
    u : Sympy expression
        Approximant solution.
    x : Sympy symbol
        Independent variable.

    Returns
    -------
    J : Sympy expression
        Functional for the given approximant function.

    """
    J = integrate(1/S(2)*(diff(u,x))**2 - 1/S(2)*u**2 - u*x, (x, 0, 1))
    return J

```

Now, let  us put all the ingredients together

```
u2, c = approx_sol(2)
```

```
J = funct_J(u2, x)
```

Thus, the approximated solution is

```
sol = solve([diff(J, c[0]), diff(J, c[1])], c)
u_ap = u2.subs(sol)
u_ap
```

The exact solution is
$$u(x) = \frac{\sin(x)}{\sin(1)} - x \enspace ,$$
and we can compare the two solutions that are really close (the relative error is $1.3778\times 10^{-5}$).

```
u_anal = sin(x)/sin(1) - x
```

```
%matplotlib inline
mpld3.enable_notebook()

vec_u = lambdify((x), u_ap, "numpy")
plt.figure(figsize=(8,4))
t = np.linspace(0, 1, 100)
y = vec_u(t)
y_anal = np.sin(t)/np.sin(1) - t
plt.plot(t, y_anal, lw=2)
plt.plot(t, y, 'r--', lw=1, markevery=10)
plt.grid(True, lw=1, color='black', alpha=0.2)
plt.xlabel("Position - x", size=15)
plt.ylabel("Solution - u", size=15)
plt.show()
```

We can compute the solution for different levels of approximation

```
plt.figure(figsize=(8,4))
t = np.linspace(0, 1, 100)
y_anal = np.sin(t)/np.sin(1) - t
plt.plot(t, y_anal, lw=2, label="Analytic")
plt.grid(True, lw=1, color='black', alpha=0.2)

t = np.linspace(0, 1, 10)
markers = ['o', 's', 'h', '*']
for cont,n in enumerate(range(1,8,2)):
    u, c = approx_sol(n)
    J = funct_J(u, x)
    sol = solve([diff(J, c[k]) for k in range(n)], c)
    u_ap = u.subs(sol)
    vec_u = lambdify((x), u_ap, "numpy")
    y = vec_u(t)
    plt.plot(t, y, marker=markers[cont], ms=8, lw=0, label="n=%i"%n)

plt.legend(loc='best')
plt.show()
```

We can compute the error for different number of terms

```
N = range(1, 23, 2)
errors = np.zeros((len(N),))
for cont, n in enumerate(N):
    u, c = approx_sol(n)
    J = funct_J(u, x)
    sol = solve([diff(J, c[k]) for k in range(n)], c)
    u_ap = u.subs(sol)
    rel_error = abs(integrate(u_ap - u_anal, (x,0,1))/integrate(u_anal, (x,0,1)))
    errors[cont] = rel_error

```

```
plt.figure(figsize=(8,4))
plt.plot(N, np.log(errors)/np.log(10), 'o')
plt.xlabel("Number of terms", size=15)
plt.ylabel("Logarithm of the relative error", size=15)
plt.grid(True, lw=1, color='black', alpha=0.2)
plt.show()
```

```

```

```
from IPython.core.display import HTML
def css_styling():
    styles = open('../styles/custom_barba.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>
/* Based on Lorena Barba template available at: https://github.com/barbagroup/AeroPython/blob/master/styles/custom.css*/
@font-face {
font-family: "Computer Modern";
src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}
div.cell{
width:800px;
margin-left:16% !important;
margin-right:auto;
}
h1 {
font-family: 'Alegreya Sans', sans-serif;
}
h2 {
font-family: 'Fenix', serif;
}
h3{
font-family: 'Fenix', serif;
margin-top:12px;
margin-bottom: 3px;
}
h4{
font-family: 'Fenix', serif;
}
h5 {
font-family: 'Alegreya Sans', sans-serif;
}
div.text_cell_render{
font-family: 'Alegreya Sans',Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
line-height: 135%;
font-size: 120%;
width:600px;
margin-left:auto;
margin-right:auto;
}
.CodeMirror{
font-family: "Source Code Pro";
font-size: 90%;
}
/* .prompt{
display: None;
}*/
.text_cell_render h1 {
font-weight: 200;
font-size: 50pt;
line-height: 100%;
color:#CD2305;
margin-bottom: 0.5em;
margin-top: 0.5em;
display: block;
}
.text_cell_render h5 {
font-weight: 300;
font-size: 16pt;
color: #CD2305;
font-style: italic;
margin-bottom: .5em;
margin-top: 0.5em;
display: block;
}
.warning{
color: rgb( 240, 20, 20 )
}
</style>


```
%matplotlib inline
from matplotlib.pyplot import style, plot
```

```
from numpy import exp, linspace, zeros
```

```
from scipy.optimize import fsolve
```

```
a = 1
b = -2
h = 1
f = lambda s: s - a - b*exp(-h*s)
```

```
t = linspace(0, 10, 100)
```

```
plot(t, f(t))
```

```
raices = fsolve(f, -10.0)
raices
```

    array([ 0.69339871])

```
plot(raices, zeros(raices.size), "o")
```

```
f = lambda s: s**2 + 1
```

```
from sympy import var, init_printing, exp, roots
init_printing()
```

```
var("s")
```

```
f = s - 1 + 2*exp(-1*s)
f
```

```
def corta(xs, e):
    if len(xs) > 1:
        for i, x in enumerate(xs[:-1]):
            if abs(xs[i+1] - x) > e:
                return [xs[:i+1], xs[i+1:]]
    return xs

def separa_grupos(xs, e):
    separados = corta(xs, e)
    if len(separados) > 1:
        x, y = separa_grupos(separados[1], e)
        return separados[0], x, y
    return xs
```

IPythonNotebooks/Sistemas con retardo en la entrada/Prueba.ipynb


# 2021-10-08 Finite Difference in 2D

## Last time

* Accuracy vs cost tradeoffs for ODE integrators
* Stiff and non-stiff problems, discussion of costs

## Today

* FD methods in 2D
* Cost profile
* The need for fast algebraic solvers

```julia
using Plots
using LinearAlgebra
using SparseArrays
using Zygote

default(linewidth=4)

function newton(residual, jacobian, u0; maxits=20)
    u = u0
    uhist = [copy(u)]
    normhist = []
    for k in 1:maxits
        f = residual(u)
        push!(normhist, norm(f))
        J = jacobian(u)
        delta_u = - J \ f
        u += delta_u
        push!(uhist, copy(u))
    end
    uhist, normhist
end

function plot_stability(Rz, title; xlims=(-2, 2), ylims=(-2, 2))
    x = LinRange(xlims[1], xlims[2], 100)
    y = LinRange(ylims[1], ylims[2], 100)
    heatmap(x, y, (x, y) -> abs(Rz(x + 1im*y)), c=:bwr, clims=(0, 2), aspect_ratio=:equal, title=title)
end
```

    plot_stability (generic function with 1 method)

# Extending advection-diffusion to 2D

## 1 dimension

\begin{align} u_t + (- \kappa u_x + w u)_x &= f(x) & \text{ on } \Omega &= (a,b) \\
u(a) &= g_D(a) & u'(b) &= g_N(b) \end{align}

* Cell Peclet number $\mathrm{Pe}_h = \frac{\lvert w \rvert h}{\kappa}$
  * $\mathrm{Pe}_h \lesssim 1$ avoids oscillations
  * $\mathrm{Pe}_h \gtrsim 1$ is non-stiff for time-dependent model
* Centered versus upwind for advection
* Need uniformly bounded $\kappa \ge \epsilon > 0$
* "Strong form" not defined at discontinuities in $\kappa$
  * Works okay using divergence form and fluxes at staggered points

## 2 dimensions

\begin{align} u_t + \nabla\cdot\big(- \kappa \nabla u + \mathbf{w} u \big) &= f(x,y) & \text{ on } \Omega & \subset \mathbb R^2 \\
u|_{\Gamma_D} &= g_D(x,y) \\
(-\kappa \nabla u + \mathbf w u) \cdot \hat n|_{\Gamma_N} &= g_N(x,y)
\end{align}

* $\Omega$ is some well-connected open set (we will assume simply connected) and the Dirichlet boundary $\Gamma_D \subset \partial \Omega$ is nonempty.
* Finite difference methods don't have an elegant/flexible way to specify boundaries
* We'll choose $\Omega = (-1, 1) \times (-1, 1)$

# On finite difference grids

* Non-uniform grids can mesh "special" domains
  * Rare in 3D; overset grids, immersed boundary methods
* Concept of staggering is complicated/ambiguous

<div>

</div>

# Time-independent advection-diffusion

## Advection

$$ \nabla\cdot(\mathbf w u) = \mathbf w \cdot \nabla u + (\nabla\cdot \mathbf w) u$$

If we choose divergence-free $\mathbf w$, we can use the stencil

$$ \mathbf w \cdot \nabla u \approx \frac 1 h \begin{bmatrix}
& w_2 & \\
-w_1 & & w_1 \\
& -w_2 &
\end{bmatrix} \!:\! \begin{bmatrix}
u_{i-1, j+1} & u_{i, j+1} & u_{i+1,j+1} \\
u_{i-1, j} & u_{i, j} & u_{i+1,j} \\
u_{i-1, j-1} & u_{i, j-1} & u_{i+1,j-1} \\
\end{bmatrix}
$$

# Diffusion

$$ -\nabla\cdot(\kappa \nabla u) = -\kappa \nabla\cdot \nabla u - \nabla\kappa\cdot \nabla u$$
* When would you trust this decomposition?
* If we have constant $\kappa$, we can write
$$
-\kappa \nabla\cdot \nabla u \approx \frac{\kappa}{h^2} \begin{bmatrix}
& -1 & \\
-1 & 4 & -1 \\
& -1 &
\end{bmatrix} \!:\! \begin{bmatrix}
u_{i-1, j+1} & u_{i, j+1} & u_{i+1,j+1} \\
u_{i-1, j} & u_{i, j} & u_{i+1,j} \\
u_{i-1, j-1} & u_{i, j-1} & u_{i+1,j-1} \\
\end{bmatrix}
$$

# Advection-diffusion in code

```julia
function advdiff_matrix(n; kappa=1, wind=[1, 1]/sqrt(2))
    h = 2 / n
    rows = Vector{Int64}()
    cols = Vector{Int64}()
    vals = Vector{Float64}()
    idx(i, j) = (i-1)*n + j
    stencil_advect = [-wind[1], -wind[2], 0, wind[1], wind[2]] / h
    stencil_diffuse = [-1, -1, 4, -1, -1] * kappa / h^2
    for i in 1:n
        for j in 1:n
            if i in [1, n] || j in [1, n]
                push!(rows, idx(i, j))
                push!(cols, idx(i, j))
                push!(vals, 1.)
            else
                append!(rows, repeat([idx(i,j)], 5))
                append!(cols, [idx(i-1,j), idx(i,j-1), idx(i,j), idx(i+1,j), idx(i,j+1)])
                append!(vals, stencil_advect + stencil_diffuse)
            end
        end
    end
    sparse(rows, cols, vals)
end
```

    advdiff_matrix (generic function with 1 method)

# Spy the matrix

```julia
A = advdiff_matrix(10, wind=[1, 3], kappa=.1)
clim = norm(vec(A), Inf)
spy(A, marker=(:square, 3), c=:bwr, clims=(-clim, clim))
```

```julia
A = advdiff_matrix(20, wind=[0, 0], kappa=.1)
ev = eigvals(Matrix(A))
scatter(real(ev), imag(ev))
```

# Plot a solution

```julia
n = 100
x = LinRange(-1, 1, n)
y = x
f = cos.(pi*x/2) * cos.(pi*y/2)'
heatmap(x, y, f, aspect_ratio=:equal)
```

```julia
A = advdiff_matrix(n, wind=[3,1], kappa=.01)
u = A \ vec(f)
heatmap(x, y, reshape(u, n, n), aspect_ratio=:equal)
```

* What happens when advection dominates?
* As you refine the grid?

# Cost breadown and optimization

```julia
using ProfileSVG
function assemble_and_solve(n)
    A = advdiff_matrix(n)
    x = LinRange(-1, 1, n)
    f = cos.(pi*x/2) * cos.(pi*x/2)'
    u = A \ vec(f)
end

@profview assemble_and_solve(600)
```

<!DOCTYPE html>
<html>
<body>
<svg version="1.1" width="960" height="436" viewBox="0 0 960 436"
     xmlns="http://www.w3.org/2000/svg" id="fig-6326d75e0427416a8c634688d8688ffe">
<defs>
    <clipPath id="fig-6326d75e0427416a8c634688d8688ffe-clip">
        <rect x="0" y="0" width="960" height="436"/>
    </clipPath>
</defs>
<style type="text/css">
    #fig-6326d75e0427416a8c634688d8688ffe text {
        pointer-events: none;
        font-family: inherit;
        font-size: 12px;
        fill: #000000;
    }
    text#fig-6326d75e0427416a8c634688d8688ffe-caption {
        font-size: 16.8px;
        fill: #000000;
        text-anchor: middle;
    }
    #fig-6326d75e0427416a8c634688d8688ffe-bg {
        fill: #FFFFFF;
    }
    #fig-6326d75e0427416a8c634688d8688ffe-viewport rect, #fig-6326d75e0427416a8c634688d8688ffe-viewport path {
        vector-effect: non-scaling-stroke;
        fill-opacity: 1;
    }
    #fig-6326d75e0427416a8c634688d8688ffe-viewport text {
        stroke: #000000;
        stroke-width: 0;
        stroke-opacity: 0.35;
    }
    #fig-6326d75e0427416a8c634688d8688ffe-viewport rect:hover, #fig-6326d75e0427416a8c634688d8688ffe-viewport path:hover {
        fill-opacity: 0.75;
        stroke: #000000;
        stroke-width: 0.5;
    }
    #fig-6326d75e0427416a8c634688d8688ffe-viewport + rect {
        fill: #FFFFFF;
        opacity: 0.8;
    }
    text#fig-6326d75e0427416a8c634688d8688ffe-details{
        fill: #000000;
    }
</style>
<g id="fig-6326d75e0427416a8c634688d8688ffe-frame" clip-path="url(#fig-6326d75e0427416a8c634688d8688ffe-clip)">
<rect id="fig-6326d75e0427416a8c634688d8688ffe-bg" x="0" y="0" width="960" height="436"/>
<text id="fig-6326d75e0427416a8c634688d8688ffe-caption" x="480" y="24">Profile results</text>
<g id="fig-6326d75e0427416a8c634688d8688ffe-viewport" transform="scale(1)">
<rect x="10" y="383" width="940.57" height="15" rx="2" fill="#FFA000" data-dinfo=""/>
<text x="10" dx="4" y="394.25"> in :-1</text>
<rect x="31.7" y="368" width="908.02" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="31.7" dx="4" y="379.25">#15 in task.jl:411</text>
<rect x="31.7" y="353" width="908.02" height="15" rx="2" fill="#E1B2FF" data-dinfo="/home/jed/.julia/packages/IJulia/e8kqU/src/"/>
<text x="31.7" dx="4" y="364.25">eventloop in eventloop.jl:8</text>
<rect x="31.7" y="338" width="908.02" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="31.7" dx="4" y="349.25">invokelatest in essentials.jl:706</text>
<rect x="31.7" y="323" width="908.02" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="31.7" dx="4" y="334.25">#invokelatest#2 in essentials.jl:708</text>
<rect x="31.7" y="308" width="908.02" height="15" rx="2" fill="#00B58B" data-dinfo="/home/jed/.julia/packages/IJulia/e8kqU/src/"/>
<text x="31.7" dx="4" y="319.25">execute_request in execute_request.jl:67</text>
<rect x="31.7" y="293" width="908.02" height="15" rx="2" fill="#E1B2FF" data-dinfo="/home/jed/.julia/packages/SoftGlobalScope/u4UzH/src/"/>
<text x="31.7" dx="4" y="304.25">softscope_include_string in SoftGlobalScope.jl:65</text>
<rect x="31.7" y="278" width="908.02" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="31.7" dx="4" y="289.25">include_string in loading.jl:1116</text>
<rect x="31.7" y="263" width="908.02" height="15" rx="2" fill="#F3004D" data-dinfo="./"/>
<text x="31.7" dx="4" y="274.25">eval in boot.jl:360</text>
<rect x="66.6" y="248" width="98.59" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="66.6" dx="4" y="259.25">assemble_and_solve in In[133]:3</text>
<rect x="66.6" y="233" width="98.12" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="66.6" dx="4" y="244.25">advdiff_matrix in In[131]:2</text>
<rect x="66.6" y="218" width="21.7" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="66.6" dx="4" y="229.25">#advdiff_matrix#21 in In[131]:11</text>
<rect x="66.6" y="203" width="21.23" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="66.6" dx="4" y="214.25">vect in array.jl:108</text>
<rect x="66.6" y="188" width="19.34" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="66.6" dx="4" y="199.25">_array_for in array.jl:670</text>
<rect x="66.6" y="173" width="19.34" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="66.6" dx="4" y="184.25">_array_for in array.jl:672</text>
<rect x="66.6" y="158" width="19.34" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="66.6" dx="4" y="169.25">similar in abstractarray.jl:784</text>
<rect x="66.6" y="143" width="19.34" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="66.6" dx="4" y="154.25">similar in abstractarray.jl:785</text>
<rect x="66.6" y="128" width="19.34" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="66.6" dx="4" y="139.25">Array in boot.jl:457</text>
<rect x="66.6" y="113" width="19.34" height="15" rx="2" fill="#FFA000" data-dinfo="./"/>
<text x="66.6" dx="4" y="124.25">Array in boot.jl:448</text>
<rect x="67.08" y="98" width="0.47" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="67.08" dx="4" y="109.25">umfpack_free_symbolic in umfpack.jl:591</text>
<rect x="67.08" y="83" width="0.47" height="15" rx="2" fill="#E1B2FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="67.08" dx="4" y="94.25">umfpack_free_numeric in umfpack.jl:606</text>
<rect x="67.08" y="68" width="0.47" height="15" rx="2" fill="#F6A60A" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="67.08" dx="4" y="79.25">umfpack_dl_free_numeric in umfpack.jl:602</text>
<rect x="85.94" y="188" width="1.89" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="85.94" dx="4" y="199.25">setindex! in array.jl:843</text>
<rect x="87.83" y="203" width="0.47" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="87.83" dx="4" y="214.25">in in operators.jl:1136</text>
<rect x="87.83" y="188" width="0.47" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="87.83" dx="4" y="199.25">iterate in array.jl:781</text>
<rect x="87.83" y="173" width="0.47" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="87.83" dx="4" y="184.25">&lt; in int.jl:448</text>
<rect x="87.83" y="158" width="0.47" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="87.83" dx="4" y="169.25">&lt; in int.jl:441</text>
<rect x="88.3" y="218" width="18.87" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="88.3" dx="4" y="229.25">#advdiff_matrix#21 in In[131]:16</text>
<rect x="88.3" y="203" width="6.61" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="88.3" dx="4" y="214.25">repeat in abstractarraymath.jl:224</text>
<rect x="88.3" y="188" width="6.61" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="88.3" dx="4" y="199.25">repeat##kw in abstractarraymath.jl:267</text>
<rect x="88.3" y="173" width="6.61" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="88.3" dx="4" y="184.25">#repeat#1 in abstractarraymath.jl:269</text>
<rect x="88.3" y="158" width="6.61" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="88.3" dx="4" y="169.25">repeat_inner_outer in abstractarraymath.jl:327</text>
<rect x="88.3" y="143" width="4.25" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="88.3" dx="4" y="154.25">repeat_outer in abstractarraymath.jl:347</text>
<rect x="88.3" y="128" width="4.25" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="88.3" dx="4" y="139.25">similar in array.jl:357</text>
<rect x="88.3" y="113" width="4.25" height="15" rx="2" fill="#FFA400" data-dinfo="./"/>
<text x="88.3" dx="4" y="124.25">Array in boot.jl:448</text>
<rect x="92.55" y="143" width="0.47" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="92.55" dx="4" y="154.25">repeat_outer in abstractarraymath.jl:348</text>
<rect x="93.02" y="143" width="1.41" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="93.02" dx="4" y="154.25">repeat_outer in abstractarraymath.jl:350</text>
<rect x="93.02" y="128" width="1.41" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="93.02" dx="4" y="139.25">setindex! in array.jl:870</text>
<rect x="93.02" y="113" width="0.47" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="93.02" dx="4" y="124.25">unsafe_copyto! in array.jl:266</text>
<rect x="93.02" y="98" width="0.47" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="93.02" dx="4" y="109.25">pointer in abstractarray.jl:1119</text>
<rect x="93.02" y="83" width="0.47" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="93.02" dx="4" y="94.25">unsafe_convert in pointer.jl:65</text>
<rect x="93.49" y="113" width="0.94" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="93.49" dx="4" y="124.25">unsafe_copyto! in array.jl:271</text>
<rect x="94.43" y="143" width="0.48" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="94.43" dx="4" y="154.25">repeat_outer in abstractarraymath.jl:352</text>
<rect x="94.91" y="203" width="3.77" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="94.91" dx="4" y="214.25">append! in array.jl:980</text>
<rect x="94.91" y="188" width="3.77" height="15" rx="2" fill="#F6A60A" data-dinfo="./"/>
<text x="94.91" dx="4" y="199.25">_growend! in array.jl:888</text>
<rect x="98.68" y="203" width="1.89" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="98.68" dx="4" y="214.25">append! in array.jl:981</text>
<rect x="98.68" y="188" width="1.89" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="98.68" dx="4" y="199.25">copyto! in array.jl:304</text>
<rect x="100.57" y="203" width="6.6" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="100.57" dx="4" y="214.25">vect in array.jl:108</text>
<rect x="100.57" y="188" width="6.6" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="100.57" dx="4" y="199.25">_array_for in array.jl:670</text>
<rect x="100.57" y="173" width="6.6" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="100.57" dx="4" y="184.25">_array_for in array.jl:672</text>
<rect x="100.57" y="158" width="6.6" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="100.57" dx="4" y="169.25">similar in abstractarray.jl:784</text>
<rect x="100.57" y="143" width="6.6" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="100.57" dx="4" y="154.25">similar in abstractarray.jl:785</text>
<rect x="100.57" y="128" width="6.6" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="100.57" dx="4" y="139.25">Array in boot.jl:457</text>
<rect x="100.57" y="113" width="6.6" height="15" rx="2" fill="#FFA000" data-dinfo="./"/>
<text x="100.57" dx="4" y="124.25">Array in boot.jl:448</text>
<rect x="107.17" y="218" width="13.21" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="107.17" dx="4" y="229.25">#advdiff_matrix#21 in In[131]:17</text>
<rect x="107.17" y="203" width="1.41" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="107.17" dx="4" y="214.25">append! in array.jl:980</text>
<rect x="107.17" y="188" width="1.41" height="15" rx="2" fill="#E4A71C" data-dinfo="./"/>
<text x="107.17" dx="4" y="199.25">_growend! in array.jl:888</text>
<rect x="108.58" y="203" width="0.95" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="108.58" dx="4" y="214.25">append! in array.jl:981</text>
<rect x="108.58" y="188" width="0.48" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="108.58" dx="4" y="199.25">copyto! in array.jl:304</text>
<rect x="109.06" y="188" width="0.47" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="109.06" dx="4" y="199.25">+ in int.jl:87</text>
<rect x="109.53" y="203" width="10.85" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="109.53" dx="4" y="214.25">vect in array.jl:108</text>
<rect x="109.53" y="188" width="10.38" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="109.53" dx="4" y="199.25">_array_for in array.jl:670</text>
<rect x="109.53" y="173" width="10.38" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="109.53" dx="4" y="184.25">_array_for in array.jl:672</text>
<rect x="109.53" y="158" width="10.38" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="109.53" dx="4" y="169.25">similar in abstractarray.jl:784</text>
<rect x="109.53" y="143" width="10.38" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="109.53" dx="4" y="154.25">similar in abstractarray.jl:785</text>
<rect x="109.53" y="128" width="10.38" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="109.53" dx="4" y="139.25">Array in boot.jl:457</text>
<rect x="109.53" y="113" width="10.38" height="15" rx="2" fill="#FFA400" data-dinfo="./"/>
<text x="109.53" dx="4" y="124.25">Array in boot.jl:448</text>
<rect x="119.91" y="188" width="0.47" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="119.91" dx="4" y="199.25">setindex! in array.jl:843</text>
<rect x="120.38" y="218" width="18.87" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="120.38" dx="4" y="229.25">#advdiff_matrix#21 in In[131]:18</text>
<rect x="120.38" y="203" width="4.24" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="120.38" dx="4" y="214.25">append! in array.jl:980</text>
<rect x="120.38" y="188" width="4.24" height="15" rx="2" fill="#F6A60A" data-dinfo="./"/>
<text x="120.38" dx="4" y="199.25">_growend! in array.jl:888</text>
<rect x="124.62" y="203" width="0.95" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="124.62" dx="4" y="214.25">append! in array.jl:981</text>
<rect x="124.62" y="188" width="0.95" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="124.62" dx="4" y="199.25">copyto! in array.jl:304</text>
<rect x="124.62" y="173" width="0.95" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="124.62" dx="4" y="184.25">_copyto_impl! in array.jl:313</text>
<rect x="124.62" y="158" width="0.95" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="124.62" dx="4" y="169.25">unsafe_copyto! in array.jl:271</text>
<rect x="125.57" y="203" width="13.68" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="125.57" dx="4" y="214.25">+ in arraymath.jl:47</text>
<rect x="125.57" y="188" width="13.68" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="125.57" dx="4" y="199.25">broadcast_preserving_zero_d in broadcast.jl:872</text>
<rect x="125.57" y="173" width="13.68" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="125.57" dx="4" y="184.25">materialize in broadcast.jl:883</text>
<rect x="125.57" y="158" width="13.68" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="125.57" dx="4" y="169.25">copy in broadcast.jl:908</text>
<rect x="125.57" y="143" width="0.47" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="125.57" dx="4" y="154.25">copyto! in broadcast.jl:936</text>
<rect x="125.57" y="128" width="0.47" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="125.57" dx="4" y="139.25">copyto! in broadcast.jl:983</text>
<rect x="125.57" y="113" width="0.47" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="125.57" dx="4" y="124.25">macro expansion in simdloop.jl:77</text>
<rect x="125.57" y="98" width="0.47" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="125.57" dx="4" y="109.25">macro expansion in broadcast.jl:984</text>
<rect x="125.57" y="83" width="0.47" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="125.57" dx="4" y="94.25">setindex! in array.jl:843</text>
<rect x="126.04" y="143" width="13.21" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="126.04" dx="4" y="154.25">similar in broadcast.jl:196</text>
<rect x="126.04" y="128" width="13.21" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="126.04" dx="4" y="139.25">similar in broadcast.jl:197</text>
<rect x="126.04" y="113" width="13.21" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="126.04" dx="4" y="124.25">similar in abstractarray.jl:784</text>
<rect x="126.04" y="98" width="13.21" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="126.04" dx="4" y="109.25">similar in abstractarray.jl:785</text>
<rect x="126.04" y="83" width="13.21" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="126.04" dx="4" y="94.25">Array in boot.jl:465</text>
<rect x="126.04" y="68" width="13.21" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="126.04" dx="4" y="79.25">Array in boot.jl:457</text>
<rect x="126.04" y="53" width="13.21" height="15" rx="2" fill="#FFA000" data-dinfo="./"/>
<text x="126.04" dx="4" y="64.25">Array in boot.jl:448</text>
<rect x="139.25" y="218" width="24.52" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="139.25" dx="4" y="229.25">#advdiff_matrix#21 in In[131]:22</text>
<rect x="139.25" y="203" width="24.52" height="15" rx="2" fill="#E1B2FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="139.25" dx="4" y="214.25">sparse in sparsematrix.jl:952</text>
<rect x="139.25" y="188" width="1.88" height="15" rx="2" fill="#2CA3FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="139.25" dx="4" y="199.25">dimlub in sparsematrix.jl:948</text>
<rect x="139.25" y="173" width="1.88" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="139.25" dx="4" y="184.25">maximum in reducedim.jl:873</text>
<rect x="139.25" y="158" width="1.88" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="139.25" dx="4" y="169.25">#maximum#687 in reducedim.jl:873</text>
<rect x="139.25" y="143" width="1.88" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="139.25" dx="4" y="154.25">_maximum in reducedim.jl:877</text>
<rect x="139.25" y="128" width="1.88" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="139.25" dx="4" y="139.25">#_maximum#689 in reducedim.jl:877</text>
<rect x="139.25" y="113" width="1.88" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="139.25" dx="4" y="124.25">_maximum in reducedim.jl:878</text>
<rect x="139.25" y="98" width="1.88" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="139.25" dx="4" y="109.25">#_maximum#690 in reducedim.jl:878</text>
<rect x="139.25" y="83" width="1.88" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="139.25" dx="4" y="94.25">mapreduce in reducedim.jl:310</text>
<rect x="139.25" y="68" width="1.88" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="139.25" dx="4" y="79.25">#mapreduce#672 in reducedim.jl:310</text>
<rect x="139.25" y="53" width="1.88" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="139.25" dx="4" y="64.25">_mapreduce_dim in reducedim.jl:318</text>
<rect x="139.25" y="38" width="1.88" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="139.25" dx="4" y="49.25">_mapreduce in reduce.jl:415</text>
<rect x="141.13" y="188" width="22.64" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="141.13" dx="4" y="199.25">sparse in sparsematrix.jl:956</text>
<rect x="141.13" y="173" width="8.02" height="15" rx="2" fill="#E1B2FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="141.13" dx="4" y="184.25">sparse in sparsematrix.jl:756</text>
<rect x="141.13" y="158" width="8.02" height="15" rx="2" fill="#F6A60A" data-dinfo="./"/>
<text x="141.13" dx="4" y="169.25">Array in boot.jl:448</text>
<rect x="149.15" y="173" width="14.62" height="15" rx="2" fill="#6FDC91" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="149.15" dx="4" y="184.25">sparse in sparsematrix.jl:767</text>
<rect x="149.15" y="158" width="1.89" height="15" rx="2" fill="#2CA3FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="149.15" dx="4" y="169.25">sparse! in sparsematrix.jl:841</text>
<rect x="149.15" y="143" width="1.89" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="149.15" dx="4" y="154.25">iterate in range.jl:674</text>
<rect x="149.15" y="128" width="1.89" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="149.15" dx="4" y="139.25">== in promotion.jl:410</text>
<rect x="151.04" y="158" width="0.47" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="151.04" dx="4" y="169.25">sparse! in sparsematrix.jl:849</text>
<rect x="151.04" y="143" width="0.47" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="151.04" dx="4" y="154.25">setindex! in array.jl:843</text>
<rect x="151.51" y="158" width="0.47" height="15" rx="2" fill="#2CA3FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="151.51" dx="4" y="169.25">sparse! in sparsematrix.jl:857</text>
<rect x="151.51" y="143" width="0.47" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="151.51" dx="4" y="154.25">&gt; in operators.jl:305</text>
<rect x="151.51" y="128" width="0.47" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="151.51" dx="4" y="139.25">&lt; in int.jl:83</text>
<rect x="151.98" y="158" width="1.42" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="151.98" dx="4" y="169.25">sparse! in sparsematrix.jl:861</text>
<rect x="151.98" y="143" width="0.94" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="151.98" dx="4" y="154.25">&gt;= in operators.jl:352</text>
<rect x="151.98" y="128" width="0.94" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="151.98" dx="4" y="139.25">&lt;= in int.jl:442</text>
<rect x="153.4" y="158" width="0.94" height="15" rx="2" fill="#2CA3FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="153.4" dx="4" y="169.25">sparse! in sparsematrix.jl:862</text>
<rect x="153.4" y="143" width="0.94" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="153.4" dx="4" y="154.25">setindex! in array.jl:843</text>
<rect x="154.34" y="158" width="2.83" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="154.34" dx="4" y="169.25">sparse! in sparsematrix.jl:864</text>
<rect x="154.34" y="143" width="1.89" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="154.34" dx="4" y="154.25">getindex in array.jl:805</text>
<rect x="156.23" y="143" width="0.94" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="156.23" dx="4" y="154.25">iterate in range.jl:674</text>
<rect x="156.23" y="128" width="0.94" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="156.23" dx="4" y="139.25">== in promotion.jl:410</text>
<rect x="157.17" y="158" width="0.47" height="15" rx="2" fill="#2CA3FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="157.17" dx="4" y="169.25">sparse! in sparsematrix.jl:875</text>
<rect x="157.17" y="143" width="0.47" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="157.17" dx="4" y="154.25">fill! in array.jl:335</text>
<rect x="157.17" y="128" width="0.47" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="157.17" dx="4" y="139.25">setindex! in array.jl:843</text>
<rect x="157.64" y="158" width="0.94" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="157.64" dx="4" y="169.25">sparse! in sparsematrix.jl:885</text>
<rect x="157.64" y="143" width="0.94" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="157.64" dx="4" y="154.25">!= in operators.jl:204</text>
<rect x="157.64" y="128" width="0.94" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="157.64" dx="4" y="139.25">== in promotion.jl:410</text>
<rect x="158.58" y="158" width="1.89" height="15" rx="2" fill="#2CA3FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="158.58" dx="4" y="169.25">sparse! in sparsematrix.jl:889</text>
<rect x="158.58" y="143" width="1.89" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="158.58" dx="4" y="154.25">+ in int.jl:87</text>
<rect x="160.47" y="158" width="0.47" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="160.47" dx="4" y="169.25">sparse! in sparsematrix.jl:920</text>
<rect x="160.47" y="143" width="0.47" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="160.47" dx="4" y="154.25">- in int.jl:86</text>
<rect x="160.94" y="158" width="0.48" height="15" rx="2" fill="#2CA3FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="160.94" dx="4" y="169.25">sparse! in sparsematrix.jl:922</text>
<rect x="160.94" y="143" width="0.48" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="160.94" dx="4" y="154.25">getindex in array.jl:805</text>
<rect x="161.42" y="158" width="2.35" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="161.42" dx="4" y="169.25">sparse! in sparsematrix.jl:926</text>
<rect x="161.42" y="143" width="0.94" height="15" rx="2" fill="#E1B2FF" data-dinfo="./"/>
<text x="161.42" dx="4" y="154.25">setindex! in array.jl:843</text>
<rect x="162.36" y="143" width="1.41" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="162.36" dx="4" y="154.25">iterate in range.jl:674</text>
<rect x="162.36" y="128" width="1.41" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="162.36" dx="4" y="139.25">== in promotion.jl:410</text>
<rect x="163.77" y="218" width="0.48" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="163.77" dx="4" y="229.25">repeat_outer in abstractarraymath.jl:345</text>
<rect x="164.72" y="233" width="0.47" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="164.72" dx="4" y="244.25">repeat##kw in abstractarraymath.jl:266</text>
<rect x="165.19" y="248" width="774.53" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="165.19" dx="4" y="259.25">assemble_and_solve in In[133]:6</text>
<rect x="165.19" y="233" width="0.47" height="15" rx="2" fill="#E1B2FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="165.19" dx="4" y="244.25">\ in linalg.jl:1564</text>
<rect x="165.19" y="218" width="0.47" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="165.19" dx="4" y="229.25">ishermitian in sparsematrix.jl:3428</text>
<rect x="165.19" y="203" width="0.47" height="15" rx="2" fill="#6FDC91" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="165.19" dx="4" y="214.25">is_hermsym in sparsematrix.jl:3437</text>
<rect x="165.19" y="188" width="0.47" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="165.19" dx="4" y="199.25">copy in array.jl:349</text>
<rect x="165.66" y="233" width="774.06" height="15" rx="2" fill="#6FDC91" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SparseArrays/src/"/>
<text x="165.66" dx="4" y="244.25">\ in linalg.jl:1567</text>
<rect x="165.66" y="218" width="70.28" height="15" rx="2" fill="#2CA3FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/LinearAlgebra/src/"/>
<text x="165.66" dx="4" y="229.25">\ in factorization.jl:104</text>
<rect x="165.66" y="203" width="70.28" height="15" rx="2" fill="#E1B2FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="165.66" dx="4" y="214.25">ldiv! in umfpack.jl:497</text>
<rect x="165.66" y="188" width="70.28" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="165.66" dx="4" y="199.25">ldiv! in umfpack.jl:510</text>
<rect x="165.66" y="173" width="70.28" height="15" rx="2" fill="#E1B2FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="165.66" dx="4" y="184.25">_Aq_ldiv_B! in umfpack.jl:527</text>
<rect x="165.66" y="158" width="70.28" height="15" rx="2" fill="#2CA3FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="165.66" dx="4" y="169.25">_AqldivB_kernel! in umfpack.jl:532</text>
<rect x="165.66" y="143" width="70.28" height="15" rx="2" fill="#E1B2FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="165.66" dx="4" y="154.25">solve! in umfpack.jl:366</text>
<rect x="235.94" y="218" width="703.78" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="235.94" dx="4" y="229.25">lu in umfpack.jl:155</text>
<rect x="235.94" y="203" width="3.31" height="15" rx="2" fill="#6FDC91" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="235.94" dx="4" y="214.25">#lu#1 in umfpack.jl:156</text>
<rect x="235.94" y="188" width="0.95" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="235.94" dx="4" y="199.25">copy in array.jl:349</text>
<rect x="236.89" y="188" width="2.36" height="15" rx="2" fill="#00B58B" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="236.89" dx="4" y="199.25">decrement in SuiteSparse.jl:15</text>
<rect x="236.89" y="173" width="1.41" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="236.89" dx="4" y="184.25">copy in array.jl:349</text>
<rect x="238.3" y="173" width="0.95" height="15" rx="2" fill="#E1B2FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="238.3" dx="4" y="184.25">decrement! in SuiteSparse.jl:12</text>
<rect x="238.3" y="158" width="0.47" height="15" rx="2" fill="#00B58B" data-dinfo="./"/>
<text x="238.3" dx="4" y="169.25">setindex! in array.jl:843</text>
<rect x="238.77" y="158" width="0.48" height="15" rx="2" fill="#2CA3FF" data-dinfo="./"/>
<text x="238.77" dx="4" y="169.25">iterate in range.jl:674</text>
<rect x="238.77" y="143" width="0.48" height="15" rx="2" fill="#6FDC91" data-dinfo="./"/>
<text x="238.77" dx="4" y="154.25">== in promotion.jl:410</text>
<rect x="239.25" y="203" width="700.47" height="15" rx="2" fill="#E1B2FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="239.25" dx="4" y="214.25">#lu#1 in umfpack.jl:161</text>
<rect x="239.25" y="188" width="700.47" height="15" rx="2" fill="#2CA3FF" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="239.25" dx="4" y="199.25">umfpack_numeric! in umfpack.jl:324</text>
<rect x="239.25" y="173" width="293.39" height="15" rx="2" fill="#6FDC91" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="239.25" dx="4" y="184.25">#umfpack_numeric!#13 in umfpack.jl:325</text>
<rect x="239.25" y="158" width="293.39" height="15" rx="2" fill="#F6A60A" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="239.25" dx="4" y="169.25">umfpack_symbolic! in umfpack.jl:304</text>
<rect x="532.64" y="173" width="407.08" height="15" rx="2" fill="#FFA000" data-dinfo="/build/julia/src/julia-1.6.3/usr/share/julia/stdlib/v1.6/SuiteSparse/src/"/>
<text x="532.64" dx="4" y="184.25">#umfpack_numeric!#13 in umfpack.jl:327</text>
</g></g>

</svg>
</body>
</html>

# What's left?

* Symmetric Dirichlet boundary conditions
* Symmetric Neumann boundary conditions
* Verification with method of manufactured solutions
* Non-uniform grids
* Upwinding for advection-dominated problems
* Variable coefficients
* Time-dependent problems
* Fast algebraic solvers


| |Pierre Proulx, ing, professeur|
|:---|:---|
|Département de génie chimique et de génie biotechnologique |** GCH200-Phénomènes d'échanges I **|

### Section10.4, chauffage par effet Brinkman: source de chauffage causé par la viscosité

>
>> Ici on traitera le problème de façon légèrement différente de Transport Phenomena, en effet, pour traiter ce problème on peut aller chercher les équations générales de l'annexe B.9.

>>Regardons donc l'équation B.9-1 qui est utilisée puisqu'on suppose que le système a une courbure négligeable et peut être traité en coordonnées cartésiennes.
Tous les termes de gauche sont nuls car:

>>> On veut le résultat en état de régime, donc la dérivée en fonction du temps est nulle.

>>> Il n'y a pas de vitesse dans la direction x ni dans la direction y, et il n'y a pas de gradient de température dans la direction z.

>>> Le seul terme dans les dérivées de la température qui reste à droite est celui dans la direction x, et le terme de source $\Phi$, donc:

>>>> ###  $k \frac {\partial^2 T}{\partial x^2}+\mu \Phi_v = 0$

>> Le terme de source visqueuse est obtenu par l'annexe B7

>

qui peut sembler très complexe (en effet) mais tous les termes s'annulent sauf celui du gradient de $v_z$ en fonction de x donc:

>>>> ### $\Phi_v =\big(\frac {\partial v_z}{\partial x} \big)^2 $
>> L'équation à résoudre sera donc directement:

>>>> ### $k \frac {\partial^2 T}{\partial x^2}+ \mu \big(\frac {\partial v_z}{\partial x} \big)^2= 0$

> Et puisque le profil de vitesse vous a déjà été donné sur la figure 10.4-2, on pourra résoudre

```python
#
# Pierre Proulx
#
# Préparation de l'affichage et des outils de calcul symbolique
#
import sympy as sp
from IPython.display import *
sp.init_printing(use_latex=True)
%matplotlib inline
```

```python
# Paramètres, variables et fonctions
#
mu,k,b,Tb,T0,vb,x=sp.symbols('mu k b T_b T_0 v_b x')
T=sp.Function('T')(x)
v=sp.Function('v')(x)
```

```python
v=vb*x/b
eq=k*T.diff(x,x)+mu*v.diff(x)**2
display(eq)
```

```python
T=sp.dsolve(eq)
display('Le profil de température, solution générale avec les constantes inconnues',T)
```

```python
condition_1=sp.Eq(T.rhs.subs(x,0),T0)
condition_2=sp.Eq(T.rhs.subs(x,b),Tb)
display('Les conditions aux limites',condition_1, condition_2)
constantes=sp.solve([condition_1,condition_2],sp.symbols('C1,C2'))
display('Les constantes après solution',constantes)
T=T.subs(constantes)
display('Le profil de température final',T)
```

```python
### Tracage
T=T.rhs
dico={'T_0':0,'T_b':10,'k':0.1,'mu':0.01,'b':0.01,'v_b':10}
T1=T.subs(dico)
dico={'T_0':0,'T_b':10,'k':0.1,'mu':0.05,'b':0.01,'v_b':10}
T2=T.subs(dico)
dico={'T_0':0,'T_b':10,'k':0.1,'mu':0.1,'b':0.01,'v_b':10}
T3=T.subs(dico)
dico={'T_0':0,'T_b':10,'k':0.1,'mu':0.15,'b':0.01,'v_b':10}
T4=T.subs(dico)
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize']=10,8
p=sp.plot((T1,(x,0,0.01)),(T2,(x,0,0.01)),(T3,(x,0,0.01)),(T4,(x,0,0.01)),
       legend=True,ylabel='T',xlabel='x',show=False)
p[0].label='mu=0.01'
p[1].label='.05'
p[2].label='.1'
p[3].label='.15'
p[0].line_color='blue'
p[1].line_color='red'
p[2].line_color='green'
p[3].line_color='black'
p.show()
```


#First-Order IIR Filter
The first order IIR filter is the simplest case of IIR filter. Both numerator (b) and denominator (a) of the transfer function of the filter are polynomials of order 1.

##Modules

```python
from numpy import pi, abs,array, arccos, angle
from scipy.signal import freqz
import matplotlib.pyplot as plt
```

## Filter *Coefficients*
```
            jw                 -jw              -jwM
   jw    B(e  )    b[0] + b[1]e    + ... + b[M]e
H(e  ) = ------ = -----------------------------------
            jw                 -jw              -jwN
         A(e  )    a[0] + a[1]e    + ... + a[N]e
```
\begin{align}
H(z)=\frac{\alpha }{1-(1-\alpha )z^{-1}}
\end{align}

\begin{align}
H(z)=\frac{b_{0}}{1+(-a_{1})z^{-1}}=\frac{b_{0}z}{z+(-a_{1})}
\end{align}

```python
b = array([1/10])
a = array([1, -0.9]) ;a1=0.9
```

##Magnitude response

```python
w, h = freqz(b,a, worN=1024)
plt.figure(dpi=130)
plt.plot(w, abs(h), linewidth=2)
plt.xlabel('Normalized frequency')
plt.ylabel('Magnitude')
plt.title('Magnitude Response $a_{1}=%.2f$'%-a[1])
plt.xlim(0, pi)
wc=arccos((((a[1]**2)-((b[0]**2)*2)+1)/(2*-a[1])))
plt.axvline(wc, color='red')
plt.axhline(0.707, color='green')
plt.text(pi/2-0.2, .707+0.01, r'$\left | H(\omega) \right |=.707$',fontsize=8)
plt.text(wc-0.06, 0.3, r'$\omega=%.2f$' %wc, rotation=90,fontsize=8)
plt.yticks([0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1])
plt.grid(True)
plt.show()
```

##Phase response

```python
plt.figure(dpi=130)
plt.plot(w, 180 * angle(h) / pi)
plt.title('Phase Response $a_{1}=%.2f$'%-a[1])
plt.xlabel('Normalized frequency')
plt.ylabel('Phase [°]')
plt.grid(True)
plt.show()
```

# Author
 Marcelo Gutiérrez


# Boundary Value Problem Example - Heated Rod
## CH EN 2450 - Numerical Methods
**Prof. Tony Saad (www.tsaad.net)
Department of Chemical Engineering
University of Utah**

```python
%matplotlib inline
%config InlineBackend.figure_format = 'svg'
import matplotlib.pyplot as plt
import numpy as np
```

# Dirichlet Boundary Conditions

The coefficient matrix looks like
\begin{equation}
\left[ {\begin{array}{*{20}{c}}
1&0&0& \cdots &0\\
1& - \left( 2 + \Delta {x^2}H \right)&1&0& \vdots \\
0& \ddots & \ddots & \ddots &0\\
 \vdots &0&1& - \left( 2 + \Delta x^2H \right)&1\\
0& \cdots &0&0&1
\end{array}} \right]
\left( {\begin{array}{*{20}{c}}
T_1\\
\begin{array}{l}
T_2\\
T_3
\end{array}\\
\begin{array}{l}
 \vdots \\
 \vdots
\end{array}\\
T_{n - 1}\\
T_n
\end{array}} \right)
= \left( {\begin{array}{*{20}{c}}
T_\rm{left}\\
- T_\infty \Delta {x^2}H\\
 \vdots \\
- T_\infty\Delta {x^2}H\\
T_\rm{right}
\end{array}} \right)
\end{equation}

```python
n = 200 # number of desired grid points
L = 1.0 # rod length
Tleft = 300
Tright = 300
Tinf = 500
h = 1 # heat transfer coefficient, due to advection of surrounding air

x = np.linspace(?,?,?) # create an array for the grid points x_i
dx = x[1] - x[0] # calculate grid spacing (grid spacing is constant here)

# create lower diagonal vector
ld = ?
ld[?] = ? # fix the end point

# create main diagonal vector
d = ?
d[0] = 1.0 # fix the end point
d[-1] = 1.0 # fix the end point

# create upper diagonal vector
ud = ?
ud[?] = ? # fix the end point

# combine those into a matrix
A = np.diag(ld,?) + np.diag(d, ?) + np.diag(ud, ?)

# build the RHS array
rhs = ?
rhs[0] = ? # fix the left boundary
rhs[-1] = ? # fix the right boundary

# solve the system of equations
sol = np.linalg.solve(?, ?)

plt.plot(?, ?)
plt.grid()
```

```python
temperature = np.tile(sol,(40,1))
plt.imshow(temperature,cmap='jet')
```

    <matplotlib.image.AxesImage at 0x119ecc910>

topics/boundary value problems/Boundary Value Problem Example - Heated Rod - With Gaps.ipynb


```python
import sympy as sp

sp.init_printing()
```

# Math 5A -- Damped Harmoic Oscillator Example

```python
# define symbols
gamma, omega_0, t, delta, F_0, alpha = sp.symbols(
    r"\gamma, \omega_0, t, \delta, F_0, \alpha"
)

# define function
x = sp.Function("x")(t)

# define differential equation
d = lambda x : sp.Derivative(x[0], t, x[1])
diff_eq = sp.Eq(
    d([x, 2]) + 2 * gamma * d([x, 1]) + omega_0**2 * x, F_0 * sp.exp(- alpha * t)
)
diff_eq
```

```python
# define the solution
x_sol = F_0 / sp.sqrt(omega_0**2 - gamma**2) / sp.sqrt(omega_0**2 + alpha**2 - 2 * alpha * gamma
    ) * sp.sin(sp.sqrt(omega_0**2 - gamma**2) * t - delta) * sp.exp(-gamma * t
    ) +  F_0 / (omega_0**2 + alpha**2 - 2 * alpha * gamma) * sp.exp(-alpha * t)
x_sol
```

```python
# you see how crazy it is
diff_eq.subs(x, x_sol)
```

```python
# doing the differential
diff_eq.subs(x, x_sol).doit()
```

```python
# for the left hand side
sp.simplify(diff_eq.subs(x, x_sol).lhs.doit())
```

Oh My God!! It's real! Those LHS craps is just $F_0 e^{-\alpha t}$.

Math HW 5A -- Damped Harmonic Oscillator Example.ipynb