# How close is close enough?

This is based on [Greg Wilson's post on testing](http://software-carpentry.org/blog/2014/10/why-we-dont-teach-testing.html), but avoids the big, difficult questions. Instead I focus on his comment about "close enough" in [the full phugoid model notebook](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_03_PhugoidFullModel.ipynb).

See also [this page from NASA on grid convergence](http://www.grc.nasa.gov/WWW/wind/valid/tutorial/spatconv.html). Also [this paper by Liu](http://ocw.mit.edu/courses/mathematics/18-304-undergraduate-seminar-in-discrete-mathematics-spring-2006/projects/xtrpltn_liu_xpnd.pdf), which is rather mathematical, on Richardson extrapolation and its extensions.

## Round off

Let's start by reminding ourselves how round-off error comes in to numerical calculations. As an example, remember that $\sin(2\pi) = 0$, and in fact $\sin(2 k \pi)=0$ for any integer $k$.

We'll use `numpy` for the calculations.

```python
%matplotlib notebook
import numpy
from matplotlib import pyplot
from numpy import pi, sin, arange
```

```python
ks = arange(10)
sins = sin(2.0*ks*pi)
for k, s in zip(ks, sins):
    print("sin(2 k pi) evaluates to {:.3g} when k={}.".format(s, k))
print("Adding these all up (result should be 0) we get {}.".format(sins.sum()))
```

    sin(2 k pi) evaluates to 0 when k=0.
    sin(2 k pi) evaluates to -2.45e-16 when k=1.
    sin(2 k pi) evaluates to -4.9e-16 when k=2.
    sin(2 k pi) evaluates to -7.35e-16 when k=3.
    sin(2 k pi) evaluates to -9.8e-16 when k=4.
    sin(2 k pi) evaluates to -1.22e-15 when k=5.
    sin(2 k pi) evaluates to -1.47e-15 when k=6.
    sin(2 k pi) evaluates to -1.71e-15 when k=7.
    sin(2 k pi) evaluates to -1.96e-15 when k=8.
    sin(2 k pi) evaluates to -2.2e-15 when k=9.
    Adding these all up (result should be 0) we get -1.1021821192326179e-14.

So we had ten different numerical calculations with errors between $0$ and $\sim 2 \times 10^{-15}$, which when combined lead to a total error $\sim 10^{-14}$. This illustrates the standard result:

Summing $N$ calculations with errors $\delta_i$, where $\delta_i \le \delta = \max_i \delta_i$, leads to a total error ${\cal E}$ which is bounded by ${\cal E} \le N \delta$.

## Going beyond sums

We'll use a very simple initial value problem (as the calculations are faster):

$$
\begin{equation}
  y' = -\sin(x), \qquad y(0) = 1
\end{equation}
$$

which has the solution $y = \cos(x)$. We'll solve this using Euler's method, as in the original notebook:

$$
\begin{equation}
  y_{n+1} = y_n - h \sin(x_n), \qquad y_0 = 1
\end{equation}
$$

where $h$ is the grid spacing and $x_n = n h$, with $n = 0, 1, \dots$ the grid step.

```python
def simple_euler(h, N):
    """
    Solve the problem y' = -sin(x), y(0) = 1 using Euler's method.

    Parameters
    ----------

    h : float
        Grid spacing
    N : int
        Number of steps

    Returns
    -------

    Y : float
        y(Nh) as approximated by Euler's method
    """

    # Initial data
    Y = 1.0
    x = 0.0
    for n in range(N):
        Y -= h*sin(x)
        x += h

    return Y
```

So, how good is this method? Check by comparing against the exact solution when $X=1$.

```python
X = 1.0
N_all = numpy.array([2**i for i in range(3, 20)])
h_all = X / N_all
Y_exact = numpy.cos(X)
Y_approx = numpy.zeros_like(h_all)
Y_errors = numpy.zeros_like(h_all)
for i, N in enumerate(N_all):
    h = h_all[i]
    Y_approx[i] = simple_euler(h, N)
    Y_errors[i] = numpy.abs(Y_approx[i] - Y_exact)
```

```python
pyplot.loglog(h_all, Y_errors, 'kx')
pyplot.xlabel(r'$h$')
pyplot.ylabel('Error');
```

So, what should we expect? If implemented correctly, we know that Euler's method behaves as $\text{Error} \propto h + {\cal O}(h^2)$. As a first guess we drop the higher order terms, giving $\text{Error} = c_1 h$, or $\log(\text{Error}) = \log(h) + \text{const}$.

We can then compute the best fit line through the data and see if it matches this assumption.

```python
simple_p = numpy.polyfit(numpy.log(h_all), numpy.log(Y_errors), 1)
pyplot.loglog(h_all, Y_errors, 'kx', label='Data')
pyplot.loglog(h_all, numpy.exp(simple_p[1])*h_all**(simple_p[0]), 'b-', label='Fit, slope={:.4f}'.format(simple_p[0]))
pyplot.legend(loc='upper left')
pyplot.xlabel(r'$h$')
pyplot.ylabel('Error');
```

So, the best fit line matches the expected slope (1) to better than $0.3\%$. Is this good enough?

First, let's do a sanity check. Why do we believe that the slope shouldn't be *exactly* $1$? It's because of our assumption: that $\text{Error} \propto h + {\cal O}(h^2)$, and that we could ignore the higher order terms. The assumption that the error takes this form is essentially saying that we've implemented the algorithm correctly (which is what we're trying to check!). The assumption that we can ignore the higher order terms is more reasonable when $h$ is small, like $10^{-5}$, but not when $h \sim 10^{-1}$. So the slope should get closer to $1$ if we ignore the results for larger $h$. Let's do that calculation:

```python
for i in range(1, len(Y_errors)-2):
    partial_p = numpy.polyfit(numpy.log(h_all[i:]), numpy.log(Y_errors[i:]), 1)
    print("The slope, when ignoring {} entries, is {:.6f}. (slope-1)={:.4g}".format(i+1, partial_p[0], partial_p[0]-1.0))
```

    The slope, when ignoring 2 entries, is 1.000313. (slope-1)=0.0003133
    The slope, when ignoring 3 entries, is 1.000176. (slope-1)=0.0001758
    The slope, when ignoring 4 entries, is 1.000099. (slope-1)=9.92e-05
    The slope, when ignoring 5 entries, is 1.000056. (slope-1)=5.638e-05
    The slope, when ignoring 6 entries, is 1.000032. (slope-1)=3.23e-05
    The slope, when ignoring 7 entries, is 1.000019. (slope-1)=1.867e-05
    The slope, when ignoring 8 entries, is 1.000011. (slope-1)=1.09e-05
    The slope, when ignoring 9 entries, is 1.000006. (slope-1)=6.439e-06
    The slope, when ignoring 10 entries, is 1.000004. (slope-1)=3.851e-06
    The slope, when ignoring 11 entries, is 1.000002. (slope-1)=2.336e-06
    The slope, when ignoring 12 entries, is 1.000001. (slope-1)=1.439e-06
    The slope, when ignoring 13 entries, is 1.000001. (slope-1)=9.026e-07
    The slope, when ignoring 14 entries, is 1.000001. (slope-1)=5.777e-07
    The slope, when ignoring 15 entries, is 1.000000. (slope-1)=3.79e-07

That's good news. We could just fit the final few entries to get closer to the expected slope, but we're still not answering how close is "close enough"

One additional question that is quite important. What's the effect of changing how far we integrate, by changing $X$? Let's make $X$ smaller.

```python
X = 1.0e-5
h_all = X / N_all
Y_exact_short = numpy.cos(X)
Y_approx_short = numpy.zeros_like(h_all)
Y_errors_short = numpy.zeros_like(h_all)
for i, N in enumerate(N_all):
    h = h_all[i]
    Y_approx_short[i] = simple_euler(h, N)
    Y_errors_short[i] = numpy.abs(Y_approx_short[i] - Y_exact_short)
```

```python
simple_p_short = numpy.polyfit(numpy.log(h_all[:-6]), numpy.log(Y_errors_short[:-6]), 1)
pyplot.loglog(h_all, Y_errors_short, 'kx', label='Data')
pyplot.loglog(h_all[:-6], numpy.exp(simple_p_short[1])*h_all[:-6]**(simple_p_short[0]), 'b-', label='Fit, slope={:.4f}'.format(simple_p_short[0]))
pyplot.legend(loc='lower right')
pyplot.xlabel(r'$h$')
pyplot.ylabel('Error');
```

We see that the algorithm converges as expected until $h$ is so small that floating point round-off errors become important. Below that point the error of each individual step does not depend on $h$ (as the truncation error of the algorithm is less than floating point round-off) but is essentially random, and has the magnitude of floating point round-off. These individual errors will then add up. To check this, we can look at the contribution from floating point errors at each step:

```python
floating_error_step = numpy.spacing(1.0e-3)
floating_error = floating_error_step * N_all
pyplot.loglog(h_all, Y_errors_short, 'kx', label='Data')
pyplot.loglog(h_all, floating_error, 'b-', label='Floating point error contribution')
pyplot.legend(loc='upper left')
pyplot.xlabel(r'$h$')
pyplot.ylabel('Error');
```

Let's check the truncation error instead. First, redo the analysis to find out *exactly* what that should look like.

```python
import sympy
sympy.init_printing()
y = sympy.Function('y')
f = sympy.Function('f')
x, h, b= sympy.symbols('x, h, b')
y_n_p_1 = y(x) + h * f(x, y(x))
```

```python
truncation_error = sympy.series(y(h),h)-y_n_p_1.subs(x, 0)
truncation_error = truncation_error.subs(f(0, y(0)), sympy.Subs(sympy.Derivative(y(x), x),(x,),(0,)))
truncation_error
```

So the truncation error should be the sum of each of these terms: for $h$ sufficiently small the higher order terms should have no effect at all. We can check, using that the derivatives of $\cos(x)$ at $x=0$ are alternatively $1$ and $0$ in magnitude:

```python
h_truncation = numpy.array([2**(-i) for i in range(4,20)])
Y_truncation_error = numpy.zeros_like(h_truncation)
Y_expected_truncation_error = numpy.zeros_like(h_truncation)
for i, h in enumerate(h_truncation):
    Y_truncation_error[i] = numpy.abs(simple_euler(h, 1) - numpy.cos(h))
    Y_expected_truncation_error[i] = h**2/2.0 - h**4/24.0 + h**6/720.0 - h**8/40320.
```

```python
fig=pyplot.figure(figsize=(12,6))
ax1=fig.add_subplot(121)
ax1.loglog(h_truncation, Y_truncation_error, 'kx', label='Data')
ax1.loglog(h_truncation, Y_expected_truncation_error, 'b-', label='Expected truncation error')
ax1.legend(loc='upper left')
ax1.set_xlabel(r'$h$')
ax1.set_ylabel('Error');
ax2=fig.add_subplot(122)
ax2.loglog(h_truncation, numpy.abs(Y_truncation_error-Y_expected_truncation_error), 'kx', label='Difference in data')
ax2.loglog(h_truncation, numpy.minimum(numpy.spacing(1.0),numpy.spacing(h_truncation**4)/numpy.spacing(1.0)), 'b-', label='Floating point limit')
ax2.legend(loc='lower right')
ax2.set_xlabel(r'$h$')
ax2.set_ylabel('Difference in truncation error');
```

We note that for $h$ sufficiently small the limitation is no longer the floating point error, but instead falls off like $h^4$.  This is (likely) because the $h^4$ term is not correctly captured in one of the calculations (as $h \lesssim 10^{-4}$ we have $h^4$ less than floating point round-off). If we went to larger $h$ we would again find that the "expected truncation error" doesn't match, as more terms in the expansion would be needed.

## Richardson extrapolation

Let's go back to a different application of the "close enough" issue. What do we actually want to use the convergence plot *for*?

At its heart, Euler's method is approximating the exact solution $y(x)$ at a point $x=X$ to some degree of accuracy. We care about two things: the exact value $y(X)$ and the error made in approximating it, $E_h$. As, in general, we don't know the exact solution, we can't know the value of the error. However, we can show how it depends on the grid step $h$, and from that we can approximately calculate how big it is for any system. That is, we can put *error bars* on our answer. For those that like the "computation as experiment" analogy, this is exactly what we should be doing.

To do this, we use *Richardson extrapolation*. We compute the solution with Euler's method twice, to get $y^{(h)}$ and $y^{(2h)}$: two approximations to the solution $y(X)$ computed with different (but related) step lengths. Using the error analysis as a basis, we *assume* that

$$
\begin{equation}
  y(X) = y^{(h)} + C h.
\end{equation}
$$

Given the two calculations $y^{(h)}$ and $y^{(2h)}$, and our assumption, we can solve for the exact solution $y(X)$ and for the error $C h$ to find

$$
\begin{equation}
  y(X) = 2 y^{(h)} - y^{(2h)}, \qquad C h = y^{(2h)} - y^{(h)}.
\end{equation}
$$

So, we can go back to our original problem and look at $y(1)$, the Richardson extrapolated "exact" solution, and the error bound that results.

```python
X = 1.0
N_all = numpy.array([2**i for i in range(3, 20)])
h_all = X / N_all
Y_richardson = numpy.zeros_like(Y_approx)
Y_richardson_error = numpy.zeros_like(Y_approx)
for i in range(1, len(h_all)):
    Y_richardson[i] = 2.0*Y_approx[i] - Y_approx[i-1]
    Y_richardson_error[i] = abs(Y_approx[i-1] - Y_approx[i])
```

```python
fig=pyplot.figure(figsize=(8,6))
ax1=fig.add_subplot(111)
ax1.loglog(h_all[1:], Y_errors[1:], 'bo', label='Data errors')
ax1.set_xscale("log", nonposx='clip')
ax1.set_yscale("log", nonposy='clip')
ax1.errorbar(h_all[1:], numpy.abs(Y_exact - Y_richardson[1:]), yerr=Y_richardson_error[1:],
             lolims=True, marker='x', color='k', ls='None', label='Richardson extrapolation and error estimate')
ax1.legend(loc='lower right')
ax1.set_xlabel(r'$h$')
ax1.set_ylabel('Error');
```

We see that the error bar found from Richardson extrapolation pretty much matches up with the error in the original calculation, as expected. So, using the two best results (i.e., those with the highest resolution, or smallest $h$) we can say that

```python
print("y(1) = {} ± {}.".format(Y_richardson[-1], Y_richardson_error[-1]))
```

    y(1) = 0.5403023058678328 ± 8.024897067970826e-07.

However, this analysis is all based on the *assumption* that the error is exactly proportional to $h$. We know this isn't true; we're neglecting higher order terms. We see this because we measure a convergence rate that isn't exactly $1$. So we need a different model for the behaviour of our algorithm. We could add more terms to the error (as expected), but this leads to more parameters to fit, which is bad (["with four parameters I can fit an elephant"](http://en.wikiquote.org/wiki/John_von_Neumann) and so on). Instead we keep a single error term, but write $y(X) = y^{(h)} + C h^s$ where $s$ is measured from the data (above we have $1.0006$). This leads to the Richardson extrapolation formulas

$$
\begin{equation}
  y(X) = \frac{2^s y^{(h)} - y^{(2h)}}{2^s - 1}, \qquad C h = \frac{y^{(2h)} - y^{(h)}}{2^s - 1}.
\end{equation}
$$

Applying this assumption to the data, we get a new set of error bars:

```python
Y_richardson_measured_s = numpy.zeros_like(Y_approx)
Y_richardson_error_measured_s = numpy.zeros_like(Y_approx)
for i in range(1, len(h_all)):
    Y_richardson_measured_s[i] = (2.0**(simple_p[0])*Y_approx[i] - Y_approx[i-1])/(2.0**(simple_p[0])-1.0)
    Y_richardson_error_measured_s[i] = abs(Y_approx[i-1] - Y_approx[i])/(2.0**(simple_p[0])-1.0)
print("y(1) = {} ± {}.".format(Y_richardson_measured_s[-1], Y_richardson_error_measured_s[-1]))
print("Difference between predicted exact values is {:.4g}.".format(abs(Y_richardson[-1]-Y_richardson_measured_s[-1])))
```

    y(1) = 0.5403023064917923 ± 8.018657471901381e-07.
    Difference between predicted exact values is 6.24e-10.

You have to look quite hard to see the difference. But the key point here is that the two exact values predicted by the different assumptions lie within each other's error bars.

Let's repeat this analysis using the data from the phugoid problem sheet. In this case we have a more complex system, a longer integration error (leading to a larger error), worse resolution (leading to a larger error), fewer data points, and no knowledge of the exact solution. However, we are still using Euler's method, so we expect the same behaviour for the error.

I explicitly give the values of the errors here rather than the code, to save time. I modified the range of values considered to

`dt_values = numpy.array([0.1*2**(-i) for i in range(8)])`

to ensure that nice factor 2 between each resolution. I then computed the differences between each using the `get_diffgrid` function, and did the best fit line getting a slope of $1.21154575$ with the data below:

```python
dt_values = numpy.array([ 0.1, 0.05, 0.025, 0.0125, 0.00625, 0.003125, 0.0015625])
diffgrid = numpy.array([ 25.4562819, 10.52418949, 4.75647465, 2.20894037, 1.01024986, 0.42865587, 0.14217568])
s = 1.21154575

Y_richardson_phugoid = numpy.zeros_like(diffgrid)
Y_richardson_phugoid_error = numpy.zeros_like(diffgrid)
Y_richardson_phugoid_measured_s = numpy.zeros_like(diffgrid)
Y_richardson_phugoid_error_measured_s = numpy.zeros_like(diffgrid)
for i in range(1, len(diffgrid)):
    Y_richardson_phugoid[i] = (2.0*diffgrid[i] - diffgrid[i-1])
    Y_richardson_phugoid_measured_s[i] = (2.0**(s)*diffgrid[i] - diffgrid[i-1])/(2.0**(s)-1.0)
    Y_richardson_phugoid_error[i] = abs(diffgrid[i-1] - diffgrid[i])
    Y_richardson_phugoid_error_measured_s[i] = abs(diffgrid[i-1] - diffgrid[i])/(2.0**(s)-1.0)
print("Phugoid limit, standard assumption = {} ± {}.".format(Y_richardson_phugoid[-1], Y_richardson_phugoid_error[-1]))
print("Phugoid limit, measured slope = {} ± {}.".format(Y_richardson_phugoid_measured_s[-1], Y_richardson_phugoid_error_measured_s[-1]))
print("Difference between predicted limits is {:.4g}.".format(abs(Y_richardson_phugoid[-1]-Y_richardson_phugoid_measured_s[-1])))
```

    Phugoid limit, standard assumption = -0.14430451 ± 0.28648019.
    Phugoid limit, measured slope = -0.07553820347702735 ± 0.21771388347702736.
    Difference between predicted limits is 0.06877.

We see the errors are much larger, but that the difference between the limiting values is within the predicted error bars of either result. Therefore the assumption that the algorithm is behaving as the idealized Euler's method does is *close enough* that the predicted result lies within the predicted error bars.

So how close *is* close enough? We need

$$
\begin{equation}
\left| \frac{2^s y^{(h)} - y^{(2h)}}{2^s - 1} - \left( 2 y^{(h)} - y^{(2h)} \right) \right| \le \left| \frac{y^{(2h)} - y^{(h)}}{2^s - 1} \right|.
\end{equation}
$$

```python
yh, y2h, s = sympy.symbols('y^h, y^{2h}, s')
Eq1 = sympy.Eq((2**s*yh-y2h)/(2**s-1)-(2*yh-y2h) , (y2h-yh)/(2**s-1))
sympy.solve(Eq1, s)
```

There's another root to check:

```python
Eq2 = sympy.Eq((2**s*yh-y2h)/(2**s-1)-(2*yh-y2h) , -(y2h-yh)/(2**s-1))
sympy.solve(Eq2, s)
```

So the threshold is $s = \log(3)/\log(2) \simeq 1.585$.

But there's also the other interval, for which we need

$$
\begin{equation}
\left| \frac{2^s y^{(h)} - y^{(2h)}}{2^s - 1} - \left( 2 y^{(h)} - y^{(2h)} \right) \right| \le \left| y^{(2h)} - y^{(h)} \right|.
\end{equation}
$$

```python
Eq3 = sympy.Eq((2**s*yh-y2h)/(2**s-1)-(2*yh-y2h) , (y2h-yh))
sympy.solve(Eq3, s)
```

```python
Eq4 = sympy.Eq((2**s*yh-y2h)/(2**s-1)-(2*yh-y2h) , -(y2h-yh))
sympy.solve(Eq4, s)
```

This gives the lower bound of $\simeq 0.585$.

This is very specific to Euler's method. What if we're using a better method with convergence rate $s_e$, so that the idealized behaviour of the algorithm is $y(X) = y^{(h)} + C h^{s_e}$? In that case, if we measure a convergence rate of $s_m$, then the results are close enough when

$$
\begin{equation}
\left| \frac{2^{s_e} y^{(h)} - y^{(2h)}}{2^{s_e} - 1} - \frac{2^{s_m} y^{(h)} - y^{(2h)}}{2^{s_m} - 1} \right| \le \left| \frac{y^{(2h)} - y^{(h)}}{2^{s_e} - 1} \right|
\end{equation}
$$

and

$$
\begin{equation}
\left| \frac{2^{s_e} y^{(h)} - y^{(2h)}}{2^{s_e} - 1} - \frac{2^{s_m} y^{(h)} - y^{(2h)}}{2^{s_m} - 1} \right| \le \left| \frac{y^{(2h)} - y^{(h)}}{2^{s_m} - 1} \right| .
\end{equation}
$$

```python
yh, y2h, se, sm = sympy.symbols('y^h, y^{2h}, s_e, s_m')
Eq5 = sympy.Eq((2**sm*yh-y2h)/(2**sm-1)-(2**se*yh-y2h)/(2**se-1) , (y2h-yh)/(2**sm-1))
sympy.solve(Eq5, sm)
```

```python
Eq6 = sympy.Eq((2**sm*yh-y2h)/(2**sm-1)-(2**se*yh-y2h)/(2**se-1) , -(y2h-yh)/(2**se-1))
sympy.solve(Eq6, sm)
```

So we can see how the bound changes with increased accuracy of the ideal algorithm:

```python
s = numpy.arange(1,10)
upper_limit = numpy.log(2.0**(s+1)-1.0)/numpy.log(2.0)
lower_limit = numpy.log(2.0**(s-1)+0.5)/numpy.log(2.0)
pyplot.plot(s, upper_limit-s, 'kx--', label='Upper limit')
pyplot.plot(s, lower_limit-s, 'ko--', label='Lower limit')
pyplot.xlabel(r'$s$')
pyplot.ylim(-1.5, 1.5)
pyplot.legend(loc='center right');
```

For algorithms where the accuracy is high (I'd say $s \ge 6$) then the measured convergence rate is *close enough* if $s_m \in s_e \pm 1$!

## Setup

The problem that's at hand is a numerical method, in this case Euler's method, that's solving a differential equation

$$
\begin{equation}
  y'(x) = f(x, y), \qquad y(0) = y_0.
\end{equation}
$$

This method uses small steps $h \ll 1$ to approximate the solution $y(x)$ from $x=0$ to some point $X > 0$. Formal analysis shows that, under certain weak assumptions (essentially about the smoothness of $f$, particularly at $x=0$), the error of the method is first order. What this means is that the difference between the numerical solution $\hat{y}(X)$ and the true solution $y(X)$ is proportional to $h$, for sufficiently small $h$. We call this the error $E(h)$. This suggests that, in the limit as $h$ goes to zero, the error will go to zero, and the numerical solution will match the true solution.

Note immediately that most of the time, as in the phugoid model, we don't know the exact solution so can't measure the error. Instead we can measure *self convergence* by checking that the numerical solution converges to *something*. In other words, we want to check that $\hat{y}(X) = Y(X) + {\cal O}(h)$, so that in the limit as $h$ goes to zero we get a single, unique solution. Further analysis is needed to show that this is the *true* solution (essentially we need to impose *consistency* of the difference equation to the differential equation), which we won't worry about here. To do this we define the *difference*

$$
  D(h) = | \hat{y}_h(X) - \hat{y}_{ref}(X) |.
$$

Here $\hat{y}_{ref}$ is some reference solution, assumed to be computed at high accuracy.

With this setup, we're measuring convergence by checking that $D(h) = {\cal O}(h)$: the solution converges to something fixed if the difference converges to zero with $h$. Again, for Euler's method, a formal analysis shows that

$$
\begin{equation}
  D(h) = a h + {\cal O}(h^2)
\end{equation}
$$

where $a$ is some constant.

## The problem

It might be that we've incorrectly implemented Euler's method and it doesn't converge in the expected fashion. There are three possibilities.

### Unstable, inconsistent

In this case we're expecting the solution doesn't converge to a limit, so that

$$
\begin{equation}
  D(h) = \alpha h^{-s} + {\cal O}(h^{-s+1}).
\end{equation}
$$

In this case the difference diverges as $h$ gets smaller: this corresponds to the error diverging. This is really bad, and the error should be really obvious. Later we'll see what bounds we can put on the coefficients given the data.

### Stable, inconsistent

This is a slightly odd case. If we had

$$
\begin{equation}
  E(h) = E_0 + \alpha h + {\cal O}(h^2)
\end{equation}
$$

then the interpretation is straightforward: the algorithm is wrong (as $h$ goes to zero the error does not), but it is converging to something.

If, on the other hand, we have

$$
\begin{equation}
  D(h) = D_0 + \alpha h + {\cal O}(h^2)
\end{equation}
$$

the the *difference* between two numerical solutions is not going to zero with $h$. This is not impossible, but would be really odd at the continuum level. However, there is one important point that we cannot ignore: the limitations of floating point arithmetic.

#### Floating point effects

We know that we can't perfectly represent a real number on a computer, leading to an intrinsic error when representing a number $z$ which we'll call $\delta_z$. We have to count on the worst case, so that adding $N$ numbers $\{ z_i \}$ leads to a total error of $N \delta$ where $\delta = \max_i \delta_{z_i}$.

However, in our case it's even worse than that. Each step of using the Euler method introduces an error, which is compounded by the previous errors: the numerical data we use to go from $x$ to $x + h$ is already wrong thanks to the earlier steps in the method. We can do further analysis (far too [briefly summarized here](http://nbviewer.ipython.org/github/IanHawke/NumericalMethods/blob/master/Lectures/14%20-%20Predictor-Corrector%20Methods.ipynb)) to show the error at $X$ will be additionally amplified by a factor $\propto e^{\lambda X}$, where $\lambda \sim \partial_y f$.

This gives us a handle on the minimum error to expect. If the initial $\delta$ comes from standard double precision floating point then it will be $\sim 10^{-16}$. Euler's method for a moderately complex system, such as the phugoid system, uses ${\cal O}(10^1)$ operations per step. Then we have to work out $\lambda$, which is the maximum eigenvalue of the Jacobian matrix $\partial_y f$. Let's use `sympy` for that.

```python
import sympy
sympy.init_printing()
```

```python
v, theta, x, y, g, v_t, C_D, C_L = sympy.symbols('v, theta, x, y, g, v_t, C_D, C_L')
```

```python
q = sympy.Matrix([v, theta, x, y])
f = sympy.Matrix([-g*sympy.sin(theta)-C_D/C_L*g/v_t**2*v**2, -g/v*sympy.cos(theta)+g/v_t**2*v, v*sympy.cos(theta), v*sympy.sin(theta)])
```

```python
J = f.jacobian(q)
r = J.eigenvals()
r
```

We can then plug in some of the numbers from the phugoid model to get rid of the constants.

```python
r1 = list(r.keys())[1]
r2 = list(r.keys())[2]
r1=r1.subs([(g, 9.81), (v_t, 30.0), (C_D, 1.0/40.0), (C_L, 1.0)])
r2=r2.subs([(g, 9.81), (v_t, 30.0), (C_D, 1.0/40.0), (C_L, 1.0)])
```

We'll then assume that $v \simeq v_t$ and plot how the eigenvalue varies with $\theta$.

```python
r1 = r1.subs(v, 30.0)
r2 = r2.subs(v, 30.0)
sympy.plot(r1);
```

```python
sympy.plot(r2);
```

In the original simulation we had $h \le 0.1$ So it seems reasonable to assume that $h \lambda \sim {\cal O}(10^{-1})$. On the original phugoid problem we had $X = 100$ so our error bound is probably $\sim \delta \times 10^1 \times e^{0.1 \times 100} \simeq 2 \times 10^{-11}$.

Summarizing this fairly lengthy discussion:

* There will be floating point error leading to a best expected accuracy.
* We can estimate this from the problem and its parameters
* In this case the estimate is ${\cal O}(10^{-11})$, so there is probably no point in trying to do better than $10^{-10}$.

### Stable, consistent

Finally, there's the case where we measure something that is converging, but isn't clearly converging perfectly in line with the analysis. The point here is to be as sure as possible that we're getting something reasonable.

The analysis shows that

$$
\begin{equation}
  D(h) = \alpha h + \beta h^2 + \dots.
\end{equation}
$$

In order to actually measure something, we *assume* that $h$ is sufficiently small that we can ignore all terms except the first. We then model $D$ as $D = \hat{\alpha} h^s$ and measure $\hat{\alpha}$ and $s$. If $s$ is close to one we believe our assumptions were reasonable and say that everything is fine.

Here is the crux of Greg's question. What does "close to one" mean here? How should we interpret our results. With what confidence can we say that the data is consistent with our algorithm being correct?

## Analysis

That was a lot of background. Let's do some back of the envelope calculations to see how bad things can be.

We want to show that our algorithm is behaving as

$$
\begin{equation}
  D(h) = h
\end{equation}
$$

for sufficiently small $h$.

Let's suppose that the algorithm we've implemented actually behaves as

$$
\begin{equation}
  D(h) = 10^{-8} h^{-1} + h + 10^{-8} h^2.
\end{equation}
$$

Over what region would it appear to be behaving "correctly"?

```python
h = numpy.logspace(-14.0, 14.0)
D = 1e-8/h + h + 1e-8*h**2
```

```python
pyplot.loglog(h, D, 'bo', label = 'Difference')
pyplot.loglog(h, h, 'r-', label = 'D = h model')
pyplot.xlabel('h')
pyplot.ylabel('Difference')
pyplot.legend(loc='lower right');
```

So, despite being *completely inconsistent, and unstable*, the algorithm appears to behave correctly over **12** orders of magnitude: from $h \sim 10^{-4}$ to $h \sim 10^{8}$.

In reality, the behaviour at the top end (large $h$) is not a concern. What's more important is how it behaves for small $h$.

It's also noticeable that in the convergence check for the phugoid model, the behaviour was only checked over 2 orders of magnitude. So, should we be concerned?

In the original phugoid model notebook the differences were computed over a fairly narrow range of $h$. If we modify that range, we can check the data in more detail.

```python
h_values = numpy.array([0.1*2**(-i) for i in range(-3,18)])
differences = numpy.array([3.70456578e+04,   5.11556903e+02,   2.68347318e+02,
         1.40645139e+02,   7.00473279e+01,   3.51807674e+01,
         1.76241342e+01,   1.15159153e+00,   5.69993649e-01,
         2.83511479e-01,   1.41334800e-01,   7.05107406e-02,
         3.51645549e-02,   1.75078941e-02,   8.68366865e-03,
         4.27258022e-03,   2.06729609e-03,   9.64716814e-04,
         4.13441038e-04,   1.37809468e-04,   0.00000000e+00])
```

Note that the final value is from the reference solution, so of course there's no information there.

First we plot this on a loglog scale to see if it's roughly correct.

```python
pyplot.loglog(h_values[:-1], differences[:-1], 'kx')
pyplot.xlabel('h')
pyplot.ylabel('Differences');
```

We see that visually the behaviour looks ok for $h \lt 10^{-2}$. Above that it isn't behaving well. So let's replot, excluding the values of $h$ which don't appear to be "sufficiently small".

```python
h_small = h_values[numpy.logical_and(h_values < 1e-2, h_values > 1e-6)]
differences_small = differences[numpy.logical_and(h_values < 1e-2, h_values > 1e-6)]
pyplot.loglog(h_small, differences_small, 'kx')
pyplot.xlabel('h')
pyplot.ylabel('Differences');
```

We now do our standard thing: assume that this is perfectly modelled by $D(h) = a h^s$ and use linear regression (of $\log(D)$ against $\log(h)$) to find the parameters $a$ and $s$. First we'll do this using the entire dataset.

```python
p_all = numpy.polyfit(numpy.log(h_small), numpy.log(differences_small), 1)
print("The measured value of s is {:.3f}".format(p_all[0]))
```

    The measured value of s is 1.052

So this is off by about 5%. Let's plot the line of best fit and see where the difference lies.

```python
pyplot.loglog(h_small, differences_small, 'kx', label = 'Differences')
pyplot.loglog(h_small, numpy.exp(p_all[1])*h_small**p_all[0], 'b-', label = "Fit, slope = {:.3f}".format(p_all[0]))
pyplot.legend()
pyplot.xlabel('h')
pyplot.ylabel('Differences');
```

This is a bit concerning: why isn't the data appearing to get better? Let's try doing the fit for all possible sets of four consecutive points.

```python
for start in range(len(h_small)-4):
    p_fourpoints = numpy.polyfit(numpy.log(h_small[start:start+4]), numpy.log(differences_small[start:start+4]), 1)
    print("Measured value starting from point {} is {:.3f}".format(start, p_fourpoints[0]))
```

    Measured value starting from point 0 is 1.009
    Measured value starting from point 1 is 1.005
    Measured value starting from point 2 is 1.004
    Measured value starting from point 3 is 1.004
    Measured value starting from point 4 is 1.007
    Measured value starting from point 5 is 1.013
    Measured value starting from point 6 is 1.027
    Measured value starting from point 7 is 1.056
    Measured value starting from point 8 is 1.121

So we get pretty good values at low resolutions: at high resolutions something odd is happening.

# More terms

```python
ks = numpy.arange(6)
hs = 2.0**(-4-ks)
y = numpy.zeros(len(ks))
for i, h in enumerate(hs):
    y[i] = 1.0 + 0.1*h+0.2*h**2
```

```python
alpha = numpy.zeros_like(y)
for i in range(len(y)-1):
    alpha[i] = y[i+1]-y[i]

a_coeff = alpha[0]*alpha[2]-alpha[1]**2
b_coeff = alpha[1]*alpha[2]-alpha[0]*alpha[3]
c_coeff = alpha[1]*alpha[3] - alpha[2]**2

sol_plus = (-b_coeff + numpy.sqrt(b_coeff**2-4.0*a_coeff*c_coeff))/(2.0*a_coeff)
sol_minus = (-b_coeff - numpy.sqrt(b_coeff**2-4.0*a_coeff*c_coeff))/(2.0*a_coeff)

print("Two solutions are {} and {}".format(-numpy.log2(sol_plus), -numpy.log2(sol_minus)))
```

    Two solutions are 0.99999999999764 and 1.999999999962835


# Variational Principle  using Symbolic Mathematics in Python

## 1. Introduction

The variational principle tells us that we can use a trial wavefunction to solve the Schrodinger equation using the following theorem:

$${{\int {{\Psi ^*}\hat H{\rm{ }}\Psi } d\tau } \over {\int {{\Psi ^*}\Psi } d\tau }} \ge {E_0}$$

We will use Sympy to solve the particle in a box problem by guessing a trial wavefunction using variational principle

```python
import sympy as sym
```

This exercise is a bit more self-guided than the other notebooks we have done. One of the most useful things you can do is **open last week's notebook to remember the commands in sympy**. Also, remember that google is your friend:

1. [Sympy tutorial](https://docs.sympy.org/latest/tutorial/index.html)
2. [Stack Overflow](https://stackoverflow.com/search?q=sympy+)
3. [Stack Exchange](https://stackexchange.com/)

## 2. Particle in a box

The wave function that we pick for a particle in a box needs to have the following properties

1. single valued
1. normalizable
1. function and its first derivative are continuous
1. boundary condition that the wave function goes to zero at the ends of the box

Particle in a box: a is a classical particle, red is real part, blue is imaginary part.

This particle only expericnes kinetic energy between the box, so the Hamiltonian for this system is

$$\hat H = {{ - {\hbar ^2}} \over {2m}}{{{d^2}} \over {d{x^2}}} + \left\{ {\matrix{{V(x) = 0} & {0 < x < a}  \cr {V(x) = \infty } & {x < 0\text{ }{\rm{ or}}\;x > a}  \cr } } \right.$$

For our purposes, that means we can consider the Hamiltonian to be

$$\hat H = {{ - {\hbar ^2}} \over {2m}}{{{d^2}} \over {d{x^2}}}$$

as long as we keep the limits of integration to be $(0,a)$

### 2.1 Trial Wave function

Although the particle in box has a well known solution

[https://en.wikipedia.org/wiki/Particle_in_a_box](https://en.wikipedia.org/wiki/Particle_in_a_box)

(or check your favorite pchem book)

We are going to guess a trial wave function:

$$\Phi (x) = \left( {{x \over a} - {{{x^3}} \over a}} \right) + \alpha \left( {{{{x^5}} \over {{a^5}}} - {1 \over 2}\left( {{{{x^7}} \over {{a^7}}} + {{{x^7}} \over {{a^7}}}} \right)} \right)$$

### 2.2 Exercise: Variational Theorem

We are going to follow the following plan:

1. Solve for the energy of the trial wave function above

$${E_{trial}} = {{\int\limits_0^a {\Phi (x){{ - {\hbar ^2}} \over {2m}}{{{d^2}} \over {d{x^2}}}\Phi (x)dx} } \over {\int\limits_0^a {\Phi {{(x)}^2}dx} }}$$

Your answer will be a function of $ m,a,\text{and } \alpha$ We will use $\alpha$ as the parameter we vary to minimize the energy and make a new trial wave function.

2. Minimize the trial energy
We will use a first derivative of the trial energy $${d \over {d\alpha }}{E_{trial}}(\alpha )$$ to find the value of $\alpha$ that gives you the lowest energy

3. Plot your new wavefunction compared to the ground state particle in a box: $${\psi _{true}}(x) = {\left( {{2 \over a}} \right)^{1/2}}\sin {{n\pi x} \over a}$$ Plot as a function of $x/a$ from $0$ to $1$. Assuming this has $m=m_e$, and $a=a_0$ use atomic (theorist) units to plot the function.

4. Compare your trial energy to the actual energy (using atomic units)
$${E_{true}}(n = 1) = {{{\hbar ^2}{\pi ^2}} \over {2m{a^2}}}$$

```python
import sympy as sym
a,x,m,alpha = sym.symbols('a,x,m,alpha')
sym.Rational(1/2)
sym.init_printing()
```

```python
phi = ((x/a)-(x**3/a**3)+ alpha*((x**5/a**5)-1/2*(x**9/a**9+x**9/a**9)))
num1 = (phi*sym.Rational(-1/2))
num2 = sym.diff(sym.diff(phi,x),x)
den1 = (phi**2)
```

```python
Numerator = sym.integrate((num1*num2),(x,0,a))
```

```python
Denominator = sym.integrate((den1),(x,0,a))
```

Your descriptions/explanations here

```python
Numerator/Denominator
```

```python
E_trial = (Numerator/Denominator).subs(a,1)
E_trial
```

```python
sym.solveset(sym.diff(E_trial,alpha))
```

```python
E_trial.subs(alpha, -0.257747652303185)
```

```python
E_trial.subs(alpha, -4.73345207773701)
```

```python
Newphi = phi.subs(alpha, -0.257747652303185)
Newphi
```

```python
Newphi2 = Newphi.subs(a,1)
Newphi2
```

```python
sym.plot(Newphi2,(x,0,1))
```

```python
sym.plot((2**1/2)*sym.sin(sym.pi*x),(x,0,1))
```

```python
sym.plot((2**1/2)*sym.sin(sym.pi*x),Newphi2,(x,0,1))
```

```python
E_true = sym.pi**2/(2*m)
E_true
```

### 2.3 Exercise: New trial wavefunction

Determine the minimum energy of the particle in a box using a new trial wavefunction $$x^\alpha(x-a)^\alpha$$

1. Find the minimum energy, $E_{trial}$
2. Plot the new trial wavefunction and compare it to the true solution and the wavefunction you found above
3. Compare you new energy to the trial energy you found above
4. Which wavefunction is better? How do you know?

```python
#Plug in 1 for a, after you solve for values, first find the first derivative of part 1 function in terms of alpha
#After you solve both plug them into E trial and see which gives you the lower E value
```

```python
phhi = (x**alpha)*(x-a)**alpha
phhi
```

```python
num11 = (phhi*sym.Rational(1/2))
num21 = sym.diff(sym.diff(phhi,x),x)
den11 = (phhi**2)
Numerator2 = sym.integrate((num11*num21),(x,0,a))
Numerator2

```

```python
Denominator2 = sym.integrate((den11),(x,0,a))
Denominator2
```

$\displaystyle \frac{a a^{4 \alpha} e^{2 i \pi \alpha} \Gamma\left(2 \alpha + 1\right) {{}_{2}F_{1}\left(\begin{matrix} - 2 \alpha, 2 \alpha + 1 \\ 2 \alpha + 2 \end{matrix}\middle| {1} \right)}}{\Gamma\left(2 \alpha + 2\right)}$

```python
E_trial2 = (Numerator2/Denominator2).subs(a,1)
E_trial2
```

$\displaystyle \frac{\alpha e^{- 2 i \pi \alpha} \Gamma\left(2 \alpha + 2\right) \int\limits_{0}^{1} \frac{x^{2 \alpha} \left(x - 1\right)^{2 \alpha} \left(4 \alpha x^{2} - 4 \alpha x + \alpha - 2 x^{2} + 2 x - 1\right)}{x^{2} \left(x - 1\right)^{2}}\, dx}{2 \Gamma\left(2 \alpha + 1\right) {{}_{2}F_{1}\left(\begin{matrix} - 2 \alpha, 2 \alpha + 1 \\ 2 \alpha + 2 \end{matrix}\middle| {1} \right)}}$

```python
sym.solveset(sym.diff(E_trial2,alpha))
```

$\displaystyle \left\{\alpha \mid \alpha \in \mathbb{C} \wedge \left(- 2 \alpha {{}_{2}F_{1}\left(\begin{matrix} - 2 \alpha, 2 \alpha + 1 \\ 2 \alpha + 2 \end{matrix}\middle| {1} \right)} \operatorname{polygamma}{\left(0,2 \alpha + 1 \right)} \int\limits_{0}^{1} \frac{x^{2 \alpha} \left(x - 1\right)^{2 \alpha} \left(4 \alpha x^{2} - 4 \alpha x + \alpha - 2 x^{2} + 2 x - 1\right)}{x^{2} \left(x - 1\right)^{2}}\, dx + 2 \alpha {{}_{2}F_{1}\left(\begin{matrix} - 2 \alpha, 2 \alpha + 1 \\ 2 \alpha + 2 \end{matrix}\middle| {1} \right)} \operatorname{polygamma}{\left(0,2 \alpha + 2 \right)} \int\limits_{0}^{1} \frac{x^{2 \alpha} \left(x - 1\right)^{2 \alpha} \left(4 \alpha x^{2} - 4 \alpha x + \alpha - 2 x^{2} + 2 x - 1\right)}{x^{2} \left(x - 1\right)^{2}}\, dx + \alpha {{}_{2}F_{1}\left(\begin{matrix} - 2 \alpha, 2 \alpha + 1 \\ 2 \alpha + 2 \end{matrix}\middle| {1} \right)} \int\limits_{0}^{1} \frac{x^{2 \alpha} \left(x - 1\right)^{2 \alpha} \left(4 x^{2} - 4 x + 2 \left(4 \alpha x^{2} - 4 \alpha x + \alpha - 2 x^{2} + 2 x - 1\right) \log{\left(x \right)} + 2 \left(4 \alpha x^{2} - 4 \alpha x + \alpha - 2 x^{2} + 2 x - 1\right) \log{\left(x - 1 \right)} + 1\right)}{x^{2} \left(x - 1\right)^{2}}\, dx - 2 i \pi \alpha {{}_{2}F_{1}\left(\begin{matrix} - 2 \alpha, 2 \alpha + 1 \\ 2 \alpha + 2 \end{matrix}\middle| {1} \right)} \int\limits_{0}^{1} \frac{x^{2 \alpha} \left(x - 1\right)^{2 \alpha} \left(4 \alpha x^{2} - 4 \alpha x + \alpha - 2 x^{2} + 2 x - 1\right)}{x^{2} \left(x - 1\right)^{2}}\, dx - \alpha \frac{d}{d \alpha} {{}_{2}F_{1}\left(\begin{matrix} - 2 \alpha, 2 \alpha + 1 \\ 2 \alpha + 2 \end{matrix}\middle| {1} \right)} \int\limits_{0}^{1} \frac{x^{2 \alpha} \left(x - 1\right)^{2 \alpha} \left(4 \alpha x^{2} - 4 \alpha x + \alpha - 2 x^{2} + 2 x - 1\right)}{x^{2} \left(x - 1\right)^{2}}\, dx + {{}_{2}F_{1}\left(\begin{matrix} - 2 \alpha, 2 \alpha + 1 \\ 2 \alpha + 2 \end{matrix}\middle| {1} \right)} \int\limits_{0}^{1} \frac{x^{2 \alpha} \left(x - 1\right)^{2 \alpha} \left(4 \alpha x^{2} - 4 \alpha x + \alpha - 2 x^{2} + 2 x - 1\right)}{x^{2} \left(x - 1\right)^{2}}\, dx\right) \Gamma\left(2 \left(\alpha + 1\right)\right) = 0 \right\} \setminus \left\{\alpha \mid \alpha \in \mathbb{C} \wedge e^{2 i \pi \alpha} \Gamma\left(2 \alpha + 1\right) {{{}_{2}F_{1}\left(\begin{matrix} - 2 \alpha, 2 \alpha + 1 \\ 2 \alpha + 2 \end{matrix}\middle| {1} \right)}}^{2} = 0 \right\}$

```python

```

Your descriptions/explanations here

### 2.4 Exercise: Design your own wavefunction!

**Now you get to make your own wavefunction!**

The only guidance I would give you is that it make sense mathematically and that it include $\alpha$ so that you can minimize the energy.

Remember that $a$ and $x$ are both length units, and that trigonometric, logarithmic, and exponential functions are all unitless

Using your new wavefunction:

1. Find the minimum energy, $E_{trial}$
2. Plot the new trial wavefunction and compare it to the true solution and the wavefunction you found above
3. Compare you new energy to the trial energy you found above
4. Which wavefunction is better? How do you know?

```python
import sympy as sym
a,x,alpha = sym.symbols('a,x,alpha')
sym.Rational(1/2)
sym.init_printing()
```

```python
patsywavefunction = (x**2)*(x-a)**alpha
patsywavefunction
```

```python
num1 = (patsywavefunction*sym.Rational(-1/2))
num2 = sym.diff(sym.diff(patsywavefunction,x),x)
den1 = (patsywavefunction**2)
```

```python
value = (num1*num2)
```

```python
Numerator = sym.integrate((value),(x,0,a))
```

```python
Denominator = sym.integrate((den1),(x,0,a))
```

```python
Numerator/Denominator
```

$\displaystyle \left(\frac{1.42081447963801 \alpha^{2}}{a} - \frac{1.68831168831169 \alpha}{a} + \frac{0.2 \left(- 10.0 \alpha - 3.0\right)}{a} + \frac{0.111111111111111 \left(- 10.0 \alpha^{2} + 36.0 \alpha\right)}{a} + \frac{1.0}{a}\right) \left(\begin{cases} \frac{1}{- \log{\left(- a \right)} - \infty \operatorname{sign}{\left(a^{4} \right)} - \frac{25}{12}} & \text{for}\: \alpha = - \frac{5}{2} \\\frac{1}{- 4 a \log{\left(- a \right)} - \frac{4 a}{3} + \infty \operatorname{sign}{\left(a^{4} \right)}} & \text{for}\: \alpha = -2 \\\frac{1}{- 6 a^{2} \log{\left(- a \right)} - 3 a^{2} - \infty \operatorname{sign}{\left(a^{4} \right)}} & \text{for}\: \alpha = - \frac{3}{2} \\\frac{1}{- 4 a^{3} \log{\left(- a \right)} - 4 a^{3} + \infty \operatorname{sign}{\left(a^{4} \right)}} & \text{for}\: \alpha = -1 \\\frac{1}{- a^{4} \log{\left(- a \right)} - \infty \operatorname{sign}{\left(a^{4} \right)}} & \text{for}\: \alpha = - \frac{1}{2} \\\frac{\left(- a\right)^{- 2 \alpha} \left(8 \alpha^{5} + 60 \alpha^{4} + 170 \alpha^{3} + 225 \alpha^{2} + 137 \alpha + 30\right)}{6 a^{5}} & \text{otherwise} \end{cases}\right)$

```python
E_trial = (Numerator/Denominator).subs(a,1)
E_trial
```

$\displaystyle \left(0.309703368526898 \alpha^{2} + 0.311688311688312 \alpha + 0.4\right) \left(\begin{cases} \frac{1}{-\infty - i \pi} & \text{for}\: \alpha = - \frac{5}{2} \\\frac{1}{\infty - 4 i \pi} & \text{for}\: \alpha = -2 \\\frac{1}{-\infty - 6 i \pi} & \text{for}\: \alpha = - \frac{3}{2} \\\frac{1}{\infty - 4 i \pi} & \text{for}\: \alpha = -1 \\\frac{1}{-\infty - i \pi} & \text{for}\: \alpha = - \frac{1}{2} \\\frac{\left(-1\right)^{- 2 \alpha} \left(8 \alpha^{5} + 60 \alpha^{4} + 170 \alpha^{3} + 225 \alpha^{2} + 137 \alpha + 30\right)}{6} & \text{otherwise} \end{cases}\right)$

```python
sym.solveset(sym.diff(E_trial,alpha))
```

```python
sym.solveset(sym.diff(E_trial,alpha))
```

```python

```

Your descriptions/explanations here

# Reading Homework

Read the following sections in Kramer

- 4.2.3 Born-Oppenheimer approximation
- 4.3.2 Secular equation
- All of 4.5

For each subsection
- write down the subchapter name
- what was the most important idea
- draw an idea digram of the main idea

**Make sure to upload this to the assignment repository**

Example idea diagram:

```python

```


# Importing and reading data

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

from scipy import integrate
import seaborn as sns; sns.set()
```

```python
# be sure to git pull upstream master before reading the data so it is up to date.
DATA_URL = 'https://raw.githubusercontent.com/blas-ko/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/'

df_confirmed = pd.read_csv(DATA_URL+'time_series_19-covid-Confirmed.csv')
df_deaths = pd.read_csv(DATA_URL+'time_series_19-covid-Deaths.csv')
df_recovered = pd.read_csv(DATA_URL+'time_series_19-covid-Recovered.csv')
```

```python
def df_to_timeseries(df):
    return df.drop(['Province/State','Lat','Long'], axis=1).groupby('Country/Region').sum().T

def df_to_timeseries_province(df):
    return df.drop(['Lat','Long'], axis=1).set_index('Country/Region')

def country_to_timeseries(df, country):
    return df.drop(['Lat','Long'],axis=1).set_index(['Country/Region','Province/State']).loc[country].T

def country_to_timeseries(df, country):
    df_country = df[ df['Country/Region'] == country ].drop(['Province/State','Lat','Long'], axis=1)

    return df_country.groupby('Country/Region').sum().T

def province_to_timeseries(df, province):
    df_province = df[ df['Province/State'] == province ]

    return df_province.set_index('Province/State').drop(['Country/Region','Lat','Long'], axis=1).T#[province]
```

```python
# testing...
country_to_timeseries(df_confirmed, 'China').plot()
country_to_timeseries(df_confirmed, 'Italy').plot()
```

# Basic model description
We use a SEIRD model to describe the spreading of the CoVid-19 virus in a population. The model distinguishes between the population that is Suceptible ($S$) to the virus, those who have been Exposed ($E$) to it but don't present any symptoms nor are infectious, those who are Infected ($I$), those who have Recovered ($R$), and those who are Deceased ($D$). This model is a simplification of the [compartamental model developed by Alison Hill and collaborators](https://alhill.shinyapps.io/COVID19seir/?fbclid=IwAR0G-qmOdznACkXRHZMMNyk4NRW-MHlk_n4I4W7Q3_MGqmm7wplUp0zpkJk)

The Suceptible population gets exposed to the virus by getting in contact with the Infected population at a rate $\lambda$. Then, the Exposed population get into an Infected stage at a rate $\sigma$. The Infected population can either Recover, which they do at a rate $\gamma$, or Die, which they do at a rate $\mu$. These dynamics are described in the following equations

\begin{align}
  \begin{aligned}
    \dot{S} &= -\lambda \frac{I}{N} S \\
    \dot{E} &= \lambda \frac{I}{N} S - \sigma E \\
    \dot{I} &= \sigma E - \gamma I - \mu I \\
    \dot{R} &= \gamma I \\
    \dot{D} &= \mu I,
  \end{aligned}
\end{align}
where $\dot{x} := \frac{dx(t)}{dt} $. Note that the equations implicitely encode that the conservation of the total population, i.e.
\begin{equation}
    S + E + I + R + D = N.
\end{equation}
This model assumes that, once an individual recovers, she will not get susceptible again. Further, we assume that the timescale of the epidemics is fast compared to the natural birth-death rates of the population. Thus, $N$ does not change in time (and, consequentially, it can be absorbed into $\lambda$).

## Simulations

```python
# Basic SEIRD model ODE
def SEIRD_model(x, t, *params):
    λ, σ, γ, μ = params
    S,E,I,R,D = x
    return [-λ*I*S,
            λ*I*S - σ*E,
            σ*E - (γ+μ)*I,
            γ*I,
            μ*I]
```

## How to choose the coefficients and initial conditions of the model?

The **symptoms** of the coronavirus, according to the [Canadian public health service](https://www.canada.ca/en/public-health/services/diseases/2019-novel-coronavirus-infection/symptoms.html#s) and the [Worldometers](https://www.worldometers.info/coronavirus/coronavirus-symptoms/) suggest that
- There are no vaccines yet
- Symptoms may take up to 14 days to appear (there's a range between 2-14 days to develop symptoms according to the [CDC](https://www.cdc.gov/coronavirus/2019-ncov/symptoms-testing/symptoms.html?CDC_AA_refVal=https%3A%2F%2Fwww.cdc.gov%2Fcoronavirus%2F2019-ncov%2Fabout%2Fsymptoms.html). We will use a exposure period of 8 days.
- The infection lasts around 14 days for mild cases and 3+ weeks for more severe cases
- 81 % of the cases are mild.

The **mortality** of the coronavirus, according to the [Worldometers](https://www.worldometers.info/coronavirus/coronavirus-death-rate/) includes:
- 3.4 % death rate as of March 03
- However, if one does the ratio of deaths/confirmed cases as of 20-March-2020, one gets 4 % death rate
- The recovery rate is around 35 % so far.

The **Basic reproductive ratio** (see [Wikipedia](https://en.wikipedia.org/wiki/Basic_reproduction_number)), according to the [CMMID](https://cmmid.github.io/topics/covid19/current-patterns-transmission/global-time-varying-transmission.html), is
- $R_0 \approx 2.2 \pm 0.2$ at 01/feb/20 in China
- $R_0 \approx 2.2 \pm 0.3$ at 01/feb/20 in Italy
- $R_0$ unknown at 01/feb/20 in the UK

Using the [World Bank transportation data](https://data.worldbank.org/indicator/IS.AIR.PSGR?locations=CN-IT-GB), we can estimate the flow between countries using as a proxy the average number of flights of any given country in 2018; these include
- China: $611$ million flights in 2018
- Italy: $28$ million flights in 2018
- UK: $165$ million flights in 2018

Specifically with travels between Italy and the UK, [Wikipedia](https://en.wikipedia.org/wiki/Italy%E2%80%93United_Kingdom_relations) says that
- Between 4 and 5 million British tourists visit Italy every year.
- 1 million Italian tourists visit the UK.
- [the UK government](https://www.gov.uk/foreign-travel-advice/italy) says that approximately 3 million British nationals visit Italy every year.

The population of countries can be found at the [Wordlometers](https://www.worldometers.info/world-population/population-by-country/), where
- China: $14.4 \times 10^9$ individuals
- Italy: $60.5 \times 10^6$ individuals
- UK: $67.9 \times 10^6$ individuals

The number of cases for each country are obtained from the [John Hopkins University repository](https://github.com/CSSEGISandData/COVID-19).

```python
# Reported China coronavirus numbers on 01st February 2020
initial_date = '2/1/20'

N = 14.4e9    # China's population
cases = df_to_timeseries(df_confirmed).loc[initial_date,'China']    # total reported cases including resolved
deaths = df_to_timeseries(df_deaths).loc[initial_date,'China']
recovered = df_to_timeseries(df_recovered).loc[initial_date,'China']

R_0 = 2.3  # Basic Reproductive Rate [people]
M = 0.034   # Mortality ratio [fraction]
P_exposure = 8     # Average exposure period [days] (should really be split up by case severity)
P_infectious = 14     # Average exposure period [days] (should really be split up by case severity)

# Compute model coefficients
γ = 1 / P_infectious
μ = γ * M
λ = R_0 * (γ + μ)
σ = 1 / P_exposure

# concatenating problem parameters
params = (λ, σ, γ, μ)

# setting initial conditions
r = 2
R0 = recovered / N
D0 = deaths / N
I0 = cases/ N - R0 - D0  # confirmed cases
E0 = r*I0 # cases without symptoms, so they are not yet detected
S0 = (1 - E0 - I0 - R0 - D0)

# initial condition at t0 = Feb 01
x0 = [S0, E0, I0, R0, D0]
```

```python
## Integrating the problem for the next year
t0,tf = (0, 365)
sol = integrate.solve_ivp(lambda t,x: SEIRD_model(x, t, *params), (t0,tf), x0)
```

```python
def plot_model(sol, country='China', log=False):
    # Basic plot of the dynamics
    plt.figure( figsize=(8,6) )

    labels = ['Suceptible', 'Exposed', 'Infected', 'Recovered', 'Deceased']
    for (i,y) in enumerate(sol.y):
        if i == 0:
            0
    #         continue
        plt.plot(sol.t, y*N, label=labels[i], lw=3)

    plt.title("Covid-19 spread in {}".format(country))
    plt.ylabel("Number of people")
    plt.xlabel("time [days since {}].".format(initial_date))
    plt.legend()

    if log:
        plt.yscale('log')

    print(f"For a population of {int(N/1e6)} million people, after {sol.t[-1]:.0f} days there were:")
    print(f"{sol.y[4][-1]*100:.1f}% total deaths, or {sol.y[4][-1]*N/1e3:.0f} thousand people.")
    print(f"{sol.y[3][-1]*100:.1f}% total recovered, or {sol.y[3][-1]*N/1e3:.0f} thousand people.")
    print(f"At the virus' maximum {sol.y[2].max()*100:.1f}% people were simultaneously infected, or {sol.y[2].max()*N/1e3:.0f} thousand people.")
    print(f"After {sol.t[-1]:.0f} days the virus was present in less than {sol.y[2][-1]*N/1e3:.0f} thousand individuals.")
```

```python
print("If no measures are taken, according to our model:\n")
plot_model(sol)
```

ToDo: Include measures of social distancing, quarantine, and the effect of the healthcare system. See [this notebook](https://github.com/TomNicholas/coronavirus/blob/master/covid-19_model.ipynb) for inspiration.

# SEIRD coupled model

```python
####----------------------------------####
#### Ilustration of the coupled model ####
####----------------------------------####
import networkx as nx

G = nx.DiGraph()
G.add_weighted_edges_from( [(1,2,4.0),(2,1, 2.0), (1,3, 1.0),(3,1, 1.0), (2,3, 1.5),(3,2, 2.0) ] )

pos = nx.spring_layout(G)  # positions for all nodes

# nodes
nx.draw_networkx_nodes(G, pos, node_size=1400, alpha=0.9)

# labels
nx.draw_networkx_labels(G, pos, labels={1:'UK',2:'Italy',3:'China'}, font_size=15, font_family='sans-serif')
# edges
edge_weights = []
for i in G.nodes:
    for j in G.nodes:
        if j != i:
            edge_weights.append( G[i][j]['weight'] )
nx.draw_networkx_edges(G, pos, width=edge_weights, )
# edge label
edge_labels = {(1,2): '$w_{12}$', (1,3): '$w_{13}$', (2,3): '$w_{23}$', }
nx.draw_networkx_edge_labels(G, pos, edge_labels=edge_labels, label_pos=0.5, font_size=15)

plt.axis('off')
plt.title("FIG 1: SEIRD Network model schema");
```

The Covid-19 is a pandemic, and, although the model $(1)$ takes a global population $N$ as its input, it does not have any structure of the interaction between the people. It is just a zeroth-order approximation of the rate of contact, and it assumes every suceptible individual has the same likelihood to get infected no matter where she is.

Here we want to extend model $(1)$ and embed it into a network, where each of its nodes represents different populations and the edges represents the coupling between them. For instance, nodes may be countries while the edges rate at which people from one country travel to another.

To simplify notation, we define the vector $\mathbf{x}^i = (S^i, E^i, I^i, R^i, D^i)$ to represent the population of country $i$.

$w_{ij}$ is the flow rate from country $i$ to country $j$. Only Suceptible and Exposed people should be able to travel (Of course Deceased and Infected people will not; but Recovered people were recently infectious, so they shouldn't travel anyways.). While the worldwide population is conserved --$N = \sum_i N^i$--, the population of each individual country can now vary over time.

$w_{ij}$ should be proportional to the individuals in country $i$, and the probability of going to country $j$ should decrease if $j$ has a lot of infected individual. Thus, we propose
\begin{equation}
    w_{ij} = \alpha_{ij} ( S^i + E^i ) e^{ - \beta I^j },
\end{equation}

where $\alpha_{ij}$ leverages the ammount of people that go from $i$ to $j$. If there's no epidemic (i.e. N^i = S^i), $w_{ij} = \alpha_{ij} S^i$ is the average number of flights per time unit from country $i$ to $j$.

$\beta$ represents the intensity to not fly to country $j$ because of its infectiousness.

Note that this rationale can be extended to any network structure, where, instead of taking the weight of a given country $i$ to every other country in the world, it only considers the set of $i$'s *possible* destinations, which we denote as $\mathcal{N}_i$.

We incorporate the flows between countries to the basic SEIRD model described previously, obtaining the following system
\begin{align}
  \begin{aligned}
    \dot{S^i} &= -\lambda \frac{I^i}{N^i} S^i &- S^i \sum_{j \in \mathcal{N}_i} \alpha_{ij}e^{-\beta I^j} + \sum_{j \in \mathcal{N}_i} S^j \alpha_{ji}e^{-\beta I^i} \\
    \dot{E^i} &= \lambda \frac{I^i}{N^i} S^i - \sigma E^i & \underbrace{ - E^i \sum_{j \in \mathcal{N}_i} \alpha_{ij}e^{-\beta I^j} }_{\text{out-flow}} + \underbrace{ \sum_{j \in \mathcal{N}_i} E^j \alpha_{ji}e^{-\beta I^i} }_{\text{in-flow}} \\
    \dot{I^i} &= \sigma E^i - \gamma I^i - \mu I^i \\
    \dot{R^i} &= \gamma I^i \\
    \dot{D^i} &= \mu I^i,
  \end{aligned}
\end{align}
where we assume that the epidemic coefficients ($\lambda, \sigma, \gamma, \mu$) stay constant.

Further, we will assume that the initial Exposed individuals are proportional to the infected (and officially registered) individuals. This is,
\begin{equation}
E^i_0 = r (I^i_0 + R^i_0 + D^i_0),
\end{equation}
for some parameter $r \geq 1$.

*Note that while the virus **may** affect the global population equally on average, different countries have developed different measures to stop the virus, effectively changing the epidemic coefficients. We will deal with those kind of measures later, when we parametrize the coefficients in terms of the action plan of each country*.

ToDo: Do the coupled SEIRD for the general case on $k$ countries joined with an adj matrix (or maybe everybody fucks with everybody). $\alpha$ is a candidate for a weighted, directed adjacency whenever it's not constant.

If $\alpha$ is constant, does everything couples exactly as we saw in the case of 2 countries?

```python
# In the SEIRD model, each country comes packed in a vector of 5 components : S,E,I,R,D
def coupled_SEIRD(x, t, *params, coupling=True):
    α, β, λ, σ, γ, μ = params # could put A, adjmatrix

    n_countries = int( len(x)/5 )
    x_dot = []
    for i in range(n_countries):
        Si, Ei, Ii, Ri, Di = x[5*i:5*(i+1)]
        # decoupled SEIRD model
        Si_dot, Ei_dot, Ii_dot, Ri_dot, Di_dot = [-λ*Ii*Si, λ*Ii*Si - σ*Ei, σ*Ei - (γ+μ)*Ii, γ*Ii, μ*Ii]

        # couplings with other countries
        coupling_Si = 0
        coupling_Ei = 0
        if coupling:
            for j in range(n_countries):
                if i != j:
                    Sj, Ej, Ij, Rj, Dj = x[5*j:5*(j+1)]
                    coupling_Si += - Si * α[i,j] * np.exp( -β*Ij ) +  Sj * α[j,i] * np.exp( -β*Ii )
                    coupling_Ei += - Ei * α[i,j] * np.exp( -β*Ij ) +  Ej * α[j,i] * np.exp( -β*Ii )

        Si_dot += coupling_Si
        Ei_dot += coupling_Ei

        x_dot += [Si_dot, Ei_dot, Ii_dot, Ri_dot, Di_dot]

    return x_dot
```

```python
####-------------####
#### Model setup ####
####-------------####
initial_date = '3/1/20'

N_uk = 67.8e6      # UK's population
cases_uk =  df_to_timeseries(df_confirmed).loc[initial_date,'United Kingdom']    # total reported cases including resolved
deaths_uk = df_to_timeseries(df_deaths).loc[initial_date,'United Kingdom']
recovered_uk = df_to_timeseries(df_recovered).loc[initial_date,'United Kingdom']
print("There were {} cases in the UK at {}".format(cases_uk, initial_date))

N_italy = 60.5e6    # Italy's population
cases_italy = df_to_timeseries(df_confirmed).loc[initial_date,'Italy']    # total reported cases including resolved
deaths_italy = df_to_timeseries(df_deaths).loc[initial_date,'Italy']
recovered_italy = df_to_timeseries(df_recovered).loc[initial_date,'Italy']
print("There were {} cases in Italy at {}".format(cases_italy, initial_date))

N_both = N_uk + N_italy

####----------------------------####
#### Estimation of coefficients ####
####----------------------------####
R_0 = 2.5             # Basic Reproductive Rate [people]
M = 0.034             # Mortality ratio [fraction]
P_exposure = 8      # Average exposure period [days]
P_infectious = 16     # Average infectious period [days] (should really be split up by case severity)

## Estimation of model coefficients
# how to compute R0 taking P_exposure into account?
γ = 1 / P_infectious
σ = 1 / P_exposure
μ = γ * M
λ = R_0 * (γ + μ + σ)
print("λ", λ)

## Estimation of coupling coefficients (source : https://data.worldbank.org/indicator/IS.AIR.PSGR?end=2018&locations=CN-IT&start=2006)
α_uk_italy = 4e6 / N_uk
α_italy_uk = 1e6 / N_italy

α = np.array([ [0.0, α_uk_italy], [α_italy_uk, 0] ]) # transition matrix
β = np.log(2) / 2e-4 # repulsion coefficient to fo to "infected" country. (made up)
# the logic behind β: if 10k people out of 50M are infected, reduce your chances of going by 1/2.
# i.e. exp( -β (10k/50M) ) ~ 1/2
print("β",β)

## MODEL PARAMETERS
params = (α, β, λ, σ, γ, μ)

####--------------------------####
#### Initial conditions setup ####
####--------------------------####
r = 7 # Ratio of unregistered vs registered cases (made up quantity)
# UK
R0_uk = recovered_uk / N_both
D0_uk = deaths_uk / N_both
I0_uk = cases_uk/ N_both - R0_uk - D0_uk
E0_uk = r*I0_uk
S0_uk = (N_uk - E0_uk - I0_uk - R0_uk - D0_uk) / N_both

x0_uk = [S0_uk, E0_uk, I0_uk, R0_uk, D0_uk]

# italy
R0_italy = recovered_italy / N_both
D0_italy = deaths_italy / N_both
I0_italy = cases_italy / N_both - R0_italy - D0_italy
E0_italy = r*I0_italy
S0_italy = (N_italy - E0_italy - I0_italy - R0_italy - D0_italy) / N_both

x0_italy = [S0_italy, E0_italy, I0_italy, R0_italy, D0_italy]

### INITIAL CONDITIONS
x0 = x0_uk + x0_italy
```

    There were 36 cases in the UK at 3/1/20
    There were 1694 cases in Italy at 3/1/20
    λ 0.47406249999999994
    β 3465.7359027997263

```python
####-----------------------------------####
#### Numerical simulation of the model ####
####-----------------------------------####
days = 19
t0,tf = (0, days) # days from 01/feb to date : 29+20
# model with coupling
%time sol = integrate.solve_ivp(lambda t,x: coupled_SEIRD(x, t, *params, coupling=True), (t0,tf), x0, t_eval=np.linspace(0,days, days+1))
# model without coupling
%time sol_nocoupling = integrate.solve_ivp(lambda t,x: coupled_SEIRD(x, t, *params, coupling=False), (t0,tf), x0, t_eval=np.linspace(0,days, days+1))

It_uk, Rt_uk, Dt_uk = (sol.y[2], sol.y[3], sol.y[4])
It_it, Rt_it, Dt_it = (sol.y[2+5], sol.y[3+5], sol.y[4+5])

It_uk_noc, Rt_uk_noc, Dt_uk_noc = (sol_nocoupling.y[2], sol_nocoupling.y[3], sol_nocoupling.y[4])
It_it_noc, Rt_it_noc, Dt_it_noc = (sol_nocoupling.y[2+5], sol_nocoupling.y[3+5], sol_nocoupling.y[4+5])

print("\nCoupled scenario:")
print("Confirmed cases after {} days in UK: {}".format(days, (It_uk[-1] + Rt_uk[-1] + Dt_uk[-1])*N_both ) )
print("Confirmed cases after {} days in Italy: {}".format(days, (It_it[-1] + Rt_it[-1] + Dt_it[-1])*N_both ) )

print("\nDecoupled scenario:")
print("Confirmed cases after {} days in UK: {}".format(days, (It_uk_noc[-1] + Rt_uk_noc[-1] + Dt_uk_noc[-1])*N_both ) )
print("Confirmed cases after {} days in Italy: {}".format(days, (It_it_noc[-1] + Rt_it_noc[-1] + Dt_it_noc[-1])*N_both ) )
```

    CPU times: user 5.34 ms, sys: 0 ns, total: 5.34 ms
    Wall time: 3.99 ms
    CPU times: user 0 ns, sys: 1.94 ms, total: 1.94 ms
    Wall time: 1.6 ms

    Coupled scenario:
    Confirmed cases after 19 days in UK: 3357.1374698537497
    Confirmed cases after 19 days in Italy: 38475.79502058507

    Decoupled scenario:
    Confirmed cases after 19 days in UK: 675.2092466621101
    Confirmed cases after 19 days in Italy: 33601.8700472445

### Results of model: Coupled Scenario

```python
plt.figure( figsize=(10,6) )

# initial date is 3/1/20. The dataset starts at 1/22/20
days_till_initialdate = (31 - 22) + 29 + 1 # days from 1/22/20 to 3/1/20

plt.plot( sol.t, (It_it + Rt_it + Dt_it)*N_both, lw=3, label='simulation')
plt.plot(country_to_timeseries(df_confirmed, 'Italy').iloc[days_till_initialdate:], lw=3, label='data')
plt.legend()
plt.xticks(rotation=45)
plt.ylabel("Number of cases")
plt.title('Italy: Coupled with the UK');
```

```python
plt.figure( figsize=(10,6) )

plt.plot( sol.t, (It_uk + Rt_uk + Dt_uk)*N_both, lw=3, label='simulation')
plt.plot(country_to_timeseries(df_confirmed, 'United Kingdom').iloc[days_till_initialdate:], lw=3, label='data')
plt.legend()
plt.xticks(rotation=45)
plt.ylabel("Number of cases")
plt.title('UK: Coupled with Italy');
```

Note that the model is already predicting the cases in Italy and the UK very well. A sensitive parameter in the model is controlling $r$, the ratio of exposed individuals (people with the virus but without symptoms) against the number of registered cases. We will see that in the decoupled scenario, Italy grows similarly as in the coupled case. However, the UK will not have enough exposure to the virus whenever it is decoupled, so the model will largely underestimate its number of cases.

Note that there is no preventive action in the considered in the model. This means that we may be highly underestimating the ratio $r$ of unregistered individual **already exposed** the virus.

### No coupling scenario

```python
plt.figure( figsize=(10,6) )

plt.plot( sol.t, (It_it_noc + Rt_it_noc + Dt_it_noc)*N_both, lw=3, label='simulation')
plt.plot(country_to_timeseries(df_confirmed, 'Italy').iloc[days_till_initialdate:], lw=3, label='data')
plt.legend()
plt.xticks(rotation=45)
plt.ylabel("Number of cases")
plt.title('Italy: No coupling with the UK');
```

```python
plt.figure( figsize=(10,6) )

plt.plot( sol.t, (It_uk_noc + Rt_uk_noc + Dt_uk_noc)*N_both, lw=3, label='simulation')
plt.plot(country_to_timeseries(df_confirmed, 'United Kingdom').iloc[days_till_initialdate:], lw=3, label='data')
plt.legend()
plt.xticks(rotation=45)
plt.ylabel("Number of cases")
plt.title('UK: No coupling with Italy');
```

## Long term dynamics (with no preventive action)

Some long term predictions of the model. These do not intend to have predictive power, I show them to see the effects of the coupling of the model.

```python
####-----------------------------------####
#### Numerical simulation of the model ####
####-----------------------------------####
days = 365
t0,tf = (0, days) # days from 01/feb to date : 29+20
%time sol = integrate.solve_ivp(lambda t,x: coupled_SEIRD(x, t, *params, coupling=True), (t0,tf), x0, t_eval=np.linspace(0,days, days+1))
%time sol_nocoupling = integrate.solve_ivp(lambda t,x: coupled_SEIRD(x, t, *params, coupling=False), (t0,tf), x0, t_eval=np.linspace(0,days, days+1))

It_uk, Rt_uk, Dt_uk = (sol.y[2], sol.y[3], sol.y[4])
It_it, Rt_it, Dt_it = (sol.y[2+5], sol.y[3+5], sol.y[4+5])

It_uk_noc, Rt_uk_noc, Dt_uk_noc = (sol_nocoupling.y[2], sol_nocoupling.y[3], sol_nocoupling.y[4])
It_it_noc, Rt_it_noc, Dt_it_noc = (sol_nocoupling.y[2+5], sol_nocoupling.y[3+5], sol_nocoupling.y[4+5])
```

    CPU times: user 17.6 ms, sys: 0 ns, total: 17.6 ms
    Wall time: 17 ms
    CPU times: user 9.23 ms, sys: 0 ns, total: 9.23 ms
    Wall time: 8.97 ms

```python
# Basic plot of the dynamics

fig, ax = plt.subplots(1, 2, figsize=(12,8))

labels = ['Suceptible', 'Exposed', 'Infected', 'Recovered', 'Deceased']
labels = labels*3
for (i,y) in enumerate(sol.y):
    if i%5 == 0:
        0
#         continue

    if i < 5:
        ax[0].plot(sol.t, y*N_both, label=labels[i], lw=3)
    else:
        ax[1].plot(sol.t, y*N_both, label=labels[i], lw=3)

print("Coupled scenario")

ax[0].legend()
ax[1].legend()
# plt.yscale('log')
```

```python
# Basic plot of the dynamics

fig, ax = plt.subplots(1, 2, figsize=(12,8))

labels = ['Suceptible', 'Exposed', 'Infected', 'Recovered', 'Deceased']
labels = labels*3
for (i,y) in enumerate(sol_nocoupling.y):
    if i%5 == 0:
        0
#         continue

    if i < 5:
        ax[0].plot(sol_nocoupling.t, y*N_both, label=labels[i], lw=3)
    else:
        ax[1].plot(sol_nocoupling.t, y*N_both, label=labels[i], lw=3)

print("Decoupled scenario")

ax[0].legend()
ax[1].legend()
# plt.yscale('log')
```

Of course this projections are not realistic. They assume that most of the population will be affected after one year time (this could indeed be possible if there is no preventive action at all). Further, the coupled case should be corrected because a substantial part of the country's population flights to the neighbouring countries. We should model a return-flight model (maybe with time delays over $S(t)$ and $E(t)$.


| |Pierre Proulx, ing, professeur|
|:---|:---|
|Département de génie chimique et de génie biotechnologique |** GCH200-Phénomènes d'échanges I **|

### Section 6.2, exemple 6.2-2

##### Dans cet exemple on traite de facon légèrement différente car on suppose une conduite lisse. Ceci explique que le débit soit plus élevé.

```python
#
# Pierre Proulx
#
# Préparation de l'affichage et des outils de calcul symbolique
#
import sympy as sp
from IPython.display import *
sp.init_printing(use_latex=True)
```

```python
# Paramètres, variables et fonctions
rho,L,dP,D,v_z,mu=sp.symbols('rho,L,dP,D,v_z,mu')
```

```python
f=1/4*(D/L)*(dP/(1/2*rho*v_z**2))            # equation définissant le facteur f
Re=rho*v_z*D/mu
f_L=16/Re                                # f calculé si Re<2100
f_T=0.0791/Re**0.25                      # si Re > 2100
```

```python
# Dictionnaire contenant les valeurs des paramètres
dico={'rho':1000,'D':7.981*2.54/100,'mu':0.001,'dP':3/14.7*101325,'L':1000*0.3048}
#
# a priori on ne sait pas si c'est laminaire ou pas,
# donc en premier on teste laminaire
#
eqL=sp.Eq(f-f_L)
v1=sp.solve((eqL,0),v_z)
Re1=Re.subs(v1)
Re1=Re1.subs(dico).evalf()
if Re1 > 2100:
    print(' Re estimé =',Re1)
    print(' Pas laminaire, on calcule en turbulent')
    eqT=sp.Eq(f-f_T)
    v2=sp.solve((eqT,0),v_z,dict=True)    #
    display(v2)
    v2=v2[0]                              # il y aura plusieurs racines
    display(v2)                           #  la première racine
    Re2=Re.subs(v2)                       #  est justement réelle
    Re2=Re2.subs(dico)
    print(' le Reynolds calculé avec Blasius pour conduite lisse')
    display(Re2.evalf())
    display(' Débit massique calculé en kg/sec')
    V=((v_z.subs(v2)).subs(dico)).evalf()        # evalf() car le résultat est symbolique
    W=rho*sp.pi*D**2/4*V                         # et on le veut en format numérique,
    display(W.subs(dico).evalf())                # c'est similaire à Matlab.
else:
    print(' le Reynolds calculé laminaire pour conduite lisse')
    display(Re1)
    V=((v_z.subs(v1)).subs(dico)).evalf()
    W=rho*sp.pi*D**2/4*V
    display(' Débit massique calculé en kg/sec')
    display(W.subs(dico).evalf())
```

```python

```


# Exercises

In this section we have two exercises:
1. Implement the polynomial kernel.
2. Implement the multiclass C-SVM.

## Shared

```python
def get_iris():
    iris = load_iris()
    data_set = iris.data
    labels = iris.target
    return data_set, labels
```

## Polynomial kernel

You need to extend the ``build_kernel`` function and implement the polynomial kernel if the ``kernel_type`` is set to 'poly'. The equation that needs to be implemented:
\begin{equation}
K=(X^{T}*Y)^{d}.
\end{equation}

```python
def build_kernel(data_set, kernel_type='linear'):
    kernel = np.dot(data_set, data_set.T)
    if kernel_type == 'rbf':
        sigma = 1.0
        objects_count = len(data_set)
        b = np.ones((len(data_set), 1))
        kernel -= 0.5 * (np.dot((np.diag(kernel)*np.ones((1, objects_count))).T, b.T)
                         + np.dot(b, (np.diag(kernel) * np.ones((1, objects_count))).T.T))
        kernel = np.exp(kernel / (2. * sigma ** 2))
    elif kernel_type == 'poly':
#         pass # put your code here
        d = len(data_set[0])
        kernel = np.matrix(kernel)**d
        kernel = np.array(kernel)
    return kernel
```

```python
# Test
data_set, _ = get_iris()
build_kernel(data_set, kernel_type='poly')
```

    array([[2.72973418e+13, 2.57264295e+13, 2.51470225e+13, ...,
            4.16696015e+13, 4.20374312e+13, 3.91971566e+13],
           [2.57264295e+13, 2.42459217e+13, 2.36998574e+13, ...,
            3.92716193e+13, 3.96182819e+13, 3.69414580e+13],
           [2.51470225e+13, 2.36998574e+13, 2.31660925e+13, ...,
            3.83871263e+13, 3.87259806e+13, 3.61094455e+13],
           ...,
           [4.16696015e+13, 3.92716193e+13, 3.83871263e+13, ...,
            6.36095738e+13, 6.41710882e+13, 5.98353431e+13],
           [4.20374312e+13, 3.96182819e+13, 3.87259806e+13, ...,
            6.41710882e+13, 6.47375597e+13, 6.03635408e+13],
           [3.91971566e+13, 3.69414580e+13, 3.61094455e+13, ...,
            5.98353431e+13, 6.03635408e+13, 5.62850541e+13]])

## Implement a multiclass C-SVM

Use the classification method that we used in notebook 7.3 and IRIS dataset to build a multiclass C-SVM classifier. Most implementation is about a function that will return the proper data set that need to be used for the prediction. You need to implement:
- ``choose_set_for_label``
- ``get_labels_count``

```python
from sklearn.datasets import load_iris
import numpy as np
from sklearn.model_selection import train_test_split
import cvxopt

def choose_set_for_label(data_set, label):
    # your code here
    data_set = data_set[labels!=2]
    labels = labels[labels!=2]

    train_data_set, test_data_set, train_labels, test_labels = train_test_split(
    data_set, labels, test_size=0.2, random_state=15)

    train_labels[train_labels<1] = -1
    test_labels[test_labels<1] = -1

    return train_data_set, test_data_set, train_labels, test_labels
```

```python
def get_labels_count(data_set):
    labels_count = len(data_set[0])
    return labels_count
```

Use the code that we have implemented earlier:

```python
def train(train_data_set, train_labels, kernel_type='linear', C=10, threshold=1e-5):
    kernel = build_kernel(train_data_set, kernel_type=kernel_type)

    P = train_labels * train_labels.transpose() * kernel
    q = -np.ones((objects_count, 1))
    G = np.concatenate((np.eye(objects_count), -np.eye(objects_count)))
    h = np.concatenate((C * np.ones((objects_count, 1)), np.zeros((objects_count, 1))))

    A = train_labels.reshape(1, objects_count)
    A = A.astype(float)
    b = 0.0

    sol = cvxopt.solvers.qp(cvxopt.matrix(P), cvxopt.matrix(q), cvxopt.matrix(G), cvxopt.matrix(h), cvxopt.matrix(A), cvxopt.matrix(b))

    lambdas = np.array(sol['x'])

    support_vectors_id = np.where(lambdas > threshold)[0]
    vector_number = len(support_vectors_id)
    support_vectors = train_data_set[support_vectors_id, :]

    lambdas = lambdas[support_vectors_id]
    targets = train_labels[support_vectors_id]

    b = np.sum(targets)
    for n in range(vector_number):
        b -= np.sum(lambdas * targets * np.reshape(kernel[support_vectors_id[n], support_vectors_id], (vector_number, 1)))
    b /= len(lambdas)

    return lambdas, support_vectors, support_vectors_id, b, targets, vector_number

def build_kernel(data_set, kernel_type='linear'):
    kernel = np.dot(data_set, data_set.T)
    if kernel_type == 'rbf':
        sigma = 1.0
        objects_count = len(data_set)
        b = np.ones((len(data_set), 1))
        kernel -= 0.5 * (np.dot((np.diag(kernel)*np.ones((1, objects_count))).T, b.T)
                         + np.dot(b, (np.diag(kernel) * np.ones((1, objects_count))).T.T))
        kernel = np.exp(kernel / (2. * sigma ** 2))
    return kernel

def classify_rbf(test_data_set, train_data_set, lambdas, targets, b, vector_number, support_vectors, support_vectors_id):
    kernel = np.dot(test_data_set, support_vectors.T)
    sigma = 1.0
    #K = np.dot(test_data_set, support_vectors.T)
    #kernel = build_kernel(train_data_set, kernel_type='rbf')
    c = (1. / sigma * np.sum(test_data_set ** 2, axis=1) * np.ones((1, np.shape(test_data_set)[0]))).T
    c = np.dot(c, np.ones((1, np.shape(kernel)[1])))
    #aa = np.dot((np.diag(K)*np.ones((1,len(test_data_set)))).T[support_vectors_id], np.ones((1, np.shape(K)[0]))).T
    sv = (np.diag(np.dot(train_data_set, train_data_set.T))*np.ones((1,len(train_data_set)))).T[support_vectors_id]
    aa = np.dot(sv,np.ones((1,np.shape(kernel)[0]))).T
    kernel = kernel - 0.5 * c - 0.5 * aa
    kernel = np.exp(kernel / (2. * sigma ** 2))

    y = np.zeros((np.shape(test_data_set)[0], 1))
    for j in range(np.shape(test_data_set)[0]):
        for i in range(vector_number):
            y[j] += lambdas[i] * targets[i] * kernel[j, i]
        y[j] += b
    return np.sign(y)
```

```python
# modify this part

data_set, labels = get_iris()

train_data_set, test_data_set, train_labels, test_labels = choose_set_for_label(data_set, labels)

objects_count = len(train_data_set)
print(objects_count)

lambdas, support_vectors, support_vectors_id, b, targets, vector_number = train(train_data_set, train_labels, kernel_type='rbf')
predicted = classify_rbf(test_data_set, train_data_set, lambdas, targets, b, vector_number, support_vectors, support_vectors_id)
predicted = list(predicted.astype(int))

from sklearn.metrics import accuracy_score

print(accuracy_score(predicted, test_labels))
```

    80
         pcost       dcost       gap    pres   dres
     0:  9.6305e+01 -1.2289e+03  2e+03  2e-01  2e-15
     1:  5.9143e+01 -1.2031e+02  2e+02  5e-03  2e-15
     2:  7.0898e+00 -1.6497e+01  2e+01  2e-16  2e-15
     3: -5.2057e-01 -3.7668e+00  3e+00  2e-16  1e-15
     4: -1.1712e+00 -1.8374e+00  7e-01  2e-16  4e-16
     5: -1.3952e+00 -1.6846e+00  3e-01  2e-16  2e-16
     6: -1.4671e+00 -1.5679e+00  1e-01  2e-16  2e-16
     7: -1.5060e+00 -1.5164e+00  1e-02  2e-16  2e-16
     8: -1.5105e+00 -1.5106e+00  1e-04  2e-16  2e-16
     9: -1.5105e+00 -1.5105e+00  1e-06  2e-16  2e-16
    Optimal solution found.
    0.85

```python

```


## Lens-focused Laguerre-Gauss beam as per GLMT:

A Finite Series sketch

```python
# Imports
import numpy as np               # Ye olde numpy
import scipy.special as sp       # Special functions
import matplotlib.pyplot as plt  # Plots results
import mpmath                    # Arbitrary precision
import scipy.integrate as intg   # Numerical integration

import inspect                   # Maybe remove later (decorator things)
import time                      # For performance evalution
```

```python
mpmath.mp.dps = 50
integrationdps = 200
```

```python
# LFLG beam parameters:
p, l = 0, 2
alpha, beta = 1, 0
r_focus = 1E-3
wavelength = 6328E-9
na = .95
w0 = 3.2 * wavelength

k = 2 * np.pi / wavelength
s = 1 / k / w0

parameters = {
    "p": p,
    "l": l,
    "alpha": alpha,
    "beta": beta,
    "r_focus": r_focus,
    "wavelength": wavelength,
    "na": na,
    "w0": w0,
    "delta": 1,
}
```

```python
# Memoization decorator
def memoized(func):
    CACHE = {}
    def memoized_func(*args, **kwargs):
        kwvals = [val for _, val in kwargs.items()]
        if (*args, *kwvals) not in CACHE:
            value = func(*args, **kwargs)
            CACHE[(*args, *kwvals)] = value
            return value
        return CACHE[(*args, *kwvals)]
    return memoized_func
```

```python
class LGBeamFS:
    def __init__(self, **kwargs):
        self.__dict__.update(parameters)
        self.__dict__.update(kwargs)

        self.k = 2 * np.pi / self.wavelength
        self.s = 1 / self.k / self.w0
        if 'delta' not in parameters and 'delta' not in kwargs:
            self.delta = 1

    def a(self, eta):
        return eta * self.r_focus * self.s * mpmath.sqrt(2)

    def lens_integrand(self, eta, n, p, kind='A'):
        if kind == 'E':
            return self.lens_integrand(eta, n + 1, p, kind='A')

        pm = 1 if kind == 'A' else -1
        if p == 0:
            lfactor = 1
        elif p >= 1:
            lfactor = mpmath.laguerre(p - 1, self.l, self.a(eta) ** 2) \
                    * self.a(eta) ** 2
        else:
            raise ValueError("Negative values of p are not acceptable.")

        kz_factor = mpmath.power(self.k ** 2 - eta ** 2, 1/ 4)
        if kind in ['A', 'B']:
            kz_factor = 1 / kz_factor
            power = n - 1
        elif kind in ['C', 'D']:
            kz_factor = kz_factor / self.k
            power = n - 2
        else:
            raise ValueError("Only kinds A, B, C, D and E recognized.")

        res = mpmath.power(eta, np.abs(l + 1)) \
            * kz_factor * lfactor * mpmath.exp(-self.a(eta) ** 2 / 2) \
            * (1 + pm * mpmath.sqrt(self.k ** 2 - eta ** 2) / self.k) \
            * mpmath.power(eta / k, power)
        return res

    def F(self, j, u):
        return (-1) ** ((j - u) / 2) \
             * mpmath.gammaprod([], [(j - u) / 2 + 1, (j + u) / 2 + 1])
    # / mpmath.factorial((j - u) / 2) / mpmath.factorial((j + u) / 2)

    @memoized
    def lens_integral(self, n, p, kind='A', intg_func=intg.quadrature):
        FUNCS = (intg.quad, intg.quadrature, intg.romberg, mpmath.quad)
        max_eta = self.k * self.na * self.delta
        np_lint = np.frompyfunc(self.lens_integrand, 4, 1)
        if intg_func not in FUNCS:
            raise ValueError("intg_func", intg_func, "not supported.")
        if intg_func is intg.quadrature:
            return intg_func(np_lint, 0, max_eta, args=(n, p, kind),
                             rtol=1E-64, maxiter=200)[0]
        elif intg_func is mpmath.quad:
            def lint(eta):
                return self.lens_integrand(eta, n, p, kind=kind)
            return intg_func(lint, [0, max_eta])
        elif intg_func is intg.romberg:
            return intg_func(np_lint, 0, max_eta, args=(n, p, kind),
                             rtol=1E-100)
        elif intg_func is intg.quad:
            return mpmath.mpc(intg.quad(np_lint, 0, max_eta, args=(n, p, kind),
                             epsrel=1E-200, limit=400)[0])
        else:
            raise NotImplemented(intg_func, "not supported")

    @memoized
    def maclaurin_nmevn(self, j, p, intg_func=intg.quad, m=0):
        lpart, rpart = 1, 1
        premul = 1j * np.sqrt(2 / np.pi) * (self.alpha + 1j * self.beta) \
               / mpmath.power(2, j)
        if j <= l:
            lpart = 0
            if j <= l - 2:
                rpart = 0

        if rpart != 0:
            rpart = self.F(j - 1, self.l - 2) \
                  * self.lens_integral(j, p, kind='B', intg_func=intg_func)
            if lpart != 0:
                lpart = self.F(j - 1, self.l) \
                      * self.lens_integral(j, 0, kind='A', intg_func=intg_func)
        if p == 1:
            if rpart != 0:
                rpart = self.F(j - 1, self.l - 2) \
                      * ((l + 1) * self.lens_integral(j, 0, kind='B',
                                                      intg_func=intg_func) \
                        - self.lens_integral(j, 1, kind='B'))
                if lpart != 0:
                    lpart = self.F(j - 1, self.l) \
                          * ((l + 1) * self.lens_integral(j, 0, kind='A',
                                                          intg_func=intg_func) \
                            - self.lens_integral(j, 1, kind='A',
                                                 intg_func=intg_func))
        elif p != 0:
            premul = premul / p
            return ((2 * p - 1 + self.l) / p * self.maclaurin(j, p - 1,
                                                              intg_func=intg_func) \
                    - (p - 1 + self.l) / p * self.maclaurin(j, p - 2,
                                                           intg_func=intg_func) \
                    - premul * (lpart + rpart)
                   )

        return premul * (lpart + rpart)

    def odd_base(self, j, p, intg_func=intg.quad, m=0):
        if m == self.l - 1:
            upper_lim = l + 1
            lower_lim = l - 1
            polarizing_factor = 1j * self.alpha - self.beta
        lpart, rpart = 1, 1
        premul = 1j * np.sqrt(2 / np.pi) * polarizing_factor \
               / mpmath.power(2, j - 1)
        if j <= upper_lim:
            lpart = 0
            if j <= lower_lim:
                rpart = 0

        if rpart != 0:
            rpart = self.F(j - 2, self.l - 2) \
                  * self.lens_integral(j, p, kind='D', intg_func=intg_func) \
                  + self.F(j - 1, self.l - 1) \
                  * self.lens_integral(j, p, kind='E', intg_func=intg_func)
            if lpart != 0:
                lpart = self.F(j - 2, self.l) \
                      * self.lens_integral(j, p, kind='C', intg_func=intg_func)
        return premul * (lpart + rpart)

    @memoized
    def maclaurin_nmodd(self, j, p, intg_func=intg.quad, m=0):
        bas = self.odd_base(j, p, intg_func=intg_func, m=m)
        if p == 1:
            bas = (self.l + 1) * self.odd_base(j, 0) - bas
        elif p != 0:
            return ((2 * p - 1 + self.l) / p \
                    * self.maclaurin(j, p - 1, intg_func=intg_func)
                    - (p - 1 + self.l) / p \
                    * self.maclaurin(j, p - 2, intg_func=intg_func) \
                    - bas / p
                   )
        return bas

    def maclaurin(self, j, p, intg_func=intg.quad, m=0):
        if (j - self.l) % 2 == 1:
            return self.maclaurin_nmevn(j, p, intg_func=intg_func, m=m)
        return self.maclaurin_nmodd(j, p, intg_func=intg_func, m=m)

    def nconv_tmbsc_lm1_evn(self, n, intg_func=intg.quad):
        premul = 1j ** n * (-1) ** ((n - self.l + 1) / 2) \
               * mpmath.power(2, n + 1.5 - self.l) * np.sqrt(np.pi) \
               / mpmath.gammaprod([(n - l + 3) / 2], [(n + l) / 2])

        fs = 0
        q = 0
        while q <= n / 2:
            inc = mpmath.power(2, -2 * q) \
                * mpmath.gammaprod([.5 + n - q], [q + 1]) \
                * self.maclaurin(n - 2 * q, p, intg_func=intg_func, m=self.l - 1)
            fs += inc
            q += 1
        return premul * fs

    def nconv_tmbsc_lm1_odd(self, n, intg_func=intg.quad):
        premul = 1j ** n * (-1) ** ((n + self.l) / 2 - 1) \
               * mpmath.power(2, n - .5 - self.l) * np.sqrt(np.pi) \
               / mpmath.gammaprod([(n - l) / 2], [(n + l + 1) / 2])

        fs = 0
        q = 0
        while q <= n / 2:
            inc = mpmath.power(2, -2 * q) \
                * mpmath.gammaprod([.5 + n - q], [q + 1]) \
                * self.maclaurin(n - 2 * q, p, intg_func=intg_func, m=self.l - 1)
            fs += inc
            q += 1
        return premul * fs

    def nconv_tmbsc_lm1(self, n, intg_func=intg.quad):
        if (n - self.l) % 2:
            return self.nconv_tmbsc_lm1_evn(n, intg_func=intg_func)
        return self.nconv_tmbsc_lm1_odd(n, intg_func=intg_func)

    def tmbsc_lm1(self, n, intg_func=intg.quad):
        with mpmath.workdps(integrationdps):
            return (-1) ** (n + 1) \
                 * mpmath.conj(self.nconv_tmbsc_lm1(n, intg_func=intg_func))
```

## Testing

#### Analyzing performance on different methods for integration

```python
# Decorator for timing functions
def timed(func):
    def timed_func(*args, **kwargs):
        print("> Evaluating", func.__name__, "...")
        start = time.time()
        val = func(*args, **kwargs)
        finish = time.time()
        print(">", func.__name__, "evaluated in", finish - start, "seconds.")
        return val
    return timed_func
```

```python
lgbeam = LGBeamFS(**parameters)

def timed_bscs(nmax, m, l, method=intg.quad, plot=True,
               dps=integrationdps, lgbeam=lgbeam, nmin=None):
    if m != l - 1:
        raise NotImplementedError("Still not available for m =", m)
    nmin = nmin if nmin is not None else m
    ns = np.arange(nmin, nmax + 1, 2)
    @timed
    def bscs():
        gs = []
        with mpmath.workdps(dps):
            for n in ns:
                gs.append(mpmath.norm(lgbeam.tmbsc_lm1(n, intg_func=method)))
        return gs

    print(">> ", len(ns), "BSCs in evaluation...")
    gs = bscs()
    if plot:
        plt.plot(ns, gs)
        plt.grid()
        plt.show()
    return gs
```

 Trusted BSCs will be saved in the $\texttt{CONTROL_BSCS}$ variable:

```python
CONTROL_BSCS = timed_bscs(500, l - 1, l, method=mpmath.quad)
```

```python
nmax = 500
m = l - 1
simargs = [nmax, m, l]
```

For now, $\texttt{mpmath.quad}$ has the longest runtime, but offers results I better trust.

Notice runtime below might seem "too short" because of memoization, which is fundamental to have in our case due to deep recursion processes.

```python
MPMATH_BSCS = timed_bscs(*simargs, method=mpmath.quad)
```

 $\texttt{scipy.integrate.quad}$: quadrature as in FORTRAN's $\texttt{QUADPACK}$

```python
QUAD_BSCS = timed_bscs(*simargs, method=intg.quad)
```

 $\texttt{scipy.integrate.quadrature}$: adaptative quadrature

```python
QUADRATURE_BSCS = timed_bscs(*simargs, method=intg.quadrature)
```

 $\texttt{scipy.integrate.romberg}$: Romberg quadrature

```python
ROMBERG_BSCS = timed_bscs(*simargs, method=intg.romberg)
```

 There also is $\texttt{scipy.integrate.simps}$ to test, but it requires sampled data which is to be worked on later.

```python
bsclsts = {
    "mpmath.quad": MPMATH_BSCS,
    "scipy.integrate.quad": QUAD_BSCS,
    "scipy.integrate.quadrature": QUADRATURE_BSCS,
    "scipy.integrate.romberg": ROMBERG_BSCS
}
linestyles = {}
l0 = 3.5
stls = ['-', '--', '-.', ':']
for ind, key in enumerate(bsclsts):
    linestyles[key] = {
        "linewidth": l0,
        "linestyle": stls[ind % len(stls)]
    }
    l0 = l0 - .5

nmin = m
ns = np.arange(nmin, nmax + 1, 2)
print("Values of n computed from", ns[0], "to", ns[-1])
plt.grid()
for name, lst in bsclsts.items():
    plt.plot(ns, lst, label=name, alpha=.75, **linestyles[name])
plt.legend()
plt.xlabel("n")
plt.ylabel("$\\left|g_{n, TM}^{l - 1}(N)\\right|$")
plt.ylim(top=4E15, bottom=0)
plt.show()
```

In sum, we have se following status for now (color code: tested as acceptable, seems acceptable, pleasehelp):
 -  $\texttt{mpmath.quad}$  _(with suboptimal performance, maybe)_
 -  $\texttt{scipy.integrate.quad}$ _(even though results seem to hold for $n < n_{problem}\approx 150$)_
 -  $\texttt{scipy.integrate.quadrature}$
 -  $\texttt{scipy.integrate.romberg}$

### Thought process:

We need the following function represented by $\texttt{gammaprod}$ in the $\texttt{mpmath}$ package:

$$ \texttt{gammaprod}(\textbf{u}, \textbf{v})  = \frac{\Gamma(u_1) \cdot \Gamma(u_2) \cdot\ldots\cdot \Gamma(u_n)}{\Gamma(v_1) \cdot \Gamma(v_2) \cdot\ldots\cdot \Gamma(v_m)}$$

For more info see entry in $\texttt{mpmath}$ documentation.

Also note that here we will compute harmonic modes in the $N$-convention, meaning we must not forget to convert the BSCs to the $P$-convention via:

$$g_{n, TX}^m (P) = (-1)^{n + 1} g_{n, TX}^{-m} (N)^*.$$

```python
# N-convention _ TM LFLG BSC _ m = l - 1 finite series
def nconv_tmlflgbsc_lm1fs(n, m, **parameters):
    pass

# [P-convention _] TM LFLG BSC _ m = l - 1 finite series
def tmlflgbsc_lm1fs(n, m, **parameters):
    return (-1 ** (n + 1)) * nconv_tmlflgbsc_lm1fs(n, -m, **parameters)
```

#### Case (ii): $m = l - 1, l > 0$
$$    \left[ g_{n, TM}^{l-1}(N) \right]_p = i^n \frac{\sqrt{\pi}}{2^{l}} (-1)^{\frac{n - l + 1}{2}} \frac{\Gamma\left( \frac{n - l + 3}{2} \right)}{\Gamma\left( \frac{n + l}{2} \right)}
    \sum_{q=0}^{\leq n / 2} 2^{\frac{1}{2}+n - 2q} \frac{\Gamma\left(\frac{1}{2}+n-q\right)}{q!} \left[ b_{n-2q} \right]_p$$

## Problem solving

Something happened in different methods of integration. Let's find out:

$$ a(\eta) = \sqrt{2}\frac{R_f}{kw_0} \eta = \sqrt{2} R_f s \eta $$

$$ A_{l0}^j= \int_0^{k\text{NA}} \frac{\eta^{|l| + 1}}{(k^2 - \eta^2)^{\frac{1}{4}}} \exp\left( -\frac{a(\eta)^2}{2} \right) \left( 1 + \frac{\sqrt{k^2 - \eta^2}}{k} \right) \left( \frac{\eta}{k} \right)^{j - 1} \text{d}\eta $$

$$ A_{lp}^j= \int_0^{k\text{NA}} \frac{\eta^{|l| + 1}}{(k^2 - \eta^2)^{\frac{1}{4}}} a(\eta)^2 L_{p-1}^l\left(a(\eta)^2\right) \exp\left( -\frac{a(\eta)^2}{2} \right) \left( 1 + \frac{\sqrt{k^2 - \eta^2}}{k} \right) \left( \frac{\eta}{k} \right)^{j - 1} \text{d}\eta $$

```python
def a(eta):
    return mpmath.sqrt(2) * r_focus / k / w0 * eta
a(na)
```

    mpf('0.000066820351589785206390909357468025590338791176126779937')

```python
def integrand(eta, l, p, j, kind='A'):
    pm = 1 if kind == 'A' else -1
    laguerre_term = 1
    if p > 0:
        laguerre_term = a(eta) ** 2 * mpmath.laguerre(p - 1, l, a(eta) ** 2)

    return ( mpmath.power(eta, np.abs(l) + 1) / mpmath.power(k ** 2 - eta ** 2, .25) \
           * laguerre_term * mpmath.exp(-(a(eta) ** 2 / 2 )) \
           * (1 + pm * mpmath.sqrt(k ** 2 - eta ** 2) / k) \
           * mpmath.power(eta / k, j - 1) )
```

```python
def eval_integral(l, p, j, kind='A', delta=1, method=mpmath.quad):
    def argument(eta):
        return integrand(eta, l, p, j, kind=kind)
    npintg = np.frompyfunc(argument, 1, 1)
    if method is mpmath.quad:
        return mpmath.quad( argument, [0, k * na * delta] )
    if method is intg.quad:
        return intg.quad( argument, 0, k * na * delta )
```

```python
def integral_performance(nmin=m, nmax=nmax, method=mpmath.quad):
    aquad_lst = []
    bquad_lst = []
    ns = np.arange(nmin, nmax + 1, 2)
    stt = time.time()
    fig, axes = plt.subplots(ncols=2)
    print("Evaluating", method.__module__, method.__name__, "for", len(ns), "values... (twice)")
    for j in ns:
        aquad_lst.append( mpmath.norm(eval_integral(l, p, j, method=method, kind='A')) )
        bquad_lst.append( mpmath.norm(eval_integral(l, p, j, method=method, kind='B')) )
    fin = time.time()
    axes[0].plot(ns, aquad_lst)
    axes[1].plot(ns, bquad_lst)
    plt.grid()
    plt.show()
    print(". Evaluated", method.__module__, method.__name__, "in", (fin - stt), "seconds.")
    return aquad_lst, bquad_lst
```

```python
mpmathint = integral_performance(method=mpmath.quad)
```

```python
quadint = integral_performance(method=intg.quad)
```

```python
difs = []
for i, v in enumerate(mpmathint):
    difs.append(np.abs(mpmathint[0][i] - quadint[0][i]) ** 2)
plt.plot(difs)
plt.grid()
plt.show()
```

From what I see here, $\texttt{scipy.integrate.quad}$ might not have to explode as seen in the cells above only because of the integration. This could be one of two cases:

 - The ever-so-slight differences in results the cell directly above show become very significant somewhere down the line as error propagates, or
 - We have somehow results shown in smaller precision in the $\texttt{scipy.integrate.quad}$ case.

### Doodles

Trying to figure out some stuff about decorators...

```python
def zero(*args, **kwargs):
    return 0
def epsilonized(inname, outname):

    def find_arg_with_name(name, arglist, func_args=[], func_kwargs={}):
        value = fkwargs.get(name)
        if value is None:
            return func_args[arglist.index(name)]
        return value

    def decorator_eps(func):
        def epsilonized_func(*args, **kwargs):
            j = find_arg_with_name(
                inname, inspect.getfullargspec(func),
                func_args=args, func_kwargs=kwargs
            )
            if j in A:
                value = 0
            else:
                value = func(*args, **kwargs)
            return value
        return epsilonized_func
    return decorator_eps
```

Quadrature integration from $\texttt{scipy.integrate}$

```python
def foo(x, a, b=0):
    return a * x + b

intg.quad(foo, 0, 1, args=(1, 1)) # args are processed in order; even kwargs
```

    (1.5, 1.6653345369377348e-14)

```python
d = {1: 2, 3: 4}
tuples = d.items()
print(tuples)

print(enumerate(d))
for i in enumerate(d):
    print(i)
```

    dict_items([(1, 2), (3, 4)])
    <enumerate object at 0x00BA0C60>
    (0, 1)
    (1, 3)

```python
kw = {1: 2}
kw.items()
```

    dict_items([(1, 2)])

```python
def foo(x, a, b=0):
    return a * x + b

def bar(x):
    return foo(x, 2, b=1)

mpmath.quad(bar, [0, 1])
```

    mpf('2.0')

```python
# Testing the "timed" decorator
@timed
def timed_foo(x):
    for _ in range(x):
        time.sleep(1)

timed_foo(1)

m
```

    > Evaluating timed_foo ...
    > timed_foo evaluated in 1.001002311706543 seconds.

    1

```python
MPMATH_BSCS = QUADRATURE_BSCS = ROMBERG_BSCS = [0 for n in ns]
```

```python
import pickle
# n-m even BSCs to dict
# tmbscdict = {(n = m + 2k, m): value of TM mode BSC}
quaddict = {}
i = m
for bsc in CONTROL_BSCS:
    quaddict[i, m] = bsc
    i += 2
with open("quad_dict.pickle", "wb") as f:
    pickle.dump(quaddict, f)
```

Fun with $\texttt{sympy}$

```python
import sympy as sym
```

```python
_l, _p, _k, _lambda, _w0, _k = sym.symbols('l, p, k, lambda, w0, k')
_rf, _s = sym.symbols("R_f s")
_eta, _j = sym.symbols('eta j')
_na = sym.symbols('\\text{NA}')
class _a(sym.Function):
    @classmethod
    def eval(cls, x):
        return _rf * sym.sqrt(2) / _k / _w0 * x
```

```python
def express(a, b, name):
    symb = sym.symbols(name)
    sol = sym.solve(a - symb, b)
    return (symb, sol[0])
```

```python
A = sym.Integral(
    ( _eta ** (sym.Abs(_l) + 1) / (_k ** 2 - _eta ** 2) ** sym.Rational(1, 4) \
    * sym.exp(-_a(_eta) ** 2 / 2) \
    * (1 + sym.sqrt(_k ** 2 - _eta ** 2) / _k) \
    * (_eta / _k) ** (_j - 1)),
    (_eta, 0, _k * _na)
)
af = sym.Function('a')
A.subs(_a(_eta), af(_eta))
```

$\displaystyle \int\limits_{0}^{\text{NA} k} \frac{\eta^{\left|{l}\right| + 1} \left(\frac{\eta}{k}\right)^{j - 1} \left(1 + \frac{\sqrt{- \eta^{2} + k^{2}}}{k}\right) e^{- \frac{R_{f}^{2} \eta^{2}}{k^{2} w_{0}^{2}}}}{\sqrt[4]{- \eta^{2} + k^{2}}}\, d\eta$

```python
A
```

$\displaystyle \int\limits_{0}^{\text{NA} k} \frac{\eta^{\left|{l}\right| + 1} \left(\frac{\eta}{k}\right)^{j - 1} \left(1 + \frac{\sqrt{- \eta^{2} + k^{2}}}{k}\right) e^{- \frac{R_{f}^{2} \eta^{2}}{k^{2} w_{0}^{2}}}}{\sqrt[4]{- \eta^{2} + k^{2}}}\, d\eta$

```python
lgbeam = LGBeamFS()
m = l - 1
ll = []
ns = np.arange(1, 10)
for n in ns:
    ll.append(mpmath.norm(lgbeam.tmbsc_lm1(n)))
plt.plot(ns, ll)
plt.show()
ll
```

```python

```