#### Jupyter notebooks

This is a [Jupyter](http://jupyter.org/) notebook using Python.  You can install Jupyter locally to edit and interact with this notebook.

# Finite difference/collocation methods

Consider the boundary value problem

$$ \begin{gather} -\frac{d^2 u}{dx^2} = f(x) \quad x \in \Omega = (-1,1) \\
u(-1) = a \quad \frac{du}{dx}(1) = b . \end{gather} $$

$f(x)$ is the "forcing" term and we have a Dirichlet boundary condition at the left end of the domain and a Neumann condition on the right end.  We need to choose
* how to represent $u(x)$, including evaluating it on the boundary,
* how to compute derivatives of $u$,
* in what sense to ask for the differential equation to be satisfied,
* where to evaluate $f(x)$ or integrals thereof,
* how to enforce boundary conditions.

## The framework

* Finite difference (FD) methods choose to represent the function $u(x)$ by its values $u_i = u(x_i)$ at a discrete set of points $$ -1 = x_0 < x_1 < \dotsb < x_n = 1 . $$
The FD framework does not uniquely specify the solution values at points outside this discrete set.
* FD computes derivatives at $x_i$ via differencing formulas involving a finite number of neighbor points (independent of the total number of points $n$).  This will be our main focus.
* FD methods ask for the differential equation to be satisfied pointwise at each $x_i$ in the interior of the domain.
* FD methods evaluate the forcing term $f$ pointwise at $x_i$.
* FD methods approximate derivatives at discrete boundary points ($x_n = 1$ above), typically using one-sided differencing formulas.

## Differencing formulas

How can we compute $\frac{du}{dx}(x_i)$ using $u_i$ and $u_{i+1}$?  How accurate is this approximation?

### Taylor series

Without loss of generality, we may assume $x_i = 0$ by shifting our function.  For notational convenience, we will also define $h = x_{i+1} - x_i$.
To determine the order of accuracy, we represent the function $u(x)$ by its Taylor series
$$ u(x) = u(0) + u'(0)x + u''(0)x^2/2! + O(x^3)$$
and substitute into the differencing formula
$$ \begin{split} u'(0) \approx \frac{u(h) - u(0)}{h} = h^{-1} \Big( u(0) + u'(0) h + u''(0)h^2/2 + O(h^3) - u(0) \Big) \\
= u'(0) + u''(0)h/2 + O(h^2) . \end{split}$$
Evidently the error in this approximation is $u''(0)h/2 + O(h^2)$. We say this method is *first order accurate*.

```python
%matplotlib inline
import numpy
from matplotlib import pyplot
pyplot.style.use('ggplot')

n = 10
h = 2/(n-1)
x = numpy.linspace(-1,1,n)
u = numpy.sin(x)
pyplot.plot(x[:-1], (u[1:] - u[:-1]) / h, 'o')
pyplot.plot(x[:-1], numpy.cos(x[:-1]));
```

This "has the right shape", but the numerical approximation is "shifted" to the left of the analytic derivative. If we differenced to the left, it would be shifted the other way.

####  Is the centered difference formula better?

Here we try

$$ \begin{split} u'(0) \approx \frac{u(h) - u(-h)}{2h} \\
= (2h)^{-1} \Big( \big[ u(0) + u'(0)h + u''(0)h^2/2 + u'''(0)h^3/6 + O(h^4) \Big] - \big[ u(0) - u'(0)h + u''(0)h^2/2 - u'''(0)h^3/6 + O(h^4) \big] \Big) \\
= u'(0) + u'''(0)h^2/6 + O(h^3) \end{split} $$

```python
def diff1l(x, u):
    return x[:-1], (u[1:] - u[:-1]) / (x[1:] - x[:-1])

def diff1r(x, u):
    return x[1:],  (u[1:] - u[:-1]) / (x[1:] - x[:-1])

def diff1c(x, u):
    return x[1:-1], (u[2:] - u[:-2]) / (x[2:] - x[:-2])

for diff in (diff1l, diff1r, diff1c):
    xx, yy = diff(x, u)
    pyplot.plot(xx, yy, 'o', label=diff.__name__)
pyplot.plot(x, numpy.cos(x), label='exact')
pyplot.legend(loc='upper right');
```

```python
grids = 2**numpy.arange(3,10)
def grid_refinement_error(f, fp, diff):
    error = []
    for n in grids:
        x = numpy.linspace(-1, 1, n)
        xx, yy = diff(x, f(x))
        error.append(numpy.linalg.norm(yy - fp(xx), numpy.inf))
    return grids, error

for diff in (diff1l, diff1r, diff1c):
    ns, error = grid_refinement_error(numpy.sin, numpy.cos, diff)
    pyplot.loglog(1/ns, error, 'o', label=diff.__name__)
hs = 2 / (grids - 1)
pyplot.loglog(hs, hs, label='$h$')
pyplot.loglog(hs, hs**2, label='$h^2$')
pyplot.xlabel('Resolution $h$')
pyplot.ylabel('Derivative error')
pyplot.legend(loc='upper right');
```

#### Stability

Are there "rough" functions for which these differencing formulas estimate $u'(x_i) = 0$?

```python
x = numpy.linspace(-1, 1, 9)
xfine = numpy.linspace(-1, 1, 200)
def f_rough(x):
    return numpy.cos(.5+4*numpy.pi*x)
def fp_rough(x):
    return -4*numpy.pi * numpy.sin(.5+4*numpy.pi*x)

pyplot.plot(x, f_rough(x), 'o-')
pyplot.plot(xfine, f_rough(xfine), '-');
```

```python
for diff in (diff1r, diff1l, diff1c):
    xx, yy = diff(x, f_rough(x))
    pyplot.plot(xx, yy, 'o', label=diff.__name__)
pyplot.plot(xfine, fp_rough(xfine), label='exact')
pyplot.legend(loc='upper right');
```

We have this function $f_{\text{rough}}(x)$ that is rough at the grid scale and has derivative identically zero according to the centered difference formula.  Except perhaps for boundary conditions (which we'll consider later), given any solution $u(x)$ to a differential equation discretized using the centered difference approximation, $$\tilde u(x) = u(x) + f_{\text{rough}}(x)$$ would also be a solution.  This non-uniqueness is a disaster for numerical algorithms and in most cases, the centered difference formula is not used directly to compute first derivatives.

When assessing a discretization we require **both accuracy and stability**.

### Second derivatives

We will need at least three grid points to compute a second derivative.
* Why?

Again, there is more than one possible approach.
* One method is to use a first derivative formula to compute $u'(x_{i+1})$ and $u'(x_{i-1})$, then apply the centered first derivative again.
* Another is to define some "staggered" points $x_{i+1/2}$ and $x_{i-1/2}$.

Why should we choose one over the other? Can we understand this in terms of accuracy and stability?

```python
def diff2a(x, u):
    xx, yy = diff1c(x, u)
    return diff1c(xx, yy)

def diff2b(x, u):
    xx, yy = diff1l(x, u)
    return diff1r(xx, yy)

x = numpy.linspace(-1, 1, 11)
u = numpy.cos(x)
for diff2 in (diff2a, diff2b):
    xx, yy = diff2(x, u)
    pyplot.plot(xx, yy, 'o', label=diff2.__name__)
pyplot.plot(x, -numpy.cos(x), label='exact')
pyplot.legend(loc='upper right');
```

```python
def grid_refinement_error2(f, fpp, diff):
    error = []
    for n in grids:
        x = numpy.linspace(-1, 1, n)
        xx, yy = diff(x, f(x))
        error.append(numpy.linalg.norm(yy - fpp(xx), numpy.inf))
    return grids, error

for diff2 in (diff2a, diff2b):
    ns, error = grid_refinement_error2(numpy.cos, lambda x: -numpy.cos(x), diff2)
    pyplot.loglog(ns, error, 'o', label=diff.__name__)
pyplot.loglog(grids, (grids-1)**(-1.), label='$h$')
pyplot.loglog(grids, (grids-1)**(-2.), label='$h^2$')
pyplot.loglog(grids, .25*(grids-1)**(-2.), label='$h^2/4$')
pyplot.xlabel('Resolution $n$')
pyplot.ylabel('Second derivative error')
pyplot.legend(loc='upper right');
```

#### Observations

* Both methods are second order accurate
* The `diff2b` method is more accurate than `diff2a` by a factor of 4
* The `diff2a` method cannot compute the derivative at points next to the boundary
* We don't know yet whether either method is stable

## Differentiation as matrices

So far we have written functions of the form `diff(x, u)` that compute derivatives.  These functions happen to have been linear in `u`.  We should be able to write differentiation as a matrix $D$ such that

$$ u'(x) = D u(x) $$

where $x$ is the vector of $n$ discrete points, thus $u(x)$ is also a vector of length $n$.

### Homework 1: Due 2018-09-16 (Sunday)

1. Fork the class repository, clone, and create a directory `hw1` inside the repository.  Add your source file(s) to that directory.
2. Write a function `diffmat(x)` that returns a square matrix $D$ that computes first derivatives at all points.
3. Write a function `diff2mat(x)` that returns a square matrix $D_2$ that computes second derivatives at all points.
4. Use test solutions to determine the order of accuracy of your methods for evenly and non-evenly spaced points.  Which norm did you use?
5. Add `README.md` in the `hw1` directory and summarize your results (one paragraph or a few bullet items is fine).
6. Commit (`git commit`) your source code and `README.md` in the `hw1` directory and push to your fork.  I'll pull from there after the due date.

* You may assume that the points `x` are monotonically increasing.
* You'll need to think about what to do at the endpoints.

## Boundary conditions

Our boundary value problem states a differential equation to be satisfied in the interior of the domain, combined with Dirichlet conditions at the left endpoint and Neumann at the right endpoint.

### Dirichlet
The left endpoint in our example BVP has a Dirichlet boundary condition,
$$u(-1) = a . $$
With finite difference methods, we have an explicit degree of freedom $u_0 = u(x_0 = -1)$ at that endpoint.
When building a matrix system for the BVP, we can implement this boundary condition by modifying the first row of the matrix,
$$ \begin{bmatrix} 1 & 0 & 0 & 0 & 0 \\ \\ & & A_{1:,:} & & \\ \\ \end{bmatrix} \begin{bmatrix} u_0 \\ \\ u_{1:} \\ \\ \end{bmatrix} = \begin{bmatrix} a \\ \\ f_{1:} \\ \\ \end{bmatrix} . $$

* This matrix is not symmetric even if $A$ is.
* We can eliminate $u_0$ and create a reduced system for $u_{1:}$.  We will describe this for a more general $2\times 2$ block system of the form
$$ \begin{bmatrix} I & 0 \\ A_{10} & A_{11} \end{bmatrix} \begin{bmatrix} u_0 \\ u_1 \end{bmatrix} = \begin{bmatrix} f_0 \\ f_1 \end{bmatrix} .$$
We can rearrange as
$$ A_{11} u_1 = f_1 - A_{10} f_0 $$
which is symmetric if $A_{11}$ is.  This is sometimes called "lifting" and is often done implicitly in the mathematics literature.  It is convenient for linear solvers and eigenvalue solvers, but inconvenient for IO and postprocessing, as well as some nonlinear problems.  For this reason, it may be preferable to write
$$ \begin{bmatrix} I & 0 \\ 0 & A_{11} \end{bmatrix} \begin{bmatrix} u_0 \\ u_1 \end{bmatrix} = \begin{bmatrix} f_0 \\ f_1 - A_{10} f_0 \end{bmatrix} $$
which is symmetric and entirely decouples the degrees of freedom associated with the boundary.  This method turns out to be relatively elegant for nonlinear solvers.

* It is sometimes useful to scale the identity by some scalar related to the norm of $A_{11}$.

## Neumann
The right endpoint in our example BVP has a Neumann boundary condition,
$$ \frac{du}{dx}(1) = b . $$
With finite difference methods, there are generally two ways to derive an equation for the value $u_n = u(x_n = 1)$.  The first is to use a one-sided difference formula as in
$$ \frac{u_n - u_{n-1}}{h} = b . $$
This requires an extra discretization choice and it may not be of the same order of accuracy as the interior discretization and may destroy symmetry.

An alternative is to temporarily introduce a ghost value $u_{n+1} = u(x_{n+1} = 1 + h)$ (possibly more) and define it to be a reflection of the values from inside the domain.  In the case $b=0$, this reflection is $u_{n+i} = u_{n-i}$.  More generally, we can bias the reflection by the slope $b$,
$$ u_{n+i} = u_{n-i} + 2b(x_n - x_{n-i}) . $$
After this definition of ghost values, we apply the interior discretization at the boundary.  For our reference equation, we would write

$$ \frac{-u_{n-1} + 2 u_n - u_{n+1}}{h^2} = f(x_n) $$

which simplifies to $$ \frac{u_n - u_{n-1}}{h^2} = f(x_n)/2 + b/h $$
after dividing by 2 and moving the boundary term to the right hand side.

```python
def laplacian(n, rhsfunc, a, b):
    x = numpy.linspace(-1, 1, n+1)
    h = 2 / n
    rhs = rhsfunc(x)
    e = numpy.ones(n)
    L = (2 * numpy.eye(n+1) - numpy.diag(e, 1) - numpy.diag(e, -1)) / h**2
    # Dirichlet condition
    L[0,:] = numpy.eye(1,n+1)
    rhs[0] = a
    rhs[1:] -= L[1:,0] * a
    L[1:,0] = 0
    # Neumann condition
    L[-1,-1] /= 2
    rhs[-1] = b / h + rhs[-1] / 2
    return x, L, rhs

laplacian(5, lambda x: 0*x+1, .5, -1)
```

    (array([-1. , -0.6, -0.2,  0.2,  0.6,  1. ]),
     array([[ 1.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ],
            [ 0.  , 12.5 , -6.25,  0.  ,  0.  ,  0.  ],
            [ 0.  , -6.25, 12.5 , -6.25,  0.  ,  0.  ],
            [ 0.  ,  0.  , -6.25, 12.5 , -6.25,  0.  ],
            [ 0.  ,  0.  ,  0.  , -6.25, 12.5 , -6.25],
            [ 0.  ,  0.  ,  0.  ,  0.  , -6.25,  6.25]]),
     array([ 0.5  ,  4.125,  1.   ,  1.   ,  1.   , -2.   ]))

```python
x, L, rhs = laplacian(160, lambda x: 0*x-1, 1, 0)
u = numpy.linalg.solve(L, rhs)
pyplot.plot(x, u);
pyplot.xlabel('x')
pyplot.ylabel('u');
```

#### Observations

* The reflection formulation maintains symmetry and order of accuracy.
* We'll need to be careful in case of nonlinearity, especially when bounds or positivity are in play. The reflected values could produce negative pressure or density, for example.

## Green's functions

A continuous [Green's function](https://en.wikipedia.org/wiki/Green%27s_function) is the solution of a differential equation where the right hand side is a [Dirac delta function](https://en.wikipedia.org/wiki/Dirac_delta_function).  We can compute discrete Green's functions by solving against columns of the identity.

```python
Linv = numpy.linalg.inv(L)
cols = [5, 8, 13]
pyplot.plot(x, Linv[:,cols])
pyplot.xlabel('x')
pyplot.ylabel('Solution u')
pyplot.legend(['$L^{-1} e_{%d}$' % c for c in cols]);
```

## Spectra and eigenfunctions

An eigenvalue $\lambda$ of the matrix $L$ satisfies
$$ L u = \lambda u $$
and $u$ is called an eigenvector.  The set of all eigenvalues $\lambda$ is called the spectrum of $L$.

```python
x, L, _ = laplacian(160, lambda x: 0*x, 1, 0)
lam, U = numpy.linalg.eigh(L)

pyplot.plot(lam, 'o')
pyplot.xlabel('Eigenvalue number')
pyplot.ylabel('Eigenvalue');
```

```python
x, L, rhs = laplacian(40, lambda x: 0*x+1, 1, -.5)
lam, U = numpy.linalg.eigh(L)
for i in (0, 1, 2, 3):
    pyplot.plot(x, U[:,i], label='$\lambda_{%d} = %5f$'%(i, lam[i]))
pyplot.legend(loc='upper right');
```

### Note on stability

Above, we see evidence of stability in that all the eigenvalues are positive and the eigenvectors associated with the smallest eigenvalues are low frequency (ignoring the mode corresponding to the Dirichlet boundary).  Specifically, there are no "noisy" functions with eigenvalues close to zero.

```python
for i in (-3, -2, -1):
    pyplot.plot(x, U[:,i], label='$\lambda_{%d} = %5f$'%(i, lam[i]))
pyplot.legend(loc='upper right');
```

## Manufactured solutions

Let's choose a smooth function with rich derivatives,
$$ u(x) = \tanh(x) . $$
Then $$ u'(x) = \cosh^{-2}(x) $$ and $$ u''(x) = -2 \tanh(x) \cosh^{-2}(x) . $$

```python
def uexact(x):
    return numpy.tanh(3*x)
def duexact(x):
    return 3*numpy.cosh(3*x)**(-2)
def negdduexact(x):
    return 3**2 * 2 * numpy.tanh(3*x) * numpy.cosh(3*x)**(-2)

x, L, f = laplacian(20, negdduexact, uexact(-1), duexact(1))
u = numpy.linalg.solve(L, f)
pyplot.plot(x, uexact(x), label='exact')
pyplot.plot(x, u, 'o', label='computed')
pyplot.legend(loc='upper left');
```

```python
def mms_error(n, discretize):
    x, L, f = discretize(n, negdduexact, uexact(-1), duexact(1))
    u = numpy.linalg.solve(L, f)
    return numpy.linalg.norm(u - uexact(x), numpy.inf)

ns = 2**numpy.arange(3,8)
errors = [mms_error(n, laplacian) for n in ns]
pyplot.loglog(2/ns, errors, 'o', label='numerical')
pyplot.loglog(2/ns, (2/ns), label='$h$')
pyplot.loglog(2/ns, (2/ns)**2, label='$h^2$')
pyplot.legend(loc='upper left');
```

## The other second order centered discretization

We have been using the discretization

$$ -u''(x_i) \approx \frac{-u_{i-1} + 2 u_i - u_{i+1}}{h^2} $$

which is sometimes summarized using the stencil

$$ h^{-2} \begin{bmatrix} -1 & 2 & -1 \end{bmatrix} . $$

There is also the discretization arising from applying centered first derivatives twice,

$$ -u''(x_i) \approx \frac{-u_{i-2} + 2 u_i - u{i+2}}{4 h^2} $$

which corresponds to the stencil

$$ (2h)^{-2} \begin{bmatrix} -1 & 0 & 2 & 0 & -1 \end{bmatrix} . $$

We'll reuse the boundary condition handling and keep the first stencil at the points adjacent to the boundaries, $x_1$ and $x_{n-1}$, but apply this discretization for true interior points.

```python
def laplacian2(n, rhsfunc, a, b):
    x, L, rhs = laplacian(n, rhsfunc, a, b)
    h = 2 / n
    L[2:-2,:] = (2 * numpy.eye(n-3, n+1, k=2)
                 - numpy.eye(n-3, n+1, k=0)
                 - numpy.eye(n-3, n+1, k=4)) / (2*h)**2
    return x, L, rhs

laplacian2(6, numpy.cos, 1, 0)
```

    (array([-1.        , -0.66666667, -0.33333333,  0.        ,  0.33333333,
             0.66666667,  1.        ]),
     array([[ 1.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ],
            [ 0.  , 18.  , -9.  ,  0.  ,  0.  ,  0.  ,  0.  ],
            [-2.25,  0.  ,  4.5 ,  0.  , -2.25,  0.  ,  0.  ],
            [ 0.  , -2.25,  0.  ,  4.5 ,  0.  , -2.25,  0.  ],
            [ 0.  ,  0.  , -2.25,  0.  ,  4.5 ,  0.  , -2.25],
            [ 0.  ,  0.  ,  0.  ,  0.  , -9.  , 18.  , -9.  ],
            [ 0.  ,  0.  ,  0.  ,  0.  ,  0.  , -9.  ,  9.  ]]),
     array([1.        , 9.78588726, 0.94495695, 1.        , 0.94495695,
            0.78588726, 0.27015115]))

This matrix is not symmetric due to the discretization used near the boundaries.

```python
x, L, f = laplacian2(20, negdduexact, uexact(-1), duexact(1))
u = numpy.linalg.solve(L, f)
pyplot.plot(x, uexact(x), label='exact')
pyplot.plot(x, u, 'o', label='computed')
pyplot.legend(loc='upper left');
```

```python
errors = [mms_error(n, laplacian) for n in ns]
errors2 = [mms_error(n, laplacian2) for n in ns]
pyplot.loglog(2/ns, errors, 'o', label='numerical')
pyplot.loglog(2/ns, errors2, 's', label='numerical2')
pyplot.loglog(2/ns, (2/ns), label='$h$')
pyplot.loglog(2/ns, (2/ns)**2, label='$h^2$')
pyplot.legend(loc='upper left');
```

```python
n = 40
x, L, _ = laplacian(n, lambda x: 0*x+1, 1, -1)
_, L2, _ = laplacian2(n, lambda x: 0*x+1, 1, -1)

lam, U = numpy.linalg.eigh(L)
lam2, U2 = numpy.linalg.eig(L2)  # Need to use nonsymmetric eigensolver
idx = numpy.argsort(lam2)
lam2 = lam2[idx]
U2 = U2[:,idx]
pyplot.plot(lam, 'o', label='original')
pyplot.plot(lam2, 'o', label='version 2')
pyplot.legend(loc='lower right');
```

```python
pyplot.figure()
for i in (0, 1, 2, 3):
    pyplot.plot(x, U[:,i], label='$\lambda_{%d} = %5f$'%(i, lam[i]))
pyplot.legend(loc='upper right')
pyplot.title('Original')

for i in (0, 1, 2, 3):
    pyplot.plot(x, U2[:,i], label='$\lambda_{%d} = %5f$'%(i, lam2[i]))
pyplot.legend(loc='upper right')
pyplot.title('Version 2');
```

### Observations about `laplacian2`

* `laplacian2` converges at the same order of accuracy as `laplacian`
* `laplacian2` is less accurate, especially on coarse/under-resolved grids
* `laplacian2` (with this boundary condition implementation) is nonsingular
* `laplacian2` is a weakly unstable discretization, even away from boundaries

# Fourier analysis of FD methods

Suppose we are operating on an infinite domain with uniform grid spacing $h=1$.  Some finite difference stencils that we have worked with are

```python
stencil_1L = [-1, 1, 0]
stencil_1C = [-1/2, 0, 1/2]
stencil_1R = [0, -1, 1]
stencil_2a = [1, -2, 1]
stencil_2b = [1/4, 0, -1/2, 0, 1/4]
```

We consider the action of these stencils on the grid functions
$$ \phi(x, \theta) = e^{i \theta x} . $$
For example, take `stencil_2a`
\begin{align}
S \phi(x, \theta) &= \phi(x-1, \theta) - 2 \phi(x, \theta) + \phi(x+1, \theta) \\
&= \underbrace{(2 \cos\theta - 2)}_{\hat S(\theta)} \phi(x, \theta) .
\end{align}

* Show this

Evidently $\phi(x, \theta)$ is an eigenvector of $S$ for every value of $\theta$ and $\hat S(\theta)$ is the corresponding eigenvalue.  We call the function $\hat S(\theta)$ the *symbol* of the stencil $S$.

* We need only consider $-\pi < \theta \le \pi$.  Why?

The exact derivatives of $\phi$ are
\begin{align}
\phi'(x, \theta) &= i \theta \phi(x, \theta) \\
\phi''(x, \theta) &= -\theta^2 \phi(x, \theta) .
\end{align}

* What is the Taylor series for the symbol $2 - 2\cos\theta$?

```python
def symbol(S, theta):
    """Compute the symbol of the stencil S(theta)"""
    if len(S) % 2 != 1:
        raise RuntimeError('Stencil length must be odd')
    w = len(S) // 2 # integer division rounds down
    phi = numpy.exp(1j * numpy.outer(range(-w, w+1), theta))
    return S @ phi

symbol(stencil_2a, [0, numpy.pi/4])
```

    array([ 0.        +0.j, -0.58578644+0.j])

```python
def plot_symbol(S, deriv=2):
    theta = numpy.linspace(-numpy.pi, numpy.pi)
    sym = symbol(S, theta)
    rsym = numpy.real_if_close((-1j)**deriv * sym)
    pyplot.plot(theta, rsym, '.', label='stencil')
    pyplot.plot(theta, theta**deriv, '-', label='exact')
    pyplot.legend()
    pyplot.title('Symbol of S={}'.format(S))
    pyplot.xlabel(r'$\theta$')
    pyplot.ylabel('approximate and $\\theta^{%d}$' % deriv)
    return theta, rsym

theta, _ = plot_symbol(stencil_2a)
pyplot.plot(theta, 2 - 2*numpy.cos(theta));
```

```python
plot_symbol(stencil_2b);
```

```python
plot_symbol(stencil_1C, deriv=1);
```

### Observations

* Plotting the symbol tells us about stability (eigenvalues near 0 for high frequency).
* The order of agreement with the exact symbol is the order of the method.
* We can't use it for variable coefficient or boundary conditions.
* We'll use this technique again to optimize solvers like multigrid.


# Conjugate gradient method

From https://en.wikipedia.org/wiki/Conjugate_gradient_method:

In mathematics, the conjugate gradient method is an algorithm for the numerical solution of particular systems of linear equations, namely those whose matrix is symmetric and positive-definite.

#### Description
Let $A\mathbf x = \mathbf b$ be a square system of $n$ linear equations.

#### The Algorithm:

$\begin{align}
& \mathbf{r}_0 := \mathbf{b} - \mathbf{A x}_0 \\
& \hbox{if } \mathbf{r}_{0} \text{ is sufficiently small, then return } \mathbf{x}_{0} \text{ as the result}\\
& \mathbf{p}_0 := \mathbf{r}_0 \\
& k := 0 \\
& \text{repeat} \\
& \qquad \alpha_k := \frac{\mathbf{r}_k^\mathsf{T} \mathbf{r}_k}{\mathbf{p}_k^\mathsf{T} \mathbf{A p}_k}  \\
& \qquad \mathbf{x}_{k+1} := \mathbf{x}_k + \alpha_k \mathbf{p}_k \\
& \qquad \mathbf{r}_{k+1} := \mathbf{r}_k - \alpha_k \mathbf{A p}_k \\
& \qquad \hbox{if } \mathbf{r}_{k+1} \text{ is sufficiently small, then exit loop} \\
& \qquad \beta_k := \frac{\mathbf{r}_{k+1}^\mathsf{T} \mathbf{r}_{k+1}}{\mathbf{r}_k^\mathsf{T} \mathbf{r}_k} \\
& \qquad \mathbf{p}_{k+1} := \mathbf{r}_{k+1} + \beta_k \mathbf{p}_k \\
& \qquad k := k + 1 \\
& \text{end repeat} \\
& \text{return } \mathbf{x}_{k+1} \text{ as the result}
\end{align}$

#### Stop Condition:
$$ \lVert X^{(k+1)} - X^{(k)} \rVert_2 \le 10^{-4}$$

```python
import numpy as np
```

```python
def conjgrad(A, b, initial_guess):
    x = initial_guess
    #Symbol of matrix multiplication in numpy is @
    r = b - A@x
    p = r

    while(1):
        Ap = A@p
        alpha = (r.T@r)/(p.T@Ap)
        alpha = np.asscalar(alpha)
        x_old = x
        x = x + alpha*p
        x_new = x
        #using norm2:
        if np.linalg.norm(x_new-x_old) < 10**(-4):
            break
        r_old = r
        r = r - alpha*Ap
        beta = (r.T@r)/(r_old.T@r_old)
        beta = np.asscalar(beta)
        p = r + beta*p
    return x

```

```python
A = np.matrix([[2.0,5.0],
               [5.0,7.0]])

b = np.matrix([[11.0],[13.0]])

initialGuess = np.matrix([[1.0],[1.0]])

sol = conjgrad(A,b,initialGuess)

print ('A:')
print(A)

print ('\nb:')
print(b)

print('\nSolution:')
print(sol)
```

    A:
    [[2. 5.]
     [5. 7.]]

    b:
    [[11.]
     [13.]]

    Solution:
    [[-1.09090909]
     [ 2.63636364]]

```python

```


# Circuits for constructing entangled states

```python
import numpy as np

from qiskit import QuantumRegister, ClassicalRegister
from qiskit import QuantumCircuit
from qiskit import execute, BasicAer

import qiskit.tools.visualization as qvis

import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning)
```

In the lecture I showed you the entangled state
\begin{eqnarray}
 |\Psi_\rangle &=& \frac{1}{\sqrt{2}} \left( |00\rangle + |11 \rangle \right).
\end{eqnarray}

But this isn't the only two-qubit entangled states. In fact, this state is just one of the four included in the so-called 'Bell basis', a set of 4 orthonormal 2-qubit entangled states. The Bell basis is especially important in the quantum teleporation protocol found in the other notebooks.

Your job here is to write small quantum circuits to construct the four states in the Bell basis
\begin{eqnarray}
|\Psi_{0}\rangle &=& \frac{1}{\sqrt{2}} \left( |00\rangle + |11 \rangle \right) \\
|\Psi_{1}\rangle &=& \frac{1}{\sqrt{2}} \left( |01\rangle + |10 \rangle \right) \\
|\Psi_{2}\rangle &=& \frac{1}{\sqrt{2}} \left( |00\rangle - |11 \rangle \right) \\
|\Psi_{3}\rangle &=& \frac{1}{\sqrt{2}} \left( |01\rangle - |10 \rangle \right) \\
\end{eqnarray}
Below is some boilerplate code for creating the circuits and outputting the state vector so you can be sure you got the right one.

```python
q = QuantumRegister(2, name='q')
bell_circuit = QuantumCircuit(q)

# YOUR CODE GOES HERE
# Construct the Bell states here by applying Qiskit gates
# You can copy the code into 4 different cells or just modify in here
# to get the different states
bell_circuit.h(q[0])
bell_circuit.cx(q[0], q[1])

# Draw the circuit
print(bell_circuit.draw())

# Execute the quantum circuit and print the state vector
backend = BasicAer.get_backend('statevector_simulator')
result = execute(bell_circuit, backend).result()
print("The state vector in the computational basis is:")
print(result.get_statevector(bell_circuit).reshape(4, 1))
```

## Higher-dimensional entangled states

Entanglement is not limited to two qubits, but can become much more complicated as there are many different 'separability structures' that a multi-qubit entangled state can take. For example, a 3-qubit state might be a tensor product of 3 single qubit states, a tensor product of a two-qubit entangled state with a single-qubit state, or even a purely entangled 3-qubit state (i.e. 'genuine multipartite entanglement').

One such state is the GHZ state,
\begin{equation}
 |GHZ\rangle = \frac{1}{\sqrt{2}} \left( |000\rangle + |111\rangle \right)
\end{equation}

```python
# YOUR CODE HERE
# Create a circuit and construct the GHZ state.
# You can copy the boilerplate code from the 2-qubit case but will of course have to
# alter the register size and the gate sequence
```


# Game instructions
Consider the following board game: A game board has 12 spaces. The swine senses the Christmas spirit and manages to run away from home couple of weeks beforehand. Fortunately for it, the butcher is a bit of a drunkard and easily distracted. The swine starts on space 7, and a butcher on space 1. On each game turn a 6-sided die is rolled. On a result of 1 to 3, the swine moves that many spaces forward. On a result of 5 or 6, the butcher moves that many spaces forward. On result 4, both advance one space forward. The swine wins if it reaches the river at space 12 (the final roll does not have to be exact, moving past space 12 is OK). The butcher wins if he catches up with the swine (or moves past it).

What are the probabilities of winning for the swine and the butcher?

Your assignment is to create a mathematical or statistical model to find these probabilities, and implement the solution as a computer program in whatever language you like. You will present it during the interview and we will discuss it with you.

Consider the following questions as well:
- Can you make your model easily extendable for different initial conditions (board size and initial positions)?
- Pros and cons of the approach?
- Can you say something about how long the game takes (also under different initial conditions)?

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import itertools
```

# Model: dynamic programming

#### Initialize game parameters

```python
board_size = 12
swine_start = 7
butcher_start = 1

if butcher_start >= swine_start:
    raise ValueError('Error in starting positions: The swine has to start ahead of the butcher.')
elif swine_start >= board_size:
    raise ValueError('Error in starting positions: The river has to lie ahead of the swine.')
```

#### DP formulation
Assume a fixed board size. Let $s$ be the position of the swine on the board, $b$ the position of the butcher on the board, and $F(s, b)$ the probability that the swine wins the game if the swine is at space $s$ and the butcher at space $b$.

The recurrence relation can be formulated as follows:

\begin{align}
    F(s, b \mid s \geq \text{board size}) = & 1 & \text{Swine has won with 100% probability if end of board is reached} \\
    F(s, b \mid s \leq b) = & 0 & \text{Swine wins with 0% probability if the butcher is ahead or at the same square} \\
    F(s, b) =  & 1/6 \cdot F(s+1, b) + & \text{DP recurrence relation} \\
               & 1/6 \cdot F(s+2, b) + \\
               & 1/6 \cdot F(s+3, b) + \\
               & 1/6 \cdot F(s+1, b+1) + \\
               & 1/6 \cdot F(s, b+5) + \\
               & 1/6 \cdot F(s, b+6)
\end{align}

The equations express that the swine's probability of winning is a weighted combination of its winning chances in all possible subsequent states.

#### Prefill winning probabilities known at start

```python
swine_positions = np.arange(swine_start, board_size+1)
butcher_positions = np.arange(butcher_start, board_size+1)
probs = pd.DataFrame(np.nan*np.ones(shape=(len(swine_positions), len(butcher_positions))), index=swine_positions, columns=butcher_positions)

# Swine has won with 100% probability if end of board is reached.
probs.loc[board_size, :] = 1.0

# Swine wins with 0% probability if the butcher is ahead or at the same square.
# I.e. upper right triangle should be zeros.
mask = np.ones(probs.shape, dtype='bool')
triu = np.triu_indices(n=probs.shape[0], m=probs.shape[0])
mask[tuple([triu[0], triu[1] + probs.shape[1] - probs.shape[0]])] = False
probs.where(mask, other=0.0, inplace=True)
```

```python
probs
```

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12

      7
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      8
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      0.0
      0.0
      0.0
      0.0
      0.0

      9
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      0.0
      0.0
      0.0
      0.0

      10
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      0.0
      0.0
      0.0

      11
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      0.0
      0.0

      12
      1.0
      1.0
      1.0
      1.0
      1.0
      1.0
      1.0
      1.0
      1.0
      1.0
      1.0
      0.0

#### Solve DP

```python
def F(probs, sp, bp):
    """Calculate the swine's probability of winning based on the current swine and butcher position.

    Arguments
        - probs: dataframe of swine's winning chances (known and unknown)
        - sp:    swine's current position
        - bp:    butcher's current position
    """

    # Check that the current positions are not lower than the starting positions.
    if sp < probs.index.min():
        sp = probs.index.min()
        print('Swine position lower than starting position: using swine starting position ({}) instead.'.format(swine_start))
    if bp < probs.columns.min():
        bp = probs.columns.min()
        print('Butcher position lower than starting position: using butcher starting position ({}) instead.'.format(butcher_start))

    # Check that neither the swine nor the butcher has already reached the end of the board.
    # If so, reset position to the last space on the board.
    if sp > probs.index.max():
#         print('Swine position exceeds board length: using highest possible position instead.')
        sp = probs.index.max()
    if bp > probs.columns.max():
#         print('Butcher position exceeds board length: using highest possible position instead.')
        bp = probs.columns.max()

    # If the requested probability is already known: return from storage.
    if not np.isnan(probs.loc[sp, bp]):
        return probs.loc[sp, bp]

    # Else: calculate the requested probability according to the DP recurrence, store and return.
    else:
        prob = 1/6 * F(probs, sp+1, bp) \
               + 1/6 * F(probs, sp+2, bp) \
               + 1/6 * F(probs, sp+3, bp) \
               + 1/6 * F(probs, sp+1, bp+1) \
               + 1/6 * F(probs, sp, bp+5) \
               + 1/6 * F(probs, sp, bp+6)

        probs.loc[sp, bp] = prob

        return prob
```

```python
swine_winning_prob = F(probs, swine_start, butcher_start)
print('The swine and the butcher start at space {} and {} respectively. The board is {} spaces long.'.format(swine_start, butcher_start, board_size))
print('The probability that the swine wins is {:.1f}%.'.format(swine_winning_prob*100))
```

    The swine and the butcher start at space 7 and 1 respectively. The board is 12 spaces long.
    The probability that the swine wins is 51.2%.


```python
# Importing the necessary packages
from sympy import *
import matplotlib.pyplot as plt
```

```python
r_1,r_2,n = symbols('r_1,r_2,n')       # Defining new symbols. You can use LaTeX here

M = Matrix([[1-r_1,r_2],[r_1,1-r_2]])  # Defining the transition matrix

display(M)                             # Display the answer (it's like printing, for symbolic objects)
```

$\displaystyle \left[\begin{matrix}1 - r_{1} & r_{2}\\r_{1} & 1 - r_{2}\end{matrix}\right]$

```python
# Numerical values of parameters ############

rONE=0.3
rTWO=0.2

#############################################

M = M.subs(r_1,rONE).subs(r_2,rTWO)         # Substitute the values of r_1 and r_2 in the matrix

psi_0 = Matrix([9,1])                       # Initial state

psi_n = M**n*psi_0                          # Arbitrary state psi_n

# Determining psi_\inf ######################

psi_n_inf = []                              # Start off with an empty vector, psi_n_inf

for el in psi_n:                            # For each element in psi_n
    psi_n_inf.append(Limit(el,n,oo).doit()) # take the limit n -> \infty, and add it to psi_n_inf

psi_n_inf = Matrix(psi_n_inf)               # Make psi_n_inf into a matrix object

# Calculating psi_n at different n ##########

temp = psi_0                                # Starting state

S_list = []                                 # Lists for S, I,and t
I_list = []
t_list = []

S_list.append(9)                            # Initialising them
I_list.append(1)
t_list.append(0)

for i in range(0,100,3):                    # For every time-step
    temp = M*temp                           # Find M^i temp
    S_list.append(temp[0])                  # Append the values of the lists
    I_list.append(temp[1])
    t_list.append(i)

# Graph the results #########################
plt.xlabel('Time(n)')
plt.ylabel('Number')
plt.plot(t_list,S_list,'o-',label='Susceptible')
plt.plot(t_list,I_list,'o-',label='Infected')
plt.legend()
plt.show()
#############################################
```

```python
def SIS(r_1,r_2,init_S,init_I):

    M = Matrix([[1-r_1,r_2],[r_1,1-r_2]])       # Defining the transition matrix

    psi_0 = Matrix([init_S,init_I])             # Initial state

    psi_n = M**n*psi_0                          # Arbitrary state psi_n

    # Determining psi_\inf ######################

    psi_n_inf = []                              # Start off with an empty vector, psi_n_inf

    for el in psi_n:                            # For each element in psi_n
        psi_n_inf.append(Limit(el,n,oo).doit()) # take the limit n -> \infty, and add it to psi_n_inf

    psi_n_inf = Matrix(psi_n_inf)               # Make psi_n_inf into a matrix object

    # Calculating psi_n at different n ##########

    temp = psi_0                                # Starting state

    S_list = []                                 # Lists for S, I,and t
    I_list = []
    t_list = []

    S_list.append(init_S)                       # Initialising them
    I_list.append(init_I)
    t_list.append(0)

    for i in range(1,20,1):                     # For every time-step
        temp = M*temp                           # Find M^i temp
        S_list.append(temp[0])                  # Append the values of the list
        I_list.append(temp[1])
        t_list.append(i)

    # Graph the results #########################
    plt.xlabel('Time(n)')
    plt.ylabel('Number')
    plt.plot(t_list,S_list,'o-')
    plt.plot(t_list,I_list,'o-')
    plt.show()
    #############################################
```

```python
SIS(0.3,0.2,55,45)
SIS(0.3,0.2,70,30)
```

```python

```


# *Sobol'* sensitivity indices

In this example we are going to quantify the correlation between the input variables and the output variable of a model thanks to the *Sobol'* indices.

The *Sobol'* indices allow to evaluate the importance of a single variable or a specific set of variables.

*Sobol'* indices range is $\left[0; 1\right]$ ; the more the indice value is close to 1 the more the variable is important toward the output variance of the function. The *Sobol'* indices can be computed at different orders.

The first order indices evaluate the importance of one variable at a time ($d$ indices, with $d$ the input dimension of the model). The $d$ total indices give the relative importance of every variables except the variable $x_i$, for every variable.

Here the *Sobol'* indices are estimated on an analytical function: *Ishigami*. It writes

$$ F(\mathbf{x}) = \sin(x_1)+7\sin(x_2)^2+0.1x_3^4\sin(x_1), \quad \mathbf{x}\in [-\pi, \pi]^3 $$
Analytical values of *Sobol'* indices for this function are available:

\begin{align}
    S_{1, 2, 3} &= [0.314, 0.442, 0.], \\
    S_{T_{1, 2, 3}} &= [0.558, 0.442, 0.244].
\end{align}

This function is interesting because it exhibits an interaction between $x_1$ and $x_3$ although $x_3$ by itself do not play a role at first order.

## The Ishigami function

```python
import openturns as ot
import numpy as np
```

```python
# Create the model and input distribution
formula = ['sin(X1)+7*sin(X2)^2+0.1*X3^4*sin(X1)']
input_names = ['X1', 'X2', 'X3']
dimension = 3
corners = [[-np.pi] * dimension, [np.pi] * dimension]
model = ot.SymbolicFunction(input_names, formula)
distribution = ot.ComposedDistribution([ot.Uniform(corners[0][i], corners[1][i])
                                        for i in range(dimension)])
true_sobol = [[0.314, 0.442, 0.], [0.558, 0.442, 0.244]]  # [first, total]
```

## Without Surrogate

```python
# Create X/Y data
ot.RandomGenerator.SetSeed(0)
size = 10000
sample = ot.SobolIndicesExperiment(distribution, size, True).generate()
output = model(sample)
```

```python
# Compute Sobol' indices using the Saltelli estimator
## first order indices
sensitivityAnalysis = ot.SaltelliSensitivityAlgorithm(sample, output, size)
first_order = sensitivityAnalysis.getFirstOrderIndices()
print(f"First order Sobol' indices: {first_order}\n"
      f"Relative error: {abs(np.array(first_order) - true_sobol[0]) * 100}%")  # maximum is 1

## total order indices
total_order = sensitivityAnalysis.getTotalOrderIndices()
print(f"Total order Sobol' indices: {total_order}\n"
      f"Relative error: {abs(np.array(total_order) - true_sobol[1]) * 100}%")  # maximum is 1
```

    First order Sobol' indices: [0.302751,0.460825,0.00669407]
    Relative error: [1.12489872 1.88252502 0.66940742]%
    Total order Sobol' indices: [0.57499,0.427147,0.256687]
    Relative error: [1.69897217 1.48529071 1.26868602]%

## With Surrogate

```python
# Create a Gaussian process surrogate model
from otsurrogate import SurrogateModel
surrogate = SurrogateModel('kriging', corners, input_names)
```

```python
# Generation of data to fit the surrogate model
sequence_type = ot.LowDiscrepancyExperiment(ot.SobolSequence(), distribution, 256)
learning_sample = sequence_type.generate()
learning_output = model(learning_sample)
```

```python
surrogate.fit(learning_sample, learning_output)
```

```python
# Compute sensitivity indices
## first order indices
output, _ = surrogate(sample)  # Do not return the information about the variance
sensitivityAnalysis = ot.SaltelliSensitivityAlgorithm(sample, output, size)
first_order = sensitivityAnalysis.getFirstOrderIndices()
print(f"First order Sobol' indices: {first_order}\n"
      f"Relative error: {abs(np.array(first_order) - true_sobol[0]) * 100}%")  # maximum is 1

## total order indices
total_order = sensitivityAnalysis.getTotalOrderIndices()
print(f"Total order Sobol' indices: {total_order}\n"
      f"Relative error: {abs(np.array(total_order) - true_sobol[1]) * 100}%")  # maximum is 1
```

    First order Sobol' indices: [0.312765,0.472573,0.0070339]
    Relative error: [0.1235497  3.05730132 0.70338973]%
    Total order Sobol' indices: [0.562337,0.447125,0.237793]
    Relative error: [0.43372069 0.51250457 0.62068279]%

## Using otsklearn

```python
surrogate = SurrogateModel('otsklearn.Kriging()', corners, input_names)
surrogate.fit(learning_sample, learning_output)
```

```python
# Compute sensitivity indices
## first order indices
#output = surrogate.evaluate(sample)
output, _ = surrogate(sample)  # Do not return the information about the variance
sensitivityAnalysis = ot.SaltelliSensitivityAlgorithm(sample, output, size)
first_order = sensitivityAnalysis.getFirstOrderIndices()
print(f"First order Sobol' indices: {first_order}\n"
      f"Relative error: {abs(np.array(first_order) - true_sobol[0]) * 100}%")  # maximum is 1

## total order indices
total_order = sensitivityAnalysis.getTotalOrderIndices()
print(f"Total order Sobol' indices: {total_order}\n"
      f"Relative error: {abs(np.array(total_order) - true_sobol[1]) * 100}%")  # maximum is 1
```

    First order Sobol' indices: [0.303545,0.459592,0.00691174]
    Relative error: [1.04549598 1.75916887 0.69117411]%
    Total order Sobol' indices: [0.577597,0.426664,0.257912]
    Relative error: [1.95973662 1.53359806 1.3912165 ]%

```python

```


## [MathJax](http://www.mathjax.org)

Times: $$28 \times 28$$

Sum and subscripts: $$\begin{eqnarray}\sigma\left(b + \sum_{l=0}^4 \sum_{m=0}^4  w_{l,m} a_{j+l, k+m} \right).\tag{125}\end{eqnarray}$$

supperscripts: $$a^1 = \sigma(b + w * a^0)$$

$$\eta = 0.1 \dot{F}$$

$$f(z)\equiv \max(0, z)$$

$$\lambda = 0.1$$

$$c^{L-1}$$

$$x = {-b \pm \sqrt{b^2-4ac} \over 2a}.$$

$$f(a) = \frac{1}{2\pi i} \oint\frac{f(z)}{z-a}dz$$

$$\cos(θ+φ)=\cos(θ)\cos(φ)−\sin(θ)\sin(φ)$$

$$\int_D ({\nabla\cdot} F)dV=\int_{\partial D} F\cdot ndS$$

$$\vec{\nabla} \times \vec{F} = \left( \frac{\partial F_z}{\partial y} - \frac{\partial F_y}{\partial z} \right) \mathbf{i}
        + \left( \frac{\partial F_x}{\partial z} - \frac{\partial F_z}{\partial x} \right) \mathbf{j} + \left( \frac{\partial
        F_y}{\partial x} - \frac{\partial F_x}{\partial y} \right) \mathbf{k}$$

$$\sigma = \sqrt{ \frac{1}{N} \sum_{i=1}^N (x_i -\mu)^2}$$

$$(\nabla_X Y)^k = X^i (\nabla_i Y)^k = X^i \left( \frac{\partial Y^k}{\partial x^i} + \Gamma_{im}^k Y^m \right)$$

$$
\left[
\begin{array}{lcr}
a1 & b22 & c333 \\
d444 & e555555 & f6
\end{array}
\right]
$$

$$
\left\{
\begin{aligned}
&a+b=c\\
&d=e+f+g
\end{aligned}
\right.
$$

$$
\begin{align}
a+b&=c\\
d&=e+f+g
\end{align}
$$

$$\frac{\partial}{\partial a}(a+b) = \frac{\partial a}{\partial a} + \frac{\partial b}{\partial a} = 1$$

$$\frac{\partial}{\partial u}uv = u\frac{\partial v}{\partial u} + v\frac{\partial u}{\partial u} = v$$

$$\frac{\partial Z}{\partial X} = (\alpha + \beta + \gamma)(\delta + \epsilon + \zeta)$$

```python

```


# "Jupyter notebooks for remote science"
> "Using Jupyter notebooks for remote science"
- toc: true
- badges: false
- comments: false

Computational notebooks have massively gained in popularity in the past years. They allow to share rich documents with text, code, mathematical formulas, images, videos etc. both in static and interactive ways. They are heavily used both in research to document projects and in teaching to provide content support. Notebooks exist in different flavors, but currently the most popular form is the Jupyter (**Ju**lia, **Py**thon, **R**) notebook. Here, we briefly describe  1. [What type of content they can be used for](#1.-What-is-a-notebook), 2. [Solutions to write/run notebooks online (no installation required)](#2.-Using-Jupyter-notebooks), 3. [How to share the documents, interactively or not and](#3.-Sharing-documents) 4. [A few examples of courses given as notebooks](#4.-Examples-of-courses).

## 1. What is a notebook

Notebooks are interactive documents with rich content that run directly in the internet browser. They are composed of series of blocks, called cells that can be executed individually. Notebooks are typically used when computer coding is involved but they provide a rich environment to create interactive documentation in general. So even if you don't need the coding part, you can still use notebooks to create documents and use the many existing resources to share them.

Notebooks are composed of mainly two types of cells: text cells and code cells.

### Text cells

#### Markdown
The content of text cells is formatted using the Markdown language, a very simple markup language allowing to create formatted text. For example *italic* and **bold** are generated by surrounding text with one ```*italic*```or two stars ```**bold**```. Titles and subtitles are generated by preceding text by a series of hash signs ```# My Title```, ```## My subtitle```.

#### Images
Images can be embedded using either a simple markdown syntax `````` or directly html language which can be customized (e.g. for image size) `````` where ```myimage``` is a path or a weblink to an image (e.g. https://img.myswitzerland.com/671846/407):

#### Equations
Fortunately, Jupyter also allows to embed $\LaTeX$ text. There a multiple solutions for this: surround your equation $\vec F = m*\vec{a}$ with dollar signs ```$\vec F = m*\vec{a}$```, use double dollar ```$$\vec F = m*\vec{a}$$``` to create a block:
$$\vec F = m*\vec{a}$$
or in a separate cell use the standard ```\begin{equation} ... \end{equation}```:

\begin{equation}
\vec F = m*\vec{a}
\end{equation}

### Code cells

#### Computational code
The most common type of code cell, is a cell with actual computational code demonstrating some algorithm or function. It has an input and if some output is generated also an output cell:

```python
def my_fun(x):
    y = x**2 + 3*x + 2
    return y

my_fun(5)
```

    42

Not only numerical calculations are possible but also symbolic ones, e.g. using sympy:

```python
import sympy
import warnings
warnings.filterwarnings("ignore")
from ipywidgets import interact, FloatSlider
import numpy as np
import matplotlib.pyplot as plt

t, k, c, y_0, ydot_0 = sympy.symbols('t k c y_0 ydot_0')
y = sympy.Function('y')
damped_oscillator = sympy.Eq(y(t).diff(t, t) + c* y(t).diff(t) + k*y(t), 0)
fun_solved = sympy.dsolve(damped_oscillator, y(t), ics={y(0):y_0, y(t).diff(t,1).subs(t, 0): ydot_0})
```

```python
damped_oscillator
```

$\displaystyle c \frac{d}{d t} y{\left(t \right)} + k y{\left(t \right)} + \frac{d^{2}}{d t^{2}} y{\left(t \right)} = 0$

```python
fun_solved
```

$\displaystyle y{\left(t \right)} = \frac{\left(\frac{y_{0} \left(c + \sqrt{c^{2} - 4 k}\right)}{2} + \dot{y}_{0}\right) e^{\frac{t \left(- c + \sqrt{c^{2} - 4 k}\right)}{2}}}{\sqrt{c^{2} - 4 k}} + \frac{\left(- \frac{c y_{0}}{2} + \frac{y_{0} \sqrt{c^{2} - 4 k}}{2} - \dot{y}_{0}\right) e^{\frac{t \left(- c - \sqrt{c^{2} - 4 k}\right)}{2}}}{\sqrt{c^{2} - 4 k}}$

#### Output code
Code cells can however also be used to display interactive content, in particular the output of an analysis e.g. with Matplotlib:

```python
plt.plot(np.arange(-10,11),my_fun(np.arange(-10,11)));
```

Results can also be shown as interactive plots which the user can play with to get an intuition:

```python
def myfun(var):
    solved = fun_solved.subs({y_0:1, ydot_0:0, k:var, c : 1})
    sympy.plot(solved.rhs,ylim = (-1,1),xlim = (0,10));
interact(myfun, var = FloatSlider(min=1, max = 10,step = .5));
```

    interactive(children=(FloatSlider(value=1.0, description='var', max=10.0, min=1.0, step=0.5), Output()), _dom_…

#### Multimedia output

In addition to these standard ways of showing code, Jupyter can also be used to display other types of information. For example a movie uploaded from Youtube:

```python
from IPython.display import YouTubeVideo
YouTubeVideo("6FZ5k9_6Vlw",560,rel=0)
```

## 2. Using Jupyter notebooks

Of course Jupyter can be installed on any computer, the simplest route being to use Anaconda or Miniconda as a base for installation. Even though this works flawlessly in a large majority of cases, it is often desirable to avoid any installation procedure, especially in the current period where in-person meetings should be avoided. As Jupyter notebooks have become very popular and as they are entirely rendered in the browser, there are many solutions to **run** the notebook on a remote computer and simply display it in the browser. The different solutions vary in the resources available, price etc.

### Renku from the Swiss Data Science Center

The SDSC provides for all academic institutions free access to a Jupyter environment on a platform called [Renku](https://renkulab.io/). It allows to create entire projects including code and data and provides a full fledged access to a Jupyter environment entirely identical to one running on a laptop. One can login with a switch.edu account (your regular @xxx.unibe.ch account) or a Github account and create both private and public projects which are in fact repositories on Gitlab. This service is free.

### Google colab

Google has developed it's own flavor of Jupyter called [Colab](https://colab.research.google.com/). It works exactly in the same way as a regular notebook, only the interface looks slightly different. Here notebooks are saved on the GoogleDrive of the account you are using to connect to Colab. One of the main advantages of this solution is the access to GPUs. This service is free and you can purchase a subscription to enjoy longer sessions and more computing power.

### Mybinder

For those already familiar with Python, Github and Jupyter, an interesting solution is the [mybinder](https://mybinder.org/) service. This allows you to turn any repository on e.g. on GitHub into an interactive Jupyter session. For that, you only have to include in your repository a file specifying the necessary packages to run your code. One negative side is that sessions are temporary, and any changes are lost on session end. This service is free.

### Private JupyterHub

If you need e.g. a highly customized Jupyter environment or the possibility to share large amounts of data you might want to create your own Jupyter service called JupyterHub. Users of such a service are simply provided a link and again connect to the service via their browser, getting access to a standard Jupyter session. ScITS has experience in setting up such services for example using [SWITCHengines](https://www.switch.ch/fr/engines/) computing resources.

## 3. Sharing documents

### File exchange
Jupyter notebooks are simple text files. So you can simply share them as any other file type. For example you can upload all of them on ilias. This means however that to open them, people have to locally install Jupyter. Again in the perspective of avoiding local installations, it is easier to give access to them on some type of server.

### Code repositories and notebook renderer
The most common solution for this is to use a code repository such as GitHub or Gitlab. If you are worried about publicly releasing your work, you can leave repositories private and share them with your students. This however requires all of them to have e.g. a GitHub account. Notebooks can be visualized as static objects by browsing through the repository. Note that even though notebooks are rendered as static objects, the links and videos will still work.
A more efficient way of displaying notebooks is however to use the [nbviewer](https://nbviewer.jupyter.org/) service, which efficiently renders any notebook. Notebooks created on Renku (see above) can be visualized on Gitlab directly.

### Interactive sharing
As mentioned above, if you want to go beyond rendering static notebooks, you can use mybinder or Google Colab to provide a quick access to interactive versions of your notebooks that collaborators and students can test and modify. You can embed directly in the notebooks badges to further simplify this procedure. Here for Colab:
[](https://colab.research.google.com/github/guiwitz/RemoteJupyter/blob/master/Jupyter_in_remote_times_colab.ipynb)
and here for mybinder:
[](https://github.com/guiwitz/RemoteJupyter/blob/master)

### Create a course website with GitHub pages
Imagine now that you have a series of notebooks for a course and you would like to assemble them into a simple website. If you are not familiar with web technologies, it can be a daunting task to create such a simple website. [GitHub pages](https://pages.github.com/) is a service integrated in GitHub which takes a lot of the pain away from that process i.e. no database, no domain to setup, no HTML to write etc. However it still involves setting up the appropriate environment to create content. This is where [Fastpages](https://github.com/fastai/fastpages) a solution built on top of GitHub pages comes to the rescue: it completely automates the process of creating and publishing the web page. The only requirement is to be minimally familiar with GitHub (forks, pull requests etc.). Faspages offers a very detailed step by step guide on how to proceed and turns your collection of notebooks into a blog. See for example [this notebook](https://guiwitz.github.io/blog/2020/03/23/Jupyter-in-remote-times.html) on my blog.

## 4. Examples of courses

We finish here by giving a few examples of courses of different nature.

### Computational courses

At ScITS we heavily use notebooks to teach programming in various areas, e.g. image processing. For that course, a [Github repository](https://github.com/guiwitz/PyImageCourse) contains all the course material. One can visualize it directly in the repository or using [nbviewer](https://nbviewer.jupyter.org/github/guiwitz/PyImageCourse/tree/master/). It can even be run interactively using the [mybinder service](https://mybinder.org/v2/gh/guiwitz/PyImageCourse/master).

### Scientific courses

Virtually all disciplines can benefit from being presented at least partially in interactive forms. To see for example how notebooks are used to teach chemistry, you can have a look at [this specific notebook](https://nbviewer.jupyter.org/github/jckantor/CBE20255/blob/master/notebooks/02.01-Balancing-Reactions.ipynb#Stoichiometry) from [this course](http://jckantor.github.io/CBE20255/). It contains videos, text, equations, symbolic calculations and is directly executable on Google Colab.

Here's another [great example](https://nbviewer.jupyter.org/github/engineersCode/EngComp3_tourdynamics/blob/master/notebooks_en/3_Get_Oscillations.ipynb) of a course on mechanics and numerical approximations. Again it uses all the resources available in notebooks. Note that if you visit the Github page, you will see this sort of badge [](https://mybinder.org/v2/gh/engineersCode/EngComp3_tourdynamics/master) telling you, you can run the material in a mybinder session.

```python

```


# For today's code challenge you will be reviewing yesterdays lecture material. Have fun!

### if you get done early check out [these videos](https://www.3blue1brown.com/neural-networks).

# The Perceptron

The first and simplest kind of neural network that we could talk about is the perceptron. A perceptron is just a single node or neuron of a neural network with nothing else. It can take any number of inputs and spit out an output. What a neuron does is it takes each of the input values, multplies each of them by a weight, sums all of these products up, and then passes the sum through what is called an "activation function" the result of which is the final value.

I really like figure 2.1 found in this [pdf](http://www.uta.fi/sis/tie/neuro/index/Neurocomputing2.pdf) even though it doesn't have bias term represented there.

If we were to write what is happening in some verbose mathematical notation, it might look something like this:

\begin{align}
 y = sigmoid(\sum(weight_{1}input_{1} + weight_{2}input_{2} + weight_{3}input_{3}) + bias)
\end{align}

Understanding what happens with a single neuron is important because this is the same pattern that will take place for all of our networks.

When imagining a neural network I like to think about the arrows as representing the weights, like a wire that has a certain amount of resistance and only lets a certain amount of current through. And I like to think about the node itselef as containing the prescribed activation function that neuron will use to decide how much signal to pass onto the next layer.

# Activation Functions (transfer functions)

In Neural Networks, each node has an activation function. Each node in a given layer typically has the same activation function. These activation functions are the biggest piece of neural networks that have been inspired by actual biology. The activation function decides whether a cell "fires" or not. Sometimes it is said that the cell is "activated" or not. In Artificial Neural Networks activation functions decide how much signal to pass onto the next layer. This is why they are sometimes referred to as transfer functions because they determine how much signal is transferred to the next layer.

## Common Activation Functions:

# Implementing a Perceptron from scratch in Python

### Establish training data

```python
import numpy as np

np.random.seed(812)

inputs = np.array([
    [0, 0, 1],
    [1, 1, 1],
    [1, 0, 1],
    [0, 1, 1]
])

correct_outputs = [[0], [1], [1], [0]]
```

### Sigmoid activation function and its derivative for updating weights

```python
def sigmoid(x):
  return 1 / (1 + np.exp(-x))

def sigmoid_derivative(x):
  sx = sigmoid(x)
  return sx * (1 - sx)
```

## Updating weights with derivative of sigmoid function:

### Initialize random weights for our three inputs

```python
weights = 2 * np.random.random((3, 1)) - 1
weights
```

    array([[-0.80108081],
           [ 0.5104585 ],
           [ 0.59516067]])

### Calculate weighted sum of inputs and weights

```python
weighted_sum = np.dot(inputs, weights)
weighted_sum
```

    array([[ 0.59516067],
           [ 0.30453837],
           [-0.20592014],
           [ 1.10561918]])

### Output the activated value for the end of 1 training epoch

```python
activated_output = sigmoid(weighted_sum)
activated_output
```

    array([[0.64454837],
           [0.57555158],
           [0.44870111],
           [0.75131149]])

### take difference of output and true values to calculate error

```python
error = correct_outputs - activated_output
error
```

    array([[-0.64454837],
           [ 0.42444842],
           [ 0.55129889],
           [-0.75131149]])

```python
adjustments = error * sigmoid_derivative(activated_output)
adjustments
```

    array([[-0.14549539],
           [ 0.09778778],
           [ 0.13111388],
           [-0.16363003]])

```python
weights += np.dot(inputs.T, adjustments)
weights
```

    array([[-0.57217915],
           [ 0.44461626],
           [ 0.51493691]])

### Put it all together

```python
for iteration in range(10000):

  # Weighted sum of inputs/weights
  weighted_sum = np.dot(inputs, weights)

  # Activate!
  activated_output = sigmoid(weighted_sum)

  # Calculate error
  error = correct_outputs - activated_output

  # Adjustments
  adjustments = error * sigmoid_derivative(activated_output)

  # Update weights
  weights += np.dot(inputs.T, adjustments)

print("Weights after training")
print(weights)

print("Output after training")
print(activated_output)
```

    Weights after training
    [[15.03729668]
     [-0.40682518]
     [-7.23231091]]
    Output after training
    [[7.22398796e-04]
     [9.99387935e-01]
     [9.99592428e-01]
     [4.81060734e-04]]


# Systems of Equations
Imagine you are at a casino, and you have a mixture of £10 and £25 chips. You know that you have a total of 16 chips, and you also know that the total value of chips you have is £250. Is this enough information to determine how many of each denomination of chip you have?

Well, we can express each of the facts that we have as an equation. The first equation deals with the total number of chips - we know that this is 16, and that it is the number of £10 chips (which we'll call ***x*** ) added to the number of £25 chips (***y***).

The second equation deals with the total value of the chips (£250), and we know that this is made up of ***x*** chips worth £10 and ***y*** chips worth £25.

Here are the equations

\begin{equation}x + y = 16 \end{equation}
\begin{equation}10x + 25y = 250 \end{equation}

Taken together, these equations form a *system of equations* that will enable us to determine how many of each chip denomination we have.

## Graphing Lines to Find the Intersection Point
One approach is to determine all possible values for x and y in each equation and plot them.

A collection of 16 chips could be made up of 16 £10 chips and no £25 chips, no £10 chips and 16 £25 chips, or any combination between these.

Similarly, a total of £250 could be made up of 25 £10 chips and no £25 chips, no £10 chips and 10 £25 chips, or a combination in between.

Let's plot each of these ranges of values as lines on a graph:

```python
%matplotlib inline
from matplotlib import pyplot as plt

# Get the extremes for number of chips
chipsAll10s = [16, 0]
chipsAll25s = [0, 16]

# Get the extremes for values
valueAll10s = [25,0]
valueAll25s = [0,10]

# Plot the lines
plt.plot(chipsAll10s,chipsAll25s, color='blue')
plt.plot(valueAll10s, valueAll25s, color="orange")
plt.xlabel('x (£10 chips)')
plt.ylabel('y (£25 chips)')
plt.grid()

plt.show()
```

Looking at the graph, you can see that there is only a single combination of £10 and £25 chips that is on both the line for all possible combinations of 16 chips and the line for all possible combinations of £250. The point where the line intersects is (10, 6); or put another way, there are ten £10 chips and six £25 chips.

### Solving a System of Equations with Elimination
You can also solve a system of equations mathematically. Let's take a look at our two equations:

\begin{equation}x + y = 16 \end{equation}
\begin{equation}10x + 25y = 250 \end{equation}

We can combine these equations to eliminate one of the variable terms and solve the resulting equation to find the value of one of the variables. Let's start by combining the equations and eliminating the x term.

We can combine the equations by adding them together, but first, we need to manipulate one of the equations so that adding them will eliminate the x term. The first equation includes the term ***x***, and the second includes the term ***10x***, so if we multiply the first equation by -10, the two x terms will cancel each other out. So here are the equations with the first one multiplied by -10:

\begin{equation}-10(x + y) = -10(16) \end{equation}
\begin{equation}10x + 25y = 250 \end{equation}

After we apply the multiplication to all of the terms in the first equation, the system of equations look like this:

\begin{equation}-10x + -10y = -160 \end{equation}
\begin{equation}10x + 25y = 250 \end{equation}

Now we can combine the equations by adding them. The ***-10x*** and ***10x*** cancel one another, leaving us with a single equation like this:

\begin{equation}15y = 90 \end{equation}

We can isolate ***y*** by dividing both sides by 15:

\begin{equation}y = \frac{90}{15} \end{equation}

So now we have a value for ***y***:

\begin{equation}y = 6 \end{equation}

So how does that help us? Well, now we have a value for ***y*** that satisfies both equations. We can simply use it in either of the equations to determine the value of ***x***. Let's use the first one:

\begin{equation}x + 6 = 16 \end{equation}

When we work through this equation, we get a value for ***x***:

\begin{equation}x = 10 \end{equation}

So now we've calculated values for ***x*** and ***y***, and we find, just as we did with the graphical intersection method, that there are ten £10 chips and six £25 chips.

You can run the following Python code to verify that the equations are both true with an ***x*** value of 10 and a ***y*** value of 6.

```python
x = 10
y = 6
print ((x + y == 16) & ((10*x) + (25*y) == 250))
```

    True

```python

```

Basics Of Algebra by Hiren/01-03-Systems of Equations.ipynb


# Sympy Solver

## CHE 341

This notebook introduces SymPy, the Python package for symbolic mathematics. First, let's import everything and set up a nice solve function:

```python
import sympy as sm
sm.init_printing()
import numpy as np
from copy import copy
import matplotlib.pyplot as plt
from sympy.abc import *

def solve(equation, variable, subs=None, unwrap=True):
    """Solve equation for the given variable; if given, a dictionary of subs
    (substitutions) can be given. This is useful if you want to solve numerically
    rather than symbolically.

    Parameters:
    equation : the sympy equation to solve
    variable : the sympy variable to solve for
    subs : the dictionary of substitutions
    unwrap : if there is only one solution, return it directly rather than returning a list.

    Returns:
    The solution (if one solution and unwrap=True), or a list of all possible solutions.

    Examples: Jupyter notebooks will give prettier printing of the output
    >>> solve(a*x**2 + b*x + c, x)
        [(-b + sqrt(-4*a*c + b**2))/(2*a), -(b + sqrt(-4*a*c + b**2))/(2*a)]

    """
    if subs is not None:
        subs = copy(subs)
        subs.pop(variable.name, None)
        out = sm.solve(equation.subs(subs), variable)
    else:
        out = sm.solve(equation, variable)
    if unwrap and len(out) == 1:
        out = out[0]
    return out
```

All capital and lowercase single letter variables (`a, b, c, A, B, C`) and Greek letters (`alpha, beta, gamma`) are defined as Sympy variables by default because of the line `from sympy.abc import *`.

For example, below we solve the quadratic equation for $x$:

```python
solns = solve(a*x**2 + b*x+c, x)
solns
```

Here's another example, solving the Boltzmann equation for $T$. In order to use the normal variables, we define $n_i$, $n_j$, and $k_B$ first:

```python
n_i, n_j, k_B = sm.symbols("n_i n_j k_B", positive=True) # Define the 3 symbols with subscripts in the Boltzmann equation

boltzmann_eqn = n_j/n_i-sm.exp(-epsilon/(k_B*T)) # We use sympy's exponential function, sm.exp
solve(boltzmann_eqn, T)
```

One final example - let's do an integral?

```python
sm.sequence(sm.exp(-epsilon*i/(k_B*T)), (i, 0, 5))
```

```python
epsilon, T = sm.symbols('epsilon T', positive=True)
```

```python

```

```python
sm.Sum(x**i, (i, 0, sm.oo)).doit()
```

```python
subs=dict(
P = 1.0,
R = 0.08206,
T = 298,
n = 1,
V=22.4)
equation = P*V-n*R*T
solve(equation, V, subs)
```

```python
n_i, n_j, k_B = sm.symbols("n_i n_j k_B")

# The sympy solver!
subs=dict(
n_i=0.5,
n_j=1,
epsilon=1e-21,
k_B=1.381e-23,
T=298)
boltzmann = n_i/n_j-sm.exp(-epsilon/(k_B*T))
boltzmann

```

```python
solve(boltzmann, n_i, subs)
```

```python
# Let's
solve(boltzmann, T)`
```

```python
solve(boltzmann, epsilon)
```

```python
rotational_energy_eqn = i*(i+1) * h**2/(8 * pi**2) * (1/(mu*R**2)) - epsilon
rotational_energy_eqn
```

```python
solve(rotational_energy_eqn, R)[1]
```

```python
!pip install pint
```

```python
import pint
u = pint.UnitRegistry()
```

```python
(u.h * 1.0).to('J s')
```

```python
def reduced_mass(m1, m2):
    return m1 * m2 / (m1 + m2)
```

```python
mu_CO = reduced_mass(12, 16)*u.amu
```

```python
mu_CO
```

```python
def boltzmann_ni(nj, energy, T):
    return nj*np.exp(-energy/ (T*1.381e-23))
```

```python
temps = np.linspace(100, 300, 5)
temps
```

```python
boltzmann_ni(1, 1e-21, temps)
```

What if I need to do this for many values of temperature? Make a new function!

```python
def boltzmann_n_i_temp(T, subs):
    subs = copy(subs)
    # The sympy solver!
    subs['T'] = T
    boltzmann = n_i/n_j-sm.exp(-epsilon/(k_B*T))
    return solve(boltzmann, n_i, subs)
```

```python
temps = np.linspace(10, 300,30)
pops = [boltzmann_n_i_temp(t, subs) for t in temps]
```

```python
plt.plot(temps, pops, 'o')
```

```python
# subs is a dictionary of substitutions - variables and their values...
subs=dict(
P = , # atm
R = 0.08206, # L atm/mol-K
T = 293, # K
n = 9.32, # mol
V= # L
)
gas_law = P*V-n*R*T
solve(gas_law, V, subs) #
```

n_i_func(1, 1e-21, 1.381e-23, 343)


# Lecture 6: Monty Hall, Simpson's Paradox

## The Monty Hall Problem

You know this problem.

* There are three doors.
* A car is behind one of the doors.
* The other two doors have goats behind them.
* You choose a door, but before you see what's behind your choice, Monty opens one of the other doors to reveal a goat.
* Monty offers you the chance to switch doors.

_Should you switch?_

### Defining the problem

Let $S$ be the event of winning when you switch.

Let $D_j$ be the event of the car being behind door $j$.

### Solving with a probability tree

With a probability tree, it is easy to represent the case where you condition on Monty opening door 2. Given that you initially choose door 1, you can quickly see that if you stick with door 1, you have a $\frac{1}{3}~$ chance of winning.

You have a $\frac{2}{3}~$ chance of winning if you switch.

### Solving with the Law of Total Probability

This is even easier to solve using the Law of Total Probability.

\begin{align}
    P(S) &= P(S|D_1)P(D_1) + P(S|D_2)P(D_2) + P(S|D_3)P(D_3) \\
    &= 0 \frac{1}{3} + 1 \frac{1}{3} + 1 \frac{1}{3}  \\
    &= \frac{2}{3}
\end{align}

### A more general solution

Let $n = 7$ be the number of doors in the game.

Let $m=3$ be the number of doors with goats that Monty opens after you select your initial door choice.

Let $S$ be the event where you win _by sticking with your original door choice of door 1_.

Let $C_j$ be the event that the car is actually behind door $j$.

Conditioning only on which door has the car, we have
\begin{align}
    & &P(S) &= P(S|C_1)P(C_1) + \dots + P(S|C_n)P(C_n) & &\text{Law of Total Probability} \\
    & &    &= P(C_1) \\
    & &    &= \frac{1}{7} \\
\end{align}

Let $M_{i,j,k}$ be the event that Monty opens doors $i,j,k$. Conditioning on Monty opening up doors $i,j,k$, we have

\begin{align}
    & &P(S) &= \sum_{i,j,k} P(S|M_{i,j,k})P(M_{i,j,k}) & &\text{summed over all i, j, k with } 2 \le i \lt j \lt k \le 7 \\
    \\
    & &\Rightarrow P(S|M_{i,j,k}) &= P(S) & &\text{by symmetry} \\
    & & &=\frac{1}{7}
\end{align}

Note that we can now generalize this  to the case where:
* there are $n \ge 3$ doors
* after you choose a door, Monty opens $m$ of the remaining doors $n-1$ doors to reveal a goat (with $1 \le m \le n-m-2$)

The probability of winning with the strategy of _sticking to your initial choice_ is $\frac{1}{n}$, whether __unconditional or conditioning on the doors Monty opens__.

After Monty opens $m$ doors, each of the remaining $n-m-1$ doors has __conditional__ probability of $\left(\frac{n-1}{n-m-1}\right) \left(\frac{1}{n}\right)$.

Since $\frac{1}{n} \lt \left(\frac{n-1}{n-m-1}\right) \left(\frac{1}{n}\right)$, you will always have a greater chance of winning if you switch.

## Simpson's Paradox

_Is it possible for a certain set of events to be more (or less) probable than another without conditioning, and then be less (or more) probable with conditioning?_

Assume that we have the above rates of success/failure for Drs. Hibbert and Nick for two types of surgery: heart surgery and band-aid removal.

### Defining the problem

Let $A$ be the event of a successful operation.

Let $B$ be the event of treatment by Dr. Nick.

Let $C$ be the event of heart surgery.

\begin{align}
    P(A|B,C) &< P(A|B^c,C) & &\text{Dr. Nick is not as skilled as Dr. Hibbert in heart surgery} \\
    P(A|B,C^c) &< P(A|B^c,C^c) & &\text{neither is he that good at band-aid removal} \\
\end{align}

And yet $P(A|B) > P(A|B^c)$?

### Explaining with the Law of Total Probability

To explain this paradox, let's try to use the Law of Total Probability.

\begin{align}
    P(A|B) &= P(A|B,C)P(C|B) + P(A|B,C^c)P(C^c|B) \\
    \\
    \text{but } P(A|B,C) &< P(A|B^c,C) \\
    \text{and } P(A|B,C^c) &< P(A|B^c,C^c)
\end{align}

Look at $P(C|B$ and $P(C|B^c)$. These weights are what makes this paradox possible, as they are what make the inequality relation sign flip.

Event $C$ is a case of __confounding__

### Another example

_Is it possible to have events $A_1, A_2, B, C$ such that_

\begin{align}
    P(A_1|B) &\gt P(A_1|C) \text{ and } P(A_2|B) \gt P(A_2|C) & &\text{ ... yet...} \\
    P(A_1 \cup A_2|B) &\lt P(A_1 \cup A_2|C)
\end{align}

Yes, and this is just another case of Simpson's Paradox.

Note that

\begin{align}
    P(A_1 \cup A_2|B) &= P(A_1|B) + P(A_2|B) - P(A_1 \cap A_2|B)
\end{align}

So this is _not_ possible if $A_1$ and $A_2$ are disjoint and $P(A_1 \cup A_2|B) = P(A_1|B) + P(A_2|B)$.

It is crucial, therefore, to consider the _intersection_ $P(A_1 \cap A_2|B)$, so let's looks at the following example where $P(A_1 \cap A_2|B) \gg P(A_1 \cap A_2|C)$ in order to offset the other inequalities.

Consider two basketball players each shooting a pair of free throws.

Let $A_j$ be the event basketball free throw scores on the $j^{th}$ try.

Player $B$ always either makes both $P(A_1 \cap A_2|B) = 0.8$, or misses both.

\begin{align}
    P(A_1|B) = P(A_2|B) = P(A_1 \cap A_2|B) = P(A_1 \cup A_2|B) = 0.8
\end{align}

Player $C$ makes free throw shots with probability $P(A_j|C) = 0.7$, independently, so we have

\begin{align}
    P(A_1|C) &= P(A_2|C) = 0.7  \\
    P(A_1 \cap A_2|C) &= P(A_1|C) P(A_2|C) = 0.49 \\
    P(A_1 \cup A_2|C) &= P(A_1|C) + P(A_2|C) - P(A_1 \cap A_2|C) \\
                      &= 2 \times 0.7 - 0.49 \\
                      &= 0.91
\end{align}

And so we have our case where

\begin{align}
    P(A_1|B) = 0.8 &\gt P(A_1|C) = 0.7 \\
    P(A_2|B) = 0.8 &\gt P(A_2|C) = 0.7 \\
    \\
    \text{ ... and  yet... } \\
    \\
    P(A_1 \cup A_2|B) &\lt P(A_1 \cup A_2|C) ~~~~ \blacksquare
\end{align}


# Nonlinear Equations

We want to find a root of the nonlinear function $f$ using different methods.

1. Bisection method
2. Newton method
3. Chord method
4. Secant method
5. Fixed point iterations

```python
%matplotlib inline
from numpy import *
from matplotlib.pyplot import *
import sympy as sym

```

```python
t = sym.symbols('t')

f_sym = t/8. * (63.*t**4 - 70.*t**2. +15.) # Legendre polynomial of order 5

f_prime_sym = sym.diff(f_sym,t)

f = sym.lambdify(t, f_sym, 'numpy')
f_prime = sym.lambdify(t,f_prime_sym, 'numpy')

phi = lambda x : 63./70.*x**3 + 15./(70.*x)
#phi = lambda x : 70.0/15.0*x**3 - 63.0/15.0*x**5
#phi = lambda x : sqrt((63.*x**4 + 15.0)/70.)

# Let's plot
n = 1025

x = linspace(-1,1,n)
c = zeros_like(x)

_ = plot(x,f(x))
_ = plot(x,c)
_ = grid()

```

```python
# Initial data for the variuos algorithms

# interval in which we seek the solution
a = 0.7
b = 1.

# initial points
x0 = (a+b)/2.0
x00 = b

```

```python
# stopping criteria
eps = 1e-10
n_max = 1000
```

## Bisection method

$$
x^k = \frac{a^k+b^k}{2}
$$
```
                   if (f(a_k) * f(x_k)) < 0:
                      b_k1 = x_k
                      a_k1 = a_k
                   else:
                      a_k1 = x_k
                      b_k1 = b_k
```

```python
def bisect(f,a,b,eps,n_max):
    assert f(a)*f(b)<0
    a_new = a
    b_new = b
    x = mean([a,b])
    err = eps + 1.
    errors = [err]
    it = 0
    while (err > eps and it < n_max):
        if ( f(a_new) * f(x) < 0 ):
            # root in (a_new,x)
            b_new = x
        else:
            # root in (x,b_new)
            a_new = x

        x_new = mean([a_new,b_new])

        #err = 0.5 *(b_new -a_new)
        err = abs(f(x_new))
        #err = abs(x-x_new)

        errors.append(err)
        x = x_new
        it += 1

    semilogy(errors)
    print(it)
    print(x)
    print(err)
    return errors

errors_bisect = bisect(f,a,b,eps,n_max)

```

```python
# is the number of iterations coherent with the theoretical estimation?
```

In order to find out other methods for solving non-linear equations, let's compute the Taylor's series of $f(x^k)$ up to the first order

$$
f(x^k) \simeq f(x^k) + (x-x^k)f^{\prime}(x^k)
$$
which suggests the following iterative scheme
$$
x^{k+1} = x^k - \frac{f(x^k)}{f^{\prime}(x^k)}
$$

The following methods are obtained applying the above scheme where

$$
f^{\prime}(x^k) \approx q^k
$$

## Newton's method
$$
q^k = f^{\prime}(x^k)
$$

$$
x^{k+1} = x^k - \frac{f(x^k)}{q^k}
$$

```python
def newton(f,f_prime,x0,eps,n_max):
    pass # TODO

%time errors_newton = newton(f,f_prime,1.0,eps,n_max)
```

## Chord method

$$
q^k \equiv q = \frac{f(b)-f(a)}{b-a}
$$

$$
x^{k+1} = x^k - \frac{f(x^k)}{q}
$$

```python
def chord(f,a,b,x0,eps,n_max):
    pass # TODO

errors_chord = chord (f,a,b,x0,eps,n_max)
```

## Secant method

$$
q^k = \frac{f(x^k)-f(x^{k-1})}{x^k - x^{k-1}}
$$

$$
x^{k+1} = x^k - \frac{f(x^k)}{q^k}
$$

Note that this algorithm requirs **two** initial points

```python
def secant(f,x0,x00,eps,n_max):
    pass # TODO

errors_secant = secant(f,x0,x00,eps,n_max)
```

## Fixed point iterations

$$
f(x)=0 \to x-\phi(x)=0
$$

$$
x^{k+1} = \phi(x^k)
$$

```python
def fixed_point(phi,x0,eps,n_max):
    pass # TODO

errors_fixed = fixed_point(phi,0.3,eps,n_max)

```

## Comparison

```python
# plot the error convergence for the methods
loglog(errors_bisect, label='bisect')
loglog(errors_chord, label='chord')
loglog(errors_secant, label='secant')
loglog(errors_newton, label ='newton')
loglog(errors_fixed, label ='fixed')
_ = legend()
```

```python
# Let's compare the scipy implmentation of Newton's method with our..
```

```python
import scipy.optimize as opt
%time opt.newton(f, 1.0, f_prime, tol = eps)
```


# Minimal distance between point and plane

Consider the case that $V = \mathbb R^3$ and $S$ is the plane spanned by the vectors

$$
b_0 = (-1,-1,0), \quad b_1 = (1,0,1).
$$

Let $u = (0,0,3)$ and let us find the closest point $s \in S$ to $u$.
There are $c_0, c_1 \in \mathbb R$ such that

$$
s = c_0 b_0 + c_1 b_1 = B c,
$$

where $B$ is the matrix with columns $b_0$ and $b_1$ and $c$ is the vector with entries $c_0$ and $c_1$. Perhaps the most natural way to find the closest point is to solve the least squares problem to minimize $|Bc - u|^2$.

```python
import numpy as np
from scipy import linalg as la
```

```python
u = np.array([0, 0, 3], dtype=float)
b0 = np.array([-1, -1, 0], dtype=float)
b1 = np.array([1, 0, 1], dtype=float)

B = np.stack((b0, b1), axis=-1)
c, _, _, _ = la.lstsq(B, u)
s = B @ c
c
```

```python
from matplotlib import pyplot as plt

import plot_vec3d as p3d
def prettify():
    p3d.plot_plane(s, c[0] * b0)
    p3d.plot_vec(u - s, 'k--', o=s)
    p3d.plot_coeffs(c, b0, b1, 'k:')
    p3d.set_equal_aspect()
    p3d.hide_ticks()
```

```python
plt.figure(dpi=120)
plt.axes(projection='3d')
p3d.plot_vec(u, 'b')
p3d.plot_vec(s, 'g')
p3d.plot_vec(b0, 'r')
p3d.plot_vec(b1, 'r')
prettify()
```

The finite element method leads to problems that are similar to the above distance minimization, but

* $V$ will then be an infinite dimensional vector space
* The corresponding least squares problem can not be solved directly

However,

* $S$ will still be finite dimensional
* We will be able to form a finite dimensional system of linear equations

As a predule, let's demonstrate this idea in the context of the above distance minimization.

By the _orthogonality implies minimality_ lemma, the closest point $s$ is characterized by

\begin{align}\tag{1}
(s,v) = (u, v) \quad \text{for all $v \in S$}.
\end{align}

In particular, equation (1) needs to hold for $v=b_0$ and $v=b_1$.
On the other hand, if (1) holds for $v=b_0$ and $v=b_1$ then it holds for all $v \in S$ by taking linear combinations. Writing again $s = c_0 b_0 + c_1 b_1$, we get the following system of equations for vector $c$ with entries $c_0$ and $c_1$

$$
\sum_{j=0}^1 c_j (b_j, b_i) = (u, b_i), \quad i=0,1.
$$

We write $K$ for the matrix with entries $(b_i, b_j)$ and $F$ for the vector with entries $(u, b_i)$. Then (1) is equivalent with $Kc = F$.

```python
# Assemble matrix K
K = np.zeros((2,2))
for i in range(2):
    for j in range(2):
        K[i, j] = np.dot(B[:, i], B[:, j])

# Assemble vector F
F = np.zeros(2)
for i in range(2):
    F[i] = np.dot(u, B[:, i])

c_alt = la.solve(K, F)

# Compare to the solution using least squares
eps = np.finfo(float).eps # Machine epsilon
la.norm(c_alt - c) / la.norm(c) < 10*eps

```

# P1 finite element

## Definition: P1 basis functions on reference domain
>$$
\psi_0(x)
= \begin{cases}
1 - x & x \in [0,1]
\\
0 & \text{otherwise},
\end{cases}
\qquad
\psi_1(x)
= \begin{cases}
x & x \in [0,1]
\\
0 & \text{otherwise}.
\end{cases}
$$

Here the reference domain means the unit interval $[0,1]$.
Viewing $\psi_0$ and $\psi_1$ as functions on $[0,1]$, they form a basis of $\mathbb P_1$.
Indeed, the standard basis of $\mathbb P_1$ is obtained as

$$
1 = \psi_0(x) + \psi_1(x), \quad x = \psi_1(x).
$$

```python
def cutoff(x):
    return (x > 0) * (x < 1)

def psi0(x):
    return cutoff(x) * (1 - x)
def psi1(x):
    return cutoff(x) * x

xs_plot = np.linspace(-0.2,1.2, 200)
plt.plot(xs_plot, psi0(xs_plot), 'r', label='$\psi_0$')
plt.plot(xs_plot, psi1(xs_plot), 'b--', label='$\psi_1$')
plt.legend();
```

## Definition: local basis functions
>Let
>
>\begin{align*}
0 = x_0 < x_1 < \dots < x_n = 1,
\end{align*}
>
> write $I_e = [x_{e-1}, x_e]$, $e=1,\dots,n$, and consider the affine function
>
>\begin{align*}
\Phi_e : \mathbb R \to \mathbb R, \quad \Phi_e(x) = \frac{x - x_{e-1}}{x_e - x_{e-1}},
\end{align*}
>
> mapping $I_e$ to $[0,1]$. The local basis functions on $I_e$ are
>
>\begin{align*}
\psi_{j,e}(x) = \psi_j (\Phi_e (x)), \quad j = 0,1.
\end{align*}

The linear interpolant $\mathcal I_h u$ of a function $u \in C(I)$, with $I = [0,1]$, can be written on $I_e \subset I$ as

$$
\mathcal I_h u(x) = u(x_{e-1}) \psi_{0,e}(x) + u(x_{e}) \psi_{1,e}(x).
$$

```python
n = 4
xs = np.linspace(0,1, n+1)

def Phi(psi, e, x):
    return psi((x - xs[e-1]) / (xs[e] - xs[e-1]))

xs_plot = np.linspace(0,1, 200)
for e in range(1, n+1):
    plt.plot(xs_plot, Phi(psi0, e, xs_plot), label=f'$\psi_0^{e}$')
plt.gca().set_xticks(xs)
plt.legend();
```

```python
for e in range(1, n+1):
    plt.plot(xs_plot, Phi(psi1, e, xs_plot), label=f'$\psi_1^{e}$')
plt.gca().set_xticks(xs)
plt.legend();
```

```python
def u(x):
    return x * (1 - x)

def I_loc(u, x):
    out = np.zeros(np.shape(x))
    for e in range(1, n+1):
        out += u(xs[e-1])*Phi(psi0, e, x) + u(xs[e])*Phi(psi1, e, x)
    return out

plt.plot(xs_plot, u(xs_plot), label='u')
plt.plot(xs_plot, I_loc(u, xs_plot), label='$I_h u$')
plt.gca().set_xticks(xs)
plt.grid(axis='x')
plt.legend();
```

Recall that the basis functions  $\phi_i$ for the space $S = P_{h,0}^1$ were defined in the _P1 finite element space_ section of the lecture notes.
They can be written as

$$
\phi_i(x) = \psi_{1,i}(x) + \psi_{0,i+1}(x), \quad i = 1,\dots,n-1.
$$

For $u$ satisfying $u(0) = 0 = u(1)$ there holds

$$
\mathcal I_h u(x) = \sum_{i=1}^{n-1} u(x_i) \phi_i(x).
$$

```python
def phi(i, x):
    return Phi(psi1, i, x) + Phi(psi0, i+1, x)

for i in range(1, n):
    plt.plot(xs_plot, phi(i, xs_plot), label=f'$\phi_{i}$')
plt.gca().set_xticks(xs)
plt.grid(axis='x')
plt.legend();
```

```python
def I(u, x):
    out = np.zeros(np.shape(x))
    for i in range(1, n):
        out += u(xs[i]) * phi(i, x)
    return out

plt.plot(xs_plot, u(xs_plot), label='u')
plt.plot(xs_plot, I(u, xs_plot), label='$I_h u$')
plt.gca().set_xticks(xs)
plt.grid(axis='x')
plt.legend();
```

# Implementation of the P1 finite element method

Let us consider the bilinear and linear forms $a$ and $L$ defined in the _a boundary value problem_ section of the lecture notes, that is,

$$
a(u, v) = \int_0^1 u'(x) v'(x) dx, \quad L(v) = \int_0^1 f(x) v(x) dx.
$$

Following the proof the _Galerkin solution_ theorem in the lecture notes, we define

$$
u_S = \sum_{j=1}^{n-1} U_j \phi_j,
\quad
K_{ij} = a(\phi_i, \phi_j),
\quad
F_i = L(\phi_i),
\qquad i,j=1,\dots,n-1,
$$

and write $U$ and $F$ for the vectors with elements $U_i$ and $F_i$, and $K$ for the matrix with elements $K_{ij}$.

The equation for the Galerkin solution

$$
a(u_S,v) = L(v) \quad \text{for all $v \in S$}.
$$

is equivalent with $K U = F$.

Let us assemble the matrix $K$. As the global basis functions $\phi_j$ are defined piecewisely on intervals $I_e$, we will consider each interval separately. On $I_e$ there holds

$$
\phi_e(x) = \psi_1^e(x), \quad \phi_{e-1}(x) = \psi_0^e(x), \quad \phi_i(x) = 0, \quad i \ne e, e-1.
$$

Differentiating the local basis functions on $I_e$, we get

$$
\phi_e'(x) = \frac{1}{x_e - x_{e-1}}, \quad \phi_{e-1}'(x) =  \frac{-1}{x_e - x_{e-1}}, \quad \phi_i'(x) = 0, \quad i \ne e, e-1.
$$

In order to handle the first and last interval in the same way as the rest, we introduce the auxiliary basis functions

$$
\phi_0(x) = \psi_{0,1}(x), \quad \phi_n(x) = \psi_{1,n}(x).
$$

```python
plt.plot(xs_plot, Phi(psi0, 1, xs_plot), label=f'$\phi_0$')
plt.plot(xs_plot, Phi(psi1, n, xs_plot), label=f'$\phi_n$')
plt.gca().set_xticks(xs)
plt.grid(axis='x')
plt.legend();
```

```python
def assemble_K_demo(x):
    n = np.size(x) - 1
    K = np.zeros((n+1, n+1))
    for e in range(1, n+1):
        dx = x[e] - x[e-1]
        dphis = [-1 / dx, 1 / dx]
        ix = [e - 1, e]
        for i in range(2):
            for j in range(2):
                K[ix[i], ix[j]] += dphis[i] * dphis[j] * dx

    return K

K = assemble_K_demo(xs)
# Condense K, that is,
# throw away the values corresponding to the auxiliary functions
K = K[1:-1,1:-1]
print(K)
```

We turn to assembly of the vector $F$. We need to evaluate the integrals $(f, \phi_j)$ numerically. Let us use the trapezium rule separately on each interval $I_e$:

$$
\int_{I_e} f(x) \phi_e(x) dx \approx \frac{x_e - x_{e-1}}{2} f(x_e),
\quad
\int_{I_e} f(x) \phi_{e-1}(x) dx \approx \frac{x_e - x_{e-1}}{2} f(x_{e-1}),
$$

and

$$
\int_{I_e} f(x) \phi_{j}(x) dx = 0, \quad j\ne e, e-1.
$$

```python
def assemble_F_demo(x, f):
    n = np.size(x) - 1
    F = np.zeros(n+1)
    for e in range(1, n+1):
        dx = x[e] - x[e-1]
        F[e] += dx / 2 * f(x[e])
        F[e-1] += dx / 2 * f(x[e-1])
    return F

def f(x):
    return 1

F = assemble_F_demo(xs, f)
# Condense F
F = F[1:-1]
print(F)
```

## Example: Poisson's equation in 1d with unit load

Let us solve the problem

$$
\begin{cases}
-u'' = 1 & \text{on $(0,1)$},
\\
u(0) = 0 = u(1).
\end{cases}
$$

```python
def assemble_poisson_demo(xs):
    K = assemble_K_demo(xs)
    K = K[1:-1,1:-1]
    F = assemble_F_demo(xs, f)
    F = F[1:-1]
    return K, F

def solve_poisson_demo(xs):
    K, F = assemble_poisson_demo(xs)
    u = np.zeros(np.size(xs))
    u[1:-1] = la.solve(K, F)
    return u
```

The solution vector `u` contains the coefficients of the Galerkin solution $u_S$ in the global basis $\phi_0, \dots, \phi_n$.
Recall that the global basis functions satisfy

$$
\phi_j(x_k) = \delta_{jk} = \begin{cases}
1 & j=k,
\\
0 & \text{otherwise}.
\end{cases}
$$

Hence we can interpret the coefficients `u` as the point values of $u_S$ at the nodes $x_j$, $j=0,\dots,n$.

```python
plt.plot(xs, solve_poisson_demo(xs), label='$u_S$ coarse')
xs_fine = np.linspace(0,1)
plt.plot(xs_fine, solve_poisson_demo(xs_fine), label='$u_S$ fine')
plt.gca().set_xticks(xs)
plt.grid(axis='x')
plt.legend();
```

# Sparsity

Most of the elements of $K$ vanish, that is, $K$ is [sparse](https://en.wikipedia.org/wiki/Sparse_matrix). (In fact, in this particular case, $K$ is [tridiagonal](https://en.wikipedia.org/wiki/Tridiagonal_matrix).)
Solving the system $KU = F$ is typically the most costly part of the finite element method, and efficiency of the method boils down to this system being sparse.

```python
K, F = assemble_poisson_demo(xs_fine)
K
```

```python
plt.spy(K);
```

We could (and should) reimplement `assemble_K_demo` using the [sparse matrix package](https://docs.scipy.org/doc/scipy/reference/sparse.html) of SciPy.

# Implementation using Scikit-fem

Let us now assemble the linear system for the same problem by using the Scikit-fem package.

```python
import skfem as fem
from skfem.helpers import dot, grad # helpers make forms look nice
```

```python
@fem.BilinearForm
def a(u, v, w):
    # Return the integrand in the bilinear form a(u, v)
    # using a notation that works in all dimensions
    return dot(grad(u), grad(v))

@fem.LinearForm
def L(v, w):
    x = w.x[0] # quadrature points for numerical integration
    # Return the integrand in the linear form L(v)
    return f(x) * v # use the same f as before

def assemble_poisson(xs):
    mesh = fem.MeshLine(xs)
    basis = fem.Basis(mesh, fem.ElementLineP1())
    K = a.assemble(basis)
    F = L.assemble(basis)
    # Empty call to get_dofs() matches all boundary degrees-of-freedom
    boundary_dofs = basis.get_dofs()
    K, F, _, _ = fem.condense(K, F, D=boundary_dofs)
    return K, F
```

```python
K, F = assemble_poisson(xs)
# K is a sparse matrix, convert it to a standard matrix
print(K.toarray())
print(F)
```

We get the same linear system as before. Let's check that this is also the case when we use a finer mesh.

```python
K1, F1 = assemble_poisson_demo(xs_fine)
K2, F2 = assemble_poisson(xs_fine)
print(la.norm(K1 - K2))
print(la.norm(F1 - F2))
```

We can solve the Poisson problem in even briefer way using Scikit-fem.

```python
from skfem.models.poisson import laplace, unit_load
import skfem.visuals.matplotlib as fem_plt

mesh = fem.MeshLine(xs)
basis = fem.Basis(mesh, fem.ElementLineP1())
K = laplace.assemble(basis)
F = unit_load.assemble(basis)
fem_plt.plot(mesh, fem.solve(*fem.condense(K, F, D=basis.get_dofs())));
```

# Computational study of convergence

Let $I = [0,1]$ and let $u$ solve the problem

$$
\begin{cases}
-u'' = f & \text{on $I$},
\\
u(0) = 0 = u(1),
\end{cases}
$$

and consider the corresponding Galerkin solution $u_S$ computed in a mesh

$$
0 = x_0 < x_1 < \dots < x_n = 1.
$$

It is typical to think that $u_S$ is parametrized by the mesh constant

$$
h = \max_{e=1,\dots,n} |x_e - x_{e-1}|,
$$

and write $u_h = u_S$.

We will study computationally how the error $u_h - u$ converges in the space $L^2(I)$ as $h \to 0$.
In order to do this, we need to set up the problem so that we know the exact solution $u$.
A way to achieve this is to choose first $u$ satisfying the boundary conditions and then compute $f$.

We set

$$
u(x) = \sin(\pi x).
$$

Then $u(0) = u(1) = 0$ and

$$
-u''(x) = \pi^2 \sin(\pi x) =: f(x).
$$

```python
def u_exact(x):
    return np.sin(np.pi * x)

def f(x):
    return np.pi**2 * np.sin(np.pi * x)

# We need to redefine this, as we didn't parametrize f
def solve_poisson_demo(xs):
    K = assemble_K_demo(xs)
    K = K[1:-1,1:-1]
    F = assemble_F_demo(xs, f)
    F = F[1:-1]
    u = np.zeros(np.size(xs))
    u[1:-1] = la.solve(K, F)
    return u
```

```python
ns = [4, 8]
for n in ns:
    xs, h = np.linspace(0,1, n+1, retstep=True)
    u = solve_poisson_demo(xs)
    plt.plot(xs, u, label=f'{h = }')
xs_plot = np.linspace(0, 1)
plt.plot(xs_plot, u_exact(xs_plot), label='exact')
plt.legend();
```

Recalling that on $I_e$

$$
\phi_e(x) = \psi_1^e(x), \quad \phi_{e-1}(x) = \psi_0^e(x), \quad \phi_i(x) = 0, \quad i \ne e, e-1,
$$

we have

$$
\|u_h - u\|^2
= \sum_{e=1}^n \int_{I_e} |u_h(x) - u(x)|^2 dx
= \sum_{e=1}^n \int_{I_e} |U_{e} \psi_1^e(x) + U_{e-1} \psi_0^e(x) - u(x)|^2 dx.
$$

Evaluating the integral using the trapezium rule yields

$$
\int_{I_e} |U_{e} \psi_1^e(x) + U_{e-1} \psi_0^e(x) - u(x)|^2 dx
\approx \frac{x_e - x_{e-1}}{2} \left( |U_{e-1} - u(x_{e-1})|^2 + |U_{e} - u(x_{e})|^2 \right).
$$

In principle, we could expand the square and compute the terms

$$
U_i U_j \int_{I_e} \psi_1^i(x) \psi_0^j(x) dx, \quad i,j=e,e-1
$$

analytically. In practice, we can use a numerical integration method, that yields an exact result for polynomials of degree 2 or less, to achieve the same accuracy as the analytic computation. Here we will use the trapezium rule for simplicity.

```python
def error_sq_demo(x, u):
    '''Squared L2 error'''
    n = np.size(x) - 1
    error = 0
    for e in range(1, n+1):
        dx = x[e] - x[e-1]
        error += dx / 2 * ( (u[e-1] - u_exact(x[e-1]))**2 + (u[e] - u_exact(x[e]))**2 )
    return error

ns = [2**n for n in range(2,6)]
N = len(ns)
errs_demo = np.zeros(N)
hs = np.zeros(N)
for i in range(N):
    n = ns[i]
    xs, h = np.linspace(0,1, n+1, retstep=True)
    u = solve_poisson_demo(xs)
    hs[i] = h
    errs_demo[i] = np.sqrt(error_sq_demo(xs, u))
```

```python
plt.loglog(hs, errs_demo, '-o', label='error')
plt.loglog(hs, hs**2, label='$h^2$')
plt.legend();
```

# Evaluation of error using Scikit-fem

```python
# We need to redefine this, as we didn't parametrize f
@fem.LinearForm
def L(v, w):
    x = w.x[0] # quadrature points for numerical integration
    # Return the integrand in the linear form L(v)
    return f(x) * v

@fem.Functional
def error_sq(w):
    x = w.x[0]
    uh = w['uh'] # uh is given in the assemble call below
    # Return the integrand in the squared L2 error
    return (uh - u_exact(x))**2

def error_poisson(xs):
    mesh = fem.MeshLine(xs)
    basis = fem.Basis(mesh, fem.ElementLineP1())
    K = a.assemble(basis)
    F = L.assemble(basis)
    boundary_dofs = basis.get_dofs()
    u = fem.solve(*fem.condense(K, F, D=boundary_dofs))
    return np.sqrt(error_sq.assemble(basis, uh=basis.interpolate(u)))

errs = np.array([
    error_poisson(np.linspace(0,1, n+1)) for n in ns])
```

```python
plt.loglog(hs, errs, 'ob-', label='skfem')
plt.loglog(hs, errs_demo, '.r--', label='demo')
plt.legend();
```

The two errors are slightly different since Scikit-fem is using more accurate numerical integration method than the trapezium rule.

```python

```