 Text provided under a Creative Commons Attribution license, CC-BY. All code is made available under the FSF-approved MIT license. (c) Kyle T. Mandli

```python
from __future__ import print_function

%matplotlib inline
import numpy
import matplotlib.pyplot as plt
import warnings
```

# Numerical Quadrature

**Goal:** Accurately Evaluate definite integrals

$$
\int^b_a f(x) dx
$$

using a finite number of samples

### Why?

1. Many integrals do not have closed form solutions

$$
   \int^b_a \sqrt{1 + \cos^2x}~ dx
$$

### A more practical example:  The Error function

The error function:

$$
   \mathrm{erf}(x) = \frac{2}{\sqrt{\pi}}\int_0^x e^{-t^2} dt
$$
is related to the cumuluative probability distribution of a Gaussian.  And is useful in probability and PDE's.  But has no closed form solution.

```python
from scipy.special import erf
x = numpy.linspace(0.,3.,100)
fig = plt.figure(figsize=(8,6))
axes = fig.add_subplot(1,1,1)
axes.plot(x, 2/numpy.sqrt(numpy.pi)*numpy.exp(-x**2),label='Gaussian')
axes.plot(x, erf(x),label='erf(x)')
axes.grid()
axes.set_xlabel('x')
axes.legend(loc='best')
plt.show()
```

Solution to even simple non-linear ordinary differential equations

   $$\frac{\text{d} u}{\text{d}t} = f(u)g(t)$$ with initial condition $u(0)=u_0$

   Can always be solved by "reduction to quadrature" i.e. the solution is given implicitly by

   $$\int_{u_0}^{u}  \frac{dv}{f(v)} = \int_0^t g(\tau)d\tau$$

which are two, definite integrals that may, or may not have closed forms (but are essentially "the area under the curve")

Solution to ordinary differential equations

   $$\frac{\text{d}^2 u}{\text{d}t^2} = f\left(u, \frac{\text{d} u}{\text{d}t}, t \right)$$

   Defining $v = \frac{\text{d} u}{\text{d}t}$ then leads to

   $$\begin{bmatrix}
   \frac{\text{d} v}{\text{d}t} \\ \frac{\text{d} u}{\text{d}t} \end{bmatrix} = \begin{bmatrix} f(u, v, t) \\ v \end{bmatrix}$$

   which can be solved by integration

   $$\begin{bmatrix}
   v \\ u \end{bmatrix} = \begin{bmatrix} v(t_0) + \int^t_{t_0} f(u, v, \hat{t}) d\hat{t} \\ u(t_0) + \int^t_{t_0} v d\hat{t} \end{bmatrix}$$

Solving partial differential equations such as "Poisson's equation"
$$
     \nabla^2 u = f
$$

using the finite element method, will use quadrature to reduce the PDE to a linear algebraic problem $A\mathbf{u}=\mathbf{f}$

Solution of poisson's equation on an irregular domain using [TerraFERMA](http://terraferma.github.io)

### Basics of Quadrature

We want to approximate an integral $I$ with some approximation $I_N$ such that
$$
    I = \int^b_a f(x) dx \approx I_N = \sum^{N}_{i=1} w_i f(x_i)
$$
where the $x_i$ are the *quadrature points* or *nodes* and the $w_i$ are the *weights*.  Usually a particular quadrature rule specifies the points $x_i$ resulting in a particular set of weights $w_i$.

Convergence requires that
$$
    \lim_{N \rightarrow \infty} I_N = I.
$$

## Riemann Sums

Given $f(x)$ and a partition of the interval $[a,b]$ with $\{x_i\}^N_{i=0}$ and $a = x_0 < x_1 < \ldots < x_N = b$ and $x^*_i \in [x_i, x_{i+1}]$ we define the Riemann integral as

$$\int^b_a f(x) dx = \lim_{N\rightarrow \infty} \sum^{N-1}_{i=0} f(x_i^*) (x_{i+1} - x_i)$$

This is a general definition and leads to a number of quadrature approaches that depend on how we pick $x_i^* \in [x_i, x_{i+1}]$.

### Midpoint Rule

Choose $x_i^*$ such that

$$x_i^* = \frac{x_{i+1} + x_i}{2}$$

so that

$$I[f] = \int^b_a f(x) dx \approx \sum^{N-1}_{i=0} f\left(\frac{x_{i+1} + x_i}{2} \right ) (x_{i+1} - x_i) = I_N[f]$$

over $\Delta x_i = x_{i+1} - x_i$

#### Example:  Integrate using midpoint rule

Calculate and illustrate the midpoint rule.  Note that we are computing the cummulative integral here:

$$
    \int^x_0 \sin(\hat{x}) d\hat{x} = \left . -\cos \hat{x} \right|^x_0 = 1 - \cos x
$$

```python
# Note that this calculates the cumulative integral from 0.0

f = lambda x: numpy.sin(x)
I = lambda x: 1.0 - numpy.cos(x)
x = numpy.linspace(0.0, 2.0 * numpy.pi, 100)

num_partitions = 10
x_hat = numpy.linspace(0.0, 2.0 * numpy.pi, num_partitions + 1)
x_star = 0.5 * (x_hat[1:] + x_hat[:-1])
delta_x = x_hat[1] - x_hat[0]
```

```python
fig = plt.figure(figsize=(8,6))
fig.subplots_adjust(hspace=.5)
axes = fig.add_subplot(2, 1, 1)

axes.plot(x, numpy.zeros(x.shape), 'k--')
axes.plot(x, f(x), 'b')

for i in range(num_partitions):
    axes.plot([x_hat[i], x_hat[i]], [0.0, f(x_star[i])], 'k--')
    axes.plot([x_hat[i + 1], x_hat[i + 1]], [0.0, f(x_star[i])], 'k--')
    axes.plot([x_hat[i], x_hat[i + 1]], [f(x_star[i]), f(x_star[i])], 'k--')

axes.set_xlabel("x")
axes.set_ylabel("$f(x)$")
axes.set_title("Partition and $f(x)$")
axes.set_xlim((0.0, 2.0 * numpy.pi))
axes.set_ylim((-1.1, 1.1))

I_hat = numpy.zeros(num_partitions)
I_hat[0] = f(x_star[0]) * delta_x
for i in range(1, num_partitions):
    I_hat[i] = I_hat[i - 1] + f(x_star[i]) * delta_x

axes = fig.add_subplot(2, 1, 2)

axes.plot(x, I(x), 'r')
# Offset due to indexing above
axes.plot(x_star + delta_x / 2.0, I_hat, 'ko')

err = numpy.abs(I(x_hat[1:]) - I_hat)

axes.set_xlabel("x")
axes.set_ylabel("$f(x)$")
axes.set_title("Integral and Approximated Integral, Max Err = {}".format(err.max()))
axes.set_xlim((0.0, 2.0 * numpy.pi))
axes.set_ylim((-0.1, 2.5))
axes.grid()

plt.show()
```

### General Newton-Cotes Quadrature

Using $n+1$ equally spaced points on a single interval $x\in[a,b]$, evaluate $f(x)$ at these points and exactly integrate the interpolating polynomial:

$$I_n[f] = \int^b_a P_n(x) dx$$

```python
func = lambda x: 1 - x**2 + numpy.sin(5*x)
x0 = 0.
x1 = .5
x_buf = .2
x = numpy.linspace(x0 - x_buf, x1+ x_buf, 100)
x_interval = numpy.linspace(x0, x1, 100)

x_samp = numpy.array([ 0.5*(x0 + x1) ])
f_samp = func(x_samp)

fig = plt.figure(figsize=(15,5))
axes = fig.add_subplot(1,3,1)
axes.plot(x,func(x),linewidth=3)
axes.plot(x_samp, f_samp, 'ro')
axes.plot(x_interval, f_samp*numpy.ones(x_interval.shape), 'k--')
axes.plot(x0*numpy.ones(2), [0., f_samp],'k--')
axes.plot(x1*numpy.ones(2), [0., f_samp],'k--')
axes.plot(x0*numpy.ones(2), [0., func(x0)],'b')
axes.plot(x1*numpy.ones(2), [0., func(x1)],'b')
axes.plot(x, numpy.zeros(x.shape),'k')
axes.text(x0-.02,-.2,'$x_i$',fontsize=16)
axes.text(x1-.02,-.2,'$x_{i+1}$',fontsize=16)
axes.set_title('Midpoint Rule',fontsize=16)
axes.set_xlabel('x', fontsize=16)
axes.set_ylabel('f(x)', fontsize=16)
axes.set_ylim((-0.25, 2.))
axes.fill_between(x_interval,0.,func(x_interval))
axes.fill_between(x_interval, 0, f_samp*numpy.ones(x_interval.shape), hatch='/', alpha=0)
axes.grid()

axes = fig.add_subplot(1,3,2)
x_samp = numpy.array([x0, x1])
f_samp = func(x_samp)
axes.plot(x,func(x),linewidth=3)
axes.plot(x_samp, f_samp, 'ro')
axes.plot(x_samp, f_samp, 'k--')
axes.plot(x0*numpy.ones(2), [0., f_samp[0]],'k--')
axes.plot(x1*numpy.ones(2), [0., f_samp[1]],'k--')
axes.plot(x0*numpy.ones(2), [0., func(x0)],'b')
axes.plot(x1*numpy.ones(2), [0., func(x1)],'b')
axes.plot(x, numpy.zeros(x.shape),'k')
axes.text(x0-.02,-.2,'$x_i$',fontsize=16)
axes.text(x1-.02,-.2,'$x_{i+1}$',fontsize=16)
axes.set_title('Trapezoidal Rule',fontsize=16)
axes.set_xlabel('x', fontsize=16)
axes.set_ylabel('f(x)', fontsize=16)
axes.set_ylim((-0.25, 2.))
axes.fill_between(x_interval,0.,func(x_interval))
axes.fill_between(x_samp,0.,f_samp, hatch='/', alpha=0)

axes.grid()

axes = fig.add_subplot(1,3,3)
x_samp = numpy.array([x0, (x0 + x1)/2., x1])
f_samp = func(x_samp)
p = numpy.polyfit(x_samp, f_samp, 2)
f_p2 = numpy.polyval(p, x_interval)
axes.plot(x,func(x),linewidth=3)
axes.plot(x_samp, f_samp, 'ro')
axes.plot(x_interval, f_p2, 'k--')
axes.plot(x0*numpy.ones(2), [0., f_samp[0]],'k--')
axes.plot(x1*numpy.ones(2), [0., f_samp[-1]],'k--')
axes.plot(x0*numpy.ones(2), [0., func(x0)],'b')
axes.plot(x1*numpy.ones(2), [0., func(x1)],'b')
axes.plot(x, numpy.zeros(x.shape),'k')
axes.text(x0-.02,-.2,'$x_i$',fontsize=16)
axes.text(x1-.02,-.2,'$x_{i+1}$',fontsize=16)
axes.set_title('Simpsons Rule',fontsize=16)
axes.set_xlabel('x', fontsize=16)
axes.set_ylabel('f(x)', fontsize=16)
axes.set_ylim((-0.25, 2.))
axes.fill_between(x_interval,0.,func(x_interval))
axes.fill_between(x_interval,0.,f_p2, hatch='/', alpha=0)

axes.grid()
plt.show()
```

#### Trapezoidal Rule

Use $n = 1$ polynomial to derive the trapezoidal rule.

Trapezoidal rule uses $n = 1$ order polynomials between each point (i.e. piece-wise defined linear polynomials).  Using the Linear Lagrange basis we can write the interpolating polynomial as

$$
    P_1(x) = f(x_i)\frac{x_{i+1}-x}{\Delta x} + f(x_{i+1})\frac{x - x_i}{\Delta x}
$$

where $\Delta x = x_{i+1}-x_i$ is the width of the interval.

Integrating this polynomial we have for a single interval

\begin{aligned}
    I_T[f] &= \int^{x_{i+1}}_{x_i} f(x_i)\ell_0(x) + f(x_{i+1})\ell_1(x) dx  \\
    &=\frac{1}{\Delta x}  \left. \left[ f(x_i)\left(x_{i+1}x - \frac{x^2}{2}\right) + f(x_{i+1})\left(\frac{x^2}{2}-xx_i\right) \right] \right|_{x_i}^{x_{i+1}} \\
    &= \frac{1}{\Delta x}  \left[ f(x_i)\left(\frac{x_{i+1}^2}{2} -x_ix_{i+1} + x_i^2\right) + f(x_{i+1})\left(\frac{x_{i+1}^2}{2} -x_ix_{i+1} + x_i^2\right) \right]\\
    &= \frac{(x_{i+1} - x_i)^2}{2\Delta x}\left[ f(x_i) + f(x_{i+1})\right]\\
    & = \frac{\Delta x}{2}\left[ f(x_i) + f(x_{i+1}) \right] \\
    & = \Delta x\left[ \frac{1}{2} f(x_i) + \frac{1}{2} f(x_{i+1})\right]
\end{aligned}

### Transformation of the interval

For quadrature,  it is often  more convenient to transform the interval from $[x_i, x_{i+1}]$ to the unit interval using the transformation

$$
    x(s) = x_i + s\Delta x
$$

where $s\in[0,1]$ is just the fractional distance across the element and $dx = \Delta x ds$.

As before,  in this frame
$$
    \ell_0(s) = (1-s), \quad \ell_1(s) = s
$$

Under this transformation,  the original integral
$$
    I_T[f] = \int^{x_{i+1}}_{x_i} f(x_i)\ell_0(x) + f(x_{i+1})\ell_1(x) dx
$$

Becomes much simpler
$$
    I_T[f] = \Delta x \int_{0}^{1} f(x_i)(1 - s) + f(x_{i+1})s\, ds
$$

or
$$
\begin{aligned}
I_T[f] &= \Delta x \left. \left[ f(x_i)(s -\frac{s^2}{2}) + f(x_{i+1}) \frac{s^2}{2} \right]  \right|_0^1\\
    & = \Delta x\left[ \frac{1}{2} f(x_i) + \frac{1}{2} f(x_{i+1})\right]
\end{aligned}
$$

### Extended Trapezoidal rule

If we now panel a larger interval with $N$ panels of size $\Delta x$,  we can calculate the *Extended Trapezoidal rule*, by summing the individual contributions of each Trapezoidal panel.

$$
    I_N[f] = \sum_{j=1}^N I_T[f]_j
$$

We can also simplify the sum over all the intervals by noting that all but the end points will have total contribution of $\Delta x$ to the entire sum such that

$$
    I_N[f] = \Delta x \left[ \frac{1}{2} (f(x_0) + f(x_N) ) + \sum^{N-1}_{j=1} f(x_j) \right]
$$

```python
# Note that this calculates the cumulative integral from 0.0

f = lambda x: numpy.sin(x)
I = lambda x: 1.0 - numpy.cos(x)
x = numpy.linspace(0.0, 2.0 * numpy.pi, 100)

num_partitions = 20
x_hat = numpy.linspace(0.0, 2.0 * numpy.pi, num_partitions + 1)
delta_x = x_hat[1] - x_hat[0]
```

```python
fig = plt.figure(figsize=(8,6))
fig.subplots_adjust(hspace=.5)
axes = fig.add_subplot(2, 1, 1)

axes.plot(x, numpy.zeros(x.shape), 'k--')
axes.plot(x, f(x), 'b')

for i in range(num_partitions):
    axes.plot([x_hat[i], x_hat[i]], [0.0, f(x_hat[i])], 'k--')
    axes.plot([x_hat[i + 1], x_hat[i + 1]], [0.0, f(x_hat[i+1])], 'k--')
    axes.plot([x_hat[i], x_hat[i + 1]], [f(x_hat[i]), f(x_hat[i+1])], 'k--')

axes.set_xlabel("x")
axes.set_ylabel("$f(x)$")
axes.set_title("Partition and $f(x)$")
axes.set_xlim((0.0, 2.0 * numpy.pi))
axes.set_ylim((-1.1, 1.1))

I_hat = numpy.zeros(num_partitions)
I_hat[0] = (f(x_hat[1]) + f(x_hat[0])) * delta_x / 2.0
for i in range(1, num_partitions):
    I_hat[i] = I_hat[i - 1] + (f(x_hat[i + 1]) + f(x_hat[i])) * delta_x / 2.0

err = numpy.abs(I(x_hat[1:]) - I_hat)

axes = fig.add_subplot(2, 1, 2)

axes.plot(x, I(x), 'r')
# Offset due to indexing above
axes.plot(x_hat[1:], I_hat, 'ko')

axes.set_xlabel("x")
axes.set_ylabel("$f(x)$")
axes.set_title("Integral and Approximated Integral, Max Err = {}".format(err.max()))
axes.set_xlim((0.0, 2.0 * numpy.pi))
axes.set_ylim((-0.1, 2.5))
axes.grid()

plt.show()
```

#### Simpson's Rule

Simpson's rule uses $N = 2$ order polynomials between each point (i.e. piece-wise defined quadratic polynomials at points $(x_i, x_i+\Delta x/2, x_{i+1}$ where $\Delta x = x_{i+1} - x_{i}$).

The polynomial in the Lagrange basis has the form

$$P_2(x) = f(x_i)\ell_0(x) + f(x_i + \Delta x/2)\ell_1(x) + f(x_{i+1})\ell_2(x)$$

Which we can just integrate to find

$$
I_S[f] = \int^{x_{i+1}}_{x_i} P_2(x) dx
$$

For this problem,  transformation of the interval  makes this much easier, and the Quadrature rule becomes

$$
\begin{aligned}
I_S[f] &= \Delta x\int_0^1  \left[f_0\ell_0(s) + f_1\ell_1(s) + f_2\ell_2(s)\right]\,ds\\
       &= \Delta x\int_0^1 \left[f_0\frac{(s-1/2)(s-1)}{1/2} + f_1 \frac{s(s-1)}{-1/4} + f_2\frac{s(s-1/2)}{1/2}\right]\,ds
\end{aligned}
$$

after a bit of work we get

$$   \begin{aligned}I_S[f] &= \Delta x\left[\frac{1}{6} f(x_i) + \frac{2}{3} f(x_i + 1/2\Delta x) + \frac{1}{6} f(x_{i+1}) \right] \\
&= \frac{\Delta x}{6}\left[f(x_i) + 4 f(x_i + 1/2\Delta x) + f(x_{i+1}) \right] \\
\end{aligned}
$$

Note:  Like all quadrature rules,  the integral over a single interval of size $\Delta x$ is always

$$
    I \approx \Delta x\sum_{i=0}^n w_i f(x_i)
$$

### Derivation of Newton Cotes formulas using the method of undetermined coefficients.

Use the general form of the quadrature rule and determine weights $w_j$ by using functions we know the solution to.  These functions can be any representation of polynomials up to order $N=2$ however the monomials $1$, $x$, $x^2$ are the easiest in this case.

$$
    I_{\Delta x}[f] = w_0 f(0) + w_1 f(\Delta x / 2) + w_2 f(\Delta x)
$$

$$\begin{aligned}
    &\text{if}~f = 1:  &I[f] =  \int^{\Delta x}_{0} 1 dx = \Delta x  \\
    &\text{if}~f = x:  &I[f] =  \int^{\Delta x}_{0} x dx = \frac{\Delta x^2}{2} \\
    &\text{if}~f = x^2:  &I[f] =  \int^{\Delta x}_{0} x^2 dx = \frac{\Delta x^3}{3}
\end{aligned}$$

What are the corresponding systems of equations?

$$
    I[f] = w_0 f(0) + w_1 f(\Delta x / 2) + w_2 f(\Delta x)
$$

$$\begin{aligned}
    &\text{if}~f = 1:  &I[f] =  \int^{\Delta x}_{0} 1 dx = \Delta x & & I_N[1] &= w_0 + w_1 + w_2 \\
    &\text{if}~f = x:  &I[f] =  \int^{\Delta x}_{0} x dx = \frac{\Delta x^2}{2} & & I_N[x] &= w_1 \frac{\Delta x}{2} + w_2\Delta x\\
    &\text{if}~f = x^2:  &I[f] =  \int^{\Delta x}_{0} x^2 dx = \frac{\Delta x^3}{3} & & I_N[x^2] &= \frac{\Delta x^2}{4} w_1 + w_2\Delta x^2\\
\end{aligned}$$

We then have the system of equations:
$$\begin{aligned}
    w_0 &+& w_1 &+& w_2 &=\Delta x \\
        &\quad& \frac{\Delta x}{2} w_1 &+& \Delta x w_2  &= \frac{\Delta x^2}{2} \\
        &\quad& \frac{\Delta x^2}{4} w_1 &+& \Delta x^2 w_2 &=\frac{\Delta x^3}{6} \\
\end{aligned}$$

or in Matrix-vector form $A\mathbf{x}=\mathbf{b}$

$$\begin{bmatrix}
1 & 1 & 1 \\
0 & \Delta x/2 &  \Delta x \\
0 & \Delta x^2/4 & \Delta x^2 \\
\end{bmatrix}
\begin{bmatrix} w_0 \\ w_1 \\  w_2 \\ \end{bmatrix}=
\begin{bmatrix} \Delta x \\ \Delta x^2/2 \\  \Delta x^3/6 \\ \end{bmatrix}
$$

or after some manipulation (and elimination) gives

$$
\begin{bmatrix}
    1 & 1 & 1 \\
    0 & 1 / 2 & 1 \\
    0 & 1 / 4 & 1 \\
\end{bmatrix} \begin{bmatrix}
    w_0 \\ w_1 \\ w_2
\end{bmatrix} = \Delta x \begin{bmatrix}
    1 \\ 1 / 2 \\ 1 / 3
\end{bmatrix}\Rightarrow
\begin{bmatrix}
    1 & 1 & 1 \\
    0 & 1 / 2 & 1 \\
    0 & 0 & 1 \\
\end{bmatrix} \begin{bmatrix}
    w_0 \\ w_1 \\ w_2
\end{bmatrix} = \Delta x \begin{bmatrix}
    1 \\   1 / 2 \\   1 / 6
\end{bmatrix}
$$

Leading to

$$ w_2 = \frac{\Delta x}{6} \quad w_1 = \frac{2}{3} \Delta x \quad w_0 = \frac{\Delta x}{6}$$

Another way to write Simpson's rule is to use intervals of three points (similar to one of the ways we did this last time).  The formulation here effectively has a $\Delta x$ half of what the intervals show but is easier to program.

```python
# Note that this calculates the cummulative integral from 0.0

f = lambda x: numpy.sin(x)
I = lambda x: 1.0 - numpy.cos(x)
x = numpy.linspace(0.0, 2.0 * numpy.pi, 100)

num_partitions = 20
x_hat = numpy.linspace(0.0, 2.0 * numpy.pi, num_partitions + 1)
delta_x = x_hat[1] - x_hat[0]
```

```python
fig = plt.figure(figsize=(8,6))
fig.subplots_adjust(hspace=.5)
axes = fig.add_subplot(2, 1, 1)

axes.plot(x, numpy.zeros(x.shape), 'k--')
axes.plot(x, f(x), 'b')

for i in range(num_partitions):
    axes.plot([x_hat[i], x_hat[i]], [0.0, f(x_hat[i])], 'k--')
    axes.plot([x_hat[i + 1], x_hat[i + 1]], [0.0, f(x_hat[i + 1])], 'k--')
    coeff = numpy.polyfit((x_hat[i], x_hat[i] + delta_x / 2.0, x_hat[i + 1]),
                          (f(x_hat[i]), f(x_hat[i] + delta_x / 2.0), f(x_hat[i+1])), 2)
    x_star = numpy.linspace(x_hat[i], x_hat[i+1], 10)
    axes.plot(x_star, numpy.polyval(coeff, x_star), 'k--')

axes.set_xlabel("x")
axes.set_ylabel("$f(x)$")
axes.set_title("Partition and $f(x)$")
axes.set_xlim((0.0, 2.0 * numpy.pi))
axes.set_ylim((-1.1, 1.1))
#axes.grid()

I_hat = numpy.zeros(num_partitions)
I_hat[0] = delta_x * (1.0 / 6.0 * (f(x_hat[0]) + f(x_hat[1])) + 2.0 / 3.0 * f(x_hat[0] + delta_x / 2.0))
for i in range(1, num_partitions):
    I_hat[i] = I_hat[i - 1] + delta_x * (1.0 / 6.0 * (f(x_hat[i]) + f(x_hat[i+1])) + 2.0 / 3.0 * f(x_hat[i] + delta_x / 2.0))

err = numpy.abs(I(x_hat[1:]) - I_hat)

axes = fig.add_subplot(2, 1, 2)

axes.plot(x, I(x), 'r')
# Offset due to indexing above
axes.plot(x_hat[1:], I_hat, 'ko')

axes.set_xlabel("x")
axes.set_ylabel("$f(x)$")
axes.set_title("Integral and Approximated Integral, Max Err = {}".format(err.max()))
axes.set_xlim((0.0, 2.0 * numpy.pi))
axes.set_ylim((-0.1, 2.5))
axes.grid()

plt.show()
```

## Error Analysis

From before we have a particular $n$-point quadrature scheme $I_n$ for a single interval which we can also write as

$$
    I_n[f] = \sum^{n-1}_{i=0} w_i f(x_i).
$$

Define the error $E[f]$ such that

$$I[f] = I_n[f] + E_n[f]$$

The *degree* of $I_n[f]$ is the integer $p$ such that $E_n[P_i] = 0 \quad \forall i \leq p$ and $\exists P_{p+1}$ such that $E[P_{p+1}] \neq 0$. In other words,  it is the maximum degree polynomial that is integrated *exactly* using $I_n$.  As we will show

* Mid-point rule: p=1 -- exact for all linear polynomials
* Trapezoidal rule: p=1 -- also exact for all linear polynomials
* Simpson's rule: p=3 -- exact for all cubic polynomials

There are multiple (related) ways to estimate Quadrature error that either use the Taylor series or the Lagrange Remainder theorem.

#### Mid-Point error

For a 1-Point quadrature rule like the Mid-point rule,  Taylor's theorem is easiest to use as we can just expand $f(x)$ around the midpoint such that

$$
    f(x) = f(x^*) + f'(x^*)(x - x^*) + \frac{f''(x^*)}{2}(x - x^*)^2 + O(f'''(x - x^*)^3)
$$

where $x^*=(x_i + x_{i+1})/2$ is the midpoint of the interval

Integrating over one element $x\in[x_i,x_{i_1}]$ Gives

$$
\begin{align}
    \int_{x_i}^{x_{i+1}} f(x) dx  &=  \int_{x_i}^{x_{i+1}} \left(f(x^*) + f'(x^*)(x - x^*) + \frac{f''(x^*)}{2}(x - x^*)^2 + O(f'''(x - x^*)^3)\right) dx\\
    &=  f(x^*)\Delta x + \int_{x_i}^{x_{i+1}}  \left(f'(x^*)(x - x^*) + \frac{f''(x^*)}{2}(x - x^*)^2 + O(f'''(x - x^*)^3)\right) dx\\
    \end{align}
$$

or
$$
    I[f] = I_M[f] + \int_{x_i}^{x_{i+1}} f'(x^*)(x - x^*)dx + \int_{x_i}^{x_{i+1}} \frac{f''(x^*)}{2}(x - x^*)^2dx + HOT
$$

where $I_M[f] = f(x^*)\Delta x$ is the Midpoint quadrature rule.

With a bit of work (or some clear thinking) you can show that the the second term on the RHS is exactly zero and that the leading order error term comes from  the third-term which evaluates to

$$
    R_M[f] = \frac{\Delta x^3 f''(x_{1/2})}{24}
$$

As the mid-point rule is exact for degree 1 polynomials (i.e. straight lines)

### Error Estimates using Polynomial interpolation

We can also use our polynomial analysis  to analyze  errors.  From Lagrange's theorem we have the remainder term as before which we can use to look at the error

$$R_N(x) = (x - x_0)(x - x_1) \cdots (x- x_N) \frac{f^{(N+1)}(c)}{(N+1)!}$$

and integrate it to find the form and magnitude of the error on a *single* interval.

#### Trapezoidal error

With $n=1$ we have

$$R_1(x) = (x - x_i) (x - x_{i+1}) \frac{f''(c)}{2}$$

Integrating this leads to

$$\int^{x_{i+1}}_{x_i} (x - x_i) (x - x_{i+1}) \frac{f''(c)}{2} dx = \frac{\Delta x^3}{12} f''(c)$$

**Note**:
   * The single-interval error for the Trapezoidal rule is of the same order as the Mid-point rule
   * Surprisingly, the magnitude of error for the Trapezoidal rule is **twice** that of the Mid-point rule!

#### Simpson's Rule Error

We could apply the same approach to Simpson's rule using the remainder term for quadratic functions

$$R_2(x) = (x - x_i) \left(x - x_i - \frac{\Delta x}{2} \right) (x - x_{i+1}) \frac{f'''(c)}{3!}$$

However we would actually find that this term cancels exactly (in the same way that the mid-point scheme has a cancellation in the Taylor series expansion).  A more detailed Taylor's series analysis shows that the leading error term for Simpson's rule is

$$E_S[f] = -\frac{1}{180}\Delta x^5 f^{(4)}(c)$$

which is exact for cubic polynomials ($p=3$). Interestingly we have gained two orders of accuracy by increasing the polynomial order by only 1!

### Review:  Newton-Cotes Formulas and errors for a single interval

given $x\in[x_i, x_{i+1}]$ and $\Delta x = x_{i+1} - x_i$, all of the   Newton-Cotes quadrature formula's can be derived by integrating the interpolating polynomial through $N$ equi-spaced points across the interval.

* Mid-point: 1-point quadrature rule
$$
    I_M[f] = \Delta x\, f(x_i + \Delta x/2) + O(\Delta x^3f'')
$$
* Trapezoidal Rule: 2-point quadrature rule
$$
    I_T[f] = \Delta x\, \left[\frac{1}{2}f(x_i) +\frac{1}{2}f(x_{i+1})\right] + O(\Delta x^3f'')
$$
* Simpson's Rule: 3-point quadrature rule
$$
    I_S[f] = \Delta x\, \left[ \frac{1}{6}f(x_i) +  \frac{2}{3}f(x_i +\Delta x/2) +\frac{1}{6}f(x_{i+1})\right] + O(\Delta x^5f^{iv})
$$

```python
func = lambda x: 1 - x**2 + numpy.sin(5*x)
x0 = 0.
x1 = .5
x_buf = .2
x = numpy.linspace(x0 - x_buf, x1+ x_buf, 100)
x_interval = numpy.linspace(x0, x1, 100)

x_samp = numpy.array([ 0.5*(x0 + x1) ])
f_samp = func(x_samp)

fig = plt.figure(figsize=(15,5))
axes = fig.add_subplot(1,3,1)
axes.plot(x,func(x),linewidth=3)
axes.plot(x_samp, f_samp, 'ro')
axes.plot(x_interval, f_samp*numpy.ones(x_interval.shape), 'k--')
axes.plot(x0*numpy.ones(2), [0., f_samp],'k--')
axes.plot(x1*numpy.ones(2), [0., f_samp],'k--')
axes.plot(x0*numpy.ones(2), [0., func(x0)],'b')
axes.plot(x1*numpy.ones(2), [0., func(x1)],'b')
axes.plot(x, numpy.zeros(x.shape),'k')
axes.text(x0-.02,-.2,'$x_i$',fontsize=16)
axes.text(x1-.02,-.2,'$x_{i+1}$',fontsize=16)
axes.set_title('Midpoint Rule',fontsize=16)
axes.set_xlabel('x', fontsize=16)
axes.set_ylabel('f(x)', fontsize=16)
axes.set_ylim((-0.25, 2.))
axes.fill_between(x_interval,0.,func(x_interval))
axes.fill_between(x_interval, 0, f_samp*numpy.ones(x_interval.shape), hatch='/', alpha=0)
axes.grid()

axes = fig.add_subplot(1,3,2)
x_samp = numpy.array([x0, x1])
f_samp = func(x_samp)
axes.plot(x,func(x),linewidth=3)
axes.plot(x_samp, f_samp, 'ro')
axes.plot(x_samp, f_samp, 'k--')
axes.plot(x0*numpy.ones(2), [0., f_samp[0]],'k--')
axes.plot(x1*numpy.ones(2), [0., f_samp[1]],'k--')
axes.plot(x0*numpy.ones(2), [0., func(x0)],'b')
axes.plot(x1*numpy.ones(2), [0., func(x1)],'b')
axes.plot(x, numpy.zeros(x.shape),'k')
axes.text(x0-.02,-.2,'$x_i$',fontsize=16)
axes.text(x1-.02,-.2,'$x_{i+1}$',fontsize=16)
axes.set_title('Trapezoidal Rule',fontsize=16)
axes.set_xlabel('x', fontsize=16)
axes.set_ylabel('f(x)', fontsize=16)
axes.set_ylim((-0.25, 2.))
axes.fill_between(x_interval,0.,func(x_interval))
axes.fill_between(x_samp,0.,f_samp, hatch='/', alpha=0)

axes.grid()

axes = fig.add_subplot(1,3,3)
x_samp = numpy.array([x0, (x0 + x1)/2., x1])
f_samp = func(x_samp)
p = numpy.polyfit(x_samp, f_samp, 2)
f_p2 = numpy.polyval(p, x_interval)
axes.plot(x,func(x),linewidth=3)
axes.plot(x_samp, f_samp, 'ro')
axes.plot(x_interval, f_p2, 'k--')
axes.plot(x0*numpy.ones(2), [0., f_samp[0]],'k--')
axes.plot(x1*numpy.ones(2), [0., f_samp[-1]],'k--')
axes.plot(x0*numpy.ones(2), [0., func(x0)],'b')
axes.plot(x1*numpy.ones(2), [0., func(x1)],'b')
axes.plot(x, numpy.zeros(x.shape),'k')
axes.text(x0-.02,-.2,'$x_i$',fontsize=16)
axes.text(x1-.02,-.2,'$x_{i+1}$',fontsize=16)
axes.set_title('Simpsons Rule',fontsize=16)
axes.set_xlabel('x', fontsize=16)
axes.set_ylabel('f(x)', fontsize=16)
axes.set_ylim((-0.25, 2.))
axes.fill_between(x_interval,0.,func(x_interval))
axes.fill_between(x_interval,0.,f_p2, hatch='/', alpha=0)

axes.grid()
plt.show()
```

##### Example 1:

Given $f(x) = \sin \pi x$. Let's consider the relative accuracy of midpoint, trapezoidal and simpson's rules for a single interval $x\in[0,1]$.

$$\begin{aligned}
    \text{Exact:}  &I[f] = \int^1_0 \sin \pi x = \left . \frac{-\cos \pi x}{\pi} \right |^1_0 = \frac{2}{\pi} \approx 0.636619772 \\
    \text{Midpoint:}  &I_M[f] = \Delta x f(1/2) = \sin (\pi / 2) = 1 \\
    \text{Trapezoid:}  &I_T[f] = \frac{\Delta x}{2} (\sin(0) + \sin(\pi)) = 0 \\
    \text{Simpson's:}  &I_S[f] = \frac{\Delta x}{6} \sin(0) + \frac{2 \Delta x}{3} \sin(\pi / 2) + \frac{\Delta x}{6} \sin(\pi) = \frac{2 \Delta x}{3} = \frac{2}{3}
\end{aligned}$$

##### Calculate the relative error for each of these estimates

```python
err=numpy.abs(2/numpy.pi - 0)
err

```

### Error in extended Newton-Cotes formulas

To find the total error we must sum the error over all the intervals:

$$I[f] = \sum_{i=0}^N \int^{x_{i+1}}_{x_i} P_n(x) dx + \sum_{i=0}^N \int^{x_{i+1}}_{x_i} R_n(x) dx = I_N[f] + E_N[f]$$

as we defined before.

####  Extended Midpoint (and Trapezoidal) rule error

If we sum up across all the intervals the total error for the mid-point rule is

$$E_N[f] = -\frac{\Delta x^3}{24} \sum_{i=0}^{N} f''(c_i)$$

or noting that $\Delta x = (b-a)/N$ we can write this a bit more physically as

$$E_N[f] = -\frac{1}{24} \Delta x^2 (b - a) \left [ \frac{1}{N} \sum^{N-1}_{i=0} f''(c_i) \right ]$$

such that expression in the brackets is the mean value of the second derivative over the interval $[a,b]$.  This also shows that the extended  trapezoidal rule converges quadratically as $\Delta x \rightarrow 0$ (or $N\rightarrow\infty$).

In general, for $N$ panels of any given Newton-Cotes  quadrature formula over the interval $x\in[a,b]$, the approximate integral and error formulas are

\begin{align}
I[f] &= \sum_{i=0}^N I_t + E_N\\
\end{align}
where
$$
    E_N = (b-a)O(\Delta x^p f^{m}(c))
$$

* Extended Mid-point:
$$
    E_{N_M} = (b-a) O(\Delta x^2 f'')
$$
* Extended Trapezoidal Rule:
$$
    E_{N_T} = (b-a) O(\Delta x^2 f'')
$$
* Extended Simpson:
$$
    E_{N_S} = (b-a) O(\Delta x^4 f^{(iv)})
$$

Where $\Delta x = \frac{b -a }{N}$

#### Convergence of Extended N-C quadrature

Here we plot the relative error for various extended Newton-Cotes quadrature formulas for the test case

$$
    I = \int_0^1 \sin(\pi x) dx = \frac{2}{\pi}
$$

```python
# Compute the error as a function of delta_x for each method
f = lambda x: numpy.sin(numpy.pi * x)

num_partitions = numpy.array([ 2**n for n in range(0, 16)])
delta_x = numpy.empty(len(num_partitions))
error_mid = numpy.empty(len(num_partitions))
error_trap = numpy.empty(len(num_partitions))
error_simpson = numpy.empty(len(num_partitions))

I_true = 2.0/numpy.pi

for (j, N) in enumerate(num_partitions):
    x_hat = numpy.linspace(0.0, 1.0, N + 1)
    delta_x[j] = x_hat[1] - x_hat[0]

    # Compute Midpoint
    x_star = 0.5 * (x_hat[1:] + x_hat[:-1])
    I_hat = 0.0
    for i in range(0, N):
        I_hat += f(x_star[i]) * delta_x[j]
    error_mid[j] = numpy.abs(I_hat - I_true)/I_true

    # Compute trapezoid
    I_hat = 0.0
    for i in range(1, N):
        I_hat += (f(x_hat[i + 1]) + f(x_hat[i])) * delta_x[j] / 2.0
    error_trap[j] = numpy.abs(I_hat - I_true)/I_true

    # Compute simpson's
    I_hat = 0.0
    for i in range(0, N):
        I_hat += delta_x[j] * (1.0 / 6.0 * (f(x_hat[i]) + f(x_hat[i+1])) + 2.0 / 3.0 * f(x_hat[i] + delta_x[j] / 2.0))
    error_simpson[j] = numpy.abs(I_hat - I_true)/I_true

fig = plt.figure(figsize=(8,6))
axes = fig.add_subplot(1, 1, 1)

order_C = lambda delta_x, error, order: numpy.exp(numpy.log(error) - order * numpy.log(delta_x))
axes.loglog(delta_x, error_mid, 'ro', label="Midpoint")
axes.loglog(delta_x, error_trap, 'bo', label="Trapezoid")
axes.loglog(delta_x, error_simpson, 'go', label="Simpson's")
axes.loglog(delta_x, order_C(delta_x[0], error_trap[0], 2.0) * delta_x**2.0, 'b--', label="2nd Order")
axes.loglog(delta_x, order_C(delta_x[0], error_simpson[0], 4.0) * delta_x**4.0, 'g--', label="4th Order")
axes.legend(loc='best')
axes.set_xlabel('$\Delta x$', fontsize=16)
axes.set_ylabel('Relative Error', fontsize=16)
axes.grid()

plt.show()
```

### Recursive Improvement of Accuracy

Say we ran the extended trapezoidal rule with step size $2 \Delta x$, we then will have

$$\begin{aligned}
			\int^{x_2}_{x_0} f(x) dx  &= \frac{2 \Delta x}{2} (f_0 + f_2) =  h (f_0 + f_2) \Rightarrow \\
			\int^b_a f(x)dx &\approx I_{2\Delta x}[f] = \sum^{N/2-1}_{j=0} \Delta x (f_{2j} + f_{2j+2}) \\
			&= \Delta x (f_{0} + f_{2})  + \Delta x (f_{2} + f_{4})  + \cdots + \Delta x (f_{N-2} + f_{N}) \\
			&= \Delta x\left ( f_0 + f_N +  2 \sum^{N/2-1}_{j=1} f_{2j} \right )
  \end{aligned}
$$

Now compare the two rules for $\Delta x$ and $2 \Delta x$:

$$\begin{align}I_{\Delta x}[f] &= \frac{\Delta x}{2} \left (f_0 + f_N + 2 \sum^{N-1}_{j=1} f_j \right)\\ I_{2 \Delta x}[f] &= \Delta x \left ( f_0 + f_N +  2 \sum^{N/2-1}_{j=1} f_{2j} \right )\end{align}$$

$$I_{\Delta x}[f] = \frac{1}{2} I_{2\Delta x} + \Delta x(f_1 + f_3 + \cdots + f_{N-1})$$

Here we see we can actually reuse the work we did to calculate $Q_{2 \Delta x}[f]$ to refine the integral.

### Arbitrary Intervals (Affine Transforms)

A lot of quadrature rules are defined on specific intervals, e.g $x\in[0,1]$ or $x\in[-1,1]$.  However any two intervals can always be mapped onto each other through an  *affine transform* or *affine map* which is a linear  transformation.

$$
    x = \alpha\xi + \beta
$$
where $\alpha$ is a scaling and $\beta$ is a shift.

Given $\alpha$ and $\beta$,  the inverse map is simply

$$
    \xi = \frac{x - \beta}{\alpha}
$$

### Example:

Map $ x \in [a,b]\rightarrow\xi \in [-1,1]$

```python
fig = plt.figure(figsize=(5,5))
axes = fig.add_subplot(1,1,1)
axes.plot(x,func(x),linewidth=3)
axes.fill_between(x_interval,0.,func(x_interval))
axes.plot(x, numpy.zeros(x.shape),'k')
axes.text(x0-.02,-.2,'$a$',fontsize=16)
axes.text((x0+x1)/2.-.02,-.2,'$x$',fontsize=16)
axes.text(x1-.02,-.2,'$b$',fontsize=16)
axes.set_xlabel('x', fontsize=16)
axes.set_ylabel('f(x)', fontsize=16)
axes.text(x0-.06,-.4,'$-1$',fontsize=16)
axes.text((x0+x1)/2.-.02,-.4,'$\\xi$',fontsize=16)
axes.text(x1-.02,-.4,'$1$',fontsize=16)
axes.set_ylim((-0.5, 2.))
axes.grid()

```

$$x =  \frac{b - a}{2}\xi + \frac{a+b}{2}$$

$$\xi = \left( x - \frac{a + b}{2}\right) \frac{2}{b-a}$$

Given an Affine mapping,  it is easy to transform any integral from one coordinate system to another

$$
    I[f] = \int^b_a f(x) dx = \int^1_{-1} f(x(\xi)) \frac{dx}{d\xi} d\xi = \frac{b - a}{2} \int^1_{-1} f(x(\xi)) d\xi\\
$$

and the $N$ point quadrature rule for a single interval is
$$
    I_n[f]_i = \frac{\Delta x_i}{2} \sum_{j=1}^{n} w_j f(x(\xi_j))
$$

and the cumulative integral over $M$ panels
$$
    I_N[f] = \sum_{i=1}^M I_n[f]_i
$$

#### Example:  Newton-Cotes Rules

We can rewrite our previous quadrature rules so that they are given on the interval $\xi \in [-1, 1]$ instead of $x \in [x_i, x_{i+1}]$.  Recall that a general quadrature rule can be written as
$$
    \sum^N_{i=1} w_i f(\xi_i)
$$
where $w_i$ are the weights and $\xi_i$ are the points specified to evaluate the function at.

 For Newton-Cotes rules we know that the points $\xi_i$ are uniformly distributed on $[-1, 1]$ but we still need to define the weights.  For a 2-Point trapezoid rule we can do this to find that
$$
    \int^1_{-1} f(x) dx \approx f(-1) + f(1)
$$
so the quadrature points are at  $\xi_0 = [-1, 1]$ with weights $w = [1, 1]$

Note that if we map this using our affine transform we would get back the original trapezoid rule:
$$
    I_2[f] = \frac{dx}{d\xi}\sum_{n=1}^{2} w_n f(x(\xi_n))  = \frac{\Delta x}{2}\left(f(x(-1)) + f(x(1))\right)
$$

Similarly for Simpson's rule we have
$$
    \xi = [-1, 0, 1] \quad \text{and} \quad w = \left[\frac{1}{3}, \frac{4}{3}, \frac{1}{3} \right].
$$

```python
def ncintegration(f, a, b, N):
    """ Approximate \int_a^b f(x)dx using  Newton-Cotes quadrature rules with N<=3 quadrature points"""

    assert (N > 0)
    assert (N < 4)
    # Build up a nested list of integration points and weights for the interval [-1, 1]:
    xNC = []
    wNC = []
    # 1 point Newton-Cotes (Mid-point)
    xNC.append(numpy.array([0.]))
    wNC.append(numpy.array([2.]))
    # 2 point Newton-Cotes quadrature (Trapezoidal)
    xNC.append(numpy.array([ -1., 1. ]))
    wNC.append(numpy.array([ 1. , 1.]))
    # 3 point Newton-Cotes quadrature (Simpson's rule)
    xNC.append(numpy.array([-1., 0., 1.]))
    wNC.append(numpy.array([1., 4., 1.])/3.)

    # make affine map between x and xi
    xi_map = lambda xi: (b - a)/2. * xi + (a + b) / 2.

    I = (b - a)/2. * wNC[N-1].dot(f(xi_map(xNC[N-1])))
    return I

```

```python
f = lambda x: numpy.sin(x)
I = lambda x: - numpy.cos(x)

a = numpy.pi/4.
b = 3*numpy.pi/4.
#b = numpy.pi/2
I_true = I(b) - I(a)
print(I_true)
```

```python
x = numpy.linspace(0.0, 2.0 * numpy.pi, 100)
x_interval = numpy.linspace(a, b, 100)
fig = plt.figure(figsize=(5,5))
axes = fig.add_subplot(1,1,1)
axes.plot(x,f(x),linewidth=3)
axes.fill_between(x_interval,0.,f(x_interval))
axes.plot(x, numpy.zeros(x.shape),'k')

axes.grid()
```

```python
# Calculate Newton-Cotes Quadrature and relative errors for 1,2, and 3 point NC quadrature schemes
for N in range(1,4):
    I = ncintegration(f,a,b,N)
    err = numpy.abs(I - I_true)/numpy.abs(I_true)
    print('N = {}, I_N = {}, err = {}'.format(N, I, err))

```

## Optimal Quadrature Methods

Newton-Cotes formulas assume a fixed, regularly spaced set of quadrature points and calculate quadrature rules based on integrating the interpolating polynomial.

Can we do better?

Actually, if we allow both the points and weights to be chosen in an optimal way, we can generate optimal quadrature rules with $N$ points that can exactly integrate polynomials up to order $p=2N-1$

### A Naive argument for $p=2N -1$ for an $N$ point quadrature scheme

Given an $N$ point quadrature scheme

$$
    \int_a^b f(x)dx = \sum_{i=1}^N w_i f(x_i)
$$

If we allow both $w_i$ and $x_i$ to be variables there are $2N$ unknowns and we can use the method of undetermined coefficients to generate $2N$ *non-linear* equations that exactly integrate all monomials up to degree $2N-1$.

##### Example 2:  2-Point Gauss-Legendre Quadrature

Consider $N=2$ on $x \in [-1,1]$

$$I_2[f] = w_0 f(x_0) + w_1 f(x_1)$$

using the method of *undetermined coefficients* we can solve for the four unknowns $w_0, w_1, x_0, x_1$ that exactly integrate the first 4 monomials, $1, x, x^2, x^3$.

Let $$I[f] = \int_{-1}^{1} f(x)dx \quad\text{and}\quad I_2[f] = w_0 f(x_0) + w_1 f(x_1)$$

Then
$$\begin{aligned}
    &I[1] &=  \int^{1}_{-1} 1 dx = 2 & & I_2[1] &= w_0 + w_1\\
    &I[x] &=  \int^{1}_{-1} x dx = 0 & & I_2[x] &= w_0 x_0 + w_1 x_1\\
    &I[x^2] &=  \int^{1}_{-1} x^2 dx = \frac{2}{3} & & I_2[x^2] &= w_0 x_0^2 + w_1 x_1^2\\
    &I[x^3] &=  \int^{1}_{-1} x^3 dx = 0 & & I_2[x^3] &= w_0 x_0^3 + w_1 x_1^3\\
\end{aligned}$$

Or as a system of non-linear equations
$$\begin{aligned}
    &w_0 + w_1 = 2\\
    &w_0 x_0 + w_1 x_1 = 0\\
    &w_0 x_0^2 + w_1 x_1^2 = \frac{2}{3}\\
    &w_0 x_0^3 + w_1 x_1^3 = 0\\
\end{aligned}$$

Note that we need to solve for 4 unknowns $x_0$, $x_1$, $w_0$, and $w_1$.  If we guess $w_0=w_1$ by symmetry, it will just fall out that $x_0=-x_1$ and

$$x_0 = -\sqrt{\frac{1}{3}}, x_1 = \sqrt{\frac{1}{3}}  $$

However, there turns out to be some much deeper math based on orthogonal polynomials that can generate both weights and quadrature points in a much more rigorous and well defined manner.

#### Quick Check

let's check this for $f(x)=x^3 - 3x^2 + 2x -1$

$$
    I[f] = \int_{-1}^1 f(x) dx = \left[ \frac{x^4}{4} - x^3 +x^2 -x \right]_{-1}^1
$$

```python
f = lambda x: x**3 -3*x**2 +2*x -1.
If = lambda x: x**4/4 - x**3 + x**2 -x
I_true = If(1.) - If(-1.)
I_true
```

```python
x = numpy.sqrt(1./3)
I_2 = f(-x) + f(x)
I_2
```

### Generalized Gaussian Quadrature

As it turns out,  both the weights and quadrature points for optimal Gaussian quadrature can be derived from the properties of families of orthogonal polynomials such as Chebyshev or Legendre polynomials.

In general,  all orthogonal polynomials form a family of polynomials of all orders with the property that

$$
    \int_a^b \omega(x)P_m(x)P_n(x) dx = 0 \quad \text{for} \quad m\neq n
$$

where $m$, $n$ are the order of the polynomial and $\omega(x)$ is a weighting function.  We say that that polynomials are orthogonal to each other under a weighted inner product. And because they are orthogonal, they are automatically linearly independent and can form a basis for the space of all polynomials up to degree $n$.

Recall something similar for vectors $x,y\in\mathbb{R}^n$ with the standard inner product (dot product)
$$
    <x, y> = \sum^N_{i=1} x_i y_i = ||x|| \cdot ||y|| \cos \theta.
$$
If $<x, y> = 0$ then the vectors $x$ and $y$ are orthogonal.

#### Example:  The Legendre Polynomials

The Legendre polynomials form one family that are orthogonal with $\omega(x)=1$ over the interval $x\in[-1,1]$.

The first few Legendre Polynomials are
\begin{align}
    P_0(x) &= 1\\
    P_1(x) &=x \\
    P_2(x) &= \frac{1}{2}(3x^2 -1)\\
    P_3(x) &= \frac{1}{2}x(5x^2 - 3)\\
       \vdots
\end{align}

```python
from scipy.special import legendre
x = numpy.linspace(-1,1,100)

fig = plt.figure(figsize=(8,6))
axes = fig.add_subplot(1,1,1)
for N in range(5):
    Pn = legendre(N)
    axes.plot(x, Pn(x), label='N={}'.format(N))

axes.grid()
axes.set_title('Legendre Polynomials', fontsize=18)
axes.set_xlabel('x',fontsize=16)
axes.legend(loc='best')
plt.show()

```

### Big idea

Let  $P_N(x)$ be an  orthogonal polynomial of degree $N$ with weighting function $\omega(x)$ on some interval $x\in[a,b]$ it follows that

$$
    \int^b_{a} \omega(x) x^i P_N(x) dx = 0 \quad \forall i < N,
$$

i.e.  $P_N(x)$ is *orthogonal* to all monomials $x^i$ ($i <N$) with respect to the weight function $\omega(x)$. (This follows from the fact that the orthogonal polynomials form a basis and all monomials can be written as a linear combination of $P_m$ for all $m<n$.

Moreover,  any general polynomial $h(x)$ up to order $p = 2N-1$ can be written as

$$
    h(x) = q(x)P_N(x) + r(x)
$$

where $q(x)$ is a quotient polynomial of degree $<N$ and $r(x)$ a remainder polynomial also of order strictly $<N$.  (Remember synthetic division?)

#### Example:  $N=2$, $p=2N-1=3$

let
$$
    h(x) = x^3 - 3x^2 + 4x + 2, \quad P_2 = \frac{1}{2}(3x^2-1)
$$

then

$$
    h(x) = \left(\frac{2}{3}x -2\right)\frac{1}{2}(3x^2-1)+ \left(\frac{13}{3}x + 1\right)
$$

or
$$
    q(x) = \left(\frac{2}{3}x -2\right),\quad r(x) = \left(\frac{13}{3}x + 1\right)
$$
both of which are 1st order polynomials

Given

$$
    h(x) = q(x)P_N(x) + r(x)
$$

By the properties of $P_N(x)$ it is easy to show two important results

$$
    \int_a^b \omega(x) h(x) dx = \int_a^b \omega(x)\left[q(x)P_N(x) + r(x)\right]dx = \int_a^b \omega(x) r(x) dx
$$

and given $x_i$ are the $N$ roots of $P_N(x)$

$$
    \sum_{i=1}^N w_i h(x_i) = \sum_{i=1}^N w_i \left[q(x_i)P_N(x_i) + r(x_i)\right] = \sum_{i=1}^N w_i r(x_i)
$$

Therefore we have a relationship between the weighted integral of $h$ and a discrete quadrature scheme with undetermined weights $w_i$.  All that remains is to find a set of weights that integrate all polynomials up to the order of $r(x)$ exactly.

Given that we know the quadrature points $x_i$ as the roots of the Legendre polynomials,  we could compute the weights by the method of undetermined coefficients.

Moreover,  we can use the lagrange basis sampled at the roots of $P_N(x)$ to expand $r(x)$ exactly as

$$
    r(x) = \sum_{i=1}^N r(x_i)\ell_i(x)
$$

and therefore
$$
\begin{align}
\int_a^b \omega(x) r(x) dx &= \int_a^b \omega(x) \sum_{i=1}^N r(x_i)\ell_i(x) \\
            &= \sum_{i=1}^N r(x_i) \int_a^b \omega(x) \ell_i(x)\\
            &=  \sum_{i=1}^N w_i r(x_i)\\
           \end{align}
$$

However, with a bit of work, the weights can can be calculated as functions of $P_n(x)$ (This is the tricky part For a proof and considerably more detail see https://en.wikipedia.org/wiki/Gaussian_quadrature.

But given a formula for the weights the general quadrature scheme becomes

$$
    \int_a^b \omega(x) h(x) dx = \sum_{i=1}^N w_i h(x_i)
$$
Choosing the correct weighting function and basis leads to a number of useful quadrature approaches:

#### Gauss-Legendre Quadrature

General Gauss-Legendre quadrature uses $\omega(x) = 1$ and |the Legendre Polynomials, which can be shown to have weights

$$w_i = \frac{2}{(1-x_i^2)(P'_n(x_i))^2}$$

and $x_i$ is the $i$th root of $P_n$.

```python
from scipy.special import legendre
x = numpy.linspace(-1,1,100)

fig = plt.figure(figsize=(8,6))
axes = fig.add_subplot(1,1,1)
for N in range(5):
    Pn = legendre(N)
    axes.plot(x, Pn(x), label='N={}'.format(N))

axes.grid()
axes.set_title('Legendre Polynomials', fontsize=18)
axes.set_xlabel('x',fontsize=16)
axes.legend(loc='best')
plt.show()

```

Which lead to the quadrature points and weights...

$$N$$	 $$x_i$$	  $$w_i$$

$$1$$	            $$0$$ 	  $$2$$

$$2$$	            $$\pm \sqrt{\frac{1}{3}}$$ 	  $$1$$

$$3$$	  $$0$$ 	  $$8/9$$

                      $$\pm \sqrt{\frac{3}{5}}$$ 	  $$5/9$$

$$4$$	  $$\pm \sqrt{\frac{3}{7} - \frac{2}{7} \sqrt{\frac{6}{5}}}$$	  $$\frac{18 + \sqrt{30}}{36}$$

                      $$\pm \sqrt{\frac{3}{7} + \frac{2}{7} \sqrt{\frac{6}{5}}}$$	 $$\frac{18 - \sqrt{30}}{36}$$

$$5$$	  $$0$$ 	  $$\frac{128}{225}$$

                      $$\pm \frac{1}{3} \sqrt{5 - 2 \sqrt{\frac{10}{7}}}$$	  $$\frac{322 + 13\sqrt{70}}{900}$$

                      $$\pm \frac{1}{3} \sqrt{5 + 2 \sqrt{\frac{10}{7}}}$$	  $$\frac{322 - 13\sqrt{70}}{900}$$

Note the weights and quadrature points for the 2-Point Gauss-Legendre quadrature and compare to what we found by undetermined coefficients.

Again, all of the weights and quadrature points are defined on the interval $x\in[-1, 1]$ however these can be transferred to any arbitrary interval by an affine transformation.

```python
def glintegration(f, a, b, N):
    """ Approximate \int_a^b f(x)dx using  Gauss-Legendre quadrature rules with N<=4 quadrature points"""

    # Build up a nested list of gauss points and weights:
    xGl = []
    wGl = []
    # 1 point Gauss-legendre quadrature
    xGl.append(numpy.array([0.]))
    wGl.append(numpy.array([2.]))
    # 2 point Gauss-legendre quadrature
    xGl.append(numpy.array([ -1./numpy.sqrt(3.), 1./numpy.sqrt(3)]))
    wGl.append(numpy.array([ 1. , 1.]))
    # 3 point Gauss-legendre quadrature
    xGl.append(numpy.array([-numpy.sqrt(3. /5.), 0., numpy.sqrt(3. / 5.)]))
    wGl.append(numpy.array([5., 8., 5.])/9.)
    # 4 point Gauss-Legendre quadrature
    xGl.append(numpy.array([-numpy.sqrt(3.0 / 7.0 - 2.0 / 7.0 * numpy.sqrt(6.0 / 5.0)),
           numpy.sqrt(3.0 / 7.0 - 2.0 / 7.0 * numpy.sqrt(6.0 / 5.0)),
          -numpy.sqrt(3.0 / 7.0 + 2.0 / 7.0 * numpy.sqrt(6.0 / 5.0)),
           numpy.sqrt(3.0 / 7.0 + 2.0 / 7.0 * numpy.sqrt(6.0 / 5.0))]))
    wGl.append(numpy.array([(18.0 + numpy.sqrt(30.0)) / 36.0, (18.0 + numpy.sqrt(30.0)) / 36.0,
         (18.0 - numpy.sqrt(30.0)) / 36.0, (18.0 - numpy.sqrt(30.0)) / 36.0]))

    # make affine map between x and xi
    xi_map = lambda xi: (b - a)/2. * xi + (a + b) / 2.

    I = (b - a)/2. * (wGl[N-1].dot(f(xi_map(xGl[N-1]))))
    return I

```

##### Example: Gauss-Legendre Quadrature single interval

let $f(x) = \cos(x)$:

$$
    I[f] = \int_0^{\pi/2} f(x) dx = \left.\sin(x)\right|_0^{\pi/2} = 1.
$$

so the affine map from $x\in[0,\pi/2]\rightarrow\xi\in[-1,1]$ is

\begin{align}
x &= \frac{\pi}{4}\xi + \frac{\pi}{4} \\
&= \frac{\pi}{4}\left(\xi + 1\right)
\end{align}

```python
f = lambda x: numpy.cos(x)
I = 1.
x_interval = numpy.linspace(0., numpy.pi/2, 100)
```

```python
x = numpy.linspace(-1.,2, 100)
fig = plt.figure(figsize=(5,5))
axes = fig.add_subplot(1,1,1)
axes.plot(x,f(x),linewidth=3)
axes.fill_between(x_interval,0.,f(x_interval))
axes.plot(x, numpy.zeros(x.shape),'k')

axes.grid()
plt.show()
```

```python
a = 0.
b = numpy.pi/2
I_true = 1.
err = numpy.zeros(4)
print('Gauss-Legendre N-point quadrature formulas\n')
for N in range(1,5):
    I = glintegration(f, a, b, N)
    err[N-1] = numpy.abs(I_true - I)
    print('N = {}, I = {}, err = {}'.format(N,I, err[N-1]))
```

```python
errNC = numpy.zeros(4)
print('Newton-Cotes N-point quadrature formulas\n')
for N in range(1,4):
    I = ncintegration(f, a, b, N)
    errNC[N-1] = numpy.abs(I_true - I)
    print('N = {}, I = {}, err = {}'.format(N,I, errNC[N-1]))
```

```python

fig = plt.figure(figsize=(8, 6))
axes = fig.add_subplot(1, 1, 1)
axes.semilogy(range(1,5),errNC,'bs', markersize=10, label='Newton-Cotes')
axes.semilogy(range(1,5),err,'ro', markersize=10, label='Gauss-Legendre')
axes.set_ylabel('Error')
axes.set_xlabel('quadrature points (N)')
axes.legend(loc='best')
axes.grid()
plt.show()
```

### Check polynomial degree

by integrating arbitrary polynomials of given order

```python
# 3rd order polynomial
f = lambda x: - 4*x**3 - 3.*x**2 + 2.*x +300.
int_f = lambda x:  - x**4 - x**3. + x**2 + x*300.

# 5th order polynomial
#f = lambda x: x**5 - 4*x**3 - 3.*x**2 + 2.*x +300.
#int_f = lambda x: x**6/6. - x**4 - x**3. + x**2 + x*300.

a = 1.
b = 4.
I_true = int_f(b) - int_f(a)
print('I_true = {}'.format(I_true))
for N in range(1,5):
    I = glintegration(f, a, b, N)
    err = numpy.abs(I - I_true)/numpy.abs(I_true)
    print('N = {}, I = {:5.6g}, rel_err = {} '.format(N,I,err))
```

```python
x = numpy.linspace(a - 1, b + 1, 100)
x_interval = numpy.linspace(a, b, 100)
fig = plt.figure(figsize=(8,6))
axes = fig.add_subplot(1,1,1)
axes.plot(x,f(x),linewidth=3)
axes.fill_between(x_interval,0.,f(x_interval))
axes.plot(x, numpy.zeros(x.shape),'k')

axes.grid()
plt.show()
```

##### Example 3:

If $f(x) = e^x$ look at the relative accuracy of midpoint, trapezoidal, Simpson's and 2-point Gauss-Legendre quadrature for a single interval $x \in [-1,1]$.

$$\begin{aligned}
    \text{Exact:} &I[f] = \int^1_{-1} e^x = \left . e^x \right |^1_{-1} = e - \frac{1}{e} \approx 2.350402387 \\
    \text{Midpoint:} &I_2[f] = 2 e^0 = 2 \\
    \text{Trapezoid:} &I_2[f] = \frac{2}{2} (e^{-1} + e^1) = e + \frac{1}{e} = 3.08616127 \\
    \text{Simpson's:} &I_2[f] = \frac{2}{6} e^{-1} + \frac{4}{3} e^0 + \frac{2}{6} e^1 = \frac{4}{3} + \frac{1}{3} (e^{-1} + e^1) \approx 2.362053757 \\
    \text{Gauss-Legendre:} &I_2[f] = e^{-\sqrt{\frac{1}{3}}} + e^{\sqrt{\frac{1}{3}}} \approx 2.342696088
\end{aligned}$$

```python
# Compute the error as a function of delta_x for each method
f = lambda x: numpy.sin(numpy.pi * x)
I = 2.0 / numpy.pi

# num_partitions = range(50, 1000, 50)
num_partitions = range(5, 50, 5)
delta_x = numpy.empty(len(num_partitions))
error_trap = numpy.empty(len(num_partitions))
error_simpson = numpy.empty(len(num_partitions))
error_2 = numpy.empty(len(num_partitions))
error_3 = numpy.empty(len(num_partitions))
error_4 = numpy.empty(len(num_partitions))

for (j, N) in enumerate(num_partitions):
    x_hat = numpy.linspace(0.0, 1.0, N)
    delta_x[j] = x_hat[1] - x_hat[0]

    # Compute trapezoid
    I_hat = 0.0
    for i in range(0, N - 1):
        I_hat += (f(x_hat[i + 1]) + f(x_hat[i])) * delta_x[j] / 2.0
    error_trap[j] = numpy.abs(I_hat - I)

    # Compute simpson's
    I_hat = 0.0
    for i in range(0, N - 1):
        I_hat += delta_x[j] * (1.0 / 6.0 * (f(x_hat[i]) + f(x_hat[i+1])) + 2.0 / 3.0 * f(x_hat[i] + delta_x[j] / 2.0))
    error_simpson[j] = numpy.abs(I_hat - I)

    # Compute Gauss-Legendre 2-point
    xi_map = lambda a,b,xi : (b - a) / 2.0 * xi + (a + b) / 2.0
    xi = [-numpy.sqrt(1.0 / 3.0), numpy.sqrt(1.0 / 3.0)]
    w = [1.0, 1.0]
    I_hat = 0.0
    for i in range(0, N - 1):
        for k in range(len(xi)):
            I_hat += f(xi_map(x_hat[i], x_hat[i+1], xi[k])) * w[k]
    I_hat *= delta_x[j] / 2.0
    error_2[j] = numpy.abs(I_hat - I)

    # Compute Gauss-Legendre 3-point
    xi_map = lambda a,b,xi : (b - a) / 2.0 * xi + (a + b) / 2.0
    xi = [-numpy.sqrt(3.0 / 5.0), 0.0, numpy.sqrt(3.0 / 5.0)]
    w = [5.0 / 9.0, 8.0 / 9.0, 5.0 / 9.0]
    I_hat = 0.0
    for i in range(0, N - 1):
        for k in range(len(xi)):
            I_hat += f(xi_map(x_hat[i], x_hat[i+1], xi[k])) * w[k]
    I_hat *= delta_x[j] / 2.0
    error_3[j] = numpy.abs(I_hat - I)

    # Compute Gauss-Legendre 4-point
    xi_map = lambda a,b,xi : (b - a) / 2.0 * xi + (a + b) / 2.0
    xi = [-numpy.sqrt(3.0 / 7.0 - 2.0 / 7.0 * numpy.sqrt(6.0 / 5.0)),
           numpy.sqrt(3.0 / 7.0 - 2.0 / 7.0 * numpy.sqrt(6.0 / 5.0)),
          -numpy.sqrt(3.0 / 7.0 + 2.0 / 7.0 * numpy.sqrt(6.0 / 5.0)),
           numpy.sqrt(3.0 / 7.0 + 2.0 / 7.0 * numpy.sqrt(6.0 / 5.0))]
    w = [(18.0 + numpy.sqrt(30.0)) / 36.0, (18.0 + numpy.sqrt(30.0)) / 36.0,
         (18.0 - numpy.sqrt(30.0)) / 36.0, (18.0 - numpy.sqrt(30.0)) / 36.0]
    I_hat = 0.0
    for i in range(0, N - 1):
        for k in range(len(xi)):
            I_hat += f(xi_map(x_hat[i], x_hat[i+1], xi[k])) * w[k]
    I_hat *= delta_x[j] / 2.0
    error_4[j] = numpy.abs(I_hat - I)

fig = plt.figure(figsize=(8, 6))
axes = fig.add_subplot(1, 1, 1)

# axes.plot(delta_x, error)
axes.loglog(delta_x, error_trap, 'o', label="Trapezoid")
axes.loglog(delta_x, error_simpson, 'o', label="Simpson's")
axes.loglog(delta_x, error_2, 'o', label="G-L 2-point")
axes.loglog(delta_x, error_3, 'o', label="G-L 3-point")
axes.loglog(delta_x, error_4, 'o', label="G-L 4-point")

order_C = lambda delta_x, error, order: numpy.exp(numpy.log(error) - order * numpy.log(delta_x))
axes.loglog(delta_x, order_C(delta_x[0], error_trap[0], 2.0) * delta_x**2.0, 'r--', label="2nd Order")
axes.loglog(delta_x, order_C(delta_x[0], error_simpson[0], 4.0) * delta_x**4.0, 'g--', label="4th Order")
axes.loglog(delta_x, order_C(delta_x[1], error_3[1], 5) * delta_x**5, 'b--', label="5th Order")
axes.loglog(delta_x, order_C(delta_x[1], error_4[1], 7.0) * delta_x**7.0, 'k--', label="7th Order")

axes.legend(loc='best',fontsize=14)
axes.set_xlabel('$\Delta x$', fontsize=16)
axes.set_xlim((5e-3, delta_x[0]))
axes.grid()

plt.show()
```

#### Other Quadrature Families

 - Clenshaw-Curtis (Gauss-Chebyshev): If $w(x) = \frac{1}{\sqrt{1 - x^2}}$ and $g(x)$ are Chebyshev polynomials then we know the roots of the polynomials to be $x_i = \cos\left(\frac{2i-1}{2N} \pi \right)$ (the Chebyshev nodes) and we can derive that $w_i = \frac{\pi}{N}$.
 - Gauss-Hermite:  If $w(x) = e^{-x^2}$ and $g(x)$ are Hermite polynomials $H_i(x)$ then
   $$w_i = \frac{2^{N-1} N! \sqrt{\pi}}{N^2 (H_{N-1}(x_i))^2}$$

## SciPy Integration Routines

SciPy has a number of integration routines that we have derived here including general purpose integrators that can control error.  For more information see [scipy.integrate](https://docs.scipy.org/doc/scipy/reference/integrate.html)

```python
import scipy.integrate as integrate
# integrate?
```


```python
from sympy.solvers import solve
from sympy import Symbol
from sympy import *
import matplotlib.pyplot as plt
import numpy as np
from scipy.sparse import *
from numpy.linalg import inv
from array import *
from scipy import linalg
from numpy import array

a = array( [2,3,4] )
x=Symbol('x')
# number of elements
nEle=3
# length of the domain
domainLen=1
xb=[]
xa=[]
for i in range(1,nEle+1):
    xb.append(i*(domainLen/3))
    xa.append((i-1)*(domainLen/3))
# problem data
a=1
c=-1
f=-x*x
k=[]
Ftemp=[]
# for ith element  # NOTE: [[element 1 k's],[element 2 k's], ...]
for i in range(nEle):
    psi_1 = (xb[i]-x)/(xb[i]-xa[i])
    psi_2 = (x-xa[i])/(xb[i]-xa[i])
    # print(psi_1)
    # print(psi_2)
    k.append([])
    Ftemp.append(integrate(f*psi_1 ,(x, xa[i], xb[i])))
    Ftemp.append(integrate(f*psi_2 ,(x, xa[i], xb[i])))
    k[i].append(integrate( a*diff(psi_1,x)*diff(psi_1,x)+c*psi_1*psi_2,(x, xa[i], xb[i])))
    k[i].append(integrate( a*diff(psi_1,x)*diff(psi_2,x)+c*psi_1*psi_2,(x, xa[i], xb[i])))
    k[i].append(integrate( a*diff(psi_2,x)*diff(psi_1,x)+c*psi_1*psi_2,(x, xa[i], xb[i])))
    k[i].append(integrate( a*diff(psi_2,x)*diff(psi_2,x)+c*psi_1*psi_2,(x, xa[i], xb[i])))

F=[]
F.append(Ftemp[0])
for i in range(0,len(Ftemp)-2,2):
    F.append(Ftemp[i+1]+Ftemp[i+2])
F.append(Ftemp[len(Ftemp)-1])

# print('k = ',k)
# print('Ftemp',Ftemp)
# print('F = ',F)
# three diagonals of the tridiagonal matrix
diagA=[]
diagB=[]
# for three element 4*4 k matrix
for i in range(nEle):
    diagA = np.append(diagA, k[i][2])
print('diagA',diagA)
# NOTE: no need for diagC as it will always be same as diagA

diagB = np.append(diagB, k[0][0])
for i in range(nEle-1):
    diagB = np.append(diagB, k[i][3]+k[i+1][0])
diagB = np.append(diagB, k[nEle-1][3])
print('diagB',diagB)

a=array([-3.05555555555556, -3.05555555555556, -3.05555555555555])
b=array([2.94444444444444, 5.88888888888889, 5.88888888888889, 2.94444444444444])

diagA=np.array(diagA, dtype=np.float64)
diagB=np.array(diagB, dtype=np.float64)
new_rr = np.array([diagB, diagA, diagA])

T = diags(new_rr, [0,-1, 1]).toarray()
print('\n',T, '\n', T[1][1],'\n')
d = np.array(T)
ainv = inv(d)
print(ainv)
print(linalg.inv(T).dot(F))
```

    diagA [-3.05555555555556 -3.05555555555556 -3.05555555555555]
    diagB [2.94444444444444 5.88888888888889 5.88888888888889 2.94444444444444]

     [[ 2.94444444 -3.05555556  0.          0.        ]
     [-3.05555556  5.88888889 -3.05555556  0.        ]
     [ 0.         -3.05555556  5.88888889 -3.05555556]
     [ 0.          0.         -3.05555556  2.94444444]]
     5.888888888888888

    [[-1.16239394 -1.4473978  -1.62713636 -1.68853773]
     [-1.4473978  -1.39476515 -1.56796777 -1.62713636]
     [-1.62713636 -1.56796777 -1.39476515 -1.4473978 ]
     [-1.68853773 -1.62713636 -1.4473978  -1.16239394]]
    [0.541326863541983 0.522652351332375 0.480108173167282 0.453152087458709]

```python
from sympy.solvers import solve
from sympy import Symbol
from sympy import *
import matplotlib.pyplot as plt
import numpy as np
from scipy.sparse import *
from numpy.linalg import inv
from array import *
from scipy import linalg
x=Symbol('x')

def kfmatix(nEle,domainLen,a,c,f):
    xb=[]
    xa=[]
    for i in range(1,nEle+1):
        xb.append(i*(domainLen/3))
        xa.append((i-1)*(domainLen/3))
    k=[]
    Ftemp=[]
    # for ith element  # NOTE: [[element 1 k's],[element 2 k's], ...]
    for i in range(nEle):
        psi_1 = (xb[i]-x)/(xb[i]-xa[i])
        psi_2 = (x-xa[i])/(xb[i]-xa[i])
        # print(psi_1)
        # print(psi_2)
        k.append([])
        Ftemp.append(integrate(f*psi_1 ,(x, xa[i], xb[i])))
        Ftemp.append(integrate(f*psi_2 ,(x, xa[i], xb[i])))
        k[i].append(integrate( a*diff(psi_1,x)*diff(psi_1,x)+c*psi_1*psi_2,(x, xa[i], xb[i])))
        k[i].append(integrate( a*diff(psi_1,x)*diff(psi_2,x)+c*psi_1*psi_2,(x, xa[i], xb[i])))
        k[i].append(integrate( a*diff(psi_2,x)*diff(psi_1,x)+c*psi_1*psi_2,(x, xa[i], xb[i])))
        k[i].append(integrate( a*diff(psi_2,x)*diff(psi_2,x)+c*psi_1*psi_2,(x, xa[i], xb[i])))

    F=[]
    F.append(Ftemp[0])
    for i in range(0,len(Ftemp)-2,2):
        F.append(Ftemp[i+1]+Ftemp[i+2])
    F.append(Ftemp[len(Ftemp)-1])

    # print('k = ',k)
    # print('Ftemp',Ftemp)
    # print('F = ',F)
    # three diagonals of the tridiagonal matrix
    diagA=[]
    diagB=[]
    # for three element 4*4 k matrix
    for i in range(nEle):
        diagA.append(k[i][2])
    # print('diagA',diagA)
    # NOTE: no need for diagC as it will always be same as diagA

    diagB.append(k[0][0])
    for i in range(nEle-1):
        diagB.append(k[i][3]+k[i+1][0])
    diagB.append(k[nEle-1][3])
    # print('diagB',diagB)
    diagA=np.array(diagA, dtype=np.float64)
    diagB=np.array(diagB, dtype=np.float64)

    K = np.array( diags([diagB,diagA,diagA], [0,-1, 1]).todense() )
    return(K,F)

def solU(k,f):
    bc1=Symbol('bc1')
    bc2=Symbol('bc2')
    print('k is ',k)
    print(k[1][0])

# length of the domain
domainLen=1
# number of elements
nEle=3
# problem data
a=1
c=-1
f=-x*x

k,f=kfmatix(nEle,domainLen,a,c,f)
print('K matrix : \n',k)
print('f matrix : \n',f)
# boundary conditions

sol = solU(k,f)
# print('Solution, U : \n',linalg.inv(k).dot(f))
```

    K matrix :
     [[ 2.94444444 -3.05555556  0.          0.        ]
     [-3.05555556  5.88888889 -3.05555556  0.        ]
     [ 0.         -3.05555556  5.88888889 -3.05555556]
     [ 0.          0.         -3.05555556  2.94444444]]
    f matrix :
     [-0.00308641975308642, -0.0432098765432099, -0.154320987654321, -0.132716049382716]
    k is  [[ 2.94444444 -3.05555556  0.          0.        ]
     [-3.05555556  5.88888889 -3.05555556  0.        ]
     [ 0.         -3.05555556  5.88888889 -3.05555556]
     [ 0.          0.         -3.05555556  2.94444444]]
    -3.05555555556

```python

```

```python

```


This notebook is part of the `galgebra` documentation: https://galgebra.readthedocs.io/.

# Introduction to using GAlgebra

This is a tutorial to introduce you to `galgebra`, a symbolic geometric algebra library for python.

A geometric algebra is defined by a set of symbols that represent the basis vectors of a real vector space, a metric tensor, and possible a set of coordinate symbols.  If coordinates are defined the metric tensor can be a function of them.

The following cell imports all the functions needed in the tutorial from `sympy`, `ga`, and `printer`.

```python
import sympy
from galgebra.ga import Ga
from galgebra.printer import latex
from IPython.display import Math

# tell sympy to use our printing by default
sympy.init_printing(latex_printer=latex, use_latex='mathjax')
```

## Printing in sympy

Sympy will show pretty $\LaTeX$ renderings of symbolic expressions by default

```python
sympy.S('n')**2
```

$\displaystyle n^{2}$

But if we want to append freeform text on the same line, we must use `Math`, `latex`, and f-strings in tandem:

```python
Math(f"y = { latex(sympy.S('n')**2) }")
```

$\displaystyle y = n^{2}$

## Creating an algebra

To start with we will define the geometric algebra of a 3 dimensional Euclidaen vector space, `o3d`, with coordinates $x$, $y$, and $z$ and unit vectors $e_x$, $e_y$, and $e_z$.

```python
xyz = (x, y, z) = sympy.symbols('x y z', real=True)
o3d = Ga('e_x e_y e_z', g=[1, 1, 1], coords=xyz)
grad = o3d.grad
```

The metric tensor $g$ is:

```python
Math(f'g = {latex(o3d.g)}')
```

$\displaystyle g = \left[\begin{array}{ccc}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{array}\right]$

## Creating multivectors

The most general element of a geometric algebra is a multivector.  To define a scalar `S`, a vector `V`, a bivector `B`, and a pseudo-scalar `P` (these are the only pure grade multivectors we can have in three dimensions):

```python
o3d.mv('S', 'scalar')
```

$\displaystyle S = S$

```python
o3d.mv('V', 'vector')
```

$\displaystyle V = V^{x} \boldsymbol{e}_{x} + V^{y} \boldsymbol{e}_{y} + V^{z} \boldsymbol{e}_{z}$

```python
o3d.mv('B', 'bivector')
```

$\displaystyle B = B^{xy} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + B^{xz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + B^{yz} \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}$

```python
o3d.mv('I', 'pseudo')
```

$\displaystyle I = I^{xyz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}$

We can also extract the basis vectors from `o3d`. If we name them `ex`, `ey`, and `ez` and form vectors from linear combinations of them:

```python
ex, ey, ez = o3d.mv()
Math(f'{latex(ex)}, {latex(ey)}, {latex(ez)}')
```

$\displaystyle  \boldsymbol{e}_{x},  \boldsymbol{e}_{y},  \boldsymbol{e}_{z}$

## Multivector operators

Binary operations that we can apply to vectors or multivectors in general are addition, `+`, subtraction, `-`, geometric product, `*`, inner (dot) product, `|`, outer (wedge) product, `^`, left contraction, `<`, right contraction, `>`.
Because operator precedence is immuatable in Python we need to always use parenthesis to determine the correct order of the operations in our expression.  Examples for `+`, `-`, `*`, `|`, and `^` follow:

```python
a = o3d.mv('a','vector')
b = o3d.mv('b','vector')
Math(fr'''
\begin{{align}}
    a &= {latex(a)} \\
    b &= {latex(b)}
\end{{align}}
''')
```

$\displaystyle
\begin{align}
    a &= a^{x} \boldsymbol{e}_{x} + a^{y} \boldsymbol{e}_{y} + a^{z} \boldsymbol{e}_{z} \\
    b &= b^{x} \boldsymbol{e}_{x} + b^{y} \boldsymbol{e}_{y} + b^{z} \boldsymbol{e}_{z}
\end{align}
$

```python
Math(fr'''
\begin{{align}}
    a+b         &= {latex(a+b)} \\
    a-b         &= {latex(a-b)} \\
    ab          &= {latex(a*b)} \\
    a\cdot b    &= {latex(a|b)} \\
    a \rfloor b &= {latex(a<b)} \\
    a \lfloor b &= {latex(a>b)} \\
    a\wedge b   &= {latex(a^b)}
\end{{align}}
''')
```

$\displaystyle
\begin{align}
    a+b         &= \left ( a^{x} + b^{x}\right ) \boldsymbol{e}_{x} + \left ( a^{y} + b^{y}\right ) \boldsymbol{e}_{y} + \left ( a^{z} + b^{z}\right ) \boldsymbol{e}_{z} \\
    a-b         &= \left ( a^{x} - b^{x}\right ) \boldsymbol{e}_{x} + \left ( a^{y} - b^{y}\right ) \boldsymbol{e}_{y} + \left ( a^{z} - b^{z}\right ) \boldsymbol{e}_{z} \\
    ab          &= \left ( a^{x} b^{x} + a^{y} b^{y} + a^{z} b^{z}\right )  + \left ( a^{x} b^{y} - a^{y} b^{x}\right ) \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + \left ( a^{x} b^{z} - a^{z} b^{x}\right ) \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + \left ( a^{y} b^{z} - a^{z} b^{y}\right ) \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \\
    a\cdot b    &= a^{x} b^{x} + a^{y} b^{y} + a^{z} b^{z} \\
    a \rfloor b &= a^{x} b^{x} + a^{y} b^{y} + a^{z} b^{z} \\
    a \lfloor b &= a^{x} b^{x} + a^{y} b^{y} + a^{z} b^{z} \\
    a\wedge b   &= \left ( a^{x} b^{y} - a^{y} b^{x}\right ) \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + \left ( a^{x} b^{z} - a^{z} b^{x}\right ) \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + \left ( a^{y} b^{z} - a^{z} b^{y}\right ) \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}
\end{align}
$

```python
B = o3d.mv('B','bivector')
B
```

$\displaystyle B = B^{xy} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + B^{xz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + B^{yz} \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}$

```python
Math(fr'''
\begin{{align}}
BB          &= {latex(B*B)} \\
a+B         &= {latex(a+B)} \\
a-B         &= {latex(a-B)} \\
aB          &= {latex(a*B)} \\
a\cdot B    &= {latex(a|B)} \\
a \rfloor B &= {latex(a<B)} \\
a \lfloor B &= {latex(a>B)} \\
a\wedge B   &= {latex(a^B)} \\
\end{{align}}
''')
```

$\displaystyle
\begin{align}
BB          &= - {\left ( B^{xy} \right )}^{2} - {\left ( B^{xz} \right )}^{2} - {\left ( B^{yz} \right )}^{2} \\
a+B         &= a^{x} \boldsymbol{e}_{x} + a^{y} \boldsymbol{e}_{y} + a^{z} \boldsymbol{e}_{z} + B^{xy} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + B^{xz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + B^{yz} \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \\
a-B         &= a^{x} \boldsymbol{e}_{x} + a^{y} \boldsymbol{e}_{y} + a^{z} \boldsymbol{e}_{z} - B^{xy} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} - B^{xz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} - B^{yz} \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \\
aB          &= \left ( - B^{xy} a^{y} - B^{xz} a^{z}\right ) \boldsymbol{e}_{x} + \left ( B^{xy} a^{x} - B^{yz} a^{z}\right ) \boldsymbol{e}_{y} + \left ( B^{xz} a^{x} + B^{yz} a^{y}\right ) \boldsymbol{e}_{z} + \left ( B^{xy} a^{z} - B^{xz} a^{y} + B^{yz} a^{x}\right ) \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \\
a\cdot B    &= \left ( - B^{xy} a^{y} - B^{xz} a^{z}\right ) \boldsymbol{e}_{x} + \left ( B^{xy} a^{x} - B^{yz} a^{z}\right ) \boldsymbol{e}_{y} + \left ( B^{xz} a^{x} + B^{yz} a^{y}\right ) \boldsymbol{e}_{z} \\
a \rfloor B &= \left ( - B^{xy} a^{y} - B^{xz} a^{z}\right ) \boldsymbol{e}_{x} + \left ( B^{xy} a^{x} - B^{yz} a^{z}\right ) \boldsymbol{e}_{y} + \left ( B^{xz} a^{x} + B^{yz} a^{y}\right ) \boldsymbol{e}_{z} \\
a \lfloor B &=  0  \\
a\wedge B   &= \left ( B^{xy} a^{z} - B^{xz} a^{y} + B^{yz} a^{x}\right ) \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \\
\end{align}
$

## More examples

Additionally, we can define multivector fields that are functions of the coordinates.  Some concrete examples are (vector and bivector fields):

```python
Vf = x**2*ex + y**2*ey + z**2*ez
Bf = x*(ey^ez) + y*(ex^ez) + z*(ex^ey)
Math(fr'''
\begin{{align}}
\text{{Vector Field:}}   && V_f &= {latex(Vf)} \\
\text{{Bivector Field:}} && B_f &= {latex(Bf)}
\end{{align}}
''')
```

$\displaystyle
\begin{align}
\text{Vector Field:}   && V_f &= x^{2} \boldsymbol{e}_{x} + y^{2} \boldsymbol{e}_{y} + z^{2} \boldsymbol{e}_{z} \\
\text{Bivector Field:} && B_f &= z \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + y \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + x \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}
\end{align}
$

In addition to binary algebraic operations the most important member functions for multivectors are `grade(i)`, `rev()`, and `norm2()`.  For a general multivector, `M`, we have:

```python
M = o3d.mv('M', 'mv')
Math('M = %s' % latex(M))
```

$\displaystyle M = M  + M^{x} \boldsymbol{e}_{x} + M^{y} \boldsymbol{e}_{y} + M^{z} \boldsymbol{e}_{z} + M^{xy} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + M^{xz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + M^{yz} \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} + M^{xyz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}$

```python
Math(fr'''
\begin{{align}}
\text{{Grade 0:}} && \left<M\right>_0 &= {latex(M.grade(0))} \\
\text{{Grade 1:}} && \left<M\right>_1 &= {latex(M.grade(1))} \\
\text{{Grade 2:}} && \left<M\right>_2 &= {latex(M.grade(2))} \\
\text{{Grade 3:}} && \left<M\right>_3 &= {latex(M.grade(3))} \\
\text{{Reverse:}} && M^\dagger        &= {latex(M.rev())}
\end{{align}}
''')
```

$\displaystyle
\begin{align}
\text{Grade 0:} && \left<M\right>_0 &= M \\
\text{Grade 1:} && \left<M\right>_1 &= M^{x} \boldsymbol{e}_{x} + M^{y} \boldsymbol{e}_{y} + M^{z} \boldsymbol{e}_{z} \\
\text{Grade 2:} && \left<M\right>_2 &= M^{xy} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + M^{xz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + M^{yz} \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \\
\text{Grade 3:} && \left<M\right>_3 &= M^{xyz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \\
\text{Reverse:} && M^\dagger        &= M  + M^{x} \boldsymbol{e}_{x} + M^{y} \boldsymbol{e}_{y} + M^{z} \boldsymbol{e}_{z} - M^{xy} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} - M^{xz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} - M^{yz} \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} - M^{xyz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}
\end{align}
$

## More printing options

A problem in displaying multivectors is that the expression can be very long and does not display nicely on the page.  To alleviate this problem one can use the multivector member function `Fmt()`.  The default is `Fmt(1)` which displays the multivector on one line, `Fmt(2)` displayes the multivector one grade per line, and `Fmt(3)` displayes the mulitvector one base or basis blade per line.  Some examples are:

```python
M.Fmt(1)
```

$\displaystyle M  + M^{x} \boldsymbol{e}_{x} + M^{y} \boldsymbol{e}_{y} + M^{z} \boldsymbol{e}_{z} + M^{xy} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + M^{xz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + M^{yz} \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} + M^{xyz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}$

```python
M.Fmt(2)
```

$\displaystyle  \begin{aligned}[t]  & M  \\  &  + M^{x} \boldsymbol{e}_{x} + M^{y} \boldsymbol{e}_{y} + M^{z} \boldsymbol{e}_{z} \\  &  + M^{xy} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + M^{xz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + M^{yz} \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \\  &  + M^{xyz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}  \end{aligned} $

```python
M.Fmt(3)
```

$\displaystyle  \begin{aligned}[t]  & M  \\  &  + M^{x} \boldsymbol{e}_{x} \\  &  + M^{y} \boldsymbol{e}_{y} \\  &  + M^{z} \boldsymbol{e}_{z} \\  &  + M^{xy} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} \\  &  + M^{xz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} \\  &  + M^{yz} \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \\  &  + M^{xyz} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}  \end{aligned} $


- - - -
# Mechpy Tutorials
a mechanical engineering toolbox

source code - https://github.com/nagordon/mechpy
documentation - https://nagordon.github.io/mechpy/web/

- - - -

Neal Gordon
2017-02-20

- - - -

## statics

* [Example 1: A simple supported beam with shear-bending plots](#Statics-Example-1)
* [Example 2: Vector calculation method to calculate 3-D moments](#Statics-Example-2)
* [Distributed Loads Calculations](#Distributed-Loads-Calculations)

## Python Initilaization with module imports

```python
# setup
import numpy as np
import sympy as sp
import scipy
from pprint import pprint
sp.init_printing(use_latex='mathjax')

import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = (12, 8)  # (width, height)
plt.rcParams['font.size'] = 14
plt.rcParams['legend.fontsize'] = 16
from matplotlib import patches

get_ipython().magic('matplotlib')  # seperate window
get_ipython().magic('matplotlib inline') # inline plotting
```

    Using matplotlib backend: Qt4Agg

- - - -
# Statics
[index](#Mechpy)
- - - -

The sum of the forces is zero
$$
\Sigma F_x =0 , \Sigma F_y =0 , \Sigma F_z =0
$$
The sum of the moments is zero
$$
\Sigma M_x =0 , \Sigma M_y =0 , \Sigma M_z =0
$$

Dot Product

$$
\vec{A} \bullet \vec{B} = ABcos\left(\theta\right)= A_xB_x + A_yB_y+A_zB_z
$$

Cross-Product

$$
\vec{C}=\vec{A} \times \vec{B} =
\begin{vmatrix}
    \widehat{i} & \widehat{j} & \widehat{k}\\
    A_{x} & A_{y} & A_{z}\\
    B_{x} & B_{y} & B_{z}
\end{vmatrix}
$$

Moment of Force
$$
M_0 = Fd \\
\vec{M_0}=\vec{r}\times \vec{F} =
\begin{vmatrix}
    \widehat{i} & \widehat{j} & \widehat{k}\\
    r_{x} & r_{y} & r_{z}\\
    F_{x} & F_{y} & F_{z}
\end{vmatrix}
$$

Moment of Force about a Specified axis
$$
\vec{M_a}=\vec{u}\bullet\vec{r}\times \vec{F} =
\begin{vmatrix}
    u_{x} & u_{y} & u_{z} \\
    r_{x} & r_{y} & r_{z} \\
    F_{x} & F_{y} & F_{z}
\end{vmatrix}
$$

### Statics-Example 1
#### A simple supported beam with shear-bending plots

```python
import os ; os.chdir('..')  # change to root from the examples folder
from mechpy.statics import simple_support, moment_calc
```

```python
simple_support()
```

### Statics-Example 2
### Vector calculation method to calculate 3-D moments

$
\vec{M_{R_0}}=\Sigma \left( \vec{r} \times \vec{F} \right) = \vec{r_A}\times\vec{F_1} +\vec{r_A}\times\vec{F_2} + \vec{r_B}\times\vec{F_3}
$

```python
moment_calc()
```

### Distributed Loads Calculations

$
F_R = \Sigma F=\int_L w(x) \,dx  = \int_A dA \,dx
$

example, hibbler, pg 187

$$
F_R=\int_A dA \, =  \int_{0}^{2} \, 60x^2 \, dx = 160 N
$$

$$
 \overline{x} = \frac{\int_A x \, dA}{\int_A dA} =  \frac{\int_{0}^{2} x60x^2 \, dx}{\int_{0}^{2} \, 60x^2 \, dx} = \frac{240}{160}
$$

```python
x = sp.var('x')
w = 60*x**2# N/m
sp.plot(w, (x,0,2));
```

```python
w.subs(x,2)
```

$$240$$

```python
sp.Integral(w,(x,0,2))
```

$$\int_{0}^{2} 60 x^{2}\, dx$$

```python
sp.integrate(w)
```

$$20 x^{3}$$

```python
sp.integrate(w,(x,0,2))
```

$$160$$

```python
sp.Integral(x*w,(x,0,2))
```

$$\int_{0}^{2} 60 x^{3}\, dx$$

```python
sp.integrate(w*x)
```

$$15 x^{4}$$

```python
sp.integrate(x*w,(x,0,2))
```

$$240$$

```python
FR= float(sp.integrate(w,(x,0,2)))
xbar = float(sp.integrate(x*w,(x,0,2)))/FR
print('center of load of an exponential distributed load is %f' % xbar)
```

    center of load of an exponential distributed load is 1.500000

```python
#hibbler pg 346

import numpy as np

r = np.array([    0, 3 , 5.25])
F = np.array([-13.5, 0 ,6.376])
M = -np.cross(r,F)

# moments
print('M_x = %f \nM_y = %f \nM_z = %f' % (M[0], M[1], M[2] ))
```

    M_x = -19.128000
    M_y = 70.875000
    M_z = -40.500000


```python
try:
    from openmdao.utils.notebook_utils import notebook_mode
except ImportError:
    !python -m pip install openmdao[notebooks]
```

# Kepler’s Equation

This example will demonstrate the use of OpenMDAO for solving an implicit equation commonly found in astrodynamics, Kepler’s Equation:

\begin{align}
E - e \sin{E} = M
\end{align}

Here $M$ is the mean anomaly, $E$ is the eccentric anomaly, and $e$ is the eccentricity of the orbit.

If we know the eccentric anomaly, computing the mean anomaly is trivial. However, solving for the eccentric anomaly when given the mean anomaly must be done numerically. We’ll do so using a nonlinear solver. In OpenMDAO, solvers converge all implicit state variables in a Group by driving their residuals to zero.

## Using a BalanceComp and NewtonSolver

In an effort to simplify things for users, OpenMDAO features a Balance component. For each implicit state variable we assign to the balance, it solves the following equation:

\begin{align}
lhs(var) \cdot mult(var) = rhs(var)
\end{align}

The _mult_ term is an optional multiplier than can be applied to the left-hand side (LHS) of the equation. For our example, we will assign the right-hand side (RHS) to the mean anomaly ($M$), and the left-hand side to $E - e \sin{E}$.

In this implementation, we rely on an ExecComp to compute the value of the LHS.

BalanceComp also provides a way to supply the starting value for the implicit state variable ($E$ in this case), via the guess_func argument. The supplied function should have a similar signature to the guess_nonlinear function of ImplicitComponent. When solving Kepler’s equation, using $M$ as the initial guess for $E$ is a good starting point.

In summary, the recipe for solving Kepler’s equation with a NewtonSolver is as follows:

1. Define a Group to contain the implicit system.
2. To that Group, add components which provide, $M$, $e$, and the left-hand side of Kepler’s equation.
3. Add a linear and nonlinear solver to the Group, since the default solvers do not iterate.
4. Setup the problem, set values for the inputs, and run the model.

```python
import numpy as np

import openmdao.api as om

prob = om.Problem()

bal = om.BalanceComp()

bal.add_balance(name='E', val=0.0, units='rad', eq_units='rad', rhs_name='M')

# Use M (mean anomaly) as the initial guess for E (eccentric anomaly)
def guess_function(inputs, outputs, residuals):
    if np.abs(residuals['E']) > 1.0E-2:
        outputs['E'] = inputs['M']

bal.options['guess_func'] = guess_function

# ExecComp used to compute the LHS of Kepler's equation.
lhs_comp = om.ExecComp('lhs=E - ecc * sin(E)',
                       lhs={'units': 'rad'},
                       E={'units': 'rad'},
                       ecc={'units': None})

prob.model.set_input_defaults('M', 85.0, units='deg')

prob.model.add_subsystem(name='lhs_comp', subsys=lhs_comp,
                         promotes_inputs=['E', 'ecc'])

prob.model.add_subsystem(name='balance', subsys=bal,
                         promotes_inputs=['M'],
                         promotes_outputs=['E'])

# Explicit connections
prob.model.connect('lhs_comp.lhs', 'balance.lhs:E')

# Set up solvers
prob.model.linear_solver = om.DirectSolver()
prob.model.nonlinear_solver = om.NewtonSolver(solve_subsystems=False, maxiter=100, iprint=2)

prob.setup()

prob.set_val('M', 85.0, units='deg')
prob.set_val('E', 85.0, units='deg')
prob.set_val('ecc', 0.6)

prob.run_model()

M = prob.get_val('M')
E = prob.get_val('E')
print(f'M = {M}')
print(f'E = {E}')
```

```{note}
BalanceComp should be placed after those components which compute its inputs.  Otherwise errors can arise due to timing issues.
```

```python
# Check that results are correct.

from openmdao.utils.assert_utils import assert_near_equal

assert_near_equal(prob.get_val('E'), 2.02317564, tolerance=1.0E-6)
```

## Using Fixed-Point Iteration with an ImplicitComponent

One approach to solving Kepler's Equation is to use fixed-point iteration:

\begin{align}
E_{n+1} = M + e \sin{E_{n}}
\end{align}

We can achieve this behavior by defining a simple ImplicitComponent that "solves itself" by overriding its `solve_nonlinear` method.

```python
class KeplerComp(om.ImplicitComponent):

    def initialize(self):
        self.options.declare('tol', types=float, default=1.0E-12, desc='convergence tolerance')
        self.options.declare('max_its', types=int, default=50, desc='maximum number of iterations')

    def setup(self):
        self.add_input('M', val=1.0, units='rad', desc='Mean anomaly')
        self.add_input('ecc', val=0.0, units=None, desc='eccentricity')
        self.add_output('E', val=1.0, units='rad', desc='Eccentric anomaly')

        self.declare_partials(of='E', wrt='*', method='cs')

    def apply_nonlinear(self, inputs, outputs, residuals):
        """ Compute the residual of output E. """
        M = inputs['M']
        e = inputs['ecc']
        E = outputs['E']
        residuals['E'] = E - M - e * np.sin(E)

    def solve_nonlinear(self, inputs, outputs):
        """ Determine the values of the implicit outputs that eliminate the residual. """
        M = inputs['M']
        e = inputs['ecc']
        E = outputs['E']

        # Compute the initial guess for iteration if the initial redisual is large.
        E = inputs['M'] if np.abs(E - M - e * np.sin(E)) > 1.0E-2 else outputs['E']

        for i in range(self.options['max_its']):
            E_old = E
            E = M + e * np.sin(E)
            if i == 0:
                print('iter ' + '   ' + 'max abs. error')
                print(5*'-' + '   '+ 14*'-')
            print(f'{i:>5} {np.max(np.abs(E - E_old)):16.12f}')
            if np.abs(E - E_old) < self.options['tol']:
                outputs['E'] = E
                break
        else:
            raise om.AnalysisError('Iteration Limit Exceeded')

```

Here the KeplerComp uses the `solve_nonlinear` method to drive it residual to zero through fixed point iteration.

```{important}
In OpenMDAO it doesn't matter how the residuals of a system of eliminated.  Inside solve_nonlinear you're free to use a variety of techniques.  As long as the resulting residual is _reasonably close to zero_, the derivatives across the implicit system will be accurate.
```

```python
import numpy as np

import openmdao.api as om

prob = om.Problem()

prob.model.add_subsystem(name='kep_comp', subsys=KeplerComp(),
                         promotes_inputs=['M', 'ecc'],
                         promotes_outputs=['E'])

prob.setup()

prob.set_val('M', 85.0, units='deg')
prob.set_val('E', 1.0, units='deg')
prob.set_val('ecc', 0.6)

prob.run_model()
M = prob.get_val('M')
E = prob.get_val('E')
print(f'M = {M}')
print(f'E = {E}')
```

```python
# Check that results are correct.

from openmdao.utils.assert_utils import assert_near_equal

assert_near_equal(prob.get_val('E'), 2.02317564, tolerance=1.0E-6)
```

In contrast to the NewtonSolver approach, this approach roughly needs the following steps:

1. Define an ImplicitComponent where the `apply_nonlinear` method computes the residuals for Kepler's Equation and `solve_nonlinear` find the value of the implicit output that eliminates the residuals.
2. Add the ImplicitComponent to the problem model.
3. Setup the problem, set values for the inputs, and run the model.

## Which approach should you use?

It's really up to you.  In this case, the Newton approach involves slightly more complicated `System` consisting of two components (the `ExecComp` and the `BalanceComp`) but it converges in just 4 iteratoins.

The ImplicitComponent involves only a single system, but writing the `solve_nonlinear` method with some error handling, initial guess generation, and print capability made for more coding on the part of the user.

```python

```


## Introduccion a Grafos
**Universidad Tecnologica Nacional - Facultad Buenos Aires**

**Ingenieria Industrial**

**Investigacion Operativa**

Autor: Martin Palazzo

Curso I4051

#### Importamos librerias

```python
# importamos las librerias clasicas
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd

# importamos librerias especiales como networkx y euclidean_distances desde scikit-learn
import networkx as nx
from sklearn.metrics.pairwise import euclidean_distances

```

```python
# instalamos una libreria especial para visualizar
#pip install decorator==4.3
```

#### Generamos ubicaciones ficticias de plantas industriales

```python
# cada planta esta caracterizada por un vector de 2 dimensiones (lat y long)
planta_0 = np.array([[0,0]])
planta_1 = np.array([[1,1]])
planta_2 = np.array([[2,1]])
planta_3 = np.array([[3,1]])
planta_4 = np.array([[1,2]])
planta_5 = np.array([[1,3]])
planta_6 = np.array([[1,4]])
planta_7 = np.array([[2,3]])
planta_8 = np.array([[2,2]])
```

```python
# concatenamos todas las ubicaciones de las plantas en un unico array
coordenadas_plantas = np.concatenate([planta_0, planta_1, planta_2,planta_3,
               planta_4,planta_5,planta_6,planta_7, planta_8], axis = 0)
```

```python
coordenadas_plantas.shape
```

    (9, 2)

#### Generamos una matriz de distancias par-a-par (matriz de adyacencia)
Utilizando la distancia euclidea entre plantas generamos una matriz de similaridad par-a-par

```python
# utilizando la funcion euclidean_distances computamos la matriz de similaridad par-a-par
matriz_distancias = euclidean_distances(coordenadas_plantas, coordenadas_plantas)
```

```python
# visualizamos la matriz de adyacencia con heatmap
sns.heatmap(matriz_distancias)
plt.title('matriz de distancias par-a-par')
plt.xlabel('plantas')
plt.ylabel('plantas')
plt.show()
```

```python
# revisamos el tipo de variable que es coordenadas_plantas
type(coordenadas_plantas)
```

    numpy.ndarray

#### construccion del grafo con networkX
Con la matriz de adyacencia construimos un elemento de networkX llamado grafo

```python
# definimos el grafo desde la matriz de adyacencia
G = nx.from_numpy_matrix(matriz_distancias)
```

```python
# revisamos el tipo de elemento
type(G)
```

    networkx.classes.graph.Graph

```python
# visualizamos la red
nx.draw(G)
```

## Grafo de estaciones de bicicletas

dataset 'bicicleteros.csv' obtenido desde https://data.buenosaires.gob.ar/dataset/bicicleteros-via-publica

```python
# importamos el archivo bicicleteros.csv con pandas y lo guardamos como un dataframe
bicicleteros = pd.read_csv('bicicleteros.csv')
```

```python
type(bicicleteros)
```

    pandas.core.frame.DataFrame

```python
# usamos la funcion de pandas .head() para visualizar los primeros renglones del dataframe
bicicleteros.head()
```

      long
      lat
      id
      nombre
      anio_de_in
      tipo
      cantidad
      ubicacion
      clasificac
      calle
      altura
      calle2
      barrio
      comuna
      codigo_postal
      codigo_postal_argentino

      0
      -58.456938
      -34.626656
      1
      LINEA A CARABOBO
      2011
      4U
      1
      VIA PUBLICA
      SUBTE
      RIVADAVIA AV.
      0
      CARABOBO AV.
      Flores
      Comuna 7
      NaN
      NaN

      1
      -58.448757
      -34.623699
      2
      LINEA A PUAN
      2011
      4U
      1
      VIA PUBLICA
      SUBTE
      RIVADAVIA AV.
      0
      PUAN
      Caballito
      Comuna 6
      NaN
      NaN

      2
      -58.441380
      -34.620616
      3
      LINEA A PRIMERA JUNTA
      2011
      4U
      1
      VIA PUBLICA
      SUBTE
      RIVADAVIA AV.
      0
      ROJAS
      Caballito
      Comuna 6
      NaN
      NaN

      3
      -58.436347
      -34.618144
      4
      LINEA A ACOYTE
      2011
      4U
      1
      VIA PUBLICA
      SUBTE
      RIVADAVIA AV.
      0
      ACOYTE AV.
      Caballito
      Comuna 6
      NaN
      NaN

      4
      -58.429896
      -34.615279
      5
      LINEA A RIO DE JANEIRO
      2011
      4U
      1
      VIA PUBLICA
      SUBTE
      RIVADAVIA AV.
      0
      RIO DE JANEIRO
      Caballito
      Comuna 6
      NaN
      NaN

```python
# revisamos que tipo de dato es la primer posicion de la columna long
type(bicicleteros.long[0])
```

    numpy.float64

estandarizamos la columna long de manera que quede con minimo = 0 y maximo = 1

\begin{align}
\hat{x} = \frac{x - \text{min}(x)}{\text{max}(x)-\text{min}(x)}
\end{align}

```python
# estandarizamos la columna long de manera que quede con minimo = 0 y maximo = 1
long_norm = (bicicleteros.long - np.min(bicicleteros.long))/(np.max(bicicleteros.long)-np.min(bicicleteros.long))
```

```python
# revisamos el type de long_norm
type(long_norm)
```

    pandas.core.series.Series

```python
# realizamos la misma estandarizacion en la columna lat
lat_norm = (bicicleteros.lat - np.min(bicicleteros.lat))/(np.max(bicicleteros.lat)-np.min(bicicleteros.lat))
```

```python
type(lat_norm)
```

    pandas.core.series.Series

```python
# visualizamos con scatterplot las estaciones donde la coordenada x = long , y = lat
sns.scatterplot(long_norm, lat_norm)
plt.show()
```

```python
# concatenamos los vectores de long y lat en un solo dataframe
coordenadas_bicicleteros = pd.concat([long_norm, lat_norm], axis = 1)
```

```python
coordenadas_bicicleteros.shape
```

    (938, 2)

```python
# visualizamos como quedo el dataframe resultante de concatenar lat y long
coordenadas_bicicleteros.head()
```

      long
      lat

      0
      0.413332
      0.402026

      1
      0.460129
      0.420248

      2
      0.502327
      0.439248

      3
      0.531118
      0.454480

      4
      0.568019
      0.472135

```python
# con el dataframe coordenadas_bicicleteros generamos una matriz de similaridad par-a-par
matriz_dist_bici = euclidean_distances(coordenadas_bicicleteros, coordenadas_bicicleteros)
```

```python
# visualizamos con heatmap la similaridad par-a-par usando distancia euclidea entre bicicleteros
sns.heatmap(matriz_dist_bici)
```

```python

```

```python
# como el grafo anterior tiene mucha densidad de arcos decidimos usar un filtro y llevamos a 0 todo arco cuya dist < 0.4
matriz_dist_bici_filt = np.where(matriz_dist_bici < 0.4, 0, matriz_dist_bici)
```

```python
# visualizamos la nueva matriz filtrada por arcos
sns.heatmap(matriz_dist_bici_filt)
```

```python
matriz_dist_bici_filt
```

    array([[0.        , 0.        , 0.        , ..., 0.        , 0.        ,
            0.41478988],
           [0.        , 0.        , 0.        , ..., 0.        , 0.        ,
            0.        ],
           [0.        , 0.        , 0.        , ..., 0.        , 0.        ,
            0.        ],
           ...,
           [0.        , 0.        , 0.        , ..., 0.        , 0.        ,
            0.        ],
           [0.        , 0.        , 0.        , ..., 0.        , 0.        ,
            0.        ],
           [0.41478988, 0.        , 0.        , ..., 0.        , 0.        ,
            0.        ]])

#### visualizacion con networkX

```python
G_bici = nx.from_numpy_matrix(matriz_dist_bici_filt)
pos_ = nx.spring_layout(G_bici, iterations=100)
```

```python
nx.draw(G_bici, node_size=10, node_color = 'skyblue', node_shape = 'o', linewidths = 0.01, edge_color="grey",
       style="solid", pos = pos_)

```

```python

```


# All together now

We have now discretized the two first order equations over a single cell. What is left is to assemble and solve the DC system over the entire mesh. To implement the divergence on the full mesh, the stencil of $\pm 1$'s must index into $\mathbf{j}$ on the entire mesh (instead of four elements). Although this can be done in a \texttt{for-loop}, it is conceptually, and often computationally, easier to create this stencil using nested kronecker products (see notebook). The volume and area terms in the divergence get expanded to diagonal matrices, and we multiply them together to get the discrete divergence operator. The discretization of the \emph{face} inner product can be abstracted to a function, $\mathbf{M}_f(\sigma^{-1})$, that completes the inner product on the entire mesh at once. The main difference when implementing this is the $\mathbf{P}$ matrices, which must index into the entire mesh.
With the necessary operators defined for both equations on the entire mesh, we are left with two discrete equations:

\begin{equation}
	\text{diag}(\mathbf{v}) \mathbf{D}\mathbf{j} =
\mathbf{q}
\\
\mathbf{M}_f(\sigma^{-1}) \mathbf{j} =
\mathbf{D}^\top \text{diag}(\mathbf{v}) \boldsymbol{\phi}.
\end{equation}

Note that now all variables are defined over the entire mesh. We could solve this coupled system or we could eliminate $\mathbf{j}$ and solve for $\phi$ directly (a smaller, second-order system).

\begin{equation}
	\text{diag}(\mathbf{v})
\mathbf{D}
\mathbf{M}_f(\sigma^{-1})^{-1}
\mathbf{D}^\top
\text{diag}(\mathbf{v})
\boldsymbol{\phi} =
\mathbf{q}.
\end{equation}

By solving this system matrix, we obtain a solution for the electric potential $\phi$ everywhere in the domain. Creating predicted data from this requires an interpolation to the electrode locations and subtraction to obtain potential differences!

Below we have wrapped up the functions in [index.ipynb](index.ipynb) in [dc_interact.py](dc_interact.py) so that we can use them with IPython widgets.

```python
%matplotlib inline
from dc_interact import dc_resistivity
from ipywidgets import interact
interact(
    dc_resistivity,
    log_sigma_background=(-4,4),
    log_sigma_block=(-4,4),
    plot_type=['potential','conductivity','current']
);
```

Moving from continuous equations to their discrete analogues is fundamental in geophysical simulations. In this tutorial, we have started from a continuous description of the governing equations for the DC resistivity problem, selected locations on the mesh to discretize the continuous functions, constructed differential operators by considering one cell at a time, assembled and solved the discrete DC equations. Composing the finite volume system in this way allows us to move to different meshes and incorporate various types of boundary conditions that are often necessary when solving these equations in practice.

# Next up ...

To learn more about SimPEG you can go to our website http://simpeg.xyz where you can find code, documentation, presentations, more tutorials and papers!


# Mandatory exercises

1.
    1. Explain briefly the difference between a deterministic and a stochastic method.
    2. Give an example of using a stochastic method to solve a deterministic problem.
    3. Give an example of a stochastic process?

2. The integral
\begin{equation}
    I =\int_0^1x^2e^x\,\rm{d}x
\end{equation}
can be computed analytically and the exact value is $I=e-2$.
We can use this integral as a test case for numerical integration methods, to see how well they are able to approximate $I$. When the Trapeziodal Rule was applied with step size $h = 0.25$, the absolute error in the computed, approximate value of $I$ was ca. $0.04$.
Using Simpson’s Rule with the same step size gave an absolute error of ca. $0.0006$.
The step size $h = 0.25$ corresponds to using $n = 5$ discretization points.

    You should now compute an approximation to $I$ by using the Monte Carlo method that is described in Hellander’s compendium. Use $n = 5$ random values. Compare the approximation to the exact value of $I$. What is the absolute error? Compare to the errors obtained for the Trapezoidal and Simpson’s Rule, respectively.

    If you repeat the experiments with more and more random numbers, explain how the error will then behave.

3. If we compare Monte Carlo with Simpsons rule for integration, for what dimension is it more preferable to use Monte Carlo methods? The error in Simpson is of order $h^4$.

4. As a model for how the price of a stock changes in time, a so called stochastic differential equation (SDE) is often used. The part of the model that describes the randomness in the value of the stock involves the same Brownian motion as you saw in the lab assignment. Assume that you are given a Matlab/Python function

    `function x = f(x0, T)`

    or

    `def f(x0, T):`

    that simulates 10 stocks according to an SDE model. The input parameters are the initial value $x_0$ and final time $T$. The vector $x_0$ contains the values of the stocks at initial time $t = 0$. The output parameter $x$ is an array with 10 elements, containing the simulated values of the stocks at time $T$. Write a Matlab/Python script (with pen and paper) that uses a Monte Carlo method to approximate $E_{x,T}$, the expected total value of the portfolio at time $T$. The program should also compute the confidence interval for the computed value, according to formula (9) in Hellander’s compendium. You can use the built in function `std(x)` in Matlab or `numpy.std()` in Python to compute the standard deviation.

    Note: you are not supposed to write the `function x=f(x0, T)`. You should use it to approximate  $E_{x,T}$.

# Non-mandatory exercises

5. In Miniproject 3 we will need to generate random numbers from an exponential distribution
\begin{equation}
    f(\tau;a_0)=
    \begin{cases}
        \begin{split}
        &a_0e^{-a_0\tau}, &\tau\ge 0\\
        &0, &\tau<0
        \end{split}
    \end{cases}
\end{equation}
by using inverse transform sampling. Our goal is to derive a formula to sample exponentially distributed random numbers $\tau$, given uniformly distributed random numbers $u$.

    1. Compute the cumulative distribution function (CDF) $F(t)=P(\tau\le t)$ for the exponential distribution.
    2. Find the inverse of $F$. That is, for a random number $u$ uniformly distributed in $[0, 1]$ find $\tau$ such that $F(\tau) = u$. Then $\tau = F^{-1} (u)$.


```python
import numpy as np
import scipy as sp
from scipy import optimize
import matplotlib.pyplot as plt
from sympy.solvers import solve
from sympy import Symbol
import math as mt
#pyomo
```

## Problem 1

```python
#define parameters
R = 5.0
C = 2.0
mu = 10.0
lambd = 8.0
rho = lambd/mu
```

```python
#define social rate function
def f(n):
    return lambd*R*((1-rho**n)/(1-rho**(n+1)))-(C*((rho/(1-rho))-((n+1)*rho**(n+1))/(1-rho**(n+1))))
```

```python
#plot expected social benefit rate vs n

for n in range(1,40,1):
    plt.plot(n, f(n), marker = "o")
plt.title('expected social benefit rate vs n')
plt.show()
```

```python
#find the exact max value and associated n value, define the benefit
#rate again
def SO(n):
    return lambd*R*((1-rho**n[0])/(1-rho**(n[0]+1)))-C*((rho/(1-rho))-((n[0]+1)*rho**(n[0]+1))/(1-rho**(n[0]+1)))
```

```python
sp.optimize.fmin(lambda n: -SO(n), np.array([0,0]))
```

    Optimization terminated successfully.
             Current function value: -33.240874
             Iterations: 56
             Function evaluations: 96

    array([ 7.86098027, -3.93528052])

### As shown above result, when n* $\approx$ 8, the social benefit rate reaches maximum where the social maximum rate is about 33.24.

```python
#calculate individual optimize system size
n_e = (R * mu)/C
print(n_e)
```

    25.0

### n* = 10 < $n_{e}$ = 25, socially optimal system size is less than the individually optimal system size $n_{e}$.

```python
p = Symbol('p')
solve(((R - p)*mu)/C - 8, p)
```

    [3.40000000000000]

### when p = 3.4, n* = $n_{e}$, the lowest toll is 3.4.

## Problem 2

```python
def z_n(n):
    return lambd*((1-rho**n)/(1-rho**(n+1)))*(R - (C*n)/mu)
```

```python
for n in range(1,40,1):
    plt.plot(n, z_n(n), marker = "o")
plt.title('expected profit rate vs n')
plt.show()
```

```python
#define the profit function again in array form for optimizer
def z_n_a(n):
    return lambd*((1-rho**n[0])/(1-rho**(n[0]+1)))*(R - (C*n[0])/mu)
```

```python
sp.optimize.fmin(lambda n: -z_n_a(n), np.array([0,0]))
```

    Optimization terminated successfully.
             Current function value: -29.507398
             Iterations: 49
             Function evaluations: 84

    array([ 3.93776678, -1.97132956])

### $n_{m}$ $\approx$ 4, $n_{m}$ < n*

## Problem 4

```python
rho4 = 0.95
def p0_cal(c):
    sec_part = 0
    for j in range(0, c, 1):
        sec_part = sec_part + ((c*rho4)**j)/mt.factorial(j)
    return (((c*rho4)**c)/((1-rho4)*mt.factorial(c)) + sec_part)**(-1)
```

```python
for c in [1, 2, 4, 8, 16, 32]:
    p0 = p0_cal(c)
    d_frac = p0*((c**c)/mt.factorial(c))*((rho4**c)/(1 - rho4))
    d_exp = (1/(1 - rho4*(1/c))) * d_frac
    print('c = ', c, ",", 'fraction delayed = ', d_frac, ', expected delay time = ', d_exp)
```

    c =  1 , fraction delayed =  0.95 , expected delay time =  18.999999999999982
    c =  2 , fraction delayed =  0.9256410256410256 , expected delay time =  1.7631257631257629
    c =  4 , fraction delayed =  0.8914189951776734 , expected delay time =  1.1690740920362932
    c =  8 , fraction delayed =  0.844168630609684 , expected delay time =  0.9579218503372301
    c =  16 , fraction delayed =  0.780192462248371 , expected delay time =  0.8294404914268395
    c =  32 , fraction delayed =  0.6957101141665399 , expected delay time =  0.7169959308640669